Communication method applied to virtual reality equipment and virtual reality equipment

Abstract

The invention discloses a communication method applied to virtual reality equipment and the virtual reality equipment, the virtual reality equipment comprises a controller, and the controller is configured to: in response to receiving an input screen projection command, display a second user interface on a display of the virtual reality equipment; the second user interface comprises information of a screen projection access address, and the screen projection access address is an access address corresponding to a screen projection service provided by the virtual reality equipment; receiving an input first identification command; in response to a received input first identification command, obtaining first text information corresponding to the screen projection access address; and voice broadcasting the first text information through a loudspeaker of the virtual reality equipment, so that the user inputs the screen projection access address to a browser of the target equipment, and the screen projection service projects the screen to the target equipment through the browser.

Description

Communication methods applied to virtual reality devices and virtual reality devices Technical Field

[0001] This application relates to the field of virtual reality technology, and in particular to a communication method and a virtual reality device applied to a virtual reality device. Background Technology

[0002] With the development of virtual reality (VR) technology, in order to enable more users to watch content played on VR devices and enhance user engagement, some VR devices offer screen mirroring applications. In some embodiments, VR devices provide a screen mirroring application that enables screen mirroring from the VR device to external devices based on a browser.

[0003] When a user wears a VR device, they can launch the screen mirroring application. The VR device's display will then show the screen mirroring access address. Because VR devices are closed systems, the user needs to remove the VR device and enter the screen mirroring access address displayed on the VR device into a browser on an external device. This allows the user to access the screen mirroring service on the VR device and project the VR device's screen onto the external device.

[0004] However, users usually do not remember the screen mirroring access address accurately at first. They need to wear and remove the VR device repeatedly to accurately input the complete screen mirroring access address to the external device, which is inconvenient. Summary of the Invention

[0005] In some embodiments of this application, a virtual reality device is provided, including:

[0006] The controller is configured as follows:

[0007] A first user interface is displayed on the monitor of the virtual reality device;

[0008] Receive input screen mirroring commands;

[0009] In response to receiving an input screen mirroring command, a second user interface is displayed on the monitor of the virtual reality device; the second user interface includes information about the screen mirroring access address, which is the access address corresponding to the screen mirroring service provided by the virtual reality device;

[0010] Receive the first identification command input for the screen projection access address;

[0011] In response to receiving the first identification command, the first text information corresponding to the screen projection access address is obtained;

[0012] The first text information is broadcast aloud through the speaker of the virtual reality device, so that the user can input the screen projection access address into the browser of the target device.

[0013] After the access address is accessed by the target device, the screen mirroring service is controlled to send the current interface of the virtual reality device to the target device through the browser.

[0014] In this way, during the screen mirroring process via URL, the user needs to remove the virtual reality device and enter the URL in the target settings. After the user removes the virtual reality device, the virtual reality device will play the screen mirroring access address through voice. The user can then enter the screen mirroring access address into the browser of the target device based on the voice prompt to complete the screen mirroring operation. This eliminates the need for the user to repeatedly put on and take off the virtual reality device or memorize the screen mirroring access address in advance, thereby improving the convenience of user operation.

[0015] In some embodiments of this application, a communication method for a virtual reality device is also provided. The method includes: displaying a first user interface on the display of the virtual reality device; receiving an input casting command; in response to receiving the input casting command, displaying a second user interface on the display of the virtual reality device; the second user interface includes information about a casting access address, the casting access address being an access address corresponding to a casting service provided by the virtual reality device; receiving an input first identification command for the casting access address; in response to receiving the input first identification command, obtaining first text information corresponding to the casting access address; broadcasting the first text information via the speaker of the virtual reality device to enable a user to input the casting access address into the browser of the target device; and after the access address is accessed by the target device, controlling the casting service to send the current interface of the virtual reality device to the target device through the browser. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 illustrates an operational scenario between a VR device and external devices such as display devices, smart devices, and servers, according to some embodiments.

[0018] Figure 2 shows a hardware configuration block diagram of a VR device 200 according to some embodiments;

[0019] Figure 3 illustrates the software configuration of a VR device 200 according to some embodiments;

[0020] Figure 4 shows an interaction flowchart between VR device 200 and target device according to some embodiments;

[0021] Figure 5 shows an icon control interface display diagram of an application in a VR device 200 according to some embodiments;

[0022] Figure 6 shows another interaction flowchart between VR device 200 and target device according to some embodiments;

[0023] Figure 7 shows another icon control interface display diagram of an application in a VR device 200 according to some embodiments;

[0024] Figure 8 shows another interaction flowchart between VR device 200 and target device according to some embodiments;

[0025] Figure 9 illustrates a flowchart of voice broadcasting from a VR device 200 according to some embodiments;

[0026] Figure 10 shows another icon control interface display diagram of an application in a VR device 200 according to some embodiments;

[0027] Figure 11 shows another interaction flowchart between VR device 200 and target device according to some embodiments;

[0028] Figure 12 shows another interaction flowchart between the VR device 200 and the target device according to some embodiments. Detailed Implementation

[0029] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0030] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0031] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0032] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0033] VR devices can be small display devices that can be worn on a user's head. VR devices achieve the effect of virtual reality by sending optical signals to the user's eyes, providing the user with an immersive visual experience.

[0034] In some embodiments, VR devices can interact with external devices such as display devices, smart devices, and servers.

[0035] Figure 1 is a schematic diagram of the operation scenario between the VR device and external devices such as display devices, smart devices, and servers according to the embodiment.

[0036] The operation scenario shown in Figure 1 may include a control device 100, a VR device 200, a display device 300, a server 400, and a smart device 500.

[0037] In some embodiments, the VR device 200 can interact with the display device 300. For example, the VR device 200 can project its displayed content onto the display device 300 so that other users can also view the content displayed on the VR device 200. The content displayed on the VR device 200 can be provided by the server 400.

[0038] In some embodiments, the VR device 200 can interact with the smart device 500. For example, users can install various applications such as games, video playback, social networking, and fitness apps on both the VR device 200 and the smart device 500. Users can then register or log in to accounts for these applications through either the VR device 200 or the smart device 500. Afterward, users can enjoy a wide variety of digital experiences and services through these applications. As another example, users can install a management application on the smart device 500 to manage the VR device 200; then, they can use this management application on the smart device 500 to associate with or reset one or more VR devices 200.

[0039] In some embodiments, the display device 300 may be a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc.

[0040] Figure 2 illustrates an exemplary configuration block diagram of a VR device 200 according to an exemplary embodiment. As shown in Figure 2, the VR device 200 may include at least one of a communicator 210, a controller 220, a display 230, a speaker 240, a memory, a power supply, and a user interface.

[0041] In some embodiments, the communicator 210 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The VR device 200 can establish the transmission and reception of control signals and data signals with the display device 300, the smart device 500, or the server 400 through the communicator 210.

[0042] The display 230 includes a display screen assembly for presenting images, a driving assembly for driving image display, a component for receiving image signals from the controller output, and a user control UI interface for displaying video content, image content, menu control interface, and user control UI interface.

[0043] The display 230 can be an LCD display, an OLED display, a head-mounted display (HMD), etc.

[0044] The controller 220 controls the operation of the VR device 200 and responds to user operations through various software control programs stored in the memory. The controller 220 controls the overall operation of the VR device 200. For example, in response to receiving a user command to select a UI object to display on the display 230, the controller 220 can perform operations related to the object selected by the user command.

[0045] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (random access memory), ROM (read-only memory), a first to an nth interface for input / output, a communication bus, etc.

[0046] Users can input commands through a graphical user interface (GUI) displayed on the monitor 230, and the user input interface receives the user input commands through the GUI. Alternatively, users can input commands by performing specific gestures, and the user input interface receives the user input commands by recognizing the gestures through sensors.

[0047] A "user interface" is the medium through which an application or operating system interacts and exchanges information with the user. It converts information from its internal form to a form that the user can accept. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0048] In some embodiments, an external controller, such as a gamepad, may also be included for use with the VR device 200. Users can interact with the VR device 200 using the gamepad.

[0049] Referring to Figure 3, in some embodiments, the VR device 200 system can be divided into four layers, from top to bottom: the Applications layer (hereinafter referred to as the "Application Layer"), the Application Framework layer (hereinafter referred to as the "Framework Layer"), the Android runtime and system library layer (hereinafter referred to as the "System Runtime Library Layer"), and the kernel layer.

[0050] In some embodiments, at least one application runs in the application layer. These applications can be Windows programs, system settings programs, or clock programs that come with the operating system; they can also be applications developed by third-party developers, such as screen mirroring, games, and video applications. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0051] In some embodiments, the application layer may further include a voice broadcast service, a text recognition service, and a screen mirroring service. The text recognition service can identify text information in an image and transmit the identified text information to the voice broadcast service. The voice broadcast service can convert the text information into speech information and broadcast the speech information through a speaker. The screen mirroring service can respond to a screen mirroring request from an external device and mirror the screen to the external device through the external device's browser.

[0052] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0053] As shown in Figure 3, the application framework layer in this embodiment includes Managers, Content Providers, View System, System Server, etc.

[0054] In some embodiments, the system runtime library layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system runs the C / C++ libraries contained in the system runtime library layer to implement the functions that the framework layer needs to perform.

[0055] In some embodiments, the kernel layer is a layer between hardware and software. As shown in Figure 3, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0056] In today's diverse digital entertainment and information-sharing environment, screen mirroring technology, as an important means of connecting different devices and achieving seamless content transmission, is increasingly favored by users. However, different screen mirroring methods have their own characteristics and limitations in terms of application scenarios, device compatibility, and implementation methods. This provides diverse choices for technology developers and users, but also brings certain challenges.

[0057] Mirroring is primarily used to copy the 2D interface of one device to another in real time. However, VR devices display a binocular 3D interface, and mirroring cannot effectively convey its depth information and immersive experience. Therefore, mirroring is not suitable for VR devices.

[0058] DLNA (Digital Living Network Alliance) screen casting is primarily designed for media and video streaming content. It relies on the support of the content source and requires a specific "TV" icon for identification and activation. The limitation of this method is that it requires the television program or video platform itself to integrate DLNA screen casting functionality, and it is only compatible with devices that support this standard.

[0059] In addition, screen mirroring and DLNA casting face challenges in cross-platform compatibility. Especially in screen mirroring attempts between VR devices and different operating systems (such as Android systems and non-Android laptops), due to differences in system architecture, protocol support, and software ecosystem, it is often difficult to achieve stable and efficient screen mirroring results.

[0060] Given the limitations of the aforementioned screen mirroring methods in VR devices, some embodiments employ a browser-based cross-platform reverse screen mirroring approach. As a cross-platform software tool, the browser exhibits unique advantages. It can mask system differences between different devices, enabling relatively flexible and widely compatible content sharing and screen mirroring through standard Web technologies and protocols (such as HTML5 and WebRTC). Browsers not only support direct playback of video streams but can also display complex interfaces through web pages, providing new ideas and possibilities for cross-platform screen mirroring.

[0061] The following is an example of a method for cross-platform reverse screen mirroring based on a browser.

[0062] As shown in Figure 4, the method for cross-platform reverse screen mirroring based on a browser can include the following steps:

[0063] S401, the VR device receives the input screen projection command.

[0064] Before receiving input casting commands, the VR device's display can show the first user interface.

[0065] In some embodiments, the first user interface may include a screen casting control. After the first user interface is displayed on the VR device's screen, the user can select a screen casting application control in the first user interface using a controller or gestures. After the user selects a screen casting application control in the first user interface, the VR device receives the input screen casting command.

[0066] In some embodiments, the first user interface may not include screen casting controls, and the user may trigger screen casting commands via voice or other means.

[0067] S402, in response to receiving an input screen mirroring command, displays a second user interface, which includes a screen mirroring access address.

[0068] In the browser-based cross-platform reverse projection method, the VR device 200 can have a projection service with a fixed projection access address pre-built. This projection service can be used to provide the content displayed in the VR device 200 to other devices.

[0069] After the VR device receives the input screen casting command, the VR device 200's display can show the second user interface as shown in Figure 5. The screen casting access address in the second user interface is the access address corresponding to the screen casting service.

[0070] S403, The target device receives the screen mirroring access address entered by the user into the browser of the target device.

[0071] In a browser-based cross-platform reverse projection solution, the target device needs to access the projection service in the VR device 200 through its browser to obtain the projection content provided by the service. Therefore, it is necessary to first enter the projection access address corresponding to the projection service in the target device's browser, which is the projection access address in the second user interface.

[0072] Because the VR device 200 is a closed device, users need to remove the VR device 200 before they can enter the screen mirroring access address displayed by the VR device 200 into the browser of the target device. However, for complex content like the screen mirroring access address, users usually cannot remember the screen mirroring access address accurately in one go. Therefore, users need to repeatedly put on the VR device 200 to view the screen mirroring access address, then remove the VR device 200 and continue to enter the screen mirroring access address into the browser of the target device until they enter the accurate and complete screen mirroring access address.

[0073] S404, after receiving the screen mirroring access address entered by the user, the target device sends a screen mirroring access request to the VR device's system service.

[0074] After the user enters the correct screen mirroring access address into the target device's browser, the target device can send a screen mirroring access request to the VR device's system service. The screen mirroring access request is used to request the VR device's screen mirroring service to send screen mirroring content to the target device.

[0075] S405, the system service sends a screen mirroring access request to the screen mirroring service.

[0076] After receiving the screen mirroring access request, the system service forwards the screen mirroring access request to the screen mirroring service.

[0077] S406, the screen mirroring service establishes a screen mirroring connection with the target device.

[0078] S407, the screen mirroring service sends a screen recording request to the screen mirroring application.

[0079] After establishing a screen mirroring connection with the target device, the screen mirroring service can send a screen recording request to the screen mirroring application.

[0080] For example, the screen mirroring service sends a single-directory screen request to the screen mirroring application to obtain a single-directory screen.

[0081] S408, the screen mirroring application records the foreground interface and generates the screen mirroring content.

[0082] After receiving the screen recording request from the screen recording service, the screen casting application begins recording the interface displayed in the foreground and generates the corresponding screen casting content.

[0083] S409, the screen mirroring application sends the screen mirroring content to the screen mirroring service.

[0084] S410, the screen mirroring service sends the screen mirrored content to the target device.

[0085] In some embodiments, after receiving the screen casting content, the screen casting service can encode, compress, or otherwise process the screen casting content, and then send the encoded, compressed, or otherwise processed screen casting content to the target device to improve the transmission effect.

[0086] S411, the target device displays the projected content through a browser.

[0087] In some embodiments, the target device may first decode and decompress the received screen projection content, and then display the decoded and decompressed screen projection content through a browser.

[0088] During step S403, the user needs to repeatedly put on and take off the VR device 200 before inputting an accurate and complete screen mirroring access address to the target device.

[0089] To improve the convenience of user operation in step S403, this application provides a VR device that can be configured with a text recognition function for the second user interface to recognize the text information corresponding to the screen projection access address in the second user interface. Then, the recognized text information can be read aloud via a voice broadcast service in the VR device. In this way, after the user removes the VR device, they can input the corresponding screen projection access address into the browser of the target device based on the voice broadcast, without needing to repeatedly put on and take off the VR device, thus improving the convenience of user operation.

[0090] To facilitate understanding of the solutions provided in the embodiments of this application, the following uses a VR device including software modules such as screen casting application, system services, text recognition services, screen casting services and voice broadcasting services as an example to illustrate the interaction process between the VR device and the target device.

[0091] As shown in Figure 6, the interaction process between the VR device and the target device includes the following steps:

[0092] S501, the VR device receives input screen projection commands.

[0093] S502, in response to receiving the input screen mirroring command, controls the screen mirroring application to display a second user interface, where the screen mirroring access address in the second user interface is the access address corresponding to the screen mirroring service.

[0094] The steps S501-S502 can be described in conjunction with the steps S401-S402, and will not be repeated here.

[0095] S503, the screen mirroring application receives input commands for recognizing the recognition area.

[0096] In some embodiments, after the VR device displays a second user interface, the user can select the content to be read aloud via voice by using a controller or gestures. The area selected by the user in the second user interface via controller or gestures can be referred to as the recognition area.

[0097] For example, as shown in Figure 7, the user can select "Please enter the following URL in the browser of the device that needs input: http: / / 192.168.1.100:1288 / steam" as the recognition area in the second user interface by operating the handle.

[0098] In some embodiments, the second user interface may include a recognition trigger control. For example, the elliptical AI control shown in Figure 7 is a recognition trigger. After the user selects a recognition area in the second user interface, they can trigger this AI control. Once the user triggers the AI ​​control, the screen mirroring application can receive the user's recognition command for the recognition area.

[0099] S504, in response to receiving an input command for recognizing the recognition area, the projection application selects the recognition area.

[0100] S505, the system service captures the image corresponding to the selected recognition area to obtain the image to be recognized.

[0101] S506, the system service sends the image to be recognized to the screen mirroring application.

[0102] S507, the screen mirroring application sends the image to be recognized to the text recognition service.

[0103] After the screen mirroring application selects a recognition area, it uses system services to capture the image corresponding to that area. Then, the system services send the image to be recognized to the text recognition service via the screen mirroring application for processing.

[0104] S508, the text recognition service identifies the text information corresponding to the screen projection access address based on the image to be recognized.

[0105] In some embodiments, the text recognition service can use artificial intelligence (AI) technology to identify the screen mirroring access address in the image to be recognized and obtain the text information corresponding to the screen mirroring access address.

[0106] For example, the text recognition service is an AI service, which can be a recognition model used to identify text content in an image. Thus, the image to be recognized can be input into this AI service, which can perform recognition processing based on the input image and output text information corresponding to the screen-sharing access address in the image.

[0107] In some embodiments, the text recognition service can use optical character recognition (OCR) technology to identify the projection access address in the image to be recognized, and obtain the text information corresponding to the projection access address. For details on OCR technology, please refer to related technologies; it will not be elaborated upon here.

[0108] S509, the text recognition service sends the recognized text information to the screen mirroring application.

[0109] On the S510, the screen mirroring application sends the recognized text information to the voice broadcast service.

[0110] S511, in response to receiving text information, the voice broadcast service starts voice broadcasting.

[0111] After the text recognition service identifies the text information corresponding to the screen mirroring access address, it sends the identified text information to the voice broadcast service through the screen mirroring application.

[0112] After receiving text information, the voice broadcast service can start voice broadcasting to read the received text information aloud.

[0113] S512, the target device receives the screen mirroring access address entered by the user into its browser.

[0114] During the process of reading text information aloud, users can remove their VR devices and enter the screen mirroring access address into the browser of the target device based on the voice broadcast.

[0115] S513, the system service receives the screen mirroring access request sent by the target device.

[0116] S514, the system service sends a screen mirroring access request to the screen mirroring service.

[0117] S515, the screen mirroring service sends a stop voice broadcast instruction message to the voice broadcast service.

[0118] S516, the voice broadcast service has stopped broadcasting.

[0119] If a user inputs the complete screen mirroring access address to the target device via voice prompts, the system service can receive the screen mirroring access request from the target device. Upon receiving the request, the screen mirroring service sends a stop voice broadcast instruction to the voice broadcast service to instruct it to cease voice broadcasting. Simultaneously, the screen mirroring service establishes a screen mirroring connection with the target device and executes the subsequent screen mirroring process.

[0120] The process of establishing a screen mirroring connection with the target device and executing the subsequent screen mirroring process can be found in the descriptions of steps S406 to S411, and will not be repeated here.

[0121] In summary, the screen projection method for VR devices provided in the above embodiments, after the VR device displays the projection access address, can convert the projection access address displayed on the VR device interface into corresponding text information through projection applications, system services, and text recognition services. Then, the text information is voice-broadcast through the VR device's voice broadcast service. In this way, after the user removes the VR device, they can input the corresponding projection access address into the target device's browser based on the voice broadcast, without requiring the user to repeatedly put on and take off the VR device or memorize the projection access address in advance, thereby improving the convenience of user operation.

[0122] To facilitate understanding of the solutions provided in the embodiments of this application, the following uses a VR device including hardware such as a display and a controller as an example to illustrate the interaction process between the VR device and the target device.

[0123] As shown in Figure 8, the interaction process between the VR device and the target device includes the following steps:

[0124] S601, when a user wears a VR device, a first user interface is displayed on the VR device's screen, and the first user interface includes screen casting application controls.

[0125] S602, the VR device's controller receives input screen mirroring commands.

[0126] After the first user interface is displayed on the VR device's screen, the user can select a screen casting application control from the first user interface using a controller or gestures. Once the user selects a screen casting application control from the first user interface, the VR device receives the input screen casting command.

[0127] S603, in response to receiving an input screen mirroring command, displays a second user interface on the VR device's display, the second user interface including a screen mirroring access address.

[0128] The steps S601 to S603 can be found in the description of steps S401 to S403, and will not be repeated here.

[0129] In step S604, the VR device controller receives an input command to identify the screen projection access address.

[0130] After receiving the input command to identify the screen projection access address, the VR device controller can execute steps S503 to S508. Please refer to the description of steps S503 to S508 for details, which will not be repeated here.

[0131] In step S605, the VR device controller obtains the text information corresponding to the screen projection access address.

[0132] After the VR device controller processes the data in step S604 above, it can obtain the text information of the screen projection access address.

[0133] It should be noted that the recognized text information is in text format, so that it can be read aloud via voice broadcast service.

[0134] Step S606: The text information is read aloud via the speaker of the VR device.

[0135] The VR device's controller can control the VR device to read text information aloud via speakers.

[0136] As shown in Figure 9, in some embodiments, the method for VR devices to broadcast text information via voice can be implemented in the following manner:

[0137] In step S6061, after the VR device obtains the text information corresponding to the screen projection access address, it displays the recognition result on the VR device's monitor.

[0138] In step S6062, the VR device receives the input broadcast command.

[0139] In step S6063, in response to receiving the input broadcast command, the VR device disables the screen-off function when not wearing the device.

[0140] Step S6064: After the VR device disables the screen-off function when not wearing the device, the VR device starts the voice broadcast service.

[0141] Step S6065: The VR device determines whether the current cumulative number of voice broadcasts has reached the maximum number of voice broadcasts.

[0142] Step S6066: If the current cumulative number of voice broadcasts has not reached the maximum number of voice broadcasts, then start broadcasting the text information via voice.

[0143] In step S6067, after the text information is read aloud via the speaker, the VR device determines whether it has detected a screen mirroring access request sent by the target device within the next preset time period.

[0144] Step S6068: If a screen mirroring access request sent by the target device is detected or the current cumulative number of voice broadcasts reaches the maximum number of voice broadcasts, then the voice broadcast service is turned off.

[0145] If no screen mirroring access request is detected from the target device, return to step S6064.

[0146] Step S6069: After turning off the voice broadcast service, turn on the screen-off function when not wearing a mask.

[0147] As exemplified in Figure 10, the recognition result may include text information and a voice broadcast selection control. For example, the voice broadcast selection control may include a voice broadcast control and a cancel control.

[0148] If the user selects the voice broadcast control, the VR device can receive the input broadcast command and then execute subsequent steps S6063 to S6068. If the user selects the cancel control, the VR device can receive the input cancel broadcast command, and the VR device will not execute subsequent steps S6063 to S6068.

[0149] In some implementations, VR devices have a screen-off function when not worn. When the screen-off function is enabled, the VR device's display switches from a lit state to a screen-off state if no user is wearing the VR device for a certain period of time. When the screen-off function is disabled, the VR device's display remains lit even if no user is wearing the VR device for a certain period of time.

[0150] In some embodiments, when the VR device's display is on, the VR device can read text information aloud normally. When the VR device's display is off, the VR device stops reading text information aloud.

[0151] Based on the above considerations, in step S6063, after the VR device receives the input voice broadcast command, it can first disable the timeout screen-off function when not wearing the device. This way, while the user is removing the VR device to input text information into the target device, the VR device's display remains on, thus ensuring that the VR device can normally broadcast text information via voice.

[0152] Furthermore, after disabling the voice broadcast service, the VR device can re-enable the screen-off function for unworn devices, so that if no user wears the VR device for a certain period of time, the VR device's display switches from a lit state to a screen-off state, thereby reducing the VR device's power consumption.

[0153] In some embodiments, after the voice broadcast service is started, the user may temporarily leave the current scene or have other matters preventing them from inputting complete text information to the target device. To avoid invalid repeated broadcasts in this situation, this embodiment of the application can, in step S6065 after starting the voice broadcast service, first determine whether the current cumulative number of voice broadcasts has reached the maximum number of voice broadcasts. If the maximum number of voice broadcasts has not been reached, the voice broadcast can be repeated once more. If the maximum number of voice broadcasts has been reached, the voice broadcast service can be turned off to end the voice broadcast.

[0154] In some embodiments, for shorter text messages, the user can input the correct text message into the target device after the VR device reads the text message aloud once. For longer text messages, the VR device may need to read the text message aloud multiple times before the user can input the correct text message into the target device. However, since the user is inputting text message into the target device while the VR device is reading the text message aloud, the user cannot control the VR device to stop or restart the voice reading.

[0155] In some embodiments, steps S6067 to S6068 described above can be used to control the VR device to stop or resume voice broadcasting.

[0156] In step S6067, after the text information is broadcast via speaker, if the user enters the accurate screen mirroring service address in the target device's browser within the next preset time period, the target device can send a screen mirroring access request to the VR device. Thus, after the text information is broadcast via speaker, if the VR device receives the screen mirroring access request within the next preset time period, the voice broadcasting service can be closed. If the VR device does not receive the screen mirroring access request within the next preset time period, the voice broadcasting service can be restarted for another voice broadcast, allowing the user to continue entering the complete screen mirroring access address to the target device based on voice broadcasting.

[0157] In this way, when the user removes the VR device, the VR device can automatically determine whether to repeat the voice broadcast after each voice broadcast by detecting whether it has received a screen projection access request from the target device. This ensures that the voice broadcast will automatically stop after the user enters complete text information on the target device, without requiring user intervention and improving user convenience.

[0158] Step S607: The target device receives the input text information.

[0159] After the user removes the VR device, they can enter the screen mirroring access address into the target device's browser via voice prompts from the VR device. Once the target device receives the correct address, it can send a screen mirroring access request to the VR device. Subsequently, upon receiving the request, the VR device's screen mirroring service can send the content to be mirrored to the target device, displaying the content shown on the VR device in the target device's browser.

[0160] In summary, the communication method for VR devices provided in the above embodiments can convert the screen projection access address displayed on the VR device's display into corresponding text information when the VR device's screen shows a second user interface. Then, the text information is aloud through the VR device's speaker. This allows users to input the corresponding screen projection access address into the target device's browser based on the voice prompt after removing the VR device, eliminating the need for repeated wearing and removing of the VR device or pre-memorizing the screen projection access address, thus improving user convenience.

[0161] It should be noted that the accuracy of text input by a user onto a target device is related to the length of the text. For example, the longer the text, the lower the accuracy of the text input.

[0162] The accuracy of text input by a user to a target device is also related to the complexity of the text information. For example, the more types of characters, such as numbers, letters, and symbols, included in the text information, the higher its complexity. Thus, the more types of characters included in the text information, the lower the accuracy of the text information input by the user to the target device.

[0163] In some embodiments, to improve the accuracy of text input by users to the target device, the following method can be used for voice broadcasting. As shown in Figure 11, the voice broadcasting method may include the following steps:

[0164] Step S701: After the VR device obtains the text information, it determines the length of the text information.

[0165] Step S702: If the length of the text information is less than the first length threshold, obtain the first short text broadcasting rule.

[0166] Step S703: Announce the message aloud according to the rules for the first short text.

[0167] In some embodiments, text information with a length less than a first length threshold can be referred to as short text information, and text information with a length greater than or equal to the first length threshold can be referred to as long text information.

[0168] For short text messages, the first short text broadcasting rule can be used for voice broadcasting.

[0169] In some embodiments, the first short text broadcasting rule may include: when the first target short text information includes characters of the same type, broadcasting the first target short text information at a first speech rate; when the first target short text information includes characters of different types, broadcasting the first target short text information at a second speech rate. The second speech rate is less than the first speech rate, and the first target short text information can be any type of short text information.

[0170] For example, when the first target short text information consists only of any one of the characters among numbers, letters, or symbols, the first target short text information can be read aloud at a faster first speech rate. For instance, if the first target short text information 1 is 89615, then 89615 can be read aloud at a faster first speech rate.

[0171] As another example, when the first target short text information includes any two or three characters among numbers, letters, or symbols, the first target short text information can be read aloud using a slower second speech rate. For example, if the first target short text information 2 is A2#3B, then A2#3B can be read aloud using a slower second speech rate.

[0172] Thus, when the complexity of the first target short text information is relatively low, a faster initial speech rate is used to read the first target short text information aloud, allowing the user to hear the entire text information as quickly as possible and complete the input to the target device, thereby enabling the target device to interact with the VR device based on the input text information as soon as possible. When the complexity of the first target short text information is relatively high, a slower initial speech rate is used to read the first target short text information aloud, which can better match the speed at which the user inputs the first target short text information to the target device, improving the accuracy of the user's input of the first target short text information.

[0173] Step S704: If the length of the text information is greater than or equal to the first length threshold, the text information is divided into multiple first short text information.

[0174] Step S705: Each first short text information is taken as the first target short text information and broadcast in sequence according to the first short text broadcasting rules.

[0175] Step S706: The target device receives the input text information.

[0176] In some embodiments, to improve the accuracy of users inputting long text information into the target device, the long text information can be divided into multiple first short text information. The length of each first short text information after division can be less than a first length threshold.

[0177] In some embodiments, long text information can be divided into multiple first short text information based on a first length threshold.

[0178] For example, taking the long text information http: / / 192.168.1.100:1288 / steam as an example, with a first length threshold of 4, http: / / 192.168.1:1288 / steam can be divided into the following multiple first short text information: "http", ": / / 1", "92.1", "68.1", ":128", "8 / st", and "eam".

[0179] Then, following the first short text broadcasting rules, "http", ": / / 1", "92.1", "68.1", ":128", "8 / st", and "eam" can be broadcast in sequence. For example, the first short text information containing the same type of characters, such as "http" and "eam", can be broadcast using the first speech speed; while the first short text information containing different types of characters, such as ": / / 1", "92.1", "68.1", ":128", and "8 / st", can be broadcast using the second speech speed.

[0180] In some embodiments, long text information can be divided into multiple first short text information based on a first length threshold and character type.

[0181] For example, while satisfying the first length threshold, efforts should be made to ensure that each segmented first short text information includes as few character types as possible.

[0182] Continuing with the example of the long text information http: / / 192.168.1.100:1288 / steam, with a first length threshold of 4, http: / / 192.168.1:1288 / steam can be divided into the following multiple first short text information: "http", ": / / ", "192", ".168", ".1:1", "288", " / st", and "eam".

[0183] In this way, the first short text information with the same type of characters, such as "http", ": / / ", "192", "288" and "eam", can be broadcast using the first speech speed, while the first short text information with different types of characters, such as ": / / 1", ".168", ".1:1" and " / st", can be broadcast using the second speech speed.

[0184] Compared to the two classification methods mentioned above, the second classification method can use the first speech rate to broadcast more short text information, thereby improving the broadcasting efficiency of long text information.

[0185] In some embodiments, the process of the VR device broadcasting any first target short text message may be interrupted. For example, when the VR device receives an activity message pushed by the server, the VR device responds to the activity message first, thus interrupting the ongoing voice broadcast. After the VR device has responded to the activity message, it can resume the voice broadcast.

[0186] When voice playback is interrupted, the user may have already input some text into the target device, and the user may not be able to accurately pinpoint the character at which the interruption occurred. Therefore, if the VR device resumes playback from the interrupted character, the characters the user continues to input may not constitute complete and accurate text information compared to the characters input before the interruption. For example, one or more characters may be missing between the characters input after the interruption. Or, one or more extra characters may be included between the characters input after the interruption.

[0187] In some embodiments, in the event of an interruption in voice broadcasting, to improve the accuracy of the user's input of text information to the target device, the following method can be used for voice broadcasting. As shown in Figure 12, this voice broadcasting method may include the following steps:

[0188] Step S901: If the voice broadcast of any first target short text information is interrupted during the process of broadcasting it through the speaker, the length of the text information is re-determined after the interruption is restored.

[0189] In some embodiments, the audio channel or processing resources may be interrupted during voice broadcasting due to other operations. After the interruption is completed, i.e., after the interruption is restored, the VR device can resume voice broadcasting.

[0190] When re-broadcasting text messages, different broadcasting strategies can be adopted for text messages of different lengths to improve the accuracy and efficiency of users inputting text messages into the target device.

[0191] Step S902: If the length of the text information is less than the second length threshold, the text information is re-broadcast by voice according to the first short text broadcasting rule.

[0192] If the length of the text information is less than the second length threshold, the text information is short. In this case, after the interruption is resolved, the first short text broadcasting rule can be used to re-broadcast the text information.

[0193] The rules for broadcasting the first short article can be found in the description of step S703, and will not be repeated here.

[0194] Step S903: If the length of the text information is greater than or equal to the second length threshold, determine at least one second short text information before interruption and at least one third short text information after interruption.

[0195] In some embodiments, if the text information is divided into multiple first short text messages, the VR device can record the location of the breakpoint short text message where the voice playback is interrupted. Then, the breakpoint short text message and each first short text message preceding it can be designated as a second short text message, and each first short text message following it can be designated as a third short text message. Alternatively, each first short text message preceding the breakpoint can be designated as a second short text message, and each first short text message following it can be designated as a third short text message.

[0196] Step S904: Obtain the second short article broadcasting rules.

[0197] The second short text broadcasting rules may include: when the second target text information contains characters of the same type, broadcasting the second target text information at a third speech rate; when the second target text information contains characters of different types, broadcasting the second target text information at a fourth speech rate. The third speech rate is greater than the first speech rate, and the fourth speech rate is greater than the second speech rate.

[0198] Step S905: Each second short text message and each third short text message are respectively used as the second target short text message. Each second short text message is re-broadcast by voice according to the second short text message broadcasting rules. And, each third short text message is continued to be broadcast by voice according to the first short text message broadcasting rules.

[0199] Since each of the second short text messages was already read before the interruption, the purpose of rereading them is primarily to allow the user to verify the input and pinpoint the interruption point. Therefore, the second short text messages can be reread at a faster pace than before the interruption.

[0200] Since each of the third short text messages was content that had not been read before the interruption, in order to match the user's input speed, the third short text messages can continue to be read aloud according to the first short text reading rules.

[0201] Step S906: The target device receives the input text information.

[0202] In this way, when voice broadcasting is interrupted, different broadcasting strategies are adopted for text information of different lengths and for text information before and after the interruption, thereby improving the accuracy and efficiency of users inputting text information into the target device.

[0203] It should be noted that the above embodiments are merely examples of VR devices implementing cross-platform reverse screen projection based on a browser, used to illustrate the solutions provided in this application, and do not imply any limitation on the application scenarios to which the solutions provided in this application are applicable. For example, the solutions provided in this application can also be applied to application scenarios such as binding and unbinding VR devices.

[0204] To easily distinguish between the three application scenarios, the recognition commands and text information corresponding to the cross-platform reverse screen projection scenario using a VR device via a browser can be referred to as the first recognition command and the first text information, respectively. The recognition commands and text information corresponding to binding a VR device can be referred to as the second recognition command and the second text information, respectively. The recognition commands and text information corresponding to unbinding a VR device can be referred to as the third recognition command and the third text information, respectively.

[0205] The following examples illustrate the methods provided in this application embodiment, using the application scenarios of binding and unbinding VR devices as examples.

[0206] Users can register an account for a management application used to manage VR devices via target devices such as mobile phones and computers. After registering an account, the account can be linked to the VR device.

[0207] In VR device pairing scenarios, the VR device can first establish a connection with the target device via Bluetooth or Wi-Fi. After the target device logs into its management application account, it prompts the user to select the VR device to pair. Once the user selects a VR device to connect to the target device, a third-party user interface is displayed on the selected VR device, which includes a pairing verification code. For example, the pairing verification code might be A#16B5.

[0208] Afterwards, the VR device can announce the binding verification code A#16B5 via voice, allowing the user to input the binding verification code A#16B5 into the target device based on the voice announcement, so that the target device can bind the VR device to the account of the management application logged in on the target device.

[0209] After the user successfully binds the VR device by entering the binding verification code A#16B5 into the target device, the target device or server can send a binding verification successful message to the VR device. Upon receiving this message, the VR device can then stop its voice broadcast.

[0210] It's important to note that an account within the same management application can be linked to one or more VR devices. Similarly, one can also reset the binding of one or more VR devices to the same account, effectively unlinking them from the account.

[0211] In the scenario of unbinding VR devices, users can launch the management application installed on the target device. Then, they can enter the device management interface, which displays a list of VR devices bound to the account logged into on the target device. Users can then select one VR device from the list to unbind it.

[0212] For example, after a user selects to unbind VR device 1, a fourth user interface is displayed on the screen of VR device 1. The fourth user interface includes an unbinding verification code. This unbinding verification code is used to verify whether the unbound VR device is VR device 1.

[0213] Afterwards, VR device 1 can announce the unbinding verification code via voice. For example, the unbinding verification code is the device serial number corresponding to VR device 1, such as 1234-4556-678, etc.

[0214] During the process of VR device 1 broadcasting the unbinding verification code, the user can input the unbinding verification code displayed by VR device 1 to the target device based on the voice broadcast, so that the target device can unbind VR device 1 from the account logged in on the target device.

[0215] After the unbinding verification code is successfully verified, the target device or server can send a message indicating that the unbinding verification has been successful to VR device 1. Upon receiving this message, VR device 1 can then stop the voice broadcast.

[0216] It should be noted that the above embodiments are only illustrative examples of binding or unbinding an application account with a VR device, and do not imply any limitation on application accounts. Accounts for other applications can also be bound or unbound to VR devices.

[0217] For example, when the VR device receives a binding command for a first application account corresponding to the target device, it displays a third user interface including a binding verification code on the VR device's screen. Then, upon receiving a second recognition command, it obtains the second text information corresponding to the binding verification code. Afterward, it broadcasts the second text information including the binding verification code via a speaker, prompting the user to input the binding verification code into the target device, allowing the target device to bind the VR device to the first application account.

[0218] In another example, when the VR device receives an unbinding command from a second application account corresponding to the target device, it displays a fourth user interface including an unbinding verification code on the VR device's screen. Then, upon receiving a third recognition command, it obtains third text information corresponding to the unbinding verification code. Afterwards, it broadcasts the third text information including the unbinding verification code via a speaker, prompting the user to input the unbinding verification code into the target device, allowing the target device to unbind the VR device from the second application account.

[0219] It should be noted that the embodiments of this application are only illustrative examples of three application scenarios that require inputting information displayed on a VR device into a target device, and do not imply any limitation on the application scenarios to which the solutions provided in these embodiments are applicable. It should be understood that the solutions provided in these embodiments can be applied to any scenario that requires inputting information displayed on a VR device into a target device, and these scenarios are not listed here.

[0220] It should also be noted that, unless otherwise specified, one or more embodiments of this specification and the features thereof can be combined with each other. This specification is not limited to any single aspect, nor to any single embodiment, nor to any combination and / or substitution of such aspects and / or embodiments. Furthermore, each aspect and / or embodiment of one or more embodiments of this specification can be used alone or in combination with one or more other aspects and / or embodiments thereof.

[0221] According to the method provided in the embodiments of this application, the embodiments of this application also provide a computer storage medium storing a computer program or instructions, which, when run on a computer, causes the computer to perform the method of any one of the embodiments of the method.

[0222] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0223] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

Claims

1. A virtual reality device, characterized in that, include: The controller is configured to display a first user interface on the display of the virtual reality device; Receive input screen mirroring commands; In response to receiving the input screen casting command, a second user interface is displayed on the display of the virtual reality device; the second user interface includes information about the screen casting access address, which is the access address corresponding to the screen casting service provided by the virtual reality device; and a first identification command for the screen casting access address is received. In response to receiving a first recognition command, the system obtains first text information corresponding to the screen projection access address; and broadcasts the first text information via the speaker of the virtual reality device so that the user can input the screen projection access address into the browser of the target device. After the access address is accessed by the target device, the screen mirroring service is controlled to send the current interface of the virtual reality device to the target device through the browser.

2. The virtual reality device according to claim 1, characterized in that, After the first text information is read aloud via the speaker of the virtual reality device, the controller is further configured to: within a preset time period, determine whether a screen projection access request sent by the target device to the virtual reality device is detected; if a screen projection access request sent by the target device to the virtual reality device is detected, then the voice reading ends; if no screen projection access request sent by the target device to the virtual reality device is detected, then the first text information is read aloud again via the speaker of the virtual reality device.

3. The virtual reality device according to claim 1, characterized in that, The controller is configured to broadcast the first text information via the speaker of the virtual reality device, comprising: the controller being configured to: determine the length of the first text information; if the length of the first text information is less than a first length threshold, then obtain a first short text broadcasting rule; wherein the first short text broadcasting rule comprises: when the first target short text information includes characters of the same type, broadcasting the first target short text information at a first speech rate; when the first target short text information includes characters of different types, broadcasting the first target short text information at a second speech rate, wherein the second speech rate is less than the first speech rate; and using the first text information as the first target short text information, broadcasting it via the speaker of the virtual reality device according to the first short text broadcasting rule.

4. The virtual reality device according to claim 3, characterized in that, The controller is further configured to: divide the first text information into multiple first short text information if the length of the first text information is greater than or equal to a first length threshold; the length of the first short text information is less than the first length threshold; obtain the first short text broadcasting rules; and take each of the first short text information as a first target short text information and broadcast it sequentially through the speaker of the virtual reality device according to the first short text broadcasting rules.

5. The virtual reality device according to claim 4, characterized in that, The controller is further configured to: if the voice broadcast of any first target short text information is interrupted during the process of broadcasting it through the speaker, then after the interruption is restored, redetermine the length of the first text information; if the length of the first text information is less than a second length threshold, then re-broadcast the first text information according to the first short text broadcasting rules; if the length of the first text information is greater than or equal to the second length threshold, then determine at least one second short text information before the interruption and at least one third short text information after the interruption. Obtain a second short text broadcasting rule; wherein the second short text broadcasting rule includes: when the second target short text information includes the same type of characters, broadcast the second target short text information using a third speech rate; when the second target short text information includes different types of characters, broadcast the second target short text information using a fourth speech rate; the third speech rate is greater than the first speech rate, and the fourth speech rate is greater than the second speech rate; each of the second short text information and the third short text information is respectively used as the second target short text information, and each of the second short text information is re-broadcast using the speaker of the virtual reality device according to the second short text broadcasting rule; and each of the third short text information is continued to be broadcast using the speaker of the virtual reality device according to the first short text broadcasting rule.

6. The virtual reality device according to any one of claims 1-5, characterized in that, Before the first text information is broadcast aloud via the speaker of the virtual reality device, the controller is further configured to: disable the screen-off function for screens that are not worn for extended periods; after the broadcast of the first text information ends, the controller is further configured to: enable the screen-off function for screens that are not worn for extended periods.

7. The virtual reality device according to claim 6, characterized in that, After disabling the screen-off function for unworn devices during timeout, the controller is further configured to: determine whether the current cumulative number of voice broadcasts has reached the maximum number of voice broadcasts; if the current cumulative number of voice broadcasts has not reached the maximum number of voice broadcasts, then broadcast the first text information by voice; if the current cumulative number of voice broadcasts has reached the maximum number of voice broadcasts, then end the voice broadcast of the first text information.

8. The virtual reality device according to any one of claims 1-5, characterized in that, The controller is configured to: receive a binding command for a first application account corresponding to a target device; in response to receiving the binding command, display a third user interface on the display; the third user interface includes a binding verification code; receive an input second recognition command for the binding verification code; and in response to receiving the input second recognition command, obtain second text information corresponding to the binding verification code. The speaker broadcasts a second text message, including the binding verification code, so that the user can input the binding verification code into the target device, allowing the target device to bind the virtual reality device to the first application account.

9. The virtual reality device according to any one of claims 1-5, characterized in that, The controller is configured to: receive an unbinding command from a second application account corresponding to a target device; in response to receiving the unbinding command, display a fourth user interface on the display; the fourth user interface includes an unbinding verification code; receive an input third recognition command for the unbinding verification code; in response to receiving the input third recognition command, obtain third text information corresponding to the unbinding verification code; and broadcast the third text information corresponding to the unbinding verification code via the speaker, so that the user inputs the unbinding verification code into the target device, allowing the target device to unbind the virtual reality device from the second application account.

10. A communication method applied to a virtual reality device, characterized in that, The term includes: A first user interface is displayed on the monitor of the virtual reality device; Receive input screen mirroring commands; In response to receiving the input screen casting command, a second user interface is displayed on the display of the virtual reality device; the second user interface includes information about the screen casting access address, which is the access address corresponding to the screen casting service provided by the virtual reality device; and a first identification command for the screen casting access address is received. In response to receiving a first recognition command, the system obtains first text information corresponding to the screen projection access address; and broadcasts the first text information via the speaker of the virtual reality device so that the user can input the screen projection access address into the browser of the target device. After the access address is accessed by the target device, the screen mirroring service is controlled to send the current interface of the virtual reality device to the target device through the browser.

Images