Screen projection display method of VR device and VR device

Abstract

The application provides a screen projection display method of a VR device and the VR device, which are used for screen projection through the VR device and improve user experience. The method comprises the following steps: after receiving a screen projection request sent by any terminal device connected with the VR device in the same local area network, a connection with the terminal device is established; when receiving screen projection data sent by the terminal device, a map position coordinate of the terminal device in a map coordinate system is converted into a screen projection position coordinate of the terminal device in a screen projection display coordinate system; according to a current position of the VR device, a target display area for displaying the screen projection data is constructed; based on the number of terminal devices sending the screen projection request, the position of at least one screen projection window in the target display area is determined; for any terminal device, the screen projection position coordinate of the terminal device and the position of the at least one screen projection window are used to determine a target screen projection window corresponding to the terminal device; and the screen projection data sent by the terminal device is displayed through the target screen projection window of the terminal device.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of virtual reality, and particularly relates to a screen projection display method of a VR device and the VR device. BACKGROUND

[0002] The screen projection technology refers to the following steps: first, connecting a screen projection source device to a wireless network in a use environment (such as a family, an office, a hotel, etc.); then, the screen projection source device sends screen projection data (such as a screen image of the screen projection source device) to a screen projection target device; and finally, the screen projection target device locally outputs and plays the screen projection data, thereby realizing the function of projecting the screen content of the screen projection source device onto the screen projection target device for display.

[0003] At present, in many scenarios, relevant screen projection content needs to be displayed on a large display screen (such as a television, etc.). For example, in an education and training scenario, interaction between a teacher and students is indispensable, and the teacher can master the learning dynamics and homework completion of the students through the screen projection content of the students. However, the display position of this screen projection mode is fixed, and the screen projection display mode is a 2D (two dimensional) display, which leads to poor user experience. SUMMARY

[0004] The present application provides a screen projection display method of a VR device and the VR device, which are used for screen projection display through the VR device. The target screen projection window of a terminal device for screen projection can be determined according to the position of the terminal device, and the screen projection display mode is a three-dimensional display. Therefore, the user experience is improved.

[0005] In a first aspect, an embodiment of the present application provides a screen projection display method of a VR device, and the method comprises the following steps.

[0006] After receiving a screen projection request sent by any one of terminal devices connected to the VR device in the same local area network, a connection is established with the terminal device.

[0007] After receiving screen projection data sent by the terminal device, a map position coordinate of the terminal device in a map coordinate system is converted into a screen projection position coordinate of the terminal device in a screen projection display coordinate system, wherein the map position coordinate is obtained based on a SLAM space positioning technology.

[0008] According to the current position of the VR device, a target display area for displaying the screen projection data is constructed.

[0009] Based on the number of terminal devices sending the screen projection request, the position of at least one screen projection window in the target display area is determined, wherein the number of screen projection windows is obtained based on the number of terminal devices.

[0010] For any one terminal device, the target projection window corresponding to the terminal device is determined by using the projection position coordinates of the terminal device and the positions of the at least one projection window; and

[0011] The projection data sent by the terminal device is displayed through the target projection window of the terminal device.

[0012] The second aspect of the application provides a VR device, comprising a processor and a memory, the processor and the memory are connected through a bus;

[0013] The memory stores a computer program, and the processor is configured to execute the following operations based on the computer program:

[0014] After receiving a projection request sent by any one terminal device connected in the same local area network as the VR device, a connection is established with the terminal device;

[0015] When receiving the projection data sent by the terminal device, the map position coordinates of the terminal device in the map coordinate system are converted into the projection position coordinates of the terminal device in the projection display coordinate system, wherein the map position coordinates are obtained based on the SLAM space positioning technology;

[0016] According to the current position of the VR device, a target display area for displaying the projection data is constructed;

[0017] Based on the number of terminal devices sending the projection request, the positions of at least one projection window in the target display area are determined, wherein the number of projection windows is obtained based on the number of terminal devices;

[0018] For any one terminal device, the target projection window corresponding to the terminal device is determined by using the projection position coordinates of the terminal device and the positions of the at least one projection window; and

[0019] The projection data sent by the terminal device is displayed through the target projection window of the terminal device.

[0020] According to the third aspect provided by the embodiment of the application, a computer storage medium is provided, the computer storage medium stores a computer program, and the computer program is used to execute the method according to the first aspect.

[0021] In the above embodiments of the application, after the VR device establishes a connection with any one of the terminal devices sending a screen projection request in the same local area network, and when receiving the screen projection data sent by the terminal device, the VR device converts the map position coordinates of the terminal device in the map coordinate system into screen projection position coordinates of the terminal device in the screen projection display coordinate system, and according to the current position of the VR device, constructs a target display area for displaying the screen projection data, and then determines the position of at least one screen projection window in the target display area based on the number of terminal devices sending a screen projection request, and then for any one terminal device, determines the target screen projection window corresponding to the terminal device by using the screen projection position coordinates of the terminal device and the position of the at least one screen projection window, and displays the screen projection data sent by the terminal device through the target screen projection window of the terminal device. Thus, in the embodiments of the application, screen projection display is realized through the VR device, and the target screen projection window of the terminal device can be determined according to the position of the terminal device for screen projection, and the screen projection display is in a three-dimensional display manner, so that the user experience is improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings described below are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0023] Figure 1 An exemplary scene schematic diagram of the screen projection display method of the VR device provided by the embodiments of the application is shown;

[0024] Figure 2 An exemplary flowchart of the screen projection display method of the VR device provided by the embodiments of the application is shown;

[0025] Figure 3 An exemplary screen projection display logic diagram of the VR device provided by the embodiments of the application is shown;

[0026] Figure 4 An exemplary diagram of the map coordinate system and the screen projection coordinate system provided by the embodiments of the application is shown;

[0027] Figure 5 An exemplary diagram of the map coordinate system and the screen projection coordinate system provided by the embodiments of the application is shown;

[0028] Figure 6 An exemplary flowchart of determining each screen projection window provided by the embodiments of the application is shown;

[0029] Figure 7An exemplary schematic diagram of a target ray in a target display area is shown;

[0030] Figure 8 An exemplary schematic diagram of a target display area is shown;

[0031] Figure 9 An exemplary schematic diagram of a target display area is shown;

[0032] Figure 10 An exemplary schematic diagram of a target display area is shown;

[0033] Figure 11 An exemplary schematic diagram of a target display area is shown;

[0034] Figures 12A-12C An exemplary schematic diagram of a target display area is shown;

[0035] Figures 13A-13B An exemplary schematic diagram of a target display area is shown;

[0036] Figure 14 An exemplary schematic diagram of a target display area is shown;

[0037] Figure 15 An exemplary schematic diagram of a target display area is shown;

[0038] Figure 16 An exemplary schematic diagram of a target display area is shown. DETAILED DESCRIPTION

[0039] For the purpose of making the objects, implementations and advantages of the present application more clear, the following will combine the drawings in the exemplary embodiments of the present application to make a clear and complete description of the exemplary implementations of the present application. Obviously, the described exemplary embodiments are only a part of the embodiments of the present application, but not all the embodiments.

[0040] Based on the exemplary embodiments described in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative labor fall within the scope of the claims of the present application. In addition, although the disclosure in the present application is introduced according to one or more examples, it should be understood that each aspect of the disclosure can also constitute a complete embodiment.

[0041] It should be noted that the brief description of the terms in this application is only for the convenience of understanding the embodiments described next, and is not intended to limit the embodiments of the application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0042] The terms "first", "second", etc. in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclusive inclusion, for example, a product or device including a series of components does not necessarily limit to the clearly listed components, but can include other components not clearly listed or inherent to these products or devices.

[0043] The term "module" used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware or / and software code capable of performing functions related to the element.

[0044] The idea of the embodiments of the present application is summarized as follows.

[0045] At present, the display position of the screen projection mode is fixed and the screen projection display mode is 2D (two-dimensional) display, resulting in poor user experience. Therefore, the VR device screen projection display method provided in the embodiments of the present application converts the map position coordinates of the terminal device in the map coordinate system into the screen projection position coordinates of the terminal device in the screen projection display coordinate system according to the current position of the VR device, constructs a target display area for displaying the screen projection data, determines the position of at least one screen projection window in the target display area based on the number of terminal devices sending screen projection requests, determines the target screen projection window corresponding to the terminal device according to the screen projection position coordinates of the terminal device and the position of the at least one screen projection window, and displays the screen projection data sent by the terminal device through the target screen projection window of the terminal device. Thus, the screen projection display through the VR device is realized in the embodiments of the present application, the target screen projection window of the terminal device can be determined according to the position of the terminal device for screen projection, and the screen projection display mode is three-dimensional display, so the user experience is improved. The VR device screen projection display method in the present application will be described in detail below in combination with the drawings.

[0046] AsFigure 1 Fig. 1 is a schematic diagram of an application scenario of a screen projection display method of a VR device in an embodiment of the present application. The application scenario includes a VR device 110 and a terminal device 120.

[0047] In a possible application scenario, the VR device 110 establishes a connection with any one of the terminal devices 120 in the same local area network after receiving a screen projection request sent by the terminal device 120. When the VR device 110 receives screen projection data sent by the terminal device 120, the VR device 110 converts a map position coordinate of the terminal device 120 in a map coordinate system into a screen projection position coordinate of the terminal device 120 in a screen projection display coordinate system, where the map position coordinate is obtained based on a SLAM space positioning technology. Then, the VR device 110 constructs a target display area for displaying the screen projection data according to a current position of the VR device 110, and determines a position of at least one screen projection window in the target display area based on a number of terminal devices sending screen projection requests, where the number of screen projection windows is obtained based on the number of terminal devices. Then, for any one of the terminal devices 120, the VR device 110 determines a target screen projection window corresponding to the terminal device 120 by using the screen projection position coordinate of the terminal device 120 and the position of the at least one screen projection window, and displays the screen projection data sent by the terminal device 120 through the target screen projection window of the terminal device 120.

[0048] wherein, Figure 1 The VR device 110 and the terminal device 120 in the application scenario can perform information interaction through a communication network. The communication network can adopt a wireless communication mode or a wired communication mode.

[0049] For example, the VR device 110 can access a network and communicate with the terminal device 120 through a cellular mobile communication technology, where the cellular mobile communication technology includes, for example, a 5th Generation Mobile Networks (5G) technology.

[0050] Optionally, the VR device 110 can access a network and communicate with the terminal device 120 through a short-range wireless communication mode, where the short-range wireless communication mode includes, for example, a Wireless Fidelity (Wi-Fi) technology.

[0051] In the description of the present application, only a single VR device 110 and three terminal devices 120 are described in detail, but those skilled in the art should understand that the VR device 110 and the terminal device 120 shown are intended to represent the operation of the VR device 110 and the terminal device 120 involved in the technical solution of the present application. It is not implied that there is a limitation on the number, type or location of the VR device 110 and the terminal device 120. It should be noted that if additional modules are added to the illustrated environment or individual modules are removed therefrom, the underlying concept of the example embodiments of the present application will not change.

[0052] It should be noted that the screen projection display method of the VR device proposed in the present application is not only applicable to the application scenarios shown, but also applicable to any screen projection display device with a VR device. Figure 1

[0053] The screen projection display method of the VR device according to the example embodiments of the present application will be described below in conjunction with the above-described application scenarios and with reference to the accompanying drawings. It should be noted that the above-described application scenarios are only shown to facilitate understanding of the method and principles of the present application, and the embodiments of the present application are not limited in this respect.

[0054] As shown in Figure 2 , a flowchart of the screen projection display method of the VR device can include the following steps:

[0055] Step 201: After receiving a screen projection request sent by any one of the terminal devices connected to the VR device in the same local area network, establish a connection with the terminal device;

[0056] The Http Server and the WebSocket Server run in the VR device in the embodiments of the present application. The Http Server in the embodiments of the present application is used to provide access to web page files, and the browser accesses resources in the web page. The screen recording logic is written in the web page. The WebSocket Server in the embodiments of the present application is used to receive the connection established by the browser, and is used to receive the screen projection data (screen recording data). The web page runs in the browser, and is used to send the screen projection data to the VR device. The StreamBuffer temporarily stores the screen projection data, and when the player in the VR device reads the screen projection data, the StreamBuffer sends the screen projection data to the player in the VR device for playing.

[0057] As shown in Figure 3 ​As shown, a schematic diagram of a screen projection display logic of a VR device is shown, a user accesses an HTTP Server through a browser in a terminal device to open a webpage, so as to connect with the VR device through the Websocket connection logic in the webpage when operating screen projection, and then send the screen projection data to the WebSocket Server at the VR device end, so that the VR device receives the screen projection data, and temporarily stores the screen projection data in the StreamBuffer, and when the VR device determines the target screen projection window corresponding to the terminal device, the screen projection data is obtained from the StreamBuffer and played through the player of the VR device.

[0058] Step 202: When receiving the screen projection data sent by the terminal device, the map position coordinates of the terminal device in the map coordinate system are converted into the screen projection position coordinates of the terminal device in the screen projection display coordinate system, wherein the map position coordinates are obtained based on the SLAM space positioning technology.

[0059] In an embodiment, the map position coordinates of the terminal device can be converted into the screen projection position coordinates in the following two ways:

[0060] Method one: If the coordinate system parameters of the map coordinate system and the coordinate system parameters of the screen projection display coordinate system are the same, the map position coordinates are scaled by a specified multiple to obtain the screen projection position coordinates of the terminal device, wherein the coordinate system parameters include the origin position, the position and direction of the horizontal coordinate axis, the position and direction of the vertical coordinate axis, and the position and direction of the vertical coordinate axis.

[0061] In the embodiment of the application, whether the map coordinate system or the screen projection coordinate system is a space rectangular coordinate system established with the center point of the VR device as the origin. And the coordinate system parameters of the map coordinate system and the screen projection coordinate system are pre-set, for example, as shown in Figure 4 The map coordinate system and the screen projection coordinate system are both established with the center point of the VR device as the origin, and the positions and directions of the coordinate axes are the same.

[0062] It should be noted that the specified multiple in the embodiment of the application can be set according to actual conditions, and the specified multiple is not limited in the embodiment of the application.

[0063] Method two: If the coordinate system parameters of the map coordinate system and the coordinate system parameters of the screen projection display coordinate system are not the same, the map position coordinates are converted by using a pre-set rotation and translation matrix between the map coordinate system and the screen projection display coordinate system to obtain the screen projection position coordinates of the terminal device.

[0064] As shown in Figure 5As shown, the directions of the x-axis and the y-axis of the map coordinate system are not the same as those of the projection display coordinate system. As can be seen from the figure, the position of p(x, y, z) in the map coordinate system is converted to (x1, y1, z1) in the projection display coordinate system.

[0065] Step 203: constructing a target display area for displaying the projection data according to the current position of the VR device;

[0066] In an embodiment, step 203 can be specifically implemented as: emitting an initial ray to the front of the VR device with the current position of the VR device as the origin; determining a spatial area corresponding to a first specified angle of the left and right field of view and a second specified angle of the upper and lower field of view of the initial ray as the target display area.

[0067] It should be noted that the range of the first specified angle in the embodiment of the present application can be (0°, 160°), and the range of the second specified angle can be (0°, 90°). However, the specific angle values of the first specified angle and the second specified angle can be set according to actual conditions, and the embodiment of the present application does not limit the specific angle values of the first specified angle and the second specified angle.

[0068] Step 204: determining the positions of at least one projection window in the target display area based on the number of terminal devices sending the projection request, wherein the number of projection windows is obtained based on the number of terminal devices;

[0069] Next, the positions of the projection windows in the target display area will be described in detail, as shown in Figure 6 FIG. 2 is a flowchart for determining the positions of the projection windows, including the following steps:

[0070] Step 601: emitting a target number of rays in the target display area with the current position of the VR device as the origin, wherein the target number is obtained based on the number of terminal devices sending the projection request, and the target number of rays are emitted in the same horizontal plane, and the included angle between any two adjacent rays is the same;

[0071] The target number in the embodiment of the present application is equal to the number of terminal devices sending the projection request. In the embodiment of the present application, the included angle between any two adjacent rays is obtained by dividing the first specified angle by the target number. As shown in Figure 7 Figure 7 In FIG. 3, the target number is 5, and the included angle between any two adjacent rays is θ.

[0072] ​Step 602: determining the display type and the position of the screen projection window corresponding to the target number of rays according to the screen projection position coordinates of the terminal device sending the screen projection request and the current position of the VR device.

[0073] The display type of the screen projection window in the embodiments of the present application includes a single-layer display type and a multi-layer display type, the single-layer display type is that the number of screen projection windows corresponding to any one ray is one, and the multi-layer display type is that the number of screen projection windows corresponding to any one ray is greater than one.

[0074] In one embodiment, the display type and the position of the screen projection window corresponding to the target number of rays can be determined in the following two ways:

[0075] Method one: if the number of terminal devices sending the screen projection request is less than a specified number, it is determined that the display type is a single-layer display type, and for any one ray, the position of the screen projection window corresponding to the ray is obtained according to a target length, wherein the target length is obtained based on the screen projection position coordinates of the terminal device sending the screen projection request and the current position of the VR device.

[0076] In one embodiment, the position of the screen projection window corresponding to any one ray can be obtained in the following way: for any one ray, the position of a target point with a distance between the end point of the ray and the target point as the target length is determined as the position of the screen projection window corresponding to the ray.

[0077] As shown in FIG. 1, Figure 8 As shown in FIG. 1, the position of the screen projection window corresponding to ray 1 is the position of point A, the position of the screen projection window corresponding to ray 2 is the position of point B, the position of the screen projection window corresponding to ray 3 is the position of point C, the position of the screen projection window corresponding to ray 4 is the position of point D, and the position of the screen projection window corresponding to ray 5 is the position of point E.

[0078] The target length is determined in the following way: for any one terminal device sending the screen projection request, the distance between the terminal device and the VR device is obtained according to the map position coordinates of the terminal device and the current position of the VR device, the distance is multiplied by a pre-set weight to obtain an intermediate length corresponding to the terminal device, and the average of the intermediate lengths of the terminal devices sending the screen projection request is determined as the target length.

[0079] Method two: if the number of terminal devices sending the screen projection request is not less than the specified number, it is determined that the display type is a multi-layer display type, and for any one ray, the positions of the multiple screen projection windows corresponding to the ray are determined according to the target length.

[0080] In one embodiment, the positions of the multiple projection windows corresponding to each ray can be determined in the following way:

[0081] For any given ray, based on the number of terminal devices and the number of each ray, the target total number of projection windows corresponding to that ray is obtained. After determining the position of a projection window corresponding to the ray by the target point whose distance from the endpoint of the ray is a target length, the current number of projection windows for that ray is updated. If the updated current number of projection windows for that ray is less than the target total number, the horizontal and vertical coordinates of a projection window are obtained by taking the target length whose distance from the endpoint of the ray is a multiple of the target length. The vertical coordinate of the previously determined projection window is then increased by a fixed multiple to obtain the position of the projection window. The process then returns to updating the current number of projection windows for that ray until the current number of projection windows for that ray is not less than the target total number, at which point the process ends. The target multiple is obtained based on the current number of projection windows for that ray.

[0082] In this embodiment, the target multiple is obtained by adding 1 to the projection window of the ray. However, this embodiment does not limit the target multiple; it can be set according to actual conditions. Furthermore, in this embodiment, the target total number of projection windows corresponding to any ray is obtained by dividing the number of terminal devices by the number of each ray. If the value obtained by dividing the number of terminal devices by the number of each ray is not an integer, the integer part of that value is added by 1 to determine the target total number of projection windows corresponding to any ray.

[0083] like Figure 9 As shown, Figure 9 The example used is that the total number of targets in the projection window corresponding to any ray is 2. From... Figure 9 As can be seen, any ray corresponds to two projection windows.

[0084] Step 205: For any terminal device, using the projection position coordinates of the terminal device and the position of the at least one projection window, determine the target projection window corresponding to the terminal device;

[0085] like Figure 10 The diagram shown illustrates the process of determining the target projection window for a terminal device, which may include the following steps:

[0086] Step 1001: For any terminal device, determine the distance between the terminal device and each ray based on the projection position coordinates of the terminal device;

[0087] In one embodiment, step 1001 can be implemented as: obtaining, for any one ray, a straight line equation corresponding to the ray according to a position of a projection window corresponding to the ray; and obtaining, for any one terminal device, distances between the terminal device and each ray according to projection position coordinates of the terminal device.

[0088] Step 1002: determining a ray with the shortest distance as a target ray corresponding to the terminal device;

[0089] As shown in FIG. 1, it can be seen from the figure that the terminal device a has the shortest distance to the ray 1, and therefore the target ray corresponding to the terminal device a is the ray 1. Figure 11

[0090] Step 1003: obtaining a target terminal device corresponding to each ray according to the target ray corresponding to each terminal device;

[0091] In one embodiment, step 1003 can be implemented as: performing iteration on the terminal devices for any one ray; for any one iterated terminal device, if the ray is the target ray of the terminal device, determining the terminal device as the target terminal device corresponding to the ray, and then returning to the step of iterating on the terminal devices until there is no terminal device that has not been iterated, and then ending.

[0092] Step 1004: for any one ray, if the number of target terminal devices corresponding to the ray is one and the number of projection windows corresponding to the ray is one, determining the projection window corresponding to the ray as a target projection window corresponding to the target terminal device;

[0093] Step 1005: for any one ray, if the number of target terminal devices corresponding to the ray is one and the number of projection windows corresponding to the ray is more than one, determining a projection window closest to the target terminal device from the projection windows corresponding to the ray as a target projection window corresponding to the target terminal device;

[0094] Step 1006: for any one ray, if the number of target terminal devices corresponding to the ray is more than one, iterating on the projection windows corresponding to the ray according to distances between the terminal devices and the VR device from small to large;

[0095] Step 1007: for any one iterated projection window, determining the projection window as a target projection window of a target terminal device closest to the projection window;

[0096] ​Step 1008: Determine whether all target terminal devices corresponding to the ray have determined the corresponding target projection window. If yes, end the process; otherwise, return to step 1006.

[0097] In one embodiment, step 205 can be specifically implemented as follows: for any terminal device, determine the distance between the terminal device and each ray based on the projection position coordinates of the terminal device; and determine the ray with the shortest distance as the target ray corresponding to the terminal device; based on the target rays corresponding to each terminal device, obtain the target terminal device corresponding to each ray; for any ray, the following three cases are included:

[0098] (1) If the number of target terminal devices corresponding to the ray is one and the number of projection windows corresponding to the ray is one, then the projection window corresponding to the ray is determined as the target projection window corresponding to the target terminal device.

[0099] like Figure 12A As shown, the screen projection between the terminal device and the VR device is a one-to-one projection. The size of the target projection window in this embodiment is preset, and the size of the target projection window can be set according to the actual situation. This embodiment does not limit the size of the target projection window.

[0100] (2) If the number of target terminal devices corresponding to the ray is one and the number of projection windows corresponding to the ray is multiple, then the projection window that is closest to the target terminal device among the projection windows corresponding to the ray is determined as the target projection window corresponding to the target terminal device.

[0101] like Figure 12B As shown in the figure, the target terminal device corresponding to ray 5 is terminal device m, and the projection windows corresponding to ray 5 are projection window P and projection window Q. It can be seen from the figure that projection window P is the projection window closest to terminal device m; therefore, projection window P is determined to be the target projection window corresponding to terminal device m.

[0102] (3) If there are multiple target terminal devices corresponding to the ray, then each projection window corresponding to the ray is traversed according to the distance from the VR device from smallest to largest. For any projection window that is traversed, the projection window is determined as the target projection window of the target terminal device that is closest to the projection window. Then, the step of traversing each projection window corresponding to the ray according to the distance from the VR device from smallest to largest is returned until the target projection window corresponding to the ray is determined for all target terminal devices.

[0103] like Figure 12CAs shown in the figure, the target terminal devices corresponding to ray 4 are terminal device a and terminal device b, and the projection windows corresponding to ray 4 are projection window R and projection window H, respectively. It can be seen from the figure that projection window R is the projection window closest to terminal device a, therefore projection window R is determined to be the target projection window corresponding to terminal device a. Similarly, projection window H is the projection window closest to terminal device b, therefore projection window H is determined to be the projection window closest to terminal device b.

[0104] Step 206: Display the projection data sent by the terminal device through the target projection window of the terminal device.

[0105] like Figure 13A As shown, this is a one-to-one screen mirroring between the terminal device and the VR device. In this scenario, only one target mirroring window is created to display the mirroring data sent by one terminal device. For example, when watching a movie, the movie resources on the terminal device can be mirrored to the VR device. The VR device can then achieve a cinema mode, providing a large-screen viewing experience with a cinematic atmosphere and a relaxed viewing experience. Figure 13B As shown, the screen projection between the terminal device and the VR device is many-to-one. Figure 13B Image a in the diagram is a single-layer display type. Image b is a multi-layer display type.

[0106] To further connect with the methods in this application, the screen projection display method of the VR device in this application will be introduced in conjunction with usage scenarios. For example... Figure 14 The diagram shown illustrates a usage scenario for screen projection display of a VR device in this application, which may include the following steps:

[0107] Step 1401: After receiving a screen mirroring request from any terminal device connected to the same local area network as the VR device, establish a connection with the terminal device;

[0108] Step 1402: When the screen projection data sent by the terminal device is received, the map position coordinates of the terminal device in the map coordinate system are converted into the screen projection position coordinates of the terminal device in the screen projection display coordinate system, wherein the map position coordinates are obtained based on SLAM spatial positioning technology;

[0109] Step 1403: Using the current position of the VR device as the origin, emit an initial ray directly in front of the VR device;

[0110] Step 1404: Using the initial ray as a reference, the spatial area corresponding to the left and right field of view of the initial ray as the first specified angle and the upper and lower field of view as the second specified angle is determined as the target display area;

[0111] Step 1405: emit a target number of rays in the target display region with the current position of the VR device as the origin, wherein the target number is based on the number of terminal devices that send the screen projection request, and the target number of emitted rays are in the same horizontal plane, and the included angle between any two adjacent rays is the same;

[0112] Step 1406: determine the display type and position of the screen projection window corresponding to the target number of rays according to the screen projection position coordinates of the terminal devices that send the screen projection request and the current position of the VR device;

[0113] Step 1407: for any one terminal device, determine the distance between the terminal device and each ray according to the screen projection position coordinates of the terminal device;

[0114] Step 1408: determine the shortest ray as the target ray corresponding to the terminal device;

[0115] Step 1409: obtain the target terminal device corresponding to each ray according to the target ray corresponding to each terminal device;

[0116] Step 1410: for any one ray, if the number of target terminal devices corresponding to the ray is one and the number of screen projection windows corresponding to the ray is one, determine the screen projection window corresponding to the ray as the target screen projection window corresponding to the target terminal device;

[0117] Step 1411: if the number of target terminal devices corresponding to the ray is one and the number of screen projection windows corresponding to the ray is multiple, determine the screen projection window closest to the target terminal device among the screen projection windows corresponding to the ray as the target screen projection window corresponding to the target terminal device;

[0118] Step 1412: if the number of target terminal devices corresponding to the ray is multiple, traverse the screen projection windows corresponding to the ray in ascending order of distance between the VR device and the screen projection window, for any one traversed screen projection window, determine the screen projection window as the target screen projection window of the target terminal device closest to the screen projection window, and return to the step of traversing the screen projection windows corresponding to the ray in ascending order of distance between the VR device and the screen projection window until the target terminal devices corresponding to the ray all determine the corresponding target screen projection window;

[0119] Step 1413: display the screen projection data sent by the terminal device through the target screen projection window of the terminal device.

[0120] Based on the same inventive concept, the screen projection display method of the VR device as described above can also be implemented by a screen projection display apparatus of a VR device. The screen projection display apparatus of the VR device has similar effects to the aforementioned method, and will not be described here again.

[0121] Figure 15 A structural schematic diagram of a screen projection display apparatus of a VR device according to an embodiment of the present disclosure.

[0122] As shown in Figure 15 , the screen projection display apparatus 1500 of the VR device of the present disclosure can include a connection module 1510, a screen projection position coordinate determination module 1520, a target display area determination module 1530, a screen projection window determination module 1540, a target screen projection window determination module 1550, and a screen projection display module 1560.

[0123] The connection module 1510 is configured to establish a connection with any one of terminal devices in the same local area network as the VR device after receiving a screen projection request sent by the terminal device.

[0124] The screen projection position coordinate determination module 1520 is configured to convert a map position coordinate of the terminal device in a map coordinate system into a screen projection position coordinate of the terminal device in a screen projection display coordinate system when receiving screen projection data sent by the terminal device, wherein the map position coordinate is obtained based on a SLAM space positioning technology.

[0125] The target display area determination module 1530 is configured to construct a target display area for displaying screen projection data according to a current position of the VR device.

[0126] The screen projection window determination module 1540 is configured to determine a position of at least one screen projection window in the target display area based on a number of terminal devices sending screen projection requests, wherein the number of screen projection windows is obtained based on the number of terminal devices.

[0127] The target screen projection window determination module 1550 is configured to determine a target screen projection window corresponding to any one of the terminal devices by using the screen projection position coordinate of the terminal device and the position of the at least one screen projection window.

[0128] The screen projection display module 1560 is configured to display screen projection data sent by the terminal device through the target screen projection window of the terminal device.

[0129] In one embodiment, the target display area determination module 1530 is specifically configured to:

[0130] emit an initial ray from the current position of the VR device as the origin to the front of the VR device.

[0131] The initial ray is taken as a reference, and a space region corresponding to a first specified angle of a left and right field of view of the initial ray and a second specified angle of an up and down field of view of the initial ray is determined as the target display region.

[0132] In one embodiment, the screen projection window determination module 1540 is specifically configured to:

[0133] The target number of rays is emitted in the target display region with the current position of the VR device as an origin, wherein the target number is obtained based on the number of terminal devices that send screen projection requests, and the target number of emitted rays are in the same horizontal plane, and the included angle between any two adjacent rays is the same;

[0134] According to the screen projection position coordinates of the terminal devices that send screen projection requests and the current position of the VR device, the display type and position of the screen projection window corresponding to the target number of rays are determined.

[0135] In one embodiment, the display type of the screen projection window includes a single-layer display type and a multi-layer display type, and the number of screen projection windows corresponding to any one ray is one in the single-layer display type, and the number of screen projection windows corresponding to any one ray is greater than one in the multi-layer display type; the screen projection window determination module 1540 is specifically configured to:

[0136] If the number of terminal devices that send screen projection requests is less than a specified number, the display type is determined to be the single-layer display type, and for any one ray, the position of the screen projection window corresponding to the ray is obtained according to a target length, wherein the target length is obtained based on the screen projection position coordinates of the terminal devices that send screen projection requests and the current position of the VR device; or,

[0137] If the number of terminal devices that send screen projection requests is not less than the specified number, the display type is determined to be the multi-layer display type, and for any one ray, the positions of the multiple screen projection windows corresponding to the ray are determined according to the target length.

[0138] In one embodiment, the target screen projection window determination module 1550 is specifically configured to:

[0139] For any one terminal device, the distance between the terminal device and each ray is determined according to the screen projection position coordinates of the terminal device; and,

[0140] The ray with the shortest distance is determined as the target ray corresponding to the terminal device;

[0141] According to the target ray corresponding to each terminal device, a target terminal device corresponding to each ray is obtained;

[0142] For any one ray, if the number of target terminal devices corresponding to the ray is one, and the number of projection windows corresponding to the ray is one, the projection window corresponding to the ray is determined as the target projection window corresponding to the target terminal device; or,

[0143] If the number of target terminal devices corresponding to the ray is one, and the number of projection windows corresponding to the ray is more than one, the projection window closest to the target terminal device among the projection windows corresponding to the ray is determined as the target projection window corresponding to the target terminal device; or,

[0144] If the number of target terminal devices corresponding to the ray is more than one, the projection windows corresponding to the ray are traversed according to the distance between the VR device from small to large, for any one traversed projection window, the projection window is determined as the target projection window of the target terminal device closest to the projection window, and then the step of traversing the projection windows corresponding to the ray according to the distance between the VR device from small to large is returned until the target terminal devices corresponding to the ray are determined to have corresponding target projection windows.

[0145] In one embodiment, the projection position coordinate determination module 1520 is specifically configured to:

[0146] If the coordinate system parameters of the map coordinate system and the coordinate system parameters of the projection display coordinate system are the same, the map position coordinates are reduced by a specified multiple to obtain the projection position coordinates of the terminal device, wherein the coordinate system parameters include the origin position, the position and direction of the horizontal coordinate axis, the position and direction of the vertical coordinate axis, and the position and direction of the vertical coordinate axis; or,

[0147] If the coordinate system parameters of the map coordinate system and the coordinate system parameters of the projection display coordinate system are not the same, the map position coordinates are converted by using a rotation translation matrix between the map coordinate system and the projection display coordinate system to obtain the projection position coordinates of the terminal device.

[0148] After introducing the projection display method and device of a VR device according to an exemplary embodiment of the present application, next, a VR device according to another exemplary embodiment of the present application is introduced.

[0149] Those skilled in the art can understand that each aspect of the present application can be implemented as a system, a method or a program product. Therefore, each aspect of the present application can be specifically implemented as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0150] In some possible embodiments, the VR device according to the present application can at least include at least one processor and at least one computer storage medium. The computer storage medium stores program codes which, when executed by the processor, cause the processor to perform the steps in the screen projection display method of the VR device according to various exemplary embodiments of the present application described above in the specification. For example, the processor can perform steps 201-206 as shown in Figure 2 .

[0151] The VR device 1600 according to this embodiment of the present application will be described below with reference to Figure 16 . Figure 16 The VR device 1600 shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0152] As shown in Figure 16 , the VR device 1600 is in the form of a general VR device. The components of the VR device 1600 can include but are not limited to the at least one processor 1601 described above, the at least one computer storage medium 1602 described above, and a bus 1603 connecting different system components including the computer storage medium 1602 and the processor 1601.

[0153] The bus 1603 represents one or more of several types of bus structures, including a computer storage medium bus or a computer storage medium controller, a peripheral bus, a processor bus, or a local bus using any of a variety of bus structures.

[0154] The computer storage medium 1602 can include readable media in the form of volatile computer storage medium, such as random access computer storage medium (RAM) 1621 and / or cache storage medium 1622, and can further include read-only computer storage medium (ROM) 1623.

[0155] The computer storage medium 1602 can further include programs / utilities 1625 with a set of (at least one) program modules 1624, such as an operating system, one or more application programs, other program modules, and program data, each of which or some combination of which can include implementation of a network environment.

[0156] The VR device 1600 can also communicate with one or more external devices 1604 such as a keyboard, a pointing device, etc.; and / or one or more devices that enable a user to interact with the VR device 1600 and / or one or more devices that enable the VR device 1600 to communicate with one or more other VR devices (such as a router, a modem, etc.). Such communication can occur via Input / Output (I / O) interface 1605. Still yet, the VR device 1600 can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the Internet) through a network adapter 1606. As depicted, the network adapter 1606 communicates with the other components of the VR device 1600 via bus 1603. It should be appreciated that although not shown, other hardware and / or software modules could be used in connection with the VR device 1600. Such modules include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0157] In some possible implementation, each aspect of the method for screen projection of a VR device provided by the present application can also be implemented as a program product in the form of a computer program or a plurality of programs. When the program product is run on a computer device, the program code is used to make the computer device execute the steps of the method for screen projection of a VR device according to various exemplary embodiments of the present application described above in the specification.

[0158] Obviously, various modifications and changes can be made to the present application by those skilled in the art without departing from the spirit and scope of the present application. Thus, it is intended that the present application encompass such modifications and changes as fall within the scope of the appended claims and their equivalents.

Claims

1. A method for casting display of a VR device, characterized in that, The method comprises: After receiving a screen projection request sent by any one of terminal devices connected to the same local area network as the VR device, a connection is established with the terminal device; After receiving screen projection data sent by the terminal device, a map position coordinate of the terminal device in a map coordinate system is converted into a screen projection position coordinate of the terminal device in a screen projection coordinate system, wherein the map position coordinate is obtained based on a SLAM space positioning technology; According to the current position of the VR device, a target display area for displaying the screen projection data is constructed; Taking the current position of the VR device as the origin, a target number of rays are emitted in the target display area, wherein the target number is obtained based on the number of terminal devices sending screen projection requests, and the target number of emitted rays are in the same horizontal plane, and the included angle between any two adjacent rays is the same; If the number of terminal devices sending screen projection requests is less than a specified number, it is determined that the display type of the screen projection window is a single-layer display type, and for any one ray, the position of the screen projection window corresponding to the ray is obtained according to a target length, wherein the target length is obtained based on the screen projection position coordinate of the terminal device sending the screen projection request and the current position of the VR device, and the number of screen projection windows corresponding to any one ray in the single-layer display type is one; or, If the number of terminal devices sending screen projection requests is not less than the specified number, it is determined that the display type of the screen projection window is a multi-layer display type, and for any one ray, the positions of a plurality of screen projection windows corresponding to the ray are determined according to a target length, wherein the target length is obtained based on the screen projection position coordinate of the terminal device sending the screen projection request and the current position of the VR device, and the number of screen projection windows corresponding to any one ray in the multi-layer display type is greater than one; For any one terminal device, the target screen projection window corresponding to the terminal device is determined by using the screen projection position coordinate of the terminal device and the positions of the at least one screen projection window; and The screen projection data sent by the terminal device is displayed through the target screen projection window of the terminal device.

2. The method of claim 1, wherein, The construction of the target display area for displaying the screen projection data according to the current position of the VR device comprises: Taking the current position of the VR device as the origin, an initial ray is emitted to the front of the VR device; Taking the initial ray as the reference, and determining a space area corresponding to the initial ray with a left and right field of view angle of a first specified angle and an up and down field of view angle of a second specified angle as the target display area.

3. The method of claim 1, wherein, The determination of the target screen projection window corresponding to the terminal device by using the screen projection position coordinate of the terminal device and the positions of the at least one screen projection window comprises: For any one terminal device, the distance between the terminal device and each ray is determined according to the screen projection position coordinate of the terminal device; and The ray with the shortest distance is determined as the target ray corresponding to the terminal device; The target terminal device corresponding to each ray is obtained according to the target ray corresponding to each terminal device; For any one ray, if the number of target terminal devices corresponding to the ray is one, and the number of projection windows corresponding to the ray is one, the projection window corresponding to the ray is determined as the target projection window corresponding to the target terminal device; or, If the number of target terminal devices corresponding to the ray is one, and the number of projection windows corresponding to the ray is more than one, the projection window closest to the target terminal device among the projection windows corresponding to the ray is determined as the target projection window corresponding to the target terminal device; or, If the number of target terminal devices corresponding to the ray is more than one, the projection windows corresponding to the ray are traversed in ascending order of distance between the VR device, for any one traversed projection window, the projection window is determined as the target projection window of the target terminal device closest to the projection window, and then the step of traversing the projection windows corresponding to the ray in ascending order of distance between the VR device is returned until the target terminal devices corresponding to the ray are all determined to have corresponding target projection windows.

4. The method of claim 1, wherein, The map position coordinates of the terminal device in the map coordinate system are converted into the projection position coordinates of the terminal device in the projection display coordinate system, including: If the coordinate system parameters of the map coordinate system and the coordinate system parameters of the projection display coordinate system are the same, the map position coordinates are reduced by a specified multiple to obtain the projection position coordinates of the terminal device, wherein the coordinate system parameters include the origin position, the position and direction of the horizontal coordinate axis, the position and direction of the vertical coordinate axis, and the position and direction of the vertical coordinate axis; or, If the coordinate system parameters of the map coordinate system and the coordinate system parameters of the projection display coordinate system are not the same, the map position coordinates are converted using a pre-set rotation and translation matrix between the map coordinate system and the projection display coordinate system to obtain the projection position coordinates of the terminal device.

5. A VR device, comprising: The processor and the memory are connected through a bus; The memory stores a computer program, and the processor is configured to execute the following operations based on the computer program: After receiving a projection request sent by any one terminal device connected to the VR device in the same local area network, a connection is established with the terminal device; When receiving the projection data sent by the terminal device, the map position coordinates of the terminal device in the map coordinate system are converted into the projection position coordinates of the terminal device in the projection display coordinate system, wherein the map position coordinates are obtained based on SLAM space positioning technology; According to the current position of the VR device, a target display area for displaying the projection data is constructed; Taking the current position of the VR device as the origin, a target number of rays are emitted in the target display area, wherein the target number is obtained based on the number of terminal devices sending the projection request, and the target number of emitted rays are in the same horizontal plane, and the included angle between any two adjacent rays is the same. If the number of terminal devices sending the screen projection request is less than the specified number, it is determined that the display type of the screen projection window is a single-layer display type, and for any one ray, the position of the screen projection window corresponding to the ray is obtained according to a target length, wherein the target length is obtained based on the screen projection position coordinates of the terminal devices sending the screen projection request and the current position of the VR device, and the single-layer display type is that the number of screen projection windows corresponding to any one ray is one; or, If the number of terminal devices sending the screen projection request is not less than the specified number, it is determined that the display type of the screen projection window is a multi-layer display type, and for any one ray, the positions of the multiple screen projection windows corresponding to the ray are determined according to a target length, wherein the target length is obtained based on the screen projection position coordinates of the terminal devices sending the screen projection request and the current position of the VR device, and the multi-layer display type is that the number of screen projection windows corresponding to any one ray is greater than one; For any one terminal device, the target screen projection window corresponding to the terminal device is determined by using the screen projection position coordinates of the terminal device and the positions of the at least one screen projection window; and The screen projection data sent by the terminal device is displayed through the target screen projection window of the terminal device.

6. The VR device of claim 5, wherein, The processor performs the construction of the target display area for displaying the screen projection data according to the current position of the VR device, and is specifically configured to: emit an initial ray to the front of the VR device with the current position of the VR device as the origin; determine a space area corresponding to the initial ray as the target display area, with the left and right field angles of the initial ray being a first specified angle and the upper and lower field angles being a second specified angle.

Images