Screen projection display method of VR equipment and VR equipment

Abstract

The invention provides a screen projection display method of VR equipment and the VR equipment, which are used for performing screen projection through the VR equipment and improving user experience. Comprising the following steps: after receiving a screen projection request sent by any terminal device connected with the VR device in the same local area network, establishing a connection with the terminal device; when screen projection data sent by the terminal equipment is received, map position coordinates of the terminal equipment in the map coordinate system are converted into screen projection position coordinates of the terminal equipment in the screen projection display coordinate system; according to the current position of the VR equipment, constructing a target display area for displaying the screen projection data; determining the position of at least one screen projection window in the target display area based on the number of the terminal devices sending the screen projection requests; for any terminal device, determining a target screen projection window corresponding to the terminal device by using the screen projection position coordinate of the terminal device and the position of the at least one screen projection window; and displaying the screen projection data sent by the terminal equipment through the target screen projection window of the terminal equipment.

Description

Background Art

[0002] Screen projection technology refers to first connecting the projection source device to the wireless network in the use environment (home, office, hotel, etc.), and then the projection source device sends the projection data (for example, the screen image of the projection source device) to the projection target device. The projection target device outputs and plays the projection data locally, thereby realizing the function of projecting the screen content of the projection source device to the projection target device for display.

[0003] At present, in many scenarios, it is necessary to display the relevant projection content through a larger display screen (for example, a TV, etc.). For example, in education and training scenarios, the interaction between teachers and students is indispensable. Teachers can understand students' learning dynamics and homework completion through students' projection content. However, the display position of this projection method is fixed, and the projection display method is 2D (two dimensional), resulting in a poor user experience. Summary of the invention

[0004] The present application provides a screen projection display method and a VR device for performing screen projection display through a VR device. The target screen projection window of the terminal device can be determined according to the position of the terminal device performing the screen projection, and the screen projection display method is three-dimensional, thereby improving the user experience.

[0005] In a first aspect, an embodiment of the present application provides a screen projection display method for a VR device, the method comprising:

[0006] After receiving a screen projection request sent by any terminal device connected to the same local area network as the VR device, a connection is established with the terminal device;

[0007] When receiving the projection data sent by the terminal device, the map position coordinates of the terminal device in the map coordinate system are converted into the projection position coordinates of the terminal device in the projection display coordinate system, wherein the map position coordinates are obtained based on the SLAM spatial positioning technology;

[0008] Constructing a target display area for displaying projection data according to the current position of the VR device;

[0009] Based on the number of terminal devices that send screen projection requests, determine the position of at least one screen projection window in the target display area, wherein the number of the screen projection windows is obtained based on the number of the terminal devices;

[0010] For any terminal device, using the projection position coordinates of the terminal device and the position of the at least one projection window, determine a target projection window corresponding to the terminal device; and,

[0011] The projection data sent by the terminal device is displayed through the target projection window of the terminal device.

[0012] A second aspect of the present application provides a VR device, including a processor and a memory, wherein the processor and the memory are connected via a bus;

[0013] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:

[0014] After receiving a screen projection request sent by any terminal device connected to the same local area network as the VR device, a connection is established with the terminal device;

[0015] When receiving the projection data sent by the terminal device, the map position coordinates of the terminal device in the map coordinate system are converted into the projection position coordinates of the terminal device in the projection display coordinate system, wherein the map position coordinates are obtained based on the SLAM spatial positioning technology;

[0016] Constructing a target display area for displaying projection data according to the current position of the VR device;

[0017] Based on the number of terminal devices that send screen projection requests, determine the position of at least one screen projection window in the target display area, wherein the number of the screen projection windows is obtained based on the number of the terminal devices;

[0018] For any terminal device, using the projection position coordinates of the terminal device and the position of the at least one projection window, determine a target projection window corresponding to the terminal device; and,

[0019] The projection data sent by the terminal device is displayed through the target projection window of the terminal device.

[0020] According to a third aspect provided by an embodiment of the present invention, a computer storage medium is provided, wherein the computer storage medium stores a computer program, and the computer program is used to execute the method as described in the first aspect.

[0021] In the above-mentioned embodiment of the present application, after the VR device establishes a connection with any terminal device that sends a screen projection request and is connected to the same local area network, and when receiving the screen projection data sent by the terminal device, the map position coordinates of the terminal device in the map coordinate system are converted into the screen projection position coordinates of the terminal device in the screen projection display coordinate system, and according to the current position of the VR device, a target display area for displaying the screen projection data is constructed, and then based on the number of terminal devices that send screen projection requests, the position of at least one screen projection window is determined in the target display area, and then for any terminal device, the screen projection position coordinates of the terminal device and the position of the at least one screen projection window are used to determine the target screen projection window corresponding to the terminal device, and the screen projection data sent by the terminal device is displayed through the target screen projection window of the terminal device. Therefore, in the embodiment of the present application, screen projection display is realized through the VR device, and the target screen projection window of the terminal device can be determined according to the position of the terminal device performing the screen projection, and the screen projection display mode is three-dimensional display, so the user experience is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1 A schematic diagram of a scene illustrating a screen projection display method of a VR device provided in an embodiment of the present application is exemplified;

[0024] Figure 2 One of the flowcharts of the screen projection display method of the VR device provided in the embodiment of the present application is exemplarily shown;

[0025] Figure 3 The following is an exemplary diagram of the screen projection display logic of the VR device provided in the embodiment of the present application;

[0026] Figure 4 One of the schematic diagrams of the map coordinate system and the projection coordinate system provided in the embodiment of the present application is exemplarily shown;

[0027] Figure 5 The second schematic diagram of the map coordinate system and the projection coordinate system provided in the embodiment of the present application is exemplarily shown;

[0028] Figure 6 The flowchart of determining each screen projection window provided in the embodiment of the present application is exemplarily shown;

[0029] Figure 7The schematic diagram of rays in the target display area provided by the embodiment of the present application is exemplarily shown;

[0030] Figure 8 One of the schematic diagrams of the screen projection window provided in the embodiment of the present application is exemplarily shown;

[0031] Fig. 9 The second schematic diagram of the screen projection window provided in the embodiment of the present application is exemplarily shown;

[0032] Fig.10 The following is a schematic diagram of a process for determining a target screen projection window of a terminal device provided in an embodiment of the present application;

[0033] Fig.11 A schematic diagram of a target ray provided in an embodiment of the present application is exemplarily shown;

[0034] Figures 12A-12C A schematic diagram of a target screen projection window provided in an embodiment of the present application is exemplarily shown;

[0035] Figures 13A-13B The following is a schematic diagram of a screen projection scenario provided by an embodiment of the present application;

[0036] Fig.14 The second flowchart of the screen projection display method of the VR device provided in the embodiment of the present application is exemplarily shown;

[0037] Fig.15 A schematic diagram of a screen projection display device of a VR device provided in an embodiment of the present application is exemplarily shown;

[0038] Fig.16 The hardware structure diagram of the VR device provided in the embodiment of the present application is exemplified. DETAILED DESCRIPTION

[0039] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0040] Based on the exemplary embodiments described in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the claims attached to this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of the disclosure can also constitute a complete implementation method separately.

[0041] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0042] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a series of components is not necessarily limited to those components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0043] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0044] The idea of ​​the embodiments of the present application is summarized below.

[0045] At present, the display position of the projection mode is fixed and the projection display mode is 2D (two-dimensional) display, resulting in a poor user experience. Therefore, the embodiment of the present application provides a projection display method for a VR device. After the VR device establishes a connection with any terminal device connected to the same local area network that sends a projection request, and when the projection data sent by the terminal device is received, the map position coordinates of the terminal device in the map coordinate system are converted into the projection position coordinates of the terminal device in the projection display coordinate system, and according to the current position of the VR device, a target display area for displaying the projection data is constructed, and then based on the number of terminal devices that send projection requests, the position of at least one projection window is determined in the target display area, and then for any terminal device, the projection position coordinates of the terminal device and the position of the at least one projection window are used to determine the target projection window corresponding to the terminal device, and the projection data sent by the terminal device is displayed through the target projection window of the terminal device. Therefore, the embodiment of the present application realizes screen projection display through VR equipment, and can determine the target screen projection window of the terminal equipment according to the position of the terminal equipment performing screen projection, and the screen projection display mode is three-dimensional display, so the user experience is improved. Below, the screen projection display method of the VR equipment in the present application is described in detail with reference to the accompanying drawings.

[0046] like Figure 1 , which is a schematic diagram of an application scenario of the screen projection display method of a VR device in an embodiment of the present application. The application scenario includes a VR device 110 and a terminal device 120.

[0047] In a possible application scenario, after receiving a screen projection request sent by any terminal device 120 connected to the same local area network as the VR device 110 , the VR device 110 establishes a connection with the terminal device 120 . When the VR device 110 receives the projection data sent by the terminal device 120, the map position coordinates of the terminal device 120 in the map coordinate system are converted into the projection position coordinates of the terminal device 120 in the projection display coordinate system, wherein the map position coordinates are obtained based on the SLAM spatial positioning technology; then the VR device 110 constructs a target display area for displaying the projection data according to the current position of the VR device 110; and based on the number of terminal devices that send projection requests, determines the position of at least one projection window in the target display area, wherein the number of projection windows is obtained based on the number of terminal devices; then for any terminal device 120, the projection position coordinates of the terminal device 120 and the position of the at least one projection window are used to determine the target projection window corresponding to the terminal device 120; and the projection data sent by the terminal device 120 is displayed through the target projection window of the terminal device 120.

[0048] in, Figure 1 The VR device 110 and the terminal device 120 can exchange information through a communication network, wherein the communication method adopted by the communication network can be divided into a wireless communication method or a wired communication method.

[0049] Exemplarily, the VR device 110 can access the network through cellular mobile communication technology and communicate with the terminal device 120, wherein the cellular mobile communication technology, for example, includes the fifth generation mobile communication (5th Generation Mobile Networks, 5G) technology.

[0050] Optionally, the VR device 110 can access the network and communicate with the terminal device 120 via a short-range wireless communication method, wherein the short-range wireless communication method, for example, includes Wireless Fidelity (Wi-Fi) technology.

[0051] Among them, the description in this application only details a single VR device 110 and three terminal devices 120, but those skilled in the art should understand that the VR device 110 and terminal device 120 shown are intended to represent the operations of the VR device 110 and terminal device 120 involved in the technical solution of this application. It does not imply any restrictions on the number, type or location of the VR device 110 and the terminal device 120. It should be noted that if additional modules are added to the illustrated environment or individual modules are removed from it, the underlying concepts of the example embodiments of this application will not be changed.

[0052] It should be noted that the screen projection display method of the VR device proposed in this application is not only applicable to Figure 1 The application scenario shown is also applicable to any screen projection display device with VR equipment.

[0053] The following describes the screen projection display method of the VR device in an exemplary embodiment of the present application in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the methods and principles of the present application, and the implementation methods of the present application are not limited in this regard.

[0054] like Figure 2 FIG. 1 is a flow chart of a method for projecting and displaying a VR device, which may include the following steps:

[0055] Step 201: after receiving a screen projection request sent by any terminal device connected to the same local area network as the VR device, establishing a connection with the terminal device;

[0056] The VR device in the embodiment of the present application runs Http Server and WebSocket Server. The Http Server in the embodiment of the present application is used to provide access to web page files, so that the browser can access the resources in the web page and write the screen recording logic in the web page. And the WebSocket Server in the embodiment of the present application is used to receive the connection established by the browser, and to receive the projection data (screen recording data). The web page runs in the browser, which is used to project the screen data and send it to the VR device. StreamBuffer temporarily stores the projection data. When the player in the VR device reads the projection data, StreamBuffer sends the projection data to the player in the VR device for playback.

[0057] like Figure 3As shown, it is a schematic diagram of the screen projection display logic of the VR device. The user accesses the HTTP Server through the browser in the terminal device to open the web page, so that when operating the screen projection, the Websocket connection logic in the web page is executed to connect to the VR device, and then the projection data is sent to the WebSocket Server on the VR device side, so that the VR device receives the projection data and temporarily stores the projection data in the StreamBuffer. When the VR device determines the target projection window corresponding to the terminal device, the projection data is obtained from the StreamBuffer and played through the player of the VR device.

[0058] Step 202: when receiving the projection data sent by the terminal device, converting the map position coordinates of the terminal device in the map coordinate system into the projection position coordinates of the terminal device in the projection display coordinate system, wherein the map position coordinates are obtained based on the SLAM spatial positioning technology;

[0059] In one embodiment, the map location coordinates of the terminal device can be converted into the projection location coordinates in the following two ways:

[0060] Method 1: If the coordinate system parameters of the map coordinate system are the same as the coordinate system parameters of the projection display coordinate system, the map position coordinates are reduced by a specified multiple to obtain the projection position coordinates of the terminal device, wherein the coordinate system parameters include the origin position, the position and direction of the horizontal axis, the position and direction of the vertical axis, and the position and direction of the vertical axis.

[0061] In the embodiment of the present application, both the map coordinate system and the projection coordinate system are spatial rectangular coordinate systems established with the center point of the VR device as the origin. And the coordinate system parameters of the map coordinate system and the projection coordinate system are pre-set, for example, Figure 4 As shown, the map coordinate system and the projection coordinate system both take the center point of the VR device as the origin, and the position and direction of each coordinate axis are the same.

[0062] It should be noted that the specified multiple in the embodiment of the present application can be set according to actual conditions, and the embodiment of the present application does not limit the specified multiple.

[0063] Method 2: If the coordinate system parameters of the map coordinate system are different from the coordinate system parameters of the projection display coordinate system, the map position coordinates will be converted using the pre-set rotation and translation matrix between the map coordinate system and the projection display coordinate system to obtain the projection position coordinates of the terminal device.

[0064] like Figure 5As shown, the directions of the x-axis and y-axis of the map coordinate system and the projection display coordinate system are not the same. It can be seen from the figure that the position of p(x, y, z) in the map coordinate system converted to the projection display coordinate system is (x1, y1, z1).

[0065] Step 203: constructing a target display area for displaying projection data according to the current position of the VR device;

[0066] In one embodiment, step 203 can be specifically implemented as follows: taking the current position of the VR device as the origin, emitting an initial ray directly in front of the VR device; taking the initial ray as a reference, and determining the spatial area corresponding to the left and right field of view angles of the initial ray as the first specified angle, and the upper and lower field of view angles as the second specified angle as the target display area.

[0067] It should be noted that: the range of the first specified angle in the embodiment of the present application may be (0°, 160°), and the range of the second specified angle may be (0°, 90°). However, the specific angle values ​​of the first specified angle and the second specified angle may be set according to actual conditions, and the embodiment of the present application does not limit the specific angle values ​​of the first specified angle and the second specified angle.

[0068] Step 204: determining a position of at least one screen projection window in the target display area based on the number of terminal devices that send screen projection requests, wherein the number of the screen projection windows is obtained based on the number of the terminal devices;

[0069] Next, the positions of each projection window in the target display area are described in detail. Figure 6 As shown, a flow chart for determining each screen projection window includes the following steps:

[0070] Step 601: Taking the current position of the VR device as the origin, emitting a target number of rays in the target display area, wherein the target number is obtained based on the number of terminal devices that send screen projection requests, and the emitted target number of rays are in the same horizontal plane, and the angles between any two adjacent rays are the same;

[0071] The number of targets in the embodiment of the present application is equal to the number of terminal devices that send screen projection requests. And the angle between any two adjacent rays in the embodiment of the present application is obtained by dividing the first specified angle by the number of targets. Figure 7 As shown, Figure 7 The number of targets is 5 and the angle between any two adjacent rays is θ, which is the rays emitted in the target display area.

[0072] Step 602: Determine the display type and position of the projection window corresponding to the target number of rays based on the projection position coordinates of the terminal device that sends the projection request and the current position of the VR device.

[0073] In the embodiment of the present application, the display types of the projection window include a single-layer display type and a multi-layer display type, and the single-layer display type means that the number of projection windows corresponding to any ray is one, and the multi-layer display type means that the number of projection windows corresponding to any ray is greater than one.

[0074] In one embodiment, the display type and position of the projection window corresponding to the target number of rays may be determined in the following two ways:

[0075] Method 1: If the number of terminal devices sending screen projection requests is less than the specified number, the display type is determined to be a single-layer display type, and for any ray, the position of the projection window corresponding to the ray is obtained according to the target length, wherein the target length is based on the projection position coordinates of the terminal device sending the screen projection request and the current position of the VR device.

[0076] In one embodiment, the position of the projection window corresponding to any ray can be obtained in the following manner: for any ray, the position of the projection window corresponding to the ray is determined by the position of the target point whose distance from the endpoint of the ray is the target length.

[0077] like Figure 8 As shown in the figure, the target length r is used as an example for explanation. The position of the projection window corresponding to ray 1 is the position of point A, the position of the projection window corresponding to ray 2 is the position corresponding to point B, the position of the projection window corresponding to ray 3 is the position of point C, the position of the projection window corresponding to ray 4 is the position of point D, and the position of the projection window corresponding to ray 5 is the position of point E.

[0078] The present application determines the target length in the following manner: for any terminal device that sends a screen projection request, the distance between the terminal device and the VR device is obtained based on the map location coordinates of the terminal device and the current location of the VR device, the distance is multiplied by a pre-set weight to obtain the intermediate length corresponding to the terminal device, and the average value of the intermediate lengths of each terminal device that has a screen projection request is determined as the target length.

[0079] Method 2: If the number of terminal devices sending screen projection requests is not less than the specified number, the display type is determined to be a multi-layer display type, and for any ray, the respective positions of the multiple screen projection windows corresponding to the ray are determined according to the target length.

[0080] In one embodiment, the positions of the multiple projection windows corresponding to each ray may be determined in the following manner:

[0081] For any ray, the target total number of projection windows corresponding to any ray is obtained according to the number of terminal devices and the number of each ray. After determining the position of a projection window corresponding to the ray by taking the position of the target point whose distance from the endpoint of the ray is the target length, the current number of projection windows of the ray is updated. If the updated current number of projection windows of the ray is less than the target total number, the horizontal position coordinate and the vertical position coordinate of a projection window are obtained by taking the target length whose distance from the endpoint of the ray is a multiple of the target, and the vertical position coordinate of the last determined projection window is enlarged by a fixed multiple. After obtaining the position of the projection window, return to the step of updating the current number of projection windows of the ray until the current number of projection windows of the ray is not less than the target total number, and then end, wherein the target multiple is obtained based on the current number of projection windows of the ray.

[0082] In the embodiment of the present application, the projection window of the ray is increased by 1 to obtain the target multiple, but the embodiment of the present application does not limit the target multiple here, and the target multiple in the embodiment of the present application can be set according to the actual situation. And in the embodiment of the present application, the number of terminal devices is divided by the number of each ray to obtain the target total number of projection windows corresponding to any ray. If the value obtained by dividing the number of terminal devices by the number of each ray is not an integer, the integer part of the value plus 1 is determined as the target total number of projection windows corresponding to any ray.

[0083] like Fig. 9 As shown, Fig. 9 In the example, the total number of screen projection windows corresponding to any ray is 2. Fig. 9 It can be seen that any ray corresponds to two projection windows.

[0084] Step 205: for any terminal device, using the projection position coordinates of the terminal device and the position of the at least one projection window, determine a target projection window corresponding to the terminal device;

[0085] like Fig.10 As shown, a flow chart for determining a target screen projection window corresponding to a terminal device may include the following steps:

[0086] Step 1001: for any terminal device, determine the distance between the terminal device and each ray according to the projection position coordinates of the terminal device;

[0087] In one embodiment, step 1001 can be specifically implemented as follows: for any ray, the equation of the straight line corresponding to the ray is obtained according to the position of the projection window corresponding to the ray; for any terminal device, the distance between the terminal device and each ray is obtained according to the projection position coordinates of the terminal device.

[0088] Step 1002: Determine the ray with the shortest distance as the target ray corresponding to the terminal device;

[0089] like Fig.11 As shown in the figure, it can be seen that the distance between terminal device a and ray 1 is the shortest, so the target ray corresponding to terminal device a is ray 1.

[0090] Step 1003: according to the target rays corresponding to the terminal devices, obtaining the target terminal devices corresponding to the rays;

[0091] In one embodiment, step 1003 can be specifically implemented as follows: for any ray, traversal of the terminal devices is performed; for any traversed terminal device, if the ray is the target ray of the terminal device, the terminal device is determined as the target terminal device corresponding to the ray, and then the step of traversing the terminal devices is returned to, and the process ends until there are no terminal devices that have not been traversed.

[0092] Step 1004: for any ray, if the number of target terminal devices corresponding to the ray is one, and the number of screen projection windows corresponding to the ray is one, the screen projection window corresponding to the ray is determined as the target screen projection window corresponding to the target terminal device;

[0093] Step 1005: For any ray, if the number of target terminal devices corresponding to the ray is one, and the number of screen projection windows corresponding to the ray is multiple, the screen projection window closest to the target terminal device among the screen projection windows corresponding to the ray is determined as the target screen projection window corresponding to the target terminal device;

[0094] Step 1006: For any ray, if the number of target terminal devices corresponding to the ray is multiple, traverse the projection windows corresponding to the ray according to the distance from the VR device from small to large;

[0095] Step 1007: for any traversed projection window, determine the projection window as the target projection window of the target terminal device that is closest to the projection window;

[0096] Step 1008: Determine whether the target terminal devices corresponding to the ray have all determined the corresponding target projection windows. If so, end; if not, return to execute step 1006.

[0097] In one embodiment, step 205 can be specifically implemented as follows: for any terminal device, according to the projection position coordinates of the terminal device, determine the distance between the terminal device and each ray respectively; and determine the ray with the shortest distance as the target ray corresponding to the terminal device; according to the target ray corresponding to each terminal device, obtain the target terminal device corresponding to each ray; for any ray, including the following three cases:

[0098] (1) If the number of target terminal devices corresponding to the ray is one, and the number of screen projection windows corresponding to the ray is one, the screen projection window corresponding to the ray is determined as the target screen projection window corresponding to the target terminal device.

[0099] like Fig. 12A As shown, the screen projection between the terminal device and the VR device is one-to-one. The size of the target screen projection window in the embodiment of the present application is pre-set, and the size of the target screen projection window can be set according to actual conditions. The embodiment of the present application does not limit the size of the target screen projection window.

[0100] (2) If the number of target terminal devices corresponding to the ray is one, and the number of screen projection windows corresponding to the ray is multiple, then the screen projection window that is closest to the target terminal device among the screen projection windows corresponding to the ray is determined as the target screen projection window corresponding to the target terminal device.

[0101] like Fig. 12B As shown, the target terminal device corresponding to ray 5 is terminal device m, and the projection windows corresponding to ray 5 are projection window P and projection window Q. As can be seen from the figure, projection window P is the projection window closest to terminal device m, so projection window P is determined to be the target projection window corresponding to terminal device m.

[0102] (3) If there are multiple target terminal devices corresponding to the ray, the projection windows corresponding to the ray are traversed according to the distance from the VR device from small to large. For any traversed projection window, the projection window is determined as the target projection window of the target terminal device closest to the projection window, and then the process returns to the step of traversing the projection windows corresponding to the ray according to the distance from the VR device from small to large, until the corresponding target projection windows are determined for all the target terminal devices corresponding to the ray.

[0103] like Fig. 12CAs shown, the target terminal devices corresponding to ray 4 are terminal device a and terminal device b, and the projection windows corresponding to ray 4 are projection window R and projection window H. As can be seen from the figure, projection window R is the projection window closest to terminal device a, so projection window R is determined to be the target projection window corresponding to terminal device a. Projection window H is the projection window closest to terminal device b, so projection window H is determined to be the projection window closest to terminal device b.

[0104] Step 206: Display the projection data sent by the terminal device through the target projection window of the terminal device.

[0105] like Fig.13A As shown in the figure, the screen is projected one-to-one between the terminal device and the VR device. In this scenario, only one target screen projection window is created to display the projection data sent by a terminal device. For example, you can watch a movie by projecting the movie resources on the terminal device to the VR device. The VR device can realize the theater mode, with a large screen and a theater atmosphere, and an easy experience. Fig. 13B As shown, the screen projection between the terminal device and the VR device is many-to-one. Fig. 13B The a image in the figure is a single-layer display type, and the b image is a multi-layer display type.

[0106] In order to further connect the method in this application, the screen projection display method of the VR device in this application is introduced in combination with the usage scenario. Fig.14 As shown, it is a flow chart of the use scenario of screen projection display of the VR device in this application, which may include the following steps:

[0107] Step 1401: after receiving a screen projection request sent by any terminal device connected to the same local area network as the VR device, establishing a connection with the terminal device;

[0108] Step 1402: when receiving the projection data sent by the terminal device, converting the map position coordinates of the terminal device in the map coordinate system into the projection position coordinates of the terminal device in the projection display coordinate system, wherein the map position coordinates are obtained based on the SLAM spatial positioning technology;

[0109] Step 1403: Taking the current position of the VR device as the origin, an initial ray is emitted toward the front of the VR device;

[0110] Step 1404: taking the initial ray as a reference, and determining a spatial region corresponding to a left-right viewing angle of the initial ray being a first specified angle and a top-bottom viewing angle being a second specified angle as the target display region;

[0111] Step 1405: Taking the current position of the VR device as the origin, emitting a target number of rays in the target display area, wherein the target number is obtained based on the number of terminal devices that send screen projection requests, and the emitted target number of rays are in the same horizontal plane, and the angles between any two adjacent rays are the same;

[0112] Step 1406: Determine the display type and position of the screen projection window corresponding to the target number of rays according to the screen projection position coordinates of the terminal device that sends the screen projection request and the current position of the VR device;

[0113] Step 1407: for any terminal device, determine the distance between the terminal device and each ray according to the projection position coordinates of the terminal device;

[0114] Step 1408: Determine the ray with the shortest distance as the target ray corresponding to the terminal device;

[0115] Step 1409: according to the target rays corresponding to the terminal devices, obtaining the target terminal devices corresponding to the rays;

[0116] Step 1410: For any ray, if the number of target terminal devices corresponding to the ray is one, and the number of screen projection windows corresponding to the ray is one, the screen projection window corresponding to the ray is determined as the target screen projection window corresponding to the target terminal device;

[0117] Step 1411: if the number of target terminal devices corresponding to the ray is one, and the number of screen projection windows corresponding to the ray is multiple, then the screen projection window that is closest to the target terminal device among the screen projection windows corresponding to the ray is determined as the target screen projection window corresponding to the target terminal device;

[0118] Step 1412: if there are multiple target terminal devices corresponding to the ray, then traverse the projection windows corresponding to the ray according to the distance from the VR device from small to large, and for any traversed projection window, determine the projection window as the target projection window of the target terminal device closest to the projection window, and then return to the step of traversing the projection windows corresponding to the ray according to the distance from the VR device from small to large, until the corresponding target projection windows are determined for all the target terminal devices corresponding to the ray;

[0119] Step 1413: Display the projection data sent by the terminal device through the target projection window of the terminal device.

[0120] Based on the same inventive concept, the screen projection display method of the VR device disclosed above can also be implemented by a screen projection display device of a VR device. The effect of the screen projection display device of the VR device is similar to that of the aforementioned method, and will not be repeated here.

[0121] Fig.15 The present invention is a structural schematic diagram of a screen projection display device of a VR device according to an embodiment of the present invention.

[0122] like Fig.15 As shown, the screen projection display device 1500 of the VR device disclosed in the present invention may include a connection module 1510, a screen projection position coordinate determination module 1520, a target display area determination module 1530, a screen projection window determination module 1540, a target screen projection window determination module 1550 and a screen projection display module 1560.

[0123] The connection module 1510 is used to establish a connection with any terminal device connected to the same local area network as the VR device after receiving a screen projection request sent by the terminal device;

[0124] The projection position coordinate determination module 1520 is used to convert the map position coordinates of the terminal device in the map coordinate system into the projection position coordinates of the terminal device in the projection display coordinate system when receiving the projection data sent by the terminal device, wherein the map position coordinates are obtained based on the SLAM spatial positioning technology;

[0125] A target display area determination module 1530 is used to construct a target display area for displaying projection data according to the current position of the VR device;

[0126] A screen projection window determination module 1540, configured to determine a position of at least one screen projection window in the target display area based on the number of terminal devices that send screen projection requests, wherein the number of screen projection windows is obtained based on the number of terminal devices;

[0127] The target projection window determination module 1550 is used to determine, for any terminal device, a target projection window corresponding to the terminal device using the projection position coordinates of the terminal device and the position of the at least one projection window;

[0128] The screen projection display module 1560 is used to display the screen projection data sent by the terminal device through the target screen projection window of the terminal device.

[0129] In one embodiment, the target display area determination module 1530 is specifically configured to:

[0130] Taking the current position of the VR device as the origin, an initial ray is emitted toward the front of the VR device;

[0131] The initial ray is used as a reference, and a spatial region corresponding to a left-right viewing angle of the initial ray being a first specified angle and a top-bottom viewing angle being a second specified angle is determined as the target display region.

[0132] In one embodiment, the projection window determination module 1540 is specifically used to:

[0133] Taking the current position of the VR device as the origin, a target number of rays are emitted in the target display area, wherein the target number is obtained based on the number of terminal devices that send screen projection requests, and the emitted target number of rays are in the same horizontal plane, and the angles between any two adjacent rays are the same;

[0134] According to the projection position coordinates of the terminal device that sends the projection request and the current position of the VR device, the display type and position of the projection window corresponding to the target number of rays are determined.

[0135] In one embodiment, the display type of the projection window includes a single-layer display type and a multi-layer display type, and the single-layer display type has one projection window corresponding to any ray, and the multi-layer display type has more than one projection window corresponding to any ray; the projection window determination module 1540 is specifically used to:

[0136] If the number of terminal devices sending screen projection requests is less than the specified number, the display type is determined to be a single-layer display type, and for any ray, the position of the screen projection window corresponding to the ray is obtained according to the target length, wherein the target length is obtained based on the screen projection position coordinates of the terminal device sending the screen projection request and the current position of the VR device; or,

[0137] If the number of terminal devices sending screen projection requests is not less than the specified number, the display type is determined to be a multi-layer display type, and for any ray, the respective positions of the multiple screen projection windows corresponding to the ray are determined according to the target length.

[0138] In one embodiment, the target projection window determination module 1550 is specifically used to:

[0139] For any terminal device, determine the distance between the terminal device and each ray according to the projection position coordinates of the terminal device; and,

[0140] Determine the ray with the shortest distance as the target ray corresponding to the terminal device;

[0141] According to the target rays corresponding to the terminal devices, the target terminal devices corresponding to the rays are obtained;

[0142] For any ray, if the number of target terminal devices corresponding to the ray is one, and the number of screen projection windows corresponding to the ray is one, the screen projection window corresponding to the ray is determined as the target screen projection window corresponding to the target terminal device; or,

[0143] If the number of target terminal devices corresponding to the ray is one, and the number of screen projection windows corresponding to the ray is multiple, then the screen projection window closest to the target terminal device among the screen projection windows corresponding to the ray is determined as the target screen projection window corresponding to the target terminal device; or

[0144] If there are multiple target terminal devices corresponding to the ray, the projection windows corresponding to the ray are traversed according to the distance from the VR device in ascending order. For any traversed projection window, the projection window is determined as the target projection window of the target terminal device closest to the projection window, and the process returns to the step of traversing the projection windows corresponding to the ray according to the distance from the VR device in ascending order until the corresponding target projection windows are determined for all the target terminal devices corresponding to the ray.

[0145] In one embodiment, the projection position coordinate determination module 1520 is specifically used to:

[0146] If the coordinate system parameters of the map coordinate system are the same as the coordinate system parameters of the projection display coordinate system, the map position coordinates are reduced by a specified multiple to obtain the projection position coordinates of the terminal device, wherein the coordinate system parameters include the origin position, the position and direction of the horizontal axis, the position and direction of the vertical axis, and the position and direction of the vertical axis; or,

[0147] If the coordinate system parameters of the map coordinate system are different from the coordinate system parameters of the projection display coordinate system, the map position coordinates will be converted using a pre-set rotation and translation matrix between the map coordinate system and the projection display coordinate system to obtain the projection position coordinates of the terminal device.

[0148] After introducing a screen projection display method and apparatus for a VR device according to an exemplary embodiment of the present invention, next, a VR device according to another exemplary embodiment of the present invention is introduced.

[0149] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0150] In some possible implementations, the VR device according to the present invention may include at least one processor and at least one computer storage medium. The computer storage medium stores program code, and when the program code is executed by the processor, the processor executes the steps of the screen projection display method of the VR device according to various exemplary embodiments of the present invention described above in this specification. For example, the processor may execute the following steps: Figure 2 Steps 201-206 shown in .

[0151] Refer to the following Fig.16 hereinafter, a VR device 1600 according to this embodiment of the present invention is described. Fig.16 The VR device 1600 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0152] like Fig.16 As shown, the VR device 1600 is presented in the form of a general VR device. The components of the VR device 1600 may include, but are not limited to: the at least one processor 1601, the at least one computer storage medium 1602, and a bus 1603 connecting different system components (including the computer storage medium 1602 and the processor 1601).

[0153] Bus 1603 represents one or more of several types of bus structures, including a computer storage media bus or computer storage media controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0154] Computer storage media 1602 may include readable media in the form of volatile computer storage media, such as random access computer storage media (RAM) 1621 and / or cache storage media 1622 , and may further include read-only computer storage media (ROM) 1623 .

[0155] The computer storage medium 1602 may also include a program / utility 1625 having a set (at least one) of program modules 1624, such program modules 1624 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0156] The VR device 1600 may also communicate with one or more external devices 1604 (e.g., keyboards, pointing devices, etc.), and may also communicate with one or more devices that enable a user to interact with the VR device 1600 and / or any device that enables the VR device 1600 to communicate with one or more other VR devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1605. Furthermore, the VR device 1600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 1606. As shown, the network adapter 1606 communicates with other modules for the VR device 1600 via a bus 1603. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the VR device 1600, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0157] In some possible implementations, various aspects of the screen projection and display method for a VR device provided by the present invention may also be implemented in the form of a program product, which includes program code. When the program product runs on a computer device, the program code is used to enable the computer device to execute the steps of the screen projection and display method for a VR device according to various exemplary embodiments of the present invention described above in this specification.

[0158] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A screen projection display method for a VR device, characterized in that: The method comprises: After receiving a screen projection request sent by any terminal device connected to the same local area network as the VR device, a connection is established with the terminal device; When receiving the projection data sent by the terminal device, the map position coordinates of the terminal device in the map coordinate system are converted into the projection position coordinates of the terminal device in the projection display coordinate system, wherein the map position coordinates are obtained based on the SLAM spatial positioning technology; Constructing a target display area for displaying projection data according to the current position of the VR device; Based on the number of terminal devices that send screen projection requests, determine the position of at least one screen projection window in the target display area, wherein the number of the screen projection windows is obtained based on the number of the terminal devices; For any terminal device, using the projection position coordinates of the terminal device and the position of the at least one projection window, determine a target projection window corresponding to the terminal device; and, The projection data sent by the terminal device is displayed through the target projection window of the terminal device.

2. The method according to claim 1, characterized in that The step of constructing a target display area for displaying projection data according to the current position of the VR device includes: Taking the current position of the VR device as the origin, an initial ray is emitted toward the front of the VR device; The initial ray is used as a reference, and a spatial region corresponding to a left-right viewing angle of the initial ray being a first specified angle and a top-bottom viewing angle being a second specified angle is determined as the target display region.

3. The method according to claim 1, characterized in that The determining the position of at least one screen projection window in the target display area based on the number of terminal devices sending the screen projection request includes: Taking the current position of the VR device as the origin, a target number of rays are emitted in the target display area, wherein the target number is obtained based on the number of terminal devices that send screen projection requests, and the emitted target number of rays are in the same horizontal plane, and the angles between any two adjacent rays are the same; According to the projection position coordinates of the terminal device that sends the projection request and the current position of the VR device, the display type and position of the projection window corresponding to the target number of rays are determined.

4. The method according to claim 3, characterized in that The display type of the projection window includes a single-layer display type and a multi-layer display type, and the single-layer display type means that the number of projection windows corresponding to any ray is one, and the multi-layer display type means that the number of projection windows corresponding to any ray is greater than one; The determining, according to the projection position coordinates of the terminal device that sends the projection request and the current position of the VR device, the display type and position of the projection window corresponding to the target number of rays includes: If the number of terminal devices sending screen projection requests is less than the specified number, the display type is determined to be a single-layer display type, and for any ray, the position of the screen projection window corresponding to the ray is obtained according to the target length, wherein the target length is obtained based on the screen projection position coordinates of the terminal device sending the screen projection request and the current position of the VR device; or, If the number of terminal devices sending screen projection requests is not less than the specified number, the display type is determined to be a multi-layer display type, and for any ray, the respective positions of the multiple screen projection windows corresponding to the ray are determined according to the target length.

5. The method according to claim 3, characterized in that: Determining a target screen projection window corresponding to the terminal device by using the screen projection position coordinates of the terminal device and the position of the at least one screen projection window includes: For any terminal device, determine the distance between the terminal device and each ray according to the projection position coordinates of the terminal device; and, Determine the ray with the shortest distance as the target ray corresponding to the terminal device; According to the target rays corresponding to the terminal devices, the target terminal devices corresponding to the rays are obtained; For any ray, if the number of target terminal devices corresponding to the ray is one, and the number of screen projection windows corresponding to the ray is one, the screen projection window corresponding to the ray is determined as the target screen projection window corresponding to the target terminal device; or, If the number of target terminal devices corresponding to the ray is one, and the number of screen projection windows corresponding to the ray is multiple, then the screen projection window closest to the target terminal device among the screen projection windows corresponding to the ray is determined as the target screen projection window corresponding to the target terminal device; or If there are multiple target terminal devices corresponding to the ray, the projection windows corresponding to the ray are traversed according to the distance from the VR device in ascending order. For any traversed projection window, the projection window is determined as the target projection window of the target terminal device closest to the projection window, and the process returns to the step of traversing the projection windows corresponding to the ray according to the distance from the VR device in ascending order until the corresponding target projection windows are determined for all the target terminal devices corresponding to the ray.

6. The method according to claim 1, characterized in that The converting the map position coordinates of the terminal device in the map coordinate system into the projection position coordinates of the terminal device in the projection display coordinate system includes: If the coordinate system parameters of the map coordinate system are the same as the coordinate system parameters of the projection display coordinate system, the map position coordinates are reduced by a specified multiple to obtain the projection position coordinates of the terminal device, wherein the coordinate system parameters include the origin position, the position and direction of the horizontal axis, the position and direction of the vertical axis, and the position and direction of the vertical axis; or, If the coordinate system parameters of the map coordinate system are different from the coordinate system parameters of the projection display coordinate system, the map position coordinates will be converted using a pre-set rotation and translation matrix between the map coordinate system and the projection display coordinate system to obtain the projection position coordinates of the terminal device.

7. A VR device, characterized in that: comprising a processor and a memory, wherein the processor and the memory are connected via a bus; The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program: After receiving a screen projection request sent by any terminal device connected to the same local area network as the VR device, a connection is established with the terminal device; When receiving the projection data sent by the terminal device, the map position coordinates of the terminal device in the map coordinate system are converted into the projection position coordinates of the terminal device in the projection display coordinate system, wherein the map position coordinates are obtained based on the SLAM spatial positioning technology; Constructing a target display area for displaying projection data according to the current position of the VR device; Based on the number of terminal devices that send screen projection requests, determine the position of at least one screen projection window in the target display area, wherein the number of the screen projection windows is obtained based on the number of the terminal devices; For any terminal device, using the projection position coordinates of the terminal device and the position of the at least one projection window, determine a target projection window corresponding to the terminal device; and, The projection data sent by the terminal device is displayed through the target projection window of the terminal device.

8. The VR device according to claim 7, characterized in that: The processor executes the step of constructing a target display area for displaying projection data according to the current position of the VR device, and is specifically configured as follows: Taking the current position of the VR device as the origin, an initial ray is emitted toward the front of the VR device; The initial ray is used as a reference, and a spatial region corresponding to a left-right viewing angle of the initial ray being a first specified angle and a top-bottom viewing angle being a second specified angle is determined as the target display region.

9. The VR device according to claim 7, characterized in that: The processor executes the step of determining the position of at least one screen projection window in the target display area based on the number of terminal devices that send the screen projection request, and is specifically configured as follows: Taking the current position of the VR device as the origin, a target number of rays are emitted in the target display area, wherein the target number is obtained based on the number of terminal devices that send screen projection requests, and the emitted target number of rays are in the same horizontal plane, and the angles between any two adjacent rays are the same; According to the projection position coordinates of the terminal device that sends the projection request and the current position of the VR device, the display type and position of the projection window corresponding to the target number of rays are determined.

10. The VR device according to claim 9, characterized in that: The display type of the projection window includes a single-layer display type and a multi-layer display type, and the single-layer display type means that the number of projection windows corresponding to any ray is one, and the multi-layer display type means that the number of projection windows corresponding to any ray is greater than one; The processor executes the step of determining the display type and position of the projection window corresponding to the target number of rays according to the projection position coordinates of the terminal device that sends the projection request and the current position of the VR device, and is specifically configured as follows: If the number of terminal devices sending screen projection requests is less than the specified number, the display type is determined to be a single-layer display type, and for any ray, the position of the screen projection window corresponding to the ray is obtained according to the target length, wherein the target length is obtained based on the screen projection position coordinates of the terminal device sending the screen projection request and the current position of the VR device; or, If the number of terminal devices sending screen projection requests is not less than the specified number, the display type is determined to be a multi-layer display type, and for any ray, the respective positions of the multiple screen projection windows corresponding to the ray are determined according to the target length.

Images