Video sharing method and system based on virtual reality display device

Abstract

The present invention provides a video sharing method and system based on a virtual reality display device. The method includes that a terminal device sends video sharing request information to a server, the server obtains information of an allocated video sharing channel from a video live platform, the server sends a video sharing instruction to the virtual reality display device, and the video sharing instruction includes the information of the video sharing channel; wherein, the virtual reality display device obtains currently displayed video data and sends the video data to the video sharing channel by using a dependency library file, the dependency library file includes a first dependency library file and a second dependency library file, the first dependency library file is encapsulated and compiled by using a first programming language, and the second dependency library file is encapsulated and compiled by using the first programming language; the terminal device obtains video data from the video sharing channel. The system implements the above method. The present invention can improve the compatibility when the virtual reality display device performs video sharing.

Description

Technical Field

[0001] The present invention relates to the technical field of video sharing. Specifically, it is a video sharing method based on a virtual reality display device and a system for implementing this method. Background Art

[0002] With the development of virtual reality technology, virtual reality display devices (such as VR glasses) are increasingly widely used in games and virtual teaching. For example, users can play virtual games by wearing virtual reality display devices, or conduct teaching in a simulated specific scenario, such as medical teaching.

[0003] Since the images presented by virtual reality display devices can usually only be seen by the users wearing the virtual reality display devices themselves, if other people, such as training personnel, want to understand the training situation, it is difficult to understand the user's training situation in real time from the images seen by the users themselves. For this reason, people hope to share the videos presented by virtual reality display devices to the terminal devices of training personnel through video sharing, and the training personnel can timely understand the images seen by the users through virtual reality display devices through terminal devices such as smart phones and tablet computers, so as to understand the user's training situation.

[0004] Generally, the video data of virtual reality display devices can be displayed on the terminal devices of training personnel through live broadcast. Before the video live broadcast, the training personnel log in to the server through a smart phone, and each virtual reality display device is also registered on the server. After the terminal device sends a video sharing request message to the server, the server will obtain a dedicated video sharing channel from the video live broadcast platform, and the server sends an instruction containing the information of the video sharing channel to the virtual reality display device, and the virtual reality display device uploads the currently presented video data to the video sharing channel.

[0005] Since the virtual reality display device needs to process video data and respond to the instructions sent by the server, an application program needs to run on the virtual reality display device to identify and parse the received patents. Usually, this application program only needs to use an SDK (software development kit) compatible with the video live broadcast platform and call the "push screen data to the specified channel" function provided by the SDK to implement the video sharing operation.

[0006] However, since the data transmission interface of the video live streaming platform is usually developed based on the Android operating system, although the virtual reality display device also runs the Android operating system, the video sharing interface may not allow the use of the video sharing interface based on the Android operating system for security reasons and can only use other types of video sharing interfaces. Such a video sharing interface may be an application based on Xamarin programming. Therefore, this application cannot directly recognize and parse the interface data of the video live streaming platform and thus cannot execute the corresponding operation instructions.

[0007] In addition, since the available video sharing interface of the virtual reality display device is not developed based on the Android operating system, the video data obtained through the video sharing interface of the virtual reality display device cannot be directly recognized by the video live streaming platform, which in turn affects the training personnel's acquisition of the video data of the virtual reality display device through the terminal device. Summary of the Invention

[0008] The first object of the present invention is to provide a video sharing method based on a virtual reality display device that can be compatible with a video live streaming platform.

[0009] The second object of the present invention is to provide a video sharing system based on a virtual reality display device that implements the above-mentioned video sharing method based on a virtual reality display device.

[0010] To achieve the first object of the present invention, the video sharing method based on a virtual reality display device provided by the present invention includes that a terminal device sends video sharing request information to a server, the server obtains information about an allocated video sharing channel from a video live streaming platform, the server sends a video sharing instruction to the virtual reality display device, and the video sharing instruction contains information about the video sharing channel; wherein, the virtual reality display device obtains the currently displayed video data and uses a dependency library file to send the video data to the video sharing channel, where the dependency library file includes a first dependency library file and a second dependency library file, the first dependency library file is encapsulated and compiled using a first programming language, and the second dependency library file is encapsulated and compiled using a second programming language; the second dependency library file is formed by binding the first dependency library file. When binding the first dependency library file to form the second dependency library file, a target function with the same name as the source function in the first dependency library file is established in the second dependency library file, and a mapping relationship between the source function and the target function is established, and a target interface with the same name as the source interface in the first dependency library file is established in the second dependency library file, and a mapping relationship between the source interface and the target interface is established; after the virtual reality display device sends the obtained video data to the video sharing channel, the terminal device obtains the video data from the video sharing channel.

[0011] As can be seen from the above solution, by binding the first dependency library file to form the second dependency library file, the conversion between the source functions of the first dependency library file and the target functions of the second dependency library file is realized, and the conversion between the source interfaces of the first dependency library file and the target interfaces of the second dependency library file is also realized, enabling the application programs on the virtual reality display device to recognize the information of the video sharing channel in the video sharing instruction sent by the server, enabling the video live streaming platform to recognize the video data transmitted by the virtual reality display device, and enabling the terminal device to parse and play the video data transmitted by the virtual reality display device.

[0012] A preferred solution is that the virtual reality display device obtains the currently displayed video data including: calling the screen recording interface of the virtual reality display device, obtaining each frame of initial video data, and converting the initial video data into video data in a target format to form a video stream.

[0013] It can be seen that the virtual reality display device obtains the initial video data by calling the screen recording interface and converts the obtained initial video data in format, enabling the video live streaming platform to recognize the video data transmitted by the virtual reality display device.

[0014] A further solution is that converting the initial video data into video data in a target format includes: using the initial video data as the original video data and converting the initial video data into a video format customized by the video live streaming platform.

[0015] Thus, it can be seen that the initial video data obtained through the screen recording interface is only used as the video source and the basis for video conversion, and the operation of video format conversion can be simply and quickly realized.

[0016] A further solution is that before converting the initial video data into a video format customized by the video live streaming platform, the following is also executed: setting up a picture listener, and when the picture listener listens for available picture frames, start obtaining the initial video data.

[0017] Thus, it can be seen that by listening for picture frames through the picture listener, it is possible to avoid the virtual reality display device from obtaining video data for a long time and improve the efficiency of obtaining video data.

[0018] A further solution is that after obtaining the initial video data, the time stamps of each frame of video data are also set to mark the order of each frame of video data through the time stamps.

[0019] A further solution is that the external call interfaces of the first dependency library file only include a video sharing start interface and a video sharing stop interface.

[0020] It can be seen that the external call interfaces of the first dependent library file are very few. When calling the first dependent library file, only the video sharing start interface therein needs to be called to start the recording of video data, and only the video sharing stop interface therein needs to be called to stop the recording of video data. Other functions all depend on the function implementation inside the first dependent library file, and the virtual reality display device does not need to perform other operations, making the acquisition of video data very simple.

[0021] A further solution is that the virtual reality display device sends the acquired video data to the video sharing channel, including: the application program on the virtual reality display device calls the dependent library file, and the dependent library file sends the video data to the video sharing channel.

[0022] It can be seen that the video data of the virtual reality display device is acquired through the dependent library file, and the video data is transmitted to the video sharing channel of the video live platform, thereby realizing the conversion of instructions between two different programming languages.

[0023] To achieve the above second object, the video sharing system based on the virtual reality display device provided by the present invention includes a terminal device, a server, and a virtual reality display device. The terminal device is used to send video sharing request information to the server; the server is used to obtain the information of the allocated video sharing channel from the video live platform and send a video sharing instruction to the virtual reality display device, and the video sharing instruction includes the information of the video sharing channel; wherein, the virtual reality display device is used to acquire the currently displayed video data and use the dependent library file to send the video data to the video sharing channel, wherein the dependent library file includes a first dependent library file and a second dependent library file. The first dependent library file is encapsulated and compiled using a first programming language, and the second dependent library file is encapsulated and compiled using a second programming language; the second dependent library file is formed by binding the first dependent library file. When binding the first dependent library file to form the second dependent library file, a target function with the same name as the source function in the first dependent library file is established in the second dependent library file, and a mapping relationship between the source function and the target function is established. A target interface with the same name as the source interface in the first dependent library file is established in the second dependent library file, and a mapping relationship between the source interface and the target interface is established; the virtual reality display device also sends the acquired video data to the video sharing channel, and the terminal device also acquires video data from the video sharing channel. Description of the Drawings

[0024] Figure 1 It is a structural block diagram of an embodiment of the video sharing system based on the virtual reality display device of the present invention.

[0025] Figure 2 It is a flowchart of an embodiment of the video sharing method based on the virtual reality display device of the present invention.

[0026] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. Specific embodiments

[0027] The video sharing method based on a virtual reality display device of the present invention is applied to a virtual reality display device and associated devices. Specifically, the virtual reality display device transmits video data to a preset video sharing channel of a video live streaming platform, and the terminal device obtains the video data from the video sharing channel, so as to achieve the purpose of viewing the video data presented by the virtual display device on the terminal device.

[0028] Embodiment of a video sharing system based on a virtual reality display device:

[0029] This embodiment includes a terminal device 11, a server 12, a virtual reality display device 13, and a video live streaming platform 14. Among them, a user can wear the virtual reality display device 13 to participate in training and games, obtain the head rotation data of the user on the virtual reality display device 13, and display corresponding images according to the head rotation situation of the user, thereby forming video data. When a trainer needs to understand the training situation of the user, the trainer can obtain the video data presented by the virtual reality display device 13 through the terminal device 11.

[0030] Specifically, the trainer can send video sharing request information to the server 12 through a terminal device 11 such as a smart phone or a tablet computer. The server 12 determines which virtual reality display devices 13 need to obtain video data according to the information received from the terminal device 11. For this purpose, multiple virtual reality display devices 13 need to be registered in the server 12 in advance. For example, each virtual reality display device 13 is provided with its own unique number, and the server 12 can mark each virtual reality display device 13 through the number of the virtual reality display device 13. Moreover, the server 12 and the virtual reality display device 13 can communicate through a wireless communication method. For example, the server 12 can send information to the virtual reality display device 13, and the virtual reality display device 13 can also feedback information such as the current working state to the server 12.

[0031] Before the terminal device 11 sends video sharing request information to the server 12, it needs to be registered and logged in on the server 12. The server 12 determines the terminal device 11 that currently sends video sharing request information. For example, each terminal device 11 is marked through the ID of the terminal device 11.

[0032] In this embodiment, the server 12 realizes the function of video sharing through the video live streaming platform 14. Specifically, after the server 12 sends video sharing request information to the video live streaming platform 14, the video live streaming platform will allocate a dedicated video sharing channel, which is a storage space allocated under the video live streaming platform 14 for storing video data uploaded from the virtual reality display device 13. Preferably, this video sharing channel has its own independent link address. The video live streaming platform 14 can return information about the video sharing channel to the server 12, including the ID of the video sharing channel, the link address, and also including the requirements of the video sharing channel for the video data format, etc.

[0033] The server 12 can send a video sharing instruction to the virtual reality display device 13. This video sharing instruction contains information about the video sharing channel. The virtual reality display device 13 obtains the currently displayed video data according to the received instruction, and transmits the video data to the video sharing channel. The terminal device 11 can obtain the video data uploaded by the virtual reality display device 13 from the video sharing channel, so as to display this video data on the terminal device 11.

[0034] Preferably, an application program runs on the virtual reality display device 13, and this application program needs to parse the information of the video sharing channel of the video live streaming platform 14. However, since the video live streaming platform 14 is edited based on the first programming language, for example, based on JAVA, and the video sharing interface of the virtual reality display device 13 is not edited based on the first programming language, for example, based on the second programming language. This results in that although the application program on the virtual reality display device 13 can obtain the currently displayed video data, it cannot send the video data to the video sharing interface of the video live streaming platform 14 through the video sharing interface. For this reason, this embodiment needs to perform conversion processing on the virtual reality display device 13 so that the virtual reality display device 13 can transmit appropriate video data to the video live streaming platform 14.

[0035] In addition, since the programming language used by the video live streaming platform 14 is different from the programming language used by the video sharing interface of the virtual reality display device 13, the video format sent by the virtual reality display device 13 is also different from the video format that the video live streaming platform 14 can recognize. For this reason, this embodiment also needs to perform format conversion on the initial video data obtained by the virtual reality display device 13 to obtain a video format that the video live streaming platform 14 can recognize.

[0036] Embodiment of the video sharing method based on a virtual reality display device:

[0037] Next, in combination with Figure 2Describe the workflow of the video sharing system based on the virtual reality display device. After the terminal device logs in to the server, it executes step S1 and sends a video sharing request message to the server. Preferably, the video sharing request message includes information about the specified virtual reality display device, such as the number of the virtual reality display device that needs to obtain video images. The server determines which virtual reality display devices need to obtain video data based on the obtained video sharing request message.

[0038] Then, the server executes step S2 and sends a request message to the video live streaming platform. The video live streaming platform allocates a video sharing channel, and this video sharing channel is dedicated to storing the video data uploaded by a specific virtual reality display device for a period of time. After allocating the video sharing channel, the video live streaming platform sends the information of the video sharing channel to the server, including the link address, number, etc. of the video sharing channel. And the video live streaming platform also sends information such as the format of the video data to the server.

[0039] Next, execute step S3. The server sends a video sharing instruction to the virtual reality display device. This video sharing instruction includes the information of the video sharing channel and also includes information such as the format requirements of the video data. After receiving the video sharing instruction, the virtual reality display device executes step S4, obtains the currently displayed video data, and encapsulates the currently displayed video data.

[0040] Specifically, the virtual reality display device creates a dependency library file, which includes a first dependency library file and a second dependency library file. The first dependency library file is encapsulated and compiled by a first programming language. For example, it is edited using JAVA. The first dependency library file is a native database based on the Android operating system. The second dependency library file is encapsulated and compiled by a second programming language. And the second dependency library file is bound through the first dependency library file.

[0041] After the virtual reality display obtains the SDK provided by the video live streaming platform, it obtains some functions in the SDK and encapsulates them to form the first dependency library file. Since the first dependency library file is a native database based on the Android operating system and the SDK provided by the video live streaming platform is compatible with the Android operating system, the first dependency library file contains multiple functions and interfaces based on the Android operating system. These functions and interfaces can obtain the image currently displayed on the virtual reality display device, that is, perform the work of obtaining video data. Preferably, these functions can call the screen recording interface of the virtual reality display device itself to implement the acquisition of video data using the screen recording interface. And the external call interfaces of the first dependency library file only include the video sharing start interface and the video sharing stop interface.

[0042] Furthermore, a picture listener can be set up to monitor whether there are available picture frames on the virtual reality display device. If there are available picture frames, the initial video data is obtained, and each frame of the initial video data is used as the original video data. Then, format conversion is performed on the original video data. For example, each frame of video data is encapsulated into a custom format of video data that can be recognized by the video live streaming platform. Preferably, each frame of the initial video data is converted into video data in the YUV420 format. In addition, the timestamp of each frame of video data needs to be set, and each frame of video data is sorted according to the timestamps of each frame of video data to form a video stream. The operation of obtaining the video stream can be implemented based on the functions and interfaces of the first dependency library file.

[0043] However, since the video sharing interface of the virtual reality display device cannot recognize files edited based on JAVA, it is necessary to perform step S5 to bind the first dependency library file to form the second dependency library file. Specifically, since the first dependency library file includes multiple functions and multiple interfaces, which are the source functions and source interfaces in the first dependency library file respectively. When binding the first dependency library file to form the second dependency library file, the source functions need to be converted into target functions in the second dependency library file, and the source interfaces need to be converted into target interfaces in the second dependency library file. Therefore, a target function with the same name as the source function is created in the second dependency library file, that is, the name of the source function is the same as that of the target function, except that the source function is edited in the first programming language and the target function is edited in the second programming language. A target interface with the same name as the source interface is created in the second dependency library file, that is, the name of the source interface is the same as that of the target interface, except that the source interface is edited in the first programming language and the target interface is edited in the second programming language. In addition, a mapping relationship needs to be established between the source function and the target function, and a mapping relationship needs to be established between the source interface and the target interface.

[0044] Specifically, the source functions and source interfaces of the first dependency library file are shown in the following table.

[0045]

[0046]

[0047] In this way, when the video sharing interface of the virtual reality display device calls the second dependency library file to parse functions and interfaces, through the mapping relationship between the target function and the source function, the corresponding data of the source function is obtained, so that the source function can be run on the virtual reality display device, and then the video data is obtained and uploaded to the video sharing channel of the video live streaming platform, that is, step S6 is executed.

[0048] After a virtual reality display device uploads a video data segment to a video sharing channel, a terminal device obtains information about the video sharing channel from a server, including the link address of the video sharing channel, and determines the specific website address where the video data to be currently viewed is stored according to the link address. The terminal device obtains the video data from the video sharing channel, and the trainer can view the video data currently seen by the user through the terminal device and judge the current training situation of the user according to the video data seen by the user.

[0049] Preferably, if the user makes an operation error or other situations during the training process, the trainer can send an instruction message through the terminal device, and the instruction message can be sent in the form of voice or text. The voice message sent by the terminal device can be directly sent to the server, and the server can push the instruction message sent by the trainer to the virtual reality display device. The virtual reality display device can add the text of the instruction message to the displayed picture, or the virtual reality display device plays the voice of the instruction message through a speaker, so that the user can know the instruction message sent by the trainer in time. Further, the video data uploaded by the virtual reality display device to the video sharing channel also contains the instruction message. Therefore, the trainer can timely know whether the instruction message sent by himself / herself is successfully sent through the video information displayed by the terminal device.

[0050] Further, the trainer can view the video data of multiple virtual reality display devices through the terminal device at the same time. For example, the terminal device sends a video sharing request message for multiple virtual reality display devices to the server, and the server requests the video live broadcast platform to allocate multiple video sharing channels. Each video sharing channel corresponds to a virtual reality display device, that is, the video data uploaded by a virtual reality display device is stored in a dedicated video sharing channel. The terminal device can obtain the video data of multiple video sharing channels. For example, the terminal device displays the video data of multiple virtual reality display devices in a nine-grid manner, so that the trainer can monitor the training situations of multiple users at the same time to implement a one-to-many training mode and improve the training efficiency.

[0051] It can be seen that the present invention can solve the problems of video data format incompatibility and program language inconsistency between the video sharing interface of the virtual reality display device and the video live broadcast platform, improve the compatibility performance of the virtual reality display device, and meet the video sharing requirements in special usage environments.

[0052] Finally, it should be emphasized that the present invention is not limited to the above embodiments. For example, changes in the first program language and the second program language, or changes in the content included in the video sharing request message, etc., should also be included in the protection scope of the claims of the present invention.

Claims

1. A video sharing method based on a virtual reality display device, comprising: The terminal device sends video sharing request information to the server, the server obtains information of an allocated video sharing channel from a video live platform, the server sends a video sharing instruction to the virtual reality display device, and the video sharing instruction includes the information of the video sharing channel; It is characterized in that: The virtual reality display device obtains the currently displayed video data, and uses a dependency library file to send the video data to the video sharing channel. Among them, the dependency library file includes a first dependency library file and a second dependency library file. The first dependency library file is encapsulated and compiled using a first programming language, and the second dependency library file is encapsulated and compiled using a second programming language; the second dependency library file is formed by binding the first dependency library file. When binding the first dependency library file to form the second dependency library file, a target function with the same name as the source function in the first dependency library file is established in the second dependency library file, and a mapping relationship between the source function and the target function is established. A target interface with the same name as the source interface in the first dependency library file is established in the second dependency library file, and a mapping relationship between the source interface and the target interface is established; After the virtual reality display device sends the obtained video data to the video sharing channel, the terminal device obtains the video data from the video sharing channel.

2. The video sharing method based on a virtual reality display device according to claim 1, characterized in that: The virtual reality display device obtaining the currently displayed video data includes: calling the screen recording interface of the virtual reality display device, obtaining each frame of initial video data, and converting the initial video data into video data in a target format to form a video stream.

3. The video sharing method based on a virtual reality display device according to claim 2, characterized in that: Converting the initial video data into video data in a target format includes: using the initial video data as the original video data and converting the initial video data into a video format customized by the video live platform.

4. The video sharing method based on a virtual reality display device according to claim 3, characterized in that: Before converting the initial video data into the video format customized by the video live platform, the following is also executed: Set a picture listener, and start obtaining initial video data when the picture listener monitors that there are available picture frames.

5. The video sharing method based on a virtual reality display device according to claim 4, characterized in that: After obtaining the initial video data, time stamps of each frame of video data are also set, and the order of each frame of video data is marked by the time stamps.

6. The video sharing method based on a virtual reality display device according to any one of claims 1 to 5, characterized in that: The external call interfaces of the first dependency library file only include a video sharing start interface and a video sharing stop interface.

7. The video sharing method based on a virtual reality display device according to any one of claims 1 to 3, characterized in that: The virtual reality display device sending the acquired video data to the video sharing channel includes: An application program on the virtual reality display device calls the dependency library file, and the dependency library file sends the video data to the video sharing channel.

8. A video sharing system based on a virtual reality display device, comprising a terminal device, a server, and a virtual reality display device, The terminal device is used to send video sharing request information to the server; The server is used to obtain information of an allocated video sharing channel from a video live platform and send a video sharing instruction to the virtual reality display device, and the video sharing instruction includes the information of the video sharing channel; It is characterized in that: The virtual reality display device is used to acquire the currently displayed video data and send the video data to the video sharing channel by using a dependency library file. The dependency library file includes a first dependency library file and a second dependency library file. The first dependency library file is encapsulated and compiled using a first programming language, and the second dependency library file is encapsulated and compiled using a second programming language; the second dependency library file is formed by binding the first dependency library file. When binding the first dependency library file to form the second dependency library file, a target function with the same name as the source function in the first dependency library file is established in the second dependency library file, and a mapping relationship between the source function and the target function is established. A target interface with the same name as the source interface in the first dependency library file is established in the second dependency library file, and a mapping relationship between the source interface and the target interface is established; The terminal device also acquires the video data from the video sharing channel.

9. The video sharing system based on a virtual reality display device according to claim 8, wherein: When the virtual reality display device acquires the currently displayed video data, it calls the screen recording interface of the virtual reality display device to acquire each frame of initial video data, and converts the initial video data into video data in a target format to form a video stream.

10. The video sharing system based on a virtual reality display device according to claim 9, wherein: When the virtual reality display device converts the initial video data into video data in a target format, it uses the initial video data as the original video data and converts the initial video data into a video format customized by the video live platform.

Images