Cloud VR interactive control data transmission method, device, intelligent terminal and storage medium

Abstract

The present invention discloses a cloud VR interactive control data transmission method, device, intelligent terminal and storage medium, the method comprising: receiving interactive control data; obtaining the category corresponding to the interactive control data based on the VR terminal that generates the interactive control data; obtaining the transport layer protocol for transmitting the interactive control data based on the category; transmitting the interactive control data based on the transport layer protocol. Compared with the prior art, the present invention establishes multiple transmission channels and adopts different transmission modes according to the characteristics of interactive control data in cloud VR, while taking into account the low latency and reliability of transmitting VR interactive control data. Even under low-bandwidth network conditions, the interactivity of VR devices can be guaranteed, and situations such as image tailing and video freeze that seriously affect the user experience can be prevented.

Description

Technical Field

[0001] The present invention relates to the field of data transmission technology, and in particular to a cloud VR interactive control data transmission method, device, intelligent terminal and storage medium. Background Art

[0002] At present, virtual reality technology VR (Virtual Reality) mostly adopts cloud VR mechanism, which stores and renders content in the cloud. Local devices only need to complete decoding and display, which reduces the requirements for device performance and effectively reduces costs. VR terminals generate interactive control data and send it to the server. The server obtains the data to be rendered from the VR system. The cloud rendering platform in the server performs real-time rendering based on the interactive control data and sends the rendered data to the VR terminal.

[0003] At present, when interactive control data is transmitted in the network, the transmission delay is relatively high due to network reasons. As a result, when it is applied to cloud VR, users experience image lag and slow response to control events during the experience, which greatly reduces the interactivity of VR devices.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide a cloud VR interactive control data transmission method, device, intelligent terminal and storage medium, aiming to solve the problems of high interactive control data delay and poor VR interactivity due to network reasons in the prior art.

[0006] In order to achieve the above object, the present invention provides a cloud VR interactive control data transmission method, wherein the method comprises:

[0007] receiving interactive control data;

[0008] Acquire the category corresponding to the interactive control data;

[0009] Based on the category, obtaining a transport layer protocol for transmitting the interactive control data;

[0010] Based on the transport layer protocol, the interactive control data is transmitted.

[0011] Optionally, obtaining a transport layer protocol for transmitting the interactive control data based on the category includes:

[0012] If the category is command data, the transport layer protocol is TCP protocol;

[0013] If the category is head interaction data or body interaction data, the transport layer protocol is UDP protocol.

[0014] Optionally, if the category is head interaction data or body interaction data, transmitting the interaction control data includes:

[0015] Determine the network transmission status based on the received statistical information reflecting the network transmission status;

[0016] If the network transmission state is network congestion, after receiving a set number of the interactive control data cumulatively, all the interactive control data are combined into a data packet, and the data packet is transmitted;

[0017] Otherwise, the interactive control data is transmitted.

[0018] Optionally, the combining all the interactive control data into a data packet, and after transmitting the data packet, further comprises:

[0019] The set number is doubled and updated.

[0020] Optionally, if the category is body interaction data and the length of the data packet is greater than the maximum transmission unit, after combining all the interaction control data into a data packet, the method further includes:

[0021] discarding the data packet;

[0022] Sorting all the interactive control data in time to form a data sequence;

[0023] discarding the interactive control data in the data sequence at intervals;

[0024] The data sequences are combined into data packets.

[0025] Optionally, if the category is header interaction data, after the cloud receives the interaction control data, the following further includes:

[0026] Based on the sequence number of the interactive control data, obtaining a judgment result according to an integrity judgment rule of the interactive control data;

[0027] If the judgment result is that the interactive control data is lost, obtain the sequence number of the lost interactive control data, obtain a set number of interactive control data cached before the sequence number, and reconstruct the lost interactive control data based on the cached interactive control data according to the Kalman filter algorithm.

[0028] In order to achieve the above-mentioned object, the second aspect of the present invention further provides a cloud VR interactive control data transmission device, comprising:

[0029] A data receiving module, used for receiving interactive control data;

[0030] A category determination module, used for obtaining the category corresponding to the interactive control data;

[0031] A protocol determination module, configured to obtain a transport layer protocol for transmitting the interactive control data based on the category;

[0032] A transmission module is used to transmit the interactive control data based on the transport layer protocol.

[0033] Optionally, a reconstruction module is also included, which is used to reconstruct the lost interactive control data based on the cached interactive control data according to the Kalman filter algorithm.

[0034] In order to achieve the above-mentioned purpose, the third aspect of the present invention also provides an intelligent terminal, which includes a memory, a processor, and a cloud VR interactive control data transmission program stored in the memory and executable on the processor. When the cloud VR interactive control data transmission program is executed by the processor, any step of the cloud VR interactive control data transmission method is implemented.

[0035] In order to achieve the above-mentioned purpose, the fourth aspect of the present invention also provides a computer-readable storage medium, on which a cloud VR interactive control data transmission program is stored. When the cloud VR interactive control data transmission program is executed by a processor, any step of the cloud VR interactive control data transmission method is implemented.

[0036] It can be seen from the above that the cloud VR interactive control data transmission method, device, intelligent terminal and storage medium of the present invention, by dividing the interactive control data into different categories and adopting different transport layer protocols for different categories of interactive control data, can realize dynamic and flexible adjustment of the transmission mode of the interactive control data, make the network transmission more stable, reduce the delay of the interactive control data, and achieve better VR interaction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 It is a flow chart of the cloud VR interactive control data transmission method provided by the present invention;

[0039] Figure 2 yes Figure 1 A schematic diagram of a specific flow chart of step S400 of the embodiment;

[0040] Figure 3 yes Figure 2A schematic diagram of a specific flow chart of step S420 of the embodiment;

[0041] Figure 4 yes Figure 1 A schematic diagram of a process of reconstructing interactive control data in the cloud of an embodiment;

[0042] Figure 5 It is a structural diagram of a cloud VR interactive control data transmission device provided by an embodiment of the present invention;

[0043] Figure 6 It is a block diagram of the internal structure principle of an intelligent terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0045] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0046] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0047] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0048] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0049] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0051] According to the location where the service runs, VR (Virtual Reality) is divided into two categories: local VR and cloud VR. Local VR stores and renders content locally. This solution requires local devices to have quite high computing power, such as high-performance PCs, all-in-one VR machines, etc., which will bring high costs to users. Cloud VR stores and renders content in the cloud, and only decoding and display need to be completed locally, thereby reducing the requirements for device performance and effectively reducing costs. Currently, the cloud VR mechanism is mostly used. However, when the interactive control data is transmitted in the network, the end-to-end transmission delay of the cloud VR platform is relatively high, resulting in poor fluency. Users experience screen lag and slow response to control events during the experience.

[0052] The cloud VR interactive control data transmission method of the present invention divides the interactive control data into different categories and adopts different transport layer protocols for different categories of interactive control data, so as to dynamically and flexibly adjust the transmission mode of the interactive control data, make the network transmission more stable, reduce the delay of the interactive control data, and achieve better VR interaction effect.

[0053] Exemplary Methods

[0054] like Figure 1 As shown, an embodiment of the present invention provides a cloud VR interactive control data transmission method. Specifically, the method includes the following steps:

[0055] Step S100: receiving interactive control data;

[0056] Specifically, the interactive control data is generated by the VR terminal and the cloud. After the VR terminal generates the interactive control data, it needs to transmit the interactive control data to the cloud by wireless or wired means, and the interactive control data generated in the cloud needs to be transmitted to the VR terminal. Usually, one or more of the above data items such as position data, posture data, and control data will be generated on the VR terminal, and the combination of single data or multiple data in the above data items is interactive control data. The specific equipment of the VR terminal is not limited, and can be a VR helmet, VR handle, somatosensory interaction equipment (such as somatosensory chairs, treadmills, somatosensory clothes, space positioning equipment, motion capture equipment, etc.); the interactive control data generated in the cloud is used to control the VR terminal to achieve certain effects. For example, the vibration feedback data generated in the cloud can achieve the somatosensory vibration effect of the VR terminal.

[0057] Step S200: obtaining the category corresponding to the interactive control data;

[0058] Specifically, since the importance of interactive control data generated by different VR terminal devices to the VR interactive effect is different, some interactive control data cannot be lost, and some interactive control data must ensure low latency. In other words, the interactive control data generated by different VR terminals have different requirements for latency and packet loss rate during transmission. According to the different requirements for latency and packet loss rate, the present invention divides the interactive control data into three categories: head interactive data, body interactive data, and command data.

[0059] Head interaction data refers to one or a combination of data items such as sensing data, capture data, feedback data, etc. collected in or acting on the head area of ​​the VR user, mainly including position data and posture data. Since the screen rendering of the user's VR perspective (FOV) is directly determined by this type of data, if the delay in the transmission of this type of data is high, it will cause dizziness to the user, so it is necessary to reduce the transmission delay of this type of data as much as possible;

[0060] Body interaction data refers to one or a combination of multiple data items such as sensing data, capture data, feedback data, etc. collected outside the head area of ​​the VR user or acting on the head area of ​​the VR user. It is mainly somatosensory interaction data, including position data and posture data of the hands, feet, handheld controllers, and body trunk. Since this type of data does not affect the VR perspective, even if there is a delay or data loss, it will not cause dizziness during use. In other words, body interaction data can allow a certain delay or loss;

[0061] The command data is mainly generated by the VR controller buttons and the vibration feedback data of the VR terminal generated by the cloud. Since people have a reaction time of about 0.2 seconds, the command data can have a delay of about 100ms, which will not affect the operating experience. However, it cannot be lost, otherwise the control event will be unresponsive or the control event response will be misplaced, which greatly reduces the interactivity of the VR device.

[0062] Therefore, after the interactive control data is classified according to the above categories, the category corresponding to the interactive control data can be obtained according to the source of the interactive control data, that is, which VR terminals the interactive control data comes from.

[0063] Optionally, the interactive control data generated by the cloud can also be classified according to the effect area produced by the action on the target VR terminal. For example, the vibration feedback data generated by the cloud usually acts on the VR helmet, and the vibration feedback data can be divided into head interaction data to achieve better interaction effect.

[0064] Step S300: Based on the category, obtaining a transport layer protocol for transmitting interactive control data;

[0065] Specifically, the existing cloud VR transmission focuses mainly on the transmission efficiency of cloud-generated videos, and does not focus on the transmission efficiency of interactive control data generated by VR devices. However, in the actual transmission process of interactive control data, data delays and packet loss are often caused due to network reasons, which directly affects the timeliness and video effects of cloud VR videos. Therefore, on the basis of classifying interactive control data by category, the present invention adopts different transport layer protocols for different interactive control data, reduces network transmission pressure, realizes low-latency transmission of interactive control data, and ensures the VR effect of cloud video.

[0066] When the category corresponding to the interactive control data is head interaction data or body interaction data, these two types of data are characterized by high frequency and small packets. The high frequency means that the frequency of these two types of data is approximately equal to the frequency of VR screen refresh rendering, that is, not less than 60hz, generally around 90-100hz, and can even reach 1000hz. The small packet means that these two types of data are just a 6-dimensional floating point coordinate with a size of 24 bytes. Therefore, these two types of data are generated at a high frequency. Although a single data packet is very small, the number of data packets is huge. When a huge amount of such small packet data is transmitted using the existing TCP, it is easy to be affected by network jitter and cause network congestion and network delay. Therefore, based on the above reasons, the transport layer protocol of the present invention selects the UDP protocol when transmitting these two types of data. Compared with the TCP protocol, the UDP protocol has high transmission efficiency and is more suitable for communications or broadcast communications with high requirements for high-speed transmission and real-time performance.

[0067] When the category corresponding to the interactive control data is command data, since command data can accept a certain delay but cannot be lost, the data generation frequency is much lower than that of head interaction data and body interaction data. Therefore, when transmitting command data, the transport layer protocol selects TCP protocol. TCP can provide reliable services. Data transmitted through TCP connection is error-free, non-lost, non-duplicate, and arrives in order.

[0068] Therefore, the present invention establishes multiple transmission channels when transmitting interactive control data, and utilizes the characteristics of different transport layer protocols to transmit different types of interactive control data, thereby alleviating network pressure and making interactive control data transmission more efficient.

[0069] In other implementation scenarios, for head interaction data or body interaction data, a UDP-based Data Transfer Protocol (UDT) transmission protocol is used to replace the UDP protocol.

[0070] Step S400: Transmitting the interactive control data based on the transport layer protocol.

[0071] Specifically, after determining the transport layer protocol to be adopted, the interactive control data generated by the VR terminal is transmitted to the cloud; and the interactive control data generated by the cloud is transmitted to the VR terminal.

[0072] In summary, the cloud VR interactive control data transmission method of the present invention includes: receiving interactive control data; obtaining the category corresponding to the interactive control data based on the VR terminal that generates the interactive control data; obtaining the transport layer protocol for transmitting the interactive control data based on the category; and transmitting the interactive control data based on the transport layer protocol. By dividing the interactive control data into different categories and using different transport layer protocols for different categories of interactive control data, the transmission mode of the interactive control data can be dynamically and flexibly adjusted, the network transmission is more stable, the delay of the interactive control data is reduced, and the VR interaction effect is better.

[0073] If the category of the interactive control data is head interactive data or body interactive data, when the interactive control data is transmitted in step S400, in some embodiments, more specifically, the following is included: Figure 2 Steps shown:

[0074] Step S410: judging the network transmission status based on the received statistical information reflecting the network transmission status;

[0075] Specifically, after the interactive control data is transmitted to the cloud, the cloud counts the data packet information of the received interactive control data, calculates the network delay and packet loss rate, and feeds back the above network delay and packet loss rate and other information. When the packet loss rate or network delay is greater than the set threshold, it is determined that the network transmission is blocked, that is, the network transmission state is network congestion. Regarding the calculation method of network delay and packet loss rate, you can refer to the calculation method in the prior art, which will not be repeated here. As for setting the threshold, it can be pre-set according to the current network speed and signal conditions; you can also count the frequency of network congestion and the degree of network delay to change the set threshold.

[0076] Step S420: if the network transmission state is network congestion, after receiving a set number of interactive control data, all the interactive control data are combined into a data packet, and the data packet is transmitted;

[0077] Step S430: Otherwise, the interactive control data is transmitted.

[0078] Specifically, based on the characteristics of interactive control data such as head interactive data and body interactive data, if network congestion still occurs when using the UDP transmission protocol for transmission, the present invention further combines multiple interactive control data packets into one data packet to reduce the transmission frequency of network transmission, thereby alleviating network transmission pressure and improving network latency. The number of combined interactive control data in the data packet can be preset. In this embodiment, the number is set to 2, which is equivalent to halving the transmission frequency. If there is no network congestion during network transmission, the interactive control data is transmitted in a timely manner.

[0079] Furthermore, after combining multiple interactive control data into a data packet, the network transmission status may still not be improved. After combining multiple control data into a data packet and transmitting, the set number can be doubled and updated so that one data packet contains more interactive control data. For example, the set number is initially 2, and after combining two interactive control data into a data packet, the set number is updated to 4. The next time the cloud still reports network congestion, four interactive control data are combined into a data packet for transmission, and the set number is updated to 8. By analogy, the set number increases geometrically to achieve the purpose of improving the network transmission status.

[0080] Since the length of the data packet transmitted by the network is not unlimited, the length needs to be less than the maximum transmission unit. After the set number increases exponentially, the length of the combined data packet may be greater than the maximum transmission unit. If the category of the interactive control data is body interaction data, the characteristic of this type of data is that even if there is data loss, it will not affect the VR interaction effect. Therefore, for body interaction data, in some embodiments, after all the interactive control data are combined into a data packet in step S420, it also includes the following Figure 3Steps shown:

[0081] Step S421: discarding the data packet;

[0082] Step S422: sorting all interactive control data by time to form a data sequence;

[0083] Step S423: discarding the interactive control data in the data sequence at intervals;

[0084] Step S424: combine the data sequences into a data packet.

[0085] Specifically, since the combined data packet exceeds the maximum transmission unit, the current data packet is discarded first, and the data packet needs to be repacked. That is, the interactive control data is sorted by time to form a data sequence, and then the interactive control data in the data sequence is discarded at intervals to reduce the length of the data packet. For example, the interactive control data with sequence numbers 2, 4, 6, etc. in the data sequence are discarded, and then the data sequences with reduced numbers are combined to form a new data packet.

[0086] By adopting the UDP transmission protocol, the delay of data transmission can be significantly reduced. However, due to the characteristics of the UDP transmission protocol, there is also a probability of head interaction data loss. However, since the head interaction data is related to the screen rendering of the user's VR perspective (FOV), it is necessary to ensure the integrity of the head interaction data as much as possible and reduce the loss of head interaction data. Therefore, the present invention further judges the integrity of the received head interaction data after receiving the interactive control data in the cloud. If the head interaction data is lost, the head interaction data is reconstructed to make the head interaction data complete and ensure the VR interaction effect.

[0087] For the head interaction data, in some embodiments, after the cloud receives the interaction control data, it also includes: Figure 4 Steps shown:

[0088] Step SA10: Based on the sequence number of the interactive control data, a judgment result is obtained according to the integrity judgment rule of the interactive control data;

[0089] Specifically, when a data packet containing header interactive control data is sent to the cloud, the header of each data packet has a serial number that increases in sequence, and the generation frequency and interval time of the header interactive data are fixed values, that is, the sending interval time of the above data packets is the same. Based on the above characteristics, when the cloud receives the interactive control data, it can determine whether the interactive control data is lost according to the sequence number of the interactive control data. For example, after receiving the interactive control data with sequence number 17, if the sequence number of the interactive control data received before is 15, it can be determined that the interactive control data with sequence number 16 is lost. It can also be determined that the interactive control data with sequence number 16 is lost if the waiting time for receiving the interactive control data with sequence number 16 exceeds the above time interval, such as if the waiting time reaches 1.5 times the above time interval.

[0090] Step SA20: If the judgment result is that the interactive control data is lost, obtain the sequence number of the lost interactive control data, obtain a set number of interactive control data cached before the sequence number, and reconstruct the lost interactive control data based on the cached interactive control data according to the Kalman filter algorithm.

[0091] Specifically, the cloud in this embodiment always caches a set number of interactive control data based on the first-in-first-out rule. For example, if the set number is 10 and the sequence number of the interactive control data currently received is 20, the interactive control data with sequence numbers from 10 to 19 are cached. When it is determined that the interactive control data is lost, as described in step SA10, the sequence number of the lost interactive control data is obtained. If the sequence number of the currently lost interactive control data is 20, the interactive control data with sequence numbers from 10 to 19 stored in the cache are read. Since the specific content of the head interactive data is based on the continuous trajectory coordinates of the time series, the interactive control data with sequence number 20 can be reconstructed according to the existing Kalman filter algorithm, so that the head interactive data remains complete.

[0092] As described above, the present invention aims at the characteristics of interactive control data in cloud VR, establishes multiple transmission channels, adopts different transmission methods, and takes into account the low latency and reliability of transmitting VR interactive control data. Even under low-bandwidth network conditions, it can ensure the interactivity of VR devices and prevent situations such as image tailing and video freeze that seriously affect user experience.

[0093] Exemplary Devices

[0094] like Figure 5 As shown in , corresponding to the cloud VR interactive control data transmission method, an embodiment of the present invention further provides a cloud VR interactive control data transmission device, which can be deployed locally or in the cloud. The cloud VR interactive control data transmission device includes:

[0095] The data receiving module 600 is used to receive interactive control data;

[0096] Specifically, when the cloud VR interactive control data transmission device is deployed locally, the data receiving module 600 is used to receive the interactive control data generated by the VR terminal. The specific equipment of the VR terminal is not limited, and can be a VR helmet, a VR handle, a somatosensory interactive device (such as a somatosensory seat, a treadmill, somatosensory clothes, a spatial positioning device, a motion capture device, etc.); when the cloud VR interactive control data transmission device is deployed in the cloud, the data receiving module 600 is used to receive the interactive control data generated in the cloud for controlling the VR terminal to achieve certain effects, such as the vibration feedback data generated in the cloud, which is used to achieve the somatosensory vibration effect of the VR terminal.

[0097] A category determination module 610 is used to obtain a category corresponding to the interactive control data;

[0098] Specifically, since the importance of interactive control data generated by different VR terminal devices to the VR interactive effect is different, some interactive control data cannot be lost, and some interactive control data must ensure low latency. In other words, the interactive control data generated by different VR terminals have different requirements for latency and packet loss rate during transmission. According to the different requirements for latency and packet loss rate, the present invention divides the interactive control data into three categories: head interactive data, body interactive data, and command data.

[0099] The head interaction data refers to one or a combination of multiple data items such as sensing data, capture data, feedback data, etc. collected in the head area of ​​the VR user or acting on the head area of ​​the VR user.

[0100] Body interaction data refers to one or a combination of multiple data items such as sensing data, capture data, feedback data, etc. collected outside the head area of ​​the VR user or acting on the head area of ​​the VR user.

[0101] The command data is mainly generated by the VR controller buttons and the vibration feedback data of the VR terminal generated by the cloud.

[0102] Therefore, after the interactive control data is classified according to the above categories, the category corresponding to the interactive control data can be obtained according to the source of the interactive control data, that is, which VR terminals the interactive control data comes from.

[0103] A protocol determination module 620, configured to obtain a transport layer protocol for transmitting the interactive control data based on the category;

[0104] Specifically, when the category corresponding to the interactive control data is head interactive data or body interactive data, the transport layer protocol selects the UDP protocol when transmitting these two types of data. Compared with the TCP protocol, the UDP protocol has high transmission efficiency and is more suitable for communications or broadcast communications that have high requirements for high-speed transmission and real-time performance.

[0105] When the category corresponding to the interactive control data is command data, the transport layer protocol selects the TCP protocol. TCP can provide reliable services. The data transmitted through the TCP connection is error-free, loss-free, non-duplicate, and arrives in order.

[0106] The transmission module 630 is used to transmit the interactive control data based on the transport layer protocol.

[0107] Specifically, after determining the transport layer protocol to be adopted, the interactive control data generated by the VR terminal is transmitted to the cloud; and the interactive control data generated by the cloud is transmitted to the VR terminal.

[0108] Optionally, a reconstruction module is also included, which is used to reconstruct the lost interactive control data based on the cached interactive control data according to the Kalman filter algorithm.

[0109] In this embodiment, the specific functions of the above-mentioned cloud VR interactive control data transmission device can refer to the corresponding description in the above-mentioned cloud VR interactive control data transmission method, and will not be repeated here.

[0110] Based on the above embodiments, the present invention further provides an intelligent terminal, whose principle block diagram can be shown as follows: Figure 6 As shown. The above-mentioned smart terminal includes a processor, a memory, a network interface and a display screen connected through a system bus. Among them, the processor of the smart terminal is used to provide computing and control capabilities. The memory of the smart terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a cloud VR interactive control data transmission program. The internal memory provides an environment for the operation of the operating system and the cloud VR interactive control data transmission program in the non-volatile storage medium. The network interface of the smart terminal is used to communicate with an external terminal through a network connection. When the cloud VR interactive control data transmission program is executed by the processor, the steps of any one of the above-mentioned cloud VR interactive control data transmission methods are implemented. The display screen of the smart terminal can be a liquid crystal display screen or an electronic ink display screen.

[0111] Those skilled in the art will understand that Figure 6 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the smart terminal to which the scheme of the present invention is applied. The specific smart terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0112] In one embodiment, a smart terminal is provided. The smart terminal includes a memory, a processor, and a cloud VR interactive control data transmission program stored in the memory and executable on the processor. When the cloud VR interactive control data transmission program is executed by the processor, the following operation instructions are performed:

[0113] receiving interactive control data;

[0114] Based on the VR terminal that generates the interactive control data, obtaining a category corresponding to the interactive control data;

[0115] Based on the category, obtaining a transport layer protocol for transmitting the interactive control data;

[0116] Based on the transport layer protocol, the interactive control data is transmitted.

[0117] An embodiment of the present invention further provides a computer-readable storage medium, on which a cloud VR interactive control data transmission program is stored. When the cloud VR interactive control data transmission program is executed by a processor, the steps of any one of the cloud VR interactive control data transmission methods provided in the embodiments of the present invention are implemented.

[0118] It should be understood that the sequence numbers of the steps in the above embodiments do not imply a sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0119] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0120] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0121] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0122] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / terminal equipment and method can be implemented in other ways. For example, the apparatus / terminal equipment embodiments described above are only illustrative, for example, the division of the above modules or units is only a logical function division, and in actual implementation, other division methods can be used, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0123] If the above-mentioned integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the above-mentioned computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0124] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features therein may be replaced by equivalents. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. Cloud VR interactive control data transmission method, characterized in that: include: receiving interactive control data; Acquire the category corresponding to the interactive control data; Based on the category, obtaining a transport layer protocol for transmitting the interactive control data; Based on the transport layer protocol, transmitting the interactive control data; If the category is head interaction data or body interaction data, the transmitting the interaction control data includes: Based on the received statistical information reflecting the network transmission status, the network transmission status is judged; if the network transmission status is network congestion, after receiving a set number of the interactive control data cumulatively, all the interactive control data are combined into a data packet and the data packet is transmitted; otherwise, the interactive control data is transmitted; The step of combining all the interactive control data into a data packet and transmitting the data packet further comprises: If the network transmission state is still network congestion when the set number of interactive control data groups are formed into data packets for transmission, the set number is increased geometrically until the network transmission state is improved; If the category is body interaction data and the length of the data packet is greater than the maximum transmission unit, after combining all the interaction control data into a data packet, the method further includes: discarding the data packet; sorting all the interactive control data in time to form a data sequence; discarding the interactive control data in the data sequence at intervals; and combining the data sequence after the interval discarding into a data packet; If the category is header interaction data, the UDP transmission protocol is used for transmission; Sorting the interactive control data according to time, and setting a unique serial number for each interactive control data according to the sorting result; If the category is header interaction data, after the cloud receives the interaction control data, the following further includes: Based on the serial number of the interactive control data, a judgment result is obtained according to the integrity judgment rule of the interactive control data; if the judgment result is that the interactive control data is lost, the serial number of the lost interactive control data is obtained, and a set number of interactive control data cached before the serial number is obtained, and based on the cached interactive control data, the lost interactive control data is reconstructed according to the Kalman filtering algorithm.

2. The cloud VR interactive control data transmission method according to claim 1, characterized in that: The obtaining, based on the category, a transport layer protocol for transmitting the interactive control data comprises: If the category is command data, the transport layer protocol is TCP protocol; If the category is head interaction data or body interaction data, the transport layer protocol is UDP protocol.

3. Cloud VR interactive control data transmission device, characterized in that: Applied to implement the cloud VR interactive control data transmission method according to any one of claims 1-2, the device comprises: A data receiving module, used for receiving interactive control data; A category determination module, used for obtaining the category corresponding to the interactive control data; A protocol determination module, used to obtain a transport layer protocol for transmitting the interactive control data based on the category; if the category is header interactive data, the UDP transport protocol is used for transmission; A transmission module, used for transmitting the interactive control data based on the transport layer protocol; Also included is a reconstruction module for reconstructing lost interactive control data based on the cached interactive control data according to a Kalman filter algorithm; If the category is header interaction data, after the cloud receives the interaction control data, the following further includes: Based on the serial number of the interactive control data, a judgment result is obtained according to the integrity judgment rule of the interactive control data; if the judgment result is that the interactive control data is lost, the serial number of the lost interactive control data is obtained, and a set number of interactive control data cached before the serial number is obtained, and based on the cached interactive control data, the lost interactive control data is reconstructed according to the Kalman filtering algorithm.

4. Intelligent terminal, characterized in that: The smart terminal includes a memory, a processor, and a cloud VR interactive control data transmission program stored in the memory and executable on the processor. When the cloud VR interactive control data transmission program is executed by the processor, the steps of the cloud VR interactive control data transmission method according to any one of claims 1 to 2 are implemented.

5. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a cloud VR interactive control data transmission program, and when the cloud VR interactive control data transmission program is executed by the processor, the steps of the cloud VR interactive control data transmission method according to any one of claims 1 to 2 are implemented.

Images