Data transmission method and device, electronic equipment, server and storage medium

Abstract

The application discloses a data transmission method and device, electronic equipment, a server and a storage medium, and belongs to the technical field of communication. The method comprises the following steps: sending header information of a message to a server, wherein the header information comprises first device information, and the header information is used for requesting the server to determine a service device for VR image data transmission with the electronic equipment based on the first device information; in the case that second device information sent by the server is received, establishing a connection with a target service device corresponding to the second device information according to the second device information; and transmitting main body information of the message to the target service device, wherein the main body information comprises position information and observation angle information of a current observation point of a user, and the main body information is used for requesting the target service device to feed back VR image data corresponding to the observation angle information according to the main body information.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of communication, and particularly relates to a data transmission method and device, electronic equipment, a server and a storage medium. BACKGROUND

[0002] With the development of science and technology, the demand for virtual reality (VR) technology is increasing in various industries. The server can transmit all video data of a VR surround image to the device end at one time in the manner of transmitting VR video data in full view, so that when the user needs to switch the video picture, the device end can complete the video data calculation and show the process of the VR surround image to the user.

[0003] However, since the amount of VR video data is large, when transmitting the VR video data in the above manner, not only the number of transmitted bits per unit time is increased, but also the transmission bandwidth is wasted, the transmission cost is increased, and the image transmission efficiency is low. SUMMARY

[0004] The embodiments of the present application provide a data transmission method and device, electronic equipment, a server and a storage medium, which can solve the problems of low VR video data transmission efficiency and high transmission cost in the related art.

[0005] In a first aspect, the embodiments of the present application provide a data transmission method applied to an electronic device, which can include:

[0006] sending header information of a message to a server, the header information including first device information, and the header information being used to request the server to determine a service device for VR image data transmission with the electronic device based on the first device information;

[0007] in a case where the second device information sent by the server is received, establishing a connection with a target service device corresponding to the second device information according to the second device information;

[0008] transmitting body information of the message to the target service device, wherein the body information includes position information and observation angle information of a current observation point of a user, and the body information is used to request the target service device to feed back VR image data corresponding to the observation angle information according to the body information.

[0009] In a second aspect, the embodiments of the present application provide a data transmission method applied to a server, which can include:

[0010] receiving header information of a message sent by an electronic device, the header information including first device information;

[0011] According to the first device information, a target service device for VR image data transmission with the electronic device is determined from a plurality of service devices of the service end;

[0012] Second device information of the target service device is sent to the electronic device, and the target device information is used to instruct the electronic device to establish a connection with the target service device corresponding to the second device information;

[0013] The subject information of the message transmitted by the electronic device is received, wherein the subject information includes position information and observation angle information of a current observation point of a user;

[0014] According to the subject information, the electronic device is fed back VR image data corresponding to the observation angle information.

[0015] In a third aspect, an embodiment of the present application provides a data transmission device applied to an electronic device, which can include:

[0016] The sending module is configured to send header information of a message to a service end, the header information including first device information, and the header information being used to request the service end to determine a service device for VR image data transmission with the electronic device based on the first device information;

[0017] The construction module is configured to, in a case where the second device information sent by the service end is received, establish a connection with a target service device corresponding to the second device information according to the second device information;

[0018] The transmission module is configured to transmit subject information of the message to the target service device, wherein the subject information includes position information and observation angle information of a current observation point of a user, and the subject information is used to request the target service device to feed back VR image data corresponding to the observation angle information according to the subject information.

[0019] In a fourth aspect, an embodiment of the present application provides a data transmission device applied to a service end, which can include:

[0020] The receiving module is configured to receive header information of a message sent by an electronic device, the header information including first device information;

[0021] The determination module is configured to determine a target service device for VR image data transmission with the electronic device from a plurality of service devices of the service end according to the first device information;

[0022] The sending module is configured to send second device information of the target service device to the electronic device, and the target device information is used to instruct the electronic device to establish a connection with the target service device corresponding to the second device information;

[0023] The receiving module is further configured to receive subject information of the message transmitted by the electronic device, wherein the subject information comprises position information and observation angle information of a current observation point of the user.

[0024] The feedback module is configured to feed back, to the electronic device, VR image data corresponding to the observation angle information according to the subject information.

[0025] In a fifth aspect, an embodiment of the present application provides an electronic device, which comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction is executed by the processor to implement the steps of the data transmission method shown in the first aspect.

[0026] In a sixth aspect, an embodiment of the present application provides a server, which comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction is executed by the processor to implement the steps of the data transmission method shown in the second aspect.

[0027] In a seventh aspect, an embodiment of the present application provides a readable storage medium, which stores a program or instruction, and the program or instruction is executed by a processor to implement the steps of the data transmission method shown in the first aspect or the steps of the data transmission method shown in the second aspect.

[0028] In an eighth aspect, an embodiment of the present application provides a chip, which comprises a processor and a display interface, the display interface is coupled with the processor, and the processor is configured to run a program or instruction to implement the steps of the data transmission method shown in the first aspect or the steps of the data transmission method shown in the second aspect.

[0029] In a ninth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium, and the program product is executed by at least one processor to implement the steps of the data transmission method shown in the first aspect or the steps of the data transmission method shown in the second aspect.

[0030] In the embodiment of the present application, the electronic device can send the header information of the message to the server, the header information includes the first device information, and the header information is used to request the server to determine the service device for VR image data transmission with the electronic device based on the first device information. In this way, in the case that the second device information sent by the server is received, the target service device corresponding to the second device information is connected according to the second device information, and then the main body information of the message is transmitted to the target service device; wherein the main body information includes the position information and the observation angle information of the current observation point of the user, and the main body information is used to request the target service device to feed back the VR image data corresponding to the observation angle information according to the main body information. Therefore, the service device in the server that can perform VR image data transmission with the electronic device can be determined through the header information in the message, so as to prevent the situation that after all the video data of the VR surround image is transmitted to the device end at one time, there is no service device that can receive the video data, resulting in data retransmission and increasing the transmission cost. In addition, based on the position identification coordinates and the angle vector information of the current user angle in the main body information of the message, the video data changed in the user visual field caused by the user turning his head can be requested to be transmitted by the service device, and the VR video data that should be transmitted to the electronic device at one time is adjusted to be transmitted in batches according to the real-time change of the user visual field. In this way, the transmission process of the entire image data is simplified, the number of bits transmitted in the same time period is reduced, the transmission delay is shortened, and the transmission bandwidth requirement is low, so as to avoid wasting the transmission bandwidth and improve the image transmission efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 A schematic diagram of a data transmission architecture provided by the embodiment of the present application;

[0032] Figure 2 A flowchart of a data transmission method based on an electronic device provided by the embodiment of the present application;

[0033] Figure 3 A structure diagram of a message sent by an electronic device provided by the embodiment of the present application;

[0034] Figure 4 A flowchart of data encapsulation of a data transmission method provided by the embodiment of the present application;

[0035] Figure 5 A flowchart of a data transmission method based on a server provided by the embodiment of the present application;

[0036] Figure 6 A structure diagram of a message sent by a server provided by the embodiment of the present application;

[0037] Figure 7An interaction schematic diagram of a data transmission method provided by an embodiment of the present application;

[0038] Figure 8 A structural schematic diagram of a data transmission device based on an electronic device provided by an embodiment of the present application;

[0039] Figure 9 A structural schematic diagram of a data transmission device based on a server provided by an embodiment of the present application;

[0040] Figure 10 A structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0041] Figure 11 A hardware structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0043] The terms “first”, “second”, and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by “first”, “second”, and the like are generally of a kind and are not limited in number, for example, the first object can be one or more. In addition, “and / or” in the specification and claims indicates at least one of the connected objects, and the character “ / ” generally indicates that the front and rear associated objects are in an “or” relationship.

[0044] To solve the problems in the related art, the data transmission method provided by the embodiments of the present application will be described in detail below in combination with the drawings Figures 1 to 7

[0045] First, the data transmission architecture provided by the embodiments of the present application will be described in detail. Figure 1

[0046] As shown in FIG. 1, the data transmission architecture provided by the embodiments of the present application includes a data transmission device based on an electronic device, a data transmission device based on a server, and an electronic device. Figure 1 ​​As shown, the data transmission architecture 10 can include a 5th Generation Mobile Communication Technology (5G) network hardware device layer 101 (5G hardware transmission system) and a resource transmission network layer 102 (software transmission system).

[0047] The 5G network hardware device layer 101 includes 5G network hardware devices, 5G connection devices, 5G indoor base stations, bridge units, base band signal processing units, core network parts, and servers. The 5G network hardware devices can be electronic devices, wearable devices, and the like. The wearable devices can be VR smart glasses, VR smart headsets, and the like. The 5G connection devices can be used to connect local devices (such as mobile phones, tablets, and computers) to the Internet. The 5G connection devices can be 5G Customer Premise Equipment (CPE). The servers can include multiple service devices.

[0048] It should be noted that the 5G network hardware devices described above can be used for VR image data transmission. With the help of the Enhanced Mobile Broadband (eMBB) and Ultra-Reliable Low-Latency Communications (uRLLC) features of the 5G network, a stable and fast transmission channel is provided for VR image transmission.

[0049] The software transmission layer 102 can include a VR image transmission system of electronic devices and servers based on the Transmission Control Protocol / Internet Protocol (TCP / IP). The system is connected based on the IPV6 address of the electronic devices and servers, encapsulates the VR image through the message system, and transmits the VR image through the 5G network hardware device layer 101 based on the 5G network.

[0050] Based on this, the software transmission layer 102 can encode and package the data to be transmitted, such as the position information of the current observation point of the user, the observation angle information, and the first image data corresponding to the position information of the current observation point of the user, through the packaging system 1 in the electronic device, and transmit the data to the server through the 5G network hardware device layer 101 using the TCP / IP protocol 1. The packaging system 2 in the server decodes and restores the received data into VR image data, and obtains the VR data that needs to be displayed by the electronic device according to the position information of the current observation point of the user and the observation angle information, so as to send the VR video data corresponding to the observation angle information to the electronic device through the network hardware device layer 101 through the TCP / IP protocol 2 again.

[0051] Therefore, the process can complete the VR image encoding and decoding and transmission process based on the network hardware device layer 101 and the resource transmission network layer 102. Due to the large bandwidth and low latency characteristics of the 5G network, the VR image can be quickly and stably transmitted. The VR image is transmitted through the IPV6 address, and the communication connection is completed based on the socket under the TCP / IP protocol, so as to complete the low-latency and high-speed transmission of the VR image based on the 5G network. Compared with the physical medium storage and network transmission method of the VR image, the data transmission process utilizes the high speed and low latency characteristics of the 5G network to ensure the high speed and stability of the transmission environment, so that the VR image with a large amount of data can be transmitted to a remote device or a server in a complete, fast and real-time manner. The user can use the 5G network hardware device more smoothly and comfortably, and no longer feel dizzy. At the same time, the user angle recognition and device recognition message are provided for identification and matching of the terminal device, and the security of device interconnection is increased.

[0052] In addition, the data transmission architecture in the embodiment of the present application can be applied not only to the scene of transmitting VR image data, but also to the transmission of augmented reality (AR) image data, real-time high-definition image, or the scene of AR remote live broadcast based on the 5G network by adjusting and modifying the transmission software based on the above-mentioned data transmission architecture.

[0053] Based on the above-mentioned data transmission architecture and its application scene, combined with Figure 2 A data transmission method based on an electronic device provided by the embodiment of the present application is described in detail.

[0054] Figure 2 A flowchart of a data transmission method based on an electronic device provided by the embodiment of the present application.

[0055] As Figure 2As shown, the data transmission method provided by the embodiments of the present application can include the following steps:

[0056] In step 210, the header information of the message is sent to the server, the header information includes the first device information, and the header information is used to request the server to determine the service device for VR image data transmission with the electronic device based on the first device information; in step 220, in the case that the second device information sent by the server is received, a connection is established with the target service device corresponding to the second device information according to the second device information; in step 230, the main body information of the message is transmitted to the target service device; wherein the main body information includes the position information and the observation angle information of the current observation point of the user, and the main body information is used to request the target service device to feed back the VR image data corresponding to the observation angle information according to the main body information.

[0057] In this way, the service device in the server that can perform VR image data transmission with the electronic device can be determined through the header information in the message first, so that the situation that all video data of the VR surround image is transmitted to the device end at one time, and there is no service device that can receive the video data, resulting in data retransmission and increasing transmission cost, etc. can be prevented. In addition, based on the position recognition coordinates and the angle vector information of the current user angle in the main body information of the message, the video data changed in the user visual field caused by the user turning his head can be requested to be transmitted by the service device respectively, and the VR video data that should be transmitted to the electronic device at one time is adjusted to be transmitted in batches according to the real-time change of the user visual field. In this way, the transmission process of the entire image data is simplified, the number of bits transmitted in the same time period is reduced, the transmission delay is shortened, and the transmission bandwidth requirement is low, so that the waste of transmission bandwidth can be avoided, and the image transmission efficiency is improved.

[0058] The above steps will be described in detail as follows.

[0059] Firstly, in one or more possible embodiments, the data transmission method can further include the following steps before step 210.

[0060] Obtaining the projection position information of the user's eyes projected on the first screen of the electronic device;

[0061] Determining the projection position information as the position information of the current observation point of the user;

[0062] Taking the position information of the current observation point as a search starting point, the direction in which the user's eyes can move searched in the same preset field of view angle range as the current observation point is determined as the moving direction;

[0063] Determining the three-dimensional coordinates corresponding to the displacement of the user's eyes in the moving direction as the three-dimensional phase information.

[0064] In addition, as shown in Figure 3 the header information in the embodiments of the present application can include an electronic device address (Client Address), a server address (Server Address) and a time point (Time). In addition, the first device information (Device info) can include the device address of the electronic device, the address of a preset service device corresponding to the electronic device, and the device status information of the electronic device.

[0065] Here, it should be noted that the byte allocation of the header information can be equal, such as 4 bytes.

[0066] Then, referring to step 230, the main information in the embodiments of the present application can include device status information (Device Status info), observation point position information (Observation Point info), observation viewing angle information (Viewing Orientation info), control information (Control info), image information (Graphic info) and fault-tolerant information (Fault-Tolerant info), as shown in Figure 3 . The device status information includes 0-deactive, 1-active, and 2-sleep, a total of 3 state information; the observation point position information includes X, Y, Z axis coordinate information; the observation viewing angle information can include three-dimensional phase information, i.e. X, Y, Z axis coordinate information, wherein the X axis coordinate information (X Orientation), the Y axis coordinate information (Y Orientation), and the Z axis coordinate information (Z Orientation); the control information (Control info) includes transmission protocol mode (Protocol info), connection information (Connection info), transmission rate (Rate info) and file information (Document info); the image information (Graphic info) includes slice image size (Graphic Size), slice number (Slice Num), and receive number (Receive Num); the fault-tolerant information (Fault-Tolerant info) includes device status information (Status info) and fault-tolerant information (Fault-Tolerant), wherein the fault-tolerant information includes backup solution (Backup Solution) and other situations (Others). The data size of each part of the message information is defined as 4 bytes in Figure 3 , so as to facilitate unified encapsulation and transmission. In addition, Figure 3A mode (mode) corresponding to a transmission protocol mode (Protocol info) shown in the middle, which can be TCP mode and UDP mode represented by 0; start information (start info) corresponding to connection information (Connection info), which can be start transmission represented by 0, keep transmission represented by 1, and stop transmission represented by 0; frame rate (frame rate) corresponding to transmission rate (Rate info); definition size (definition size) corresponding to file information (Document info); and height (Height), length (length) and width (width) corresponding to slice image size (Graphic Size); and device status (Status) corresponding to device status information (Status info).

[0067] In one or more possible embodiments, before step 230, the data transmission method can further include:

[0068] According to the position information and the observation angle information of the current observation point, a target transmission protocol is selected from a plurality of preset transmission protocols, the target transmission protocol being a Transmission Control Protocol (TCP) or a User Datagram Protocol (UDP), wherein the observation angle information includes a moving direction within a same preset field of view angle range as the position information of the current observation point and three-dimensional phase information in the moving direction;

[0069] According to the target transmission protocol, a data transmission channel is established.

[0070] Based on this, step 230 can specifically include:

[0071] Step 2301, transmitting the main body information of the message to the target service device through the data transmission channel.

[0072] Specifically, in the case where the main body information further includes first image data corresponding to the position information of the current observation point, the first image data can include transmission protocol, image segment slice quantity, transmission rate and other parameters of the image data; based on this, step 2301 can specifically include:

[0073] Step 23011, obtaining a first data attribute of the main body information, the first data attribute including a data size of the first image data and a transmission rate of the data transmission channel;

[0074] Step 23012: Based on the association between preset data attributes and preset slicing conditions, obtain the first slicing condition corresponding to the first data attribute;

[0075] Step 23013: According to the first slicing conditions, slice the first image data to obtain N image segments, where N is a positive integer greater than 1;

[0076] Step 23014: Transmit N image segments to the target service device through the data transmission channel.

[0077] Furthermore, if the location information of the current observation point includes the three-dimensional coordinate information of the current observation point, that is, when the head rotation viewpoint changes, the electronic device requests the viewpoint file corresponding to the new viewpoint from the server. Specifically, step 23014 may include:

[0078] The three-dimensional coordinate information of the current observation point, the observation view information, and N image segments are sequentially encapsulated to obtain the first encapsulated data;

[0079] The first encapsulated data is transmitted to the target service device through the data transmission channel, in accordance with the transmission control protocol and the Internet protocol address corresponding to the transmission control protocol.

[0080] like Figure 4 As shown, after establishing the transmission channel, image data transmission begins. Step 40 involves detecting the file size, the set transmission rate, the length, width, and height of the image data, the number of slices (N), the 3D coordinates of the current observation point, and the observation viewpoint. Steps 41-44 encapsulate the 3D coordinates of the current observation point, the observation viewpoint, and N image slices. Step 45 obtains the first encapsulated data based on the encapsulation in steps 41-44. Then, step 46 is executed to transfer the file via TCP or UDP, so that the data can be decapsulated and restored after reaching the target service device.

[0081] Secondly, combining Figure 5 This application provides a detailed description of a server-based data transmission method.

[0082] Figure 5 A flowchart illustrating a server-based data transmission method provided in this application embodiment.

[0083] like Figure 5 As shown, the data transmission method provided in this application embodiment may include the following steps:

[0084] Step 510, receiving the header information of the message sent by the electronic device, the header information including the first device information; step 520, determining the target service device for VR image data transmission with the electronic device from the plurality of service devices of the server according to the first device information; step 530, sending the second device information of the target service device to the electronic device, the target device information being used to instruct the electronic device to establish a connection with the target service device corresponding to the second device information; step 540, receiving the body information of the message transmitted by the electronic device, wherein the body information includes the position information and the observation angle information of the current observation point; step 550, feeding back the VR image data corresponding to the observation angle information to the electronic device according to the body information.

[0085] The above steps are described in detail as follows.

[0086] Firstly, in relation to step 540, the observation angle information involved in this step can include the moving direction within the same preset field of view angle range as the position information of the current observation point and the three-dimensional phase information in the moving direction.

[0087] Next, in relation to step 550, in one or more possible embodiments, this step 550 can specifically include:

[0088] Step 5501, obtaining the VR image data corresponding to the body information according to the body information;

[0089] Step 5502, obtaining the second data attribute of the VR image data, the second data attribute including the data size of the VR image data and the transmission rate of the data transmission channel, the data transmission channel being the channel for the target service device to transmit the VR image data;

[0090] Step 5503, obtaining the second slicing condition corresponding to the second data attribute according to the association relationship between the preset data attribute and the preset slicing condition;

[0091] Step 5504, slicing the VR image data according to the second slicing condition to obtain M image segments, M being a positive integer greater than 1;

[0092] Step 5505, transmitting the M image segments to the electronic device through the data transmission channel.

[0093] Further, before this step 5505, the data transmission method can further include:

[0094] sequentially packaging the M image segments to obtain second packaging data;

[0095] Based on this, the step 5505 can specifically include:

[0096] The second encapsulated data is transmitted to the electronic device through a data transmission channel according to a transmission control protocol corresponding to the electronic device and an Internet protocol address corresponding to the transmission control protocol.

[0097] Here, as shown in Figure 6 , the main information of the message can include device state information (Device State info), slice image information (Document info), i.e., information of M image segments, state information (State info), fault-tolerant information (Fault-Tolerant), and slice information (Slice info). The image information (Graphic document) mainly refers to the size of the slice image byte stream. The state information (Status info) mainly includes: state information (Status), a backup solution (Backup Solution), and other modes (Error mode). The transmission information mainly refers to the slice information (Slice info), and the number information is divided into high information and low information. The data sizes of each part of the message information, such as 4 bytes and 84 bytes, are defined in Figure 6 , so as to facilitate unified encapsulation and transmission.

[0098] In addition, as shown in Figure 6 , the bits stream corresponding to the device state information (Device State info), the high part corresponding to the fault-tolerant information (Fault-Tolerant), and the low part corresponding to the state information (State info).

[0099] Based on the above, the data transmission method provided by the embodiments of the present application further provides an interactive process of data transmission between an electronic device and a server, which is specifically described in combination with Figure 7 .

[0100] As shown in Figure 7 , the message includes a message header (Message Header) and a message body (Message Body).As shown, taking the VR smart glasses as an example, the VR smart glasses are in a shutdown state before running, start to send header information to identify a service device after starting, establish a connection channel after identifying a target service device, and select a corresponding process according to state information (0-deactive, 1-active, 2-sleep) of the service device. After the service device is activated, the VR smart glasses collect position information and observation angle information of a current observation point of a user, and after determining a TCP / UDP protocol, start a handshake protocol to establish a transmission channel, and transmit first image data, the position information and the observation angle information of the current observation point of the user through the transmission channel. After the service device receives corresponding data, the image picture is processed and VR image data corresponding to the observation angle information is sent to the VR smart glasses until the transmission is completed. In addition, the VR smart glasses are started at the same time, and the state and fault tolerance information of the VR smart glasses are detected synchronously. Different device state information is respectively directed to: normal operation, shutdown, restart, start backup scheme and other schemes.

[0101] Therefore, the service device in the service end that can perform VR image data transmission with the electronic device can be determined through the header information in the message, so that the situation that all video data of the VR surround image is transmitted to the device end at one time, and there is no service device that can receive the video data, resulting in data retransmission and increasing transmission cost, and the like, can be prevented. In addition, based on the position identification coordinates and the angle vector information of the current user angle in the main body information of the message, the video data changed in the user field of view caused by the user turning his head can be requested to be transmitted by the service device, the VR video data that should be transmitted to the electronic device at one time is adjusted to be transmitted in batches according to the real-time change of the user field of view, so that the transmission process of the entire image data is simplified, the number of bits transmitted in the same time period is reduced, the transmission delay is shortened, and the transmission bandwidth requirement is low, so that the waste of transmission bandwidth can be avoided, and the image transmission efficiency is improved.

[0102] The data transmission method provided in the embodiment of the application can be executed by the data transmission device. In the embodiment of the application, the data transmission method executed by the data transmission device is taken as an example to describe the data transmission device provided in the embodiment of the application.

[0103] Based on the same inventive concept, the application also provides a data transmission device based on an electronic device. The data transmission device based on an electronic device is described in detail below with reference to the embodiments of the application. Figure 8

[0104] Figure 8 A structure diagram of the data transmission device based on an electronic device provided in the embodiment of the application.

[0105] As Figure 8 ​As shown, the data transmission apparatus 80 is applied to an electronic device, and specifically can include:

[0106] The sending module 801 is configured to send header information of the message to the server, the header information including the first device information, and the header information being used to request the server to determine the service device for the VR image data transmission with the electronic device based on the first device information.

[0107] The construction module 802 is configured to, in a case where the second device information sent by the server is received, establish a connection with a target service device corresponding to the second device information according to the second device information.

[0108] The transmission module 803 is configured to transmit body information of the message to the target service device, wherein the body information includes position information and observation angle information of the current observation point, and the body information is used to request the target service device to feed back VR image data corresponding to the observation angle information according to the body information.

[0109] The data transmission apparatus 80 will be described in detail as follows.

[0110] In one or more possible embodiments, the data transmission apparatus 80 in the embodiment of the present application can further include a screening module and a construction module, wherein,

[0111] The screening module is configured to screen a target transmission protocol from a plurality of preset transmission protocols according to the position information and the observation angle information of the current observation point, the target transmission protocol being a transmission control protocol or a user datagram protocol, and the observation angle information including a moving direction within a same preset field of view angle range as the position information of the current observation point and three-dimensional phase information in the moving direction.

[0112] The construction module is configured to establish a data transmission channel according to the target transmission protocol.

[0113] The transmission module 803 can also be configured to transmit the body information of the message to the target service device through the data transmission channel.

[0114] In another or more possible embodiments, the data transmission apparatus 80 in the embodiment of the present application can further include an acquisition module and a slicing module, wherein,

[0115] The acquisition module is configured to, in a case where the body information further includes first image data corresponding to the position information of the current observation point, acquire a first data attribute of the body information, the first data attribute including a data size of the first image data and a transmission rate of the data transmission channel.

[0116] The acquisition module can also be configured to acquire a first slicing condition corresponding to the first data attribute according to an association between the preset data attribute and the preset slicing condition.

[0117] The slicing module is configured to slice the first image data according to the first slicing condition to obtain N image segments, where N is a positive integer greater than 1.

[0118] The transmission module 803 can also be configured to transmit the N image segments to a target service device through a data transmission channel.

[0119] In yet another or more possible embodiments, the data transmission device 80 in the embodiments of the present application can further include an encapsulation module, wherein

[0120] The encapsulation module is configured to sequentially encapsulate the three-dimensional coordinate information of the current observation point, the observation angle information, and the N image segments to obtain first encapsulation data when the position information of the current observation point includes the three-dimensional coordinate information of the current observation point.

[0121] The transmission module 803 can also be configured to transmit the first encapsulation data to the target service device through a data transmission channel according to a transmission control protocol and an Internet protocol address corresponding to the transmission control protocol.

[0122] Based on the same inventive concept, the present application also provides a service-side data transmission device. Details are described below with reference to the specific embodiments. Figure 9

[0123] Figure 9 A structure diagram of a service-side data transmission device provided by the embodiments of the present application.

[0124] As shown in Figure 9 , the data transmission device 90 is applied to a service side, and the data transmission device 90 can specifically include:

[0125] The receiving module 901 receives header information of a message sent by an electronic device, and the header information includes first device information.

[0126] The determination module 902 is configured to determine a target service device for VR image data transmission with the electronic device from a plurality of service devices of the service side according to the first device information.

[0127] The sending module 903 is configured to send second device information of the target service device to the electronic device, and the target device information is used to instruct the electronic device to establish a connection with the target service device corresponding to the second device information.

[0128] ​The receiving module 901 is further configured to receive subject information of a packet transmitted by the electronic device, wherein the subject information comprises position information and observation angle information of a current observation point of a user.

[0129] The feedback module 904 is configured to feed back, to the electronic device, VR image data corresponding to the observation angle information according to the subject information.

[0130] The data transmission apparatus 90 is described in detail as follows:

[0131] In one or more possible embodiments, the data transmission apparatus 90 can further comprise an acquisition module, a slicing module and a first transmission module, wherein the acquisition module is configured to acquire, according to the subject information, VR image data corresponding to the subject information.

[0132] The acquisition module is configured to acquire, according to the subject information, VR image data corresponding to the subject information.

[0133] The acquisition module is further configured to acquire a second data attribute of the VR image data, wherein the second data attribute comprises a data size of the VR image data and a transmission rate of a data transmission channel, and the data transmission channel is a channel through which the target service device transmits the VR image data.

[0134] The acquisition module is further configured to acquire, according to an association between the preset data attribute and a preset slicing condition, a second slicing condition corresponding to the second data attribute.

[0135] The slicing module is configured to slice the VR image data according to the second slicing condition to obtain M image segments, wherein M is a positive integer greater than 1.

[0136] The first transmission module is configured to transmit the M image segments to the electronic device through the data transmission channel.

[0137] In another or more possible embodiments, the data transmission apparatus 90 can further comprise an encapsulation module and a second transmission module, wherein the encapsulation module is configured to sequentially encapsulate the M image segments to obtain second encapsulation data.

[0138] The encapsulation module is configured to sequentially encapsulate the M image segments to obtain second encapsulation data.

[0139] The second transmission module is configured to transmit the second encapsulation data to the electronic device through the data transmission channel according to a transmission control protocol corresponding to the electronic device and an Internet protocol address corresponding to the transmission control protocol.

[0140] The data transmission apparatus in the embodiments of the present application can be an electronic device or a component in the electronic device, for example, an integrated circuit or a chip. The electronic device can be a terminal or other device than the terminal. For example, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and can also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a cashier machine, or a self-service machine, and the like, and the embodiments of the present application are not limited in this regard.

[0141] The data transmission apparatus in the embodiments of the present application can be an apparatus with an operating system. The operating system can be an Android operating system, an IOS operating system, or other possible operating system, and the embodiments of the present application are not limited in this regard.

[0142] The device coordination apparatus provided by the embodiments of the present application can achieve Figures 1 to 7 The data transmission method embodiments shown in the embodiments of the present application achieve the same technical effects as the processes, and thus the details are not repeated here.

[0143] Based on this, the device coordination apparatus provided by the embodiments of the present application can be used to enable the electronic device to send the header information of the message to the server, the header information including the first device information, the header information being used to request the server to determine the service device for VR image data transmission with the electronic device based on the first device information. In this way, in the case where the second device information sent by the server is received, the target service device corresponding to the second device information is connected according to the second device information, and then the main body information of the message is transmitted to the target service device. The main body information includes the position information and the observation angle information of the current observation point of the user, and the main body information is used to request the target service device to feed back the VR image data corresponding to the observation angle information according to the main body information. In this way, the service device in the server that can perform VR image data transmission with the electronic device can be determined through the header information in the message, so as to prevent the situation that after all the video data of the VR surround image is transmitted to the device end at one time, there is no service device that can receive the video data, resulting in data retransmission and increase of transmission cost. In addition, based on the position identification coordinates and the angle vector information of the current user angle in the main body information of the message, the video data changed in the user visual field caused by the user turning his head can be requested to be transmitted by the service device, the VR video data that should be transmitted to the electronic device at one time is adjusted to be transmitted in batches according to the real-time change of the user visual field. In this way, the transmission process of the entire image data is simplified, the number of bits transmitted in the same time period is reduced, the transmission delay is shortened, and the transmission bandwidth requirement is low, so as to avoid waste of the transmission bandwidth and improve the image transmission efficiency.

[0144] Optionally, as shown in Figure 10 The embodiments of the present application also provide an electronic device 100, which includes a processor 1001 and a memory 1002. The memory 1002 stores programs or instructions executable on the processor 1001. When the programs or instructions are executed by the processor 1001, each step of the above data transmission method embodiment is implemented, and the same technical effect is achieved. To avoid repetition, details are not described here.

[0145] It should be noted that the electronic device in the embodiments of the present application includes the mobile electronic device and the non-mobile electronic device described above.

[0146] Figure 11 A hardware structure schematic diagram of an electronic device provided by the embodiments of the present application is provided.

[0147] The electronic device 1100 includes but is not limited to the following components: a radio frequency unit 1101, a network module 1102, an audio output unit 1103, an input unit 1104, a sensor 1105, a display unit 1106, a user input unit 1107, an interface unit 1108, a memory 1109, a processor 1110, etc.

[0148] Those skilled in the art can understand that the electronic device 1100 can also include a power supply (such as a battery) for supplying power to various components, and the power supply can be logically connected to the processor 1110 through a power management system, so that the power management system can realize functions such as management of charging, discharging, and power consumption management. Figure 11 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than those shown, or combine certain components, or different component arrangements, which are not described here.

[0149] In the embodiment of the present application, the network module 1102 is configured to send header information of a message to a server, the header information including first device information, and the header information being used to request the server to determine a service device for VR image data transmission with the electronic device based on the first device information. The processor 1110 is configured to, in a case where second device information sent by the server is received, establish a connection with a target service device corresponding to the second device information according to the second device information. The network module 1102 is configured to transmit body information of the message to the target service device; wherein the body information includes position information and observation angle information of a current observation point, and the body information is used to request the target service device to feed back VR image data corresponding to the observation angle information according to the body information.

[0150] The electronic device 1100 will be described in detail as follows:

[0151] In one or more possible embodiments, the processor 1110 in the embodiment of the present application can also be configured to filter a target transmission protocol from a plurality of preset transmission protocols according to the position information and the observation angle information of the current observation point, the target transmission protocol being a transmission control protocol or a user datagram protocol, wherein the observation angle information includes a moving direction within a same preset field of view angle range as the position information of the current observation point and three-dimensional phase information in the moving direction; and establish a data transmission channel according to the target transmission protocol. The network module 1102 can also be configured to transmit the body information of the message to the target service device through the data transmission channel.

[0152] In another or more possible embodiments, the processor 1110 can also be configured to, in a case where the body information further includes first image data corresponding to the position information of the current observation point, obtain a first data attribute of the body information, the first data attribute including a data size of the first image data and a transmission rate of the data transmission channel.

[0153] The processor 1110 can also be configured to obtain a first slicing condition corresponding to the first data attribute according to an association relationship between a preset data attribute and a preset slicing condition.

[0154] The processor 1110 is further configured to slice the first image data according to a first slice condition to obtain N image segments, N being a positive integer greater than 1.

[0155] The network module 1102 is further configured to transmit the N image segments to the target service device through a data transmission channel.

[0156] In yet another or more possible embodiments, the processor 1110 is further configured to, when the position information of the current observation point comprises three-dimensional coordinate information of the current observation point, sequentially encapsulate the three-dimensional coordinate information of the current observation point, the observation visual angle information and the N image segments to obtain first encapsulation data.

[0157] The network module 1102 is further configured to transmit the first encapsulation data to the target service device through a data transmission channel according to a transmission control protocol and an Internet protocol address corresponding to the transmission control protocol.

[0158] It should be understood that the input unit 1104 can include a graphics processor (GPU) 11041 and a microphone 11042. The graphics processor 11041 processes image data of a still image or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1106 can include a display panel, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1107 includes at least one of a touch panel 11071 and other input devices 11072. The touch panel 11071 is also called a touch screen. The touch panel 11071 can include two parts of a touch detection device and a touch display. The other input devices 11072 can include, but are not limited to, a physical keyboard, function keys (such as volume display keys, on-off keys, etc.), trackballs, mice, joysticks, etc., which will not be described here.

[0159] The memory 1109 can be used to store software programs and various data, and can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 1109 can include a volatile memory or a non-volatile memory, or the memory 1109 can include both volatile and non-volatile memories. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 1109 in the embodiments of the present application includes but is not limited to these and any other suitable types of memories.

[0160] The processor 1110 can include one or more processing units; optionally, the processor 1110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes a wireless display signal, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 1110.

[0161] The embodiments of the present application also provide a readable storage medium, and the readable storage medium stores programs or instructions, the programs or instructions are executed by a processor to realize various processes of the above-mentioned data transmission method embodiments and achieve the same technical effects. To avoid repetition, details are not described here.

[0162] The processor is a processor in the electronic device in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disc, or an optical disc, etc.

[0163] In addition, the embodiment of the present application further provides a chip, which comprises a processor and a display interface. The display interface is coupled with the processor. The processor is used to run programs or instructions, to realize various processes of the above data transmission method embodiments, and to achieve the same technical effects. To avoid repetition, details are not described herein.

[0164] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system-level chip, a system chip, a chip system, or a system-on-chip chip, etc.

[0165] The embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to realize various processes of the above data transmission method embodiments, and to achieve the same technical effects. To avoid repetition, details are not described herein.

[0166] It should be noted that in this document, the term "comprise", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0167] In addition, it should be noted that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, but can also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved, for example, the described method can be performed in an order different from the described order, and various steps can also be added, omitted or combined. In addition, the features described with reference to some examples can be combined in other examples.

[0168] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned example methods can be realized by means of software and a necessary general hardware platform, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product in essence or in the form of a part that contributes to the prior art, which is stored in a storage medium (such as a ROM / RAM, a magnetic disc, an optical disc), and includes a plurality of instructions for causing a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present application.

[0169] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative, not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

Claims

1. A data transmission method, characterized in that, Applied to electronic devices, including: The header information of the message sent to the server includes first device information. The header information is used to request the server to determine the service device that performs VR image data transmission with the electronic device based on the first device information. The first device information includes the device address of the electronic device, the address of the preset service device corresponding to the electronic device, and the device status information of the electronic device. Upon receiving the second device information sent by the server, a connection is established with the target service device corresponding to the second device information based on the second device information. The method transmits the main body information of the message to the target service device; wherein, the main body information includes the location information of the user's current observation point and the observation perspective information, the observation perspective information includes the movement direction within the same preset field of view as the location information of the current observation point and the three-dimensional phase information in the movement direction, the main body information is used to request the target service device to feed back VR image data corresponding to the observation perspective information based on the main body information; wherein, the method further includes: obtaining the projection position information of the user's eyes projected onto the first screen of the electronic device; determining the projection position information as the location information of the current observation point; using the location information of the current observation point as the search starting point, determining the movement direction of the user's eyes within the same preset field of view as the movement direction; The main information also includes first image data corresponding to the location information of the current observation point; the transmission of the main information of the message to the target service device includes: obtaining a first data attribute of the main information, the first data attribute including the data size of the first image data and the transmission rate of the data transmission channel; obtaining a first slicing condition corresponding to the first data attribute according to the association between preset data attributes and preset slicing conditions; slicing the first image data according to the first slicing conditions to obtain N image segments, where N is a positive integer greater than 1; and transmitting the N image segments to the target service device through the data transmission channel.

2. The method according to claim 1, characterized in that, Before transmitting the body information of the message to the target service device, the method further includes: Based on the location information and observation angle information of the current observation point, a target transmission protocol is selected from multiple preset transmission protocols. The target transmission protocol is either the Transmission Control Protocol or the User Datagram Protocol. Establish a data transmission channel according to the target transmission protocol; The transmission of the message body information to the target service device includes: The main information of the message is transmitted to the target service device through the data transmission channel.

3. The method according to claim 1, characterized in that, The location information of the current observation point includes the three-dimensional coordinate information of the current observation point; the transmission of the main body information of the message to the target service device through the data transmission channel includes: The three-dimensional coordinate information of the current observation point, the observation viewpoint information, and the N image segments are sequentially encapsulated to obtain the first encapsulated data; The first encapsulated data is transmitted to the target service device through the data transmission channel, in accordance with the transmission control protocol and the Internet protocol address corresponding to the transmission control protocol.

4. A data transmission method, characterized in that, Applied to the server side, including: The header information of a message sent by an electronic device is received. The header information includes first device information, which includes the device address of the electronic device, the address of a preset service device corresponding to the electronic device, and the device status information of the electronic device. Based on the first device information, a target service device for transmitting VR image data with the electronic device is determined from among multiple service devices of the server. Send the second device information of the target service device to the electronic device, wherein the second device information is used to instruct the electronic device to establish a connection with the target service device corresponding to the second device information; The system receives the main body information of the message transmitted by the electronic device, wherein the main body information includes the location information of the user's current observation point and the observation angle information; the observation angle information includes the movement direction within the same preset field of view as the location information of the current observation point and the three-dimensional phase information in the movement direction, wherein the location information of the current observation point is determined by the projection position information of the user's eyes projected onto the first screen of the electronic device; the movement direction is determined by the direction in which the user's eyes can move, searched from the location information of the current observation point as the search starting point within the same preset field of view as the current observation point; the main body information also includes the position information of the user's current observation point and the observation angle information in the first screen of the electronic device. The first image data corresponding to the location information of the current observation point; the main information of receiving the message transmitted by the electronic device includes: receiving N image segments transmitted by the electronic device through a data transmission channel, wherein the N image segments are determined by the following steps: obtaining a first data attribute of the main information, wherein the first data attribute includes the data size of the first image data and the transmission rate of the data transmission channel; obtaining a first slicing condition corresponding to the first data attribute according to the correlation between preset data attributes and preset slicing conditions; slicing the first image data according to the first slicing conditions to obtain the N image segments, where N is a positive integer greater than 1; Based on the subject information, VR image data corresponding to the observation perspective information is fed back to the electronic device.

5. The method according to claim 4, characterized in that, The step of feeding back VR image data corresponding to the observation perspective information to the electronic device based on the subject information includes: Based on the subject information, obtain VR image data corresponding to the subject information; Obtain a second data attribute of the VR image data, the second data attribute including the data size of the VR image data and the transmission rate of the data transmission channel, the data transmission channel being the channel through which the target service device transmits the VR image data; Based on the association between preset data attributes and preset slicing conditions, obtain the second slicing condition corresponding to the second data attribute; According to the second slicing condition, the VR image data is sliced ​​to obtain M image segments, where M is a positive integer greater than 1; The M image segments are transmitted to the electronic device through the data transmission channel.

6. The method according to claim 5, characterized in that, Before transmitting the M image segments to the electronic device through the data transmission channel, the method further includes: The M image segments are sequentially encapsulated to obtain the second encapsulated data; The transmission of the M image segments to the electronic device via the data transmission channel includes: The second encapsulated data is transmitted to the electronic device through the data transmission channel, according to the transmission control protocol corresponding to the electronic device and the Internet protocol address corresponding to the transmission control protocol.

7. A data transmission device, characterized in that, Applied to electronic devices, including: The sending module is used to send the header information of a message to the server. The header information includes first device information. The header information is used to request the server to determine the service device that performs VR image data transmission with the electronic device based on the first device information. The first device information includes the device address of the electronic device, the address of the preset service device corresponding to the electronic device, and the device status information of the electronic device. The construction module is used to establish a connection with the target service device corresponding to the second device information based on the second device information when the second device information is received from the server. A transmission module is used to transmit the main information of the message to the target service device; wherein, the main information includes the location information of the user's current observation point and the observation view information, the observation view information includes the movement direction within the same preset field of view angle range as the location information of the current observation point and the three-dimensional phase information in the movement direction, and the main information is used to request the target service device to feed back VR image data corresponding to the observation view information based on the main information; The data transmission device further includes an acquisition module and a determination module; wherein, the acquisition module is used to acquire the projection position information of the user's eyes projected onto the first screen of the electronic device; the determination module is used to determine the projection position information as the position information of the current observation point; and using the position information of the current observation point as the search starting point, the direction in which the user's eyes can move within the same preset field of view as the current observation point is determined as the movement direction; The acquisition module is further configured to acquire a first data attribute of the subject information, the first data attribute including the data size of the first image data corresponding to the location information of the current observation point and the transmission rate of the data transmission channel; acquire a first slicing condition corresponding to the first data attribute according to the association between preset data attributes and preset slicing conditions; the data transmission device further includes a slicing module, configured to slice the first image data according to the first slicing condition to obtain N image segments, where N is a positive integer greater than 1; the sending module is specifically configured to transmit the N image segments to the target service device through the data transmission channel.

8. An electronic device, characterized in that, include: A processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the data transfer steps as described in any one of claims 1-4.

9. A server, characterized in that, include: A processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the data transfer steps as described in any one of claims 5-6.

Images