Data transmission method and device, communication equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses a data transmission method and device, communication equipment and a computer readable storage medium, and the method comprises the steps that network equipment receives a first request sent by a terminal, and the first request is used for requesting three-dimensional model data; the first request comprises first information, and the first information at least comprises a model definition level of each object in a first object set under a first visual angle of the user; determining a first detail level corresponding to each object in the first object set based on the first information; wherein each object has three-dimensional model data of a plurality of detail levels; sending second information to the terminal, wherein the second information comprises the first data set; the first data set comprises first model data, corresponding to the first detail level, of each object in the first object set; the second information is used for the terminal to generate a three-dimensional model of each object in the first object set.

Description

Technical Field

[0001] The present invention relates to the field of wireless network technology, and in particular to a data transmission method, device, communication equipment and computer-readable storage medium. Background Art

[0002] In services such as Extended Reality (XR) and the Metaverse, the three-dimensional (3D) scene perceived by users is composed of various 3D models. Usually, the processing flow of 3D models includes reconstruction (when represented by multiple views or depth maps), rendering, and display. If it is rendered in the cloud, it needs to be encoded and streamed, and then transmitted to the terminal for display via the network. In this case, the object encoded is a two-dimensional image, but for the perception of 3D services, 3D models need to be transmitted. Compared with two-dimensional images, 3D models have a large amount of data. At present, there is no standard for unified encoding of 3D models, and high network bandwidth requirements are required. Moreover, when users request multiple 3D object models, the amount of network transmission data will be even greater. Currently, there is no suitable transmission solution for services that transmit multiple 3D models. Summary of the invention

[0003] To solve the existing technical problems, the embodiments of the present invention provide a data transmission method, an apparatus, a communication device and a computer-readable storage medium.

[0004] To achieve the above object, the technical solution of the embodiment of the present invention is implemented as follows:

[0005] In a first aspect, an embodiment of the present invention provides a data transmission method, including:

[0006] The network device receives a first request sent by the terminal, the first request being used to request three-dimensional model data; the first request includes first information, the first information at least including a model clarity level of each object in a first object set under a first viewing angle of a user;

[0007] Determining a first level of detail corresponding to each object in the first object set based on the first information; wherein each object has three-dimensional model data of multiple levels of detail;

[0008] Sending second information to the terminal, the second information includes a first data set; the first data set includes first model data corresponding to the first detail level for each object in the first object set; the second information is used by the terminal to generate a three-dimensional model for each object in the first object set.

[0009] In the above scheme, determining the first detail level corresponding to each object in the first object set based on the first information includes: determining the second detail level corresponding to each object in the first object set based on the first mapping relationship and the first information; wherein the first mapping relationship includes detail levels corresponding to different model clarity levels; determining the first detail level corresponding to each object according to network resource information and the second detail level corresponding to each object in the first object set.

[0010] In the above scheme, determining the first detail level corresponding to each object according to the network resource information and the second detail level corresponding to each object in the first object set includes: determining a data volume threshold for data transmission based on the network resource information; when the data volume of the three-dimensional model data corresponding to the second detail level of all objects in the first object set does not exceed the data volume threshold, determining the second detail level corresponding to each object as the first detail level corresponding to each object; when the data volume of the three-dimensional model data corresponding to the second detail level of all objects in the first object set exceeds the data volume threshold, lowering the second detail level of at least one object in the first object set to determine the first detail level corresponding to each object; wherein, after the detail level is lowered, the data volume of the three-dimensional model data corresponding to the first detail level of all objects in the first object set does not exceed the data volume threshold.

[0011] In the above scheme, lowering the second detail level of at least one object in the first object set includes: when the second detail levels of the first object and the second object are the same, alternately lowering the second detail level of the first object and the second detail level of the second object according to the model clarity level; when the model clarity levels of the first object and the second object are different, and the second detail levels of the first object and the second object are different, continuously lowering the second detail level of the first object and / or the second detail level of the second object according to the order of detail levels, wherein the second detail level of the object with a lower model clarity level is preferentially lowered; wherein the first object and the second object are any two objects in the first object set.

[0012] In the above scheme, the first information also includes the display capability information of the terminal; determining the first detail level corresponding to each object in the first object set based on the first information includes: determining the detail level range of each object according to the display capability information; determining the first detail level corresponding to each object in the first object set within the detail level range based on the first information.

[0013] In the above scheme, the method also includes: the network device receives a second request sent by the terminal, the second request is used to request to update the three-dimensional model data; the second request includes third information, and the third information includes the model clarity level of each object in the second object set under the user's second perspective; based on the third information, the third detail level corresponding to each object in the second object set is determined; fourth information is sent to the terminal, the fourth information includes a second data set, and the second data set includes second model data corresponding to the third detail level of at least some objects in the second object set; the fourth information is used by the terminal to generate a three-dimensional model of each object in the second object set.

[0014] In the above scheme, before sending the fourth information to the terminal, the method also includes: determining a third object and / or a fourth object based on the first object set and the second object set, wherein the third object is any object belonging to the first object set and the second object set and the corresponding first detail level is lower than the third detail level, and the fourth object is any object belonging to the second object set and not belonging to the first object set; accordingly, the second data set only includes the second model data of all the third objects and / or the fourth objects corresponding to the third detail level.

[0015] In a second aspect, an embodiment of the present invention provides a data transmission method, including:

[0016] The terminal sends a first request to the network device, the first request being used to request three-dimensional model data; the first request includes first information, the first information at least including a model clarity level of each object in a first object set under a first perspective of a user;

[0017] Receive second information sent by the network device, and generate a three-dimensional model of each object in the first object set based on the second information; the second information includes a first data set, and the first data set includes first model data corresponding to a first detail level for each object in the first object set; the first detail level is determined by the network device based on the first information; wherein each object has three-dimensional model data of multiple detail levels.

[0018] In the above scheme, before the terminal sends the first request to the network device, the method also includes: the terminal determines the first object set based on the first perspective, and determines the first distance and the second distance of each object in the first object set, wherein the first distance is the vertical distance between the object and the viewpoint plane corresponding to the first perspective, and the second distance is the vertical distance between the object and the first straight line, and the first straight line is the straight line where the observation direction corresponding to the first perspective is located; determine the model clarity level of each object based on the first distance and the second distance.

[0019] In the above scheme, the method also includes: the terminal sends a second request to the network device, the second request is used to request updating of the three-dimensional model data; the second request includes third information, and the third information includes the model clarity level of each object in the second object set under the user's second perspective; receiving fourth information sent by the network device, and generating a three-dimensional model of each object in the second object set based on the fourth information; the fourth information includes a second data set, and the second data set includes second model data corresponding to a third detail level of at least some objects in the second object set; the third detail level is determined by the network device based on the third information.

[0020] In the above scheme, the second data set only includes the second model data of all third objects and / or fourth objects corresponding to the third detail level, wherein the third object is any object belonging to the first object set and the second object set and the corresponding first detail level is lower than the third detail level, and the fourth object is any object belonging to the second object set and not belonging to the first object set.

[0021] In the above scheme, the terminal sends a second request to the network device, including: the terminal sends the second request when the second perspective satisfies the first condition; wherein the first condition includes at least one of the following: there are new objects under the second perspective relative to the objects under the first perspective; the change of the observation distance under the second perspective relative to the observation distance under the first perspective exceeds a first value; the change of the observation direction under the second perspective relative to the observation direction under the first perspective exceeds a second value.

[0022] In the above scheme, before the terminal sends the second request to the network device, the method also includes: the terminal determines the second object set based on the second perspective, and determines the third distance and the fourth distance of each object in the second object set, wherein the third distance is the vertical distance between the object and the viewpoint plane corresponding to the second perspective, and the fourth distance is the vertical distance between the object and the second straight line in the observation direction, and the second straight line is the straight line where the observation direction corresponding to the second perspective is located; determine the model clarity level of each object based on the third distance and the fourth distance.

[0023] In a third aspect, an embodiment of the present invention provides a data transmission device, which is applied to a network device, and includes a first communication unit and a first processing unit; wherein:

[0024] The first communication unit is configured to receive a first request sent by a terminal, the first request being used to request three-dimensional model data; the first request includes first information, the first information at least including a model clarity level of each object in a first object set under a first viewing angle of a user;

[0025] The first processing unit is configured to determine a first level of detail corresponding to each object in the first object set based on the first information; wherein each object has three-dimensional model data of multiple levels of detail;

[0026] The first communication unit is also used to send second information to the terminal, the second information including a first data set; the first data set includes first model data corresponding to the first detail level for each object in the first object set; the second information is used by the terminal to generate a three-dimensional model for each object in the first object set.

[0027] In the above scheme, the first processing unit is used to determine the second detail level corresponding to each object in the first object set based on the first mapping relationship and the first information; wherein the first mapping relationship includes detail levels corresponding to different model clarity levels; and, based on the network resource information and the second detail level corresponding to each object in the first object set, determine the first detail level corresponding to each object.

[0028] In the above scheme, the first processing unit is used to determine the data volume threshold for data transmission based on the network resource information; when the data volume of the three-dimensional model data corresponding to the second detail level of all objects in the first object set does not exceed the data volume threshold, the second detail level corresponding to each object is determined as the first detail level corresponding to each object; when the data volume of the three-dimensional model data corresponding to the second detail level of all objects in the first object set exceeds the data volume threshold, the second detail level of at least one object in the first object set is lowered to determine the first detail level corresponding to each object; wherein, after the detail level is lowered, the data volume of the three-dimensional model data corresponding to the first detail level of all objects in the first object set does not exceed the data volume threshold.

[0029] In the above scheme, the first processing unit is used to alternately lower the second detail level of the first object and the second detail level of the second object according to the model clarity level when the second detail levels of the first object and the second object are the same; and to continuously lower the second detail level of the first object and / or the second detail level of the second object according to the order of detail levels when the model clarity levels of the first object and the second object are different and the second detail levels of the first object and the second object are different, wherein the second detail level of the object with a lower model clarity level is lowered preferentially; wherein the first object and the second object are any two objects in the first object set.

[0030] In the above scheme, the first information also includes the display capability information of the terminal; the first processing unit is used to determine the detail level range of each object according to the display capability information; and determine the first detail level corresponding to each object in the first object set within the detail level range based on the first information.

[0031] In the above scheme, the first communication unit is also used to receive a second request sent by the terminal, and the second request is used to request to update the three-dimensional model data; the second request includes third information, and the third information includes the model clarity level of each object in the second object set under the user's second perspective; the first processing unit is also used to determine the third detail level corresponding to each object in the second object set based on the third information; the first communication unit is also used to send fourth information to the terminal, and the fourth information includes a second data set, and the second data set includes second model data corresponding to the third detail level of at least some objects in the second object set; the fourth information is used by the terminal to generate a three-dimensional model of each object in the second object set.

[0032] In the above scheme, the first processing unit is also used to determine the third object and / or the fourth object based on the first object set and the second object set, wherein the third object is any object that belongs to the first object set and the second object set and whose corresponding first detail level is lower than the third detail level, and the fourth object is any object that belongs to the second object set and does not belong to the first object set; accordingly, the second data set only includes the second model data of all the third objects and / or the fourth objects corresponding to the third detail level.

[0033] In a fourth aspect, an embodiment of the present invention provides a data transmission device, the device is applied in a terminal, the device includes a second communication unit,

[0034] The second communication unit is used to send a first request to a network device, wherein the first request is used to request three-dimensional model data; the first request includes first information, wherein the first information at least includes a model clarity level of each object in a first object set under a first perspective of a user; and, receiving second information sent by the network device, and generating a three-dimensional model of each object in the first object set based on the second information; the second information includes a first data set, wherein the first data set includes first model data corresponding to a first detail level for each object in the first object set; the first detail level is determined by the network device based on the first information; wherein each object has three-dimensional model data of multiple detail levels.

[0035] In the above scheme, the device also includes a second processing unit, which is used to determine the first object set based on the first perspective, and to determine the first distance and the second distance of each object in the first object set, wherein the first distance is the vertical distance between the object and the viewpoint plane corresponding to the first perspective, and the second distance is the vertical distance between the object and the first straight line, and the first straight line is the straight line where the observation direction corresponding to the first perspective is located; and is also used to determine the model clarity level of each object based on the first distance and the second distance.

[0036] In the above scheme, the second communication unit is also used to send a second request to the network device, the second request is used to request to update the three-dimensional model data; the second request includes third information, the third information includes the model clarity level of each object in the second object set under the user's second perspective; and, receives fourth information sent by the network device; the second processing unit is used to generate a three-dimensional model of each object in the second object set based on the fourth information; the fourth information includes a second data set, the second data set includes second model data corresponding to a third detail level of at least some objects in the second object set; the third detail level is determined by the network device based on the third information.

[0037] In the above scheme, the second data set only includes the second model data of all third objects and / or fourth objects corresponding to the third detail level, wherein the third object is any object belonging to the first object set and the second object set and the corresponding first detail level is lower than the third detail level, and the fourth object is any object belonging to the second object set and not belonging to the first object set.

[0038] In the above scheme, the second communication unit is also used to send the second request when the second perspective satisfies the first condition; wherein the first condition includes at least one of the following: there are new objects under the second perspective relative to the objects under the first perspective; the change in the observation distance under the second perspective relative to the observation distance under the first perspective exceeds a first value; the change in the observation direction under the second perspective relative to the observation direction under the first perspective exceeds a second value.

[0039] In the above scheme, the second processing unit is also used to determine the second object set based on the second perspective, and to determine the third distance and the fourth distance of each object in the second object set, wherein the third distance is the vertical distance between the object and the viewpoint plane corresponding to the second perspective, and the fourth distance is the vertical distance between the object and the second straight line in the observation direction, and the second straight line is the straight line where the observation direction corresponding to the second perspective is located; and, based on the third distance and the fourth distance, determine the model clarity level of each object.

[0040] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect or the second aspect.

[0041] In a sixth aspect, an embodiment of the present invention provides a communication device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect or the second aspect when executing the program.

[0042] The data transmission method, apparatus, communication equipment and computer-readable storage medium provided in the embodiments of the present invention can be applied to the transmission of three-dimensional model data of multiple objects. When a terminal requests three-dimensional models of multiple objects from a network device, the network device can decide the three-dimensional model data of the corresponding model detail level based on the model accuracy requirement of the terminal (i.e., the model clarity level), thereby realizing variable bit rate three-dimensional model data transmission, saving bandwidth resources, reducing service response delay and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The data transmission method according to the embodiment of the present invention is shown in FIG. Figure 1 ;

[0044] Figure 2 A schematic diagram of the positional relationship between a user's viewpoint and various objects in a virtual scene according to an embodiment of the present invention;

[0045] Figure 3 The data transmission method according to the embodiment of the present invention is shown in FIG. Figure 2 ;

[0046] Figure 4 A schematic diagram of a first distance and a second distance according to an embodiment of the present invention;

[0047] Figure 5 A flowchart of an application example of the data transmission method according to an embodiment of the present invention;

[0048] Fig. 6A Schematic diagram of the combination of different LOD levels of 3D objects in an application example of the data transmission method of an embodiment of the present invention Figure 1 ;

[0049] Figure 6B Schematic diagram of the combination of different LOD levels of 3D objects in an application example of the data transmission method of an embodiment of the present invention Figure 2 ;

[0050] Figure 7 A schematic diagram of a user's viewing angle change in an application example of a data transmission method according to an embodiment of the present invention;

[0051] Figure 8 The structure diagram of the data transmission device according to the embodiment of the present invention is shown in FIG. Figure 1 ;

[0052] Fig. 9The structure diagram of the data transmission device according to the embodiment of the present invention is shown in FIG. Figure 2 ;

[0053] Fig.10 Schematic diagram of the structure of a communication device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] In the description of the present invention, it should be noted that the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. These terms are only used to distinguish one element (or threshold or application or instruction or operation) from another element (or threshold or application or instruction or operation). For example, a first operation can be called a second operation, and a second operation can also be called a first operation without departing from the scope of the present invention. Both the first operation and the second operation are operations, but they are not the same operation.

[0056] The term "and / or" in the embodiments of the present invention refers to any and all possible combinations of one or more of the associated enumerated items. It should also be noted that when used in this specification, "include / comprise" specifies the existence of the stated features, integers, steps, operations, elements and / or components, but does not exclude the existence or addition of one or more other features, integers, steps, operations, elements and / or components and / or their groups.

[0057] The steps in the embodiments of the present invention do not necessarily have to be processed in the order of the described steps. The steps can be selectively rearranged, or the steps in the embodiments can be deleted, or the steps in the embodiments can be added as needed. The step description in the embodiments of the present invention is only an optional sequence combination and does not represent all step sequence combinations in the embodiments of the present invention. The step order in the embodiments cannot be regarded as a limitation of the present invention.

[0058] The network device of the embodiment of the present invention may also be referred to as a wireless access network device, and the wireless access network device may be: a base station, an evolved base station (eNB, evolved node B), a home base station, an access point (AP, Access Point) in a wireless fidelity (WIFI, Wireless Fidelity) system, a wireless relay node, a wireless backhaul node, a transmission point (TP, Transmission Point) or a transmission and reception point (TRP, Transmission and Reception Point), etc. It may also be a gNB in ​​a new radio (NR, New Radio) system, or it may also be a component or a part of a device constituting a base station, etc. It should be understood that in each embodiment of the present invention, the specific technology and specific device form adopted by the network device are not limited. In the present invention, the network device may be implemented in the form of a cloud device, or it may also be the above-mentioned wireless access network device deployed with the cloud device, etc. The cloud device may be, for example, an edge cloud, an edge device, an edge node, a cloud server, etc.

[0059] The terminal (such as a mobile terminal) of an embodiment of the present invention can be implemented in various forms. For example, the terminal described in the embodiment of the present invention may include a mobile terminal such as a VR head-mounted display device, AR glasses, a mobile phone, a smart phone, a laptop computer, a digital broadcast receiver, a personal digital assistant (PDA, Personal Digital Assistant), a tablet computer (PAD), a portable multimedia player (PMP, Portable Media Player), a navigation device, etc., and a fixed terminal such as a digital TV, a smart TV, a desktop computer, etc. Below, it is assumed that the terminal is a mobile terminal. However, it will be understood by those skilled in the art that, in addition to elements specifically for mobile purposes, the construction according to the embodiment of the present invention can also be applied to a fixed type of terminal.

[0060] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0061] An embodiment of the present invention provides a data transmission method. Figure 1 The data transmission method according to the embodiment of the present invention is shown in FIG. Figure 1 ,like Figure 1 As shown, the method includes:

[0062] Step 101: A network device receives a first request sent by a terminal, wherein the first request is used to request three-dimensional model data; the first request includes first information, wherein the first information includes at least a model clarity level of each object in a first object set under a first viewing angle of a user;

[0063] Step 102: determining a first level of detail corresponding to each object in the first object set based on the first information; wherein each object has three-dimensional model data of multiple levels of detail;

[0064] Step 103: Send second information to the terminal, where the second information includes a first data set; the first data set includes first model data corresponding to the first detail level for each object in the first object set; and the second information is used by the terminal to generate a three-dimensional model for each object in the first object set.

[0065] The data transmission method of this embodiment can be applied to three-dimensional (3D) business scenarios, which may include virtual reality (VR), augmented reality (AR), metaverse and other businesses with 3D experience requirements, such as real-time visits to themed virtual exhibition halls.

[0066] In this embodiment, the first request is used to request the 3D model data of each object in the first object set under the user's first perspective. It can be understood that the first perspective is the field of view corresponding to the user's viewpoint in the virtual scene, and the first object set is the set of all 3D objects that need to be presented within the field of view. For example, as the user moves or rotates in a virtual scene, the 3D objects presented in the user's perspective will change.

[0067] In some embodiments, the first information also includes identification information of each object in the first object set.

[0068] In step 101, the model clarity level may be used to measure the user's requirements for the model quality, model details or model clarity of each object. A higher model clarity level of an object means that the user's requirements for the model clarity of the object are higher.

[0069] In some embodiments, the model clarity level of each object may be determined based on a vertical distance between each object and the user's sight direction and a vertical distance between each object and the user's viewpoint plane at the first viewing angle. Figure 2 FIG. 1 is a schematic diagram of the positional relationship between the user's viewpoint and each object in a virtual scene according to an embodiment of the present invention. Figure 2As shown in the figure, the vertical distance D(A) between object A and the user's viewpoint plane is smaller than the vertical distance D(B) between object B and the user's viewpoint plane, indicating that object A is closer to the user's viewpoint. Accordingly, the user's model quality requirements for object A should be higher. However, at the same time, the vertical distance VER_D(A) between object A and the user's line of sight is larger than the vertical distance VER_D(B) between object B and the user's line of sight, indicating that object B is closer to the user's area of ​​interest. Accordingly, the user's model quality requirements for object B should be higher. The model clarity levels of objects A and B can be determined after comprehensive consideration.

[0070] In some embodiments, the model clarity level can be distinguished by different identifiers, numerical values ​​or numerical ranges, etc. For example, the model clarity level can be represented based on a numerical value between 1 and 100, and a larger numerical value indicates that the model clarity level corresponding to the object is higher or lower; or, the model clarity level can be identified based on characters with a fixed order, such as letters A, B, C, D, etc., and a higher order indicates that the model clarity level corresponding to the object is higher or lower.

[0071] In various embodiments of the present invention, the network device may store three-dimensional model data corresponding to a plurality of objects, wherein each object has three-dimensional model data of a plurality of detail levels, and the data volume of the three-dimensional model data corresponding to different detail levels of each object is different. In step 102, the network device determines the first detail level corresponding to each object based on the model clarity level of each object in the first object set, thereby determining the data volume of the three-dimensional model data to be sent to the terminal based on the model clarity requirement of each object under the user's first perspective.

[0072] In step 103, the network device sends first model data of each object corresponding to a first detail level to the terminal, and the terminal generates a three-dimensional model of each object from a first perspective of the user based on the first model data of each object.

[0073] In some embodiments, the network device may train the reconstruction, rendering, and encoding functions of the three-dimensional model in a neural network based on the Neural radiance Field (NerF) technology to generate three-dimensional models with different code rates. As an example, a feature network representation method may be used to implement three-dimensional models with different code rates and different levels of detail. For example, multiple code rates may be implemented by depth truncation of a multi-resolution feature vector tree, or the feature vectors of the feature network may be stored in a codebook, and three-dimensional models with different code rates may be generated by using quantization parameters, such as quantizing different bit widths (bw, bitwith), which may implement coding compression of the three-dimensional model and reduce the bandwidth of the three-dimensional model data, wherein the larger the bw, the higher the accuracy of the three-dimensional model. That is, in this embodiment, the first model data is no longer the traditional information encoded based on slice (encoding after dividing the image into several strips) or tile (encoding after dividing the image into several rectangles), and the first model data may be a trained neural network, which may specifically include neurons and / or model parameters of the trained neural network.

[0074] The data transmission method of the embodiment of the present invention can be applied to the transmission of three-dimensional model data of multiple objects. When a terminal requests three-dimensional models of multiple objects from a network device, the network device can decide the three-dimensional model data of the corresponding model detail level based on the terminal's model accuracy requirements (i.e., the model clarity level), thereby realizing three-dimensional model data transmission with variable bit rate, saving bandwidth resources, reducing service response delay, and improving user experience.

[0075] In an optional embodiment of the present invention, determining the first detail level corresponding to each object in the first object set based on the first information may include: determining the second detail level corresponding to each object in the first object set based on a first mapping relationship and the first information; wherein the first mapping relationship includes detail levels corresponding to different model clarity levels; and determining the first detail level corresponding to each object according to network resource information and the second detail level corresponding to each object in the first object set.

[0076] In this embodiment, the first mapping relationship may be, for example, a mapping function from a model definition level to a detail level. The higher the model definition level, the higher the corresponding detail level. Exemplarily, the model definition level may be represented by a numerical value from 1 to 100, and the detail level may include levels 1 to 5. For example, the model definition level in the range of 1 to 20 may be mapped to a detail level 1, the model definition level in the range of 21 to 40 may be mapped to a detail level 2, the model definition level in the range of 41 to 60 may be mapped to a detail level 3, the model definition level in the range of 61 to 80 may be mapped to a detail level 4, and the model definition level in the range of 81 to 100 may be mapped to a detail level 5. It can be understood that the model definition level may be a fine-grained model accuracy division determined by user-side requirements, and the detail level may be a coarse-grained model accuracy division stored on the network side. The first mapping relationship may be used to determine a unique second detail level for each object according to the model definition level of each object in the first request.

[0077] This embodiment also takes into account network resource information, which may include transmission resources between the network device and the terminal, such as bandwidth, transmission bit rate, etc. That is, after determining the second level of detail corresponding to each object according to the first mapping relationship, the first level of detail of each object in the first data set finally transmitted to the terminal is also determined based on the network resource information.

[0078] The data transmission method of the embodiment of the present invention maps the fine-grained model accuracy requirements of the user side to the coarse-grained detail level through the first mapping relationship, which can simply and quickly adapt to the division of network resources.

[0079] In some embodiments, determining the first detail level corresponding to each object according to the network resource information and the second detail level corresponding to each object in the first object set may include: determining a data volume threshold for data transmission based on the network resource information; when the data volume of the three-dimensional model data corresponding to the second detail level of all objects in the first object set does not exceed the data volume threshold, determining the second detail level corresponding to each object as the first detail level corresponding to each object; when the data volume of the three-dimensional model data corresponding to the second detail level of all objects in the first object set exceeds the data volume threshold, lowering the second detail level of at least one object in the first object set to determine the first detail level corresponding to each object; wherein the data volume of the three-dimensional model data corresponding to the first detail level of all objects in the first object set after the detail level is lowered does not exceed the data volume threshold.

[0080] In this embodiment, the data volume threshold may be, for example, a transmission data volume determined based on the network resource information. When it is determined that the total amount of data of the three-dimensional models of the second detail level corresponding to all objects in the first object set does not exceed the data volume threshold, the second detail level may be determined as the first detail level of the first model data transmitted to the terminal; when it is determined that the total amount of data of the three-dimensional models of the second detail level corresponding to all objects in the first object set exceeds the data volume threshold, the second detail level of at least one object is adjusted, for example, the detail level of some or all objects is reduced on the basis of the second detail level, until the total amount of data of the three-dimensional model data corresponding to the reduced detail level does not exceed the data volume threshold, and the reduced detail level is determined as the first detail level of the first model data transmitted to the terminal.

[0081] The data transmission method of the embodiment of the present invention can adjust the detail level of the three-dimensional model according to the network resource information, and can give priority to model resources with higher model clarity level requirements when the network resources are poor, thereby optimizing the user experience.

[0082] In an optional embodiment of the present invention, lowering the second detail level of at least one object in the first object set may include: when the second detail levels of the first object and the second object are the same, alternately lowering the second detail level of the first object and the second detail level of the second object according to the model clarity level; when the model clarity levels of the first object and the second object are different and the second detail levels of the first object and the second object are different, continuously lowering the second detail level of the first object and / or the second detail level of the second object according to the order of detail levels, wherein the second detail level of the object with a lower model clarity level is preferentially lowered; wherein the first object and the second object are any two objects in the first object set.

[0083] As an implementation mode, when the model clarity levels of the first object and the second object are different and the second detail levels of the first object and the second object are the same, the second detail level of the first object and the second detail level of the second object are alternately lowered according to the model clarity level. Exemplarily, if the model clarity level of the first object is higher than the model clarity level of the second object, the second detail level of the second object may be lowered first. If, after lowering the second detail level of the second object, the total amount of three-dimensional model data corresponding to the respective second detail levels of all objects (including the second detail level of the second object after the lowering) exceeds the data amount threshold, the second detail level of the first object is further lowered. At this time, it is determined whether the total amount of data exceeds the data amount threshold. If it still exceeds the data amount threshold, the above process of alternately lowering the detail level of the objects is continued until the total amount of three-dimensional model data corresponding to the respective second detail levels of all objects after adjustment does not exceed the data amount threshold, and the adjustment process ends.

[0084] As another implementation, when the model clarity level of the first object is the same as that of the second object, and the second detail level of the first object is the same as that of the second object, the second detail level of the first object and the second detail level of the second object are alternately lowered according to the model clarity level. Exemplarily, when the model clarity level of the first object is the same as that of the second object, the second detail level of the first object (or the second object) may be lowered first. If, after lowering the second detail level of the first object (or the second object), the total amount of three-dimensional model data corresponding to the respective second detail levels (including the lowered second detail level) of all objects exceeds the data volume threshold, the second detail level of the second object (or the first object) is further lowered. At this time, it is determined whether the total amount of data exceeds the data volume threshold. If it still exceeds the data volume threshold, the above process of alternately lowering the detail level of the objects is continued until the total amount of three-dimensional model data corresponding to the respective second detail levels of all objects after adjustment does not exceed the data volume threshold, and the adjustment process ends.

[0085] As another embodiment, when the model clarity levels of the first object and the second object are different, and the second detail levels of the first object and the second object are different, the second detail level of the first object and / or the second detail level of the second object are continuously lowered in the order of detail levels, wherein the second detail level of the object with the lower model clarity level among the first object and the second object is preferentially lowered. That is, firstly consider lowering the second detail level of the object with the lower model clarity level, and if the total amount of data does not exceed the data amount threshold in the process of continuously lowering the second detail level of the object with the lower model clarity level, the adjustment process ends; if the second detail level of the object with the lower model clarity level has been adjusted to the lowest detail level, but the total amount of data still exceeds the data amount threshold, consider lowering the second detail level of the object with the higher model clarity level. Exemplarily, if the model clarity level of the first object is higher than the model clarity level of the second object, the second detail level of the second object is preferentially lowered. If, after the second detail level of the second object is lowered, the total amount of three-dimensional model data corresponding to the respective second detail levels of all objects (including the second detail level of the second object after the lowering) exceeds the data amount threshold, the second detail level of the second object is further lowered. At this time, it is determined whether the total amount of data exceeds the data amount threshold. If it still exceeds the data amount threshold, the second detail level of the second object is continued to be lowered and the above judgment is performed until the second detail level of the second object has reached the minimum detail level. If the total amount of data still exceeds the data amount threshold at this time, the second detail level of the object with higher model clarity (i.e., the first object) is further lowered or continuously lowered until the total amount of data of the three-dimensional models of all objects corresponding to their respective second detail levels after adjustment does not exceed the data amount threshold, and the adjustment process ends.

[0086] It should be noted that, after each adjustment of the second detail level of an object, the relationship between the second detail levels of each object in the first object set may change. At this time, the two objects compared for the next adjustment may change from the two objects compared for the previous adjustment. In this embodiment, for the convenience of description, "first object" and "second object" are uniformly used for description. It can be understood that the first object and the second object in this embodiment refer to any two objects that meet the above-mentioned relationship conditions of model clarity level and detail level.

[0087] Exemplarily, taking object A and object B as examples, each object includes four different detail levels. For example, the four detail levels of object A are A_LOD1, A_LOD2, A_LOD3 and A_LOD4 in order from high to low detail levels, and the four detail levels of object B are B_LOD1, B_LOD2, B_LOD3 and B_LOD4 in order from high to low detail levels. If the model clarity level LA of object A is higher than the model clarity level LB of object B, and the second detail level A_LOD2 of object A before adjustment is the same as the second detail level B_LOD2 of object B before adjustment, then the selection priority of each detail level combination is (A_LOD2, B_LOD2), (A_LOD2, B_LOD3), (A_LOD3, B_LOD3), (A_LOD3, B_LOD4), (A_LOD4, B_LOD4).

[0088] If the model clarity level LA of object A is higher than the model clarity level LB of object B, and the second detail level A_LOD2 of object A before adjustment is higher than the second detail level B_LOD3 of object B before adjustment, the second detail level of object B is lowered first. At this time, the selection priority of each detail level combination is (A_LOD2, B_LOD3), (A_LOD2, B_LOD4), (A_LOD3, B_LOD4), (A_LOD4, B_LOD4).

[0089] If the model clarity level LA of object A is the same as the model clarity level LB of object B, and the second detail level A_LOD2 of object A before adjustment is also the same as the second detail level B_LOD2 of object B before adjustment, the object whose second detail level is to be lowered may be randomly selected, that is, the selection priority of each detail level combination may be (A_LOD2, B_LOD2), (A_LOD2, B_LOD3) or (A_LOD3, B_LOD2), (A_LOD3, B_LOD3), (A_LOD3, B_LOD4) or (A_LOD4, B_LOD3), (A_LOD4, B_LOD4).

[0090] Therefore, when adjusting the second detail level of object A and the second detail level of object B, the detail levels of object A and object B can be determined according to their respective model clarity levels and their respective second detail levels, according to the corresponding priorities mentioned above, until the amount of three-dimensional model data of all objects in the first object set is less than the data amount threshold corresponding to the network resource information.

[0091] The data transmission method of an embodiment of the present invention decides a set of three-dimensional model data to be transmitted to the terminal based on the first request of the terminal and the network resource information. When network resources are limited, the model clarity level and the model detail level of each object are compared, and the transmission of models with lower levels is selected in descending order (alternatingly) or always prioritized in descending order, so as to optimize the user experience to the greatest extent.

[0092] In an optional embodiment of the present invention, the first information also includes display capability information of the terminal; determining the first detail level corresponding to each object in the first object set based on the first information may include: determining the detail level range of each object according to the display capability information; determining the first detail level corresponding to each object in the first object set within the detail level range based on the first information.

[0093] In this embodiment, the display capability information may refer to terminal capability information related to the three-dimensional model display, which may include the resolution level supported by the terminal, the bit rate range of the three-dimensional model display supported by the terminal, the three-dimensional model clarity range supported by the terminal, or the three-dimensional model detail level range supported by the terminal, etc. For example, if the display resolution level of the terminal is 2K, there is no need for the network device to transmit a 4K three-dimensional model.

[0094] In some embodiments, determining the detail level range of each object according to the display capability information may include: the network device determines the data volume range corresponding to the three-dimensional model data according to the display capability information, and determines the detail level range corresponding to each object based on the data volume range.

[0095] In some embodiments, determining the first level of detail corresponding to each object in the first object set in the level of detail range based on the first information may include: determining the first level of detail corresponding to each object in the first object set in the level of detail range based on the first information and the first mapping relationship. The first mapping relationship may refer to the description of the aforementioned embodiment, which is not repeated here.

[0096] In one embodiment, determining the first detail level corresponding to each object in the first object set within the detail level range based on the first information and the first mapping relationship may include: determining the second detail level corresponding to each object in the first object set based on the first mapping relationship and the first information, lowering the second detail level of at least one object in the first object set according to the detail level range, and determining the first detail level corresponding to each object, wherein the first detail level of each object in the first object set after the detail level is lowered belongs to the detail level range. Exemplarily, lowering the second detail level of at least one object in the first object set according to the detail level range may include: if the second detail level of the first object does not belong to the detail level range, adjusting the second detail level of the first object to the detail level range, the first object being any object in the first object set.

[0097] In this embodiment, after the network device determines the second detail level corresponding to each object based on the first mapping relationship and the model clarity level of each object, if the second detail level of any object does not fall within the detail level range, the second detail level of the object is lowered until the lowered second detail level falls within the detail level range, and the lowered second detail level is determined to be the first detail level of the object.

[0098] In another embodiment, the determining the first detail level corresponding to each object in the first object set in the detail level range based on the first information and the first mapping relationship may include: determining the second detail level corresponding to each object in the first object set based on the first mapping relationship and the first information, and determining the first detail level corresponding to each object in the detail level range according to the network resource information and the second detail level corresponding to each object in the first object set. Based on the previous embodiment, this embodiment may further consider network resource information.

[0099] Exemplarily, determining the first detail level corresponding to each object in the detail level range according to the network resource information and the second detail level corresponding to each object in the first object set may include: determining a data volume threshold for data transmission based on the network resource information; when the data volume of the three-dimensional model data corresponding to the second detail level of all objects in the first object set does not exceed the data volume threshold, and the second detail level of each object in the first object set belongs to the detail level range, determining the second detail level corresponding to each object as the first detail level corresponding to each object; when the data volume of the three-dimensional model data corresponding to the second detail level of all objects in the first object set exceeds the data volume threshold, lowering the second detail level of at least one object in the first object set in the detail level range to determine the first detail level corresponding to each object; wherein, after the detail level is lowered, the data volume of the three-dimensional model data corresponding to the first detail level of all objects in the first object set does not exceed the data volume threshold, and the adjusted first detail level of each object in the first object set belongs to the detail level range. Among them, the process of lowering the second detail level of at least one object in the first object set within the detail level range can refer to the description of the aforementioned embodiment. It is only necessary to ensure that the adjustment range does not exceed the detail level range. To save space, it will not be repeated here.

[0100] In another embodiment, the determining of the first detail level corresponding to each object in the first object set in the detail level range based on the first information and the first mapping relationship may include: determining a second mapping relationship based on the first mapping relationship and the detail level range, the second mapping relationship including detail levels corresponding to different model clarity levels and belonging to the detail level range; determining the second detail level corresponding to each object in the first object set based on the second mapping relationship and the first information, and determining the first detail level corresponding to each object according to the network resource information and the second detail level corresponding to each object in the first object set. In this embodiment, the first mapping relationship may be first corrected based on the detail level range to determine the second mapping relationship, and the second detail level determined based on the second mapping relationship and the model clarity level of each object all belongs to the detail level range. In addition, the first detail level corresponding to each object may be determined according to the network resource information and the second detail level corresponding to each object in the first object set, and the description of the aforementioned embodiment may be referred to, and will not be repeated here.

[0101] The data transmission method of the embodiment of the present invention can exclude the model detail levels of each object that exceed the display capability of the terminal based on the first request of the terminal, avoid transmitting the three-dimensional model with too high detail level, and reduce the waste of network resources.

[0102] The embodiment of the present invention also provides a data transmission method. Based on the above embodiment, the method further includes: the network device receives a second request sent by the terminal, the second request is used to request to update the three-dimensional model data; the second request includes third information, the third information includes the model clarity level of each object in the second object set under the second perspective of the user; based on the third information, the third detail level corresponding to each object in the second object set is determined; fourth information is sent to the terminal, the fourth information includes a second data set, the second data set includes second model data corresponding to the third detail level of at least some objects in the second object set; the fourth information is used by the terminal to generate a three-dimensional model of each object in the second object set.

[0103] In this embodiment, the second request is used to request three-dimensional model data of each object in a second object set under the user's second perspective. The second perspective is the perspective of the user after the perspective changes based on the first perspective. For example, as the user moves or rotates in a virtual scene, the user's perspective changes. The second object set is the set of all objects under the second perspective.

[0104] In some embodiments, the third information also includes identification information of each object in the second object set.

[0105] In some embodiments, determining the third detail level corresponding to each object in the second object set based on the third information may include: determining the fourth detail level corresponding to each object in the second object set based on the first mapping relationship and the third information, and determining the third detail level corresponding to each object according to network resource information and the fourth detail level corresponding to each object in the second object set.

[0106] As an optional implementation, determining the third detail level corresponding to each object based on network resource information and the fourth detail level corresponding to each object in the second object set may include: determining a data volume threshold for data transmission based on the network resource information; determining the fourth detail level corresponding to each object as the third detail level corresponding to each object, when the data volume of the three-dimensional model data corresponding to the fourth detail level of all objects in the two object sets does not exceed the data volume threshold; when the data volume of the three-dimensional model data corresponding to the fourth detail level of all objects in the second object set exceeds the data volume threshold, lowering the fourth detail level of at least one object in the second object set to determine the third detail level corresponding to each object; wherein the data volume of the three-dimensional model data corresponding to the third detail level of all objects in the second object set after the detail level is lowered does not exceed the data volume threshold.

[0107] In one embodiment, lowering the fourth detail level of at least one object in the second object set may include: when the fourth detail levels of the fifth object and the sixth object are the same, alternately lowering the fourth detail level of the fifth object and the fourth detail level of the sixth object according to the model clarity level; when the model clarity levels of the fifth object and the sixth object are different, and the fourth detail levels of the fifth object and the sixth object are different, continuously lowering the fourth detail level of the fifth object and / or the fourth detail level of the sixth object according to the order of detail levels, wherein the fourth detail level of the object with a lower model clarity level among the fifth object and the sixth object is preferentially lowered; wherein the fifth object and the sixth object are any two objects in the second object set.

[0108] In an optional embodiment of the present invention, before sending the fourth information to the terminal, the method further includes: determining a third object and / or a fourth object based on the first object set and the second object set, wherein the third object is any object belonging to the first object set and the second object set and the corresponding first detail level is lower than the third detail level, and the fourth object is any object belonging to the second object set and not belonging to the first object set; accordingly, the second data set only includes the second model data corresponding to the third detail level of all the third objects and / or the fourth objects.

[0109] In this embodiment, the network device compares the second object set with the first object set, determines a third object that belongs to both the first object set and the second object set and whose second detail level is higher than the first detail level, and / or determines a fourth object that belongs to the second object set and does not belong to the first object set. It can be understood that the third object is any object whose model detail level needs to be updated upward relative to the first perspective from the second perspective, and the fourth object is any object that is newly added relative to the first perspective from the second perspective. Accordingly, in the fourth information sent by the network device to the terminal, the second data set only includes the second model data of all third objects and / or all fourth objects.

[0110] In some embodiments, determining the third detail level corresponding to each object in the second object set based on the third information may include: determining the fourth detail level corresponding to each object in the second object set based on the first mapping relationship and the third information, and determining the third detail level corresponding to each third object and / or each fourth object according to network resource information and the fourth detail level corresponding to all third objects and / or fourth objects in the second object set.

[0111] As an optional implementation, determining the third detail level corresponding to each third object and / or each fourth object based on the network resource information and the fourth detail level corresponding to each of the third objects and / or the fourth objects in the second object set may include: determining a data volume threshold for data transmission based on the network resource information; determining the fourth detail level corresponding to each third object and / or each fourth object as the third detail level corresponding to each third object and / or each fourth object, when the data volume of the three-dimensional model data corresponding to the fourth detail level of all the third objects and / or the fourth objects in the second object set does not exceed the data volume threshold; and preferentially lowering the fourth detail level of at least one fourth object to determine the third detail level corresponding to each third object and / or each fourth object, when the data volume of the three-dimensional model data corresponding to the fourth detail level of all the third objects and / or the fourth objects in the second object set exceeds the data volume threshold. The fourth detail level of at least one fourth object is determined; wherein, after the detail level is lowered, the data volume of the three-dimensional model data corresponding to the third detail level of all the third objects and / or the fourth objects in the second object set does not exceed the data volume threshold.

[0112] The data transmission method of the embodiment of the present invention can determine the model detail level of each object sent by the network device according to the model clarity level of each object sent by the terminal in real time, so as to realize the transmission of three-dimensional model data with different bit rates according to the model clarity requirement on the terminal side. At the same time, during the real-time transmission process, redundant information in the transmitted model data can be removed according to the changes in user posture, thereby avoiding the waste of unlimited uplink and downlink resources and improving the utilization rate of unlimited resources and the user interaction experience of three-dimensional services.

[0113] The embodiment of the present invention also provides a data transmission method. Figure 3 The data transmission method according to the embodiment of the present invention is shown in FIG. Figure 2 ,like Figure 3 As shown, the method includes:

[0114] Step 201: The terminal sends a first request to a network device, where the first request is used to request three-dimensional model data; the first request includes first information, where the first information at least includes a model clarity level of each object in a first object set under a first viewing angle of a user;

[0115] Step 202: Receive second information sent by the network device, and generate a three-dimensional model of each object in the first object set based on the second information; the second information includes a first data set, and the first data set includes first model data corresponding to a first detail level for each object in the first object set; the first detail level is determined by the network device based on the first information; wherein each object has three-dimensional model data of multiple detail levels.

[0116] The relevant description of step 201 to step 202 of this embodiment can be specifically referred to the relevant description of step 101 in the above embodiment, and will not be repeated here to save space.

[0117] In this embodiment, the network device may pre-store three-dimensional model data corresponding to multiple objects, wherein each object has three-dimensional model data at multiple detail levels, and the data volume of the three-dimensional model data corresponding to different detail levels of each object is different.

[0118] In an optional embodiment of the present invention, before the terminal sends the first request to the network device, the method further includes: the terminal determines the first object set based on the first perspective, and determines the first distance and the second distance of each object in the first object set, wherein the first distance is the vertical distance between the object and the viewpoint plane corresponding to the first perspective, and the second distance is the vertical distance between the object and the first straight line, and the first straight line is the straight line where the observation direction corresponding to the first perspective is located; the model clarity level of each object is determined based on the first distance and the second distance. It can be understood that the first distance is the vertical distance between the object and the user's viewpoint plane, and the second distance is the vertical distance between the object and the user's line of sight, and the viewpoint plane is perpendicular to the line of sight.

[0119] As an optional implementation, the determining of the first distance and the second distance of each object in the first object set may include: determining the first distance of each object based on a first unit vector of the observation direction corresponding to the first viewing angle and a first coordinate vector of each object in the first object set, and determining the second distance of each object based on the first coordinate vector, the first unit vector, and a second coordinate vector corresponding to the user's viewpoint under the first viewing angle. The first unit vector, the first coordinate vector, and the second coordinate vector are all vectors in the coordinate system corresponding to the virtual scene, and the first coordinate vector is a coordinate vector corresponding to the center coordinates of the object.

[0120] Figure 4 Schematic diagram of the first distance and the second distance of an embodiment of the present invention, such as Figure 4 As shown in the figure, a virtual scene coordinate system is established with the center of the user's viewpoint as the origin. The position coordinates of the user's viewpoint in the virtual scene coordinate system are (0,0,0), and the corresponding position vector is P o , the unit vector of the observation direction corresponding to the first perspective in the virtual scene coordinate system is P e (X e ,Y e ,Z e ), the position coordinates of the center of the 3D object in the virtual scene coordinate system are (X m ,Y m ,Z m ), the corresponding position vector is P m , the first distance D can be calculated by the following formula:

[0121]

[0122] The second distance VER_D can be calculated by the following formula:

[0123] VER_D=|P m -P o|×|P m -P e | / |P e -P o |

[0124] Among them, X e , Y e , Z e The X, Y, and Z axis components of the unit vector of the user's current viewing direction in the virtual scene coordinate system, respectively. m , Y m , Z m They respectively represent the X, Y, and Z axis coordinate components of the 3D object center in the virtual scene coordinate system.

[0125] In some embodiments, determining the model clarity level of the object based on the first distance and the second distance may include: determining the model clarity level of the object based on a weighted sum of the first distance and the second distance. Exemplarily, the model clarity level may be L=∝D+βVER_D, where ∝ represents the weight of the first distance, β represents the weight of the second distance, and optionally, the larger the value of the model clarity level, the lower the model clarity level.

[0126] In an optional embodiment of the present invention, the first information further includes display capability information of the terminal; the display capability information is used by the network device to determine a detail level range that satisfies the display capability of the terminal.

[0127] In this embodiment, the first information may also include display capability information of the terminal so that the network device can filter out three-dimensional models with a high level of detail, avoid transmitting three-dimensional model data of too high a level, and reduce the waste of network resources.

[0128] In an optional embodiment of the present invention, the method also includes: the terminal sends a second request to the network device, the second request is used to request updating of the three-dimensional model data; the second request includes third information, and the third information includes the model clarity level of each object in the second object set under the user's second perspective; receiving fourth information sent by the network device, and generating a three-dimensional model of each object in the second object set based on the fourth information; the fourth information includes a second data set, and the second data set includes second model data corresponding to a third detail level of at least some objects in the second object set; the third detail level is determined by the network device based on the third information.

[0129] In an optional embodiment of the present invention, the second data set only includes second model data of all third objects and / or fourth objects corresponding to the third detail level, wherein the third object is any object belonging to the first object set and the second object set and the corresponding first detail level is lower than the third detail level, and the fourth object is any object belonging to the second object set and not belonging to the first object set.

[0130] In an optional embodiment of the present invention, the terminal sends a second request to the network device, including: the terminal sends the second request when the second perspective satisfies a first condition; wherein the first condition includes at least one of the following: an object under the second perspective is newly added relative to the object under the first perspective; a change in the observation distance under the second perspective relative to the observation distance under the first perspective exceeds a first value; a change in the observation direction under the second perspective relative to the observation direction under the first perspective exceeds a second value.

[0131] In this embodiment, the terminal can monitor user operations and / or changes in user actions in real time, and send the second request when it detects that a new object is added in the second perspective of the user relative to the first perspective, or that the user's observation distance and / or observation direction has changed significantly. It should be noted that the changes in the observation distance and observation direction in the user's perspective can be determined by detecting the user's movement in the real environment, such as the user experiencing the virtual scene through AR or VR devices; or by detecting the user's operation in the virtual environment, such as the user inputting movement instructions in the virtual scene through an input device; or it can be a combination of the two, which is not limited in the present invention.

[0132] In an optional embodiment of the present invention, before the terminal sends the second request to the network device, the method also includes: the terminal determines the second object set based on the second perspective, and determines a third distance and a fourth distance of each object in the second object set, wherein the third distance is a vertical distance between the object and the viewpoint plane corresponding to the second perspective, and the fourth distance is a vertical distance between the object and a second straight line in the observation direction, and the second straight line is a straight line in the observation direction corresponding to the second perspective; and determines a model clarity level of each object based on the third distance and the fourth distance.

[0133] In some embodiments, determining the model clarity level of each object based on the third distance and the fourth distance may include: determining the model clarity level of each object based on a weighted sum of the third distance and the fourth distance. Optionally, the larger the value corresponding to the model clarity level, the lower the model clarity level.

[0134] As an optional implementation, the determining of the third distance and the fourth distance of each object in the second object set may include: determining the third distance of each object based on the second unit vector of the observation direction corresponding to the second perspective and the third coordinate vector of each object in the second object set, and determining the fourth distance of each object based on the second unit vector, the third coordinate vector and the fourth coordinate vector corresponding to the user's viewpoint under the second perspective. The second unit vector, the third coordinate vector and the fourth coordinate vector are all vectors in the coordinate system corresponding to the virtual scene, and the third coordinate vector is a coordinate vector corresponding to the center coordinates of the object.

[0135] The data transmission scheme of the embodiment of the present invention is described below in conjunction with specific application scenarios.

[0136] This example takes a 3D business scenario (for example, a user visits a themed virtual exhibition hall in real time through a terminal) as an example. The three-dimensional models of each 3D object (i.e., the objects in the aforementioned embodiments) in the 3D business scenario can be cached in the edge cloud. The edge cloud can be deployed on the base station side or the core network side to reduce the service transmission delay. In this example, the edge cloud or the base station or core network device deployed with the edge cloud is the network device in the aforementioned embodiments.

[0137] Figure 5 This is a flowchart of an application example of the data transmission method of an embodiment of the present invention. This example can adjust the 3D model data transmitted by the network device according to the real-time motion state of the user, such as Figure 5 As shown in the figure, the specific implementation process includes:

[0138] Step 301: The terminal calculates the model definition level of the 3D object according to the position information (D, VER_D) of the 3D object. In this example, the position information is the position information of the 3D object in the virtual exhibition hall scene. Figure 2 As shown, D represents the depth of the position information of the 3D object (i.e., the first distance), which is the vertical distance between the object and the user's viewpoint plane. The larger D is, the farther the corresponding 3D object is from the user's viewpoint. For 3D objects of the same size, the farther the object is, the smaller the model display range is; VER_D represents the absolute distance (i.e., the second distance) between the 3D object and the user's gaze direction (i.e., the first straight line). The larger VER_D is, the farther the 3D object is from the user's field of view of interest, and the corresponding model clarity requirement is also lower. The terminal calculates the model clarity level L of all 3D objects observed by the user's current perspective (i.e., the first perspective) in the three-dimensional business scene, where the model clarity level L is a fine-grained level division, and the value of L can be, for example, 1 to 100, to meet the description of the user's expected model clarity under multiple objects and multiple position relationships.

[0139] The model clarity level L is calculated from the position information (D, VER_D) of the 3D object in the virtual scene relative to the user's viewpoint. For example, L = ∝D + βVER_D, where ∝ represents the weight of the depth D and β represents the weight of the absolute distance VER_D. Figure 4 As shown in the figure, a virtual scene coordinate system is established with the center of the user's viewpoint as the origin. The position coordinates of the user's viewpoint in the virtual scene coordinate system are (0,0,0), and the corresponding position vector is P o , the unit vector of the user's current perspective (i.e., the first perspective) in the virtual scene coordinate system is P e (X e ,Y e ,Z e ), the position coordinates of the center of the 3D object in the virtual scene coordinate system are (X m ,Y m ,Z m ), the corresponding position vector is P m , the depth D can be calculated by the following formula:

[0140]

[0141] The absolute distance VER_D can be calculated by the following formula:

[0142] VER_D=|P m -P o |×|P m -P e | / |P e -P o |

[0143] Among them, X e , Y e , Z e The X, Y, and Z axis components of the unit vector of the user's current viewing direction in the virtual scene coordinate system, respectively. m , Y m , Z m They respectively represent the X, Y, and Z axis coordinate components of the 3D object center in the virtual scene coordinate system.

[0144] Thus, the terminal can obtain the model clarity level L corresponding to each 3D object under the user's current viewing angle.

[0145] Step 302: The network device caches 3D model data corresponding to multiple levels of detail (LOD) of multiple 3D objects. LOD indicates the richness of the details of the 3D model. The smaller the LOD value, the clearer the corresponding multiple 3D models and the richer the details. In this example, the network device can cache the 3D model data of each 3D object in the 3D business scene, and each 3D object can be neurally encoded into models with multiple levels of detail.

[0146] Step 303, the terminal requests multiple 3D model resources from the network device (i.e., sends a first request), carries the model definition level L information of each 3D object (i.e., the first information), and uploads the terminal display level RES_L. In this example, the terminal requests multiple 3D model resources from the network device, and can upload three types of information, including the model identifier of the 3D object, the model definition level L of the 3D object, and the terminal display level RES_L (i.e., display capability information), where RES_L represents the display capability of the terminal, which can be used by the network device to filter out high-level models, avoid transmitting 3D models of too high a level, and waste network resources. For example, RES_L can be a resolution level. If the terminal display resolution is 2K, there is no need to transmit a 4K model.

[0147] Step 304: The network device decides the 3D model data to be sent according to RES_L, L and network resources (i.e., network resource information). Specifically, the process may include the following:

[0148] (1) By excluding models whose LOD exceeds the display range of the terminal through RES_L, the network equipment can obtain the wireless resource status of the network (base station side or core network side) and transmit 3D model resources of different bit rates according to the user's viewpoint requirements;

[0149] (2) Execute mapping f: L→LOD (f refers to the mapping function from L to LOD, i.e., the first mapping relationship). Usually, the model definition level L is a fine-grained level division, such as 1 to 100, and the LOD level is a coarse-grained level division. It can be divided into single-digit levels according to the bit rate of the model, such as levels 1 to 4, so as to simply and quickly adapt to the division of network resources;

[0150] (3) Determine the combination of 3D model LOD levels based on network resource conditions. The bitrate (a measure of data volume) of each LOD level model is different. The bitrates of multiple 3D models are cumulatively added. If the network resources can meet the transmission conditions of the added bitrate, the 3D model data of the corresponding LOD level is sent according to the mapping result of step (2). If the network resources do not meet the transmission conditions of the added bitrate, the transmission of high-level L model resources is prioritized to optimize the user experience. The specific rules are as follows:

[0151] Case 1: If the model L of any two objects is different and the LOD level after mapping is the same, the LOD levels of the two objects are descended in the order of the model L of the two objects according to the network resources;

[0152] Case 2: If the model L of any two objects is different and the LOD levels after mapping are different, the LOD level of the object with lower model L level is prioritized in descending order according to network resources;

[0153] Case 3: If the model L of any two objects is different and the LOD levels after mapping are different, the LOD levels of the two objects are descended in sequence according to the network resources. Specifically, the descending logic in Case 1 can be reused.

[0154] The following takes the case where a terminal requests a network device to transmit models of 3D objects A and B as an example, and three situations are described respectively.

[0155] For case 1, Fig. 6A Schematic diagram of the combination of different LOD levels of 3D objects in an application example of the data transmission method of an embodiment of the present invention Figure 1 ,like Fig. 6A As shown, the network device can cache models at multiple LOD levels, such as Fig. 6A In the model LOD1, LOD2, LOD3 and LOD4 corresponding to object A or object B, the model data volume corresponding to different LOD levels of the same model is different. For example, the model data volume of LOD1 level of object A is 563kB, and the model data volume of LOD2 level is 160kB. For example, the LOD of object A and object B after mapping are both at the second level, that is, Fig. 6A In the first column on the left, 3D objects A and B are both at level LOD2. If the model clarity level LA of object A is higher than the model clarity level LB of object B, and the LOD level of object A after mapping is the same as the LOD level of object B after mapping, then the LOD levels of object B and object A are sorted in descending order according to network resources, and the priority order of model combination is: (A:LOD2, B:LOD2)>(A:LOD2, B:LOD3)>(A:LOD3, B:LOD3)>(A:LOD3, B:LOD4)>(A:LOD4, B:LOD4). In other words, if the sum of the LOD2 model data volume of object A after mapping (160kB) and the LOD2 model data volume of object B after mapping (131kB) exceeds the data volume that can be transmitted by network resources, the LOD of object B is sorted in descending order, such as Fig. 6AAs shown in the second column, the amount of data to be transmitted is determined, that is, the amount of LOD2 model data of object A (160kB) and the amount of LOD3 model data of object B (37kB), and it is judged again whether it exceeds the amount of data that the network resources can transmit. If it still exceeds, the LOD of object A is sorted in descending order, the amount of data is determined again, and it is judged whether it exceeds the amount of data that the network resources can transmit, and so on, until it does not exceed the amount of data transmitted by the network resources or it is reduced to the lowest LOD of the models of objects A and B.

[0156] For case 2, Figure 6B Schematic diagram of the combination of different LOD levels of 3D objects in an application example of the data transmission method of an embodiment of the present invention Figure 2 ,like Figure 6B As shown, taking the LOD of object A as the second level after mapping and the LOD of object B as the third level after mapping as an example, that is, Figure 7 In the first column on the left, 3D object A is model level LOD2, and object B is model level LOD3. If the model clarity level LA of object A is higher than the model clarity level LB of object B, and the LOD level of object A after mapping is higher than the LOD level of object B after mapping, then the LOD level of object B is prioritized to be descended. At this time, the priority order of model combination is: (A:LOD2, B:LOD3)>(A:LOD2, B:LOD4)>(A:LOD3, B:LOD4)>(A:LOD4, B:LOD4). In other words, if the sum of the LOD2 model data volume of object A after mapping (160kB) and the LOD2 model data volume of object B after mapping (37kB) exceeds the amount of data that network resources can transmit, then object B's LOD is always prioritized to be descended until object B drops to the lowest model LOD or does not exceed the amount of data that network resources can transmit. If object B drops to the lowest model LOD (such as Figure 6B If the LOD4 in the figure still exceeds the data volume that the network resources can transmit, the LOD of object A is considered to be descended, and it is determined whether it exceeds the data volume that the network resources can transmit, until the data volume that the network resources can transmit is met or it is reduced to the lowest LOD of the model of object A.

[0157] For case three, the descending logic in case one can be reused. However, when the model LODs of object A and object B are the same, the model LOD of object A or object B can be randomly selected to descend first. The rest of the process can refer to case one and will not be repeated here.

[0158] Step 305: The network device transmits 3D model data of multiple objects to the terminal.

[0159] Step 306: When the user's observation action changes, the 3D objects under the observation angle of view (i.e., the second angle of view) are updated, and the model clarity level L corresponding to each 3D object is recalculated. In this example, the terminal monitors the user's action changes in real time. When the type of 3D object under the user's perspective is added and / or the user's action changes significantly, the L of the 3D object is recalculated to avoid frequent uploading of information and wasting uplink resources. Among them, the significant change in user action can be measured by the size of the change in user space distance and / or observation direction, such as moving more than N meters, rotating the observation direction angle by more than M degrees, etc.

[0160] Figure 7 This is a schematic diagram of the change of user perspective in an application example of the data transmission method of an embodiment of the present invention. As the user moves and rotates in a virtual scene, the 3D objects presented in the user's viewpoint will undergo two changes: one is the addition or reduction of 3D objects, and the other is the change in the distance from the 3D object to the viewpoint. Figure 7 As shown, when the user's viewpoint is at position X, the corresponding field of view includes objects A and B. As the user moves and rotates to position Y, object B exceeds the user's field of view, and object C enters the user's field of view. Further, when the user moves and rotates to position Z, object B re-enters the user's field of view.

[0161] Step 307: The terminal uploads update information (ie, the second request) to the network device, which may carry information such as identification information of each object at the current viewing angle and model definition level.

[0162] Step 308: The network device re-decides the 3D model data to be transmitted to the terminal according to the update information and network resources. If data update occurs, step 309 is executed. The specific process may include the following:

[0163] (1) Execute mapping f: L → LOD;

[0164] (2) For newly added 3D objects (i.e., the third objects) and 3D objects whose model LOD levels are updated (i.e., the fourth objects), the model LOD of the objects to be transmitted is adjusted according to the descending rules in the above-mentioned cases 1, 2, and 3 based on network resources. In addition, during the adjustment process, the current model data of the 3D object is compared with the previously sent model data in real time. If the current model LOD is lower than the previously sent model LOD level, it is not sent. The 3D model quality of other objects (e.g., newly added objects and / or objects with higher LOD than before) is guaranteed with the network resources as the upper limit, thereby determining the updated 3D model data to be sent to the terminal.

[0165] Step 309: The network device sends the updated 3D model data (ie, the fourth information) to the terminal.

[0166] by Figure 7 For example, when the user's viewpoint is at position X, the terminal requests the 3D model data of objects A and B from the network device, and carries the corresponding L and RES_L information. At this time, the model clarity level LA of object A is higher than the model clarity level LB of object B. After mapping, the detail level LOD_A of object A is also higher than the detail level LOD_B of object B. The network device first filters the model with a high level according to RES_L. At this time, the 3D model data to be sent is determined according to the second case of step (3) in the above step 304, and the quality of model A is prioritized when the network is limited.

[0167] When the user's viewpoint moves to position Y, the terminal requests the 3D model data of objects A and C from the network device and carries the corresponding L, where the model clarity level LA of object A is higher than the model clarity level LC of object C. The network device makes a decision. If the model LOD level of object A needs to be updated upward based on f mapping, the 3D model data of objects A and C are sent while giving priority to the model quality of object A, otherwise only the 3D model data of object C is sent.

[0168] When the user's viewpoint moves to position Z, the terminal requests the 3D model data of objects A, B, and C from the network and carries the corresponding L. The network device makes a decision. At this time, the model clarity level LB of object B is higher than the model clarity level LA of object A, and the model clarity level LA of object A is higher than the model clarity level of object C. Based on the f mapping, the LOD level of object B is updated upward, the LOD level of object A is updated downward, and the LOD level of object C is not updated. In this case, only the 3D model data of object B is sent.

[0169] This example proposes transmission schemes with different bit rates for multiple neurally encoded 3D models. The terminal calculates and sends the model clarity level of each object to the network device in real time based on the position depth information of each 3D object in the virtual scene relative to the user's viewpoint and the absolute distance relative to the annotation direction. The network device decides to transmit a set of 3D model data based on wireless network resources. Specifically, when network resources are limited, the model clarity level and model detail level of each object are compared, and the model detail level of each object is selected in descending order or the model detail level of objects with low model clarity level is always prioritized in descending order. In addition, the terminal determines that when the user's posture changes significantly based on the user's position and observation angle information, it requests to update the 3D model data. The network filters the redundant model information and sends the updated 3D model data, avoiding the waste of uplink and downlink wireless resources, improving the utilization of wireless resources, and improving the user interaction experience of 3D services.

[0170] An embodiment of the present invention further provides a data transmission device, which is applied in a network device. Figure 8The structure diagram of the data transmission device according to the embodiment of the present invention is shown in FIG. Figure 1 ,like Figure 8 As shown, the data transmission device 400 includes a first communication unit 401 and a first processing unit 402; wherein,

[0171] The first communication unit 401 is configured to receive a first request sent by a terminal, the first request being used to request three-dimensional model data; the first request comprising first information, the first information comprising at least a model clarity level of each object in a first object set under a first viewing angle of a user;

[0172] The first processing unit 402 is configured to determine a first detail level corresponding to each object in the first object set based on the first information; wherein each object has three-dimensional model data of multiple detail levels;

[0173] The first communication unit 401 is also used to send second information to the terminal, the second information including a first data set; the first data set including first model data corresponding to the first detail level for each object in the first object set; the second information is used by the terminal to generate a three-dimensional model for each object in the first object set.

[0174] In an optional embodiment of the present invention, the first processing unit 402 is used to determine the second detail level corresponding to each object in the first object set based on the first mapping relationship and the first information; wherein the first mapping relationship includes detail levels corresponding to different model clarity levels; and, based on the network resource information and the second detail level corresponding to each object in the first object set, determine the first detail level corresponding to each object.

[0175] In an optional embodiment of the present invention, the first processing unit 402 is used to determine a data volume threshold for data transmission based on the network resource information; when the data volume of the three-dimensional model data corresponding to the second detail level of all objects in the first object set does not exceed the data volume threshold, the second detail level corresponding to each object is determined as the first detail level corresponding to each object; when the data volume of the three-dimensional model data corresponding to the second detail level of all objects in the first object set exceeds the data volume threshold, the second detail level of at least one object in the first object set is lowered to determine the first detail level corresponding to each object; wherein the data volume of the three-dimensional model data corresponding to the first detail level of all objects in the first object set after the detail level is lowered does not exceed the data volume threshold.

[0176] In an optional embodiment of the present invention, the first processing unit 402 is used to alternately lower the second detail level of the first object and the second detail level of the second object according to the model clarity level when the second detail levels of the first object and the second object are the same; and continuously lower the second detail level of the first object and / or the second detail level of the second object according to the order of detail levels when the model clarity levels of the first object and the second object are different and the second detail levels of the first object and the second object are different, wherein the second detail level of the object with a lower model clarity level is preferentially lowered; wherein the first object and the second object are any two objects in the first object set.

[0177] In an optional embodiment of the present invention, the first information also includes display capability information of the terminal; the first processing unit 402 is used to determine the detail level range of each object according to the display capability information; and determine the first detail level corresponding to each object in the first object set within the detail level range based on the first information.

[0178] In an optional embodiment of the present invention, the first communication unit 401 is further used to receive a second request sent by the terminal, the second request is used to request to update the three-dimensional model data; the second request includes third information, and the third information includes the model clarity level of each object in the second object set under the user's second perspective; the first processing unit 402 is further used to determine the third detail level corresponding to each object in the second object set based on the third information; the first communication unit 401 is further used to send fourth information to the terminal, the fourth information includes a second data set, and the second data set includes second model data corresponding to the third detail level of at least some objects in the second object set; the fourth information is used by the terminal to generate a three-dimensional model for each object in the second object set.

[0179] In an optional embodiment of the present invention, the first processing unit 402 is further used to determine a third object and / or a fourth object based on the first object set and the second object set, wherein the third object is any object that belongs to the first object set and the second object set and whose corresponding first detail level is lower than the third detail level, and the fourth object is any object that belongs to the second object set and does not belong to the first object set; accordingly, the second data set only includes the second model data of all the third objects and / or the fourth objects corresponding to the third detail level.

[0180] In an embodiment of the present invention, the first processing unit 402 in the data transmission device 400 can be implemented by a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU) or a programmable gate array (FPGA) in the network device in actual applications; the first communication unit 401 in the data transmission device 400 can be implemented by a communication module (including: basic communication kit, operating system, communication module, standardized interface and protocol, etc.) and a transceiver antenna in actual applications.

[0181] An embodiment of the present invention further provides a data transmission device, which is applied in a terminal. Fig. 9 The structure diagram of the data transmission device according to the embodiment of the present invention is shown in FIG. Figure 2 ,like Fig. 9 As shown, the data transmission device 500 includes a second communication unit 501,

[0182] The second communication unit 501 is used to send a first request to a network device, wherein the first request is used to request three-dimensional model data; the first request includes first information, wherein the first information at least includes a model clarity level of each object in a first object set under a first perspective of a user; and, receiving second information sent by the network device, and generating a three-dimensional model of each object in the first object set based on the second information; the second information includes a first data set, wherein the first data set includes first model data corresponding to a first detail level for each object in the first object set; the first detail level is determined by the network device based on the first information; wherein each object has three-dimensional model data of multiple detail levels.

[0183] In an optional embodiment of the present invention, the device also includes a second processing unit, which is used to determine the first object set based on the first perspective, and determine the first distance and the second distance of each object in the first object set, wherein the first distance is the vertical distance between the object and the viewpoint plane corresponding to the first perspective, and the second distance is the vertical distance between the object and a first straight line, and the first straight line is the straight line where the observation direction corresponding to the first perspective is located; and is also used to determine the model clarity level of each object based on the first distance and the second distance.

[0184] In an optional embodiment of the present invention, the first information further includes display capability information of the terminal; the display capability information is used by the network device to determine a detail level range that satisfies the display capability of the terminal.

[0185] In an optional embodiment of the present invention, the second communication unit 501 is also used to send a second request to the network device, the second request is used to request to update the three-dimensional model data; the second request includes third information, the third information includes the model clarity level of each object in the second object set under the user's second perspective; and, receive fourth information sent by the network device; the second processing unit is used to generate a three-dimensional model of each object in the second object set based on the fourth information; the fourth information includes a second data set, the second data set includes second model data corresponding to a third detail level of at least some objects in the second object set; the third detail level is determined by the network device based on the third information.

[0186] In an optional embodiment of the present invention, the second data set only includes second model data of all third objects and / or fourth objects corresponding to the third detail level, wherein the third object is any object belonging to the first object set and the second object set and the corresponding first detail level is lower than the third detail level, and the fourth object is any object belonging to the second object set and not belonging to the first object set.

[0187] In an optional embodiment of the present invention, the second communication unit 501 is further used to send the second request when the second perspective satisfies a first condition; wherein the first condition includes at least one of the following: an object under the second perspective is newly added relative to the object under the first perspective; a change in the observation distance under the second perspective relative to the observation distance under the first perspective exceeds a first value; a change in the observation direction under the second perspective relative to the observation direction under the first perspective exceeds a second value.

[0188] In an optional embodiment of the present invention, the second processing unit is further used to determine the second object set based on the second perspective, and determine a third distance and a fourth distance of each object in the second object set, wherein the third distance is a vertical distance between the object and the viewpoint plane corresponding to the second perspective, and the fourth distance is a vertical distance between the object and a second straight line in the observation direction, and the second straight line is a straight line in the observation direction corresponding to the second perspective; and, determine a model clarity level of each object based on the third distance and the fourth distance.

[0189] In an embodiment of the present invention, the second processing unit in the data transmission device 500 can be implemented by the CPU, DSP, MCU or FPGA in the terminal in actual applications; the second communication unit 501 in the data transmission device 500 can be implemented by a communication module (including: basic communication kit, operating system, communication module, standardized interface and protocol, etc.) and a transceiver antenna in actual applications.

[0190] It should be noted that: the data transmission device provided in the above embodiment only uses the division of the above program modules as an example when performing data transmission. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the data transmission device provided in the above embodiment and the data transmission method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0191] Fig.10 6 is a schematic diagram of the structure of a communication device according to an embodiment of the present invention. The communication device 600 may be a network device or a terminal according to the aforementioned embodiment. Fig.10 The communication device 600 shown includes: at least one processor 601, a memory 602 and at least one network interface 603. The various components in the communication device 600 are coupled together via a bus system 604. It is understood that the bus system 604 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 604 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 604 is not described in detail. Fig.10 Various buses are labeled as bus system 604 .

[0192] It can be understood that the memory 602 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 602 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memories.

[0193] The memory 602 in the embodiment of the present invention is used to store various types of data to support the operation of the communication device 600. Examples of such data include: any computer program used to operate on the communication device 600, such as a program for implementing the method of the embodiment of the present invention.

[0194] The method disclosed in the above embodiment of the present invention can be applied to the processor 601, or implemented by the processor 601. The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 601 or the instruction in the form of software. The above processor 601 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 601 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 602. The processor 601 reads the information in the memory 602 and completes the steps of the above method in combination with its hardware.

[0195] In an exemplary embodiment, the communication device 600 may be implemented by at least one application specific integrated circuit (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic component to execute the aforementioned method.

[0196] In an exemplary embodiment, the embodiment of the present invention further provides a computer-readable storage medium, such as a memory 602 including a computer program, and the computer program can be executed by a processor 601 of a communication device 600 to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, FlashMemory, magnetic surface memory, optical disk, or CD-ROM; or it can be various devices including one or any combination of the above memories, such as a computer, a tablet device, etc.

[0197] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0198] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0199] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0200] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0201] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0202] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0203] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0204] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0205] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A data transmission method, characterized in that: The method comprises: The network device receives a first request sent by the terminal, the first request being used to request three-dimensional model data; the first request includes first information, the first information at least including a model clarity level of each object in a first object set under a first viewing angle of a user; Determining a first level of detail corresponding to each object in the first object set based on the first information; wherein each object has three-dimensional model data of multiple levels of detail; Sending second information to the terminal, the second information includes a first data set; the first data set includes first model data corresponding to the first detail level for each object in the first object set; the second information is used by the terminal to generate a three-dimensional model for each object in the first object set.

2. The method according to claim 1, characterized in that The determining, based on the first information, a first level of detail corresponding to each object in the first object set includes: Determine a second level of detail corresponding to each object in the first object set based on a first mapping relationship and the first information; wherein the first mapping relationship includes levels of detail corresponding to different model clarity levels; The first detail level corresponding to each object in the first object set is determined according to the network resource information and the second detail level corresponding to each object in the first object set.

3. The method according to claim 2, characterized in that The determining, according to the network resource information and the second detail level corresponding to each object in the first object set, the first detail level corresponding to each object includes: Determining a data volume threshold for data transmission based on the network resource information; When the amount of the three-dimensional model data corresponding to the second detail level of all objects in the first object set does not exceed the data amount threshold, determining the second detail level corresponding to each object as the first detail level corresponding to each object; When the data amount of the three-dimensional model data corresponding to the second detail level of all objects in the first object set exceeds the data amount threshold, the second detail level of at least one object in the first object set is lowered to determine the first detail level corresponding to each object; wherein the data amount of the three-dimensional model data corresponding to the first detail level of all objects in the first object set after the detail level is lowered does not exceed the data amount threshold.

4. The method according to claim 3, characterized in that The step of lowering the second detail level of at least one object in the first object set comprises: When the second detail levels of the first object and the second object are the same, alternately lowering the second detail level of the first object and the second detail level of the second object according to the model clarity level; When the model definition levels of the first object and the second object are different and the second detail levels of the first object and the second object are different, continuously lowering the second detail level of the first object and / or the second detail level of the second object according to the detail level order, wherein the second detail level of the object with the lower model definition level is lowered preferentially; The first object and the second object are any two objects in the first object set.

5. The method according to any one of claims 1 to 4, characterized in that: The first information further includes display capability information of the terminal; and determining a first detail level corresponding to each object in the first object set based on the first information includes: determining a detail level range for each object according to the display capability information; A first level of detail corresponding to each object in the first object set is determined in the level of detail range based on the first information.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The network device receives a second request sent by the terminal, the second request being used to request updating of the three-dimensional model data; the second request includes third information, the third information including a model clarity level of each object in the second object set under a second viewing angle of the user; Determine a third level of detail corresponding to each object in the second object set based on the third information; Sending fourth information to the terminal, the fourth information including a second data set, the second data set including second model data corresponding to the third detail level of at least some objects in the second object set; the fourth information is used by the terminal to generate a three-dimensional model of each object in the second object set.

7. The method according to claim 6, characterized in that Before sending the fourth information to the terminal, the method further includes: Determine a third object and / or a fourth object according to the first object set and the second object set, wherein the third object is any object that belongs to the first object set and the second object set and whose corresponding first detail level is lower than the third detail level, and the fourth object is any object that belongs to the second object set and does not belong to the first object set; Correspondingly, the second data set only includes the second model data of all the third objects and / or the fourth objects corresponding to the third level of detail.

8. A data transmission method, characterized in that: The method comprises: The terminal sends a first request to the network device, the first request being used to request three-dimensional model data; the first request includes first information, the first information at least including a model clarity level of each object in a first object set under a first perspective of a user; Receive second information sent by the network device, and generate a three-dimensional model of each object in the first object set based on the second information; the second information includes a first data set, and the first data set includes first model data corresponding to a first detail level for each object in the first object set; the first detail level is determined by the network device based on the first information; wherein each object has three-dimensional model data of multiple detail levels.

9. The method according to claim 8, characterized in that Before the terminal sends the first request to the network device, the method further includes: The terminal determines the first object set based on the first viewing angle, and determines a first distance and a second distance of each object in the first object set, wherein the first distance is a vertical distance between the object and a viewpoint plane corresponding to the first viewing angle, and the second distance is a vertical distance between the object and a first straight line, and the first straight line is a straight line where an observation direction corresponding to the first viewing angle is located; A model clarity level of each object is determined based on the first distance and the second distance.

10. The method according to claim 8, characterized in that The method further comprises: The terminal sends a second request to the network device, where the second request is used to request to update the three-dimensional model data; the second request includes third information, where the third information includes a model clarity level of each object in the second object set under the second perspective of the user; Receive fourth information sent by the network device, and generate a three-dimensional model of each object in the second object set based on the fourth information; the fourth information includes a second data set, and the second data set includes second model data corresponding to a third detail level of at least some objects in the second object set; the third detail level is determined by the network device based on the third information.

11. The method according to claim 10, characterized in that The second data set only includes second model data corresponding to the third detail level of all third objects and / or fourth objects, wherein the third object is any object that belongs to the first object set and the second object set and whose corresponding first detail level is lower than the third detail level, and the fourth object is any object that belongs to the second object set and does not belong to the first object set.

12. The method according to claim 10, characterized in that The terminal sending a second request to the network device includes: The terminal sends the second request when the second viewing angle satisfies a first condition; wherein the first condition includes at least one of the following: The object under the second viewing angle is newly added relative to the object under the first viewing angle; The change of the observation distance at the second viewing angle relative to the observation distance at the first viewing angle exceeds a first value; A change in the observation direction at the second viewing angle relative to the observation direction at the first viewing angle exceeds a second value.

13. The method according to any one of claims 10 to 12, characterized in that: Before the terminal sends the second request to the network device, the method further includes: The terminal determines the second object set based on the second viewing angle, and determines a third distance and a fourth distance of each object in the second object set, wherein the third distance is a vertical distance between the object and a viewpoint plane corresponding to the second viewing angle, and the fourth distance is a vertical distance between the object and a second straight line in an observation direction, and the second straight line is a straight line where the observation direction corresponding to the second viewing angle is located; A model clarity level of each object is determined based on the third distance and the fourth distance.

14. A data transmission device, characterized in that: The device is applied to a network device, and comprises a first communication unit and a first processing unit; wherein, The first communication unit is configured to receive a first request sent by a terminal, the first request being used to request three-dimensional model data; the first request includes first information, the first information at least including a model clarity level of each object in a first object set under a first viewing angle of a user; The first processing unit is configured to determine a first level of detail corresponding to each object in the first object set based on the first information; wherein each object has three-dimensional model data of multiple levels of detail; The first communication unit is also used to send second information to the terminal, the second information including a first data set; the first data set includes first model data corresponding to the first detail level for each object in the first object set; the second information is used by the terminal to generate a three-dimensional model for each object in the first object set.

15. A data transmission device, characterized in that: The device is applied in a terminal, and comprises a second communication unit. The second communication unit is used to send a first request to the network device, wherein the first request is used to request three-dimensional model data; the first request includes first information, and the first information at least includes a model clarity level of each object in a first object set under a first perspective of a user; And, receiving second information sent by the network device, and generating a three-dimensional model of each object in the first object set based on the second information; the second information includes a first data set, and the first data set includes first model data corresponding to a first detail level for each object in the first object set; the first detail level is determined by the network device based on the first information; wherein each object has three-dimensional model data of multiple detail levels.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented; or, when the program is executed by a processor, the steps of the method described in any one of claims 8 to 13 are implemented.

17. A communication device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method described in any one of claims 1 to 7 are implemented; or, when the processor executes the program, the steps of the method described in any one of claims 8 to 13 are implemented.

Images