A data sending method, receiving method and device

Abstract

The present application discloses a data sending method, a receiving method and a device. The sending method includes: a network device obtains the location information and movement attribute information of a first terminal device at a first moment, where the movement attribute information includes the movement speed and direction of the first terminal device; determines a target area according to the location information and movement attribute information of the first terminal device, where the target area is the area predicted to be reached by the first terminal device at a second moment, the network device obtains first data corresponding to the target area, and sends the first data to the first terminal device, where the first data includes a target anchor point set and a target object set, and the target anchor point set is used to mark at least one 3D object that constitutes the target object set; in this method, the network device pre-distributes the anchor point information and 3D objects corresponding to the target area to the terminal device, thereby saving the time for the user to wait for the 3D object to be loaded and improving the user experience.

Description

Technical Field

[0001] This application relates to the field of terminals, and in particular, to a data sending method, a receiving method, and a device based on an anchor. Background Art

[0002] An anchor can be used to locate the position and size of an object. In an Augmented Reality (AR) environment, a user identifies a set of feature points of an object in front of the camera of a mobile terminal through the camera of the mobile terminal. This set of feature points can uniquely mark the features of the real environment. After the mobile phone captures the anchor information using the camera, it sends the information to a cloud network device, such as a cloud server or an AR Cloud. The cloud network device can determine the position and pose of the mobile phone based on this anchor information. At the same time, other virtual objects can also be loaded based on the anchor information, and the positions and poses of these virtual objects in the real environment can be set, enabling the user to obtain more information about the currently scanned object. Among them, the virtual object is a three-dimensional (3D) object developed by a developer through a tool. The position and pose of the 3D object depend on the anchor. The mobile phone first identifies at least one anchor through the camera, and then loads the 3D object (or 3D object image) on the display screen of the mobile phone based on these anchors, so that the position and pose of the 3D object are fixed relative to all the anchors, thus achieving the effect of truly projecting the 3D object into the real environment.

[0003] The cloud server or the AR Cloud maintains the anchor information received from the terminal device, and the stored anchor information can also facilitate the sharing of 3D objects among different mobile phone users. For example, user A identifies an anchor through the mobile phone, places a 3D object based on the anchor, and sends the anchor and the set of feature points of the 3D object to the cloud server or the AR Cloud. After user B identifies the same anchor through another mobile phone, user B can load the 3D object or 3D object image placed by user A from the cloud, thus realizing the sharing of 3D objects.

[0004] However, in the process of user B realizing the sharing of 3D objects mentioned above, a series of method steps are required, such as the mobile phone scanning the anchor, identifying the anchor information and object feature information, uploading the anchor information and feature information, the cloud server matching the anchor, and sending down the anchor and the 3D object. As a result, the user may wait for a relatively long time, such as at least 5 seconds (s), in front of the currently viewed object before obtaining the 3D object image corresponding to the object, and the user experience is poor. Summary of the Invention

[0005] This application provides a data sending method, a receiving method, and a device for reducing the waiting time for a user to load a 3D object image. This method can pre-load the 3D object image, thereby improving the user experience. Specifically, the following technical solutions are disclosed in this application:

[0006] In a first aspect, the present application provides a data sending method, the method comprising: a network device obtaining position information and movement attribute information of a first terminal device at a first moment, the movement attribute information including the movement speed and direction of the first terminal device; the network device determining a target area according to the position information and movement attribute information of the first terminal device, the target area being an area predicted to be reached by the first terminal device at a second moment, the second moment being the next moment after the first moment; the network device obtaining first data corresponding to the target area, and sending the first data to the first terminal device.

[0007] Wherein, the first data includes a target anchor point set and a target object set, the target area includes at least one anchor point, the target anchor point set is composed of the at least one anchor point, the at least one anchor point is used to mark at least one 3D object in the target area, and the at least one 3D object forms the target object set.

[0008] In the method provided in this embodiment, the cloud server uses the position and movement attributes of the terminal device to predict in advance the target area that the user will reach, and then sends the anchor point information and 3D objects corresponding to the target area to the terminal device in advance, so that the terminal device can pre-load the 3D object information to be displayed, avoiding the user starting to scan the current environment in the target area, and going through a series of operation processes such as identifying environmental feature information, uploading feature information, successfully matching anchor points in the cloud, and loading. This method saves the time for the user to wait for the 3D object to be loaded and improves the user experience.

[0009] In combination with the first aspect, in a possible implementation manner of the first aspect, the network device obtaining the first data corresponding to the target area includes: the network device obtaining the correspondence between at least one area and at least one anchor point set, the at least one area including the target area; the network device searching in the correspondence for the target anchor point set associated with the target area; the network device determining the target object set according to the 3D objects marked by each anchor point in the target anchor point set.

[0010] This implementation manner uses at least one target anchor point to mark 3D objects, and thus can use the scanned target anchor points to determine the target object set, thereby improving the search efficiency.

[0011] In combination with the first aspect, in another possible implementation of the first aspect, after the network device obtains the first data corresponding to the target area, it further includes: the network device filters out second data from the first data according to the context information of the first terminal device, and the second data includes the target anchor point set and a part of the target object set; the context information includes one or more of: user identifier, device type, device capabilities, and cache size.

[0012] When the network device sends the first data to the first terminal device, it includes: the network device sends the second data to the first terminal device.

[0013] In combination with the first aspect, in yet another possible implementation of the first aspect, the network device filters out second data from the first data according to the context information of the first terminal device, including one or more of the following combinations:

[0014] The network device deletes the 3D objects that the user does not have access to from the target object set according to the user identifier in the context information, and obtains the remaining 3D object set, and the user identifier is used to indicate whether the user has access to each 3D object;

[0015] Alternatively, the network device filters out the 3D object set suitable for the device type from the target object set according to the device type in the context information;

[0016] Alternatively, the network device filters out the 3D object set suitable for the device capabilities from the target object set according to the device capabilities in the context information, and the device capabilities include the level of detail of the device for rendering 3D objects;

[0017] Alternatively, the network device filters out the 3D object set whose storage capacity does not exceed the cache size of the first terminal device from the target object set according to the cache size in the context information.

[0018] This implementation uses the context information of the first terminal device to further screen the found target anchor point set and target object set, and excludes the 3D objects that do not meet the user characteristics or context, thereby saving storage space, reducing transmission delay, and further improving the user experience.

[0019] In combination with the first aspect, in another possible implementation manner of the first aspect, the network device obtains the location information of the first terminal device at the first moment, including: the network device receives the first anchor point information sent by the first terminal device at the first moment; the network device looks up in the anchor point database whether the first anchor point information is stored, and if so, determines the location information of the first terminal device according to the first anchor point information, and the anchor point database includes the anchor point information of at least one anchor point.

[0020] In this implementation manner, the anchor point database stored by the network device can quickly find the first anchor point information, thereby determining the location of the first terminal device, improving the data search and sending efficiency.

[0021] It should be understood that the location information of the first terminal device can also be obtained through other means, such as obtaining the location information of the first terminal device in real time through GPS positioning technology.

[0022] In combination with the first aspect, in another possible implementation manner of the first aspect, the network device obtains the correspondence between at least one area and at least one anchor point set, including: the network device receives the scan information sent by the second terminal device, and the scan information includes the anchor point information of at least one anchor point scanned by the second terminal device in the current area and the 3D objects marked by each of the anchor points; the network device obtains the correspondence between the current area and the anchor point information of at least one anchor point in the current area according to the scan information.

[0023] In addition, it further includes that the network device stores the correspondence.

[0024] In this implementation manner, by pre-scanning the surrounding environment and setting anchor point information, a reference basis is provided for marking 3D objects, preparing for subsequent data search and sending.

[0025] In the second aspect, the present application further provides a data receiving method, and the method includes: the first terminal device receives the first data sent by the network device, and when the first terminal device enters the target area and scans the target anchor point set in the first data, the target object set is displayed on the first terminal device.

[0026] Wherein, the first data includes a target anchor point set and a target object set, the target anchor point set is composed of at least one anchor point in the target area, the at least one anchor point is used to mark at least one 3D object in the target area, the at least one 3D object forms the target object set, and the target area is the area predicted that the first terminal device will reach at the second moment.

[0027] Optionally, in a possible implementation, the first terminal device obtains at least one anchor point information by scanning the current surrounding environment, and compares whether the anchor point information matches the anchor point information of the target device pre-stored. If so, at least one target 3D object is determined according to the corresponding relationship between the pre-established target anchor point set and the target object set.

[0028] Combined with the second aspect, in a possible implementation manner of the second aspect, before the first terminal device receives the first data sent by the network device, it further includes: the first terminal device scans the external environment to obtain first anchor point information, where the first anchor point information includes the anchor point information of at least one anchor point included in the scanned external environment; the first terminal device sends the first anchor point information to the network device at the first moment, and the first anchor point information is used to determine the position information of the first terminal device.

[0029] Combined with the second aspect, in another possible implementation manner of the second aspect, the method further includes: the first terminal device sends the context information of the first terminal device to the network device, and the context information is used to screen out second data from the first data, and the second data includes a part of the target anchor point set and the target object set.

[0030] Combined with the second aspect, in yet another possible implementation manner of the second aspect, the context information includes one or more of: user identifier, device type, device capability, and cache size;

[0031] If the context information includes a user identifier, then a part of the target object set in the second data includes: deleting 3D objects that the user does not have access rights from the target object set to obtain the remaining 3D object set, and the user identifier is used to indicate whether the user has access rights to each 3D object;

[0032] If the context information includes a device type, then a part of the target object set in the second data includes: a 3D object set suitable for the device type in the target object set;

[0033] If the context information includes a device capability, then a part of the target object set in the second data includes: a 3D object set suitable for the device capability in the target object set, where the device capability includes the detail level of the device for rendering 3D objects;

[0034] If the context information includes a cache size, then a part of the target object set in the second data includes: a 3D object set whose storage capacity in the target object set does not exceed the cache size of the first terminal device.

[0035] In a third aspect, the present application further provides a data sending device, which can be applied to a network device. The device includes:

[0036] An acquisition unit, configured to acquire the position information and movement attribute information of the first terminal device at a first moment, where the movement attribute information includes the movement speed and direction of the first terminal device; a processing unit, configured to determine a target area according to the position information and movement attribute information of the first terminal device, and acquire first data corresponding to the target area; a sending unit, configured to send the first data to the first terminal device.

[0037] Wherein, the target area is an area predicted to be reached by the first terminal device at a second moment, the second moment is the next moment of the first moment, the first data includes a target anchor point set and a target object set, the target area includes at least one anchor point, the target anchor point set is composed of the at least one anchor point, and the at least one anchor point is used to mark at least one 3D object in the target area, and the at least one 3D object forms the target object set.

[0038] In combination with the third aspect, in a possible implementation manner of the third aspect, the processing unit is further configured to acquire the correspondence between at least one area and at least one anchor point set, and search for the target anchor point set associated with the target area in the correspondence; and determine the target object set according to the 3D objects marked by each anchor point in the target anchor point set; the at least one area includes the target area.

[0039] In combination with the third aspect, in another possible implementation manner of the third aspect, after acquiring the first data corresponding to the target area, the processing unit is further configured to screen out second data from the first data according to the context information of the first terminal device, and the second data includes a part of the target anchor point set and the target object set; the context information includes one or more of: user identifier, device type, device capability, and cache size; the sending unit is further configured to send the second data to the first terminal device.

[0040] In combination with the third aspect, in still another possible implementation manner of the third aspect, the processing unit is further configured to delete, according to the user identifier in the context information, the 3D objects that the user does not have access rights to in the target object set, and obtain the remaining 3D object set, where the user identifier is used to indicate whether the user has access rights to each 3D object;

[0041] Alternatively, according to the device type in the context information, filter out a set of 3D objects suitable for the device type from the set of target objects;

[0042] Alternatively, according to the device capabilities in the context information, filter out a set of 3D objects suitable for the device capabilities from the set of target objects, where the device capabilities include the level of detail for the device to render 3D objects;

[0043] Alternatively, according to the cache size in the context information, filter out a set of 3D objects whose storage capacity does not exceed the cache size of the first terminal device from the set of target objects.

[0044] Combined with the third aspect, in another possible implementation manner of the third aspect, it further includes a receiving unit, and the receiving unit is further configured to receive first anchor point information sent by the first terminal device at the first moment; the processing unit is further configured to check whether the first anchor point information is stored in the anchor point database, and if so, determine the location information of the first terminal device according to the first anchor point information, and the anchor point database includes anchor point information of at least one anchor point.

[0045] Combined with the third aspect, in another possible implementation manner of the third aspect, the receiving unit is further configured to receive scan information sent by the second terminal device, where the scan information includes anchor point information of at least one anchor point scanned by the second terminal device in the current area and 3D objects marked by each of the anchor points; the processing unit is further configured to obtain the correspondence between the current area and the anchor point information of at least one anchor point in the current area according to the scan information.

[0046] In a fourth aspect, the present application further provides a data receiving device, which can be applied to a first terminal device, such as a UE. The device includes: a receiving unit, configured to receive first data sent by a network device, and a processing unit, configured to display the set of target objects on the first terminal device when the first terminal device enters the target area and scans the set of target anchor points in the first data.

[0047] Wherein, the first data includes: a set of target anchor points and a set of target objects. The set of target anchor points consists of at least one anchor point in the target area, and the at least one anchor point is used to mark at least one 3D object in the target area. The at least one 3D object constitutes the set of target objects, and the target area is the area predicted to be reached by the first terminal device at the second moment.

[0048] Combined with the fourth aspect, in a possible implementation manner of the fourth aspect, it further includes a sending unit,

[0049] The processing unit is further configured to scan the external environment to obtain first anchor information before the receiving unit receives the first data, where the first anchor information includes the anchor information of at least one anchor included in the scanned external environment; the sending unit is configured to send the first anchor information to the network device at the first moment, and the first anchor information is used to determine the location information of the first terminal device.

[0050] In combination with the fourth aspect, in another possible implementation manner of the fourth aspect, the sending unit is configured to send the context information of the first terminal device to the network device, and the context information is used to screen out second data from the first data, and the second data includes a part of the target anchor set and the target object set.

[0051] Optionally, the context information includes one or more of: user identifier, device type, device capability, and cache size;

[0052] In combination with the fourth aspect, in another possible implementation manner of the fourth aspect, the processing unit is further configured to, if the context information includes a user identifier, delete the 3D objects that the user does not have access rights to in the target object set to obtain the remaining 3D object set, where the user identifier is used to indicate whether the user has access rights to each 3D object; if the context information includes a device type, obtain the 3D object set suitable for the device type in the target object set; if the context information includes a device capability, obtain the 3D object set suitable for the device capability in the target object set, where the device capability includes the detail level of the device for rendering 3D objects; if the context information includes a cache size, obtain the 3D object set whose storage capacity does not exceed the cache size of the first terminal device in the target object set.

[0053] In a fifth aspect, the present application further provides a data transmission system, where the system includes at least one terminal device and a network device, where the at least one terminal device includes a first terminal device.

[0054] Further, the first terminal device includes the data receiving device in the foregoing fourth aspect and any implementation manner of the fourth aspect; the network device includes the data sending device in the foregoing third aspect and any implementation manner of the third aspect.

[0055] In addition, the system further includes a second terminal device, which sends scan information to the network device. The scan information includes the anchor information of at least one anchor scanned by the second terminal device in the current area and the 3D objects marked by each of the anchors. The network device receives the scan information sent by the second terminal device and obtains the correspondence between the current area and the anchor information of at least one anchor in the current area according to the scan information.

[0056] In a sixth aspect, the present application further provides a communication device, which includes at least one processor and a memory. The memory is used to store instructions provided by the at least one processor. The at least one processor is used to execute the instructions to implement the methods in the foregoing first aspect and various implementation manners of the first aspect, or execute the methods in the foregoing second aspect and various implementation manners of the second aspect.

[0057] Optionally, the processor and the memory may be integrated in a chip system or a chip circuit. The chip system or the chip circuit further includes an input / output interface, which is used to implement communication between the chip system / chip circuit and other external modules.

[0058] Optionally, the processor is a logic circuit.

[0059] Optionally, the network device or network node is a cloud server, a server, an AR Cloud, etc.

[0060] In a seventh aspect, the present application further provides a computer-readable storage medium, in which instructions are stored such that when the instructions are run on a computer or a processor, they can be used to execute the methods in the foregoing first aspect and various implementation manners of the first aspect, or execute the methods in the foregoing second aspect and various implementation manners of the second aspect.

[0061] In addition, the present application further provides a computer program product, which includes computer instructions. When the instructions are executed by a computer or a processor, the methods in the foregoing first aspect and various implementation manners of the first aspect, and the methods in the foregoing second aspect and various implementation manners of the second aspect can be implemented.

[0062] It should be noted that the beneficial effects corresponding to the technical solutions of the various implementation manners of the foregoing second aspect to the seventh aspect are the same as those of the foregoing first aspect and various implementation manners of the first aspect. For specific reference, see the description of the beneficial effects in the foregoing first aspect and various implementation manners of the first aspect, and details will not be repeated. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1Schematic structural diagram of a wireless communication system provided by this application;

[0064] Figure 2 Schematic structural diagram of a terminal device provided by this application;

[0065] Figure 3 Schematic flow diagram of the process for obtaining object position information and virtual object sharing implemented by Google in this application;

[0066] Figure 4 Flow chart of a data sending method provided by this application;

[0067] Figure 5 Schematic diagram of predicting the target area where a user will move at the next moment in an indoor venue provided by this application;

[0068] Figure 6 Flow chart of another data sending method provided by this application;

[0069] Figure 7 Schematic structural diagram of a multi-functional entertainment venue provided by this application;

[0070] Figure 8 Signaling flow chart of a data transmission method provided by this application;

[0071] Figure 9 Schematic structural diagram of a data sending device provided by this application;

[0072] Figure 10 Schematic structural diagram of a network device provided by this application. Detailed implementation manners

[0073] In order to enable those skilled in the art of this technology to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0074] Before describing the technical solutions of the embodiments of the present application, the application scenarios of the embodiments of the present application will be described with reference to the accompanying drawings first. The technical solutions of the present application can be applied to various communication systems, such as Wireless Local Area Network (WLAN), global system for mobile communications (GSM) system, code division multiple access (CDMA) system, wide band code division multiple access (WCDMA) system, general packet radio service (GPRS), long term evolution (LTE) system, LTE frequency division duplex (FDD) system, LTE time division duplex (TDD), universal mobile telecommunication system (UMTS), world wide interoperability for microwave access (WiMAX) communication system, future fifth generation (5G) system or new radio (NR), etc. As Figure 1 shown, in any of the above communication systems, it includes: terminal devices carried by at least one user and cloud network devices, as well as at least one anchor point and objects, environments, etc. located by the anchor point.

[0075] The environment includes: public viewing scenarios such as exhibitions, museums, exhibition halls, venues, etc. In addition, it can also include the vehicle - to - everything (V2X) in the vehicle - to - everything network, such as scenarios in the fields of autonomous driving and driverless driving, such as parking lots, shopping malls, etc.

[0076] Among them, the terminal device can be a portable device, such as a smart terminal, a mobile phone, a laptop computer, a tablet computer, a personal computer (PC), a personal digital assistant (PDA), a foldable terminal, a wearable device with wireless communication function (such as a smart watch or bracelet), a user device (UE), a smart home device (such as a TV), a car, a motorcycle helmet, an in-vehicle computer, a game console, and an augmented reality (AR) / virtual reality (VR) device, etc. The embodiments of the present application do not limit the specific device form of the terminal device. In addition, the above various terminal devices include but are not limited to those equipped with Apple (IOS), Android, Microsoft, or other operating systems.

[0077] Figure 2 is a schematic diagram of the hardware structure of the terminal device provided by the embodiments of the present application. As Figure 2 shown, the terminal device 100 may include a processor 110, a memory 120, a universal serial bus (USB) interface 130, a radio frequency circuit 140, a mobile communication module 150, a wireless communication module 160, a camera 170, a display screen 180, a SIM card interface 190, a touch sensor 200, a pressure sensor 210, and a key 220, etc.

[0078] Among them, the processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modulation and demodulation processor, a graphics processing unit (GPU), an image signal processor (ISP), a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processor (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors, such as integrated in a system on a chip (SoC). A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory may store the instructions or data that the processor 110 has just used or recycled.

[0079] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface 190, and / or a USB interface 130, etc.

[0080] The memory 120 may be used to store computer-executable program code, and the executable program code includes instructions. The memory 120 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area may store data created during the use of the terminal device 100 (such as audio data, a phone book). In addition, the memory 120 may include one or more storage units. For example, it may include a volatile memory, such as a dynamic access memory (RAM), and may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, etc. The processor 110 executes various functional applications and data processing of the terminal device 100 by running the instructions stored in the memory 120 and / or the instructions stored in the memory provided in the processor.

[0081] The wireless communication function of the terminal device 100 may be implemented through a radio frequency circuit 140, a mobile communication module 150, a wireless communication module 160, a modulation and demodulation processor, and a baseband processor, etc.

[0082] The radio frequency circuit 140 may include at least one antenna 141 for transmitting and receiving electromagnetic wave signals. Each antenna in the terminal device 100 can be used to cover a single or multiple communication frequency bands. In some embodiments, the antenna can be used in combination with a tuning switch.

[0083] The mobile communication module 150 can provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the terminal device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 141, filter and amplify the received electromagnetic waves, and then transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through the antenna 141 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 can be provided in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be provided in the same device.

[0084] The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (including but not limited to a speaker, a receiver, etc.), or displays an image or video through the display screen 180. In some embodiments, the modulation and demodulation processor can be an independent device. In other embodiments, the modulation and demodulation processor can be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0085] The wireless communication module 160 may include a wireless fidelity (WiFi) module, a bluetooth (BT) module, a GNSS module, a near field communication (NFC) module, an infrared (IR) module, etc. The wireless communication module 160 can be one or more devices integrating at least one of the above modules. The wireless communication module 160 receives electromagnetic waves via the antenna 141, frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be transmitted from the processor 110, frequency-modulate and amplify it, and convert it into electromagnetic waves through the antenna 141 for radiation.

[0086] In the embodiments of the present application, the wireless communication functions of the terminal device 100 may include, for example, Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), 5th Generation Mobile Networks New Radio (5G NR), BT, GNSS, WLAN, NFC, FM, and / or IR, etc. GNSS may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), Beidou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0087] The camera 170 is used to capture static images or videos. The camera 170 includes a lens and a photosensitive element. An object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a Complementary Metal-Oxide-Semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard RGB, YUV, RYYB, etc. formats. In some embodiments, the terminal device 100 may include one or N cameras 170, where N is a positive integer greater than 1.

[0088] The NPU is a neural-network (NN) computing processor. By learning from the structure of biological neural networks, such as the transmission pattern between human brain neurons, it can quickly process input information and can also continuously learn by itself. Through the NPU, applications such as intelligent cognition of the terminal device 100 can be realized, such as image recognition, face recognition, voice recognition, etc.

[0089] The display screen 180 is used to display images, videos, etc. The display screen 180 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include 1 or N display screens 180, where N is a positive integer greater than 1.

[0090] The touch sensor 200, also known as the "touch control device". The touch sensor 200 can be disposed on the display screen 180, and the touch sensor 200 and the display screen 180 form a touch screen, also known as the "touch control screen". The touch sensor 200 is used to detect touch operations acting on or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 180. In other embodiments, the touch sensor 200 can also be disposed on the surface of the terminal device 100, at a different position from the display screen 180. The pressure sensor 210 is used to measure the pressure value of the user's touch on the screen. In addition, other sensors can also be included, such as a gyroscope sensor, an acceleration sensor, a temperature sensor, etc.

[0091] The keys 220 include a power-on key, a volume key, etc. The keys 220 can be mechanical keys or touch keys. The terminal device 100 can receive key inputs and generate key signal inputs related to the user settings and function controls of the terminal device 100.

[0092] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown in the figures, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0093] In addition, the network device includes, but is not limited to, cloud servers, servers, data centers, computing units, ARCloud, etc., and the functions of this network device may be the same as or different from the structure of the terminal device 100 shown. This embodiment does not specifically limit the structure and form of the network device. Figure 2 The following describes the embodiments of cloud anchor scanning and virtual object sharing.

[0094] The following describes the embodiments of cloud anchor scanning and virtual object sharing.

[0095] Google ( ) has proposed a method for obtaining object position information and realizing virtual object sharing. Google's AR Core is an AR toolkit for mobile application developers and users, which includes the Cloud Anchor function, allowing developers to upload anchor point information to the cloud, and at the same time match the saved anchor points from the cloud through feature information to realize the sharing of position information and virtual objects. See Figure 3 , and the specific method includes:

[0096] Step 1: User A identifies the environmental feature information through a mobile phone, and anchor point information will be generated, such as including at least one anchor point included in the currently scanned environment, the position coordinates of each anchor point, etc.

[0097] Step 2: User A uploads the anchor point information to the cloud using the mobile phone.

[0098] Step 3: User A places a virtual object, such as a 3D object, based on the anchor point, and uploads the 3D object based on this anchor point to the cloud.

[0099] Step 4: User B uses another mobile phone to scan and identify the environment in the same environment to obtain the environmental feature information.

[0100] Step 5: User B uploads the scanned environmental feature information to the cloud using the mobile phone. The cloud matches the cloud anchor according to this feature information and sends the anchor point and the 3D object to User B's mobile phone.

[0101] Step 6: After receiving it, User B's mobile phone displays the 3D object created by User A according to the anchor point, realizing the sharing of the 3D object.

[0102] In the above method, when user B wants to share a 3D object, it is necessary to start from scanning the current environment (step four), and go through a series of processes such as identifying environmental feature information, uploading feature information, successfully matching an anchor point in the cloud, and sending the anchor point and the 3D object to user B's mobile phone. This often takes a long time to wait, for example, waiting for more than 5 seconds, which results in a poor experience for user B.

[0103] To reduce the waiting time of users and improve the user experience, this embodiment provides a method for sending anchor point information, as Figure 4 shown. This method is executed by a network device, such as a cloud server or an AR Cloud. Specifically, this method includes:

[0104] 101: The network device obtains the position information and movement attribute information of the first terminal device at the first moment.

[0105] The position information of the first terminal device is the coordinate position where the first terminal device is currently located at the first moment. The movement attribute information represents the movement ability of the first terminal device. The movement attribute information of the terminal device includes movement speed and direction. Specifically, the movement speed depends on the movement state when the user uses the device, such as walking, cycling, driving, etc. This movement speed and direction can be configured by the network device according to the user's current state. For example, if the user holds the first terminal device and walks in the museum to view the exhibits, the movement speed of the first terminal device can be determined according to the average walking speed of people indoors, and the movement direction can be measured by the gyroscope sensor of the first terminal device.

[0106] Optionally, in one implementation, the first moment can be when the network device detects that the first terminal device carried by the user enters the exhibition area, or it can also be when the network device detects that the first terminal device enters the first anchor point coverage area within the first area, and then performs the acquisition operation in step 101.

[0107] Or, in another possible implementation, the user turns on the scanning function of the first terminal device, such as starting the Cloud Anchor function APP, and starts scanning the current exhibits / items being visited. The first terminal device will automatically send a command to the network device in the cloud. The first moment is when the network device receives the command sent by the first terminal device, indicating that the user has turned on the scanning function to perform the scanning operation of the exhibits / items. At this time, the foregoing step 101 is executed.

[0108] It should be understood that the first moment can also be other time nodes, that is, as long as it is any moment before the user moves to the target area or scans the environment of the target area, it can be the foregoing first moment.

[0109] In addition, in step 101, the network device can communicate with the first terminal device in any one or more ways such as GPS, WiFi, Bluetooth, etc., to obtain the location information and movement attribute information of the first terminal device. In a possible implementation, the first terminal device captures environmental feature information using a camera and then uploads it to the network device. After receiving it, the network device determines the location information of the first terminal device through the environmental feature information. The location information of the first terminal device may be an outdoor location or an indoor location.

[0110] 102: The network device determines a target area according to the location information and movement attribute information of the first terminal device. The target area is the area predicted to be reached by the first terminal device at a second moment, and the second moment is the next moment after the first moment.

[0111] For example, the network device, such as a cloud server, pre-divides the space into multiple areas, and each area contains one or more anchor points. As Figure 5 shown, in the indoor scene of an exhibition hall, it is pre-divided into 6 areas, namely the first area to the sixth area. At least one exhibit is displayed in each area, and the location and size of each exhibit can be calibrated by at least one anchor point. For example, object 1 is located in the first area, and the size and location of object 1 are represented by at least one anchor point ( Figure 5 not shown).

[0112] Among them, the target area can be a preset location. For example, when a user browses a museum, they will browse according to the museum's route. Then, the target area can be determined according to the path planning of the museum. When the target area does not have a planned route, it can be determined according to at least one path between the user's current location and the final location. At this time, the target area can be one or more.

[0113] Furthermore, the network device determines the area that the first user may move to at the second moment according to the location, moving speed, and moving direction of the first terminal device carried by the user, such as a mobile phone. Let this area be the target area.

[0114] 103: Obtain first data corresponding to the target area. The first data includes a target anchor point set and a target object set.

[0115] Among them, the target area includes at least one anchor point. The target anchor point set is composed of the at least one anchor point. The at least one anchor point is used to mark at least one 3D object in the target area, and the at least one 3D object constitutes the target object set.

[0116] In addition, before step 103, it further includes: The network device obtains a target anchor point set and a target object set corresponding to the target area. One implementation includes: The network device obtains the correspondence between at least one area and at least one anchor point set, and the at least one area includes the target area and at least one target anchor point.

[0117] In step 102 above, after obtaining the target area, the network device searches for the target anchor point set associated with the target area in the correspondence; and determines the target object set according to the 3D objects marked by each anchor point in the target anchor point set.

[0118] For example, the target area is the second area, and there are two anchor points in this second area, namely anchor point 1 and anchor point 2. Among them, anchor point 1 is used to mark object 21, and anchor point 2 is used to mark object 22. Then the target anchor point set is determined as {anchor point 1, anchor point 2}, and the target object set is determined as {object 21, object 22}.

[0119] If the first terminal device is a mobile phone and the user holds the mobile phone and walks indoors, the cloud server can determine the target area that the user is about to enter according to the user's current position, moving speed, and moving direction. Then the cloud server filters out the relevant information of at least one anchor point and 3D object existing in the target area to obtain the target anchor point set and the target object set. If the first terminal device is a vehicle, the cloud server determines the target street that the vehicle is about to enter according to the vehicle's current position, moving direction, and moving speed, and then filters out in advance the target anchor point set and target object set corresponding to the target street.

[0120] In addition, the objects in the target object set are 3D objects, and each 3D object in the target object set contains all the information forming the 3D object, such as information and data like the 3D images of object 21 and object 22.

[0121] 104: The network device sends the first data to the first terminal device. Correspondingly, the first terminal device receives the first data.

[0122] 105: The first terminal device loads the first data and displays the target object set on the first terminal device.

[0123] Specifically, the first terminal device scans the surrounding environment where the user is located in real time. One or more anchors are included in the scanned surrounding environment. The first terminal device matches the scanned anchors with the pre-stored anchors and determines whether the current user has entered the target area according to the matching result. If they match, it is determined that the first terminal device is already located in the target area. And the anchors scanned by the first terminal are matched in real time, and the matched anchors are determined as the target anchors. A plurality of target anchors form the target anchor set. Also, since the first data includes the correspondence between the target anchor set and the target object set, the target object set can be determined, and the target object set is displayed on the display screen.

[0124] In an example, when the user moves to the target area, a 3D image of the target object will be automatically displayed on the display screen of the user's mobile phone. For example, if object 21 is a ceramic cup, a 3D stereoscopic image of the ceramic cup will be displayed on the mobile phone's display screen, or content of graphic analysis of the ceramic cup. Additionally, 3D image information of object 22 can also be displayed. For example, if object 22 is a portrait, 3D image information of the portrait will be displayed on the mobile phone's display screen, such as portrait story introduction, voice reading, etc. The specific display method of the target 3D object on the display screen of the first terminal device in this embodiment is not limited.

[0125] For the method provided in this embodiment, the cloud server uses the location and movement attributes of the terminal device to predict in advance the target area that the user will reach, and then sends the anchor information and 3D objects corresponding to the target area to the terminal device in advance, so that the terminal device can pre-load the 3D object information to be displayed, avoiding a series of operation processes for the user to start scanning the current environment in the target area, identifying environmental feature information, uploading feature information, successfully matching anchors in the cloud, and loading. This method saves the time for the user to wait for the 3D object to be loaded and improves the user experience.

[0126] Furthermore, in the above embodiment, before step 102, the method further includes: the first terminal device sends the context information of the first terminal device to the network device. Wherein, the context information of the first terminal device includes at least one of user identification, device type, device capability, and cache size.

[0127] Among them, the user identifier is used to uniquely mark the identity of the user using the terminal device. The cloud server or AR Cloud determines whether the current user has access rights to different 3D objects according to the user identifier. For example, the user identifier can mark whether the user identity is an "ordinary user" or a "VIP user"; or, according to the user's age, mark which age group the user belongs to as a visitor, such as "teenager", "middle-aged", "elderly", etc. Because for different user identifiers, different 3D object image information can be matched.

[0128] The device type is used to indicate which type of device the user is using. The device type includes but is not limited to mobile phones, tablets, PCs, AR / VR glasses, cars, motorcycle helmets, etc. Since the display effects of different 3D objects are different on different user terminals, there are some 3D objects that are specifically customized for a certain device type. For example, 3D objects customized for AR glasses.

[0129] The device capability represents the rendering capability of the terminal device for 3D objects. The rendering capability depends on the processing capabilities determined by the GPU, CPU, memory, etc. of the terminal device. The higher the device capability, the richer the detail level (abbreviation: "detail degree") of the objects that can be rendered. On the contrary, for some 3D objects with rich details and relatively large volumes, if an attempt is made to render them on a terminal device with weak rendering capabilities, it may cause phenomena such as overheating of the device and too long loading time. Therefore, it is necessary to distinguish and mark the capabilities of the terminal device.

[0130] The detail degree can be represented by the resolution of the target 3D object image to be displayed. The higher the resolution of the target 3D object image, the clearer the image and the richer the detail degree; conversely, the lower the resolution, the rougher the detail degree of the image. In addition, when configuring 3D object information with software, the 3D image to be displayed should be obtained as faithfully and detailed as possible, and there may be some areas where such detailed parts do not need to be faithfully stored. Therefore, in this example, 3D images with different detail degrees are provided through software modules, so that the resolution (pixels per inch) of the compressed image can be changed over the entire 3D object image area, and the desired resolution (detail degree) for each 3D image can be determined automatically in a way controlled by software and / or the user before compression. Based on this, the 3D object information in this embodiment can be stored in a lower resolution manner, or the less important image areas can be stored in a low resolution manner. Or, it is also possible to store a clear 3D object image in a high resolution.

[0131] The cache size represents the cache capacity of the terminal device, that is, the size of the storage space. If the cache capacity of the terminal device is larger, it means that the terminal device can store more data / information of 3D objects locally. Conversely, the smaller the cache capacity, the less 3D data / information can be stored. Therefore, different target 3D object sets are determined according to the cache size of the terminal device.

[0132] In this embodiment, the first terminal device can report the context information multiple times or report the context information of the first terminal device at one time. Specifically, the reporting method can be through wireless communication methods such as WiFi and Bluetooth.

[0133] See Figure 6 , after the network device obtains the first data corresponding to the target area in step 103 above, it further includes:

[0134] 104’: The network device filters out the second data from the first data according to the context information of the first terminal device, and the second data includes a part of the target anchor point set and the target object set.

[0135] Example 1:

[0136] The network device deletes the 3D objects that the user does not have access rights to in the target object set according to the user identifier in the context information, and obtains the remaining 3D object set. For example, if the user identifier indicates that the user is a "general user", then the exhibition rights of some 3D objects are not open to general users and are only open to VIP users. Then, the information of the 3D objects that are not open is blocked, and the 3D object set that is only open to "general users" is obtained, that is, the data information of the 3D objects that VIP users can access is excluded from the target object set.

[0137] Example 2:

[0138] The network device filters out the 3D object set suitable for the device type from the target object set according to the device type in the context information, and deletes the 3D objects that are not applicable to the first terminal device from the 3D object set. For example, if the device type of the first terminal device is an AR glasses, then only the 3D object set that VR glasses can recognize is retained, and the data information of the 3D objects of other types of terminal devices is excluded from the target object set.

[0139] Example 3:

[0140] Based on the device capabilities in the context information, the network device filters out a 3D object set that suits the device capabilities from the target object set. The device capabilities include the level of detail for the device to render 3D objects. For example, if the context information indicates that the rendering capability of the first terminal device is "high", then 3D objects with more details are selected as the target 3D object set, and 3D objects with blurred image features or without more details and rendering effects are blocked, or 3D objects that cannot be presented and rendered on the first terminal device are deleted from the target object set, so as to provide 3D objects with high-definition and high-detail rendering effects for the user.

[0141] Example 4:

[0142] Based on the cache size in the context information, the network device filters out a 3D object set whose storage capacity does not exceed the cache size of the first terminal device from the target object set. The network device determines that the remaining cache space in the current first terminal device is small according to the cache size and cannot load all the target object set and target anchor set in the target area. Then, it eliminates the data / information of at least one 3D object that is far from the first terminal device, and retains the data information of 3D objects that are close to the first terminal device, and the storage space occupied by the data information of these 3D objects that are close is less than the cache space size of the first terminal device.

[0143] In addition, the network device also determines the anchor set associated with these 3D objects. If there are many 3D object sets determined in step 103, a part of the 3D object sets needs to be selected according to the cache size of the user terminal. The cloud server can select several 3D objects in front in the order in which the first terminal device may contact the anchors, and requires that the storage capacity occupied by these 3D objects is less than the cache size.

[0144] 105’: The network device sends the second data to the first terminal device.

[0145] In the method provided in this embodiment, the network device further filters the target object set according to the context information of the first terminal device held by the user, and eliminates 3D objects that do not conform to the user identification, device type, device capabilities, and cache size, so that the remaining part of the target object set after filtering is more suitable for the first terminal device, thereby improving the user experience.

[0146] In addition, compared with transmitting all the first data, the second data obtained by filtering the first data is transmitted to the first terminal device, the data information of the transmitted 3D objects is reduced, and thus the data volume transmitted to the terminal device is reduced, and the transmission overhead is reduced.

[0147] It should be noted that in the process of determining the second data in step 104', it can be any combination of two or more of the above examples 1 to 4. For example, if the context information reported by the first terminal device to the network device includes all of the user identifier, device type, device capabilities, and cache size, then when screening the second data, each piece of information is sequentially judged whether it meets the conditions of the terminal device or the user, and then the intersection is taken from all the screening results to obtain the second data.

[0148] It should be understood that the context information of the terminal device may further include other parameter characteristics, and / or screening conditions, and this embodiment does not limit the specific process of screening out a part of the first data as the second data.

[0149] In a specific embodiment, as Figure 7 shown, it is a multi-functional entertainment venue, which includes three rooms: a living room, a toy room, and a lounge. Among them, several tables are placed in the toy room, and different toys are placed on each table. The activity organizer has pre-created several 3D toys in the toy room, and the position and size of each 3D toy can be marked by an anchor point.

[0150] See Figure 8 , a data transmission method provided in this embodiment includes the following steps:

[0151] 301: The organizer terminal device scans the current area, identifies at least one anchor point in the current area, creates 3D object information based on the at least one anchor point, and configures the 3D object information on the associated at least one anchor point.

[0152] Specifically, the organizer, such as UE2, first identifies 5 anchor points on the tables in the toy room through its own mobile phone. These 5 anchor points are respectively marked in the toy room as: anchor point 2, anchor point 3, anchor point 4, anchor point 5, and anchor point 6. Anchor point 1 and anchor point 7 are the anchor points set in the living room. The above anchor points 2 to 6 are used to mark the features such as the edges, corners, and decorations of 5 tables, and these anchor points are unique in the toy room. Based on the at least one scanned anchor point, UE2 sets 3D toy information according to the position and attitude of each anchor point, that is, the 3D toy virtual image. For example, the 3D toy virtual images marked by anchor points 2 to 6 include a toy Mercedes-Benz model, a dolphin doll, a ninja model, a doll, and a teddy bear, and the corresponding relationship shown in Table 1 is obtained.

[0153] Table 1. Corresponding relationship between anchor points and 3D objects

[0154]

[0155]

[0156] AsFigure 7 As shown, a 3D dolphin doll is set around Anchor Point 3, and thus the position and pose of the dolphin doll in the toy room can be marked through Anchor Point 3. Similarly, a doll is set around Anchor Point 5, so the position and pose of the doll are marked through Anchor Point 5. The 3D object information includes relevant data such as the position, pose, and size of the dolphin toy and the doll in the toy room. And a correspondence relationship is established between the 3D object information of Anchor Point 3 and the dolphin toy, and the 3D object information of Anchor Point 5 and the doll.

[0157] 302: UE2 sends the at least one anchor point information and the 3D object information to the cloud server. The anchor point information includes: the positions of all anchor points in the toy room and the anchor point coordinate systems established by each anchor point; the 3D object information includes the correspondence relationship between the at least one anchor point and the configured 3D object information.

[0158] Optionally, the above method further includes: UE2 sets filtering conditions for these 3D objects. For example, the access permission of the Mercedes-Benz car model is only allowed for users with the identification of "member" or "VIP user"; the toy ninja is only visible to devices with the device capability of "high" of the terminal device; the doll can only be viewed through a terminal device with the device type of "AR glasses". Customers entering the multi-functional entertainment venue can use their own terminal devices to enter the toy room to view the 3D objects placed by the organizer UE2, and the devices used by different customers can be the same or different.

[0159] Correspondingly, the cloud server receives the anchor point information and the 3D object information sent from the UE2. A correspondence relationship is established between at least one anchor point information and the 3D object information according to the obtained information, as shown in Table 1 above.

[0160] In addition, according to the pre-divided regions and the anchor points in each region, the cloud server also establishes a correspondence relationship between at least one region and the set of anchor points, and then obtains the correspondence relationship between different sets of anchor points and the 3D object information. For example, the multi-functional entertainment venue is divided into 3 regions according to the rooms, namely the first region, the second region, and the third region, corresponding to the living room, the toy room, and the lounge in sequence. Each region contains at least one anchor point, and then a correspondence relationship between the set of anchor points and the 3D object information is established. As shown in Table 2, map<set of anchor points, 3D object information>. Among them, the 3D object information can represent all the information of the 3D object.

[0161] Table 2. Correspondence relationship between the set of anchor points and the 3D object information

[0162]

[0163] It should be understood that a 3D object information can be marked by one anchor point, or can also be marked by two or more anchor points. In addition, one anchor point can be used to mark one or more 3D objects. In this embodiment, the number of 3D objects to be marked and the number of 3D objects that each anchor point can mark are not limited.

[0164] Optionally, each area can be divided smaller. For example, taking the range that each anchor point can mark as an area, the "anchor point set 2" in Table 2 can be split into 5 anchor point sets, and each anchor point set contains one anchor point. In this embodiment, the way of dividing the area, as well as the number and position of anchor points configured in each area are not limited.

[0165] 303: The customer enters the multi-functional entertainment venue and holds the first terminal device UE1 to scan and identify the first anchor point set. The first anchor point set includes the feature information of anchor point 1.

[0166] For example, UE1 is a mobile phone. When the customer enters the room and observes, when the customer walks towards the toy room, the customer will scan and identify anchor point 1 at the toy room door.

[0167] 304: UE1 sends the first anchor point set to the cloud server. At the same time, UE1 also uploads mobile attribute information, as well as context information such as user identification, device type, device capability, and cache size. In this example, it is assumed that the user identification reported by UE1 is an ordinary user, the device type is a mobile phone, the device capability is medium, the cache size is 1G, and the mobile attribute is 0.5m / s from the living room to the toy room direction.

[0168] 305: The cloud server receives the first anchor point set, mobile attribute information and UE1 context information reported by UE1. In the anchor point database of the cloud server, anchor point 1 is matched according to the feature of anchor point 1 in the first anchor point set, the position of UE1 is determined, and the target area for movement at the next moment is determined according to the mobile attribute information of UE1.

[0169] For example, in this example, the cloud server identifies that the position of anchor point 1 is at the toy room door, so as to determine that the position of the mobile phone UE1 is at the toy room door; then combined with the mobile attribute of the mobile phone of 0.5m / s from the living room to the toy room direction, it is predicted that the mobile phone may enter the toy room in 3 seconds. In this example, the target area is the toy room.

[0170] 306: The cloud server determines the target anchor point set and 3D object set in the target area.

[0171] Specifically, the cloud server determines that the target area, that is, the toy room, contains the target anchor point set {anchor points 2, 3, 4, 5, 6} and the corresponding 3D object set includes {Mercedes-Benz car model, dolphin doll, ninja model, doll and teddy bear}.

[0172] 307: The cloud server filters based on the context information of the UE1 in the target anchor set and the 3D object set to determine the 3D object set that the UE1 can display and its corresponding anchor set.

[0173] Specifically, the cloud server determines that the customer is an ordinary user based on the "user identification" of the customer. Therefore, it is determined that the Mercedes - Benz car model in the 3D object set is invisible to this customer because the Mercedes - Benz car model is only visible to VIP members. Additionally, based on the "device type" of the user terminal being a mobile phone, it can be determined that the doll is invisible to this customer because the doll is only visible to users with a "device type" of "AR glasses"; based on the "device capability" of the terminal device being medium, it can be determined that the ninja model is invisible because the ninja model has rich details and requires a high - capability device to be rendered and presented; and based on the "cache size" of the UE1 being 1G, which can accommodate the data / information of the remaining two 3D objects, the dolphin doll and the teddy bear. Finally, after removing the Mercedes - Benz car model, the doll, and the ninja model from the 3D object set, the remaining 3D object set that the UE1 can display is {dolphin doll and teddy bear}, and the corresponding anchor set is {anchor 3 and 6}.

[0174] 308: The cloud server sends the 3D object set and its corresponding anchor set to the UE1.

[0175] In this example, the 3D object set sent by the cloud server to the UE1 includes a dolphin doll and a teddy bear, and the anchor set is anchor 3 and anchor 6.

[0176] In addition, the cloud server can also send all the anchor information in the toy room to the UE1, such as sending the attached information of anchor 2, 4, 5. Among them, the 3D object corresponding to anchor 2 is only visible to users with a "user identification" of "member" or "VIP user"; the 3D object corresponding to anchor 4 requires a device with a "device capability" of high; the 3D object corresponding to anchor 5 can only be displayed with a "device type" of AR glasses.

[0177] 309: After the UE1 enters the target area and identifies the anchor, it displays the 3D object set on the display screen of the UE1.

[0178] Specifically, when the customer holds the mobile phone UE1 and enters the toy room, the customer holds the mobile phone and scans in the toy room. When the mobile phone identifies anchor 3, the local matches successfully with anchor 3, and the dolphin doll can be immediately displayed on the mobile phone. When the mobile phone identifies anchor 6, the local matches successfully with anchor 6, and the 3D image of the teddy bear can be immediately displayed on the mobile phone.

[0179] When the mobile phone identifies anchor 2, 4, 5, corresponding prompt information is displayed. Among them, the prompt information explains that there is an undisplayed 3D object here and the reason why the 3D object is not displayed.

[0180] In the method provided in this embodiment, when the user holds the terminal device and scans the environment, an instant 3D object loading experience can be obtained, eliminating the waiting time for interaction with the cloud and improving the user experience.

[0181] Next, a device embodiment corresponding to the above method embodiment of the present application will be introduced.

[0182] Figure 9 It is a schematic structural diagram of a device provided in an embodiment of the present application. In one embodiment, the device may include: an acquisition unit 901, a processing unit 902, and a sending unit 903. In addition, the device may further include a receiving unit 904, a storage unit 905, and more or fewer units and modules. The structure of the device in this embodiment is not limited.

[0183] Among them, when the device is a data sending device, the acquisition unit 901 is configured to acquire the position information and movement attribute information of the first terminal device at the first moment, and the movement attribute information includes the movement speed and direction of the first terminal device. The processing unit 902 is configured to determine a target area according to the position information and movement attribute information of the first terminal device, and acquire first data corresponding to the target area; the sending unit 903 is configured to send the first data to the first terminal device.

[0184] Among them, the target area is the area predicted to be reached by the first terminal device at the second moment, and the second moment is the next moment after the first moment. The first data includes a target anchor point set and a target object set. The target area includes at least one anchor point, and the target anchor point set is composed of the at least one anchor point. The at least one anchor point is used to mark at least one 3D object in the target area, and the at least one 3D object forms the target object set;

[0185] Optionally, in some embodiments, the processing unit 902 is further configured to acquire the correspondence between at least one area and at least one anchor point set, search for the target anchor point set associated with the target area in the correspondence; and determine the target object set according to the 3D objects marked by each anchor point in the target anchor point set. The at least one area includes the target area.

[0186] Optionally, in some other embodiments, after the processing unit 902 obtains the first data corresponding to the target area, it further filters out second data from the first data according to the context information of the first terminal device, and the second data includes a part of the target anchor set and the target object set. The context information includes one or more of: user identification, device type, device capabilities, and cache size. The sending unit 903 is further configured to send the second data to the first terminal device.

[0187] Optionally, in some other embodiments, the processing unit 902 is further configured to delete, according to the user identification in the context information, the 3D objects to which the user does not have access rights from the target object set, and obtain the remaining 3D object set, where the user identification is used to indicate whether the user has access rights to each 3D object;

[0188] Alternatively, according to the device type in the context information, filter out a set of 3D objects suitable for the device type from the target object set;

[0189] Alternatively, according to the device capabilities in the context information, filter out a set of 3D objects suitable for the device capabilities from the target object set, where the device capabilities include the level of detail for the device to render 3D objects;

[0190] Alternatively, according to the cache size in the context information, filter out a set of 3D objects whose storage capacity does not exceed the cache size of the first terminal device from the target object set.

[0191] Optionally, in some other embodiments, the receiving unit 904 is further configured to receive first anchor information sent by the first terminal device at the first moment; the processing unit 902 is further configured to check whether the first anchor information is stored in the anchor database, and if so, determine the location information of the first terminal device according to the first anchor information, where the anchor database includes anchor information of at least one anchor.

[0192] In addition, if the processing unit 902 does not find the first anchor information, the matching fails, that is, the location information of the first terminal device cannot be determined. At this time, the user needs to rescan.

[0193] Optionally, in some other embodiments, the receiving unit 904 is further configured to receive scan information sent by a second terminal device, where the scan information includes anchor information of at least one anchor scanned by the second terminal device in the current area and 3D objects marked by each of the anchors. The processing unit 902 is further configured to obtain the correspondence between the current area and the anchor information of at least one anchor in the current area according to the scan information.

[0194] A storage unit 905 for storing the correspondence between the current area and the anchor information of at least one anchor of the current area.

[0195] On the other hand, when the device is a data receiving device, a receiving unit 904 is configured to receive first data sent by a network device; a processing unit 902 is configured to display the target object set on the first terminal device when the first terminal device enters the target area and scans a target anchor set in the first data.

[0196] The first data includes a target anchor set and a target object set. The target anchor set is composed of at least one anchor in the target area, and the at least one anchor is used to mark at least one 3D object in the target area. The at least one 3D object forms the target object set. The target area is the area predicted to be reached by the first terminal device at a second moment.

[0197] Optionally, in some embodiments, the processing unit 902 is further configured to scan the external environment to obtain first anchor information before the receiving unit receives the first data. The first anchor information includes the anchor information of at least one anchor included in the scanned external environment; a sending unit 903 is configured to send the first anchor information to the network device at the first moment, and the first anchor information is used to determine the position information of the first terminal device.

[0198] Optionally, in some other embodiments, the sending unit 903 is configured to send the context information of the first terminal device to the network device, and the context information is used to filter out second data from the first data. The second data includes a part of the target anchor set and the target object set.

[0199] Furthermore, the context information includes one or more of: user identifier, device type, device capability, and cache size. The processing unit 902 is further configured to:

[0200] If the context information includes a user identifier, delete the 3D objects that the user does not have access rights to in the target object set to obtain the remaining 3D object set. The user identifier is used to indicate whether the user has access rights to each 3D object.

[0201] If the context information includes a device type, select the 3D object set suitable for the device type in the target object set.

[0202] If the context information includes a device capability, select the 3D object set suitable for the device capability in the target object set. The device capability includes the detail level of the device for rendering 3D objects.

[0203] If the context information includes a cache size, a set of 3D objects whose storage capacity in the set of target objects does not exceed the cache size of the first terminal device.

[0204] In addition, the processing unit 902 is further configured to scan the surrounding environment, determine at least one anchor point, match the at least one anchor point with the anchor points stored locally to determine the set of target anchor points, and search for the corresponding set of target objects in the pre-stored corresponding relationship according to the set of target anchor points.

[0205] In addition, in a hardware implementation, an embodiment of the present application further provides a network device, which may be the cloud server or AR Cloud in the foregoing embodiments, etc., for implementing the methods in the foregoing embodiments.

[0206] Wherein, the structure of the network device may be the same as or different from the structure of the foregoing terminal device. In some embodiments, referring to Figure 10 , a schematic structural diagram of a network device is shown. The network device may include: a processor 10, a memory 20, and at least one communication interface 30. Among them, the processor 10, the memory 20, and the at least one communication interface 30 are coupled through a communication bus.

[0207] Among them, the processor 10 is the control center of the network device and can be used to complete communications in the wireless communication system, including data transmission with at least one terminal device; and communications with other network devices, etc.

[0208] Further, the processor 10 may be composed of an integrated circuit (IC). For example, it may be composed of a single packaged IC, or may be composed of multiple packaged ICs with the same or different functions connected. For example, the processor 10 may include a central processing unit (CPU) or a digital signal processor (DSP), etc.

[0209] In addition, the processor 10 may further include a hardware chip, which may be a logic circuit, an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0210] The memory 20 is used for storing and exchanging various types of data or software, such as the location information of the terminal device, the mobile attribute information, the set of target anchors, and the set of target objects. In addition, a computer program or code may be stored in the memory 20.

[0211] Specifically, the memory 20 may include a volatile memory, such as a random access memory (RAM); it may also include a non-volatile memory, such as a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD). The memory 20 may also include a combination of the above types of memories.

[0212] Optionally, the memory 20 may be integrated in the processor 10 as a storage medium, or may be configured outside the processor 10. This embodiment does not limit this.

[0213] At least one communication interface 30, which may use any device such as a transceiver, is used for communicating with other devices or communication networks, such as Ethernet, WLAN, etc. For example, communication with the UE1 is performed using at least one communication interface 30.

[0214] In addition, the above network device also includes a mobile communication module, a wireless communication module, etc. The mobile communication module includes: a module with wireless communication function. In addition, it can also include a filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. In some embodiments, at least part of the functional modules of the mobile communication module can be set in the processor. The wireless communication module can provide solutions for wireless communications including WLAN, Bluetooth (BT), global navigation satellite system (GNSS), etc. applied to the switch.

[0215] It should be understood that the above network device may also include more or fewer components, and the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the network device. Figure 9 or Figure 10 The components shown may be implemented in hardware, software, firmware, or any combination thereof.

[0216] When implemented using software, it can be implemented in whole or in part in the form of a computer program product. Figure 9 The receiving unit 904 and the sending unit 903 in the data sending device shown can be implemented by at least one communication interface 30 , the functions of the acquiring unit 901 and the processing unit 902 can be implemented by the processor 10 , and the function of the storage unit 905 can be implemented by the memory 20 .

[0217] In addition, the present application also provides a wireless communication system, the structure of which can be similar to the above Figure 1 The system includes at least one terminal device, such as UE1, UE2, and a network device. The structure of the network device can be similar to Figure 10 The structure of the terminal device can be the same as that of the aforementioned Figure 2 The structures shown are the same.

[0218] Optionally, the structure of the network device can also be Figure 2 The structures of the terminal devices shown are the same, and this embodiment does not limit this.

[0219] The present application also provides a computer program product, which includes one or more computer program instructions. When a computer loads and executes the computer program instructions, the processes or functions described in the above embodiments are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0220] The computer program instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one network node, computer, server, or data center to another node in a wired or wireless manner.

[0221] Optionally, in another possible implementation of the above data sending device, the device can be a wireless communication device or a chip in a wireless communication device. Specifically, the device includes: at least one input / output interface and a logic circuit. Among them, the input / output interface can be an input / output circuit. The logic circuit can be a signal processor, a chip, or other integrated circuits that can implement the method of the present application. Among them, at least one input / output interface is used for input or output of signals or data. In addition, the input / output interface can also be used to implement communication transmission with at least one terminal device.

[0222] Among them, the logic circuit is used to execute some or all of the steps of any method provided by the embodiments of the present application.

[0223] In addition, in the description of the present application, unless otherwise specified, "a plurality of" means two or more than two. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. do not necessarily limit differences.

[0224] The above-described embodiments of the present application do not constitute a limitation to the protection scope of the present application.

Claims

1. A data sending method, characterized in that, The method includes: The network device obtains the location information and movement attribute information of the first terminal device at the first moment, where the movement attribute information includes the movement speed and direction of the first terminal device; The network device determines a target area according to the location information and movement attribute information of the first terminal device. The target area is the area predicted to be reached by the first terminal device at the second moment, and the second moment is the next moment after the first moment; The network device obtains first data corresponding to the target area. The first data includes a target anchor point set and a target object set. The target area includes at least one anchor point. The target anchor point set is composed of the at least one anchor point. The at least one anchor point is used to mark at least one 3D object in the target area, and the at least one 3D object forms the target object set; The network device sends the first data to the first terminal device; After the network device obtains the first data corresponding to the target area, it further includes: The network device filters out second data from the first data according to the context information of the first terminal device. The second data includes a part of the target anchor point set and the target object set; The context information includes one or more of: user identifier, device type, device capability, and cache size; The network device sending the first data to the first terminal device includes: The network device sending the second data to the first terminal device; The network device filtering out second data from the first data according to the context information of the first terminal device includes: The network device deletes the 3D objects that the user does not have access rights to in the target object set according to the user identifier in the context information, and obtains the remaining 3D object set. The user identifier is used to indicate whether the user has access rights to each 3D object; Or, the network device filters out the 3D object set suitable for the device type from the target object set according to the device type in the context information; Or, the network device filters out the 3D object set suitable for the device capability from the target object set according to the device capability in the context information. The device capability includes the detail level of the device for rendering 3D objects; Or, the network device filters out the 3D object set whose storage capacity does not exceed the cache size of the first terminal device from the target object set according to the cache size in the context information.

2. The method according to claim 1, characterized in that, The network device obtaining the first data corresponding to the target area includes: The network device obtains the correspondence between at least one area and at least one anchor point set, where the at least one area includes the target area; The network device looks up the target anchor point set associated with the target area in the correspondence; The network device determines the target object set according to the 3D objects marked by each anchor point in the target anchor point set; 3. The method according to any one of claims 1 to 2, characterized in that, The network device obtaining the location information of the first terminal device at the first moment includes: The network device receives the first anchor information sent by the first terminal device at the first moment; The network device looks up in the anchor database whether the first anchor information is stored. If so, it determines the location information of the first terminal device according to the first anchor information. The anchor database includes the anchor information of at least one anchor.

4. The method according to claim 2, characterized in that, The network device obtains the correspondence between at least one area and at least one anchor set, including: The network device receives the scan information sent by the second terminal device. The scan information includes the anchor information of at least one anchor scanned by the second terminal device in the current area and the 3D objects marked by each of the anchors. The network device obtains the correspondence between the current area and the anchor information of at least one anchor in the current area according to the scan information.

5. A data receiving method, characterized in that, The method includes: The first terminal device receives the first data sent by the network device. The first data includes a target anchor set and a target object set. The target anchor set consists of at least one anchor in the target area. The at least one anchor is used to mark at least one 3D object in the target area. The at least one 3D object constitutes the target object set. The target area is the area predicted to be reached by the first terminal device at the second moment; When the first terminal device enters the target area and scans the target anchor set in the first data, the target object set is displayed on the first terminal device; The method further includes: The first terminal device sends the context information of the first terminal device to the network device. The context information is used to screen out the second data from the first data. The second data includes a part of the target anchor set and the target object set; The context information includes one or more of: user identifier, device type, device capability, and cache size. If the context information includes a user identifier, a part of the target object set in the second data includes: deleting the 3D objects that the user does not have access rights to in the target object set to obtain the remaining 3D object set. The user identifier is used to indicate whether the user has access rights to each 3D object. If the context information includes a device type, a part of the target object set in the second data includes: the 3D object set suitable for the device type in the target object set. If the context information includes a device capability, a part of the target object set in the second data includes: the 3D object set suitable for the device capability in the target object set. The device capability includes the detail level of the device for rendering 3D objects. If the context information includes a cache size, a part of the target object set in the second data includes: the 3D object set whose storage capacity in the target object set does not exceed the cache size of the first terminal device.

6. The method according to claim 5, characterized in that, Before the first terminal device receives the first data sent by the network device, it further includes: The first terminal device scans the external environment to obtain first anchor point information, where the first anchor point information includes the anchor point information of at least one anchor point included in the scanned external environment; The first terminal device sends the first anchor point information to the network device at a first moment, where the first anchor point information is used to determine the location information of the first terminal device.

7. A data sending device, characterized in that, The apparatus includes: An obtaining unit, configured to obtain the location information and movement attribute information of a first terminal device at a first moment, where the movement attribute information includes the movement speed and direction of the first terminal device; A processing unit, configured to determine a target area according to the location information and movement attribute information of the first terminal device, and obtain first data corresponding to the target area, where the target area is an area predicted to be reached by the first terminal device at a second moment, the second moment is the next moment after the first moment, the first data includes a target anchor point set and a target object set, at least one anchor point is included in the target area, the target anchor point set is composed of the at least one anchor point, the at least one anchor point is used to mark at least one 3D object in the target area, and the at least one 3D object forms the target object set; A sending unit, configured to send the first data to the first terminal device; The processing unit is further configured to, after obtaining the first data corresponding to the target area, screen out second data from the first data according to the context information of the first terminal device, where the second data includes a part of the target anchor point set and the target object set; the context information includes one or more of: user identifier, device type, device capability, and cache size; the sending unit is further configured to send the second data to the first terminal device; The processing unit is further configured to delete, according to the user identifier in the context information, the 3D objects that the user does not have access rights to in the target object set to obtain a remaining 3D object set, where the user identifier is used to indicate whether the user has access rights to each 3D object; or screen out, according to the device type in the context information, the 3D object set suitable for the device type in the target object set; or screen out, according to the device capability in the context information, the 3D object set suitable for the device capability in the target object set, where the device capability includes the detail level of the device for rendering 3D objects; or screen out, according to the cache size in the context information, the 3D object set whose storage capacity does not exceed the cache size of the first terminal device in the target object set.

8. The device according to claim 7, characterized in that, The processing unit is further configured to obtain the correspondence between at least one area and at least one anchor point set, and find the target anchor point set associated with the target area in the correspondence; and determine the target object set according to the 3D objects marked by each anchor point in the target anchor point set; the at least one area includes the target area.

9. The device according to any one of claims 7 to 8, characterized in that It further includes a receiving unit, The receiving unit is further configured to receive first anchor information sent by the first terminal device at the first moment; The processing unit is further configured to check whether the first anchor information is stored in the anchor database. If so, the position information of the first terminal device is determined according to the first anchor information. The anchor database includes anchor information of at least one anchor.

10. The device according to claim 8, characterized in that The processing unit is further configured to receive scan information sent by the second terminal device. The scan information includes anchor information of at least one anchor scanned by the second terminal device in the current area and 3D objects marked by each of the anchors. The processing unit is further configured to obtain a correspondence between the current area and the anchor information of at least one anchor in the current area according to the scan information.

11. A data receiving device, characterized in that Applied to a first terminal device, the apparatus includes: A receiving unit, configured to receive first data sent by a network device. The first data includes a target anchor set and a target object set. The target anchor set is composed of at least one anchor in a target area. The at least one anchor is used to mark at least one 3D object in the target area. The at least one 3D object forms the target object set. The target area is the area predicted to be reached by the first terminal device at the second moment; A processing unit, configured to display the target object set on the first terminal device when the first terminal device enters the target area and scans the target anchor set in the first data; A sending unit, configured to send context information of the first terminal device to the network device. The context information is used to filter out second data from the first data. The second data includes a part of the target anchor set and the target object set; The context information includes one or more of: user identifier, device type, device capability, and cache size. The processing unit is further configured to, if the context information includes a user identifier, delete 3D objects that the user does not have access rights to in the target object set to obtain a remaining 3D object set. The user identifier is used to indicate whether the user has access rights to each 3D object; if the context information includes a device type, a 3D object set suitable for the device type in the target object set; if the context information includes a device capability, a 3D object set suitable for the device capability in the target object set. The device capability includes the detail level of the device for rendering 3D objects; if the context information includes a cache size, a 3D object set whose storage capacity does not exceed the cache size of the first terminal device in the target object set.

12. The device according to claim 11, characterized in that It further includes a sending unit, The processing unit is further configured to scan the external environment to obtain first anchor information before the receiving unit receives the first data. The first anchor information includes anchor information of at least one anchor included in the scanned external environment; The sending unit is configured to send the first anchor information to the network device at a first moment, where the first anchor information is used to determine the location information of the first terminal device.

13. A data transmission system, characterized in that The system includes a terminal device and a network device. The network device includes the device according to any one of claims 7 to 10. The terminal device includes the device according to any one of claims 11 to 12.

14. The system according to claim 13, characterized in that The system further includes a second terminal device. The second terminal device sends scan information to the network device, where the scan information includes the anchor information of at least one anchor scanned by the second terminal device in the current area and the 3D object marked by each of the anchors. The network device receives the scan information and obtains the correspondence between the current area and the anchor information of at least one anchor in the current area according to the scan information.

15. A communication device, characterized in that It includes at least one processor and a memory. The memory is configured to store instructions provided by the at least one processor. The at least one processor is configured to execute the instructions to implement the method according to any one of claims 1 to 4, or any one of claims 5 to 6.

16. A computer-readable storage medium, characterized in that Computer program instructions are stored in the computer-readable storage medium. When the computer program instructions are run, the method according to any one of claims 1 to 4, or any one of claims 5 to 6 is implemented.

Images