An augmented reality (AR) data processing method, apparatus, and related device

Abstract

The application discloses a processing method and device of augmented reality (AR) data and related equipment. The method is applied to a network edge computing node, and includes the following steps: calculating the encryption density of AR data according to the AR data and the encryption level of the AR data, determining target AR data that needs to be encrypted in the AR data, and then encrypting the target AR data to obtain encrypted AR data. In this way, the encryption of the target AR data is only needed, and the whole disk encryption processing of the AR data is not needed, so that the encryption and decryption efficiency is higher, the encryption of Web AR data is suitable, and the requirement of a user for a smooth experience of a Web AR function is met.

Description

Technical Field

[0001] This application belongs to the field of augmented reality technology, and in particular relates to a method, apparatus and related equipment for processing augmented reality (AR) data. Background Technology

[0002] Augmented Reality (AR) technology is a technique that cleverly integrates virtual information with the real world. It widely utilizes multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, and sensing technologies to simulate and apply computer-generated text, images, 3D models, music, videos, and other virtual information to the real world. The two types of information complement each other, thus "enhancing" the real world. However, existing AR data is easily illegally scraped, tampered with, and disseminated, seriously affecting public understanding of AR data.

[0003] Existing methods for encrypting AR data involve encrypting and decrypting the entire AR file. While these methods can encrypt AR data, their encryption and decryption efficiency is low, making them unsuitable for encrypting Web AR data and failing to meet users' needs for a smooth Web AR experience. Summary of the Invention

[0004] This application provides a method, apparatus, and related equipment for processing augmented reality (AR) data, which can encrypt AR data with high encryption and decryption efficiency. It is suitable for encrypting Web AR data and meets the user's need for a smooth experience of Web AR functions.

[0005] In a first aspect, embodiments of this application provide a method for processing augmented reality (AR) data, applied to a network edge computing node, including: Obtain AR data and the encryption level of the AR data; The encryption density of the AR data is determined based on the encryption level. Based on the encryption density, determine the target AR data in the AR data that needs to be encrypted; The target AR data is encrypted to obtain encrypted AR data.

[0006] Secondly, embodiments of this application provide an augmented reality (AR) data processing apparatus applied to a network edge computing node, comprising: The acquisition module is used to acquire AR data and the encryption level of the AR data; The first determining module is used to determine the encryption density of the AR data based on the encryption level. The second determining module is used to determine the target AR data that needs to be encrypted in the AR data based on the encryption density. An encryption module is used to encrypt the target AR data to obtain encrypted AR data.

[0007] Thirdly, embodiments of this application provide an electronic device, the device including: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the augmented reality (AR) data processing method as described above.

[0008] Fourthly, embodiments of this application provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement the augmented reality (AR) data processing method described in any of the above claims.

[0009] Fifthly, embodiments of this application provide a computer program product, wherein instructions in the computer program product, when executed by a processor of an electronic device, cause the electronic device to perform the augmented reality (AR) data processing method as described in any of the above claims.

[0010] The augmented reality (AR) data processing method, apparatus, and related devices of this application are applied to network edge computing nodes. They can calculate the encryption density of AR data based on the AR data and its encryption level, determine the target AR data within the AR data that needs encryption, and then encrypt the target AR data to obtain encrypted AR data. Thus, this application only requires encrypting the target AR data within the AR data, rather than encrypting the entire AR data, resulting in high encryption and decryption efficiency. It is suitable for encrypting Web AR data and meets users' needs for a smooth Web AR experience. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating the augmented reality (AR) data processing method provided in an embodiment of this application. Figure 2 This is a schematic diagram showing that the encryption density of augmented reality (AR) data provided in this application embodiment is 50%.

[0013] Figure 3 This is a schematic diagram of the structure of the augmented reality (AR) data processing device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0014] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0015] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0016] Augmented Reality (AR) technology is a technology that cleverly integrates virtual information with the real world. It widely uses various technologies such as multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, and sensing to simulate and apply computer-generated virtual information such as text, images, 3D models, music, and videos to the real world. The two types of information complement each other, thereby achieving "enhancement" of the real world.

[0017] Augmented Reality (AR) technology is a relatively new technology that integrates information from the real world and the virtual world. It uses computer science to simulate and overlay physical information that is difficult to experience in the real world, effectively applying virtual information to the real world while allowing it to be perceived by human senses, thus achieving a sensory experience that transcends reality. After the real environment and virtual objects overlap, they can coexist in the same scene and space. Augmented Reality technology not only effectively reflects real-world content but also enables the display of virtual information, with these subtle elements complementing and overlaying each other.

[0018] In existing AR solutions, users need to download and install large, heavyweight apps to experience AR, but the limited memory, battery, and storage capacity of mobile phones restrict the development of AR. Web AR, on the other hand, uses a browser / server (B / S) architecture and leverages web technologies such as HTML5, JavaScript, 3D modeling, and VR to achieve a lightweight and seamless augmented reality experience. It eliminates the need for bulky apps, allowing users to access and experience the latest Web AR data anytime. However, both app-based AR content and Web AR data are vulnerable to illegal scraping, tampering, and dissemination, severely impacting public awareness of AR data. Traditional protection methods are inadequate for AR data protection, especially Web AR data. Existing protection methods use common methods to encrypt and decrypt the entire AR file. While these methods can encrypt AR data, their encryption and decryption efficiency is low and unsuitable for Web AR data, failing to meet users' needs for a smooth Web AR experience.

[0019] To address the problems of the prior art, embodiments of this application provide a method, apparatus, and related equipment for processing augmented reality (AR) data. The method for processing AR data provided in this application embodiment will be described first below.

[0020] Figure 1 A flowchart illustrating a method for processing augmented reality (AR) data according to an embodiment of this application is shown. Figure 1 As shown, an augmented reality (AR) data processing method, applied to a network edge computing node, may include steps S101 to S104: S101. Obtain AR data and the encryption level of AR data; S102. Determine the encryption density of the AR data based on the encryption level; S103. Based on the encryption density, determine the target AR data that needs to be encrypted in the AR data; S104. Encrypt the target AR data to obtain encrypted AR data.

[0021] In this embodiment, applied to a network edge computing node, the encryption density of AR data is calculated based on the AR data and its encryption level. The target AR data that needs encryption is then identified, and the target AR data is encrypted to obtain the encrypted AR data. Thus, this embodiment only requires encryption of the target AR data, rather than encrypting the entire AR data set. This results in high encryption and decryption efficiency, making it suitable for encrypting Web AR data and meeting users' needs for a smooth Web AR experience.

[0022] The above-mentioned augmented reality (AR) data processing method, applied to network edge computing nodes, has high processing capabilities. At the same time, since the resources of network edge computing nodes are geographically very close to users and information sources, the latency of network response to user requests is greatly reduced, and the possibility of network congestion in the transmission network and core network is also reduced, thus achieving the requirements of high reliability and low latency transmission.

[0023] Specifically, edge computing can be achieved through fourth-generation mobile communication networks (4G) and fifth-generation mobile communication networks (5G).

[0024] 5G represents an innovation and transformation in network technology. The International Telecommunication Union's Radiocommunication Bureau has defined three typical application scenarios for 5G: Enhanced Mobile Broadband (eMBB), Ultra-Reliable and Low-Latency Communication (uRLLC), and Massive Machine Type Communication (mMTC). eMBB primarily targets high-bandwidth services such as Virtual Reality (VR) / Augmented Reality (AR) and online high-definition video (4K / 8K); uRLLC primarily targets latency-sensitive, high-performance applications in vertical industries such as smart transportation and smart healthcare; and mMTC primarily targets IoT applications with high connection density requirements, such as smart cities, smart industries, and smart water meters, involving hundreds of billions of devices.

[0025] Because 5G boasts higher bandwidth technology eMBB compared to 4G, AR can be better and more widely applied in 5G edge computing technology within vertical application scenarios. 5G communication networks are more decentralized, requiring the deployment of small-scale or portable data centers at the network edge to offload some terminal requests to local computing for processing, thus meeting the ultra-low latency requirements of uRLLC and mMTC. Therefore, Mobile Edge Computing (MEC) is one of the core technologies of 5G and can realize the augmented reality (AR) data processing method described in this application.

[0026] The specific implementation methods for each of the above steps are described below.

[0027] In S101, the aforementioned AR data can be a combination of real-world and virtual-world information. It can be a two-dimensional planar image or a three-dimensional stereoscopic image.

[0028] The encryption level of the aforementioned AR data can be determined by classifying and grading the data according to predefined rules. In this embodiment, the encryption level can be... It means that among them This indicates that Web AR data has the highest level of encryption. This indicates the lowest level of encryption for Web AR data. Different Web AR data can be classified into different encryption levels according to security and confidentiality requirements. Different parts of the same Web AR data can also be processed with different encryption levels according to different security and confidentiality requirements. The encryption level of Web AR data can be determined according to predefined rules. Classification processing is performed. For example, predefined rules may include the three-level classification of the "Guidelines for Industrial Data Classification and Grading (Trial)" issued by the Ministry of Industry and Information Technology of the People's Republic of China, the five-level classification of JR / T 0197-2020 "Guidelines for Financial Data Security Classification" issued by the National Financial Standardization Technical Committee, the four-level classification of YD / T 3813-2020 "Methods for Data Classification and Grading of Basic Telecommunications Enterprises" issued by the Ministry of Industry and Information Technology of the People's Republic of China, and the four-level classification of the "Practice Guidelines for Network Security Standards - Guidelines for Network Data Classification and Grading" issued by the National Information Security Standardization Technical Committee, etc.

[0029] In S102, the encryption density mentioned above can represent the relative density of the target AR data that needs to be encrypted within the AR data. For example, the highest encryption density for processing AR data can be 100%, meaning all AR data is encrypted, while the lowest encryption density can be 0, meaning the original AR data.

[0030] The encryption density of AR data, as described above based on the encryption level, can be determined through 5G edge computing (MEC). Higher encryption levels result in higher encryption densities.

[0031] In this embodiment, the encryption density can be used as follows: The formula is expressed as follows: (1) in, Encryption density The highest encryption density is 100%. The minimum value is 0.

[0032] In S103, the aforementioned target AR data can be data that needs to be encrypted within the AR data.

[0033] The above method of determining the target AR data that needs to be encrypted in AR data based on encryption density can be achieved by determining the target AR data based on the encryption density of uniformly distributed pixels in the AR data.

[0034] For example, such as Figure 2As shown, this is AR data with an encryption density of 50%, where the dark area represents the selected target AR data.

[0035] In S104, the above-mentioned encryption of the target AR data to obtain encrypted AR data can be achieved by encrypting the target AR data according to the Gaussian blur function.

[0036] As another implementation of this application, in order to focus on encrypting key temporal features in AR data, before S104, the following may also be included: The key temporal features that need to be encrypted in AR data are extracted using a temporal feature extraction function. The temporal feature extraction function is obtained by learning the correspondence between key scene features labeled according to the content display order of AR data samples. Determine the position coordinates of the pixels containing key temporal features in a preset coordinate system; Based on the encryption weights and coordinates of pixels at different locations in the preset AR data, calculate the encryption weight matrix of the target AR data; The target AR data is encrypted to obtain encrypted AR data, including: The target AR data is encrypted based on the encryption weight matrix to obtain encrypted AR data.

[0037] The aforementioned key temporal features can be the data in AR data that requires intensive encryption.

[0038] The aforementioned preset coordinates can be coordinates identical to the Earth's coordinate system, or coordinates consistent with the coordinates of the device used to acquire the real-world scene. The position coordinates of the pixels containing the key temporal features in the preset coordinate system can be two-dimensional position coordinates (X,Y) in a two-dimensional image, or three-dimensional position coordinates (X,Y,Z) in a three-dimensional object.

[0039] The aforementioned temporal feature extraction function is obtained by learning the correspondence between AR data samples and key scene features labeled according to the content display order of the AR data samples. For example, a temporal function is constructed in a three-dimensional coordinate system, and a modified recurrent neural network algorithm is used to extract numerical key temporal features from the AR data. The calculation formula is as follows: (2) in This represents the activation function or an encapsulated feedforward neural network. This represents the weight coefficients within the loop unit. Indicates the first time, This represents the three-dimensional spatial coordinates (x, y, and height) of the AR data. Key temporal features are extracted from the AR data according to the order in which it is displayed. Initialization involves random allocation. The improved recurrent neural network model is trained by taking AR data samples from various time series as input, performing iterative calculations, outputting key temporal features of the AR data, and comparing the output results with the expected results. If the results match the expectations, the iteration stops; otherwise, adjustments are made. Continue iterating until the output matches the expected key temporal features.

[0040] The encryption weights of pixels at different locations in the aforementioned preset AR data can be set by the user based on the AR data. The encryption weight of pixels at key temporal features can be higher than that at other locations. Alternatively, if it is not necessary to focus on encrypting key temporal features, the encryption weight of pixels at key temporal features can be the same as that at other locations, for example, all equal to 1.

[0041] The above-mentioned encryption of target AR data to obtain encrypted AR data can be achieved by encrypting each pixel in the target AR data using a Gaussian blur function and obtaining the encrypted AR data based on the encryption weight matrix.

[0042] In this embodiment, by extracting key temporal features from AR data that require encryption, and utilizing the different weights of each pixel in the encryption weight matrix, the key temporal features in the target AR data can be encrypted in a focused manner, thereby enhancing the encryption effect of the AR data.

[0043] In some embodiments, encrypting the target AR data according to the encryption weight matrix to obtain encrypted AR data may include: Obtain the component pixel values ​​of the pixel corresponding to the target AR data in each color channel; Based on the encryption weight matrix, the component pixel values ​​of each color channel are encrypted to obtain the encrypted AR data.

[0044] The aforementioned color channels can be channels that store color information in an image. Each image has one or more color channels. The default number of color channels in an image depends on its color mode; that is, the color mode of an image determines the number of its color channels. Each color channel stores information about the color elements in the image. The colors in all color channels are superimposed and mixed to produce the color of a pixel in the image. For example, in an RGB mode image, R is the red channel, G is the green channel, and B is the blue channel. It should be noted that color channels are not limited to red, green, and blue; they can also be other color channels. This application does not specifically limit the type of color channel used.

[0045] The aforementioned component pixel values ​​can be the pixel values ​​of the R (red) channel, G (green) channel, and B (blue) channel in an RGB mode image. Each of RGB has 256 levels of brightness, represented numerically from 0, 1, 2... up to 255.

[0046] The above method encrypts the component pixel values ​​of each color channel according to the encryption weight matrix to obtain encrypted AR data. It can be the pixel point with three-dimensional position coordinates in the target AR data determined by the encryption level and encryption density. The component pixel values ​​of each color channel of the pixel point are weighted and encrypted according to the encryption weight matrix to obtain encrypted AR data.

[0047] In this embodiment, by weighted encryption of the component pixel values ​​of the pixel points corresponding to the target AR data in each color channel, the AR data can be encrypted in different color channels, which can effectively enhance the protection effect.

[0048] Optionally, in some embodiments, determining the position coordinates of the pixel containing the key temporal feature in a preset coordinate system may specifically include: determining the three-dimensional position coordinates of the pixel containing the key temporal feature in a three-dimensional coordinate system.

[0049] In this embodiment, in real-life scenarios, the environment is a complex three-dimensional space. In order to meet the various complex application requirements of real-life scenarios and determine the coordinates of the pixels where key temporal features are located, the three-dimensional position coordinates can be determined in a three-dimensional coordinate system to meet the needs of complex application scenarios.

[0050] Furthermore, in some embodiments, in three-dimensional spatial application scenarios, encrypting the target AR data to obtain encrypted AR data may include: The target AR data is encrypted using a multidimensional Gaussian blur function to obtain encrypted AR data; The multidimensional Gaussian blur function is: (3) Where x, y, and z are the horizontal, vertical, and height coordinates in a three-dimensional coordinate system, t is the key temporal feature, and r is the encryption level. For encryption density, Represents the covariance matrix. This represents the average value of the corresponding dimension in the AR data.

[0051] In this embodiment, the multidimensional Gaussian blur function processes AR data in six dimensions: horizontal coordinate, vertical coordinate, height coordinate, key temporal features, encryption level, and encryption density. The target AR data is determined and encrypted by the encryption level and encryption density, without overall encryption. This satisfies the requirements of efficient encryption and smooth experience of AR data. The three-dimensional coordinates can meet the needs of complex three-dimensional spatial applications and avoid the situation where a large number of key factors are easily lost when processing with a two-dimensional Gaussian function. Furthermore, by extracting key temporal features, key areas in the AR data are encrypted in a focused manner, enhancing the protection effect.

[0052] In the above embodiments, acquiring AR data may include: Collect image or video data corresponding to real-world scenes; The scene recognition and tracking model identifies the target scene in image or video data. AR data is obtained by overlaying a virtual scene model corresponding to the target scene onto the image or video data corresponding to the real scene.

[0053] The aforementioned acquisition of image or video data corresponding to real-world scenes can be achieved through wearable AR devices or mobile phones and other hardware devices with camera capabilities, capturing videos or images of the real-world environment. A video can be understood as a collection of frame data consisting of a series of images, denoted by a frame data collection S containing a total of S images. S is a positive integer greater than or equal to 1.

[0054] The aforementioned scene recognition and tracking model is trained based on image data and corresponding scenes. Specifically, it can involve establishing a scene recognition and tracking module. The 5G mobile edge computing (MEC) trains M specific scene recognition and tracking datasets using image pattern recognition algorithms. When given a large number of images containing M specific scenes and other unrelated scenes, if the model can identify the M specific scenes, then the M specific scene recognition and tracking datasets are considered valid and usable. Otherwise, they are invalid datasets and require retraining using image pattern recognition. M is a positive integer greater than or equal to 1.

[0055] The aforementioned virtual scene model can be constructed using 3D virtual modeling software, comprising N specific virtual scene models, each containing at least one of the following: people, animals, plants, landscapes, and buildings. N is a positive integer greater than or equal to 1.

[0056] The above-mentioned method of overlaying a virtual scene model corresponding to the target scene onto the image or video data corresponding to the real scene to obtain AR data can be achieved by outputting the collected real environment video or image to the scene recognition and tracking module for comparison to identify a specific scene, and then the virtual scene model building module overlaying a specific virtual scene model onto this specific image to form an AR augmented reality effect and obtain AR data.

[0057] In this embodiment, the mobile edge computing (MEC) determines the user's location through network data (such as location information fed back from the RAN side), and utilizes a local AR server to provide real-time AR data matching calculation and push, achieving real-time aggregation of local real-world data and AR data. In this way, Web AR achieves a lightweight and seamless augmented reality (AR) experience using a browser / server (B / S) architecture and web technologies such as HTML5, JavaScript, 3D modeling, and VR. It eliminates the need to install bulky apps and allows for access to Web AR data anytime.

[0058] As another implementation of this application, in order to view the encrypted AR data, it is necessary to decrypt the encrypted AR data. Therefore, after obtaining the encrypted AR data, it may also include: Receive a request from a terminal device to obtain AR data, the request including the key corresponding to the AR data; Once the key verification is successful, the authorized terminal device can access the AR data.

[0059] The aforementioned key can be a combination key containing tokens, expiration dates, signatures, etc.

[0060] In this embodiment, AR data is decrypted and accessed through key verification, presenting the original AR data with encryption protection. This can prevent leakage or cracking of AR data that has already undergone efficient deep blurring when unauthorized access to Web AR data is made or when illegal crawlers crawl and tamper with Web AR data.

[0061] To facilitate understanding of the augmented reality (AR) data processing method in the embodiments of this application, the actual application process of this AR data processing method is described as follows: The following steps are taken to encrypt and protect Web AR data that is dynamic, temporal, spatially variable, and complex in various scenarios: First, based on the encryption level of the Web AR data Determining the encryption density of Web AR through 5G mobile edge computing This allows us to identify the target Web AR data that needs encryption; based on a temporal feature extraction function, we extract key temporal features from the target Web AR data that may require intensive encryption. Determine the position coordinates of the pixels containing key temporal features in the three-dimensional coordinate system. At this point, based on the encryption level Encryption density Key time series characteristics and pixel position coordinates The multidimensional Gaussian blur function is used to encrypt the AR data, resulting in initial encrypted AR data.

[0062] Secondly, to focus on encrypting key temporal features, an encryption weight matrix for the target AR data can be calculated based on the preset encryption weights of pixels at different locations in the AR data and their corresponding coordinates. Then, the initial encrypted AR data is weighted according to this encryption weight matrix to obtain weighted encrypted AR data. : (4) Third, to achieve better encryption protection for colors, the pixel points corresponding to the target AR data in the Web AR data are also combined with the RGB red, green, and blue primary colors. The component pixel values ​​of each channel are encrypted and protected across the three color channels to obtain the final encrypted AR data. At this point, the improved multidimensional Gaussian blur function is: (5) Finally, the key is used for decryption to legally authorize users to access Web AR data. For example, a wearable device or mobile phone can send the key corresponding to the AR data to the network edge computing node. If the authorization is successful, that is, if the key verification is successful, the terminal device can be authorized to access the AR data, that is, to view the Web AR data normally.

[0063] Based on the augmented reality (AR) data processing method provided in the above embodiments, this application also provides specific implementations of an augmented reality (AR) data processing apparatus. Please refer to the following embodiments.

[0064] like Figure 3 As shown, the augmented reality (AR) data processing device 300 provided in this application embodiment may include: Module 301 is used to acquire AR data and the encryption level of the AR data. The first determining module 302 is used to determine the encryption density of AR data according to the encryption level; The second determining module 303 is used to determine the target AR data that needs to be encrypted in the AR data based on the encryption density; Encryption module 304 is used to encrypt the target AR data to obtain encrypted AR data.

[0065] In this embodiment, applied to network edge computing nodes, the encryption density of AR data is calculated based on the AR data and its encryption level. The target AR data that needs encryption is then identified, and the target AR data is encrypted to obtain encrypted AR data. By calculating the encryption density based on the encryption level and identifying the target AR data based on that density, only the target AR data needs encryption, rather than encrypting the entire AR data. This results in high encryption and decryption efficiency, making it suitable for encrypting Web AR data and meeting users' needs for a smooth Web AR experience.

[0066] In some embodiments, it also includes: The extraction module is used to extract key temporal features that need to be encrypted from AR data using a temporal feature extraction function. The temporal feature extraction function is obtained by learning the correspondence between key scene features labeled according to the content display order of AR data samples. The second determining module 303 is also used to determine the position coordinates of the pixel where the key temporal feature is located in the preset coordinate system; The weight calculation module is used to calculate the encryption weight matrix of the target AR data based on the encryption weights and position coordinates of pixels at different locations in the preset AR data. The encryption module 304 is also used to encrypt the target AR data according to the encryption weight matrix to obtain the encrypted AR data.

[0067] In this embodiment, by extracting key temporal features from AR data that require focused encryption and using an encryption weight matrix, the key temporal features in the target AR data can be encrypted in a focused manner, thereby enhancing the encryption effect.

[0068] In some embodiments, the acquisition module 301 is further configured to acquire the component pixel values ​​of the pixel corresponding to the target AR data in each color channel; The encryption module 304 is also used to encrypt the component pixel values ​​of each color channel according to the encryption weight matrix to obtain encrypted AR data.

[0069] In this embodiment, by weighted encryption of the component pixel values ​​of the pixel points corresponding to the target AR data in each color channel, the AR data can be encrypted in different color channels, which can effectively enhance the protection effect.

[0070] Optionally, in some embodiments, the second determining module 303 is further used to determine the three-dimensional position coordinates of the pixel point where the key temporal feature is located in the three-dimensional solid coordinate system.

[0071] In this embodiment, in real-life scenarios, the environment is a complex three-dimensional space. In order to meet the various complex application requirements of real-life scenarios and determine the coordinates of the pixels where key temporal features are located, the three-dimensional position coordinates can be determined in a three-dimensional coordinate system to meet the needs of complex application scenarios.

[0072] Furthermore, in some embodiments, the encryption module 304 is also used to encrypt the target AR data using a multidimensional Gaussian blur function to obtain encrypted AR data; The multidimensional Gaussian blur function is: Where x, y, and z are the horizontal, vertical, and height coordinates of the three-dimensional coordinate system, t is the key temporal feature, and r is the encryption level. For encryption density, Represents the covariance matrix. This represents the average value of the corresponding dimension in the AR data.

[0073] In this embodiment, the multidimensional Gaussian blur function processes AR data in six dimensions: horizontal coordinate, vertical coordinate, height coordinate, key temporal features, encryption level, and encryption density. The target AR data is determined and encrypted by the encryption level and encryption density, without overall encryption. This satisfies the requirements of efficient encryption and smooth experience of AR data. The three-dimensional coordinates can meet the needs of complex three-dimensional spatial applications and avoid the situation where a large number of key factors are easily lost when processing with a two-dimensional Gaussian function. Furthermore, by extracting key temporal features, key areas in the AR data are encrypted in a focused manner, enhancing the protection effect.

[0074] In the above embodiments, the acquisition module 301 is also used to acquire image or video data corresponding to the real scene; The acquisition module 301 is also used to identify the target scene of the image or video data through the scene recognition and tracking model; the scene recognition and tracking model is trained based on the image data and the corresponding scene. The acquisition module 301 is also used to overlay the virtual scene model corresponding to the target scene onto the image or video data corresponding to the real scene to obtain AR data.

[0075] In this embodiment, the mobile edge computing (MEC) determines the user's location through network data (such as location information fed back from the RAN side), and utilizes a local AR server to provide real-time AR data matching calculation and push, achieving real-time aggregation of local real-world data and AR data. In this way, Web AR achieves a lightweight and seamless augmented reality (AR) experience using a browser / server (B / S) architecture and web technologies such as HTML5, JavaScript, 3D modeling, and VR. It eliminates the need to install bulky apps and allows for access to Web AR data anytime.

[0076] The above embodiments also include: The decryption module is used to receive a request from the terminal device to obtain AR data, the request including the key corresponding to the AR data; The decryption module is also used to authorize terminal devices to access AR data when the key verification is successful.

[0077] In this embodiment, AR data is decrypted and accessed through key verification, presenting the original AR data with encryption protection. This can prevent leakage or cracking of AR data that has already undergone efficient deep blurring when unauthorized access to Web AR data is made or when illegal crawlers crawl and tamper with Web AR data.

[0078] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application is shown.

[0079] An electronic device may include a processor 401 and a memory 402 storing computer program instructions.

[0080] Specifically, the processor 401 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0081] Memory 402 may include mass storage for data or instructions. For example, and not limitingly, memory 402 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 402 may include removable or non-removable (or fixed) media. Where appropriate, memory 402 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 402 is non-volatile solid-state memory.

[0082] In a particular embodiment, memory 402 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.

[0083] The processor 401 reads and executes computer program instructions stored in the memory 402 to implement any of the augmented reality (AR) data processing methods in the above embodiments.

[0084] In one example, the electronic device may also include a communication interface 403 and a bus 410. For example, Figure 4 As shown, the processor 401, memory 402, and communication interface 403 are connected through bus 410 and complete communication with each other.

[0085] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0086] Bus 410 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 410 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0087] The electronic device can execute the augmented reality (AR) data processing method in the embodiments of this application, thereby achieving a combination of Figure 1 and Figure 3 The method and apparatus for processing augmented reality (AR) data are described.

[0088] Furthermore, in conjunction with the augmented reality (AR) data processing methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the augmented reality (AR) data processing methods in the above embodiments.

[0089] In conjunction with the augmented reality (AR) data processing methods in the above embodiments, this application also provides a computer program product, wherein the instructions in the computer program product, when executed by the processor of an electronic device, cause the electronic device to perform any of the augmented reality (AR) data processing methods in the above embodiments.

[0090] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0091] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0092] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0093] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0094] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for processing augmented reality (AR) data, characterized in that, Applied to network edge computing nodes, including: Obtain AR data and the encryption level of the AR data; Based on the encryption level, the encryption density of the AR data is determined, whereby the encryption density represents the proportion of target AR data that needs to be encrypted within the AR data. Based on the encryption density, determine the target AR data in the AR data that needs to be encrypted; The target AR data is encrypted to obtain encrypted AR data; Before encrypting the target AR data to obtain the encrypted AR data, the process also includes: The key temporal features that need to be encrypted in the AR data are extracted using a temporal feature extraction function. The key temporal features include numerical feature values ​​{t1, t2, ..., ti} in the order of display time, where i represents the i-th time. The temporal feature extraction function is obtained by learning the correspondence between the AR data samples and the key scene features labeled according to the content display order of the AR data samples. Determine the position coordinates of the pixel containing the key temporal feature in a preset coordinate system; The encryption weight matrix of the target AR data is calculated based on the encryption weights of pixels at different locations in the preset AR data and the location coordinates. The step of encrypting the target AR data to obtain encrypted AR data includes: The target AR data is encrypted according to the encryption weight matrix to obtain encrypted AR data.

2. The method according to claim 1, characterized in that, The target AR data is encrypted according to the encryption weight matrix to obtain encrypted AR data, including: Obtain the component pixel values ​​of the pixel points corresponding to the target AR data in each color channel; Based on the encryption weight matrix, the component pixel values ​​of each color channel are encrypted to obtain encrypted AR data.

3. The method according to claim 1, characterized in that, Determining the position coordinates of the pixel containing the key temporal feature in the preset coordinate system specifically includes: Determine the three-dimensional position coordinates of the pixel containing the key temporal feature in the three-dimensional coordinate system.

4. The method according to claim 3, characterized in that, The target AR data is encrypted to obtain encrypted AR data, including: The target AR data is encrypted using a multidimensional Gaussian blur function to obtain encrypted AR data; The multidimensional Gaussian blur function is: Where x, y, and z are the x-coordinate, y-coordinate, and height coordinates of the three-dimensional position coordinates, t is the key temporal feature, and r is the encryption level. The encryption density is... Represents the covariance matrix. This represents the average value of the corresponding dimension in the AR data, which contains at least one encryption level and encryption density, with different encryption levels and encryption densities corresponding to different parts of the AR data.

5. The method according to any one of claims 1-4, characterized in that, Acquire AR data, including: Collect image or video data corresponding to real-world scenes; The scene recognition and tracking model identifies the target scene in the image or video data; the scene recognition and tracking model is trained based on the image data and the corresponding scene. AR data is obtained by overlaying a virtual scene model corresponding to the target scene onto the image or video data corresponding to the real scene.

6. The method according to any one of claims 1-4, characterized in that, After obtaining the encrypted AR data, the process also includes: The terminal device receives a request to obtain AR data, the request including a key corresponding to the AR data; If the key verification is successful, the terminal device is authorized to access the AR data.

7. An apparatus for processing augmented reality (AR) data, characterized in that, Applied to network edge computing nodes, including: The acquisition module is used to acquire AR data and the encryption level of the AR data; The first determining module is used to determine the encryption density of the AR data according to the encryption level, wherein the encryption density is used to represent the proportion of target AR data that needs to be encrypted in the AR data; The second determining module is used to determine the target AR data that needs to be encrypted in the AR data based on the encryption density. An encryption module is used to encrypt the target AR data to obtain encrypted AR data; The device further includes: The extraction module is used to extract key temporal features that need to be encrypted from the AR data using a temporal feature extraction function. The key temporal features include numerical feature values ​​{t1, t2, ..., ti} in the order of display time, where i represents the i-th time. The temporal feature extraction function is obtained by learning the correspondence between the AR data samples and the key scene features labeled according to the content display order of the AR data samples. The second determining module is further configured to determine the position coordinates of the pixel containing the key temporal feature in a preset coordinate system; The weight calculation module is used to calculate the encryption weight matrix of the target AR data based on the encryption weights of pixels at different locations in the preset AR data and the location coordinates. The encryption module is further configured to encrypt the target AR data according to the encryption weight matrix to obtain encrypted AR data.

8. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the augmented reality (AR) data processing method as described in any one of claims 1-6.

9. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the augmented reality (AR) data processing method as described in any one of claims 1-6.

10. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device causes the electronic device to perform the augmented reality (AR) data processing method as described in any one of claims 1-6.

Images