Interactive display system, augmented reality device and cloud server

Abstract

The embodiment of the invention relates to an interactive display system, augmented reality equipment and a cloud server, an AR application originally deployed in the augmented reality equipment is deployed to the cloud server, so that AR application development does not need to deeply adapt to the equipment performance of the augmented reality equipment and is not influenced by the performance of the augmented reality equipment, the development difficulty is reduced, and the development period is shortened; the running effect of the AR application is not limited to the hardware performance of the augmented reality equipment, and complex functions can be realized through the AR application deployed in the cloud server; when the new function or the augmented reality device is updated, the AR application does not need to be redeveloped under the local architecture of the augmented reality device, the new function can be flexibly added in the cloud server or the new generation of augmented reality device is adapted, and the flexibility and expansibility of the AR application are improved; and the sensing analysis data is adjusted into the interface rendering data through the mobile terminal, so that the augmented reality equipment can be ensured to present the interface rendering data with a better effect, and the use experience is improved.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the technical field of AR device, in particular to an interactive display system, an augmented reality device and a cloud server. BACKGROUND

[0002] In recent years, with the development of science and technology, the augmented reality (AR) device is very popular among users. At present, the traditional AR application interaction implementation mode in the AR device is to integrate the processor, the memory and other hardware devices on the augmented reality device, to develop a specific running system based on the augmented reality device, and to customize and develop the software suitable for the specific running system.

[0003] However, the above-mentioned traditional AR application interaction implementation mode depends on the hardware performance of the AR device, which limits the running effect of the AR application, and it is difficult to realize complex functions. After the hardware device selection and device change of different models of AR devices, the system and the AR application need to be re-adapted to different device performance, the expansibility is low, and the use experience is poor. SUMMARY

[0004] Therefore, one of the purposes of the embodiment of the present application is to provide an interactive display system, an augmented reality device and a cloud server, which aims to solve the technical problem of low expansibility of the system and the application program of the AR device in the prior art.

[0005] To solve the above technical problems, the embodiment of the present application provides the following technical solutions: In a first aspect, the embodiment of the present application provides an interactive display system, comprising: An augmented reality device, a mobile terminal and a cloud server, the mobile terminal is in communication connection with the augmented reality device and the cloud server respectively, the cloud server deploys at least one AR application, and the AR application is a cloud application running in an AROS system; The augmented reality device is configured to obtain initial interaction data and send the initial interaction data to the mobile terminal; The mobile terminal is configured to send the initial interaction data to the cloud server; The cloud server is configured to select an AR application corresponding to the business type of the initial interaction data as a target AR application among at least one AR application, control the target AR application to process the initial interaction data to obtain interaction analysis data, and send the interaction analysis data to the mobile terminal, and the mobile terminal adjusts the interaction analysis data into interface rendering data meeting the presentation requirements of the augmented reality device, so that the augmented reality device presents the interface rendering data.

[0006] In some embodiments, the mobile terminal deploys a plurality of terminal applications, and the mobile terminal is configured to send the initial interaction data to the cloud server, including: generating a function selection instruction according to the initial interaction data, the function selection instruction being used to represent a service type of the initial interaction data; determining a terminal application corresponding to the data type of the initial interaction data as a target terminal application among the plurality of terminal applications; controlling the target terminal application to perform data format conversion on the initial interaction data to obtain converted initial interaction data; sending the function selection instruction and the converted initial interaction data to the cloud server.

[0007] In some embodiments, the cloud server is configured to select an AR application corresponding to the service type of the initial interaction data as a target AR application among at least one AR application, including: obtaining the function selection instruction sent by the mobile terminal; determining the service type of the initial interaction data according to the function selection instruction; selecting an AR application corresponding to the service type of the initial interaction data as a target AR application among at least one AR application.

[0008] In some embodiments, the cloud server is configured to control the target AR application to process the initial interaction data to obtain interaction analysis data, including: determining a format of the initial interaction data and a standard format supported by the target AR application; in response to the format of the initial interaction data being different from the standard format, controlling the target AR application to convert the initial interaction data into initial interaction data in the standard format; controlling the target AR application to process the initial interaction data in the standard format to obtain interaction analysis data.

[0009] In some embodiments, the interaction analysis data includes media data and presentation instructions, and the mobile terminal adjusts the interaction analysis data into interface rendering data conforming to a presentation requirement of the augmented reality device, including: parsing the interaction analysis data to extract the media data and the presentation instructions, the media data including text data, image data and / or audio data, and the presentation instructions being used to represent characteristic parameters of the augmented reality device for presenting the media data; obtaining presentation parameters of the augmented reality device; adjusting the media data based on the presentation instructions and the presentation parameters to obtain the interface rendering data.

[0010] In some embodiments, the presentation instruction comprises a presentation order of the media data, the presentation parameter comprises a display area size of the text data and the image data, a play parameter value of the audio data, and the mobile terminal adjusts the media data based on the presentation instruction and the presentation parameter to obtain the interface rendering data, comprising: parsing the presentation instruction to extract the presentation order of the media data; adjusting the size of the text data and the image data to conform to the display area size of the text data and the image data, and / or adjusting the play parameter of the audio data to conform to the play parameter value according to the presentation order of the media data, to obtain adjusted text data, image data and / or audio data; sending the adjusted text data, image data and / or audio data to the augmented reality device in the presentation order, so that the augmented reality device presents the adjusted text data, image data and / or audio data, and the adjusted text data, image data and / or audio data are the interface rendering data.

[0011] In some embodiments, the augmented reality device presents the interface rendering data, comprising: displaying the text data and the image data in the display area corresponding to the text data and the image data in the presentation order, and / or playing the audio data.

[0012] In some embodiments, the augmented reality device displays the text data and the image data in the display area corresponding to the text data and the image data in the presentation order, comprising: in response to the actual occupied area of the text data overlapping with the actual occupied area of the image data, displaying the text data and the image data in the order of displaying the image data first and then displaying the text data.

[0013] In a second aspect, the embodiments of the present application provide an augmented reality device, which is configured to: obtain initial interaction data and send the initial interaction data to a mobile terminal, so that the mobile terminal sends the initial interaction data to a cloud server, and the cloud server deploys at least one AR application, and the AR application is a cloud application running on an AROS system; receive interface rendering data sent by the mobile terminal, the interface rendering data being data obtained by the mobile terminal adjusting the interaction analysis data sent by the cloud server to meet the presentation requirements of the augmented reality device, the interaction analysis data being data obtained by the cloud server controlling a target AR application to process the initial interaction data, the target AR application being an AR application corresponding to the service type of the initial interaction data among the at least one AR application; present the interface rendering data.

[0014] In a third aspect, an embodiment of the present application provides a cloud server, the cloud server deploying at least one AR application, the AR application being a cloud application running on an AROS system, and the cloud server being configured to: receive initial interaction data sent by a mobile terminal, the initial interaction data being data obtained by an augmented reality device, and the augmented reality device sending the initial interaction data to the mobile terminal; select an AR application corresponding to the service type of the initial interaction data as a target AR application among the at least one AR application, and control the target AR application to process the initial interaction data to obtain interaction analysis data; send the interaction analysis data to the mobile terminal, the mobile terminal adjusting the interaction analysis data to interface rendering data meeting the presentation requirements of the augmented reality device, and sending the interface rendering data to the augmented reality device to enable the augmented reality device to present the interface rendering data.

[0015] The embodiment of the present application has the following beneficial effects: Different from the prior art, the interaction display system provided by the embodiment of the present application includes an augmented reality device, a mobile terminal and a cloud server, the mobile terminal is in communication connection with the augmented reality device and the cloud server respectively, the cloud server deploys at least one AR application, the AR application is a cloud application running on an AROS system, the augmented reality device is configured to obtain initial interaction data and send the initial interaction data to the mobile terminal, the mobile terminal is configured to send the initial interaction data to the cloud server, the cloud server is configured to select an AR application corresponding to the service type of the initial interaction data as a target AR application among the at least one AR application, control the target AR application to process the initial interaction data to obtain interaction analysis data, and send the interaction analysis data to the mobile terminal, the mobile terminal adjusts the interaction analysis data to interface rendering data meeting the presentation requirements of the augmented reality device, so that the augmented reality device presents the interface rendering data.

[0016] The AR application originally deployed on the augmented reality device is deployed to the cloud server in the embodiments of the present application, the AR application development does not need to be deeply adapted to the device performance of the augmented reality device, is not affected by the performance of the augmented reality device, the development difficulty is reduced, and the development cycle is shortened; and the running effect of the AR application is not limited to the hardware performance of the augmented reality device, various complex functions can be realized through the AR application deployed on the cloud server; in addition, when new functions are added or the augmented reality device is updated, the AR application does not need to be redeveloped under the local architecture of the augmented reality device, the new functions or the new generation of augmented reality devices can be flexibly added on the cloud server, the flexibility and expansibility of the AR application are improved; and the interface rendering data adjusted by the mobile terminal from the interaction analysis data to meet the presentation requirements of the augmented reality device can guarantee that the augmented reality device presents the interface rendering data with better effect, and improves the use experience. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the prior art or the embodiments. Obviously, the drawings described below only show some of the embodiments of the present application, and should not be regarded as a limitation to the protection scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0018] Figure 1a is a structural schematic diagram of an interactive display system provided by some embodiments of the present application; Figure 1b is a schematic diagram of an application scenario of an interactive display system in some embodiments of the present application; Figure 1c is a schematic diagram of an augmented reality device obtaining initial interaction data in some embodiments of the present application; Figure 2 is a schematic diagram of the interaction between the augmented reality device, the mobile terminal and the cloud server in the interactive display system provided by some embodiments of the present application; Figure 3 is a schematic diagram of the cloud server processing initial interaction data and the mobile terminal processing interaction analysis data in some embodiments of the present application; Figure 4a is a schematic diagram of the mobile terminal processing initial interaction data in some embodiments of the present application; Figure 4b is a schematic diagram of the interaction between the mobile terminal and the cloud server in some embodiments of the present application; Figure 5a is a schematic diagram of the cloud server processing initial interaction data in some embodiments of the present application; Figure 5bFig. 1 is a schematic diagram of interaction between a mobile terminal and a cloud server in some embodiments of the present application; Figure Two Figure 6 Fig. 2 is a schematic diagram of processing initial interaction data by an AR application in a cloud server in some embodiments of the present application; Figure 7a Fig. 3 is a schematic diagram of processing interaction analysis data by a mobile terminal in some embodiments of the present application; Figure 7b Fig. 4 is a schematic diagram of interaction between an augmented reality device, a mobile terminal and a cloud server in some embodiments of the present application; Figure Two Figure 7c Fig. 5 is a schematic diagram of interaction between a mobile terminal and an augmented reality device in some embodiments of the present application; Figure 8a Fig. 6 is a schematic diagram of rendering data of a presentation interface by an augmented reality device in some embodiments of the present application; Figure 8b Fig. 7 is a schematic diagram of interaction between a mobile terminal and an augmented reality device in some embodiments of the present application; Figure Two Figure 9a Fig. 8 is a schematic diagram of rendering data of a presentation interface by an augmented reality device in some embodiments of the present application; Figure Two Figure 9b Fig. 9 is a schematic diagram of interaction between a mobile terminal and an augmented reality device in some embodiments of the present application; Figure Three Figure 10a Fig. 10 is a schematic diagram of a flow of steps performed by an augmented reality device in some embodiments of the present application; Figure 10b Fig. 11 is a schematic diagram of a flow of steps performed by a cloud server in some embodiments of the present application. DETAILED DESCRIPTION

[0019] In order to make the objects and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. The following detailed description of the embodiments of the present application in the drawings is not intended to limit the scope of the present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0020] ​​​​​It should be noted that, if not constitute a conflict, the following described embodiments of the application are involved in each of the technical features can be combined with each other, and are within the scope of the protection of the present application. In addition, although the functional module division is carried out in the device or structure schematic diagram, the logical order is shown in the flow chart, but in some cases, the steps shown or described can be executed in different from the module division in the device, or different from the order in the flow chart. In addition, the "first", "second", "third" and other similar expressions used herein are not limited to the order of data and execution, only for the purpose of facilitating the explanation and the function and the same or similar items with the same or similar items are distinguished, and can not be understood as indicating or implying the relative importance or implicit indication of the number of technical features.

[0021] Unless otherwise defined, the technical terms and scientific terms used in the specification are the same as those commonly understood by those skilled in the art to which the present application belongs. The terms used in the specification are only for the purpose of describing the specific embodiments and are not used to limit the present application. It should be understood that the term "and / or" used in the specification includes any and all combinations of one or more listed items.

[0022] Please refer to Figure 1a and Figure 1b , Figure 1a The structure of the interactive display system provided by some embodiments of the present application is schematically shown, Figure 1b The application scenario of the interactive display system in some embodiments of the present application is schematically shown.

[0023] As Figure 1a and Figure 1b The interactive display system 1000 includes an augmented reality device 100, a mobile terminal 200 and a cloud server 300, the mobile terminal 200 is respectively connected with the augmented reality device 100 and the cloud server 300 through a network, the network includes but is not limited to the Internet, intranet, LAN, mobile communication network and combination thereof.

[0024] Specifically, the cloud server 300 deploys at least one AR application (for example Figure 1bAs shown in AR applications 301, 302, 303, and 304, AR applications are cloud applications running on the AROS system. AR applications (such as SLAM positioning and real-time 3D rendering) require a large amount of computing resources. The local CPU, GPU, and memory of the augmented reality device 100 are difficult to efficiently handle the concurrent operation of multiple AR applications. In addition, these AR applications not only require more high-performance processors, larger capacity memory, and larger batteries, but also require more heat dissipation components to maintain the normal operation of the augmented reality device 100. Therefore, deploying AR applications on the augmented reality device 100 will make the augmented reality device 100 bloated and complex.

[0025] In this embodiment, computationally intensive AR applications (such as image recognition, 3D rendering, and environment modeling) are offloaded from the augmented reality device 100 and deployed to the cloud server 300. The augmented reality device 100 retains only the necessary sensors, display module, and communication module. By deploying the AR applications originally deployed on the augmented reality device 100 to process its initial sensory data to the cloud server 300, the hardware requirements of the augmented reality device 100 are significantly reduced (e.g., CPU / GPU specifications, battery capacity, and memory). The cloud server 300, as a high-performance backend, can efficiently handle concurrent requests from multiple AR applications. Furthermore, the AROS system, as a cloud-based operating system, is responsible for coordinating application scheduling, data synchronization, and resource allocation, optimizing the low latency and real-time performance of the augmented reality device 100 for AR scenarios. This approach enables the augmented reality device 100 to achieve local lightweighting, meeting lightweight requirements, and allows for extremely lightweight development of AR applications on the cloud server 300. It is understood that... Figure 1b Four AR applications are shown only as an illustration and do not impose any limitations on the type or number of AR applications.

[0026] Please see Figure 1c and Figure 2 In this embodiment, the augmented reality device 100 is configured to acquire initial sensory interaction data and send the initial sensory interaction data to the mobile terminal 200. The initial sensory interaction data includes, but is not limited to, audio data, video data, image data, visual tracking data, gesture recognition data, head posture data, voice command data, spatial anchor point data, motion trajectory data, and display interface operation command data. The file types of the initial sensory interaction data include, but are not limited to, text, voice, video, and images (e.g., text, voice, video, and images). Figure 1c (The text 101, voice 102, and image 103 shown). Specifically, in this embodiment, the initial interaction data consists of raw data collected by the augmented reality device (such as AR glasses) from the environment in which the augmented reality device is located, and raw data of user operation control commands received or perceived by the AR glasses.

[0027] In the embodiment, the identification of the initial interaction data is completed by the cloud server 300 or the mobile terminal 200, that is, the augmented reality device 100 only retains basic functions such as audio acquisition, video acquisition, control signal sensing, screen-off, shutdown and the like, and the augmented reality device 100 only needs to send all the data collected or sensed and containing the initial interaction data to the cloud server 300 or the mobile terminal 200. The identification of the initial interaction data can be completed by the cloud server 300 according to needs, which can reduce the hardware performance requirement of the mobile terminal 200. The identification of the initial interaction data can be completed by the mobile terminal 200 according to needs, which can filter and / or convert data, and reduce the data amount in the transmission process of uploading to and downloading from the cloud server 300.

[0028] After receiving the initial interaction data sent by the augmented reality device 100, the mobile terminal 200 is configured to send the initial interaction data to the cloud server 300.

[0029] The cloud server 300 is configured to select an AR application corresponding to the service type of the initial interaction data as a target AR application among the at least one AR application (that is, select an AR application suitable for processing the initial interaction data as the target AR application among the at least one AR application), control the target AR application to process the initial interaction data to obtain interaction analysis data, and send the interaction analysis data to the mobile terminal 200. The interaction analysis data is structured result data obtained by data analysis and processing of the initial interaction data, for example, result data obtained by processing of original data such as feature extraction, algorithm processing, pattern recognition, data fusion and the like. Specifically, it can be target coordinate parameters obtained by processing of visual tracking data, a three-dimensional map grid obtained by processing of environment sensing data, instruction code result data obtained by analyzing gesture recognition images and the like.

[0030] In the embodiment, the augmented reality device 100 is in communication connection with the mobile terminal 200, and is not directly in communication connection with the cloud server 300, and transmits and processes data through the data relay of the mobile terminal 200. In this way, the high-performance communication unit does not need to be arranged on the augmented reality device 100, but the high-performance communication unit on the mobile terminal 200 (such as a mobile phone or the like) is used to communicate and transmit data with the cloud server 300, so that the stability and efficiency of data transmission can be ensured, and the data interaction delay can be reduced.

[0031] In an embodiment of the present application, after receiving the interaction analysis data sent by the cloud server 300, the mobile terminal 200 is further configured to perform visual adaptation or rendering on the interaction analysis data, so that the interaction analysis data meets the presentation requirements of the augmented reality device 100, to obtain interface rendering data meeting the presentation requirements of the augmented reality device 100, and send the interface rendering data to the augmented reality device 100 through the network, so that the augmented reality device 100 presents the interface rendering data. Specifically, the interaction analysis data can be visualized and adapted (such as coordinate conversion, graphic rendering, and interface layout conversion), and finally form data that can be directly presented on the near-eye display device.

[0032] It should be understood that in another embodiment of the present application, meeting the presentation requirements of the augmented reality device means meeting the display parameters of the display device of the augmented reality device, the playback requirements of the audio playback device, and the like. For example, when the interaction analysis data includes images and voice, the resolution of the images is adjusted so that the images can be displayed on the display area of the display device of the augmented reality device 100 with better display effect, and the volume and channel of the voice are adjusted so that the voice can be played on the augmented reality device 100 with better sound effect.

[0033] For example, referring to Figure 3 , the initial interaction data obtained by the augmented reality device 100 includes voice 102. After the mobile terminal 200 sends the voice 102 to the cloud server 300, the cloud server 300 selects the AR application 302 corresponding to the service type of the initial interaction data as the target AR application among the AR application 301, the AR application 302, the AR application 303, and the AR application 304, controls the AR application 302 to process the voice 102, and obtains the interaction analysis data (i.e., the voice 102A). The voice 102A is sent to the mobile terminal 200.

[0034] It can be understood that the cloud server 300 controls the AR application 302 to process the voice 102, and the processing operation includes but is not limited to removing noise in the voice 102, analyzing the content of the voice 102, and the like, to obtain the interaction analysis data. Of course, when the initial interaction data also includes other types of data, other target AR applications can also be used to perform corresponding processing operations on the other types of data, and are not limited to the above voice processing operations.

[0035] After receiving the voice 102A sent by the cloud server 300, the mobile terminal 200 adjusts the volume and channel of the voice 102A, to obtain the voice 102B meeting the presentation requirements of the augmented reality device 100. Of course, when the interaction analysis data also includes other types of data, corresponding processing operations can also be performed on the other types of data to meet the presentation requirements of the augmented reality device 100, and are not limited to the above voice processing operations.

[0036] In the embodiments of the present application, the augmented reality device 100 can be a smart AR glasses. Obviously, in order to obtain the initial interaction data (i.e. text data, voice data and image data, etc.), the augmented reality device 100 needs to be equipped with corresponding devices and sensors (such as a camera, a voice collection device) and the like.

[0037] In the embodiments of the present application, the mobile terminal 200 can be any suitable terminal device such as a tablet computer, a smart phone, a smart bracelet and a smart watch, of course, the mobile terminal 200 can also be any other suitable type of device or apparatus, and is not limited to the smart phone shown in the embodiments of the present application. Obviously, in order to process the interaction analysis data to adjust the interaction analysis data into the interface rendering data meeting the presentation requirements of the augmented reality device 100, the mobile terminal 200 needs to be equipped with corresponding software (such as image processing, voice processing, text processing, interface rendering software, etc.) and hardware (such as a memory, a processor, an analog-to-digital converter, a controller, etc.), so that the mobile terminal 200 has certain computing and control capabilities, thereby being able to adjust the interaction analysis data to make the interaction analysis data meet the presentation requirements of the augmented reality device 100.

[0038] In the embodiments of the present application, the cloud server 300 can be any suitable cloud server such as an ECS (Elastic Compute Service) of Ali Cloud, a lightweight application server and a GPU / FPGA cloud server, an EC2, a LightSail and an Outposts of Amazon Web Services (AWS), of course, the cloud server 300 can also be any other suitable cloud server. It is easy to understand that, in order to deploy the AR application to the cloud server 300, the environment, configuration, capacity and the like of the cloud server 300 need to be equipped as the target environment, the target configuration and the target capacity required, so that the AR application can be adapted to process the initial interaction data.

[0039] As described above, in the embodiments of the present application, by deploying the AR application originally deployed on the augmented reality device to the cloud server, the AR application development does not need to be deeply adapted to the device performance of the augmented reality device, is not affected by the hardware performance of the augmented reality device, reduces the development difficulty and shortens the development cycle; and the running effect of the AR application is not limited to the hardware performance of the augmented reality device, various complex functions can be realized through the AR application deployed on the cloud server; in addition, when a new function is added or the augmented reality device is updated, the AR application does not need to be redeveloped under the local architecture of the augmented reality device, the new function can be flexibly added or the new generation of augmented reality device can be adapted on the cloud server, thereby improving the flexibility and expansibility of the AR application; and by adjusting the interaction analysis data into the interface rendering data meeting the presentation requirements of the augmented reality device through the mobile terminal, the augmented reality device can present the interface rendering data with better effect, thereby improving the user experience.

[0040] In some embodiments, the mobile terminal 200 is deployed with a plurality of terminal applications, the terminal applications being configured to perform data format conversion on the initial interaction data to obtain converted initial interaction data. Different terminal applications correspond to processing initial interaction data of different data types, i.e., performing data format conversion on initial interaction data of different types.

[0041] For example, referring to Figure 4a , the mobile terminal 200 is deployed with three terminal applications, i.e., terminal application 201, terminal application 202, and terminal application 203, wherein the terminal application 201 is configured to perform recognition processing on initial interaction data of a text type to remove part of noise data, the terminal application 202 is configured to perform recognition processing on initial interaction data of an audio type to remove part of noise data, and the terminal application 203 is configured to perform recognition processing on initial interaction data of an image type to remove part of noise data.

[0042] For example, referring to Figure 4b , the mobile terminal is configured to send the initial interaction data after recognition processing to a cloud server, specifically including the following steps: S401: The mobile terminal 200 generates a function selection instruction according to the initial interaction data.

[0043] S402: The mobile terminal 200 determines a terminal application corresponding to the data type of the initial interaction data as a target terminal application among the plurality of terminal applications.

[0044] S403: The mobile terminal 200 controls the target terminal application to perform data format conversion on the initial interaction data to obtain converted initial interaction data.

[0045] S404: The mobile terminal 200 sends the function selection instruction and the converted initial interaction data to the cloud server 300.

[0046] The function selection instruction is used to represent the service type of the initial interaction data. For example, if the text in the initial interaction data is to be translated in real time, the function selection instruction is a translation instruction, which instructs the cloud server 300 to perform the service function of real-time translation on the text. For example, if the image in the initial interaction data is to be processed for picture quality improvement, the function selection instruction is a picture quality improvement instruction, which instructs the cloud server 300 to perform the service function of quality improvement on the image.

[0047] It can be understood that the service type refers to the type of service function to be implemented, and has no necessary connection with the data type of the initial interaction data. For example, when performing a real-time translation service function, the initial interaction data of pure text, text in the image of the initial interaction data, and voice of the initial interaction data can be translated in real time, and the real-time translation service function has no necessary connection with the data type of the object to be translated (i.e., pure text, text in the image, and voice of the initial interaction data).

[0048] In step S401, the service type of the initial interaction data is determined according to the service function to be performed on the initial interaction data, and then the corresponding function selection instruction is generated according to the service type of the initial interaction data. For example, if real-time translation is required for the initial interaction data, the service type of the initial interaction data is determined to be translation, and the corresponding function selection instruction is generated as a translation instruction.

[0049] In step S402, the data type of the initial interaction data can be audio, text, or image, and the mobile terminal 200 determines a terminal application corresponding to the data type of the initial interaction data as the target terminal application among a plurality of terminal applications, for example, please refer to Figure 4a When the initial interaction data is audio type data, the terminal application 202 corresponding to the audio type is determined as the target terminal application, and then the terminal application 202 is used to perform data format conversion on the audio type initial interaction data.

[0050] Of course, the initial interaction data can also include data of multiple data types. When including data of multiple data types, a plurality of terminal applications corresponding to the data types of the initial interaction data are determined as the target terminal applications, for example, please refer to Figure 4a When the initial interaction data includes text type and audio type data, the terminal applications 201 and 202 corresponding to the text type and the audio type are determined as the target terminal applications, and the terminal applications 201 and 202 are used to perform data format conversion on the text type and the audio type initial interaction data, respectively.

[0051] In step S403, after determining the target terminal application, the mobile terminal 200 controls the determined target terminal application to perform data format conversion on the initial interaction data to obtain converted initial interaction data. For example, please refer to Figure 4a The initial interaction data is image type data (i.e., image 103) and the data format is RAW format, the terminal application 203 is determined as the target terminal application, and the terminal application 203 is used to perform data format conversion on the RAW format image 103 to obtain the converted data format image 103A in JPEG format.

[0052] In the embodiments of the present application, the data format conversion processing of the initial interaction data is performed to reduce the data volume, so as to reduce the data volume transmitted by the mobile terminal 200 to the cloud server 300, improve the transmission speed, and reduce the bandwidth requirement.

[0053] In step S404, after obtaining the function selection instruction and the format-converted initial interaction data, the function selection instruction and the format-converted initial interaction data are transmitted to the cloud server 300 through the network.

[0054] In some embodiments, referring to Figure 5a and Figure 5b , the cloud server is configured to select, among the at least one AR application, an AR application corresponding to the service type of the initial interaction data as a target AR application, including the following steps: S501: The cloud server 300 acquires the function selection instruction sent by the mobile terminal.

[0055] S502: The cloud server 300 determines the service type of the initial interaction data according to the function selection instruction.

[0056] S503: The cloud server 300 selects, among the at least one AR application, an AR application corresponding to the service type of the initial interaction data as a target AR application.

[0057] In step S501, after receiving the function selection instruction sent by the mobile terminal, the cloud server 300 can acquire the function selection instruction sent by the mobile terminal from a storage medium.

[0058] In step S502, after acquiring the function selection instruction, the cloud server 300 determines the service type of the initial interaction data according to the function selection instruction. For example, the function selection instruction is a translation instruction, and it is determined that the service type of the initial interaction data is translation, that is, the business function performed on the initial interaction data is real-time translation.

[0059] In step S503, after determining the service type of the initial interaction data, the cloud server 300 selects, among the at least one deployed AR application, an AR application corresponding to the service type of the initial interaction data as a target AR application, that is, an AR application used to perform a corresponding business function. For example, the service type of the initial interaction data is translation, and it is determined that the AR application 301 corresponding to translation is the target AR application, and the subsequent real-time translation business function is performed on the initial interaction data by using the AR application 301.

[0060] In some embodiments, referring to Figure 5b , the cloud server is configured to control the target AR application to process the initial interaction data to obtain interaction analysis data, including the following steps: S504: The cloud server 300 determines the format of the initial interaction data and the standard format supported by the target AR application.

[0061] S505: The cloud server 300 controls the target AR application to convert the initial interaction data into initial interaction data in the standard format in response to the format of the initial interaction data being different from the standard format.

[0062] S506: The cloud server 300 controls the target AR application to process the initial interaction data in the standard format to obtain interaction analysis data.

[0063] In step S504, after determining the target AR application, the cloud server 300 acquires the format of the initial interaction data and the standard format supported by the target AR application, thereby determining the format of the initial interaction data and the standard format supported by the target AR application.

[0064] In step S505, after determining the format of the initial interaction data and the standard format supported by the target AR application, the cloud server 300 compares the format of the initial interaction data with the standard format, and determines whether the format of the initial interaction data is the format supported by the target AR application. If the format of the initial interaction data is the same as the standard format, the initial interaction data can be directly input to the target AR application, and no data format conversion is needed. If the format of the initial interaction data is different from the standard format, it indicates that the initial interaction data cannot be directly input to the target AR application, and the cloud server 300 controls the target AR application to convert the initial interaction data into initial interaction data in the standard format.

[0065] In step S506, after converting the initial interaction data into initial interaction data in the standard format, the cloud server 300 controls the target AR application to process the initial interaction data in the standard format to obtain interaction analysis data.

[0066] For example, referring to Figure 5b and Figure 6 , the initial interaction data is audio data, and the data format thereof is MPS format when transmitted to the cloud server 300. The target AR application is the AR application 301, and the standard format of audio supported by the AR application 301 is PCM format. Obviously, the MPS format is different from the PCM format, and the cloud server 300 controls the AR application 301 to convert the audio data in the MPS format into audio data in the PCM format, and controls the AR application 301 to process the audio data in the PCM format to obtain interaction analysis data.

[0067] In some embodiments, the interaction analysis data includes media data and presentation instructions, the media data is data required to be presented on the augmented reality device, and the media data includes text data, image data, and / or audio data, etc., and the presentation instructions are instructions for indicating presentation parameters of the media data, i.e., the presentation instructions are used to represent characteristic parameters of the augmented reality device presenting the media data, and the characteristic parameters (i.e., the presentation parameters) include font size, style, and display position of the text data, resolution and display position of the image data, and timbre and volume of the audio data, etc.

[0068] Specifically, please refer to Figure 7a and Figure 7b , the mobile terminal adjusts the interaction analysis data into interface rendering data conforming to the presentation requirements of the augmented reality device, including: S701: The mobile terminal 200 parses the interaction analysis data and extracts the media data and the presentation instructions.

[0069] S702: The mobile terminal 200 obtains the presentation parameters of the augmented reality device.

[0070] S703: The mobile terminal 200 adjusts the media data based on the presentation instructions and the presentation parameters to obtain the interface rendering data.

[0071] Specifically, after receiving the interaction analysis data sent by the cloud server 300, the mobile terminal 200 parses the interaction analysis data, extracts the media data and the presentation instructions in the interaction analysis data, and obtains the presentation parameters of the augmented reality device 100 from the local storage or the augmented reality device 100. The presentation parameters include but are not limited to the size, resolution, display brightness, display position of images / texts, and supported styles or formats of display components of the augmented reality device 100, the volume limit, supported timbre, and sound channels of the audio playback components of the augmented reality device 100, etc.

[0072] In step S703, after obtaining the media data, the presentation instructions, and the presentation parameters, the mobile terminal 200 adjusts the format, style, display position, display parameters, volume size, and sound channels of the media data (i.e., text data, image data, and / or audio data) according to the presentation instructions and the presentation parameters, so that the media data conforms to the presentation requirements of the augmented reality device 100, thereby obtaining the interface rendering data conforming to the presentation requirements of the augmented reality device 100.

[0073] In some embodiments, please refer to Figure 7c , the mobile terminal adjusts the media data based on the presentation instructions and the presentation parameters to obtain the interface rendering data, including: S7031: The mobile terminal 200 parses the presentation instructions and extracts the presentation order of the media data.

[0074] S7032: The mobile terminal 200 adjusts the size of the text data and the image data to conform to the display area size of the text data and the image data according to the presentation order of the media data, and / or adjusts the playing parameter of the audio data to conform to the playing parameter value, to obtain the adjusted text data, image data and / or audio data.

[0075] S7033: The mobile terminal 200 sends the adjusted text data, image data and / or audio data to the augmented reality device in the presentation order, so that the augmented reality device presents the adjusted text data, image data and / or audio data.

[0076] In this embodiment, the presentation instruction includes the presentation order of the media data, and the presentation parameter includes the display area size of the text data and the image data, and the playing parameter value of the audio data. The display area refers to the area of the display component of the augmented reality device 100 for displaying the text data or the image data, and the playing parameter value refers to the audio format, the channel and the timbre supported by the audio playing component of the augmented reality device 100, and the volume limit of the audio playing component, etc.

[0077] In step S7031, the mobile terminal 200 parses the presentation instruction and extracts the presentation order of the media data, i.e. the presentation order of the text data, the image data and / or the audio data. It should be understood that any one or more of the text data, the image data and the audio data can be presented simultaneously, and the display position of the text data and the image data can be set according to actual needs, which is the position of the display area of the augmented reality device 100 for displaying the text data or the image data.

[0078] In step S7032, the mobile terminal 200 adjusts the size of the text data and the image data to conform to the display area size of the text data and the image data according to the presentation order of the media data, so that the text data and the image data can be displayed in the display area of the text data and the image data with better display effect, and / or adjusts the playing parameter of the audio data to conform to the playing parameter value, so that the audio data can be played in the audio playing component of the augmented reality device 100 with better sound effect, to finally obtain the adjusted text data, image data and / or audio data, which is the interface rendering data.

[0079] It can be understood that adjusting the size of the text data to be consistent with the size of the display area of the text data means adjusting the size of the text data to a target size, so that the adjusted text data can be displayed completely and clearly in the display area of the text data. The target size is less than or equal to the size of the display area of the text data, and the designer can adjust the text data to the size with the best display effect according to actual needs. Similarly, adjusting the size of the image data to be consistent with the size of the display area of the image data means adjusting the size of the image data to a reference size, so that the adjusted image data can be displayed completely and clearly in the display area of the image data. The reference size is less than or equal to the size of the display area of the image data, and the designer can adjust the image data to the size with the best display effect according to actual needs.

[0080] In step S7033, the mobile terminal 200 sends the adjusted text data, image data and / or audio data to the augmented reality device 100 in the presentation order through the network, so that the augmented reality device 100 presents the adjusted text data, image data and / or audio data.

[0081] For example, in some embodiments, referring to Figure 8a and Figure 8b , the augmented reality device presents the interface rendering data, including: S801: The augmented reality device 100 displays the text data and the image data in the display area corresponding to the text data and the image data according to the presentation order, and / or plays the audio data.

[0082] For example, after receiving the adjusted text data, image data and / or audio data sent by the mobile terminal 200, the augmented reality device 100 presents the interface rendering data (i.e. the adjusted text data, image data and / or audio data) in the corresponding components according to the presentation order, that is, displays the text data and the image data in the display area corresponding to the text data and the image data in the display component, and / or controls the audio playing component to play the audio content corresponding to the audio data.

[0083] It can be easily understood that the presentation order can be determined by the cloud server 300 according to actual display needs after processing the obtained text data, image data and / or audio data. Or the presentation order can also be the order of obtaining the text data, image data and / or audio data directly.

[0084] In some embodiments, referring to Figure 8aAs shown, the interface rendering data includes text data, image data and audio data, the overall display area 10 of the display component of the augmented reality device 100 is divided into a display area 11 corresponding to the image data and a display area 12 corresponding to the text data, the augmented reality device 100 displays the text data (i.e. "as shown in the figure above, it is a laboratory") in the display area 12 corresponding to the text data in the display component, displays the image data (i.e. an image of a laboratory) in the display area 11 corresponding to the image data in the display component, and controls the audio playback component 13 to play the audio content corresponding to the audio data.

[0085] It can be understood that, due to the limitation of the hardware of the augmented reality device, special design, more text data or larger text data, etc., the actual occupied area of the text data and the actual occupied area of the image data may overlap when displaying the text data and the image data in practice, and if the text data and the image data are displayed at the same time, it is necessary to ensure that the image data does not block the text data, so that the text data and the image data have better display effect.

[0086] Specifically, please refer to Figure 9a and Figure 9b , the augmented reality device displays the text data and the image data in the display area corresponding to the text data and the image data according to the presentation order, including: S8011: In response to the fact that the actual occupied area of the text data overlaps with the actual occupied area of the image data, the augmented reality device 100 displays the text data and the image data according to the order of displaying the image data first and then displaying the text data.

[0087] In this embodiment, the actual occupied area refers to the actual area occupied by the text data or the image data when displayed, and the size of the actual occupied area of the text data or the image data may exceed the size of the display area corresponding to the text data or the image data.

[0088] For example, when it is detected that the actual occupied area of the text data overlaps with the actual occupied area of the image data, in order to make the image data not block the text data, the augmented reality device 100 displays the text data and the image data according to the order of displaying the image data first and then displaying the text data, i.e. displaying the image data on the lower layer first and displaying the text data on the upper layer, so that the image data is located below the layer of the text data, thereby the image data does not block the text data, and the text data and the image data have better display effect.

[0089] Please refer to Figure 9aAs shown, the interface rendering data includes text data, image data and audio data, the overall display area 10 of the display component of the augmented reality device 100 is divided into a display area 11 corresponding to the image data and a display area 12 corresponding to the text data, the augmented reality device 100 displays the image data (i.e. an image of a laboratory) in the display area 11 corresponding to the image data and displays the text data (i.e. "the image shown in the above figure is a laboratory") in the display area 12 corresponding to the text data, at this time, the actual occupied area of the text data overlaps with the actual occupied area of the image data, the augmented reality device 100 displays the text data and the image data in the order of displaying the image data first and then displaying the text data, and displays the image data layer below the text data layer.

[0090] To sum up, the interactive display system provided by the embodiments of the present application includes an augmented reality device, a mobile terminal and a cloud server, the mobile terminal is in communication connection with the augmented reality device and the cloud server respectively, the cloud server deploys at least one AR application, the AR application is a cloud application running on an AROS system, the augmented reality device is configured to obtain initial interaction data and send the initial interaction data to the mobile terminal, the mobile terminal is configured to send the initial interaction data to the cloud server, the cloud server is configured to select an AR application corresponding to the business type of the initial interaction data as a target AR application from the at least one AR application, control the target AR application to process the initial interaction data to obtain interaction analysis data, and send the interaction analysis data to the mobile terminal, and the mobile terminal adjusts the interaction analysis data into interface rendering data meeting the presentation requirements of the augmented reality device, so that the augmented reality device presents the interface rendering data.

[0091] The embodiments of the present application deploy the AR application originally deployed on the augmented reality device to the cloud server, the AR application development does not need to be deeply adapted to the device performance of the augmented reality device, is not affected by the hardware performance of the augmented reality device, reduces the development difficulty and shortens the development cycle; and the running effect of the AR application is not limited by the hardware performance of the augmented reality device, various complex functions can be realized through the AR application deployed on the cloud server; in addition, when a new function is added or the augmented reality device is updated, the AR application does not need to be redeveloped under the local architecture of the augmented reality device, the new function can be flexibly added or the new generation of augmented reality device can be adapted on the cloud server, the flexibility and expansibility of the AR application are improved; and the interaction analysis data is adjusted into interface rendering data meeting the presentation requirements of the augmented reality device through the mobile terminal, which can ensure that the augmented reality device presents the interface rendering data with better effect and improves the use experience.

[0092] The embodiments of the present application also provide an augmented reality device, please refer to Figure 10a The augmented reality device is configured to perform the following steps S1001-S1003: S1001: Obtain initial interaction data, and send the initial interaction data to a mobile terminal, so that the mobile terminal sends the initial interaction data to a cloud server.

[0093] In step S1001, the cloud server deploys at least one AR application, and the AR application is a cloud application running in an AROS system. For example, an augmented reality device obtains initial interaction data through sensors equipped on the augmented reality device, and the initial interaction data includes, but is not limited to, text data, voice data, and image data. The initial interaction data is sent to a mobile terminal, so that the mobile terminal sends the initial interaction data to a cloud server.

[0094] S1002: Receive interface rendering data sent by the mobile terminal, the interface rendering data is data obtained by the mobile terminal adjusting interaction analysis data sent by the cloud server to meet the presentation requirements of the augmented reality device, and the interaction analysis data is data obtained by the cloud server controlling a target AR application to process the initial interaction data, and the target AR application is an AR application corresponding to the business type of the initial interaction data in the at least one AR application.

[0095] In step S1002, the mobile terminal sends the initial interaction data to the cloud server, the cloud server selects an AR application corresponding to the business type of the initial interaction data as a target AR application from the at least one AR application, controls the target AR application to process the initial interaction data, obtains interaction analysis data, and sends the interaction analysis data to the mobile terminal. The mobile terminal adjusts the interaction analysis data sent by the cloud server to interface rendering data that meets the presentation requirements of the augmented reality device, sends the interface rendering data to the augmented reality device, and the augmented reality device receives the interface rendering data sent by the mobile terminal.

[0096] S1003: Present the interface rendering data.

[0097] In step S1003, the interface rendering data includes text data, image data, and / or audio data. After receiving the interface rendering data sent by the mobile terminal, the augmented reality device presents the interface rendering data, that is, displays the text data and the image data on a display component, and controls an audio playback component to play audio content corresponding to the audio data.

[0098] The embodiment of the application also provides a cloud server, and the cloud server deploys at least one AR application, please refer to Figure 10b , the cloud server is configured to perform the following steps S1004-S1006: S1004: Receive initial interaction data sent by a mobile terminal, and the initial interaction data is data obtained by an augmented reality device, and the augmented reality device sends the initial interaction data to the mobile terminal.

[0099] In step S1004, the cloud server receives the initial interaction data sent by the mobile terminal, the initial interaction data being data acquired by the augmented reality device through the sensors equipped on the augmented reality device, the augmented reality device sending the initial interaction data to the mobile terminal, and the mobile terminal sending the initial interaction data to the cloud server.

[0100] In step S1005, an AR application corresponding to the service type of the initial interaction data is selected as a target AR application from among the at least one AR application deployed on the cloud server, and the target AR application is controlled to process the initial interaction data to obtain interaction analysis data.

[0101] In step S1005, an AR application corresponding to the service type of the initial interaction data is selected as a target AR application from among the at least one AR application deployed on the cloud server, and the target AR application is controlled to process the initial interaction data to obtain interaction analysis data.

[0102] In step S1006, the interaction analysis data is sent to the mobile terminal, the mobile terminal adjusts the interaction analysis data into interface rendering data conforming to the presentation requirements of the augmented reality device, and sends the interface rendering data to the augmented reality device to enable the augmented reality device to present the interface rendering data.

[0103] In step S1006, the interaction analysis data is sent to the mobile terminal, the mobile terminal adjusts the interaction analysis data into interface rendering data conforming to the presentation requirements of the augmented reality device, and sends the interface rendering data to the augmented reality device to enable the augmented reality device to present the interface rendering data.

[0104] Those skilled in the art can understand that the embodiments provided in the present application are only illustrative, the writing order of each step in the method of the embodiments does not mean a strict execution order and does not constitute any limitation on the implementation process, and the order can be adjusted, combined and deleted according to actual needs. The modules or sub-modules, units or sub-units, etc. in the device or system of the embodiments can be combined, divided and deleted according to actual needs. For example, the division of the unit is only a logical function division, and another division mode can also be used in actual implementation. For example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0105] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus a general hardware platform, and of course, can also be implemented by hardware. Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of the method. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM), etc.

[0106] It should be noted that the above embodiments are intended to illustrate the technical concepts and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and implement it accordingly, and cannot be used to limit the scope of protection of the present application. Those skilled in the art can understand that all or part of the processes of the above-mentioned embodiments can be modified according to the technical solutions described in the embodiments of the present application, or some technical features can be replaced equivalently. It can be understood that these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application, and should be regarded as equivalent changes and modifications based on the embodiments of the present application, and should belong to the scope of protection of the claims of the present application.

Claims

1. An interactive display system, characterized in that, include: The system includes an augmented reality device, a mobile terminal, and a cloud server. The mobile terminal is communicatively connected to both the augmented reality device and the cloud server. The cloud server deploys at least one AR application, which is a cloud application running on the AROS system. The augmented reality device is configured to acquire initial interaction data and send the initial interaction data to the mobile terminal; The mobile terminal is configured to send the initial sensory data to the cloud server; The cloud server is configured to select, among at least one AR application, an AR application corresponding to the service type of the initial interactive data as the target AR application, control the target AR application to process the initial interactive data, obtain interactive analysis data, and send the interactive analysis data to the mobile terminal. The mobile terminal adjusts the interactive analysis data into interface rendering data that meets the presentation requirements of the augmented reality device, so that the augmented reality device can present the interface rendering data.

2. The system according to claim 1, characterized in that, The mobile terminal deploys multiple terminal applications, and the mobile terminal is configured to send the initial interaction data to the cloud server, including: The function selection instruction is generated based on the initial interaction data, and the function selection instruction is used to characterize the service type of the initial interaction data; Among the multiple terminal applications, the terminal application corresponding to the data type of the initial sensory data is determined as the target terminal application; The target terminal application is controlled to convert the initial sensory data into a data format to obtain the converted initial sensory data. The function selection command and the converted initial sensory data are sent to the cloud server.

3. The system according to claim 2, characterized in that, The cloud server is configured to select, among at least one of the AR applications, the AR application corresponding to the service type of the initial interactive data as the target AR application, including: Obtain the function selection instruction sent by the mobile terminal; The service type of the initial sensory data is determined according to the function selection instruction; Among the various AR applications, select the AR application that corresponds to the service type of the initial sensory data as the target AR application.

4. The system according to claim 3, characterized in that, The cloud server is configured to control the target AR application to process the initial sensory interaction data to obtain sensory interaction analysis data, including: Determine the format of the initial sensory data and the standard format supported by the target AR application; In response to the fact that the format of the initial sensory data is different from the standard format, the target AR application is controlled to convert the initial sensory data into the standard format initial sensory data; The target AR application is controlled to process the initial sensory interaction data in the standard format to obtain sensory interaction analysis data.

5. The system according to any one of claims 1-4, characterized in that, The interactive analysis data includes media data and presentation instructions. The mobile terminal adjusts the interactive analysis data into interface rendering data that meets the presentation requirements of the augmented reality device, including: The sensory analysis data is parsed to extract the media data and the presentation instructions. The media data includes text data, image data, and / or audio data. The presentation instructions are used to characterize the feature parameters of the augmented reality device in presenting the media data. Obtain the rendering parameters of the augmented reality device; Based on the presentation instructions and the presentation parameters, the media data is adjusted to obtain the interface rendering data.

6. The system according to claim 5, characterized in that, The presentation instructions include the presentation order of the media data, and the presentation parameters include the display area size of the text data and the image data, and the playback parameter values ​​of the audio data. Based on the presentation instructions and the presentation parameters, the mobile terminal adjusts the media data to obtain the interface rendering data, including: Parse the presentation instructions and extract the presentation order of the media data; Based on the presentation order of the media data, adjust the size of the text data and the image data to match the display area size of the text data and the image data, and / or adjust the playback parameters of the audio data to match the playback parameter values, to obtain the adjusted text data, image data and / or audio data; The adjusted text data, image data, and / or audio data are sent to the augmented reality device in the order of presentation, so that the augmented reality device can present the adjusted text data, image data, and / or audio data, which are the interface rendering data.

7. The system according to claim 6, characterized in that, The augmented reality device presents the interface rendering data, including: According to the presentation order, the text data and the image data are displayed in the display areas corresponding to the text data and the image data, and / or the audio data is played.

8. The system according to claim 7, characterized in that, The augmented reality device displays the text data and the image data in the display areas corresponding to the text data and the image data according to the presentation order, including: In response to the overlap between the actual occupied area of ​​the text data and the actual occupied area of ​​the image data, the text data and the image data are displayed in the order of displaying the image data first and then the text data.

9. An augmented reality device, characterized in that, The augmented reality device is configured to: Acquire initial interaction data and send the initial interaction data to a mobile terminal, so that the mobile terminal sends the initial interaction data to a cloud server, wherein the cloud server deploys at least one AR application, and the AR application is a cloud application running on the AROS system; The interface rendering data sent by the mobile terminal is obtained by the mobile terminal by adjusting the interaction analysis data sent by the cloud server to meet the presentation requirements of the augmented reality device. The interaction analysis data is obtained by the cloud server controlling the target AR application to process the initial interaction data. The target AR application is at least one AR application in the AR application that corresponds to the service type of the initial interaction data. Present the interface rendering data.

10. A cloud server, characterized in that, The cloud server deploys at least one AR application, which is a cloud application running on the AROS system. The cloud server is configured as follows: The device receives initial interaction data sent by a mobile terminal, wherein the initial interaction data is data acquired by the augmented reality device, and the augmented reality device sends the initial interaction data to the mobile terminal. Select an AR application that corresponds to the service type of the initial sensory data from at least one of the AR applications as the target AR application, and control the target AR application to process the initial sensory data to obtain sensory analysis data; The interaction analysis data is sent to the mobile terminal, which adjusts the interaction analysis data into interface rendering data that meets the presentation requirements of the augmented reality device, and sends the interface rendering data to the augmented reality device so that the augmented reality device can display the interface rendering data.

Images