Augmented reality data processing method and system and computer readable storage medium

Abstract

The embodiment of the invention discloses an augmented reality data processing method and system and a computer readable storage medium. According to the specific scheme, the method comprises the steps that a visible light signal emitted by second equipment is acquired; the visible light signal is decoded, target information carried by the visible light signal is obtained, and the target information is related to the target reality scene; sending the target information to a server, so that the server determines a target AR material based on the target information, and returns the target AR material; and superposing the target AR material on the real scene view of the target real scene to obtain an AR view, and displaying the AR view. According to the method and the device, the target information is acquired based on visual light communication, and the AR material matched with the real scene is determined based on the target information, so that the problem of performing augmented reality display on different real scenes can be solved.

Description

Technical Field

[0001] This application relates to the technical fields of visible light communication and augmented reality, and in particular to an augmented reality data processing method, system and computer-readable storage medium. Background Technology

[0002] Augmented Reality (AR) is a technology that overlays virtual information onto real-world scenes. AR technology can provide users with a richer interactive experience.

[0003] Currently, AR technology is being used more and more in various places. These places contain a variety of different real-world scenarios, and how to display these different scenarios in augmented reality has become an important technical problem. Summary of the Invention

[0004] This application provides an augmented reality data processing method, system, and computer-readable storage medium.

[0005] This application provides the following solution:

[0006] According to the first aspect, an augmented reality data processing method is provided, including:

[0007] Acquire the visible light signal emitted by the second device;

[0008] The visible light signal is decoded to obtain the target information carried by the visible light signal. The target information is related to the target's real-world scene.

[0009] The target information is sent to the server so that the server can determine the target AR material based on the target information and return the target AR material.

[0010] An AR view is obtained by overlaying the target AR material onto the real-world view of the target real-world scene, and then the AR view is displayed.

[0011] According to the second aspect, another augmented reality data processing method is provided, including:

[0012] The first device receives target information sent by the first device. The target information is obtained by the first device through decoding the visible light signal emitted by the second device. The target information is related to the target's real-world scene.

[0013] Identify target AR materials based on target information;

[0014] The target AR material is returned to the first device, so that the first device can overlay the target AR material onto the real scene view of the target real scene to obtain an AR view and display the AR view.

[0015] According to the third aspect, another augmented reality data processing system is provided, including a first device, a second device, and a server;

[0016] The second device is used to generate and transmit visible light signals carrying target information;

[0017] The first device is used to acquire the visible light signal emitted by the second device, decode the visible light signal, acquire the target information carried by the visible light signal, and send the target information to the server. The target information is related to the target's real-world scene.

[0018] The server is used to determine the target AR material based on the target information and return the target AR material to the first device;

[0019] The first device is also used to overlay the target AR material onto the real-world view of the target real-world scene to obtain an AR view, and to display the AR view.

[0020] According to a fourth aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any one of the first aspects described above.

[0021] According to the fifth aspect, an electronic device is provided, comprising:

[0022] One or more processors; and

[0023] A memory associated with one or more processors, the memory being used to store program instructions that, when read and executed by one or more processors, perform the steps of the method described in any of the first aspects above.

[0024] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0025] In this application, a visible light signal carrying target information is emitted by a second device, enabling a first device to decode the visible light signal and obtain the target information. This target information is related to the target's real-world scene, allowing the identification of suitable AR materials for the target's real-world scene based on this information, thereby solving the problem of augmented reality display for different real-world scenes.

[0026] In addition, compared with the method of deploying additional signal enhancement equipment in related technologies, the second device in this application can be implemented using the lighting equipment already deployed in the scene, which is an effective use of existing infrastructure, without the need for additional equipment deployment, and greatly reduces deployment costs. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a system architecture diagram applicable to the embodiments of this application;

[0029] Figure 2 This is a flowchart of an augmented reality data processing method provided in an embodiment of this application;

[0030] Figure 3 This is a flowchart of another augmented reality data processing method provided in the embodiments of this application;

[0031] Figure 4 This is a flowchart illustrating a specific implementation of the augmented reality data processing method provided in this application.

[0032] Figure 5 This is a schematic diagram of the structure of an augmented reality data processing device provided in an embodiment of this application;

[0033] Figure 6 This is a schematic diagram of another augmented reality data processing device provided in an embodiment of this application;

[0034] Figure 7 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0035] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0036] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0037] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0038] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0039] Augmented Reality (AR) is a technology that overlays virtual information onto real-world scenes.

[0040] Currently, AR technology is being used more and more in various venues to provide users with a better on-site interactive experience.

[0041] For example, AR technology can be applied to various exhibition venues. Exhibition venues generally have many different real-world scenarios, and how to perform augmented reality displays for these scenarios has become an important technical problem.

[0042] Understandably, when displaying augmented reality (AR) content on a real-world scene, the AR assets used to overlay the scene onto the real-world view need to be compatible with the view to achieve a good augmented reality effect. Therefore, it is necessary to provide compatible AR assets for different real-world scenes, and to overlay these compatible AR assets when the user's device displays the real-world scene view, thereby achieving an augmented reality display of the real-world scene.

[0043] In related technologies, augmented reality (AR) displays of real-world scenes rely on the user's device location information. This location information determines the user's current location, and then AR materials tailored to that scene are pushed to the user's device for AR display. However, real-world scenes are often complex, such as indoor environments like exhibition halls, where user device location information is primarily obtained using Global Positioning System (GPS) signals. Numerous obstructions in indoor environments can significantly attenuate location signals (such as GPS signals), and in crowded indoor environments, multiple location signals can interfere with each other, making it impossible to accurately obtain the user's location information and effectively display the real-world scene in an AR manner.

[0044] To address the issue of ineffective positioning of user equipment in indoor environments, many related technologies require the deployment of dedicated positioning signal enhancement equipment (such as signal amplifiers or additional positioning base stations) in indoor environments, which incurs additional deployment costs.

[0045] In view of this, this application provides a new approach. To facilitate understanding of this application, the system architecture on which this application is based is first described. For example... Figure 1 The diagram illustrates an augmented reality data processing system applicable to embodiments of this application. The augmented reality data processing system includes a server, a first device, and a second device.

[0046] The first device is used to display the target real-world scene in augmented reality. The first device is an augmented reality-enabled device, which may include, but is not limited to, mobile phones, tablets, smart wearable devices, and smart car terminals. The smart wearable device may be a smartwatch, smart glasses, smart bracelet, etc. In this embodiment, the first device typically has an information display function, such as a display screen. In addition, it may also have an operation command input function, such as inputting operation commands via touchscreen, keyboard, or voice.

[0047] The second device is a visible light signal emitting device, which may include, but is not limited to, light-emitting diode (LED) lighting.

[0048] The server can be a dedicated server, a server cluster, or a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a hosting product within the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) services, such as high management difficulty and weak service scalability.

[0049] The server is mainly used to identify the target AR material and send the target AR material to the first device, so that the first device can display the target real-world scene based on the target AR material.

[0050] The first device and the server can interact via a network, which can be wired or wireless. The first device and the second device can interact via Visible Light Communication (VLC).

[0051] VLC is a communication technology that uses light waves as a medium to transmit data. This technology utilizes the characteristics of visible light to transmit information. It achieves communication by modulating the brightness, frequency, or phase of light, hiding data information in the optical signal, and then transmitting the data through the transmission of visible light signals.

[0052] It should be understood that Figure 1 The number of servers, first devices, and second devices shown is merely illustrative. Depending on implementation needs, there can be any number of servers, first devices, and second devices.

[0053] Figure 2 This is a flowchart of an augmented reality data processing method provided in an embodiment of this application. The method can be... Figure 1 The first device in the system shown performs the operation. For example... Figure 2 As shown, the method may include the following steps:

[0054] Step S210: Acquire the visible light signal emitted by the second device;

[0055] Step S220: Decode the visible light signal to obtain the target information carried by the visible light signal. The target information is related to the target's real-world scene.

[0056] Step S230: Send the target information to the server so that the server can determine the target AR material based on the target information and return the target AR material;

[0057] Step S240: Overlay the target AR material onto the real-world view of the target real-world scene to obtain an AR view, and display the AR view.

[0058] When displaying augmented reality (AR) content in different real-world scenarios, the challenge lies in providing suitable AR materials. As the above process demonstrates, this application uses a second device to emit a visible light signal carrying target information, enabling the first device to decode the signal and obtain the target information. This target information is related to the target real-world scene, allowing the determination of suitable AR materials based on this information, thus solving the problem of displaying AR content in different real-world scenarios.

[0059] In addition, compared with the method of deploying additional signal enhancement equipment in related technologies, the second device in this application can be implemented using the lighting equipment already deployed in the scene, which is an effective use of existing infrastructure, without the need for additional equipment deployment, and greatly reduces deployment costs.

[0060] The following describes in detail each step of the above process and the effects that can be further produced, with reference to the embodiments.

[0061] First, the above step S210, namely "acquiring the visible light signal emitted by the second device", will be described in detail with reference to the embodiments.

[0062] The second device may include a visible light signal transmitting module, which is used to encode and transmit visible light signals.

[0063] The first device has a visible light signal acquisition module, which acquires the visible light signal emitted by the second device.

[0064] In one possible approach, in this step, the first device can respond to receiving interactive prompt information, generate and display an interactive prompt interface based on the interactive prompt information, the interactive prompt interface including a visible light signal acquisition control, the interactive prompt information being triggered and sent in response to the first device being located within the associated area of ​​the target real scene; in response to detecting the user's trigger operation on the visible light signal acquisition control, the visible light signal acquisition module of the first device is invoked to acquire visible light signals.

[0065] The associated area of ​​the target real scene can be understood as the area within a certain range around the target real scene. When the user is in this area, the visible light signal acquisition module of the first device can be used to acquire visible light signals and can also acquire the real scene view of the target real scene, thereby supporting the subsequent augmented reality display of the target real scene.

[0066] In this embodiment, when a user moves the first device to an associated area of ​​the target real-world scene, the sending of interactive prompt information can be automatically triggered. After the interactive prompt information is sent to the first device, the first device can generate and display an interactive prompt interface based on the interactive prompt information. This interactive prompt interface is used to guide the user to interact based on augmented reality technology. Specifically, the interactive prompt interface includes a visible light signal acquisition control. After the interactive prompt interface guides the user to trigger the visible light signal acquisition control, the first device will acquire visible light signals and perform augmented reality display based on the visible light signals, thereby realizing interaction with the user.

[0067] As one possible approach, the associated area is a geofenced area created based on the geofence service, and the interactive prompt information is sent to the first device by the geofence system in response to detecting that the first device is located within the geofenced area of ​​the target real scene.

[0068] Specifically, a geofencing area of ​​the target real-world scene can be defined based on the geofencing service, and this area can be used as the aforementioned associated region. When the first device approaches the geofencing area of ​​the target real-world scene, the geofencing system automatically sends an interactive prompt message to the first device.

[0069] The following describes in detail step S220, namely "decoding the visible light signal to obtain the target information carried by the visible light signal, the target information being related to the target's real-world scene," with reference to the embodiments.

[0070] Visible light signals are essentially obtained by encoding target information into visible light using the visible light signal transmitting module of a second device, and then carrying the encoded information into visible light. This is achieved using VLC (Visible Light Communication) technology.

[0071] After acquiring the visible light signal, the first device can decode the visible light signal based on a preset visible light signal decoding algorithm to obtain the target information.

[0072] The target real-world scene is the scene where the user of the first device is currently located, which is also the real-world scene where augmented reality display needs to be performed. The target information is related to the target real-world scene, enabling the target information to represent some relevant information of the target real-world scene, and allowing the target AR materials that are compatible with the target real-world scene to be determined based on the target information.

[0073] As an example, target information may include, but is not limited to, the identification information of the second device, the location information of the target real-world scene, and the scene identifier of the target real-world scene.

[0074] The second device can be a device associated with the target real-world scene, enabling the identification information of the second device to determine the target AR material that is compatible with the target real-world scene.

[0075] As an example, the second device can be an LED light source near the target real-world scene, capable of illuminating the target real-world scene and also emitting visible light signals to the vicinity of the target real-world scene.

[0076] The following describes in detail step S230, namely "sending the target information to the server so that the server can determine the target AR material based on the target information and return the target AR material", with reference to the embodiments.

[0077] After the first device decodes the target information, it can send the target information to the server, and the server determines the target AR material based on the target information.

[0078] Specifically, when the target information includes the identification information of the second device, since the second device is associated with the target real-world scene, the target real-world scene that needs to be displayed in augmented reality can be determined based on the identification information of the second device, thereby determining the target AR material that is compatible with the target real-world scene.

[0079] When the target information includes the location information of the target real scene, the location information can represent the location of the target real scene, so that the target real scene that needs to be displayed in augmented reality can be determined based on the location information, and thus the target AR material that is compatible with the target real scene can be determined.

[0080] When the target information includes a scene identifier of the target real-world scene, the scene identifier can serve as a unique identifier for the target real-world scene, enabling the determination of the target real-world scene that needs to be displayed in augmented reality based on the scene identifier, thereby determining the target AR material that is compatible with the target real-world scene.

[0081] The target AR material can include, but is not limited to, 3D models, animations, videos, etc. For example, in games such as grabbing red envelopes, the target AR material is an animation of red envelopes falling.

[0082] The server can pre-configure an AR material library and determine the target AR material from the library based on the target information.

[0083] The following describes in detail step S240, namely "overlaying the target AR material onto the real scene view of the target real scene to obtain an AR view and displaying the AR view", with reference to the embodiments.

[0084] In this embodiment of the application, after receiving the target AR material returned by the server, the first device can overlay the target AR material onto the real scene view of the target real scene to obtain an AR view and display the AR view, thereby realizing the augmented reality display of the target real scene.

[0085] AR view refers to a technical view that combines virtual information with the real world. In the embodiments of this application, it is a view obtained by overlaying real-world scenes with 3D models, animations, videos, etc.

[0086] As one possible approach, after displaying the AR view, the above method also includes:

[0087] In response to detecting user actions based on the AR view on the first device, the action data is sent to the server.

[0088] Users can perform actions based on the AR view. The first device can send the action data to the server, providing a basis for the server to perform subsequent processing based on the action data.

[0089] As an example, the AR view is an AR view that overlays a falling red envelope animation onto a real-world image. Users can grab red envelopes by clicking on the falling red envelopes on the screen. After the user completes the grabbing action, the action data can be sent to the server.

[0090] Figure 3 This is a flowchart of an augmented reality data processing method provided in an embodiment of this application. The method can be... Figure 1 The server in the system shown executes this. For example... Figure 3 As shown, the method may include the following steps:

[0091] Step S310: Receive target information sent by the first device. The target information is obtained by the first device by decoding the visible light signal emitted by the second device. The target information is related to the target real scene.

[0092] Step S320: Determine the target AR material based on the target information.

[0093] Step S330: Return the target AR material to the first device so that the first device can overlay the target AR material onto the real scene view of the target real scene to obtain an AR view and display the AR view.

[0094] When displaying augmented reality (AR) content in different real-world scenarios, the challenge lies in providing suitable AR materials. As the above process demonstrates, this application uses a second device to emit a visible light signal carrying target information, enabling the first device to decode the signal and obtain the target information. This target information is related to the target's real-world scene, allowing the determination of suitable AR materials to address this issue and solve the problem of displaying AR content in different real-world scenarios.

[0095] In addition, compared with the method of deploying additional signal enhancement equipment in related technologies, the second device in this application can be implemented using the lighting equipment already deployed in the scene, which is an effective use of existing infrastructure, without the need for additional equipment deployment, and greatly reduces deployment costs.

[0096] The following describes in detail each step of the above process and the effects that can be further produced, with reference to the embodiments.

[0097] First, the above step S310, namely "receiving target information sent by the first device, the target information being obtained by the first device through decoding the visible light signal emitted by the second device, and the target information being related to the target real scene", will be described in detail with reference to the embodiments.

[0098] The second device may include a visible light signal transmitting module, which is used to encode and transmit visible light signals. The first device has a visible light signal acquisition module, which acquires the visible light signals transmitted by the second device.

[0099] Visible light signals are essentially obtained by encoding target information into visible light using the visible light signal transmitting module of a second device, and then carrying the encoded information into visible light. This is achieved using VLC (Visible Light Communication) technology.

[0100] After acquiring the visible light signal, the first device can decode the visible light signal based on a preset visible light signal decoding algorithm to obtain the target information.

[0101] The target real-world scene is the scene where the user of the first device is currently located, which is also the real-world scene where augmented reality display needs to be performed. The target information is related to the target real-world scene, enabling the target information to represent some relevant information of the target real-world scene, and allowing the target AR materials that are compatible with the target real-world scene to be determined based on the target information.

[0102] As an example, target information may include, but is not limited to, the identification information of the second device, the location information of the target real-world scene, and the scene identifier of the target real-world scene.

[0103] The second device can be a device associated with the target real-world scene, enabling the identification information of the second device to determine the target AR material that is compatible with the target real-world scene.

[0104] As an example, the second device can be an LED light source near the target real-world scene, capable of illuminating the target real-world scene and also emitting visible light signals to the vicinity of the target real-world scene.

[0105] The following describes step S320, "Determining target AR material based on target information," in detail with reference to an embodiment.

[0106] After the first device decodes the target information, it can send the target information to the server, and the server determines the target AR material based on the target information.

[0107] Specifically, when the target information includes the identification information of the second device, since the second device is associated with the target real-world scene, the target real-world scene that needs to be displayed in augmented reality can be determined based on the identification information of the second device, thereby determining the target AR material that is compatible with the target real-world scene.

[0108] When the target information includes the location information of the target real scene, the location information can represent the location of the target real scene, so that the target real scene that needs to be displayed in augmented reality can be determined based on the location information, and thus the target AR material that is compatible with the target real scene can be determined.

[0109] When the target information includes a scene identifier of the target real-world scene, the scene identifier can serve as a unique identifier for the target real-world scene, enabling the determination of the target real-world scene that needs to be displayed in augmented reality based on the scene identifier, thereby determining the target AR material that is compatible with the target real-world scene.

[0110] The target AR material can include, but is not limited to, 3D models, animations, videos, etc. For example, in games such as grabbing red envelopes, the target AR material is an animation of red envelopes falling.

[0111] The server can pre-configure an AR material library and determine the target AR material from the library based on the target information.

[0112] As a feasible approach, when determining target AR materials based on target information, the target AR material corresponding to the target information can be determined based on a first preset correspondence between the target information and the target AR material. Specifically, a first preset correspondence can be established between the target information and the target AR material to quickly determine the target AR material corresponding to the target information.

[0113] As another feasible approach, when determining target AR materials based on target information, the candidate AR materials corresponding to the target information can be determined first based on the second preset correspondence between the target information and the candidate AR materials; then, the target AR materials can be selected from the candidate AR materials based on the user-related information of the user of the first device.

[0114] A second preset correspondence can be established between target information and candidate AR materials, so that the candidate AR materials corresponding to the target information can be quickly determined based on the second preset correspondence.

[0115] User-related information can characterize some of the user's personalized features. By filtering candidate AR materials based on user-related information, the selected target AR materials can not only be adapted to the target real-world scene, but also better match the user's personalized features.

[0116] The aforementioned user-related information includes, but is not limited to, the user's user identifier, the user's historical operational behavior, and the user's relevant attribute information.

[0117] Among these, historical operation behaviors can be the user's actions based on AR views that have been displayed on the first device. For example, if the AR view is a real-world view overlaid with a falling red envelope animation, the historical operation behavior is to tap the falling red envelopes on the screen to grab them.

[0118] When selecting target AR materials from candidate AR materials based on historical user behavior, the following two methods can be used, but are not limited to:

[0119] The first method is to determine the type of target AR material based on the user's historical operation behavior, and then select the AR material of the determined type from the candidate AR materials as the target AR material.

[0120] In this implementation, historical operation behavior can reflect some of the user's operation preferences. Based on the user's historical operation behavior, the type of AR material preferred by the user can be determined, and then AR material of the determined type can be selected from the candidate AR material as the target AR material.

[0121] For example, a user's historical action might be grabbing red envelopes (digital cash gifts). Based on this historical action, it can be determined that the user is highly likely to grab red envelopes again the next time they interact with the AR view. Therefore, candidate AR materials related to grabbing red envelopes can be displayed to the user as target AR materials.

[0122] Besides being based on user preferences, this implementation method also allows for pre-planning of AR implementation paths on the server side for real-world scenarios. For example, from a user entering an exhibition hall to exiting, a path of "Welcome - Hall Introduction - Activity 1 - Activity 2 - Red Envelope Giveaway" can be designed. Upon receiving the first target message from the user, the target AR material is determined to be an animation containing a welcome message; upon receiving the second target message, the target AR material is determined to be a hall introduction video; upon receiving the third target message, the target AR material is determined to be Activity Animation 1; upon receiving the fourth target message, the target AR material is determined to be Activity Animation 2; and upon receiving the fifth target message, the target AR material is determined to be a red envelope giveaway.

[0123] The second method is to determine the AR materials that the user has used based on the user's historical operation behavior, and then select the target AR material from the candidate AR materials other than the AR materials that have been used.

[0124] Historical user behavior can also reflect how users use AR assets. For AR assets that have already been used by users, it's crucial to avoid repeatedly displaying them, as this could negatively impact the user experience. Therefore, used AR assets can be excluded from the candidate AR assets, and the remaining candidate AR assets can be selected as the target AR assets. This avoids repeatedly displaying used AR assets and ensures a positive user experience.

[0125] As an alternative approach, certain exhibition venues, such as museums, may pre-set AR content display groups, which need to contain several types of AR content. When displaying AR content to users, it is necessary to ensure that users can see as many types of AR content as possible from the display group. This allows for the selection of target AR content by identifying AR content types that have already been shown to users based on their historical user behavior, determining AR content types that have not yet been shown in the display group as target AR content types, and then selecting AR content of the determined types from the candidate AR content as the target AR content.

[0126] The following describes in detail step S330, "The target AR material is returned to the first device so that the first device can overlay the target AR material onto the real scene view of the target real scene to obtain an AR view and display the AR view," with reference to the embodiments.

[0127] In this embodiment of the application, after receiving the target AR material returned by the server, the first device can overlay the target AR material onto the real scene view of the target real scene to obtain an AR view and display the AR view, thereby realizing the augmented reality display of the target real scene.

[0128] As one possible approach, after the first device displays the AR view, the method may further include: the server responding to receiving behavioral data of a preset operation behavior, determining a real-world scene to be recommended associated with the target real-world scene, wherein the behavioral data is sent by the first device in response to detecting that the user has performed a preset operation behavior based on the AR view; generating guidance information and sending the guidance information to the first device, wherein the guidance information is used to guide the user to move to the associated area of ​​the real-world scene to be recommended.

[0129] After the user performs an action based on the AR view, the first device will send the action data to the server.

[0130] There are certain service connections between different real-world scenarios, and relationships can be established between these scenarios. When a user performs a preset action in a target real-world scenario, the user can be recommended to move to an associated, yet-to-be-recommended real-world scenario. This latter scenario will then be augmented reality-enabled, allowing the user to interact with the other user within the two related scenarios, thus enhancing the user experience.

[0131] As an example, the AR view of the target real-world scene is a live view overlaid with a falling red envelope animation. The user's preset action is to tap the falling red envelope on the screen and successfully grab it. At this point, the server can send guidance information to the first device. For example, the guidance information could be a text message like "Move to area X to continue grabbing red envelopes," or a path guidance message towards area X. This area X is the associated area of ​​the real-world scene to be recommended. By interacting with the recommended real-world scene within this associated area and continuing the red envelope grabbing operation, the user's interaction with different real-world scenes can be deepened, enhancing the user's interactive experience.

[0132] The methods provided in this application embodiment can be applied to a variety of application scenarios, including but not limited to: augmented reality display for different real-world scenarios in the indoor environments of various performance and exhibition venues.

[0133] As an example, Figure 4 This is a flowchart illustrating an optional implementation of the augmented reality data processing method provided in this application.

[0134] The process includes the following steps:

[0135] Step S410: The VLC emitting module of the LED emits a VLC light signal.

[0136] The LED lighting corresponds to the first device described above in the embodiments of this application. In this example, the VLC transmitting module is integrated into the LED lighting and is used to encode and transmit VLC signals carrying target information.

[0137] Step S420: The mobile phone identifies and decodes the VLC signal through the camera.

[0138] The mobile phone corresponds to the first device described in this application embodiment. The user's mobile phone may include a visible light signal acquisition module for acquiring VLC signals emitted by the LED lighting lamp. The visible light signal acquisition module in the mobile phone may be a camera. The user's mobile phone may have a built-in VLC decoding algorithm for decoding the VLC signal. Target information is obtained from the VLC signal, and this target information may be the identification information of the LED lighting lamp.

[0139] Step S430: The mobile phone sends the identification information obtained from decoding the VLC signal to the server.

[0140] Step S440: The server retrieves AR materials from the AR material library and sends the AR materials to the mobile phone.

[0141] The server can have a pre-built AR material library. The AR material library stores AR materials and can be indexed by identification information such as LEDs, making it easy to detect the corresponding target AR materials.

[0142] Step S450: Display the AR material on the mobile phone.

[0143] The user's phone may also include an AR rendering engine, which overlays the target AR material with the real-world scene view to create an AR view and display it, so that the target AR material can blend with the real environment.

[0144] Step S460: Collect user interaction data and submit it to the server.

[0145] When users interact with the AR-based view, their mobile phones can collect behavioral data about the user's actions and report it to the server. The server also includes an interaction module, which receives and analyzes the user interaction data reported by the user's mobile phone.

[0146] The steps of the augmented reality data processing method in this application can be executed through applications (APPs), plugins within applications, mini-programs, etc., installed on a mobile phone.

[0147] This technical solution aims to enhance the interactivity of the user's on-site experience by combining a smartphone app with VLC communication and AR technologies. In this solution, the venue's LED lighting system can send VLC-encoded signals. The smartphone app identifies and decodes these light signals through its camera, subsequently triggering AR materials corresponding to the target information, such as 3D models, animations, and videos, thereby enhancing the user's on-site interactive experience.

[0148] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0149] In summary, the augmented reality data processing system provided in this application includes a first device, a second device, and a server.

[0150] The second device is used to generate and transmit a visible light signal carrying target information.

[0151] The first device is used to acquire the visible light signal emitted by the second device, decode the visible light signal, acquire the target information carried by the visible light signal, and send the target information to the server. The target information is related to the target's real-world scene.

[0152] The server is used to determine the target AR material based on the target information and return the target AR material to the first device.

[0153] The first device is also used to overlay the target AR material onto the real-world view of the target real-world scene to obtain an AR view, and to display the AR view.

[0154] In addition to the effects described above, the technical solutions provided by the embodiments of this application can further achieve the following technical effects:

[0155] 1) In this application, the user can interact based on the AR view, and the first device sends the generated operation behavior to the server. The server can determine the target AR material to be sent to the first device based on the user's historical operation behavior, thereby achieving accurate matching of the target AR material.

[0156] 2) In this application, the first device can receive interactive prompt information when it is located in the associated area of ​​the target real scene (e.g., a geofence area), and generate and display an interactive prompt interface based on the interactive prompt information to prompt the user to trigger the visible light signal acquisition space to acquire visible light signals. This method is more convenient for users and improves the user experience.

[0157] 3) The server can determine the type of target AR material based on the user's historical operation behavior, and select the AR material of the determined type as the target AR material, thereby realizing a strong correlation between the target AR material and the user's preferences or historical operations, so that the pushed target AR material is more in line with the user's needs or the service requirements.

[0158] 4) The server can determine the AR materials that the user has used based on the user's historical operation behavior, and avoid sending the AR materials that the user has used repeatedly when determining the target AR materials to be sent to the first device, thereby further improving the user experience.

[0159] 5) In this application, based on the interaction between the user and the AR view, the server can determine the real scene to be recommended that is associated with the target real scene based on the user's operation behavior, and generate guidance information to guide the user to move to the associated area of ​​the real scene to be recommended. This way of guiding the user can help the user quickly determine the positional relationship between multiple target real scenes, save the user's time and improve the user experience.

[0160] Of course, any product implementing this application does not necessarily need to achieve all of the above advantages at the same time.

[0161] According to another embodiment, an augmented reality data processing apparatus is provided. Figure 5 A schematic block diagram of an augmented reality data processing apparatus according to one embodiment is shown, the apparatus being disposed in Figure 1 The first device in the illustrated architecture. (As shown) Figure 5 As shown, the augmented reality data processing device 500 includes: a visible light signal acquisition module 510, a visible light signal decoding module 520, an AR material acquisition module 530, and an AR display module 540. The main functions of each component module are as follows:

[0162] The visible light signal acquisition module 510 is used to acquire the visible light signal emitted by the second device.

[0163] The visible light signal decoding module 520 is used to decode the visible light signal and obtain the target information carried by the visible light signal. The target information is related to the target's real-world scene.

[0164] AR material acquisition module 530 is used to send target information to the server so that the server can determine the target AR material based on the target information and return the target AR material.

[0165] AR display module 540 is used to overlay the target AR material onto the real scene view of the target real scene to obtain an AR view, and to display the AR view.

[0166] As one possible implementation, the above-mentioned device further includes:

[0167] The behavior data reporting module (not shown in the figure) is used to send the behavior data of the operation behavior to the server after the AR view is displayed in response to the detection of the operation behavior of the user of the first device based on the AR view.

[0168] As one possible approach, the visible light signal acquisition module 510 is specifically used for:

[0169] In response to receiving an interactive prompt message, an interactive prompt interface is generated and displayed based on the interactive prompt message. The interactive prompt interface includes a visible light signal acquisition control. The interactive prompt message is triggered and sent in response to the first device being located within the associated area of ​​the target real scene.

[0170] In response to the detection of a user's trigger operation on the visible light signal acquisition control, the visible light signal acquisition module of the first device is invoked to acquire the visible light signal.

[0171] As one possible approach, the associated area is a geofenced area created based on the geofence service, and the interactive prompt information is sent to the first device by the geofence system in response to detecting that the first device is located within the geofenced area of ​​the target real scene.

[0172] According to another embodiment, an augmented reality data processing apparatus is provided. Figure 6A schematic block diagram of an augmented reality data processing apparatus according to one embodiment is shown, the apparatus being disposed in Figure 1 The server-side in the architecture shown. For example... Figure 6 As shown, the augmented reality data processing device 600 includes: a target information receiving module 610, an AR material determination module 620, and an AR material return module 630. The main functions of each component module are as follows:

[0173] The target information receiving module 610 is used to receive target information sent by the first device. The target information is obtained by the first device by decoding the visible light signal emitted by the second device. The target information is related to the target's real-world scene.

[0174] AR material determination module 620 is used to determine target AR materials based on target information.

[0175] AR material return module 630 is used to return the target AR material to the first device, so that the first device can overlay the target AR material onto the real scene view of the target real scene to obtain an AR view and display the AR view.

[0176] As one possible approach, the AR material determination module 620 is specifically used for:

[0177] Based on a first preset correspondence between target information and target AR material, determine the target AR material corresponding to the target information; or...

[0178] Based on the second preset correspondence between target information and candidate AR materials, candidate AR materials corresponding to the target information are determined; based on the user-related information of the user of the first device, target AR materials are selected from the candidate AR materials.

[0179] As one possible approach, when the AR material determination module 620 filters target AR materials from candidate AR materials based on user-related information of the user of the first device, it is specifically used for at least one of the following:

[0180] Based on the user's historical user behavior, determine the type of the target AR material, and select the AR material of the determined type from the candidate AR materials as the target AR material; or...

[0181] Based on the user's historical operation behavior, determine the AR materials that the user has used, and then determine the target AR material from the candidate AR materials other than the AR materials that have been used.

[0182] As one possible implementation, the above-described device further includes a guiding module (not shown in the figure), which is used for:

[0183] After the first device displays the AR view, in response to receiving behavioral data of preset operation behavior, it determines the recommended real scene associated with the target real scene. The behavioral data is sent by the first device in response to detecting that the user has performed a preset operation behavior based on the AR view.

[0184] Generate guidance information and send it to the first device. The guidance information is used to guide the user to move to the associated area of ​​the real-world scene to be recommended.

[0185] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0186] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0187] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the methods in the foregoing method embodiments.

[0188] And an electronic device, comprising:

[0189] One or more processors; and

[0190] A memory associated with one or more processors, the memory being used to store program instructions that, when read and executed by one or more processors, perform the steps of any of the methods in the foregoing method embodiments.

[0191] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the methods in the foregoing method embodiments.

[0192] in, Figure 7 An exemplary architecture of an electronic device is shown, which may include a processor 710, a video display adapter 711, a disk drive 712, an input / output interface 713, a network interface 714, and a memory 720. The processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, and memory 720 can communicate with each other via a communication bus 730.

[0193] The processor 710 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0194] The memory 720 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 720 can store the operating system 721 for controlling the operation of the electronic device 700, and the basic input / output system (BIOS) 722 for controlling the low-level operations of the electronic device 700. Additionally, it can store a web browser 723, a data storage management system 724, and 725, etc. The aforementioned augmented reality data processing system 725 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when the technical solution provided in this application is implemented through software or firmware, the relevant program code is stored in the memory 720 and is called and executed by the processor 710.

[0195] Input / output interface 713 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0196] Network interface 714 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0197] Bus 730 includes a pathway for transmitting information between various components of the device, such as processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, and memory 720.

[0198] It should be noted that although the above-described device only shows the processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, memory 720, bus 730, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0199] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0200] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An augmented reality data processing method, characterized in that, include: Acquire the visible light signal emitted by the second device; The visible light signal is decoded to obtain the target information carried by the visible light signal, and the target information is related to the target's real-world scene; The target information is sent to the server so that the server can determine the target augmented reality (AR) material based on the target information and return the target AR material. An AR view is obtained by overlaying the target AR material onto the real-world view of the target real-world scene, and then the AR view is displayed.

2. The method according to claim 1, characterized in that, After displaying the AR view, the method further includes: In response to detecting user actions based on the AR view, the behavioral data of the actions is sent to the server.

3. The method according to claim 1 or 2, characterized in that, The acquisition of the visible light signal emitted by the second device includes: In response to receiving interactive prompt information, an interactive prompt interface is generated and displayed based on the interactive prompt information. The interactive prompt interface includes a visible light signal acquisition control. The interactive prompt information is triggered and sent in response to the first device being located within the associated area of ​​the target real scene. In response to detecting a user's trigger operation on the visible light signal acquisition control, the visible light signal acquisition module of the first device is invoked to acquire the visible light signal.

4. The method according to claim 3, characterized in that, The associated area is a geofenced area created based on the geofence service, and the interactive prompt information is sent to the first device by the geofence system in response to detecting that the first device is located within the geofenced area of ​​the target real scene.

5. An augmented reality data processing method, characterized in that, include: The device receives target information sent by a first device, which is obtained by the first device through decoding a visible light signal emitted by a second device, and the target information is related to the target's real-world scene. The target AR material is determined based on the target information; The target AR material is returned to the first device, so that the first device overlays the target AR material onto the real scene view of the target real scene to obtain an AR view, and displays the AR view.

6. The method according to claim 5, characterized in that, The step of determining the target AR material based on the target information includes: Based on a first preset correspondence between the target information and the target AR material, the target AR material corresponding to the target information is determined; or... Based on the second preset correspondence between the target information and the candidate AR material, the candidate AR material corresponding to the target information is determined; based on the user-related information of the user of the first device, the target AR material is selected from the candidate AR material.

7. The method according to claim 6, characterized in that, Based on user-related information of the user of the first device, the target AR material is selected from the candidate AR materials, including at least one of the following: Based on the user's historical user behavior, determine the type of the target AR material, and select the AR material of the determined type from the candidate AR materials as the target AR material; or... Based on the user's historical operation behavior, the AR materials that the user has used are determined, and the target AR material is determined from the candidate AR materials other than the AR materials that have been used.

8. The method according to any one of claims 5-7, characterized in that, After the first device displays the AR view, the method further includes: In response to receiving behavioral data of a preset operation behavior, a recommended real-world scene associated with the target real-world scene is determined. The behavioral data is sent by the first device in response to detecting that the user has performed a preset operation behavior based on the AR view. Generate guidance information and send the guidance information to the first device. The guidance information is used to guide the user to move to the associated area of ​​the real-world scene to be recommended.

9. An augmented reality data processing system, characterized in that, Includes the first device, the second device, and the server; The second device is used to generate and transmit a visible light signal carrying target information; The first device is used to acquire the visible light signal emitted by the second device, decode the visible light signal, acquire the target information carried by the visible light signal, and send the target information to the server. The target information is related to the target real-world scene. The server is used to determine the target AR material based on the target information and return the target AR material to the first device; The first device is further configured to overlay the target AR material onto a real-world view of the target real-world scene to obtain an AR view, and to display the AR view.

10. The system according to claim 9, characterized in that, When the server determines the target AR material based on the target information, it is specifically used for: Based on a first preset correspondence between the target information and the target AR material, the target AR material corresponding to the target information is determined; or... Based on the second preset correspondence between the target information and the candidate AR material, the candidate AR material corresponding to the target information is determined; Based on the user-related information of the user of the first device, the target AR material is selected from the candidate AR materials.

11. The system according to claim 9 or 10, characterized in that, When the first device acquires the visible light signal emitted by the second device, it is specifically used for: In response to receiving interactive prompt information, an interactive prompt interface is generated and displayed based on the interactive prompt information. The interactive prompt interface includes a visible light signal acquisition control. The interactive prompt information is triggered and sent in response to the first device being located within the associated area of ​​the target real scene. In response to detecting a user's trigger operation on the visible light signal acquisition control, the visible light signal acquisition module of the first device is invoked to acquire the visible light signal.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1-8.

13. An electronic device, characterized in that, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1-8.

14. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-8.

Images