Multi-terminal AR content synchronization triggering method, system, electronic device and storage medium

Abstract

The present application relates to a multi-terminal AR content synchronization triggering method, wherein the method includes: multiple experience terminals, collecting real-time images of the same preset object at different positions of the target scene, generating real-time positioning requests based on the real-time images respectively, and sending the real-time positioning requests to the local visual positioning terminal; the local visual positioning terminal generates corresponding real-time scene poses according to each real-time positioning request, and returns the real-time scene poses to each experience terminal respectively; sending a synchronization trigger instruction to each experience terminal, instructing each experience terminal to synchronously display AR special effects content that interacts with the preset object at different angles according to its own real-time scene pose. Through this application, the problem of poor flexibility of the multi-terminal AR content synchronization triggering technology is solved. User terminals at different positions can display AR special effects content that interacts with the actual scene from different angles, which has better flexibility and improves the user's overall experience.

Description

Technical Field

[0001] The present application relates to the field of visual positioning, and in particular to a multi-terminal AR content synchronization triggering method, system, electronic device and computer-readable storage medium. Background Art

[0002] Visual positioning navigation service is a user navigation service experience terminal service based on a specified scene location and bound to the geographic location. With the rapid development of the Internet of Things, visual positioning navigation service is increasingly entering people's daily lives.

[0003] In related technologies, the solutions for visual positioning services and content interactive display are as follows:

[0004] 1. Based on a single device, each device performs visual positioning services and triggers interactive content display. Due to the lack of a synchronization controller, this type of technical solution cannot achieve the effect of multi-terminal synchronous triggering.

[0005] 2. Multiple terminals only handle the synchronous display of content and do not involve visual positioning services. Due to the lack of visual positioning services, this type of technical solution is not flexible in the timing and location of content display, resulting in a poor sense of augmented reality.

[0006] Currently, no effective solution has been proposed to address the problem of poor flexibility in multi-terminal AR content synchronization triggering technology. Summary of the Invention

[0007] The embodiments of the present application provide a multi-terminal AR content synchronization triggering method, system, electronic device, and computer-readable storage medium to at least solve the problem of poor flexibility of the multi-terminal AR content synchronization triggering technology in the related art.

[0008] In a first aspect, an embodiment of the present application provides a method for synchronously triggering AR content on multiple terminals, the method comprising:

[0009] Multiple experience terminals collect real-time images of the same preset object at different locations in the target scene.

[0010] Generate a real-time positioning request based on the real-time image respectively, and send the real-time positioning request to a local visual positioning terminal, wherein the local visual positioning terminal is deployed in the target scene, and the local visual positioning terminal integrates an intranet penetration service and a visual positioning service;

[0011] The local visual positioning terminal generates a corresponding real-time scene pose according to each real-time positioning request, and returns the real-time scene pose to each experience terminal respectively;

[0012] The synchronization control server sends a synchronization trigger instruction to each experience terminal, instructing each experience terminal to synchronously display AR special effect content that interacts with the preset object at different angles according to its own real-time scene posture.

[0013] In some embodiments, when the experience terminal communicates with the local visual positioning terminal via a local area network:

[0014] The experience terminal finds the local visual positioning terminal through the local area network discovery protocol, and sends the real-time positioning request to the local visual positioning terminal through the local area network connection channel;

[0015] After generating the real-time scene posture, the local visual positioning terminal returns the real-time scene posture to each experience terminal through the local area network connection channel.

[0016] In some embodiments, when the experience terminal accesses the local visual positioning terminal through the public network:

[0017] The experience terminal sends the real-time positioning request to the cloud host cluster through the public network channel;

[0018] The cloud host cluster establishes a connection channel with the intranet penetration module of the local visual positioning terminal through the intranet penetration server, and forwards the real-time positioning request to the local visual positioning terminal through the connection channel;

[0019] After generating the real-time scene posture, the local visual positioning terminal returns the real-time scene posture to each experience terminal through the connection channel.

[0020] In some embodiments, after the experience terminal sends the real-time positioning request to the cloud host cluster through a public network channel, the method further includes:

[0021] The cloud host cluster allocates the optimal node to the positioning request through a load balancing module;

[0022] The cloud host cluster queries the local visual positioning terminal corresponding to the optimal node through the service registration center and obtains its link address;

[0023] The cloud host cluster instructs the intranet penetration server to establish the connection channel with the intranet penetration module in the local visual positioning terminal according to the link address.

[0024] In some embodiments, the synchronization control server is deployed in the target scenario or in a cloud host cluster;

[0025] The synchronization control server generates the synchronization trigger instruction after receiving the signal that all experience terminals have completed visual positioning.

[0026] In a second aspect, an embodiment of the present application provides another multi-terminal AR content synchronization triggering method, the method comprising:

[0027] Multiple experience terminals collect real-time images of the same preset object at different locations in the target scene.

[0028] Generating real-time positioning requests based on the real-time images respectively, and sending the real-time positioning requests to a visual positioning service through a public network channel, wherein the visual positioning service is deployed in a cloud host cluster;

[0029] The positioning service generates corresponding real-time scene poses according to each real-time positioning request, and returns the real-time scene poses to each experience terminal through the public network channel;

[0030] Through the synchronization control server, a synchronization trigger instruction is sent to each experience terminal to instruct each experience terminal to synchronously display the AR special effect content that interacts with the preset object at different angles according to its own real-time scene posture.

[0031] In some embodiments, multiple experience terminals send the real-time positioning request to the positioning service through a public network channel, including:

[0032] The cloud host cluster allocates the optimal node to the positioning request through a load balancing module;

[0033] The cloud host cluster queries the link address corresponding to the optimal node through the service registration center, and forwards the real-time positioning request to the visual positioning service corresponding to the link address.

[0034] In a third aspect, an embodiment of the present application provides a multi-terminal AR content synchronization triggering system, the system comprising: an experience terminal, a local visual positioning terminal, and a synchronization control server, wherein;

[0035] The experience terminal is used to collect real-time images of the same preset object at different locations in the target scene, generate real-time positioning requests based on the real-time images, and send the real-time positioning requests to the local visual positioning terminal, wherein the local visual positioning terminal is deployed in the target scene and integrates an intranet penetration service and a visual positioning service;

[0036] The local visual positioning terminal is used to generate corresponding real-time scene poses according to each real-time positioning request, and return the real-time scene poses to each experience terminal respectively;

[0037] The synchronization control server is used to send a synchronization trigger instruction to each experience terminal, thereby instructing each experience terminal to synchronously display AR special effect content that interacts with the preset object at different angles according to its own real-time scene posture.

[0038] In a fourth aspect, an embodiment of the present application provides another multi-terminal AR content synchronization triggering system, the system comprising: an experience terminal, a positioning service, and a synchronization control server, wherein;

[0039] The experience terminal is used to collect real-time images of the same preset object at different locations in the target scene, generate real-time positioning requests based on the real-time images, and send the real-time positioning requests to the visual positioning service through a public network channel, wherein the visual positioning service is deployed in a cloud host cluster;

[0040] The positioning service is used to generate corresponding real-time scene poses according to each real-time positioning request, and return the real-time scene poses to each experience terminal through a public network channel;

[0041] The synchronization control server is used to send a synchronization trigger instruction to each experience terminal to instruct each experience terminal to synchronously display AR special effect content that interacts with the preset object at different angles according to its own real-time scene posture.

[0042] In a fifth aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in the first and second aspects above is implemented.

[0043] In the first aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in the first and second aspects above.

[0044] Compared with related technologies, the embodiment of the present application provides a multi-terminal AR content synchronization triggering method. Through multiple experience terminals, real-time images of the same preset object are collected at different locations of the target scene, and real-time positioning requests are generated based on the real-time images, and the real-time positioning requests are sent to the local visual positioning terminal; the visual positioning service generates corresponding real-time scene poses according to each real-time positioning request, and returns the real-time scene poses to each experience terminal through the local area network connection channel; after receiving the trigger command, each experience terminal can synchronously display AR special effects content that interacts with the preset object at different angles. This solves the problem of poor flexibility in the multi-terminal AR content synchronization triggering technology in related technologies. User terminals at different locations can display AR special effects content that interacts with the actual scene from different angles, which has better flexibility and can enhance the user's overall experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 2 is a schematic diagram of an application environment of a multi-terminal AR content synchronization triggering method according to an embodiment of the present application;

[0047] Figure 2 This is a flowchart of a multi-terminal AR content synchronization triggering method according to an embodiment of the present application;

[0048] Figure 3 This is a flow chart of configuring and forwarding a positioning request by a cloud host cluster according to an embodiment of the present application;

[0049] Figure 4 is a flowchart of another multi-terminal AR content synchronization triggering method according to an embodiment of the present application;

[0050] Figure 5 This is a structural block diagram of a multi-terminal AR content synchronization triggering system according to an embodiment of the present application;

[0051] Figure 6 is a structural block diagram of another multi-terminal AR content synchronization triggering system according to an embodiment of the present application;

[0052] Figure 7 Schematic diagram of the internal structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application.

[0054] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0055] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0056] Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by a person of ordinary skill in the technical field to which this application belongs. The words "one", "a", "the" and the like used in this application do not indicate a limit on quantity and may indicate the singular or plural. The terms "include", "comprise", "have" and any variations thereof used in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units that are inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The word "multiple" used in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0057] It should be understood that the terms herein may be technical means for implementing a part of the present invention or other summary technical terms. For example, the terms may include:

[0058] AR (Argument Reality): Augmented reality is a technology that cleverly integrates virtual information with the real world. It simulates computer-generated virtual information such as text, images, three-dimensional models, and videos, and applies them to the real world. The two types of information complement each other to achieve "enhancement" of the real world.

[0059] Pose: position and attitude (orientation), for example: in two dimensions it is generally (x, y, yaw), in three dimensions it is generally (x, y, z, yaw, pitch, roll), including 6 degrees of freedom, 6of (6Degrees Of Freedom), the last three elements describe the attitude of the object, where yaw is the heading angle, rotating around the Z axis, pitch is the pitch angle, rotating around the Y axis, and roll is the roll angle, rotating around the X axis.

[0060] Visual positioning: The terminal device captures an offline scene image, and the server matches and processes the image in the scene map to obtain the real-time positioning information (6dof) of the terminal device.

[0061] Intranet penetration: Intranet penetration, also known as NAT penetration, is performed to ensure that data packets with a specific source IP address and source port number are not blocked by the NAT device and are correctly routed to the intranet host.

[0062] Load balancing: When there are a large number of user requests at the same time, the user requests are distributed to multiple server nodes for processing, so that the work tasks can be completed together, achieving the technical effect of improving service response speed.

[0063] The multi-terminal AR content synchronization triggering method provided in this application can be applied to Figure 1 In the application environment shown, Figure 1 : is a schematic diagram of an application environment of a multi-terminal AR content synchronization triggering method according to an embodiment of the present application, such as Figure 1 As shown, the terminal 10 communicates with the positioning server 11 through the network. The terminal 10 can be a terminal device such as a smart phone, tablet computer, and smart wearable device with camera and real-time data processing and transmission functions. The positioning server can be a hardware device deployed in an offline scene or deployed on a public network cluster. In the technical solution of this application, multiple terminals 10 can, under the control signal of the synchronization control server 12, according to the actual scene posture returned by the positioning server, synchronously display AR special effects for specific objects in the offline scene from different angles.

[0064] Example 1

[0065] Figure 2 : is a flowchart of a multi-terminal AR content synchronization triggering method according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0066] S201, multiple experience terminals collect real-time images of the same preset object at different locations in the target scene;

[0067] The experience terminal may be a smartphone, and the target scene may be a museum, a shopping mall, a scenic spot, an amusement park, etc.; it can be further understood that the preset object is a real object offline, which may be a plant (such as a Christmas tree), a sculpture, or a larger building or mountain in the scene;

[0068] A specific image acquisition scenario is as follows: the preset object is a "Lake Center Pavilion" in a historical and cultural scenic area. Four users, A, B, C, and D, each use their smartphone to shoot the "Lake Center Pavilion" from the east, south, west, and north of the shore, obtaining real-time images from different angles.

[0069] It should be noted that, in practical applications, the present application solution is not limited to the above-mentioned scenarios, and can also be applied to smaller or larger scenarios.

[0070] In step S202, multiple experience terminals generate real-time positioning requests based on real-time images and send the real-time positioning requests to a local visual positioning terminal, where the local visual positioning terminal is deployed in the target scene and integrates an intranet penetration service and a visual positioning service.

[0071] Furthermore, the process of generating a positioning request based on a real-time image, that is, encapsulating the captured real-time image in a specific format of the communication protocol, and how to encapsulate and then generate a positioning request are irrelevant to the core invention of this application and can be achieved through conventional means in this field, so they will not be repeated in this embodiment.

[0072] Specifically, when the user is in the above-mentioned target scenario, the local area network is preferred for communication, that is, the positioning request is sent to the local terminal through the local area network channel; in addition, the above-mentioned positioning request can also be forwarded to the local visual positioning terminal through the cloud host through the public network channel based on the intranet penetration technology.

[0073] S203: The local visual positioning terminal generates corresponding real-time scene poses according to each real-time positioning request, and returns the real-time scene poses to each experience terminal;

[0074] It should be noted that, in this application, visual positioning technology is applied to obtain a positioning result that matches the image taken by the user. The positioning result is the scene pose (6dof) of the user terminal when taking real-time images.

[0075] Specifically, the local visual positioning terminal pre-stores a three-dimensional positioning map of the target scene. The local visual positioning terminal extracts features from the real-time image and matches the features in the positioning map to obtain a similar map frame corresponding to the image. Further, by establishing a 2D-3D observation between the local features of the image and the similar map frame, the real-time scene pose is solved based on the 2D-3D observation.

[0076] Among them, the methods for extracting feature points and global and local descriptors from images can be SIFT, SURF, ORB, BOW, VLAD, etc., which can be configured and replaced according to the hardware computing power level and specific scene requirements.

[0077] It can be understood that the local visual positioning terminal will process the positioning requests sent by each terminal device, and then generate the scene posture of each terminal when taking the image; after that, the generated scene posture will be returned to each experience terminal through the local area network connection channel or the public network channel.

[0078] S204, the synchronization control server sends a synchronization trigger instruction to each experience terminal, instructing each experience terminal to synchronously display AR special effect content that interacts with the preset object at different angles according to its own real-time scene posture.

[0079] Among them, after obtaining the scene posture, each experience terminal will send a heartbeat to the synchronization control server to notify the synchronization control server that the visual positioning has been completed and the synchronization trigger is ready.

[0080] In this embodiment, the synchronization control server can be deployed in the target scenario or in a cloud host cluster; specifically, different deployment locations use different communication methods (including local area network channels or public network channels).

[0081] The synchronization controller can automatically trigger the synchronous display of AR special effects. For example, after receiving that all terminals have completed visual positioning, it automatically sends a display instruction to each experience terminal.

[0082] Synchronous display can also be triggered manually. For example, after receiving the visual positioning of all terminals, the synchronization controller pushes a ready signal to the smart device of the operator in the target scene. The operator then transmits the trigger command to the synchronization controller through voice control, terminal interface touch, etc., and the synchronization controller controls all user terminals to display AR content synchronously. Each experience terminal can display the same AR content from different observation angles, which can improve flexibility and enhance user experience.

[0083] A further example is given below with reference to the specific scenario in step S201: After obtaining their respective scene positions, the terminals of the four users A, B, C, and D located in the southeast, northwest, and northeast of the lake pavilion send a ready signal to the synchronization control server. The synchronization server instructs these experience terminals to display the same AR special effect from different angles based on their own real-time positions.

[0084] It should be noted that the AR special effect can produce interactive effects with the preset object "Pavilion in the Center of the Lake", such as a "Phoenix" flying out of the Pavilion in the Center of the Lake; or at night, multiple gradually rising "Kongming Lanterns" are displayed around the Pavilion in the Center of the Lake; and what the four users A, B, C and D see from the terminal interface will be an AR image that is a superposition of virtual and real, and each has a different observation angle, which has better flexibility and a better overall user experience.

[0085] Through the above steps S201 to S204, compared with the existing synchronization method, which has the problem of insufficient flexibility due to the lack of visual positioning, the embodiment of the present application uses a local visual positioning terminal deployed in the scene to provide real-time feedback of the posture data of the user terminal. Multiple user terminals can display AR special effects corresponding to the same object in the scene from different observation angles based on the posture data, thereby improving the flexibility of the existing synchronization triggering solution and enhancing the user experience.

[0086] In some embodiments, when the experience terminal and the visual positioning service are located in the same local area network,

[0087] The experience terminal obtains the local visual positioning terminal through the LAN discovery protocol and sends the real-time positioning request to the local visual positioning terminal through the LAN connection channel;

[0088] After generating the real-time scene pose, the local visual positioning terminal returns the real-time scene pose to each experience terminal through the local area network connection channel.

[0089] In some embodiments, when the experience terminal accesses the local visual positioning terminal through the public network, the experience terminal sends a real-time positioning request to the cloud host cluster through the public network channel;

[0090] The cloud host cluster establishes a connection channel with the intranet penetration service in the local visual positioning terminal through the intranet penetration server, and forwards the real-time positioning request to the local visual positioning terminal through the connection channel;

[0091] After generating the real-time scene pose, the local visual positioning terminal returns the real-time scene pose to each experience terminal through the connection channel.

[0092] It can be seen that in this embodiment, the above two communication interaction modes are provided, and users can flexibly choose according to their own operating habits; it should be noted that since the positioning terminal is deployed in the target scene, it is preferred to use the intranet channel established through the local area network to realize the transmission of positioning requests and positioning results, thereby reducing the delay of the data transmission process and having higher real-time performance.

[0093] In some embodiments, it is considered that when public network communication is adopted, multiple users will access the visual positioning service at the same time. If there are too many users online at the same time, there may be risks of data congestion, low service efficiency, or even unavailability.

[0094] In this embodiment, after the experience terminal sends the real-time positioning request to the cloud host cluster through the public network channel, the following steps are performed to effectively deal with the above risks: Figure 3 This is a flow chart of configuring and forwarding a positioning request by a cloud host cluster according to an embodiment of the present application. Figure 3As shown, this step includes the following steps:

[0095] Step S301: The cloud host cluster allocates at least one optimal node to the positioning request through the load balancing module;

[0096] Step S302: The cloud host cluster queries the local visual positioning terminal corresponding to the optimal node through the service registration center and obtains its link address;

[0097] In step S303, the cloud host cluster instructs the intranet penetration server to establish a connection channel with the intranet penetration service in the local visual positioning terminal according to the link address.

[0098] Through the above steps S301 to S303, user access requests are evenly distributed to each available local visual positioning terminal, thereby avoiding the problem of multiple positioning requests being concentrated on a few positioning terminals, resulting in low service efficiency, data channel congestion and other problems. It can effectively improve the efficiency of user request positioning and ensure the reliability of visual positioning services.

[0099] Example 2

[0100] Figure 4 FIG. 1 is a flowchart of another multi-terminal AR content synchronization triggering method according to an embodiment of the present application. Figure 4 As shown, the process includes the following steps:

[0101] S401, multiple experience terminals collect real-time images of the same preset object at different locations in the target scene;

[0102] In step S402, multiple experience terminals generate real-time positioning requests based on real-time images, and send the real-time positioning requests to the visual positioning service through a public network channel. The visual positioning service is deployed in a cloud host cluster.

[0103] S403: The positioning service generates corresponding real-time scene poses according to each real-time positioning request, and returns the real-time scene poses to each experience terminal through the public network channel;

[0104] S404, sending a synchronization trigger instruction to each experience terminal through the synchronization control server to instruct each experience terminal to synchronously display AR special effect content that interacts with the preset object at different angles according to its own real-time scene posture.

[0105] The difference between Example 2 and Example 1 lies in the different deployment locations of the "visual positioning service". In Example 1, the positioning service is deployed in an offline scenario in the form of hardware, which integrates the intranet penetration service and the visual positioning service; in Example 2, the "visual positioning service" is deployed on a public network cloud host.

[0106] Except for the different data transmission methods, the specific synchronization control method and the display method of AR special effects are the same as those in Example 1. Therefore, in this embodiment, more specific implementation details will not be expanded or illustrated.

[0107] This embodiment also provides a multi-terminal AR content synchronization triggering system, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be repeated here. As used below, the terms "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0108] Figure 5 This is a structural block diagram of a multi-terminal AR content synchronization triggering system according to an embodiment of the present application. Figure 5 As shown, the system includes: an experience terminal 50, a local visual positioning terminal 51 and a synchronization control server 52, wherein;

[0109] The experience terminal 50 is used to collect real-time images of the same preset object at different locations in the target scene, generate real-time positioning requests based on the real-time images, and send the real-time positioning requests to the local visual positioning terminal 51, wherein the local visual positioning terminal 51 is deployed in the target scene and integrates the intranet penetration service and the visual positioning service;

[0110] The local visual positioning terminal 51 is used to generate corresponding real-time scene poses according to each real-time positioning request, and return the real-time scene poses to each experience terminal 50 through the local area network connection channel;

[0111] The synchronization control server 52 is used to send synchronization trigger instructions to each experience terminal 50, instructing each experience terminal 50 to synchronously display AR special effect content that interacts with preset objects at different angles according to its own real-time scene posture.

[0112] Figure 6 is a structural block diagram of another multi-terminal AR content synchronization triggering system according to an embodiment of the present application, such as Figure 6 As shown, the system includes: an experience terminal 60, a positioning service 61 and a synchronization control server 62, wherein;

[0113] The experience terminal 60 is used to collect real-time images of the same preset object at different locations in the target scene, generate real-time positioning requests based on the real-time images, and send the real-time positioning requests to the visual positioning service through the public network channel. The visual positioning service is deployed in the cloud host cluster;

[0114] The positioning service 61 is used to generate corresponding real-time scene poses according to each real-time positioning request, and return the real-time scene poses to each experience terminal through the public network channel;

[0115] The synchronization control server 62 is used to send synchronization trigger instructions to each experience terminal to instruct each experience terminal to synchronously display AR special effect content that interacts with the preset object at different angles according to its own real-time scene posture.

[0116] In one embodiment, Figure 7 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, such as Figure 7 As shown, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The electronic device includes a processor, a network interface, an internal memory, and a non-volatile memory connected via an internal bus, wherein the non-volatile memory stores an operating system, a computer program, and a database. The processor is used to provide computing and control capabilities, the network interface is used to communicate with an external terminal via a network connection, the internal memory is used to provide an environment for the operation of the operating system and the computer program, and when the computer program is executed by the processor, a multi-terminal AR content synchronization triggering method is implemented, and the database is used to store data.

[0117] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0118] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, which can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0119] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A multi-terminal AR content synchronization triggering method, characterized in that: The method comprises: Multiple experience terminals collect real-time images of the same preset object at different locations in the target scene. Generate a real-time positioning request based on the real-time image respectively, and send the real-time positioning request to a local visual positioning terminal through a local area network channel or based on an intranet penetration technology, wherein the local visual positioning terminal is deployed in the target scene, and the local visual positioning terminal integrates an intranet penetration service and a visual positioning service; The local visual positioning terminal generates a corresponding real-time scene pose according to each real-time positioning request, and returns the real-time scene pose to each experience terminal, including: extracting features of the real-time image and matching the features in the positioning map to obtain a similar map frame corresponding to the image; establishing a 2D-3D observation between the local features of the image and the similar map frame, and solving the real-time scene pose based on the 2D-3D observation; The synchronization control server sends a synchronization trigger instruction to each experience terminal automatically or manually, instructing each experience terminal to synchronously display AR special effect content that interacts with the preset object at different angles according to its own real-time scene posture.

2. The method according to claim 1, characterized in that When the experience terminal communicates with the local visual positioning terminal via a local area network: The experience terminal finds the local visual positioning terminal through the local area network discovery protocol, and sends the real-time positioning request to the local visual positioning terminal through the local area network connection channel; After generating the real-time scene posture, the local visual positioning terminal returns the real-time scene posture to each experience terminal through the local area network connection channel.

3. The method according to claim 1, characterized in that When the experience terminal accesses the local visual positioning terminal through the public network: The experience terminal sends the real-time positioning request to the cloud host cluster through the public network channel; The cloud host cluster establishes a connection channel with the intranet penetration module of the local visual positioning terminal through the intranet penetration server, and forwards the real-time positioning request to the local visual positioning terminal through the connection channel; After generating the real-time scene posture, the local visual positioning terminal returns the real-time scene posture to each experience terminal through the connection channel.

4. The method according to claim 3, characterized in that After the experience terminal sends the real-time positioning request to the cloud host cluster through the public network channel, the method further includes: The cloud host cluster allocates the optimal node to the positioning request through a load balancing module; The cloud host cluster queries the local visual positioning terminal corresponding to the optimal node through the service registration center and obtains its link address; The cloud host cluster instructs the intranet penetration server to establish the connection channel with the intranet penetration module in the local visual positioning terminal according to the link address.

5. The method according to claim 1, wherein The synchronization control server is deployed in the target scenario or in a cloud host cluster; The synchronization control server generates the synchronization trigger instruction after receiving the signal that all experience terminals have completed visual positioning.

6. A multi-terminal AR content synchronization triggering method, characterized in that: The method comprises: Multiple experience terminals collect real-time images of the same preset object at different locations in the target scene. Generating real-time positioning requests based on the real-time images respectively, and sending the real-time positioning requests to a visual positioning service through a public network channel, wherein the visual positioning service is deployed in a cloud host cluster; The positioning service generates corresponding real-time scene poses according to each real-time positioning request, and returns the real-time scene poses to each experience terminal through the public network channel; Through the synchronization control server, a synchronization trigger instruction is sent to each experience terminal to instruct each experience terminal to synchronously display the AR special effect content that interacts with the preset object at different angles according to its own real-time scene posture.

7. The method according to claim 6, characterized in that Multiple experience terminals send the real-time positioning request to the positioning service through the public network channel, including: The cloud host cluster allocates the optimal node to the positioning request through a load balancing module; The cloud host cluster queries the link address corresponding to the optimal node through the service registration center, and forwards the real-time positioning request to the visual positioning service corresponding to the link address.

8. A multi-terminal AR content synchronization triggering system, characterized in that: The system includes: an experience terminal, a local visual positioning terminal and a synchronization control server, wherein; The experience terminal is used to collect real-time images of the same preset object at different locations in the target scene, generate real-time positioning requests based on the real-time images, and send the real-time positioning requests to the local visual positioning terminal through a local area network channel or based on intranet penetration technology, wherein the local visual positioning terminal is deployed in the target scene and integrates intranet penetration service and visual positioning service; The local visual positioning terminal is used to generate corresponding real-time scene poses according to each real-time positioning request, and return the real-time scene poses to each experience terminal, including: extracting features of the real-time image and matching the features in the positioning map to obtain a similar map frame corresponding to the image; establishing 2D-3D observations between local features of the image and the similar map frames, and solving the real-time scene poses based on the 2D-3D observations; The synchronization control server is used to send synchronization trigger instructions to each experience terminal in an automatic or manual manner, instructing each experience terminal to synchronously display AR special effect content that interacts with the preset object at different angles according to its own real-time scene posture.

9. A multi-terminal AR content synchronization triggering system, characterized in that: The system includes: an experience terminal, a positioning service and a synchronization control server, wherein; The experience terminal is used to collect real-time images of the same preset object at different locations in the target scene, generate real-time positioning requests based on the real-time images, and send the real-time positioning requests to the visual positioning service through a public network channel, wherein the visual positioning service is deployed in a cloud host cluster; The positioning service is used to generate a corresponding real-time scene pose according to each real-time positioning request, and return the real-time scene pose to each experience terminal through a public network channel, including: extracting features of the real-time image and matching the features in the positioning map to obtain a similar map frame corresponding to the image; establishing 2D-3D observations between local features of the image and the similar map frame, and solving the real-time scene pose based on the 2D-3D observations; The synchronization control server is used to send synchronization trigger instructions to each experience terminal automatically or manually, so as to instruct each experience terminal to synchronously display AR special effects content that interacts with the preset object at different angles according to its own real-time scene posture.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images