An AR real scene map dynamic superimposed label method, system, device and medium

Abstract

The application provides an AR real scene map dynamic superimposed label method, system, device and medium, which is applied to a webpage client and includes the following steps: obtaining video decoding information; obtaining AR label information by analyzing SEI frame information in the video decoding information; generating and rendering an operable label layer according to the AR label information; changing the operable label layer in response to user operation; obtaining new AR label information according to the changed operable label layer; outputting the new AR label information to a backend server to enable the backend server to update SEI frame information according to the new AR label information; and re-rendering the operable label layer in response to update information sent by the client after listening to the updated SEI frame information. Through webpage program development, the AR real scene map dynamic superimposed label method provided by the application has cross-platform and dynamic properties, has low development cost, and is flexible to publish. The preview layer and the operable label layer of the application are decoupled, the program complexity is reduced, and the business flexibility is improved.

Description

Technical Field

[0001] This invention relates to the field of augmented reality technology, specifically to a method, system, device, and medium for dynamically overlaying labels on an AR real-world map. Background Technology

[0002] AR (Augmented Reality) maps are an innovative map service that combines Augmented Reality (AR) technology with Geographic Information System (GIS). By overlaying virtual information onto a user's real-time video stream, it allows users to see additional data and images in the real world, thereby enhancing their perception and interactive experience. With continuous technological advancements and further market expansion, AR maps are expected to play an even more important role in the future.

[0003] In the existing AR real-time dynamic overlay label solution, several AR labels to be drawn in the real-time video are determined based on the gimbal information and the information in the AR labels, and coordinate transformation is performed. The backend device draws and generates the AR label interface, which is overlaid with the video frame data of the real-time video to obtain the video frame data to be displayed and sent to the front-end device to display the interactive video screen. When the user operation coordinate information sent by the front-end device is received, the interactive object and the corresponding user operation response result are determined and the AR label interface is updated. New video frame data is generated and sent to the front-end device to display the updated interactive video screen.

[0004] The above technical solution involves drawing the AR tag interface separately on the backend device, overlaying the AR interface and video frames to generate real-time video data, and then transmitting it to the frontend device. In this process, the video frame overlay processing will inevitably cause algorithm delays. Moreover, the frontend device can only receive the overlaid video frames. If it needs to perform operations such as modifying or hiding tags, it needs to notify the backend device to modify the video frame data. The communication between the frontend and backend devices is too frequent, which can easily affect the smoothness of the backend device's operation. Summary of the Invention

[0005] This application provides a method, system, device, and medium for dynamically overlaying labels on AR real-scene maps, which makes the smooth operation of backend devices less affected.

[0006] The first aspect of this application provides a method for dynamically overlaying labels on an AR real-scene map.

[0007] A method for dynamically overlaying labels on an AR real-world map includes the following steps:

[0008] Obtain video decoding information;

[0009] AR tag information is obtained by parsing the SEI frame information in the video decoding information;

[0010] Generate an operable tag layer based on AR tag information;

[0011] In response to user actions, change the operable label layer;

[0012] Based on the changed operable label layer, obtain the new AR label information;

[0013] Output the new AR tag information to the backend server so that the backend server can update the SEI frame information based on the new AR tag information;

[0014] In response to the update information sent by the client after listening for updates to the SEI frame information, render the operable tag layer to obtain interactive video footage.

[0015] Preferably, generating an operable tag layer based on AR tag information includes:

[0016] The AR tag coordinate information in the AR tag information is denormalized based on the preset layer size;

[0017] The label layer is generated by processing the denormalized AR label coordinate information using Canvas.

[0018] Preferably, after obtaining the video decoding information, the method further includes:

[0019] Extract video frame information from the video decoding information;

[0020] A preview layer is generated below the operable label layer based on the video frame information.

[0021] Preferably, the video frame images in the video frame information are in YUV format.

[0022] Preferably, in response to user actions, changing the operable label layer includes the following steps:

[0023] Based on the operation information sent by the user through the client, the label information in the operable label layer can be added, edited, or deleted; the label information is associated with the AR label coordinate information.

[0024] A second aspect of this application provides an AR real-scene map dynamic overlay label system.

[0025] An AR real-scene map dynamic overlay label system is characterized by comprising: a web client for executing an AR real-scene map dynamic overlay label method, wherein the web client, the client and the backend server are all connected by communication connections, and the backend server is coupled to multiple AR panoramic cameras.

[0026] Preferably, the backend server includes:

[0027] The device management service module is used to acquire the video stream and device information transmitted by the AR panoramic camera and process the video stream and device information to obtain a video stream with AR tag information.

[0028] The media forwarding service module is used to send video streams with AR tag information to the client;

[0029] The client is used to decode video streams with AR tag information to obtain video decoding information.

[0030] Preferably, the backend server further includes:

[0031] The application service module is used to receive updated AR tag information reported to the device management service module to update the corresponding SEI frame information.

[0032] An electronic device is provided in the third aspect of this application.

[0033] An electronic device includes a processor, a memory, and a transceiver. The memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform an AR real-scene map dynamic overlay label method.

[0034] A computer-readable storage medium is provided in the fourth aspect of this application.

[0035] A computer-readable storage medium includes a processor, a memory, and a transceiver, wherein the memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform an AR real-scene map dynamic overlay label method.

[0036] In summary, this application includes at least one of the following beneficial technical effects:

[0037] 1. By developing a web application, the AR real-scene map dynamic overlay label method of this application has cross-platform and dynamic characteristics, low development cost, and flexible release.

[0038] 2. The preview layer and operable label layer of this application are decoupled, which reduces program complexity and improves business flexibility; since the original video stream does not need to be updated after decoupling, communication between front-end and back-end devices is reduced, thus reducing latency. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating an AR real-scene map dynamic overlay label method according to an embodiment of this application;

[0040] Figure 2 This is a detailed flowchart illustrating step S3 of an AR real-scene map dynamic overlay label method according to an embodiment of this application;

[0041] Figure 3 This is a scene diagram of an AR real-scene map dynamic overlay label system according to an embodiment of this application;

[0042] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0043] Explanation of reference numerals in the attached diagram: 1. Web client; 2. Client; 3. Backend server; 31. Device management service module; 32. Media forwarding service module; 33. Application service module; 4. AR panoramic camera. Detailed Implementation

[0044] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0045] In the description of the embodiments in this application, words such as "illustrative," "for example," or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "illustrative," "for example," or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of words such as "illustrative," "for example," or "for example" is intended to present the relevant concepts in a specific manner.

[0046] In the description of the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, or A and B existing simultaneously. Furthermore, unless otherwise stated, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0047] Scenario: This application is mainly applied to the improvement of computer vision technology. By adding a web client, it improves the latency problem caused to the backend server by frequent information interaction between the client and the backend server and video frame overlay processing in the existing technology, reduces program complexity and reduces communication latency between front-end and back-end devices.

[0048] The proper nouns used in this application include:

[0049] AR Real-View Map: Based on video spatialization, it uses computer vision technology to merge the virtual and the real, allowing users to see the real environment and more information on the map.

[0050] AR panoramic camera: A panoramic shooting device that combines augmented reality (AR) technology. It can capture 360-degree images of the surrounding environment and, after software processing, overlay virtual information onto the real world to provide users with an immersive visual experience.

[0051] AR tags: visual markers used in augmented reality applications, primarily for tracking and locating the position of virtual objects in the real world.

[0052] SEI frames are a mechanism in video coding standards used to transmit supplementary information unrelated to image encoding and decoding. SEI belongs to the bitstream category and provides a method for adding extra information to the video bitstream; it is one of the features of the H.264 and H.265 standards.

[0053] YUV format: An important color encoding method that optimizes the compression and transmission of video and images by separating luminance and chrominance components, while taking into account the characteristics of the human visual system, thus reducing the required bandwidth while maintaining visual quality.

[0054] WebSocket: An application-layer network protocol based on TCP. It establishes connections through the HTTP protocol upgrade mechanism and enables full-duplex real-time communication. WebSocket is designed to overcome the shortcomings of HTTP in terms of real-time performance and interactivity, making it suitable for application scenarios requiring real-time data exchange.

[0055] WebGL is a technology for drawing and rendering 3D graphics on web pages, allowing developers to use it directly in the browser without installing any plugins or libraries. Based on the OpenGL ES 2.0 standard, it supports programmable shaders, enabling developers to create complex 3D scenes and models. WebGL works by using a rasterization engine to draw basic shapes such as points, lines, and triangles based on code, thereby constructing more complex 3D content.

[0056] Canvas: A new element added in HTML5 that allows developers to draw images on web pages using JavaScript. Canvas itself is a rectangular area, but it doesn't have built-in drawing functionality; drawing operations are performed using JavaScript. Canvas provides 2D and 3D drawing APIs, enabling developers to draw various graphics and animations on the canvas. Its applications are very wide-ranging, including but not limited to data visualization, game development, and chart creation.

[0057] Plugin monitoring: refers to the technology of monitoring and responding to specific events or behaviors during plugin development.

[0058] Reference Figure 1 and Figure 2 A method for dynamically overlaying labels on AR real-world maps includes the following steps:

[0059] S1: Obtain video decoding information;

[0060] Specifically: The original video information of the video decoding information is collected by the AR panoramic camera. The AR panoramic camera 4 transmits the original video information to the backend server 3. The backend server 3 calculates and stores the corresponding AR tag coordinate information based on the received original video information and device information. The original video information can be understood as a video stream. The device information includes the rotation angle and elevation angle information of the current AR panoramic camera 4 and AR tag information.

[0061] After storing the current coordinate information of the AR tag, the backend server 3 packages the processed video stream and AR tag information and pushes it to the client 2. The pushed video stream contains the real-time preview stream of the current AR panoramic camera 4, while the AR tag information is stored in the SEI frame of the video stream. It should be noted that the AR tag information at this time includes the AR tag identifier, business information, and AR tag coordinate information corresponding to the AR tag identifier. The AR tag coordinate information is normalized pixel coordinates. It should also be noted that the backend server 3 stores the SEI frame of the video stream.

[0062] After receiving the pushed video stream, client 2 uses the client 2 plugin to decode the video stream, obtain video decoding information, and sends the video decoding information to web client 1; thus enabling web client 1 to obtain video decoding information; the video decoding information includes video frame information and SEI frame information; the client 2 plugin can be implemented in various forms, such as an executable program (e.g., an .exe under the Windows operating system), a browser plugin, or a WebAssembly that can be directly called by web programs, etc., and its main function is for video decoding.

[0063] S2: Obtain AR tag information by parsing the SEI frame information in the video decoding information;

[0064] Specifically: Web client 1 uses video processing libraries or tools, such as FFmpeg, to parse SEI frame information and extract AR tag information from the SEI frame information; there are multiple SEI frames in the video decoding information, and there are also multiple corresponding SEI frames for AR tag information.

[0065] S3: Generate and render an operable tag layer based on AR tag information;

[0066] Specifically: Step S3 includes the following steps:

[0067] S31: Parse the AR tag information to obtain the AR tag coordinate information;

[0068] Specifically: AR tag coordinate information is stored in a specific area. By finding the area through location lookup, the AR tag coordinate information can be obtained, thus completing the parsing process to obtain the AR tag coordinate information;

[0069] S32: Denormalize AR tag coordinate information based on layer size;

[0070] Specifically, in the fields of computer vision and image processing, "inverse normalization" generally refers to the process of restoring normalized data to its original range or distribution before normalization. This involves adapting the inverse normalized AR label coordinate information to the layer size of the web client 1 to facilitate subsequent operations.

[0071] S33: Generate an operable label layer by processing the denormalized AR label coordinate information using Canvas;

[0072] Specifically, web client 1 can use JavaScript on the Canvas to draw the denormalized AR tag coordinate information, thereby generating an operable tag layer; after the operable tag layer is generated, web client 1 will call the Canvas to render the operable tag layer; it should be noted that there are multiple SEI frames corresponding to the operable tag layer.

[0073] S4: Extract video frame information from the video decoding information, and generate a preview layer below the operable label layer based on the video frame information;

[0074] Specifically, video frame information can be understood as the video frames in the video stream corresponding to the AR tag coordinate information. The format of the video frame images in the video frame information is converted to YUV format. This step can be performed on web client 1, or in other embodiments on client 2. Based on the video frame images, a preview layer is generated below the operable tag layer. This preview layer can be generated by directly importing video frame images. It should be noted that the preview layer serves as a reference to the operable layer in this embodiment, facilitating the restoration of the operable layer. Steps S4 and S2-S3 can be performed simultaneously and have no sequential relationship.

[0075] It is also worth noting that after the preview layer and the operable label layer are generated for the first time, the web client 1 uses Webgl+Canvas to render the screen and overlay multiple operable label layers, i.e., the AR interface.

[0076] S5: In response to user actions, change the operable label layer;

[0077] Specifically: Based on the operation information issued by the user through client 2, label information in multiple operable label layers can be added, edited, or deleted. Label information can be understood as the association information of AR label coordinates in the operable label layers. The user can operate on client 2 using the keyboard, and client 2 issues corresponding commands to control the drawing tools on web client 1 to add, edit, or delete label information in the operable label layers.

[0078] S6: Obtain new AR tag information based on the changed operable tag layer;

[0079] Specifically, since the operable label layer is obtained by denormalizing AR label coordinate information, the web client 1 can convert the label information in the modified operable label layer into new AR label coordinate information, replace the AR label coordinate information in the AR label information, and thus obtain new AR label information.

[0080] S7: Output the new AR tag information to the backend server 3 so that the backend server 3 can update the SEI frame information according to the new AR tag information;

[0081] Specifically, after the personnel operate on all the operable label layers, the client 2 issues a confirmation command. Then, the web client 1 rewrites the new AR label information into the corresponding SEI frame information and transmits it to the backend server 3. The backend server 3 identifies the new AR label information and updates the corresponding SEI frame information, which can be understood as replacing the old SEI frame information with the new SEI frame information.

[0082] S8: In response to the update information sent by client 2 after listening for the update of SEI frame information, re-render the operable label layer.

[0083] Specifically, client 2 has a listening plugin that can monitor updates to AR tag information in the 3SEI frame information of the backend server. Upon detecting an update, client 2 transmits the update information to web client 1. Web client 1, upon receiving the update information, re-renders the operable tag layers and overlays all operable tag layers on the webpage, thus creating an interactive video display that allows users to understand the rendering effect of the modified AR video. It should be noted that the preview layer remains unchanged throughout this process; in all...

[0084] The implementation principle of the AR real-scene map dynamic overlay label method provided in this application embodiment is as follows: Labels and video stream data are separated, and rendering is performed in a browser using Web technology. This increases the flexibility of label operations, reduces program complexity, and reduces communication between front-end and back-end devices, thereby further reducing video preview latency. Furthermore, due to the cross-platform and dynamic nature of Web applications, a single codebase can run on multiple platforms, reducing development costs; moreover, Web application deployment is dynamic, and client-side programs can be updated in real time.

[0085] Reference Figure 3 An AR real-scene map dynamic overlay label system includes a web client 1, a backend server 3, and a client 2 for sending video decoding information. The web client 1, client 2, and backend server 3 are all connected by network communication. The backend server 3 is coupled with multiple AR panoramic cameras 4.

[0086] The backend server 3 includes a device management service module 31, a media forwarding service module 32, and an application service module 33. The device management service module 31, the media forwarding service module 32, and the application service module 33 are all software programs of the backend server 3.

[0087] Among them, the device management service module 31 is used to obtain the video stream and device information transmitted by the AR panoramic camera 4 and process the video stream and device information to obtain a video stream with AR tag information.

[0088] The media forwarding service module 32 is used to send video streams with AR tag information to client 2.

[0089] Application service module 33 is used to receive updated AR tag information reported to device management service module 31 to update the corresponding SEI frame information.

[0090] Client 2 is used to decode the video stream with AR tag information to obtain video decoding information, and then transmit the video decoding information to web client 1.

[0091] The implementation principle of the AR real-scene map dynamic overlay label system disclosed in this embodiment is as follows: The device management service module 31 obtains the rotation angle and elevation angle of the AR panoramic camera 4 currently selected by the user, and calculates the coordinate information of the corresponding AR label; the media forwarding service module 32 pulls the video stream from the device management service and pushes the video stream to the client 2. The media stream data includes the real-time preview stream of the current AR panoramic camera 4, where the AR label information is stored in the SEI frame of the video stream; the client 2 plugin converts the video stream information into YUV image format and passes it to the web client 2, and splits the SEI frame information and passes it to the web client 2; the web client 2 renders the video stream using Webgl+Canvas and parses the AR label information, and draws it on the page using Canvas; after the user adds, edits or deletes label information, the web client 2 reports the updated content to the application service module 33; the application service module 33 reports the updated information to the device management service module 31, the SEI frame information in the video stream pulled by the media forwarding service module 32 is updated, the client 2 listens for the AR label information update, and re-renders the updated AR label on the page.

[0092] Reference Figure 4 The present application provides a schematic diagram of the structure of an electronic device. The electronic device 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002.

[0093] The communication bus 1002 is used to realize the connection and communication between these components.

[0094] The user interface 1003 may include a display screen and a camera. Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.

[0095] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0096] The processor 1001 may include one or more processing cores. The processor 1001 connects to various parts within the server 1000 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by calling data stored in the memory 1005. Optionally, the processor 1001 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 1001 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 1001 and may be implemented as a separate chip.

[0097] The memory 1005 may include random access memory (RAM) or read-only memory. Optionally, the memory 1005 may include a non-transitory computer-readable storage medium. The memory 1005 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. The memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for the AR real-scene map dynamic overlay label method.

[0098] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0099] In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 1001 can be used to call the application program stored in the memory 1005 for the AR real-scene map dynamic overlay label method. When executed by one or more processors, the electronic device performs one or more of the methods described in the above embodiments.

[0100] An electronic device readable storage medium stores instructions. When executed by one or more processors, these instructions cause the electronic device to perform one or more of the methods described in the above embodiments.

[0101] Those skilled in the art will clearly understand that the technical solutions of this application can be implemented using software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware capable of independently or in conjunction with other components to perform a specific function. Hardware may include, for example, a Field-Programmable Gate Array (FPGA), an Integrated Circuit (IC), etc.

[0102] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0103] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0104] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0106] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0107] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0108] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0109] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for dynamically overlaying labels on an AR real-scene map, characterized in that, Includes the following steps: Obtain video decoding information; AR tag information is obtained by parsing the SEI frame information in the video decoding information; Generate several operable label layers based on AR label information; In response to user actions, change the operable label layer; Based on the changed operable label layer, obtain the new AR label information; Output the new AR tag information to the backend server (3) so that the backend server (3) can update the SEI frame information according to the new AR tag information; In response to the update information sent by the client (2) after listening for the update of the SEI frame information, several operable label layers are rendered and superimposed to obtain an interactive video screen; Generate an operable tag layer based on AR tag information, including: The AR tag coordinate information in the AR tag information is denormalized based on the preset layer size; The label layer is generated by processing the denormalized AR label coordinate information using Canvas; After obtaining the video decoding information, it also includes: Extract video frame information from the video decoding information; A preview layer is generated below the operable label layer based on the video frame information; The client (2) has a listening plugin. The client (2) plugin can listen to the update of AR tag information in the SEI frame information of the backend server (3). After the client (2) plugin listens to the update of AR tag information, it transmits the update information to the web client. After receiving the update information, the web client will re-render the operable tag layer and render and overlay all operable tag layers on the web page to obtain an interactive video screen, so that people can understand the rendering effect of the modified AR video.

2. The AR real-scene map dynamic overlay label method according to claim 1, characterized in that: The video frame images in the video frame information are in YUV format.

3. The AR real-scene map dynamic overlay label method according to claim 1, characterized in that, In response to user actions, the operable label layer is modified, including the following steps: Based on the operation information sent by the user through the client (2), the label information in the operable label layer can be added, edited or deleted; wherein, the label information is associated with the AR label coordinate information.

4. An AR real-scene map dynamic overlay label system, characterized in that, include: A web client (1) is used to execute the AR real-scene map dynamic overlay label method as described in any one of claims 1-3. The web client (1), client (2) and backend server (3) are all connected by communication. The backend server (3) is coupled to multiple AR panoramic cameras (4).

5. The AR real-scene map dynamic overlay label system according to claim 4, characterized in that, The backend server (3) includes: The device management service module (31) is used to obtain the video stream and device information transmitted by the AR panoramic camera (4) and process the video stream and device information to obtain a video stream with AR tag information; The media forwarding service module (32) is used to send video streams with AR tag information to the client (2); The client (2) is used to decode the video stream with AR tag information to obtain video decoding information.

6. The AR real-scene map dynamic overlay label system according to claim 4, characterized in that, The backend server (3) also includes: The application service module (33) is used to receive updated AR tag information reported to the device management service module (31) to update the corresponding SEI frame information.

7. An electronic device, characterized in that, The device includes a processor, a memory, and a transceiver, wherein the memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, A computer program includes program instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-3.

Images