Method and device for transmitting augmented reality video stream

Abstract

The aim of the present application is to provide a method for transmitting an augmented reality video stream. The method comprises: by means of a target monitoring platform and a shared channel corresponding to a target camera apparatus, acquiring and presenting a target video stream collected in real time by the corresponding target camera apparatus; acquiring an augmented reality operation of a user with respect to the target video stream, and superposing corresponding augmented reality information onto some / all of the video frames of the target video stream; and generating a corresponding augmented reality video stream on the basis of the augmented reality information and the target video stream, and by means of the channel corresponding to the target camera apparatus that is shared by the target monitoring platform with other video platforms, pushing the augmented reality video stream to the other video platforms. In the present solution, augmented reality information is superposed onto a video stream, and an augmented reality video stream is pushed to other video platforms, such that multi-platform video interaction is implemented, thereby improving the efficiency of monitoring video stream information interaction, and improving the usage experience of users.

Description

Method and device for transmitting augmented reality video stream

[0001] This application is based on and claims priority to an application with CN application number 202310945913.X and application date 2023.07.28. The disclosed content of the CN application is hereby incorporated into this application as a whole. Technical Field

[0002] The present application relates to the field of communications, and in particular to a technology for transmitting augmented reality video streams. Background Art

[0003] Video surveillance includes front-end cameras, transmission cables, and video surveillance platforms. Cameras can be divided into network digital cameras and analog cameras, which can be used to collect front-end video image signals. A complete video surveillance system is composed of five major parts: recording, transmission, control, display, and recording and registration. The camera transmits the video image to the control host through a network cable or a coaxial video cable. The control host then distributes the video signal to each monitor and recording device. At the same time, the voice signal to be transmitted can be synchronously recorded into the recorder. Through the control host, the operator can issue instructions to control the up, down, left, and right movements of the pan / tilt head and to focus and zoom the lens, and can also switch between multiple cameras through the video matrix. Using a special recording processing mode, images can be recorded, played back, called out, and stored.

[0004] Summary of the Invention

[0005] One object of the present application is to provide a method and device for transmitting augmented reality video streams.

[0006] According to one aspect of the present application, a method for transmitting an augmented reality video stream is provided, wherein the method includes:

[0007] Acquire and present the target video stream captured in real time by the corresponding target camera device through the shared channel corresponding to the target monitoring platform and the target camera device;

[0008] Acquire an augmented reality operation of the user on the target video stream, and overlay corresponding augmented reality information on some / all video frames of the target video stream;

[0009] A corresponding augmented reality video stream is generated according to the augmented reality information and the target video stream, and the augmented reality video stream is pushed to the other video platforms through the channel corresponding to the target camera device shared by the target monitoring platform to the other video platforms.

[0010] According to one aspect of the present application, a device for transmitting an augmented reality video stream is provided, wherein the device includes:

[0011] a module for acquiring and presenting a target video stream captured in real time by a corresponding target camera device through a shared channel corresponding to the target monitoring platform and the target camera device;

[0012] Module one and module two are used to obtain the user's augmented reality operation on the target video stream and superimpose corresponding augmented reality information on part or all of the video frames of the target video stream;

[0013] Module 13 is used to generate a corresponding augmented reality video stream based on the augmented reality information and the target video stream, and push the augmented reality video stream to the other video platforms through the channel corresponding to the target camera device shared by the target monitoring platform to other video platforms.

[0014] According to one aspect of the present application, a computer device is provided, wherein the device includes:

[0015] processor; and

[0016] A memory arranged to store computer executable instructions, which when executed cause the processor to perform the steps of any of the methods described above.

[0017] According to one aspect of the present application, a computer-readable storage medium is provided, on which a computer program / instruction is stored, characterized in that when the computer program / instruction is executed, the system performs the steps of any of the methods described above.

[0018] According to one aspect of the present application, a computer program product is provided, comprising a computer program / instruction, wherein the computer program / instruction implements the steps of any of the above methods when executed by a processor.

[0019] Compared to existing technologies, this application uses the shared channel of the target camera device to obtain the target video stream and generate a corresponding augmented reality video stream based on the user's augmented reality operation. This augmented reality video stream is then pushed to other video platforms through the shared channel. This solution achieves the superposition of augmented reality information in the video stream and pushes the augmented reality video stream to other video platforms, realizing multi-platform video interaction, improving the efficiency of monitoring video stream information interaction, and enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0021] FIG1 shows a flow chart of a method for transmitting an augmented reality video stream according to an embodiment of the present application;

[0022] FIG2 shows functional modules of a computer device 100 according to another embodiment of the present application;

[0023] FIG3 illustrates an exemplary system that may be used to implement various embodiments described herein.

[0024] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0025] The present application is described in further detail below with reference to the accompanying drawings.

[0026] In a typical configuration of the present application, the terminal, the device of the service network and the trusted party all include one or more processors (eg, a central processing unit (CPU)), an input / output interface, a network interface and a memory.

[0027] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory. Memory is an example of a computer-readable medium.

[0028] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PCM), programmable random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computer device.

[0029] The devices referred to in this application include but are not limited to user devices, network devices, or devices formed by integrating user devices and network devices through a network. The user devices include but are not limited to any mobile electronic product that can interact with a user (for example, through a touchpad), such as a smartphone, a tablet computer, a surveillance camera, etc. The mobile electronic product can use any operating system, such as the Android operating system, the iOS operating system, etc. Among them, the network device includes an electronic device that can automatically perform numerical calculations and information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc. The network device includes but is not limited to a computer, a network host, a single network server, a plurality of network servers, or a cloud composed of multiple servers; here, the cloud is composed of a large number of computers or network servers based on cloud computing, wherein cloud computing is a type of distributed computing, a virtual supercomputer composed of a group of loosely coupled computers. The network includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, a wireless self-organizing network (Ad Hoc network), etc. Preferably, the device may also be a program running on the user device, the network device, or a device formed by integrating the user device and the network device, the network device and the touch terminal, or the network device and the touch terminal via a network.

[0030] Of course, those skilled in the art should understand that the above-mentioned devices are only examples, and other existing or future devices that are applicable to this application should also be included in the scope of protection of this application and are included here by reference.

[0031] In the description of the present application, “plurality” means two or more, unless otherwise clearly defined.

[0032] FIG1 shows a method for transmitting an augmented reality video stream according to one aspect of the present application, which is applied to a computer device and specifically includes steps S101, S102, and S103. In step S101, a target video stream captured in real time by a corresponding target camera device is acquired and presented through a shared channel corresponding to the target monitoring platform and the target camera device; in step S102, a user's augmented reality operation on the target video stream is acquired, and corresponding augmented reality information is superimposed on part / all of the video frames of the target video stream; in step S103, a corresponding augmented reality video stream is generated based on the augmented reality information and the target video stream, and the augmented reality video stream is pushed to the other video platforms through the channel corresponding to the target camera device shared by the target monitoring platform to the other video platforms. For example, the computer device includes but is not limited to user devices, network devices, or a combination of user devices and network devices; wherein, the user device includes but is not limited to any electronic product that can interact with the user, such as smart phones, tablet computers, smart glasses, smart helmets, web cameras, etc.; the network device includes but is not limited to computers, video surveillance platforms, network hosts, single network servers, multiple network server sets or a cloud composed of multiple servers, such as a ground control center, a video surveillance platform, etc.

[0033] Specifically, in step S101, the target video stream captured in real time by the corresponding target camera device is acquired and presented through the shared channel corresponding to the target monitoring platform and the target camera device. For example, the target monitoring platform includes a video platform that takes network centralized management and network transmission as its core to complete information transmission, control, management and storage; through this monitoring platform, monitoring networking and centralized management are realized, and authorized users can monitor the monitoring site in real time on any computer on the network, providing a powerful and flexible network centralized monitoring platform. The target camera device includes a camera device used in security, which usually has higher corresponding pixels and resolution than a computer video head and lower than a professional digital camera. Most monitoring cameras are just single video capture devices, and specific appearances include gun-type, hemispherical, high-speed ball and other types. The target camera device is used to encode the collected image data and transmit it to the video platform. The channel corresponding to the target camera device refers to the physical location of the video input of the target camera device on the monitoring matrix or hard disk recorder device. The target camera device is connected to the target monitoring platform via the GB / T28181 protocol. For example, the target camera device is directly connected to the target monitoring platform, or the target camera device is directly connected to other video platforms (such as other monitoring platforms, network video recorders, etc.), and is configured and connected to the target monitoring platform through other video platforms or other video platforms. GB / T28181 "Technical Requirements for Information Transmission, Exchange, and Control of Public Security Video Surveillance Network Systems" is a national standard proposed by the Science and Technology Information Bureau of the Ministry of Public Security, under the jurisdiction of the National Security Alarm System Standardization Technical Committee (SAC / TC 100), and jointly drafted by multiple units including the First Institute of the Ministry of Public Security. This standard specifies the interconnection structure and communication protocol structure for information transmission, exchange, and control in urban monitoring and alarm network systems, the basic requirements and security requirements for transmission, exchange, and control, as well as technical requirements such as control and transmission processes and protocol interfaces. This standard applies to the design, system testing, acceptance, and related equipment development and production of security and prevention monitoring and alarm network systems, and can be used as a reference for other information systems. Specifically, the target camera device can achieve access to the target monitoring platform by setting the corresponding national standard code, video channel code ID, and configuring the SIP server IP address on the configuration page. After the target monitoring platform establishes a connection with the target camera device / the video platform to which the target camera device is cascaded, the corresponding target camera device / the video platform to which the target camera device is cascaded can set some or all channels of the target camera device to be shared. For example, in the aforementioned configuration interface, the corresponding video channel code ID is configured. If the configuration is empty, it means that the target monitoring platform is not connected as a shared channel, etc., and the video stream collected by the corresponding video channel code ID of the target camera device is pushed to the target monitoring platform as the upper-level video platform. The video channel of the target camera device can be one or more, which is not limited here.

[0034] In some embodiments, the method further includes step S104 (not shown), in which one or more video platforms are cascaded with the target monitoring platform, wherein at least one channel of the video platform and the target monitoring platform is set as a shared channel. For example, the target monitoring platform can serve as an upper-level monitoring platform, one or more lower-level video platforms / camera devices can be registered to the upper-level target monitoring platform, and the upper-level target monitoring platform can play the captured video of one or more lower-level video platforms / camera devices. For example, the corresponding lower-level video platform / camera device can set the channels of all / part of the camera devices to be shared, thereby pushing the video stream to the upper-level target monitoring platform. In other cases, the target monitoring platform can also be registered as a lower-level monitoring platform to one or more upper-level video platforms, and the channels of all / part of the video streams obtained by the target monitoring platform are shared to one or more upper-level video platforms, thereby pushing all / part of the video streams to the upper-level video platform, etc.

[0035] In some embodiments, the target camera device or the video platform corresponding to the target camera device is registered to the target monitoring platform through national standard cascading. For example, the target monitoring platform serves as the upper-level monitoring platform, and the target camera device or the video platform corresponding to the target camera device can be registered to the target monitoring platform through national standard cascading to achieve cascading with the target monitoring platform, etc., wherein cascading means that there is a superior-subordinate relationship between two signaling security routing gateways, and the lower-level signaling security routing gateway actively initiates registration with the upper-level signaling security routing gateway, and inter-system communication can only be carried out after authentication and certification by the upper-level signaling security routing gateway. Furthermore, the video platform to which the target camera device / target camera device is cascaded can set all / part of the channels to be shared, and push the target video stream shot by the corresponding shared channel to the target monitoring platform based on the corresponding communication connection.

[0036] Typically, after the target monitoring platform obtains the video stream collected by the target camera device corresponding to the shared channel through the national standard cascade, the corresponding video stream is presented through the display device of the monitoring platform (for example, a display screen or a projector, etc.). In some cases, the target monitoring platform, as the upper-level monitoring platform, will simultaneously obtain the video streams collected by multiple video platforms or camera devices corresponding to the shared channel, and the target monitoring platform can simultaneously present multiple video streams in a split-screen manner. For example, in some embodiments, the method further includes step S105 (not shown), in which one or more video streams pushed by one or more video platforms through the corresponding shared channel are simultaneously presented through the display device of the target monitoring platform. For example, one or more video platforms are registered with the target monitoring platform through the national standard cascade, and the corresponding channels are shared with the target monitoring platform through the one or more video platforms. For example, the corresponding one or more video platforms can set all / part of the channels of the camera device to be shared, thereby pushing one or more video streams captured by the corresponding shared channels to the target monitoring platform. The target monitoring platform can present one of the one or more video streams on the display screen based on the user's playback operation on one of the one or more acquired video streams, such as touching a play button, etc. If playback operations on multiple video streams are acquired, the selected multiple video streams can be presented on the display screen at the same time, such as presenting multiple video streams through split-screen, wherein the split-screen ratio can be customized by the user, etc. In some embodiments, after presenting the video stream on the display screen, the target monitoring platform can adjust the presentation of the video stream on the display screen based on the user's control operation, such as touching the pan / tilt button on the display screen to display the video stream after rotating the pan / tilt, or touching the zoom button on the display screen to display the zoomed video stream, etc. In some cases, the target monitoring platform can locate or identify the features of the corresponding scene by the video stream captured by the target camera device corresponding to the shared channel obtained by the target monitoring platform, and filter out other video streams whose position difference is less than the position difference threshold or have similar features from the video streams captured by other shared channels obtained by the target monitoring platform, and perform split-screen presentation of the target video stream and the filtered out other video streams.

[0037] In step S102, the user's augmented reality operation on the target video stream is obtained, and the corresponding augmented reality information is superimposed on part / all of the video frames of the target video stream. For example, when the target monitoring platform presents the target video stream through a display device, the corresponding user can perform augmented reality operations during the video presentation process, and the augmented reality operation is used to superimpose virtual information during the video presentation process, such as superimposing labels in the video frames, corresponding labels such as point labels, line labels, surface labels, gun camera labels, ball camera labels, building labels, place labels and alarm labels. One or more combinations thereof. In some embodiments, the enabling setting of the augmented reality operation function can be further performed. If the channel corresponding to the target video stream enables the augmented reality operation function, the user can perform augmented reality operations on the presented target video stream. The target monitoring platform determines the user's operating position as the image position of the augmented reality information, and superimposes and presents the corresponding augmented reality information at the corresponding image position. In some cases, the target monitoring platform may also continue to overlay the label in subsequent video frames based on the currently overlaid augmented reality information, such as overlaying the corresponding augmented reality information at the same image location, or calculating the spatial location of the augmented reality information and, after subsequently adjusting the camera angle, overlaying and presenting the augmented reality information at the corresponding image location in the video frame based on the same spatial location. For example, in some embodiments, the target camera device includes a gun camera device; wherein, in step S102, a user's augmented reality operation regarding the image location of a video frame in the target video stream is obtained, and the corresponding augmented reality information is overlaid and presented at the corresponding image location in some / all video frames of the target video stream. For example, when only a scene needs to be monitored, a gun camera can be used. The corresponding position and posture of the video stream captured by the gun camera is generally fixed, and the target monitoring platform cannot automatically obtain the corresponding position and posture information of the gun camera device. Therefore, for a gun-style camera, the operation position of the user's augmented reality operation in the video frame can be directly used as the image position of the augmented reality operation, and based on this image position, augmented reality information can be superimposed and a subsequent augmented reality video stream can be generated, such as superimposing augmented reality information at this image position in subsequent video frames. In some embodiments, the target monitoring platform supports tag search. For example, the target monitoring platform presents the target video stream and virtual tags superimposed on the target video stream through a display device. Based on the user's search operation, the platform can search for virtual tags within the visual range of the current target video stream.In some embodiments, the target monitoring platform supports certain business settings. For example, the user can filter by the type of virtual tag, and after filtering, the target monitoring platform only displays the selected type of virtual tag on the video stream. For example, the user can use the carousel setting to play the video stream collected by the target monitoring platform through the corresponding shared channel of the lower-level video platform / camera device and the augmented reality information superimposed on the video stream. For example, the user can use the carousel setting to display the relevant information of the augmented reality information corresponding to the target video stream. For example, the display content includes any one or more of the following: text description, ordinary picture, panoramic picture, local video, monitoring channel, PDF document, location, name, etc. Preferably, the relevant information of the augmented reality information can also be set with certain animation effects, for example, display in sequence, display duration, etc.

[0038] In step S103, a corresponding augmented reality video stream is generated based on the augmented reality information and the target video stream, and the augmented reality video stream is pushed to the other video platforms through the channel corresponding to the target camera device shared by the target monitoring platform to other video platforms. For example, the video platform / camera device (as a lower-level video platform) is registered to the target monitoring platform through the national standard cascade, and the corresponding video platform / camera device can set all / part of the camera device's channels to be shared, thereby pushing one or more video streams collected by the corresponding shared channel to the target monitoring platform. When the target monitoring platform presents the target video stream collected by the shared channel, the corresponding user can perform augmented reality operations during the video presentation process. After determining the corresponding augmented reality information, the target monitoring platform can generate the corresponding augmented reality video stream. The target monitoring platform is registered to the other video platform (as the upper-level video platform) through the national standard cascade, and the target monitoring platform can choose to set the channel corresponding to the target video stream to be shared (the shared channel is the same channel as the channel shared by the aforementioned lower-level video platform to the target monitoring platform), thereby pushing the augmented reality video stream to other video platforms. The augmented reality video stream may be a merged video stream comprising a target video stream, augmented reality information in the target video stream, and position information of the augmented reality information (e.g., a stored image position or a spatial position of an image position calculated by a user, etc.), or a new video stream generated by superimposing the augmented reality information onto the target video stream. In some embodiments, the generation of a corresponding augmented reality video stream based on the augmented reality information and the target video stream includes: fusing the augmented reality information with some / all video frames to generate a fused video frame, and replacing some / all video frames with the fused video frame to generate a corresponding augmented reality video stream. For example, while presenting the target video stream and superimposing the augmented reality information on the target video stream, the target monitoring platform fuses the augmented reality information into the target video stream based on the corresponding image position to generate a new fused video stream, and uses the fused video stream as the corresponding augmented reality video stream. The target video stream may be the original video stream captured by the camera device, or a fused video stream generated by fusing the augmented reality information in other lower-level video platforms.

[0039] In some embodiments, the method further includes step S106 (not shown), in which the platform operation of the user of the target monitoring platform regarding the one or more video platforms is obtained, and the cascade relationship between the one or more video platforms and the target monitoring platform is adjusted based on the platform operation. For example, the target monitoring platform is one of the multiple cascaded platforms, and the target monitoring platform can perform corresponding platform operations on its cascaded upper-level video platform and lower-level video platform, thereby adjusting the cascade relationship between the target monitoring platform and one or more video platforms based on the corresponding platform operations. Specifically, a platform operation page for one or more cascaded video platforms is provided for the corresponding target monitoring platform, and platform-related information of the one or more video platforms cascaded by the target monitoring platform is presented on the platform operation page, such as platform identification information, platform configuration information (such as SIP server number, SIP server IP address, SIP server port, etc.) and shared channel identification information, etc. In some embodiments, the platform operation includes at least one of the following: deactivating a platform; editing a platform; deleting a platform; updating a platform. For example, the platform operation includes deactivating the platform. The target monitoring platform can deactivate the cascade between the selected video platform and the target monitoring platform based on the user's deactivation operation on a certain video platform in the platform operation page, suspend all shared channels between the video platform and the target monitoring platform, etc. For example, the platform operation includes editing the platform, such as editing the platform configuration information of the video platform, editing the platform identification information, platform configuration information or shared channels (such as adding or deleting shared channels, etc.) of the video platform, thereby adjusting the relevant content in the cascade relationship between the video platform and the target monitoring platform based on the editing operation. For example, the platform operation includes deleting the platform. The target monitoring platform can delete the selected video platform based on the user's deletion operation on a certain video platform in the platform operation page, and remove the cascade relationship of the corresponding video platform from the corresponding list page. The platform operation includes updating the platform. The updating platform is used to update the platform-related information of the video platform cascaded by the target monitoring platform, such as updating all currently cascaded video platforms in real time and presenting them on the platform operation page.

[0040] In some embodiments, the method further includes step S107 (not shown), in which camera parameter information corresponding to the target video stream is obtained; wherein, in step S102, a user's augmented reality operation on the target video stream is obtained, and corresponding augmented reality information is superimposed on the corresponding image positions of some / all video frames of the target video stream based on the camera parameter information. For example, the target camera device includes a PTZ (Pan / Tilt / Zoom) camera device, and the computer device can obtain the camera parameter information corresponding to the video stream while obtaining the target video stream, such as by obtaining the camera parameter information of the PTZ camera device through a corresponding SDK based on direct communication with the PTZ camera device, or by performing OCR recognition on the video frames of the target video stream based on the video stream, thereby identifying the camera parameter information of the corresponding video frames in the video stream. After the target monitoring platform obtains the corresponding camera parameter information, it can overlay the aforementioned augmented reality information on all / part of the video frames (such as subsequent video frames) of the video stream based on the camera parameter information. For example, the image position corresponding to the augmented reality information in the subsequent video frame is calculated based on the camera parameter information of the subsequent video frame and the spatial position of the augmented reality information (calculated and determined based on the image position of the augmented reality operation in the video frame and the camera parameter information corresponding to the video frame), thereby overlaying and presenting the augmented reality information at the corresponding image position. For example, the camera parameter information includes the Pan / Tilt / Zoom values ​​corresponding to the video frames captured by the PTZ camera device (which can be each video frame or the video frames filtered according to a preset rule). For another example, the camera parameter information includes internal and / or external parameter information corresponding to the video frames captured by the camera device. For another example, the camera parameter information includes information used to calculate the corresponding internal and / or external parameters, specifically, Pan / Tilt / Zoom information or Pan / Tilt / Zoom information and camera position information used to calculate the corresponding internal and / or external parameters. Specifically, since the position of the PTZ camera is fixed, the computer device can read the corresponding PTZ camera's location in the geographic system, or obtain the corresponding location through a communication connection with the PTZ camera. In some embodiments, obtaining the camera parameter information corresponding to the target video stream includes performing optical character recognition (OCR) on key frames in the target video stream to determine the corresponding camera parameter information. In some embodiments, the camera parameter information is superimposed (e.g., as a watermark) on one or more key frames of the target video stream, where the key frames can be all video frames or video frames filtered according to preset rules.After the target monitoring platform acquires the target video stream, it can perform OCR (Optical Character Recognition) on the keyframes to identify the corresponding PTZ camera's camera parameter information, such as the values ​​of Pan, Tilt, and Zoom parameters. The recognition result information includes the PTZ camera's camera parameter information in the corresponding keyframe. The computer device can determine that the recognition result information includes the PTZ camera's camera parameter information. In some cases, the recognition result information also includes camera parameter identification information for the PTZ camera, such as letters such as P, T, and Z that indicate the corresponding parameter. After acquiring the corresponding recognition result information, the computing device can determine the camera parameter information for each keyframe based on the recognition result information for each keyframe, such as identifying the values ​​of Pan, Tilt, and Zoom parameters based on a preset order. For example, when the recognition result information also includes parameter identification information for each camera parameter, the corresponding parameter value can be determined based on the parameter values ​​near each camera parameter identification information, thereby acquiring the camera parameter information for the keyframe. After the computing device determines the camera parameter information for each key frame, it can further calculate the augmented reality information overlay position based on the camera parameter information. Due to the different manufacturers of the camera devices, the form in which the camera parameter information is presented in the video frame when the corresponding overlay setting information is overlaid may also vary, such as different overlay positions, different overlay fonts, different parameter formats, or different overlay colors. In order to improve recognition accuracy, we can set different OCR recognition algorithms for camera devices from different manufacturers. Here, the different OCR recognition algorithms can refer to different recognition algorithm models, or different algorithm branches within the same recognition algorithm model. Generally, the OCR recognition algorithm of a current PTZ camera device can be selected and determined based on user operation, or based on system recognition, such as determining the corresponding camera manufacturer identification information by performing bitstream analysis on the video stream, and calling the corresponding OCR recognition algorithm based on the camera manufacturer identification information, or determining the corresponding camera manufacturer identification information based on user input, and calling the corresponding OCR recognition algorithm based on the camera manufacturer identification information, etc.

[0041] In some embodiments, the method further includes step S108 (not shown), in which a user selection operation on the superimposed augmented reality information is obtained; one or more other video streams are determined based on the spatial location of the augmented reality information, wherein the spatial location is within the acquisition range of the one or more other video streams; and at least one of the one or more other video streams is presented. For example, while the target monitoring platform presents the target video stream and augmented reality information, it can also call other video streams based on the user's selection operation on the augmented reality information. The selected augmented reality information can be an augmented reality tag added by the user in the current target monitoring platform in the video stream, or an augmented reality tag added and integrated into the target video stream by other video platforms. The target monitoring platform can obtain the spatial location of the augmented reality information (such as reading it from a storage space, or calculating and determining it based on the image location of the augmented reality information added to the video frame and the corresponding camera parameter information of the video frame, etc.), and determine one or more other video streams nearby based on the spatial location, wherein the one or other video streams are video streams pushed by the target monitoring platform based on other camera devices or lower-level video platforms through a shared channel. Among them, one or more video streams can be determined based on the position difference between the camera position and the spatial position of the corresponding camera device. If the corresponding position difference is less than or equal to the camera position difference threshold, it is determined that the spatial position is within the acquisition range of the corresponding camera device, and the video stream collected by it is determined to be one of one or more other video streams. In some cases, in addition to matching based on the camera position, the target monitoring platform also determines the corresponding other video streams based on the camera angle information of the corresponding camera device. If the spatial position is within the camera angle of a certain camera device, the video stream collected by it is determined to be one of one or more other video streams. After the target monitoring platform obtains one or more other video streams, it can synchronously present the target video stream and the corresponding one or more other video streams, or present the target video stream and any one or more of the one or more other video streams, etc.

[0042] In some embodiments, the method of obtaining the user's augmented reality operation on the target video stream and superimposing the corresponding augmented reality information in the corresponding image position of some / all video frames of the target video stream according to the camera parameter information includes: obtaining the stored / added augmented reality information and the spatial position of the augmented reality information; determining the image position of the augmented reality information in the target video stream according to the camera parameter information and the spatial position of the augmented reality information, and superimposing the corresponding augmented reality information in the corresponding image position of some / all video frames of the target video stream. For example, the target monitoring platform can filter other video streams in the video stream obtained by the target monitoring platform based on the augmented reality information added by the user, and can also superimpose and present other augmented reality information in the currently obtained target video stream, wherein the other augmented reality information can include the augmented reality information added in other video streams. Specifically, the target monitoring platform can obtain the augmented reality tags superimposed in other video streams, and at the same time obtain the spatial position of the corresponding augmented reality tags. When the spatial position is within the camera angle of the camera device corresponding to the target video stream, the target monitoring platform obtains the camera parameter information of the target video stream, and can calculate the image position of the augmented reality tag in the target video stream based on the camera parameter information and the spatial position of the augmented reality tag, thereby superimposing the augmented reality tag in the target video stream, such as superimposing the corresponding augmented reality information in the corresponding image position of all / part of the video frames of the target video stream (such as video frames containing the corresponding spatial position, etc.).

[0043] The above mainly introduces various embodiments of a method for transmitting an augmented reality video stream according to the present application by way of examples. In addition, the present application also provides specific devices capable of implementing the above embodiments.

[0044] FIG2 shows a computer device 100 for transmitting an augmented reality video stream according to one aspect of the present application, specifically comprising a module 11 101, a module 12 102, and a module 13 103. Module 11 101 acquires and presents a target video stream captured in real time by a corresponding target camera device through a shared channel corresponding to the target monitoring platform and the target camera device; module 12 102 acquires a user's augmented reality operation on the target video stream, and overlays corresponding augmented reality information on part / all of the video frames of the target video stream; module 13 103 generates a corresponding augmented reality video stream based on the augmented reality information and the target video stream, and pushes the augmented reality video stream to the other video platforms through the channel corresponding to the target camera device shared by the target monitoring platform to the other video platforms.

[0045] Here, the specific implementations corresponding to the module 101, the module 102 and the module 103 shown in Figure 2 are the same or similar to the embodiments of the aforementioned steps S101, S102 and S103, and are therefore not repeated here and are included herein by reference.

[0046] In some embodiments, the device further includes a quad module (not shown) for cascading one or more video platforms with the target monitoring platform, wherein at least one channel of the video platform and the target monitoring platform is set as a shared channel.

[0047] In some implementations, the target camera device or the video platform corresponding to the target camera device is registered to the target monitoring platform through national standard cascading.

[0048] In some embodiments, the device further includes a module (not shown) for simultaneously presenting one or more video streams pushed by one or more video platforms through corresponding shared channels through a display device of the target monitoring platform.

[0049] In some embodiments, the target camera device includes a gun camera device; wherein, module 102 is used to obtain the user's augmented reality operation on the image position of the video frame in the target video stream, and superimpose corresponding augmented reality information at the corresponding image position in part / all of the video frames of the target video stream.

[0050] In some embodiments, generating a corresponding augmented reality video stream based on the augmented reality information and the target video stream includes: fusing the augmented reality information with part / all of the video frames to generate fused video frames, and replacing part / all of the video frames with the fused video frames to generate the corresponding augmented reality video stream.

[0051] In some embodiments, the device further includes a module (not shown) for obtaining platform operations of the user of the target monitoring platform regarding the one or more video platforms, and adjusting the cascade relationship between the one or more video platforms and the target monitoring platform based on the platform operations. In some embodiments, the platform operations include but are not limited to: deactivating a platform; editing a platform; deleting a platform; and updating a platform. For example, the platform operations include deactivating a platform, and the target monitoring platform can deactivate the cascade between the selected video platform and the target monitoring platform based on the user's deactivation operation on a certain video platform in the platform operation page, suspend all shared channels between the video platform and the target monitoring platform, etc.

[0052] In some embodiments, the device further includes a module (not shown) configured to obtain camera parameter information corresponding to the target video stream; wherein module (102) is configured to obtain a user's augmented reality operation on the target video stream and, based on the camera parameter information, overlay corresponding augmented reality information on corresponding image positions of some / all video frames of the target video stream. In some embodiments, obtaining the camera parameter information corresponding to the target video stream includes performing OCR on key frames in the target video stream to determine the corresponding camera parameter information.

[0053] In some embodiments, the device further includes a module (not shown) for obtaining a user's selection operation on the superimposed augmented reality information; determining one or more other video streams based on the spatial position of the augmented reality information, wherein the spatial position is within the acquisition range of the one or more other video streams; and presenting at least one of the one or more other video streams.

[0054] In some embodiments, obtaining the user's augmented reality operation on the target video stream and superimposing the corresponding augmented reality information in the corresponding image positions of some / all video frames of the target video stream according to the camera parameter information includes: obtaining the stored / added augmented reality information and the spatial position of the augmented reality information; determining the image position of the augmented reality information in the target video stream according to the camera parameter information and the spatial position of the augmented reality information, and superimposing the corresponding augmented reality information in the corresponding image positions of some / all video frames of the target video stream.

[0055] Here, the specific implementations corresponding to the four modules to the eight modules are the same as or similar to the embodiments of the aforementioned steps S104 to S108, and are therefore not described in detail and are included herein by reference.

[0056] In addition to the methods and devices described in the above embodiments, the present application also provides a computer-readable storage medium, which stores computer code. When the computer code is executed, the method described in any of the above items is executed.

[0057] The present application also provides a computer program product. When the computer program product is executed by a computer device, the method described in any one of the preceding items is executed.

[0058] The present application also provides a computer device, comprising:

[0059] one or more processors;

[0060] a memory for storing one or more computer programs;

[0061] When the one or more computer programs are executed by the one or more processors, the one or more processors are caused to implement the method as described in any one of the preceding items.

[0062] FIG3 illustrates an exemplary system that may be used to implement various embodiments described herein;

[0063] As shown in FIG3 , in some embodiments, system 300 can function as any of the aforementioned devices in the various embodiments. In some embodiments, system 300 may include one or more computer-readable media (e.g., system memory or NVM / storage device 320) having instructions and one or more processors (e.g., processor(s) 305) coupled to the one or more computer-readable media and configured to execute the instructions to implement the modules and thereby perform the actions described herein.

[0064] For one embodiment, system control module 310 may include any suitable interface controller to provide any suitable interface to at least one of processor(s) 305 and / or any suitable device or component in communication with system control module 310 .

[0065] The system control module 310 may include a memory controller module 330 to provide an interface to the system memory 315. The memory controller module 330 may be a hardware module, a software module, and / or a firmware module.

[0066] System memory 315 can be used, for example, to load and store data and / or instructions for system 300. For one embodiment, system memory 315 can include any suitable volatile memory, such as a suitable DRAM. In some embodiments, system memory 315 can include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0067] For one embodiment, system control module 310 may include one or more input / output (I / O) controllers to provide interfaces to NVM / storage device 320 and communication interface(s) 325 .

[0068] For example, NVM / storage 320 may be used to store data and / or instructions. NVM / storage 320 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0069] NVM / storage device 320 may include storage resources that are physically part of the device on which system 300 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 320 may be accessed over a network via communication interface(s) 325.

[0070] Communication interface(s) 325 may provide an interface for system 300 to communicate over one or more networks and / or with any other suitable devices. System 300 may wirelessly communicate with one or more components of a wireless network in accordance with any of one or more wireless network standards and / or protocols.

[0071] For one embodiment, at least one of the processor(s) 305 may be packaged together with the logic of one or more controllers of the system control module 310 (e.g., the memory controller module 330). For one embodiment, at least one of the processor(s) 305 may be packaged together with the logic of one or more controllers of the system control module 310 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 305 may be integrated on the same die with the logic of one or more controllers of the system control module 310. For one embodiment, at least one of the processor(s) 305 may be integrated on the same die with the logic of one or more controllers of the system control module 310 to form a system-on-chip (SoC).

[0072] In various embodiments, system 300 may be, but is not limited to, a server, a workstation, a desktop computer device, or a mobile computer device (e.g., a laptop computer device, a handheld computer device, a tablet computer, a netbook, etc.). In various embodiments, system 300 may have more or fewer components and / or a different architecture. For example, in some embodiments, system 300 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0073] It should be noted that the application can be implemented in software and / or a combination of software and hardware, for example, can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the application can be executed by a processor to realize the steps or functions described above. Similarly, the software program of the application (including relevant data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the application can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0074] In addition, a part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present application through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes but is not limited to a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0075] Communication media include media by which communication signals containing, for example, computer-readable instructions, data structures, program modules, or other data are transmitted from one system to another. Communication media may include guided transmission media such as cables and wires (e.g., fiber optic, coaxial, etc.) and wireless (unguided transmission) media capable of propagating energy waves, such as acoustic, electromagnetic, RF, microwave, and infrared. Computer-readable instructions, data structures, program modules, or other data may be embodied as, for example, a modulated data signal in a wireless medium such as a carrier wave or similar mechanism such as that embodied as part of spread spectrum technology. The term "modulated data signal" refers to a signal that has one or more of its characteristics changed or set in such a manner as to encode information in the signal. Modulation may be analog, digital, or a hybrid modulation technique.

[0076] By way of example and not limitation, computer-readable storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. For example, computer-readable storage media include, but are not limited to, volatile memory, such as random access memory (RAM, DRAM, SRAM); and non-volatile memory, such as flash memory, various read-only memories (ROM, PROM, EPROM, EEPROM), magnetic and ferromagnetic / ferroelectric memories (MRAM, FeRAM); and magnetic and optical storage devices (hard disks, magnetic tapes, CDs, DVDs); or other media now known or later developed that can store computer-readable information / data for use by a computer system.

[0077] Here, according to one embodiment of the present application, a device is included, which includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein, when the computer program instructions are executed by the processor, the device is triggered to run the methods and / or technical solutions based on the aforementioned multiple embodiments of the present application.

[0078] It is obvious to those skilled in the art that the present application is not limited to the details of the above-mentioned exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. Words such as first and second are used to indicate names and do not indicate any particular order.

Claims

1. A method for transmitting an augmented reality video stream, wherein: The method includes: Obtain and present the target video stream collected in real time by the corresponding target camera device through the shared channel corresponding to the target monitoring platform and the target camera device; Acquire an augmented reality operation of the user on the target video stream, and overlay corresponding augmented reality information on part / all of the video frames of the target video stream; A corresponding augmented reality video stream is generated according to the augmented reality information and the target video stream, and the augmented reality video stream is pushed to the other video platforms through the channel corresponding to the target camera device shared by the target monitoring platform to the other video platforms.

2. The method according to claim 1, wherein: The method further comprises: One or more video platforms are cascaded with the target monitoring platform, wherein at least one channel of the video platform and the target monitoring platform is set as a shared channel.

3. The method according to claim 2, wherein: The target camera device or the video platform corresponding to the target camera device is registered to the target monitoring platform through national standard cascading.

4. The method according to claim 2 or 3, wherein: The method further comprises: One or more video streams pushed by one or more video platforms through corresponding shared channels are simultaneously presented through the display device of the target monitoring platform.

5. The method according to claim 2 or 3, wherein: The method further comprises: The platform operation of the user of the target monitoring platform on the one or more video platforms is obtained, and the cascade relationship between the one or more video platforms and the target monitoring platform is adjusted based on the platform operation.

6. The method according to claim 5, wherein: The platform operation includes at least one of the following: Deactivate the Platform; Editing platform; Delete the platform; Update the platform.

7. The method according to claim 1, wherein: The method further comprises: Obtaining the camera parameter information corresponding to the target video stream; Wherein, the obtaining of the user's augmented reality operation on the target video stream is performed in part of the target video stream. The corresponding augmented reality information is superimposed on each / all video frames, including: The user's augmented reality operation on the target video stream is obtained, and corresponding augmented reality information is superimposed in corresponding image positions of part / all of the video frames of the target video stream according to the camera parameter information.

8. The method according to claim 7, wherein: The obtaining of the camera parameter information corresponding to the target video stream includes: Perform OCR recognition on key frames in the target video stream to determine corresponding camera parameter information.

9. The method according to claim 7, wherein: The method further comprises: Acquiring a user's selection operation on the superimposed augmented reality information; Determine one or more other video streams based on the spatial position of the augmented reality information, wherein the spatial position is within a collection range of the one or more other video streams; At least one of the one or more other video streams is presented.

10. The method according to claim 7 or 8, wherein: The obtaining of the user's augmented reality operation on the target video stream, and superimposing corresponding augmented reality information in corresponding image positions of some / all video frames of the target video stream according to the camera parameter information, includes: Acquire the stored / added augmented reality information and the spatial position of the augmented reality information; The image position of the augmented reality information in the target video stream is determined according to the camera parameter information and the spatial position of the augmented reality information, and the corresponding augmented reality information is superimposed in the corresponding image position of part / all of the video frames of the target video stream.

11. The method according to claim 1, wherein: The target camera device includes a gun camera device; wherein the acquiring of the user's augmented reality operation on the target video stream and superimposing corresponding augmented reality information on part / all of the video frames of the target video stream includes: The user's augmented reality operation on the image position of the video frame in the target video stream is obtained, and the corresponding augmented reality information is superimposed and presented at the corresponding image position in part / all of the video frames of the target video stream.

12. The method according to claim 1, wherein: The generating a corresponding augmented reality video stream according to the augmented reality information and the target video stream comprises: The augmented reality information is fused with the part / all of the video frames to generate a video frame to be fused, and the part / all of the video frames are replaced with the fused video frames to generate a corresponding augmented reality video stream.

13. A device for transmitting an augmented reality video stream, wherein: The equipment includes: A module is used to obtain and present the target monitoring platform and the target camera device through the shared channel corresponding to the target monitoring platform and the target camera device. Now corresponds to the target video stream collected in real time by the target camera device; Module one or two, used to obtain the user's augmented reality operation on the target video stream, and superimpose corresponding augmented reality information on part / all of the video frames of the target video stream; A module 13 is used to generate a corresponding augmented reality video stream according to the augmented reality information and the target video stream, and push the augmented reality video stream to the other video platform through the channel corresponding to the target camera device shared by the target monitoring platform to other video platforms.

14. A computer device, wherein: The equipment includes: Processor; and A memory arranged to store computer executable instructions which, when executed, cause the processor to perform the steps of the method as claimed in any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: The computer program / instructions, when executed, cause the system to perform the steps of the method as claimed in any one of claims 1 to 12.

16. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images