Multimedia data stream pushing method, terminal, system and storage medium

Abstract

The invention relates to the technical field of stream pushing, and provides a multimedia data stream pushing method, a terminal, a system and a storage medium, and the method comprises the steps: controlling AR equipment connected with the terminal to carry out multimedia data collection in response to a setting operation of live broadcast through the AR equipment; receiving a multimedia data stream transmitted by the AR equipment, and caching the multimedia data stream; and reading the multimedia data stream, and pushing the multimedia data stream to a streaming media server so as to perform live broadcast based on the streaming media server. According to the embodiment of the invention, multimedia data stream pushing based on the AR equipment is realized, so that live broadcast through the AR equipment is realized.

Description

Technical Field

[0001] This application relates to the field of streaming technology, and in particular to a multimedia data streaming method, terminal, system and storage medium. Background Technology

[0002] With technological advancements, live video streaming has become widely used in e-commerce, education, and other fields. Currently, it typically relies on third-party live streaming platforms to provide cross-platform open SDKs (Software Development Kits) and APIs (Application Programming Interfaces) for live video streaming on terminals such as mobile phones, enabling users to conduct live video streaming through these devices. However, for AR (Augmented Reality) devices, such as AR glasses, the corresponding APIs for live video streaming have not yet been opened or configured, and third-party live streaming platforms have not provided corresponding SDKs, making it impossible to implement live streaming functionality using AR devices. Summary of the Invention

[0003] This application provides a multimedia data streaming method, terminal, system, and storage medium, which aims to enable multimedia data streaming based on AR devices, thereby achieving live streaming through AR devices.

[0004] In a first aspect, this application provides a multimedia data streaming method based on an AR device, the method comprising: In response to the setup operation of live streaming using AR devices, control the AR devices that establish a connection with the terminal to collect multimedia data; Receive the multimedia data stream transmitted by the AR device and cache the multimedia data stream; The multimedia data stream is read and pushed to the streaming media server for live streaming.

[0005] Secondly, this application also provides a terminal, the terminal including a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the steps of the multimedia data streaming method based on the AR device as described above.

[0006] Thirdly, this application also provides a streaming system, which includes an AR device and a terminal as described above, wherein the terminal establishes a connection with the AR device.

[0007] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described multimedia data streaming method based on an AR device.

[0008] This application discloses a multimedia data streaming method, device, system, and storage medium based on AR devices. In response to the setup operation of live streaming using AR devices, the method controls the AR device connected to the terminal to collect multimedia data, receives and caches the multimedia data stream transmitted by the AR device, reads the cached multimedia data stream, and pushes the multimedia data stream to a streaming media server for live streaming. This realizes multimedia data streaming based on AR devices and live streaming of content collected by AR devices, solving the problem of not being able to use AR devices for live streaming. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating a multimedia data streaming method based on an AR device, provided as an embodiment of this application; Figure 2 This is a schematic diagram of a UI interface display provided in an embodiment of this application; Figure 3 A flowchart illustrating another multimedia data streaming method based on an AR device provided in this application embodiment; Figure 4 A timing diagram for registering a virtual device is provided as an embodiment of this application; Figure 5 This application provides an embodiment of a system architecture diagram consisting of AR glasses and a mobile phone. Figure 6 A timing diagram of a streaming process provided in an embodiment of this application; Figure 7 This application provides a schematic diagram of a data read / write process using a double buffer. Figure 8 A schematic block diagram of a terminal provided for an embodiment of this application; Figure 9 This is a schematic block diagram of a streaming system provided in an embodiment of this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0013] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0014] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0015] Embodiments of this application provide a multimedia data streaming method, terminal, system, and storage medium for implementing multimedia data streaming based on AR devices, thereby enabling live streaming via AR devices. The AR devices include, but are not limited to, AR glasses.

[0016] Please see Figure 1 , Figure 1 This is a flowchart illustrating a multimedia data streaming method based on an AR device according to an embodiment of this application. This method can be applied to terminals, including but not limited to electronic products such as mobile phones, tablets, and live streaming cameras. This application does not limit the application scenarios of this method.

[0017] like Figure 1 As shown, the multimedia data streaming method based on AR devices specifically includes steps S101 to S103.

[0018] S101, In response to the setup operation of using AR device for live streaming, control the AR device that has established a connection with the terminal to perform multimedia data acquisition.

[0019] For example, a streaming media client is installed on the terminal, which includes, but is not limited to, a third-party live streaming app client. After the user opens the streaming media client, when the user wants to conduct live streaming based on an AR device, the user can perform the corresponding settings operation for live streaming using the AR device. For example, when opening a third-party live streaming app, the corresponding UI controls for live streaming based on the AR device are displayed on the UI (User Interface), where the UI controls include, but are not limited to, selection controls. The user performs the settings operation for live streaming using the AR device by clicking on the selection control. When the terminal detects the settings operation for live streaming using the AR device, it controls the AR device connected to the terminal to open the camera device (such as a camera) to collect multimedia data. The multimedia data includes, but is not limited to, audio data and video data.

[0020] For example, taking AR glasses as the AR device and a mobile phone as the terminal, such as... Figure 2 As shown, after a third-party live streaming app is opened on a mobile phone, a selection control such as "Switch to AR Glasses" is displayed in the UI interface. When a user wants to use AR glasses for live streaming, the user clicks on the selection control to perform the settings operation of using AR glasses for live streaming. When the mobile phone detects the user's settings operation, it controls the AR glasses that have established a connection with the mobile phone to open the camera to collect video data.

[0021] In some embodiments, before controlling the AR device that establishes a connection with the terminal to perform multimedia data acquisition, the method includes: establishing a point-to-point connection between the terminal and the AR device based on a target communication protocol, the target communication protocol including the Wi-Fi Direct protocol.

[0022] The target communication protocol can be Wi-Fi Direct, or other different communication protocols, which are not specifically limited in this application.

[0023] For example, the terminal is configured with a Network module. The Network module is a software module in the terminal responsible for network-related functions and can provide protocol support, including but not limited to TCP (Transmission Control Protocol), UDP (User Datagram Protocol), and Wi-Fi Direct protocol. Through this Network module, the terminal and the AR device establish a point-to-point connection based on the target communication protocol (such as Wi-Fi Direct protocol), bypassing the router and thus reducing latency.

[0024] For example, taking AR glasses as the AR device and a mobile phone as the terminal, the target communication protocol is set to Wi-Fi Direct protocol, and information such as IP address (e.g., IP: 192.168.xx) and port (e.g., Port: 1234) is bound. The AR glasses, as the client, actively connects through the Wi-Fi Direct protocol. When the mobile phone detects the AR glasses connecting, it accepts the connection, thereby completing the point-to-point connection between the mobile phone and the AR glasses.

[0025] S102. Receive the multimedia data stream transmitted by the AR device and buffer the multimedia data stream.

[0026] For example, the terminal and the AR device each create a Socket. A Socket is a programming interface in computer networks used to enable bidirectional communication between processes on different hosts (such as between a terminal and an AR device). It can be seen as an endpoint for communication between processes on different hosts, providing a unified data exchange mechanism for the application layer. The AR device transmits the acquired multimedia data stream to the terminal through the Socket. The terminal receives the multimedia data stream transmitted by the AR device through the Network module and buffers the multimedia data stream after receiving it. For example, the multimedia data stream transmitted by the AR device is uncompressed raw data in YUV420 format. Compared to transmitting compressed data, although the bandwidth consumption is higher, it ensures low latency in data transmission.

[0027] In some embodiments, such as Figure 3 As shown, step S104 precedes step S102, and step S102 includes sub-step S1021.

[0028] S104. Create a physical memory pool based on the memory manager; S1021. Write the multimedia data stream to the physical memory pool.

[0029] For example, the terminal system's HAL (Hardware Abstraction Layer) creates a physical memory pool (such as real_buf) through ION (Inter-Process Memory Manager). After receiving the multimedia data stream transmitted from the AR device, the multimedia data stream is written into the physical memory pool. This physical memory pool serves as shared memory, meaning that writing the multimedia data stream into shared memory avoids multiple data copies.

[0030] For example, by using DMA (Direct Memory Access) to write multimedia data directly to the physical memory pool, DMA bypasses the CPU (Central Processing Unit) cache, thus reducing CPU usage.

[0031] It should be noted that step S104 can be executed before or after step S101, and no specific restrictions are imposed in this application.

[0032] S103. Read the multimedia data stream and push the multimedia data stream to the streaming media server for live streaming based on the streaming media server.

[0033] For example, a multimedia data stream is read from a physical memory pool without copying the data, and then pushed to a streaming media server. For instance, if YUV frames are read, they are pushed to the streaming media server. This streaming media server includes, but is not limited to, a third-party live streaming app server. By distributing the multimedia data stream through the streaming media server, live streaming of the content captured by the AR device can be achieved.

[0034] In some embodiments, the multimedia data streaming method based on AR devices further includes: registering virtual devices, generating virtual device nodes, and mapping a physical memory pool through memory mapping operations; In step S103, reading the multimedia data stream includes: reading the multimedia data stream written to the physical memory pool based on the virtual device node.

[0035] For example, a virtual device (such as a virtual camera) is declared to the terminal system through the `hw_device_t` structure, and virtual device registration is performed, including configuring virtual device information and generating the corresponding virtual device node. The virtual device information includes, but is not limited to, the virtual device type and virtual device ID (Identity document). For instance, taking the registration of a virtual camera as an example, the virtual Camera HAL is registered to the terminal system. When the virtual Camera HAL starts, it extends `CameraDeviceFactory` and adds the virtual camera ID as `virtual_cam_0`. `CameraDeviceFactory` is an abstract factory class in the factory pattern used to create camera devices; its core function is to dynamically generate instances of specific camera devices. Figure 4As shown, the Virtual Camera HAL calls `addCameraDevice(Virtual_cam_0)` during initialization to register a virtual camera with the ID "Virtual_cam_0" with the system. The CameraService requests `getCameraCharacteristics()` to obtain the characteristics of this virtual camera. The CameraService manages the camera device lifecycle and request scheduling, coordinating the interaction between the HAL and the application. In `getCameraCharacteristics()`, the Virtual Camera HAL declares and returns information such as the resolution, frame rate, and color gamut format supported by the virtual camera. For example, the virtual camera supports 1080P resolution, 30fps frame rate, and YUV420 format, resolving compatibility issues with third-party live streaming apps. By registering the virtual device, virtual device nodes are generated, such as the ` / dev / video1` node. The Virtual Camera HAL then uses `mmap` (memory mapping) to associate the physical memory pool and the virtual device nodes in the address space, establishing a mapping relationship between their address spaces. Based on this mapping relationship, the virtual device nodes are mapped to the physical memory pool.

[0036] Registered virtual devices are recognized as native devices by the terminal system. Taking Android as an example, Android has very strict permission control. Recognizing virtual devices as native devices can ensure the security, compatibility and controllability of data processing, avoid the problem of third-party live streaming apps not being able to recognize them, and ensure compatibility with various related applications.

[0037] CameraService enumerates available devices (enumerateCameras). Registered virtual devices are included in the list of available devices, and the list of devices is returned to third-party live streaming apps, including physical cameras and virtual cameras [real_cam_0, virtual_cam_0]).

[0038] For example, the terminal is pre-configured with a software interface associated with the AR device, such as setActiveSource(). When a user wants to conduct live streaming based on the AR device, the user can click on the UI control to trigger the software interface, which notifies CameraService to open the virtual device, such as switching the terminal's camera ID from the physical camera ID (e.g., real_cam_0) to the virtual camera ID (e.g., virtual_cam_0), thereby seamlessly switching the terminal's physical camera to the virtual camera. The switching time is very short, less than 100ms, achieving a seamless switching experience for the user.

[0039] For example, when switching from a terminal's physical camera to a virtual device, the writing of multimedia data streams can be paused, such as pausing the writing of a single frame. This allows for a seamless switching experience for the user, while pausing the writing process avoids resource contention and reduces CPU usage.

[0040] A data stream (configureStream) is created using a virtual Camera HAL to configure a stable data transmission channel between the virtual device and the third-party live streaming app. The virtual Camera HAL sends the multimedia data stream from the physical memory pool mapped by the Surface returned by the third-party live streaming app to the app, skipping the traditional v4l2 loopback "user space ↔ kernel space" data copy operation, thus reducing data latency and improving performance.

[0041] For example, taking AR glasses as the AR device and a mobile phone as the terminal, such as... Figure 5 and Figure 6 As shown, when a user wants to conduct live streaming using AR glasses, they can click on a UI control, such as the "Switch to AR Glasses" UI control. This triggers the `setActiveSource()` interface, notifying the `CameraService` to activate the virtual camera (e.g., `virtual_cam_0`). The `CameraService` then seamlessly switches the phone's physical camera to the virtual camera and establishes a Wi-Fi Direct point-to-point connection between the phone and the AR glasses via the `Network` module. Afterward, the `Network` module sends a real-time YUV frame data request to the AR glasses. The AR glasses directly transmit YUV frame data (e.g., YUV420 format video frames) to the phone, which receives the data through the `Network` module and writes it to the physical memory pool. The physical memory pool is mapped to virtual device nodes via `mmap` using the virtual `Camera HAL`. Instead of directly manipulating the physical memory pool, the `CameraService` injects the read YUV frame data into the third-party live streaming app, eliminating the need for data copying and achieving low-latency data injection.

[0042] In some embodiments, the physical memory pool includes a dual buffer, such as a first buffer and a second buffer, for writing and reading multimedia data streams. For example, writing a multimedia data stream to the physical memory pool includes: writing the multimedia data stream to the first buffer and alternating between the first and second buffers; reading the multimedia data stream includes: reading the multimedia data stream from the alternating second buffer.

[0043] The AR device acts as the producer, and the terminal as the consumer. The AR device and terminal alternately use a dual buffer. The AR device writes multimedia data streams to the first buffer, and then alternates between the first and second buffers, for example, by swapping the pointers of the first and second buffers. After the first and second buffers are alternated, the terminal can read the multimedia data stream from the alternated second buffer (which is also the first buffer before the alternate). This alternation between the AR device and terminal reduces collisions.

[0044] In some embodiments, before writing the multimedia data stream to the first buffer, the method includes: detecting whether the first buffer is currently idle; writing the multimedia data stream to the first buffer includes: if the first buffer is currently idle, then writing the multimedia data stream to the first buffer.

[0045] AR devices send multimedia data streams (such as YUV frames) to the terminal via Socket. The terminal checks whether the first buffer is currently free. If the first buffer is free, the multimedia data stream is written to the first buffer. Otherwise, if the first buffer is not free, such as being occupied by other threads, the multimedia data stream is discarded, thus avoiding contention.

[0046] In some embodiments, detecting whether the first buffer is currently free includes: detecting the current value of a preset counting parameter, the preset counting parameter being related to the thread associated with the first buffer; if the current value is a first value, then determining that the first buffer is currently free.

[0047] A preset counting parameter maps to the current thread status associated with the first buffer, such as whether a thread is currently using the first buffer. For example, the preset counting parameter `refcount` is set to a first value of 0. After the AR device sends a multimedia data stream to the terminal, the receiving thread checks the current value of `refcount`. If the current value of `refcount` is 0, it means the first buffer is currently unoccupied and idle. In this case, the first buffer is locked, and the multimedia data stream is written to it. If the current value of `refcount` is greater than 0, it means the first buffer is currently occupied. In this case, the multimedia data stream is discarded to avoid contention.

[0048] For example, still using AR glasses as the AR device and a mobile phone as the terminal, assuming the first buffer is BufferA and the second buffer is BufferB, such as... Figure 7As shown, the AR glasses send the current frame data (such as a YUV frame) to the phone, execute the AR glasses data writing thread (receiving thread), check the current value of refcount. If refcount = 0, then lock BufferA and write the current frame data into BufferA. If refcount > 0, then discard the current frame data.

[0049] In some embodiments, after writing the multimedia data stream to the first buffer, the method includes: updating a preset write flag parameter to a second value; and reading the multimedia data stream from the alternating second buffer, including: if the preset write flag parameter is the second value, then reading the multimedia data stream from the alternating second buffer.

[0050] For example, a preset write flag parameter w_flag is set, with the corresponding second value being 1. After writing the multimedia data stream to the first buffer, w_flag is set to 1. When w_flag=1, the multimedia data stream is read from the alternating second buffer. For example, as... Figure 7 As shown, after locking BufferA and writing the current frame data into BufferA, w_flag=1 is set to notify the virtual Camera HAL to read the current frame data from the alternating BufferB.

[0051] In some embodiments, after writing the multimedia data stream to the first buffer, the method includes: updating the current value in a first update manner, the first update manner including increasing a preset value; after reading the multimedia data stream from the alternating second buffer, the method includes: updating the current value in a second update manner, the second update manner including decreasing the preset value.

[0052] For example, a preset value of 1 is set, but this preset value can also be other values, which are not specifically limited in this application. After the multimedia data stream is successfully written to the first buffer, refcount is incremented by 1, changing the current value of refcount from 0 to 1. When refcount > 0, no new data is written to the first buffer until the written multimedia data stream has been read. The first and second buffers are alternated. After reading the multimedia data stream from the alternated second buffer, refcount is decremented by 1, changing the current value of refcount from 1 to 0. When refcount = 0, new multimedia data streams can continue to be written to the first buffer. For example, as shown... Figure 7As shown, setting w_flag to 1 notifies the virtual Camera HAL to read the current frame data from the alternating BufferB. After the current frame data is read, refcount is decremented by 1. By alternating the use of BufferA and BufferB as double buffers, combined with refcount and w_flag, mismatch between data writing and reading speeds is prevented, thus achieving synchronization and balancing data integrity and real-time performance.

[0053] In the above embodiments, in response to the setup operation of using an AR device for live streaming, the AR device connected to the terminal is controlled to collect multimedia data, receive and cache the multimedia data stream transmitted by the AR device, read the cached multimedia data stream, and push the multimedia data stream to a streaming media server for live streaming. This achieves live streaming of the content collected by the AR device, solving the problem of not being able to use AR devices for live streaming. Furthermore, compared to live streaming through mobile phones or other terminals, live streaming through AR devices can bring users a different visual experience and enhance the user experience.

[0054] Please see Figure 8 , Figure 8 This is a schematic block diagram of a terminal provided in an embodiment of this application. The terminal includes, but is not limited to, electronic products such as mobile phones, tablet computers, and live streaming cameras.

[0055] like Figure 8 As shown, the terminal 100 may include a processor 110 and a memory 120, wherein the processor 110 and the memory 120 are connected via a bus, such as an I2C (Inter-integrated Circuit) bus.

[0056] Specifically, the processor 110 can be a microcontroller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP), etc.

[0057] Specifically, the memory 120 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc. The memory 120 stores various computer programs for the processor 110 to execute.

[0058] The processor 110 is configured to run a computer program stored in the memory, and to perform the following steps when executing the computer program: In response to the setup operation of live streaming using AR devices, control the AR devices that establish a connection with the terminal to collect multimedia data; Receive the multimedia data stream transmitted by the AR device and cache the multimedia data stream; The multimedia data stream is read and pushed to the streaming media server for live streaming.

[0059] In some embodiments, before implementing the caching of the multimedia data stream, the processor 110 is configured to: Create a physical memory pool based on a memory manager; The caching of the multimedia data stream includes: The multimedia data stream is written to the physical memory pool.

[0060] In some embodiments, the processor 110 is further configured to implement: Perform virtual device registration, generate virtual device nodes, and map the physical memory pool through memory mapping operations; When implementing the reading of the multimedia data stream, the processor 110 is configured to: The multimedia data stream written to the physical memory pool is read based on the virtual device node.

[0061] In some embodiments, the physical memory pool includes a first buffer and a second buffer; when implementing the writing of the multimedia data stream into the physical memory pool, the processor 110 is configured to: The multimedia data stream is written into the first buffer, and the first buffer and the second buffer are alternately buffered. When implementing the reading of the multimedia data stream, the processor 110 is configured to: The multimedia data stream is read from the second buffer after the alternation.

[0062] In some embodiments, before writing the multimedia data stream to the first buffer, the processor 110 is configured to perform the following: Check if the first buffer is currently free; When the processor 110 implements the step of writing the multimedia data stream into the first buffer, it is configured to: If the first buffer is currently free, the multimedia data stream is written to the first buffer.

[0063] In some embodiments, when implementing the detection of whether the first buffer is currently free, the processor 110 is configured to: The current value of a preset count parameter is detected, the preset count parameter being related to the thread associated with the first buffer; If the current value is the first value, then the first buffer is determined to be currently free.

[0064] In some embodiments, after writing the multimedia data stream to the first buffer, the processor 110 is configured to: The current value is updated using a first update method, which includes adding a preset value. After implementing the reading of the multimedia data stream from the alternating second buffer, the processor 110 is configured to: The current value is updated using a second update method, which includes reducing the preset value.

[0065] In some embodiments, after writing the multimedia data stream to the first buffer, the processor 110 is configured to: Update the preset write flag parameter to the second value; When implementing the reading of the multimedia data stream from the alternating second buffer, the processor 110 is configured to: If the preset write flag parameter is the second value, then the multimedia data stream is read from the alternating second buffer.

[0066] In some embodiments, before the AR device that establishes a connection with the control terminal performs multimedia data acquisition, the processor 110 is configured to: A point-to-point connection is established between the terminal and the AR device based on a target communication protocol, which includes the Wi-Fi Direct protocol.

[0067] Terminal 100 can execute the multimedia data streaming method based on AR device provided in the relevant embodiments of this application. Therefore, it can achieve the beneficial effects that the multimedia data streaming method based on AR device provided in the relevant embodiments of this application can achieve. For details, please refer to the previous embodiments, which will not be repeated here.

[0068] Please see Figure 9 , Figure 9 This is a schematic block diagram of a streaming system provided in an embodiment of this application. Figure 9 As shown, the streaming system 1000 includes a terminal 100 and an AR device 200. The terminal 100 can be... Figure 8The terminal 100 and AR device 200 shown include, but are not limited to, AR glasses. Therefore, the streaming system 1000 can achieve the beneficial effects of the multimedia data streaming method based on AR devices provided in the embodiments of this application, as detailed in the preceding embodiments, and will not be repeated here.

[0069] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the multimedia data streaming method based on an AR device as described above.

[0070] The computer-readable storage medium can be an internal storage unit of the terminal or streaming system described in the foregoing embodiments, such as a hard disk or memory of the terminal or streaming system. Alternatively, the computer-readable storage medium can be an external storage device of the terminal or streaming system, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, etc., equipped on the terminal or streaming system.

[0071] Since the computer program stored in the storage medium can execute any of the multimedia data streaming methods based on AR devices provided in the embodiments of this application, the beneficial effects that any of the multimedia data streaming methods based on AR devices provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0072] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0073] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application.

Claims

1. A method for pushing multimedia data based on an AR device, characterized in that, The method comprises: In response to a live broadcast setting operation using an AR device, controlling the AR device connected with the terminal to collect multimedia data; Receiving a multimedia data stream transmitted by the AR device and buffering the multimedia data stream; Reading the multimedia data stream and pushing the multimedia data stream to a streaming media server for live broadcast based on the streaming media server.

2. The method of claim 1, wherein, Before the buffering of the multimedia data stream, the method comprises: Creating a physical memory pool based on a memory manager; The buffering of the multimedia data stream comprises: Writing the multimedia data stream into the physical memory pool.

3. The method of claim 2, wherein, The method further comprises: Registering a virtual device, generating a virtual device node, and mapping the physical memory pool through a memory mapping operation; The reading of the multimedia data stream comprises: Reading the multimedia data stream written in the physical memory pool based on the virtual device node.

4. The method of claim 2, wherein, The physical memory pool comprises a first buffer and a second buffer; and the writing of the multimedia data stream into the physical memory pool comprises: Writing the multimedia data stream into the first buffer and alternately buffering the first buffer and the second buffer; The reading of the multimedia data stream comprises: Reading the multimedia data stream from the second buffer after alternation.

5. The method of claim 4, wherein, Before the writing of the multimedia data stream into the first buffer, the method comprises: Detecting whether the first buffer is currently idle; The writing of the multimedia data stream into the first buffer comprises: If the first buffer is currently idle, writing the multimedia data stream into the first buffer.

6. The method of claim 5, wherein, The detection of whether the first buffer is currently idle comprises: Detecting a current value of a preset parameter, the preset parameter being related to a thread associated with the first buffer; If the current value is a first numerical value, determining that the first buffer is currently idle.

7. The method of claim 6, wherein, After the writing of the multimedia data stream into the first buffer, the method comprises: Updating the current value in a first updating manner, the first updating manner comprising increasing a preset value; After the reading of the multimedia data stream from the second buffer after alternation, the method comprises: Updating the current value in a second updating manner, the second updating manner comprising decreasing the preset value.

8. The method of claim 4, wherein, After the writing of the multimedia data stream into the first buffer, the method comprises: Updating a preset write flag parameter to a second numerical value; The reading of the multimedia data stream from the second buffer after alternation comprises: If the preset write flag parameter is the second numerical value, reading the multimedia data stream from the second buffer after alternation.

9. A terminal, characterized by comprising: The terminal comprises a memory and a processor; The memory is configured to store a computer program; The processor is configured to execute the computer program and implement the steps of the method according to any one of claims 1 to 8 when executing the computer program.

10. A plug flow system characterized by, The push streaming system comprises an AR device and a terminal according to claim 9, the terminal being connected with the AR device.

11. A computer readable storage medium characterized by, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 8.

Images