Screen mirroring method and device based on code rate adjustment, equipment and storage medium

Abstract

The invention provides a screen mirroring method and device based on code rate adjustment, equipment and a storage medium. The method comprises the following steps: acquiring to-be-played audio and video data; monitoring a real-time rate of a USB transmission channel, and determining a coding bit rate; according to the coding bit rate, coding the audio and video data to be played; segmenting the encoded audio and video data to obtain an audio and video data packet; transmitting the audio and video data packet to a receiving end through the USB transmission channel; and recombining and playing the audio and video data packet through the receiving end. According to the invention, the real-time rate of the USB transmission channel is monitored, and the coding bit rate is determined by using the mapping relation stored in the preset database, so that the function of automatically adjusting the coding parameters according to the actual transmission condition is realized.

Description

Technical Field

[0001] The present invention relates to the field of screen mirroring technology, and in particular to a screen mirroring method, device, equipment and storage medium based on bit rate adjustment. Background Art

[0002] Currently, with the continuous development of application scenarios such as multimedia display and remote collaboration, screen mirroring technology has become an important technical means widely used in fields such as conferencing, education, entertainment, and remote assistance. Traditional screen mirroring solutions mainly use fixed encoding parameters and simple data transmission methods, and have poor adaptability to network or USB transmission channel conditions. For example, the screen mirroring methods used in existing technologies often do not monitor USB transmission bandwidth, load, and other conditions in real time, making it difficult to dynamically adjust the encoding bit rate according to actual transmission conditions. This can lead to problems such as increased data transmission delays, reduced video quality, and audio and video asynchrony when bandwidth fluctuates or load changes significantly.

[0003] In addition, traditional solutions have limitations in terms of packet transmission of audio and video data, data integrity assurance, transmission scheduling strategies, and data packet reassembly and synchronous playback. General practices cannot effectively handle the problems of packet loss, disordered order, and inconsistent pace of different data streams that may occur during transmission, thus affecting the final playback quality and user experience. Especially in the USB transmission channel, due to factors such as limited bandwidth and fluctuating transmission rates, fixed encoding and transmission schemes are more likely to cause data congestion or transmission errors. Existing technologies still have a lot of room for improvement in optimizing USB transmission efficiency, reducing system latency, and achieving high-quality synchronous playback of audio and video.

[0004] In summary, the defects in the existing technology need to be solved urgently. Summary of the Invention

[0005] The present invention provides a screen mirroring method, device, equipment and storage medium based on bit rate adjustment, which are used to solve the defects in the existing technology and realize automatic adjustment of encoding parameters according to actual transmission conditions to ensure high quality and smoothness of video transmission.

[0006] The present invention provides a screen mirroring method based on bit rate adjustment, comprising:

[0007] Collect audio and video data to be played;

[0008] Monitor the real-time rate of the USB transmission channel and determine the encoding bit rate;

[0009] Encoding the audio and video data to be played according to the encoding bit rate;

[0010] Segment the encoded audio and video data to obtain audio and video data packets;

[0011] Transmitting the audio and video data packets to a receiving end via the USB transmission channel;

[0012] The audio and video data packets are reassembled and played through the receiving end.

[0013] According to a screen mirroring method based on bit rate adjustment provided by the present invention, the step of monitoring the bandwidth and load of the USB transmission channel and determining the encoding bit rate specifically includes:

[0014] Monitor the real-time rate of USB transmission channel;

[0015] According to the real-time rate, matching the corresponding encoding bit rate in a preset database;

[0016] The preset database is used to store the mapping relationship between the real-time rate and the encoding bit rate.

[0017] According to a screen mirroring method based on bit rate adjustment provided by the present invention, the step of encoding the audio and video data to be played according to the encoding bit rate specifically includes:

[0018] Selecting a suitable target encoding format based on the supported formats and predetermined application scenarios fed back by the receiving end;

[0019] The audio and video data to be played that is collected in real time is encoded according to the target encoding format and the encoding bit rate.

[0020] According to a screen mirroring method based on bit rate adjustment provided by the present invention, after the step of segmenting the encoded audio and video data to obtain audio and video data packets, the method further includes:

[0021] Add header information to the audio and video data packet, wherein the header information includes a sequence number and a timestamp.

[0022] According to a screen mirroring method based on bit rate adjustment provided by the present invention, the audio and video data packets include audio data packets and video data packets, and the step of transmitting the audio and video data packets to the receiving end through the USB transmission channel specifically includes:

[0023] Arrange the divided video data packets and audio data packets in a cross arrangement and establish an audio and video data packet queue;

[0024] The audio and video data packets are transmitted to a receiving end through the USB transmission channel according to the audio and video data packet queue.

[0025] According to a screen mirroring method based on bit rate adjustment provided by the present invention, the step of reassembling and playing the audio and video data packets by the receiving end specifically includes:

[0026] sorting the audio data packets and the video data packets according to the sequence number and timestamp information included in the header information;

[0027] Synchronizing the audio data packet and the video data packet according to the timestamp information;

[0028] Decode and play the synchronized audio and video data packets.

[0029] According to a screen mirroring method based on bit rate adjustment provided by the present invention, before the step of reassembling and playing the audio and video data packets by the receiving end, the method further includes:

[0030] Performing a data packet integrity check based on the sequence number included in the header information;

[0031] When there are missing audio and video data packets, a data retransmission request is generated according to the sequence number corresponding to the missing audio and video data packets.

[0032] The present invention also provides a screen mirroring device based on bit rate adjustment, comprising:

[0033] Data acquisition module, used to collect audio and video data to be played;

[0034] Rate monitoring module, used to monitor the real-time rate of the USB transmission channel and determine the encoding bit rate;

[0035] A data encoding module, configured to encode the audio and video data to be played according to the encoding bit rate;

[0036] A data segmentation module is used to segment the encoded audio and video data to obtain audio and video data packets;

[0037] A data transmission module, used for transmitting the audio and video data packets to a receiving end through the USB transmission channel;

[0038] The reassembly and playback module is used to reassemble and play the audio and video data packets through the receiving end.

[0039] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the screen mirroring method based on bit rate adjustment as described above is implemented.

[0040] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the screen mirroring method based on bit rate adjustment as described above is implemented.

[0041] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-described screen mirroring methods based on bit rate adjustment.

[0042] The screen mirroring method, device, equipment and storage medium based on bit rate adjustment provided by the present invention collect audio and video data to be played; monitor the real-time rate of the USB transmission channel to determine the encoding bit rate; encode the audio and video data to be played according to the encoding bit rate; segment the encoded audio and video data to obtain audio and video data packets; transmit the audio and video data packets to the receiving end through the USB transmission channel; and reassemble and play the audio and video data packets through the receiving end. The present invention monitors the real-time rate of the USB transmission channel and determines the encoding bit rate using the mapping relationship stored in a preset database, thereby realizing the function of automatically adjusting the encoding parameters according to the actual transmission conditions. The invention can reduce the video encoding bit rate when the USB transmission rate is low to reduce the amount of data per frame and avoid transmission congestion; and appropriately increase the bit rate when the transmission rate is high, thereby ensuring the high quality and smoothness of video transmission and improving the user viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 1 is a flow chart of a screen mirroring method based on bit rate adjustment provided by the present invention;

[0045] Figure 2 1 is a schematic structural diagram of a screen mirroring device based on bit rate adjustment provided by the present invention;

[0046] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0047] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0048] In order to solve the problems in the prior art, the present invention proposes a screen mirroring method based on bit rate adjustment to automatically adjust the encoding parameters according to the actual transmission conditions to ensure high quality and smoothness of video transmission. The screen mirroring method based on bit rate adjustment is described below. Figure 1 As shown, including but not limited to the following steps:

[0049] Step 110: Collect audio and video data to be played.

[0050] In this step, the system uses the data acquisition module to capture the screen image to be played and the corresponding audio signal in real time. The specific implementation methods include:

[0051] Use the interface provided by the operating system (such as the Capture API under the MacOS platform) to capture screen content, and obtain synchronized audio data through the audio capture interface.

[0052] During the data acquisition process, an asynchronous callback mechanism is used to buffer the collected video frames and audio data in real time into the temporary storage area of the acquisition module for subsequent encoding.

[0053] To ensure the continuity and real-time nature of data, the acquisition module cooperates with hardware acceleration technology (such as GPU-assisted data acquisition) to improve data acquisition efficiency.

[0054] Step 120: Monitor the real-time rate of the USB transmission channel and determine the encoding bit rate.

[0055] This step is the key process of dynamic bit rate adjustment, which is used to determine the bit rate used for subsequent encoding based on the current actual situation of the USB transmission channel. The specific implementation method includes:

[0056] The USB transmission channel monitoring module periodically reads the current USB interface transmission rate, bandwidth occupancy and load information, which are obtained through the hardware detection module or the driver interface.

[0057] The collected real-time transmission rate is matched with the rate-coding bit rate mapping relationship stored in the database preset in the storage medium. The database records the optimal coding bit rate parameters corresponding to each transmission rate.

[0058] The control module determines the current encoding bit rate based on the matching results and feeds this parameter back to the data encoding module as a key parameter for subsequent encoding. A closed-loop control algorithm can be used here to smooth out rate fluctuations to avoid system instability caused by frequent adjustments.

[0059] Step 130: Encode the audio and video data to be played according to the encoding bit rate.

[0060] In this step, the system encodes the original audio and video data output by the acquisition module in real time according to the encoding bit rate determined by the information feedback, specifically including:

[0061] The appropriate encoding target format is selected based on the target formats supported by the receiver (such as H.264 or H.265 for video, AAC for audio) and the application scenario. The receiver parameters are pre-configured during system initialization or dynamically obtained through a feedback mechanism.

[0062] The data encoding module compresses and encodes the real-time captured video frames and audio data according to a specified encoding bit rate. The encoding process uses hardware or software encoders to efficiently process the data, while also adjusting parameters such as frame rate, resolution, and compression level to further adapt to real-time transmission requirements.

[0063] The encoding process adopts a fixed time window processing method to ensure the smoothness and synchronization of continuous data frames. At the same time, the generated encoded data is accompanied by necessary timestamp information for subsequent use.

[0064] Step 140: Segment the encoded audio and video data to obtain audio and video data packets.

[0065] After the encoded audio and video data is output in the form of a data stream, it enters the data segmentation stage. The specific implementation method is as follows:

[0066] The data segmentation module splits the continuous audio and video data stream into segments based on predefined packet sizes and data transmission strategies. During segmentation, header information is added to each data packet. This header information includes a sequence number and a corresponding timestamp for packet sorting and integrity checking.

[0067] Audio and video data packets are divided into audio packets and video packets. Based on the identification information within the packet, the two are prepared for transmission independently or in an interleaved manner. The size of each packet must meet the requirements of USB transmission efficiency and ensure that the packets can be accurately restored when reassembled.

[0068] Step 150: Transmit the audio and video data packets to a receiving end through the USB transmission channel.

[0069] This step is the data transmission process, and its implementation depends on the application of cross-transmission strategy. The specific steps are as follows:

[0070] The split video and audio packets are arranged in a pre-set interleaved pattern to form an audio and video packet queue. The purpose of this interleaved pattern is to ensure that the two data types appear alternately during transmission, avoiding audio and video desynchronization caused by congestion on a single path.

[0071] The USB transmission module sequentially extracts audio and video packets from the queue and sends them to the receiving end via the USB transmission channel. The transmission process uses a low-latency transmission protocol, combined with the high-speed transmission characteristics of USB, to achieve efficient and stable data packet output.

[0072] The transmission module also supports an error detection mechanism, which monitors in real time whether packet loss or transmission errors occur during data packet transmission. If an anomaly is detected, a data retransmission request is triggered through a feedback mechanism to ensure the integrity of the transmitted data.

[0073] Step 160: Reassemble and play the audio and video data packets through the receiving end.

[0074] At the receiving end, the audio and video data packets are reassembled according to the sequence number and timestamp information in the header information to complete decoding and synchronous playback. The specific implementation steps include:

[0075] The data receiving module at the receiving end stores the audio and video data packets received through the USB transmission channel into the buffer in sequence and sorts them according to the sequence number of the data packet header to ensure that the order of the data packets is consistent with the original data.

[0076] The reassembly module synchronizes the sorted audio and video packets based on their timestamps. If any packets are missing or out of sequence, the error recovery mechanism is triggered based on the sequence number to request retransmission of the missing packets.

[0077] After the reorganization is completed, the video data packet and audio data packet are sent to their respective decoders for decoding. The decoded audio and video data are synchronously rendered and output by the playback module to ensure that users have a smooth, low-latency screen mirroring experience.

[0078] During the entire playback process, the system can also monitor the playback status in real time, continuously optimize the data reception and decoding process through a feedback mechanism, and implement closed-loop control to improve playback quality and stability.

[0079] As a further optional embodiment, the step of monitoring the bandwidth and load of the USB transmission channel and determining the encoding bit rate specifically includes:

[0080] Monitor the real-time rate of USB transmission channel;

[0081] According to the real-time rate, matching the corresponding encoding bit rate in a preset database;

[0082] The preset database is used to store the mapping relationship between the real-time rate and the encoding bit rate.

[0083] In this embodiment, to further improve the adaptability of the screen mirroring system under different transmission conditions, a method for dynamically determining the encoding bit rate based on the USB transmission channel bandwidth and load is provided. The method specifically includes the following three sub-steps:

[0084] Monitor the real-time rate of USB transmission channels

[0085] In this step, a USB transfer rate monitoring module is introduced. By directly interacting with the USB controller, it periodically collects real-time transfer rate data from the USB transmission channel. This data accurately reflects the current bandwidth usage and load level of the USB transmission channel, providing a quantitative basis for the subsequent matching process. The collected rate data includes instantaneous data rate and bandwidth utilization, reflecting the dynamically changing transmission environment in real time.

[0086] Match the corresponding encoding bit rate in the preset database

[0087] The system maintains a pre-set encoding bit rate mapping database on the storage medium, which stores the optimal encoding bit rates for different real-time rates. After monitoring the real-time rate, the control module matches this rate data with the mapping relationships in the pre-set database, finding the encoding bit rate parameters that best match the current rate value. This mapping relationship is determined during design based on historical transmission data and actual application scenarios, and can be calibrated and dynamically updated using experimental data to ensure the accuracy and applicability of the mapping results.

[0088] Determine the encoding bit rate

[0089] Once the match is complete, the system outputs the corresponding encoding bit rate based on the mapping relationship in the database and feeds this parameter back to the data encoding module for subsequent audio and video data encoding processing. This dynamically determined encoding bit rate ensures that when the USB transmission rate is low, a lower encoding bit rate is used to reduce data volume and thus transmission latency; while at higher transmission rates, the encoding bit rate is appropriately increased to improve the transmitted image quality and detail. In this way, the system can effectively cope with bandwidth fluctuations and load changes in the USB transmission environment, achieving the goal of optimizing image quality and transmission efficiency.

[0090] As a further optional embodiment, the step of encoding the to-be-played audio and video data according to the encoding bit rate specifically includes:

[0091] Selecting a suitable target encoding format based on the supported formats and predetermined application scenarios fed back by the receiving end;

[0092] The audio and video data to be played that is collected in real time is encoded according to the target encoding format and the encoding bit rate.

[0093] To further enhance the screen mirroring system's adaptability and encoding flexibility in different usage scenarios and receiving end environments, this embodiment introduces a feedback mechanism for the receiving end's supported formats and application scenarios during the data encoding process, enabling dynamic selection of the appropriate encoding format based on the encoding bit rate and real-time encoding. The specific implementation includes the following two sub-steps:

[0094] 1: Choose the appropriate target encoding format

[0095] Receiver feedback data acquisition

[0096] During the startup or operation of the system, the system receives parameter information fed back by the receiving device through a preset feedback channel. This information includes but is not limited to the encoding format, resolution, frame rate, audio and video decoding capabilities supported by the receiving end, and predetermined application scenarios (such as conference mode, entertainment mode, or remote assistance mode).

[0097] Pre-booked application scenario analysis

[0098] The control module combines receiver feedback with the current application scenario to comprehensively evaluate the advantages of each supported format. For example, in scenarios requiring high video quality and detail preservation, the highly efficient H.265 encoding method is preferred; when compatibility is high or decoding resources are limited, the H.264 encoding method is preferred. For audio, AAC or other suitable audio encoding formats are selected based on the receiver's processing capabilities.

[0099] Target encoding format determination

[0100] Based on this analysis, the control module determines and selects the target encoding format to be used for this encoding operation. This format not only meets the receiver's decoding compatibility requirements but also matches the current USB transmission channel conditions (determined by the encoding bit rate described above), ensuring that the final data is transmitted and decoded with optimal quality.

[0101] 2: Dynamic encoding operation of real-time data collection

[0102] Encoding parameter configuration

[0103] Once the target encoding format is determined, the data encoding module will use the predetermined encoding bit rate as the basic parameter and configure the encoder in combination with the characteristics of the target encoding format. This configuration includes setting the parameters of the video encoder (such as H.264 or H.265) and audio encoder (such as AAC), such as bit rate, frame rate, resolution, GOP (Group of Pictures) length, and other key encoding control parameters.

[0104] Real-time data encoding and processing

[0105] The real-time audio and video data to be played is captured by the data acquisition module and immediately fed into the encoder for processing. The encoder compresses and encodes the data based on the input information, encoding bit rate, and target encoding format, generating a digital media data stream that meets the target format requirements.

[0106] Real-time feedback and adjustments

[0107] During the encoding process, the system monitors the quality of the output encoded data stream and dynamically adjusts encoding parameters based on subsequent transmission and receiving feedback, ensuring high video quality and playback smoothness despite changes in USB transmission channel conditions. This feedback mechanism forms a closed-loop control loop, enabling the encoding module to continuously optimize the encoding process based on real-time transmission conditions and application requirements.

[0108] This embodiment, by introducing a receiving-end feedback mechanism and pre-determined application scenario judgment, enables accurate selection of the appropriate target encoding format during the encoding phase. It then efficiently compresses and encodes the real-time captured audio and video data in combination with a predetermined encoding bit rate, thereby achieving optimal data transmission quality and decoding and playback effects. This implementation not only improves the system's adaptability in changing transmission environments, but also addresses the multiple requirements of different application scenarios for image quality, latency, and compatibility, significantly enhancing the overall screen mirroring experience.

[0109] As a further optional embodiment, after the step of segmenting the encoded audio and video data to obtain audio and video data packets, the method further includes:

[0110] Add header information to the audio and video data packet, wherein the header information includes a sequence number and a timestamp.

[0111] After the encoded audio and video data is segmented, in order to ensure smooth reassembly and synchronous playback of subsequent data at the receiving end, this embodiment adds the following steps:

[0112] 1. Header information design and definition

[0113] Serial number

[0114] A unique sequence number is defined for each segmented audio and video data packet. The sequence number increments according to the order in which the data packets are generated and is used to identify the position of the data packet in the entire data stream, making it easier for the receiving end to sort and reassemble the data in the correct order.

[0115] Timestamp

[0116] To ensure accurate synchronization of audio and video data, a timestamp field is defined to record the generation or display time of the packet. This timestamp information can be used at the receiving end to correct audio and video synchronization and manage playback delays, ensuring that each packet is seamlessly connected according to the predetermined timing after decoding.

[0117] 2. Header information addition operation

[0118] After the segmentation operation is completed, the data segmentation module processes each generated audio and video data packet.

[0119] This module calls a pre-defined encapsulation function and appends the designed header information (including the sequence number and timestamp) to the front of each data packet. The entire data packet format can therefore be expressed as: [header information][encoded data].

[0120] The header information and the encoded data are combined in a unified data encapsulation format to ensure the correct parsing and reading of each field information during subsequent transmission and reassembly.

[0121] 3. Data packet encapsulation standards and protocol support

[0122] The format of the header information and its field definitions can be flexibly adjusted according to application scenarios and protocol requirements.

[0123] To ensure the reliability of data transmission, this embodiment also recommends setting a check code or control flag field in the header information to help the receiving end quickly detect whether the data packet is complete or has a transmission error.

[0124] This information not only supports the sequencing and synchronization of data packets, but can also be used by upper-layer applications to implement data packet retransmission and error recovery mechanisms, thereby further improving the overall transmission stability and reliability of the system.

[0125] As a further optional embodiment, the audio and video data packet includes an audio data packet and a video data packet, and the step of transmitting the audio and video data packet to the receiving end through the USB transmission channel specifically includes:

[0126] Arrange the divided video data packets and audio data packets in a cross arrangement and establish an audio and video data packet queue;

[0127] The audio and video data packets are transmitted to a receiving end through the USB transmission channel according to the audio and video data packet queue.

[0128] In this embodiment, the audio and video data packets generated after the data segmentation processing include audio data packets and video data packets respectively, and each data packet has additional header information (including serial number and timestamp) to ensure the integrity of the identification data required for subsequent sorting and synchronization.

[0129] To avoid local transmission bottlenecks or data asynchrony problems that may be caused by continuous transmission of data packets on a single path, the control module cross-arranges the divided audio and video data packets. The specific implementation method includes:

[0130] First, the video data packets and audio data packets are collected separately and preliminarily arranged according to the generation order or preset timestamps;

[0131] Then, the audio and video data packets are arranged alternately according to a predetermined rule (for example, in a rotational order), thereby forming a data stream with audio and video interleaved. For example, the data packets can be arranged in the following order: video data packet 1, audio data packet 1, video data packet 2, audio data packet 2, and so on.

[0132] The cross-arranged packets are sequentially stored in a queue, which serves as the basis for scheduling data transmission by the USB transmission module. Each packet in the queue is stored in the order in which it is arranged, ensuring that transmission is strictly carried out in the cross-arranged order. This not only improves data transmission efficiency but also facilitates the sorting, reassembly, and synchronous playback of subsequent packets at the receiving end.

[0133] The USB transmission module leverages the high-speed data transmission capabilities of the USB interface to sequentially extract data packets from the audio and video data packet queue and transmit them to the receiving end via the USB transmission channel according to the established transmission protocol. During the transmission process, the module monitors the transmission status of the USB channel in real time to ensure that each data packet is sent with low latency and high transmission efficiency.

[0134] During the transmission process, the USB transmission module combines the scheduling algorithm to dynamically adjust the data packets in the queue:

[0135] When the USB transmission channel is detected to be stable, packets are sent one by one in the preset queue order;

[0136] When there is a transmission anomaly or congestion, the module temporarily adjusts the queue scheduling through the feedback mechanism, giving priority to sending key data packets (such as key frame data packets or audio synchronization packets) to ensure data integrity and synchronization at the receiving end.

[0137] After each transmission, the USB transmission module records the packet transmission status and, based on the receiving end's feedback, determines whether the packet was received correctly. If a packet is lost or a transmission error is detected, the corresponding retransmission mechanism is triggered, further ensuring the reliability and stability of the overall transmission process.

[0138] By interleaving the segmented video and audio data packets, establishing a dedicated audio and video data packet queue, and then sequentially sending the packets over the USB transmission channel based on the queue, this embodiment effectively balances the order and synchronization requirements of audio and video data during transmission. This method not only improves the efficiency and stability of data transmission, but also facilitates the rapid sorting, accurate reassembly, and synchronized playback of subsequent audio and video data packets at the receiving end, thereby achieving a low-latency, high-quality screen mirroring effect.

[0139] As a further optional embodiment, the step of reassembling and playing the audio and video data packets by the receiving end specifically includes:

[0140] sorting the audio data packets and the video data packets according to the sequence number and timestamp information included in the header information;

[0141] Synchronizing the audio data packet and the video data packet according to the timestamp information;

[0142] Decode and play the synchronized audio and video data packets.

[0143] To ensure efficient reassembly and synchronous playback of audio and video data at the receiving end, this embodiment introduces a sorting and synchronization mechanism based on the sequence number and timestamp data in the header information at the receiving end. The specific steps are as follows:

[0144] 1. Audio and video data packet sorting

[0145] The receiving end first receives each audio and video data packet with header information from the USB transmission channel. The header information contains a unique serial number and a corresponding timestamp.

[0146] The control module arranges the received audio and video packets in the order they were generated, based on the sequence number information in the packets. This step ensures that any confusion in the order of the packets during physical transmission can be corrected, providing a basis for subsequent accurate reassembly.

[0147] 2. Audio and video data packet synchronization

[0148] After the sorting is completed, the receiving end further synchronizes the audio and video data packets based on the timestamp data in the data packet header information.

[0149] The synchronization module compares and corrects the timestamps in the audio and video data packets, aligning the two data streams along the same timeline to ensure seamless synchronization of the audio and video during decoding and playback.

[0150] If a large time deviation or asynchrony is detected, the system can trigger a preset delay or acceleration strategy to fine-tune the relevant data packets to achieve precise synchronization in the final output.

[0151] 3. Decoding and playback

[0152] The audio data packets and video data packets that have been sorted and synchronized are sent to the audio decoder and video decoder for decoding respectively.

[0153] The original media data generated after decoding is scheduled by the playback module and played synchronously according to the corrected time sequence, ensuring that users have a low-latency, high-quality and synchronized audio-visual experience during the viewing process.

[0154] The playback module can also detect the playback status in real time. If any playback anomalies are encountered, feedback will be given to subsequent modules to initiate data retransmission or further synchronization adjustment mechanisms, thereby improving the stability and reliability of the overall system.

[0155] As a further optional embodiment, before the step of reassembling and playing the audio and video data packets by the receiving end, the method further includes:

[0156] Performing a data packet integrity check based on the sequence number included in the header information;

[0157] When there are missing audio and video data packets, a data retransmission request is generated according to the sequence number corresponding to the missing audio and video data packets.

[0158] To improve data integrity and transmission reliability during transmission, this embodiment adds steps for packet integrity checking and retransmission request generation before the receiving end sorts, synchronizes, decodes, and plays the audio and video data packets. The specific implementation is as follows:

[0159] Packet integrity check

[0160] After receiving the audio and video data packet with header information, the receiving end first parses the sequence number contained in the header information.

[0161] The control module compares the serial numbers of consecutive data packets to determine whether each data packet is complete.

[0162] If a jump or missing sequence number is found, it is determined that the corresponding data packet has been lost or abnormal during transmission.

[0163] Generate data retransmission request

[0164] For a missing data packet, the control module generates a corresponding data retransmission request according to a sequence number corresponding to the missing data packet.

[0165] The retransmission request is sent to the sender through the feedback channel, requesting the retransmission of the missing audio and video data packets to ensure that the receiver can obtain the complete data stream.

[0166] At the same time, after receiving the retransmission request, the sender will give priority to re-encoding the missing data packets or directly retransmit the original data packets to restore the continuity and integrity of the audio and video data as soon as possible.

[0167] Connection with subsequent data reorganization

[0168] After completing the data packet integrity check and generating the necessary retransmission request, the receiving end waits for and receives the supplementary audio and video data packets.

[0169] After receiving the complete data packet, the system reassembles, decodes and plays the data packet according to the previously defined sorting and synchronization process to ensure that the final audio and video output has good synchronization and integrity.

[0170] The screen mirroring device based on bit rate adjustment provided by the present invention is described below. Figure 2 As shown, the screen mirroring device based on bit rate adjustment described below and the screen mirroring method based on bit rate adjustment described above can correspond to each other.

[0171] A screen mirroring device based on bit rate adjustment, comprising:

[0172] Data collection module 210, used to collect audio and video data to be played;

[0173] The rate monitoring module 220 is used to monitor the real-time rate of the USB transmission channel and determine the encoding bit rate;

[0174] A data encoding module 230, configured to encode the audio and video data to be played according to the encoding bit rate;

[0175] The data segmentation module 240 is used to segment the encoded audio and video data to obtain audio and video data packets;

[0176] The data transmission module 250 is used to transmit the audio and video data packets to the receiving end through the USB transmission channel;

[0177] The reassembly and playback module 260 is configured to reassemble and play the audio and video data packets through the receiving end.

[0178] Figure 3 An example of a physical structure diagram of an electronic device is shown below. Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the screen mirroring method based on bit rate adjustment, which includes:

[0179] Collect audio and video data to be played;

[0180] Monitor the real-time rate of the USB transmission channel and determine the encoding bit rate;

[0181] Encoding the audio and video data to be played according to the encoding bit rate;

[0182] Segment the encoded audio and video data to obtain audio and video data packets;

[0183] Transmitting the audio and video data packets to a receiving end via the USB transmission channel;

[0184] The audio and video data packets are reassembled and played through the receiving end.

[0185] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0186] On the other hand, the present invention further provides a computer program product, comprising a computer program, which may be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the screen mirroring method based on bit rate adjustment provided by the above methods, the method comprising:

[0187] Collect audio and video data to be played;

[0188] Monitor the real-time rate of the USB transmission channel and determine the encoding bit rate;

[0189] Encoding the audio and video data to be played according to the encoding bit rate;

[0190] Segment the encoded audio and video data to obtain audio and video data packets;

[0191] Transmitting the audio and video data packets to a receiving end via the USB transmission channel;

[0192] The audio and video data packets are reassembled and played through the receiving end.

[0193] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the screen mirroring method based on bit rate adjustment provided by the above methods, the method comprising:

[0194] Collect audio and video data to be played;

[0195] Monitor the real-time rate of the USB transmission channel and determine the encoding bit rate;

[0196] Encoding the audio and video data to be played according to the encoding bit rate;

[0197] Segment the encoded audio and video data to obtain audio and video data packets;

[0198] Transmitting the audio and video data packets to a receiving end via the USB transmission channel;

[0199] The audio and video data packets are reassembled and played through the receiving end.

[0200] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0201] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0202] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A screen mirroring method based on bit rate adjustment, characterized in that: include: Collect audio and video data to be played; Monitor the real-time rate of the USB transmission channel and determine the encoding bit rate; Encoding the audio and video data to be played according to the encoding bit rate; Segment the encoded audio and video data to obtain audio and video data packets; Transmitting the audio and video data packets to a receiving end via the USB transmission channel; The audio and video data packets are reassembled and played through the receiving end.

2. The screen mirroring method based on bit rate adjustment according to claim 1, characterized in that: The step of monitoring the bandwidth and load of the USB transmission channel and determining the encoding bit rate specifically includes: Monitor the real-time rate of USB transmission channel; According to the real-time rate, matching the corresponding encoding bit rate in a preset database; The preset database is used to store the mapping relationship between the real-time rate and the encoding bit rate.

3. The screen mirroring method based on bit rate adjustment according to claim 1, characterized in that: The step of encoding the audio and video data to be played according to the encoding bit rate specifically includes: Selecting a suitable target encoding format based on the supported formats and predetermined application scenarios fed back by the receiving end; The audio and video data to be played that is collected in real time is encoded according to the target encoding format and the encoding bit rate.

4. The screen mirroring method based on bit rate adjustment according to claim 1, wherein: After the step of segmenting the encoded audio and video data to obtain audio and video data packets, the method further includes: Add header information to the audio and video data packet, wherein the header information includes a sequence number and a timestamp.

5. The screen mirroring method based on bit rate adjustment according to claim 4, characterized in that: The audio and video data packets include audio data packets and video data packets. The step of transmitting the audio and video data packets to the receiving end through the USB transmission channel specifically includes: Arrange the divided video data packets and audio data packets in a cross arrangement and establish an audio and video data packet queue; The audio and video data packets are transmitted to a receiving end through the USB transmission channel according to the audio and video data packet queue.

6. The screen mirroring method based on bit rate adjustment according to claim 5, characterized in that: The step of reassembling and playing the audio and video data packets by the receiving end specifically includes: sorting the audio data packets and the video data packets according to the sequence number and timestamp information included in the header information; Synchronizing the audio data packet and the video data packet according to the timestamp information; Decode and play the synchronized audio and video data packets.

7. The screen mirroring method based on bit rate adjustment according to claim 6, characterized in that: Before the step of reassembling and playing the audio and video data packets by the receiving end, the method further includes: Performing a data packet integrity check based on the sequence number included in the header information; When there are missing audio and video data packets, a data retransmission request is generated according to the sequence number corresponding to the missing audio and video data packets.

8. A screen mirroring device based on bit rate adjustment, characterized in that: include: Data acquisition module, used to collect audio and video data to be played; Rate monitoring module, used to monitor the real-time rate of the USB transmission channel and determine the encoding bit rate; A data encoding module, configured to encode the audio and video data to be played according to the encoding bit rate; A data segmentation module is used to segment the encoded audio and video data to obtain audio and video data packets; A data transmission module, used for transmitting the audio and video data packets to a receiving end through the USB transmission channel; The reassembly and playback module is used to reassemble and play the audio and video data packets through the receiving end.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the screen mirroring method based on bit rate adjustment according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the screen mirroring method based on bit rate adjustment according to any one of claims 1 to 7 is implemented.

Images