Audio and video data optimization method and device for screen projection and storage medium

Abstract

The invention relates to the technical field of communication, in particular to an audio and video data optimization method and device for screen projection and a storage medium, and the method comprises the steps that transmitting end equipment transmits audio and video data to receiving end equipment through a first channel, and the transmitting end equipment transmits the audio and video data to a cache region through a second channel; and if it is monitored that packet loss occurs when the receiving end device receives the audio and video data transmitted by the first channel, the receiving end device calls the audio and video substitute data in the cache region, and integrates the audio and video data transmitted by the first channel and the audio and video substitute data. According to the method and the device, the technical problem of ensuring the efficiency and the stability of audio and video data transmission in the screen projection process in some scenes with relatively poor network environments or relatively much wireless interference is solved, so that the data packet loss rate is reduced, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and in particular to a method, device and storage medium for optimizing audio and video data for screen projection. Background Art

[0002] Screen projection technology is of great significance in modern technological life. It allows users to project content such as videos, pictures, documents, etc. on mobile devices (such as mobile phones and tablets) or computers onto large-screen devices (such as TVs and projectors), greatly improving the visual experience. In terms of home entertainment, users can enjoy watching movies and playing games on a large screen; in business scenarios, it is convenient to display presentations and improve communication efficiency.

[0003] During the screen projection transmission process, the quality of the picture and audio is easily affected by network conditions. When the network speed is slow, the transmission of audio and video data during the screen projection process is unstable, and its transmission rate cannot meet the needs of smooth playback. Furthermore, when the network is complex or unstable, data packets are prone to loss or damage during the audio and video data transmission process, and the receiving end cannot completely restore the picture at this time; and under this network condition, the source device and the target device need to constantly re-establish the connection, adjust the transmission parameters or switch channels, which not only easily leads to device delays, but also frequent channel switching easily leads to reduced efficiency and increased risk of transmission interruption. In addition, if the performance of the screen projection receiving device is insufficient (such as processor, memory, graphics card, etc.), it will also affect the stability and effect of wireless screen projection, resulting in poor transmission effects such as audio and video playback jams, incomplete pictures, and sound interruptions. Therefore, in some scenarios with poor network environment or more wireless interference, it is extremely important to improve the efficiency and stability of audio and video data transmission during screen projection.

[0004] The Chinese invention patent with the name of "screen projection audio and video data transmission method and related equipment" and the announcement number CN114697733B has been published. This invention patent builds a wireless local area network and a peer-to-peer network channel between the source device and the target device, selects a channel with short delay to transmit audio, selects a channel with high remaining bandwidth to transmit video, and uses multiple channels to cross-send data to balance the load, or multiple channels to send the same data concurrently to reduce the packet loss rate. However, using multiple channels to send a large amount of the same data concurrently can easily occupy network bandwidth resources and increase the consumption of computing resources.

[0005] The Chinese invention patent with the name of "Data Transmission Method and Device" and the announcement number CN111628847B, which has been published, includes monitoring the network status of the audio and video data connection channel during screen projection. When the network status deteriorates, the corresponding transmission method is selected and executed from the preset transmission strategy according to the index value of the network status. This invention patent solves the problems of delay and freeze during screen projection and improves the display effect. However, relying solely on network status monitoring cannot fully reflect the complex situation in actual transmission. If a small number of data packets are lost during the transmission process, this patent can only be solved by switching the transmission method. At this time, the switching process may introduce additional delays. Summary of the invention

[0006] The purpose of this application is to provide an audio and video data optimization method, device and storage medium for screen projection, which solves the technical problem of ensuring the efficiency and stability of audio and video data transmission during screen projection in certain scenarios with poor network environment or more wireless interference, so as to reduce the data packet loss rate and improve the user experience.

[0007] In order to solve the above technical problems, the solution adopted by this application is as follows: The present invention provides an audio and video data optimization method for screen projection, characterized in that its implementation includes the following steps: Step S1: The receiving end receives the audio and video data within one cycle transmitted by the sending end; the audio and video data received by the receiving end is the original data; at the same time, the buffer area receives and stores the audio and video data within one cycle transmitted by the sending end; the audio and video data stored in the buffer area is the substitute data; The substitute data and the original data belong to the audio and video data within a cycle sent by the sending end; Step S2: The receiving end detects whether there is packet loss in the original data within the period, and if so, executes step S3, otherwise, executes step S4; Step S3: according to the packet loss situation, the substitute data or part of the substitute data is retrieved from the buffer area and integrated with the original data to obtain complete audio and video data; and step S5 is executed; Step S4: The original data is complete audio and video data; execute step S5; Step S5: the receiving end outputs the complete audio and video data; the receiving end enters the next audio and video data receiving cycle and repeats step S1; The implementation of the above step S3 includes the following steps: Step S31: the receiving end counts the missing audio and video data packets to obtain a missing data packet list; the missing data packet list includes the total number of missing audio and video data packets and the missing audio and video data packet numbers; Step S32: the receiving end retrieves a replacement data packet from the buffer area according to the missing data packet list, and integrates the replacement data packet into the original data to obtain complete audio and video data; The replacement data includes a replacement data packet, and the replacement data packet number is the same as the missing audio and video data packet number on the missing data packet list.

[0008] In some embodiments, in step S1, the receiving end receives audio and video data within a period transmitted by the sending end through the first channel; at the same time, the buffer area receives and stores audio and video data within a period transmitted by the sending end through the second channel.

[0009] In some embodiments, when the receiving end and the buffer area receive audio and video data respectively, the network status of the first channel and the second channel are monitored and compared at a preset frequency, and the audio and video data transmission mode is selected according to the comparison result, which includes the following steps: Step S111: respectively monitoring the network status of the first channel and the second channel; Step S112: comparing the network status of the first channel and the second channel; if the network status of the first channel is better than that of the second channel, executing step S113; if the network status of the second channel is better than that of the first channel, executing step S114; Step S113: the receiving end receives the audio and video data within a period transmitted by the transmitting end through the first channel; at the same time, the buffer area receives and stores the audio and video data within a period transmitted by the transmitting end through the second channel; Step S114: the receiving end receives the audio and video data within a period transmitted by the sending end through the second channel; at the same time, the buffer area receives and stores the audio and video data within a period transmitted by the sending end through the first channel.

[0010] In some embodiments, in step S111, the formula: Among them, y represents the network status score of the channel; x i Represents the i-th network state parameter; a i represents the weight of the i-th network state parameter; Thus, the network status score values ​​of the first channel and the second channel are obtained; the network status score value of the first channel is compared with the network status score value of the second channel, and the audio and video data transmission method is selected according to the comparison result.

[0011] The network status parameters include bandwidth utilization rate, packet loss rate, and delay.

[0012] In some embodiments, the first channel includes a first IP address, and the second channel includes a second IP address; the sending end device establishes connections with the receiving end and the cache area through the first IP address and the second IP address respectively.

[0013] In some embodiments, the buffer area is a circular buffer area, and the audio and video data are written into the circular buffer area in sequence according to the order of data packet labels.

[0014] In some embodiments, in step S1, the transmitting end encodes and packages the video and audio and labels them, and the specific steps include: Step S101: the transmitting end device encodes the video and the audio respectively to obtain a video data segment and an audio data segment; Step S102: Packing the video data segments and audio data segments in one cycle according to the data volume to obtain a plurality of video data packets and a plurality of audio data packets; Step S103: label each data packet and list the label numbers; Step S104: After the sending end completes sending the audio and video data packets and the number list within the cycle, step S101 is repeated to package and number the audio and video data within the next cycle; The number list is sent from the sending end to the receiving end, and the receiving end counts the missing audio and video data packets through the number list.

[0015] The present invention also provides an audio and video data optimization device for screen projection, characterized in that, in order to implement an audio and video data optimization method for screen projection as described in any one of claims 1 to 7, the method comprises: A communication module, used to establish a first channel and a second channel between the sending end and the receiving end; A sending module, comprising a sending end device, wherein the sending end device is used to send audio and video data through the first channel and the second channel; The receiving module includes a receiving end device, wherein the receiving end device is used for: Acquire raw data through the first channel or the second channel; Count the missing audio and video data packets and obtain a list of missing data packets; Retrieving a replacement data packet from a buffer area according to the missing data packet list, and integrating the replacement data packet with the original data; A cache module, comprising a cache area, wherein the cache area is used to store substitute data; Monitoring module for: Monitor the network status of the first channel and the second channel respectively at a preset frequency and compare them, and select an audio and video data transmission method according to the comparison result; According to the packet loss of the original data within the period detected by the receiving end device, the frequency of monitoring the network status of the first channel and the second channel is increased; The receiving end establishes a connection with the buffer area through the monitoring module; Label module for: The audio and video data packets packaged by the sending device are numbered and a number list is made.

[0016] The present invention also provides a computer device, characterized in that the computer device includes a memory and a processor, the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, it implements any of the above-mentioned methods for optimizing audio and video data for screen projection.

[0017] The present invention also provides a computer-readable storage medium, characterized in that a program is stored in the computer-readable storage medium, and when the computer executes the program, any one of the above-mentioned methods for optimizing audio and video data for screen projection is executed.

[0018] The technical solution of the present application has at least the following advantages and beneficial effects: 1. The present invention sets up dual channels, one channel is used to transmit the audio and video data of the sending end device to the receiving end device, and the other channel transmits the audio and video data of the sending end device to the buffer area for temporary storage; if the data packet received by the receiving end device is lost, the receiving end device retrieves the audio and video data packet that is the same as the lost packet from the buffer area, so as to prevent the delay caused by channel switching when a small amount of packet loss occurs; 2. The present invention will also comprehensively evaluate the current network status, effectively identify abnormal conditions in the network and switch transmission channels, avoiding unnecessary mis-switching and frequent channel switching; the present invention effectively ensures the integrity of audio and video data, avoiding problems such as audio and video playback freezes and incompleteness caused by data packet loss, thereby improving the user's viewing experience in the projection scene; 3. In the present invention, the receiving end sends a list of missing data packets to the buffer area, retrieves replacement data packets according to the list and transmits them to the receiving end, and the receiving end integrates the data into complete data, which not only improves the reliability of data transmission and ensures the integrity of audio and video data, but also the buffer area can quickly identify and extract corresponding replacement data packets according to the list and transmit them to the receiving end, avoiding the delay caused by the sending end retransmission or waiting for the sending end to retransmit in the traditional method, so that the sending end can send data according to the periodic timing; in addition, the buffer area only extracts necessary data packets according to the missing data packet list provided by the receiving end, instead of resending the entire data stream, avoiding unnecessary data transmission and storage, which can significantly reduce the data flow in the network and reduce the network load. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a flow chart of the method for optimizing audio and video data for screen projection of the present invention; Figure 2 For the present invention Figure 1 A schematic flow chart of a specific embodiment of step S3; Figure 3 A structural schematic diagram of the audio and video data optimization device for screen projection of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] It should be noted that similar numbers and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further defined and explained in the subsequent figures. If the terms "center", "upper", "lower", "inner", "outer", etc. indicate an orientation or position relationship based on the orientation or position relationship shown in the figure, or the orientation or position relationship in which the product of the application is usually placed when in use, it is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application. It should also be noted that unless otherwise clearly specified and limited, if the terms "set", "install", and "connect" appear, they should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or it can be a connection between the two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0022] Example 1 Please refer to Figure 1-Figure 2 The present invention provides an audio and video data optimization method for screen projection, and its implementation includes the following steps: Step S1: The receiving end receives the audio and video data within one cycle transmitted by the sending end; the audio and video data received by the receiving end is the original data; at the same time, the buffer area receives and stores the audio and video data within one cycle transmitted by the sending end; the audio and video data stored in the buffer area is the substitute data; The substitute data and the original data belong to the audio and video data within a cycle sent by the sending end; Step S2: The receiving end detects whether there is packet loss in the original data within the period, and if so, executes step S3, otherwise, executes step S4; Step S3: according to the packet loss situation, the substitute data or part of the substitute data is retrieved from the buffer area and integrated with the original data to obtain complete audio and video data; and step S5 is executed; Step S4: The original data is complete audio and video data; execute step S5; Step S5: the receiving end outputs the complete audio and video data; the receiving end enters the next audio and video data receiving cycle and repeats step S1; The implementation of the above step S3 includes the following steps: Step S31: the receiving end counts the missing audio and video data packets to obtain a missing data packet list; the missing data packet list includes the total number of missing audio and video data packets and the missing audio and video data packet numbers; Step S32: the receiving end retrieves a replacement data packet from the buffer area according to the missing data packet list, and integrates the replacement data packet into the original data to obtain complete audio and video data; The replacement data includes a replacement data packet, and the replacement data packet number is the same as the missing audio and video data packet number on the missing data packet list.

[0023] It should be noted that in the above step S5, the receiving end outputs the complete audio and video data to the screen projection device for screen projection; It should be noted that the packet loss situation includes the lost data packets and the number of lost packets; the first channel and the second channel both include the video transmission channel and the audio transmission channel; It should be noted that the replacement data in the buffer area is used as backup data at the receiving end when data packets are lost, so as to ensure the integrity and continuity of the data.

[0024] It needs to be explained that, during the transmission of audio and video data, the original data received by the receiving end may be lost due to network conditions. At this time, the receiving end sends a list of missing data packets to the cache area, retrieves the replacement data packets according to the list and transmits them to the receiving end. The receiving end integrates the data into complete data, which not only improves the reliability of data transmission and ensures the integrity of the audio and video data, but also the cache area can quickly identify and extract the corresponding replacement data packets according to the list and transmit them to the receiving end, avoiding the delay caused by the sender's retransmission or waiting for the sender to retransmit in the traditional method, so that the sender can send data according to the periodic timing instead of waiting for the sending instruction; in addition, the cache area only extracts the necessary data packets based on the missing data packet list provided by the receiving end, instead of resending the entire data stream, avoiding unnecessary data transmission and storage, which can significantly reduce the data flow in the network and reduce the network load.

[0025] In the present invention, the sending end is a device that sends audio and video data, including but not limited to mobile phones and tablets; the receiving end device is a device that receives and plays audio and video data, including but not limited to televisions, projectors, and computers.

[0026] It should be noted that in step S1, the transmitting end encodes and packages the video and audio and labels them, and the specific steps include: Step S101: the transmitting end device encodes the video and the audio respectively to obtain a video data segment and an audio data segment; Step S102: Packing the video data segments and audio data segments in one cycle according to the data volume to obtain a plurality of video data packets and a plurality of audio data packets; Step S103: label each data packet and list the label numbers; Step S104: After the sending end completes sending the audio and video data packets and the number list within the cycle, step S101 is repeated to package and number the audio and video data within the next cycle; The number list is sent from the sending end to the receiving end, and the receiving end counts the missing audio and video data packets through the number list.

[0027] It should be explained that the label list records the label information of the data packets within a cycle; in this embodiment, the label list is sent together with the first data packet within a cycle. The purpose is that after the receiving end receives the label list, it can directly judge the missing status of subsequent data packets within the cycle, avoiding the need for the receiving end to wait until the end of the entire cycle to judge the integrity of the data packet, reducing processing delays and potential data loss risks caused by missing data packets.

[0028] It should be noted that, in this embodiment, the video encoding uses the H.264 video encoding standard to obtain video data segments; the audio encoding uses the ACC audio encoding standard to obtain audio data segments.

[0029] In step S31, the receiving end counts missing audio and video data packets to obtain a missing data packet list, including the following steps: After receiving the label list within a cycle, the receiving end will traverse and compare the label information of the received data packets on the label list in turn; after the comparison is completed, the remaining labels on the label list are the missing data packets; according to the remaining labels on the label list, they are sorted to obtain a list of missing data packets.

[0030] In this embodiment, by formula: Among them, m represents the label value; n represents the size of the hash table; mod represents the modulus operation; h(k) represents the hash value of the m label.

[0031] The above formula is used to calculate the hash value of the label of the data packet in the cache area within a cycle and the hash value of the label of the data packet in the missing data packet list, and the hash value of the label in the missing data packet list is compared with the hash value of the cache area label; the data packet with the same hash value is extracted and transmitted to the receiving end, which integrates it with the original data.

[0032] It should be explained that the advantage of hash comparison labels is that there is no need for linear retrieval when searching for replacement data packets of lost audio and video data packets in the cache area, which improves the search efficiency.

[0033] In step S1, the receiving end receives the audio and video data within a period transmitted by the transmitting end through the first channel; at the same time, the buffer area receives and stores the audio and video data within a period transmitted by the transmitting end through the second channel; It should be noted that the first channel includes a first IP address, and the second channel includes a second IP address; the sending end device establishes connections with the receiving end and the buffer area through the first IP address and the second IP address respectively; In this embodiment, the data transmitted by the transmitting device through the first channel and the second channel are performed simultaneously, the purpose of which is to ensure the smoothness and continuity of the receiving device in retrieving data in the buffer area.

[0034] Furthermore, when the receiving end and the buffer area receive audio and video data respectively, the network status of the first channel and the second channel are monitored and compared according to a preset frequency, and the audio and video data transmission mode is selected according to the comparison result, which includes the following steps: Step S111: respectively monitoring the network status of the first channel and the second channel; Step S112: comparing the network status of the first channel and the second channel; if the network status of the first channel is better than that of the second channel, executing step S113; if the network status of the second channel is better than that of the first channel, executing step S114; Step S113: the receiving end receives the audio and video data within a period transmitted by the transmitting end through the first channel; at the same time, the buffer area receives and stores the audio and video data within a period transmitted by the transmitting end through the second channel; Step S114: the receiving end receives the audio and video data within a period transmitted by the sending end through the second channel; at the same time, the buffer area receives and stores the audio and video data within a period transmitted by the sending end through the first channel.

[0035] It is worth noting that, taking the case where the network status of the second channel is better than that of the first channel as an example, after switching channels, the receiving end and the transmitting end transmit audio and video data through the second channel, and the buffer area and the transmitting end transmit audio and video data through the first channel, which may easily lead to the fact that the missing audio and video data packets of the receiving end cannot be supplemented by the replacement data packets of the buffer area, thereby leading to the phenomenon that the receiving end cannot integrate the complete audio and video data. Therefore, in this embodiment, after switching channels, the audio and video data sent by the transmitter to the buffer area also includes redundant data packets; the network status of the channel between the transmitter and the buffer area is monitored and evaluated in real time, and the proportion of redundant data packets is adjusted according to the network status to ensure that there are enough replacement data packets in the buffer area for the receiver to retrieve. The purpose of this design is to strike a balance between ensuring data reliability and reducing delays; although this design may introduce a certain delay, compared with the traditional retransmission mechanism, it can more efficiently utilize network resources, reduce delays, and improve the reliability and stability of data transmission.

[0036] It should be noted that when transmitting redundant data packets, the sender gives priority to transmitting redundant data packets containing key frames.

[0037] Furthermore, in step S111, by formula: Among them, y represents the network status score of the channel; x i Represents the i-th network state parameter; a i represents the weight of the i-th network state parameter; Thus, the network status score values ​​of the first channel and the second channel are obtained; the network status score value of the first channel is compared with the network status score value of the second channel, and the audio and video data transmission method is selected according to the comparison result.

[0038] The network status parameters include bandwidth utilization rate, packet loss rate, and delay.

[0039] It should be explained that bandwidth utilization refers to the ratio of network bandwidth used to total bandwidth; packet loss rate refers to the ratio of the number of data packets lost during data transmission to the total number of data packets sent; latency refers to the time it takes for data to be transmitted from the sender to the receiver; It should be noted that, for the packet loss rate, the total number of sent packets is provided by the number list, and the number of lost packets is provided by the missing packet list.

[0040] Specifically, taking switching the transmission channel when it is monitored that the network status of the second channel is better than the network status of the first channel as an example, the steps include: Step S401: After the transmitting end completes data transmission within a cycle and the receiving end and the buffer area complete receiving audio and video data within a cycle, the receiving end disconnects from the first channel and the buffer area disconnects from the second channel; Step S402: The receiving end counts the lost data packets and schedules the replacement data packets in the buffer area, and at the same time the receiving end requests to connect to the second channel; the buffer area requests to connect to the first channel; Step S403: After the receiving end device and the buffer area complete the channel connection, the receiving end and the buffer area receive the audio and video data transmitted by the sending end through the second channel and the first channel respectively.

[0041] It should be noted that this design can ensure that the receiving device can integrate the data packets within the received cycle when switching the transmission channel, avoiding the delay phenomenon when switching channels of traditional audio and video.

[0042] Furthermore, in the present invention, the missing data packet list is also used to speed up the network status detection frequency of the first channel and the second channel.

[0043] Specifically, the network status detection frequency of the first channel and the second channel is accelerated according to the missing data packet list, and its implementation includes the following steps: Step S501: compare the number of labels in the missing data packet list with a preset threshold; if the number of labels on the list is greater than the preset threshold, execute step S503; if the number of labels on the list is less than the preset threshold, execute step S502; Step S502: speeding up the preset frequency, and monitoring the network status of the first channel and the network status of the second channel according to the new preset frequency; Step S503: Maintaining the preset frequency to monitor the network status of the first channel and the network status of the second channel.

[0044] It needs to be explained that the purpose of speeding up the monitoring frequency of the network status is to promptly discover abnormal situations in the network and dynamically adjust the transmission strategy of audio and video data according to the actual situation.

[0045] In the present invention, the buffer area is a circular buffer area, and the audio and video data are sequentially written into the circular buffer area according to the order of the data packet labels; It should be noted that the circular buffer area can be used cyclically without any unused space; when the buffer area is full of data, the new data will overwrite the earliest data, thereby improving space utilization; data can flow continuously in the buffer area and will not be lost due to the buffer area being full.

[0046] It should be noted that the cache area is located on the server.

[0047] like Figure 3 As shown, the present invention also provides an audio and video data optimization device for screen projection to implement the audio and video data optimization method for screen projection, including: A communication module, used to establish a first channel and a second channel between the sending end and the receiving end; A sending module, comprising a sending end device, wherein the sending end device is used to send audio and video data through the first channel and the second channel; The receiving module includes a receiving end device, wherein the receiving end device is used for: Acquire raw data through the first channel or the second channel; Count the missing audio and video data packets and obtain a list of missing data packets; Retrieving a replacement data packet from a buffer area according to the missing data packet list, and integrating the replacement data packet with the original data; A cache module, comprising a cache area, wherein the cache area is used to store substitute data; Monitoring module for: Monitor the network status of the first channel and the second channel respectively at a preset frequency and compare them, and select an audio and video data transmission method according to the comparison result; According to the packet loss of the original data within the period detected by the receiving end device, the frequency of monitoring the network status of the first channel and the second channel is increased; The receiving end establishes a connection with the buffer area through the monitoring module; Label module for: The audio and video data packets packaged by the sending device are numbered and a number list is made.

[0048] The present invention also provides a computer device, which includes a memory and a processor. The memory stores a computer program that can be run on the processor. When the computer program is executed by the processor, the above-mentioned audio and video data optimization method for screen projection is implemented.

[0049] The present invention also provides a computer-readable storage medium, in which a program is stored. When a computer executes the program, the above-mentioned audio and video data optimization method for screen projection is executed.

[0050] During the screen projection process, the traditional single-channel transmission of audio and video data can easily cause network congestion. In order to ensure the efficiency and stability of audio and video data transmission, the existing audio and video data transmission between the sending device and the receiving device is generally multi-channel concurrent transmission. Although this audio and video data transmission method can improve the transmission capacity to a certain extent, it will also increase the consumption of computing resources and easily generate a large amount of redundant data. In order to solve the problem of audio and video data transmission and the generation of a large amount of redundant data during screen projection, some audio and video transmissions use dynamic switching of transmission channels to avoid generating a large amount of redundant data while ensuring the stability and process of data transmission; however, in some scenarios with poor network environment or more wireless interference, it is easy to cause frequent switching of transmission channels, which will not only reduce transmission efficiency, but also increase the risk of transmission interruption.

[0051] In order to solve the above problems, the present invention sets up dual channels, one channel is used to transmit the audio and video data of the sending device to the receiving device, and the other channel transmits the audio and video data of the sending device to the buffer area for temporary storage; if the data packet received by the receiving device is lost, the receiving device retrieves the audio and video data packet that is the same as the lost packet from the buffer area to prevent the channel switching from causing delays when there is a small amount of packet loss; the present invention will also comprehensively evaluate the current network status, effectively identify abnormal conditions in the network and switch transmission channels, avoiding unnecessary mis-switching and frequent channel switching. In this way, the present invention effectively ensures the integrity of audio and video data, avoids problems such as audio and video playback jamming and incompleteness caused by data packet loss, thereby improving the user's viewing experience in the screen projection scenario.

[0052] So far, various embodiments of the present invention have been described in detail. In order to avoid obscuring the concept of the present invention, some details known in the art are not described. Those skilled in the art can fully understand how to implement the technical solution invented here based on the above description, and the scope of the present invention is defined by the attached claims.

Claims

1. A method for optimizing audio and video data for screen projection, characterized in that: Its implementation includes the following steps: Step S1: The receiving end receives the audio and video data within one cycle transmitted by the sending end; the audio and video data received by the receiving end is the original data; at the same time, the buffer area receives and stores the audio and video data within one cycle transmitted by the sending end; the audio and video data stored in the buffer area is the substitute data; The substitute data and the original data belong to the audio and video data within a cycle sent by the sending end; Step S2: The receiving end detects whether there is packet loss in the original data within the period, and if so, executes step S3, otherwise, executes step S4; Step S3: according to the packet loss situation, the substitute data or part of the substitute data is retrieved from the buffer area and integrated with the original data to obtain complete audio and video data; and step S5 is executed; Step S4: The original data is complete audio and video data; execute step S5; Step S5: the receiving end outputs the complete audio and video data; the receiving end enters the next audio and video data receiving cycle and repeats step S1; The implementation of the above step S3 includes the following steps: Step S31: the receiving end counts the missing audio and video data packets to obtain a missing data packet list; the missing data packet list includes the total number of missing audio and video data packets and the missing audio and video data packet numbers; Step S32: the receiving end retrieves a replacement data packet from the buffer area according to the missing data packet list, and integrates the replacement data packet into the original data to obtain complete audio and video data; The replacement data includes a replacement data packet, and the replacement data packet number is the same as the missing audio and video data packet number on the missing data packet list.

2. The method for optimizing audio and video data for screen projection according to claim 1, characterized in that: In the step S1, the receiving end receives the audio and video data within a period transmitted by the sending end through the first channel; At the same time, the buffer area receives and stores the audio and video data within a period transmitted by the transmitter through the second channel.

3. The method for optimizing audio and video data for screen projection according to claim 2, characterized in that: When the receiving end and the buffer area receive audio and video data respectively, the network status of the first channel and the second channel are monitored and compared according to the preset frequency, and the audio and video data transmission mode is selected according to the comparison result, which includes the following steps: Step S111: respectively monitoring the network status of the first channel and the second channel; Step S112: comparing the network status of the first channel and the second channel; if the network status of the first channel is better than that of the second channel, executing step S113; if the network status of the second channel is better than that of the first channel, executing step S114; Step S113: the receiving end receives the audio and video data within a period transmitted by the transmitting end through the first channel; at the same time, the buffer area receives and stores the audio and video data within a period transmitted by the transmitting end through the second channel; Step S114: the receiving end receives the audio and video data within a period transmitted by the sending end through the second channel; at the same time, the buffer area receives and stores the audio and video data within a period transmitted by the sending end through the first channel.

4. The method for optimizing audio and video data for screen projection according to claim 3, characterized in that: In step S111, the formula: Among them, y represents the network status score of the channel; x i Represents the i-th network state parameter; a i represents the weight of the i-th network state parameter; Thus, the network status scores of the first channel and the second channel are obtained; the network status score of the first channel is compared with the network status score of the second channel, and the audio and video data transmission mode is selected according to the comparison result; The network status parameters include bandwidth utilization rate, packet loss rate, and delay.

5. The method for optimizing audio and video data for screen projection according to claim 2, characterized in that: The first channel includes a first IP address, and the second channel includes a second IP address; the sending end device establishes connections with the receiving end and the buffer area through the first IP address and the second IP address respectively.

6. The method for optimizing audio and video data for screen projection according to claim 1, characterized in that: The buffer area is a circular buffer area, and the audio and video data are written into the circular buffer area in sequence according to the order of data packet labels.

7. The method for optimizing audio and video data for screen projection according to claim 1, characterized in that: In step S1, the transmitting end encodes and packages the video and audio and labels them, and the specific steps include: Step S101: the transmitting end device encodes the video and the audio respectively to obtain a video data segment and an audio data segment; Step S102: Packing the video data segments and audio data segments in one cycle according to the data volume to obtain a plurality of video data packets and a plurality of audio data packets; Step S103: label each data packet and list the label numbers; Step S104: After the sending end completes sending the audio and video data packets and the number list within the cycle, step S101 is repeated to package and number the audio and video data within the next cycle; The number list is sent from the sending end to the receiving end, and the receiving end counts the missing audio and video data packets through the number list.

8. An audio and video data optimization device for screen projection, characterized in that: To implement a method for optimizing audio and video data for screen projection according to any one of claims 1 to 7, comprising: A communication module, used to establish a first channel and a second channel between the sending end and the receiving end; A sending module, comprising a sending end device, wherein the sending end device is used to send audio and video data through the first channel and the second channel; The receiving module includes a receiving end device, wherein the receiving end device is used for: Acquire raw data through the first channel or the second channel; Count the missing audio and video data packets and obtain a list of missing data packets; Retrieving a replacement data packet from a buffer area according to the missing data packet list, and integrating the replacement data packet with the original data; A cache module, comprising a cache area, wherein the cache area is used to store substitute data; Monitoring module for: Monitor the network status of the first channel and the second channel respectively at a preset frequency and compare them, and select an audio and video data transmission method according to the comparison result; According to the packet loss of the original data within the period detected by the receiving end device, the frequency of monitoring the network status of the first channel and the second channel is increased; The receiving end establishes a connection with the buffer area through the monitoring module; Label module for: The audio and video data packets packaged by the sending device are numbered and a number list is made.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, it implements the audio and video data optimization method for screen projection as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the computer executes the program, a method for optimizing audio and video data for screen projection as described in any one of claims 1 to 7 is executed.

Images