Wireless terminal and method for sending and receiving audio and video data

Abstract

The present application discloses a wireless terminal, a method for transmitting audio and video data, a method for receiving audio and video data, and an audio and video data transmission system. The first wireless terminal includes an acquisition module for acquiring first data, which is raw audio and video data; and a first WiFi module, which includes a first encoding unit and a transmitting unit, wherein the first encoding unit is configured to encode the first data and generate second data; and the transmitting unit is configured to transmit the second data to a second wireless terminal.

Description

Technical Field

[0001] The present application relates to the field of audio and video data transmission, and in particular to a wireless terminal, a method for sending audio and video data, a method for receiving audio and video data, and an audio and video communication system. Background Art

[0002] In related technologies, during the process of encoding audio and video data by the sending device, transmission over the wireless air interface, and decoding by the receiving device, transmission delays may occur due to factors such as the hardware of the audio and video data sending device or the audio data receiving device and the link status of the wireless air interface, resulting in audio and video playback interruptions. Summary of the Invention

[0003] In view of this, the embodiments of the present application provide a wireless terminal, a method for sending audio and video data, a method for receiving audio and video data, and an audio and video data transmission system, so as to at least solve the problem of transmission delay generated when transmitting video data or audio data in related technologies, resulting in audio and video playback jamming.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] An embodiment of the present application provides a method for sending audio and video data in a first wireless terminal, wherein the first wireless terminal includes a collection module and a first WiFi module, wherein the method includes:

[0006] Collecting first data based on the acquisition module; the first data is original audio and video data;

[0007] encoding the first data based on a first encoding unit in the first WiFi module to generate the second data;

[0008] The second data is sent to the second wireless terminal based on the sending unit in the first WiFi module.

[0009] In the above solution, the encoding process of the first data by the first encoding unit in the first WiFi module to generate the second data includes:

[0010] If the encoding capability of the first encoding unit meets the encoding requirement of the first data, the first encoding unit is started to encode the first data to generate the second data.

[0011] In the above solution, the sending unit in the first WiFi module sends the second data to the second wireless terminal, including:

[0012] The wireless air interface status is monitored and the air interface status is fed back to the first encoding unit, so that the first encoding unit determines the encoding bit rate according to the air interface status.

[0013] In the above solution, the encoding process of the first data by the first encoding unit in the first WiFi module to generate the second data further includes:

[0014] Each frame of the first data is divided into at least two slices of data according to the air interface state; and the first data is encoded using a single slice of data as a unit to generate the second data.

[0015] An embodiment of the present application further provides a method for receiving audio and video data in a second wireless terminal, wherein the second wireless terminal includes a second WiFi module and a playback module, wherein the method includes:

[0016] receiving the second data based on a wireless link established between the receiving unit of the second WiFi module and the sending unit of the first WiFi module;

[0017] Decoding the second data by a decoding unit based on the second WiFi module to obtain third data;

[0018] The playing module plays the corresponding audio and video according to the third data.

[0019] In the above solution, the decoding unit in the second WiFi module decodes the second data to obtain the third data, including: if it is detected that the second data is normally encoded data, controlling the decoding unit to decode the second data.

[0020] In the above solution, the decoding unit in the second WiFi module decodes the second data to obtain the third data, and further includes: determining a resolution corresponding to the second data;

[0021] The playing module playing the corresponding audio and video includes: playing the audio data corresponding to the third data according to the resolution.

[0022] In the above solution, the decoding unit based on the second WiFi module decodes the second data to obtain third data, and further includes: if it is detected that the second data is encoding abnormality data, controlling the receiving unit to send data abnormality feedback to the sending unit, so that the first WiFi module re-encodes the first data and sends the second data.

[0023] An embodiment of the present application further provides a first wireless terminal, configured to send audio and video data, including:

[0024] an acquisition module, configured to acquire first data, where the first data is original audio and video data; and

[0025] A first WiFi module includes a first encoding unit and a sending unit, wherein the first encoding unit is used to encode the first data and generate second data; and the sending unit is used to send the second data to the second wireless terminal.

[0026] In the above solution, the first wireless terminal further includes:

[0027] The processor is configured to start the first encoding unit to encode the first data to generate the second data if the processor determines that the encoding capability of the first encoding unit meets the encoding requirements of the first data.

[0028] In the above scheme, the sending unit is used to establish a wireless link with the receiving unit in the second wireless terminal to transmit the second data. The sending unit is also used to monitor the air interface status of the wireless link and feed back the air interface status to the first encoding unit so that the first encoding unit determines the encoding bit rate according to the air interface status.

[0029] In the above solution, the first encoding unit is further used to divide each frame of data in the first data into at least two slices of data according to the air interface state; and encode the first data in units of a single slice of data to generate the second data.

[0030] In the above solution, the processor and the acquisition module are electrically connected to the first WiFi module respectively.

[0031] In the above solution, the acquisition module, the processor and the first WiFi module are electrically connected in sequence; the processor includes a second encoding unit; and the second encoding unit is used to encode the first data if the processor determines that the second encoding unit is to encode the first data.

[0032] The embodiment of the present application further provides a second wireless terminal for receiving audio and video data, which is applied to the first wireless terminal and includes:

[0033] A second WiFi module includes a receiving unit and a decoding unit, wherein the receiving unit is used to receive the second data; the decoding unit is used to decode the second data to obtain third data; and

[0034] A playing module is used to play the audio and video corresponding to the third data.

[0035] In the above solution, the second WiFi module is further configured to control the decoding unit to decode the second data if it is detected that the second data is normally encoded data.

[0036] In the above solution, the decoding unit is further used to determine the resolution corresponding to the second data; and the playing module is further used to play the audio and video corresponding to the third data according to the resolution.

[0037] In the above solution, the second WiFi module is further configured to control the receiving unit to send data abnormality feedback to the sending unit if it is detected that the second data is encoding abnormality data, so that the first WiFi module re-encodes the first data and sends the second data.

[0038] An embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0039] The embodiment of the present application also provides an audio and video data transmission system, including a first wireless terminal and a second wireless terminal,

[0040] The first wireless terminal is used to send audio and video data, and includes: an acquisition module for acquiring first data, where the first data is original audio and video data; and a first WiFi module, where the first WiFi module includes a first encoding unit and a transmitting unit, where the first encoding unit is used to encode the first data and generate second data; and the transmitting unit is used to transmit the second data to the second wireless terminal;

[0041] The second wireless terminal is used to receive audio and video data, and the second wireless terminal includes: a second WiFi module, including a receiving unit and a decoding unit, the receiving unit is used to receive the second data; the decoding unit is used to decode the second data to obtain third data; and a playing module is used to play the audio and video corresponding to the third data.

[0042] In an embodiment of the present application, the first wireless terminal can complete the encoding of audio and video data in the first encoding unit in the first WiFi module, and can directly send the audio and video data through the sending unit in the first WiFi module, thereby reducing the transmission link of the audio and video data in the first wireless terminal, thereby shortening the hardware transmission path of the audio and video data in the first wireless terminal, reducing the transmission delay of the audio and video data, improving the transmission efficiency of the audio and video data, and avoiding the phenomenon of jamming during audio and video playback. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1A schematic structural diagram of a first wireless terminal provided in one embodiment of the present application;

[0044] Figure 2 A schematic structural diagram of a first wireless terminal provided in yet another embodiment of the present application;

[0045] Figure 3 A schematic diagram of a first wireless terminal sending audio and video data according to an embodiment of the present application;

[0046] Figure 4 A schematic diagram of feedback of the air interface status of a wireless link provided in one embodiment of the present application;

[0047] Figure 5 A schematic structural diagram of a first wireless terminal provided in one embodiment of the present application;

[0048] Figure 6 A schematic structural diagram of a first wireless terminal provided in yet another embodiment of the present application;

[0049] Figure 7 A schematic structural diagram of a second wireless terminal provided in one embodiment of the present application;

[0050] Figure 8 A schematic diagram of an implementation flow of a method for transmitting audio and video data in an audio and video data transmitting device provided in an embodiment of the present application;

[0051] Figure 9 A schematic diagram of an implementation flow of a method for receiving audio and video data in a second wireless terminal provided in an embodiment of the present application;

[0052] Figure 10 A structural diagram of an audio and video data transmission system provided in one embodiment of the present application;

[0053] Figure 11 A structural diagram of an audio and video communication system provided in yet another embodiment of the present application. DETAILED DESCRIPTION

[0054] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0056] It should be noted that the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0057] In addition, in the embodiments of the present application, "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0058] Before describing the technical solutions of the embodiments of the present application in detail, a brief description of the audio and video data transmission method in the related art is first given.

[0059] In the related art, the delay of audio and video transmission can be divided into three types. The first is the delay at the sending terminal, which can be further divided into the delay caused by audio and video data acquisition, pre-processing, and encoding. The delays in these three cases are mainly related to the hardware capabilities and the encoding algorithm used. For example, the audio sound card signal conversion delay is approximately between 1-30ms, and the encoding delay is approximately between 2.5-60ms. In addition, the delay at the first wireless terminal also includes the transmission delay of the audio and video data after processing through the main control central processing unit (CPU) to the wireless transmission module, and the delay inside the wireless transmission module. The second type of delay is the wireless air interface transmission delay, which depends on the link status of the wireless air interface. The third type of delay is the delay at the second wireless terminal, which depends on the internal delay of the wireless receiving module, the decoding delay, and the display delay. In addition, no matter whether the first wireless terminal or the second wireless terminal is in operation, the CPU and cache will process requests from multiple applications and external devices at the same time. When the CPU is occupied by other requests, the processing of audio and video requests will be delayed.

[0060] Based on this, in an embodiment of the present application, by adjusting the hardware path of audio and video data transmission, the transmission delay of audio and video data when transmitted at the hardware level is reduced.

[0061] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] The embodiment of the present application provides a first wireless terminal 10 for sending audio and video data. Figure 1 This is a structural diagram of the first wireless terminal 10 provided in an embodiment of the present application. Figure 1 As shown, the first wireless terminal 10 includes:

[0063] The acquisition module 101 is configured to acquire first data, where the first data is original audio and video data; and

[0064] The first WiFi module 102 includes a first encoding unit 1021 and a sending unit 1022 . The first encoding unit 1021 is used to encode the first data and generate second data; the sending unit 1022 is used to send the second data to the second wireless terminal 70 .

[0065] In an embodiment of the present application, the acquisition module 101 is used to acquire raw audio and video data. It is understandable that the acquisition module 101 may include an image acquisition unit and / or a sound acquisition unit. The image acquisition unit is used to sense ambient light and acquire raw image data; the image acquisition unit may be an image sensor, for example, a CCD image sensor or a CMOS image sensor. The sound acquisition unit is used to acquire audio data, and the sound acquisition unit may be, for example, a microphone. The acquisition module 101 can acquire a large amount of raw audio and video data (i.e., first data). By encoding and compressing a large amount of first data, the amount of audio and video data to be transmitted can be reduced, thereby improving the transmission speed of the audio and video data. In actual applications, the first WiFi module 102 can be electrically connected to the acquisition module 101. After the acquisition module 101 acquires the first data, the first data can be transmitted to the first WiFi module 102 via the electrical connection between the acquisition module 101 and the first WiFi module 102, such as a Mobile Industry Processor Interface (MIPI). The first encoding unit 1021 in the first WiFi module 102 can encode the first data acquired by the acquisition module 101 to generate second data, and then transmit the second data to the second wireless terminal via the transmitting unit 1022. Since the first data acquired by the acquisition module 101 is directly transmitted to the first WiFi module 102, the first encoding unit 1021 in the first WiFi module 102 encodes the first data, and the transmitting unit 1022 in the first WiFi module 102 transmits the encoded second data, there is no need to transmit the first data to other hardware units. This greatly reduces the transmission path of audio and video data between different hardware units, thereby reducing transmission delay at the hardware level.

[0066] See Figure 2In some embodiments of the present application, the first wireless terminal 10 further includes a processor 201, which can be electrically connected to the first WiFi module 102 and configured to determine whether the encoding capability of the first encoding unit 1021 meets the encoding requirements of the first data. If the processor 201 determines that the encoding capability of the first encoding unit 1021 meets the encoding requirements of the first data, the first encoding unit 1021 is started to encode the first data to generate the second data.

[0067] It is understandable that different data imposes different encoding requirements on the encoding capabilities of the first encoding unit 1021. Therefore, before encoding the first data, the processor 201 needs to determine whether the first encoding unit 1021 has the ability to encode the first data. The processor 201 can determine the encoding capabilities of the first encoding unit 1021 through command parameters. If the processor 201 determines that the encoding capabilities of the first encoding unit 1021 can meet the encoding requirements of the first data, the first encoding unit 1021 is activated to encode the first data. The acquisition module 101 can directly transmit the first data to the first WiFi module 102, eliminating the need to transmit the first data to other hardware units. In actual applications, the processor 201 can detect the encoding capabilities of the first encoding unit 1021 when the first wireless terminal 10 is powered on. The encoding capabilities of the first encoding unit 1021 can be determined based on indicators such as the maximum data volume, compression capacity, and compliance with audio and video encoding formats that the first encoding unit 1021 can process.

[0068] In one embodiment, if Figure 3 As shown, the sending unit 1022 is used to establish a wireless link with the receiving unit 7011 in the second wireless terminal 70 to transmit the second data. The sending unit 1022 is also used to monitor the air interface status of the wireless link and feed back the air interface status to the first encoding unit 1021 so that the first encoding unit 1021 determines the encoding bit rate according to the air interface status.

[0069] Here, when the sending unit 1022 sends the second data to the second wireless terminal 70, it is necessary to establish a wireless link with the receiving unit 7011 of the second wireless terminal 70, and transmit the second data to the second wireless terminal 70 based on the established wireless link. In actual applications, the air interface state of the wireless link can affect the transmission efficiency of the second data. For example, when the air interface interference is large, the transmission state of the wireless link will be poor, thereby causing the accumulation of second data packets and a large transmission delay. Therefore, it is necessary to monitor the air interface state of the wireless link through the sending unit 1022 and feed back the monitoring results to the first encoding unit 1021. The first encoding unit 1021 can adjust the encoding bit rate of the first data according to the air interface state of the wireless link, such as Figure 4 As shown, Figure 4 A schematic diagram of feedback on the air interface status of a wireless link is shown. For example, if the air interface status of the wireless link is poor, the first encoding unit 1021 can lower the encoding bit rate of the first data, so that the first encoding unit 1021 can encode the first data faster and avoid excessive accumulation of second data. If the air interface status of the wireless link is good, the first encoding unit 1021 can increase the encoding bit rate of the first data, thereby improving the transmission efficiency of audio and video data. In actual applications, the feedback time of the air interface status of the wireless link by the transmitting unit 1022 needs to be determined according to the current transmission rate. For example, if the current transmission rate is 30fps, the corresponding feedback time is 100-200ms. In actual applications, the resolution of the first data encoding process can also be adjusted in real time according to the air interface status of the wireless link. For example, if the air interface status of the wireless link is poor, the resolution of the first data encoding process can be lowered, and the amount of second data generated can be reduced. This is conducive to improving the transmission efficiency of the second data when the air interface status of the wireless link is poor, thereby reducing buffer accumulation during the transmission of audio and video data and reducing the transmission delay of audio and video data.

[0070] In one embodiment, the first encoding unit 1021 is further configured to divide each frame of the first data into at least two slices of data according to the air interface state; and perform encoding processing on the first data in units of a single slice of data to generate the second data.

[0071] Here, during the encoding process of the first data, the first encoding unit 1021 may divide each frame of the first data into multiple slices of data. The first encoding unit 1021 encodes the first data of each slice of data in units of slices, thereby reducing the amount of data required to be processed by the first encoding unit 1021 for a single encoding. Furthermore, after encoding the first data of the single slice of data, the first encoding unit 1021 generates second data of the single slice of data. The communication transmission unit 1022 transmits the second data of the single slice of data to the second wireless terminal 70, thereby improving the transmission efficiency of audio and video data. In particular, when the air interface status is poor, the transmission delay of audio and video data can be reduced. In practical applications, when the air interface status is good, the amount of audio and video data transmitted should be increased. Therefore, during the encoding process of the first data, the first encoding unit 1021 can directly encode the first data without further dividing each frame of the first data into multiple slices of data. This ensures smooth transmission of audio and video data while increasing the single data processing capacity of the first encoding unit 1021.

[0072] In one embodiment, if Figure 5 As shown, the processor 201 and the acquisition module 101 are electrically connected to the first WiFi module 102 respectively.

[0073] Here, the processor 201 and the acquisition module 101 are electrically connected to the first WiFi module 102 respectively, so that in terms of hardware connection, the first data collected by the acquisition module 101 can be directly transmitted to the first WiFi module 102, the first data is encoded by the first encoding unit 1021 in the first WiFi module 102, and the second data is sent by the sending unit 1022 in the first WiFi module 102, thereby realizing the shortest hardware transmission path between audio and video data acquisition and transmission, thereby reducing the transmission delay of audio and video data at the hardware level.

[0074] In one embodiment, if Figure 6 As shown, the acquisition module 101, the processor 201 and the first WiFi module 102 are electrically connected in sequence; the processor 201 includes a second encoding unit 2011; the second encoding unit 2011 is used to encode the first data if the processor 201 determines that the second encoding unit 2011 is to encode the first data.

[0075] Here, the acquisition module 101 is electrically connected to the processor 201, and the processor 201 is electrically connected to the first WiFi module 102. The processor 201 also includes a second encoding unit 2011. When the acquisition module 101 acquires the first data, the processor 201 determines whether the first encoding unit 1021 in the first WiFi module 102 encodes the first data. For example, it can be determined whether the encoding capability of the first encoding unit 1021 meets the encoding requirements of the first data. If the processor 201 determines that the first encoding unit 1021 encodes the first data, the first data is transmitted from the processor 201 to the first encoding unit 1021 and encoded by the first encoding unit 1021. Element 1021 completes the encoding processing of the first data. If the processor 201 determines that the second encoding unit 2011 is to encode the first data, the acquisition module 101 will transmit the acquired first data to the second encoding unit 2011. The second encoding unit 2011 encodes the first data to generate second data, and transmits the second data to the sending unit 1022 of the first WiFi module 102 to the second wireless terminal 70. Under this electrical connection, there is no need to increase or decrease the chip pins of the processor 201. The processor 201 can dynamically control the transmission of audio and video data at the hardware layer, and can reduce the data transmission delay at the hardware layer to a certain extent.

[0076] In the above embodiment, the acquisition module 101 is electrically connected to the processor 201; the processor 201 includes a second encoding unit 2011; the second encoding unit 2011 is used to encode the first data if the processor 201 determines that the second encoding unit 2011 is to encode the first data, thereby dynamically controlling the transmission of audio and video data at the hardware layer, reducing hardware changes, and reducing the delay of data transmission at the hardware layer to a certain extent.

[0077] The embodiment of the present application further provides a second wireless terminal 70, such as Figure 7 As shown, the second wireless terminal 70 includes:

[0078] A second WiFi module 701; the second WiFi module 701 includes a receiving unit 7011 and a decoding unit 7012; the receiving unit 7011 is used to establish a wireless link with the sending unit 1022 of the first WiFi module 102, and is used to receive second data transmitted by the wireless link; the decoding unit 7012 is used to decode the second data to obtain third data; and a playing module 702, which can be electrically connected to the second WiFi module 701, and is used to play audio and video corresponding to the third data.

[0079] Here, the second wireless terminal 70 includes a second WiFi module 701 and a playback module 702, wherein the second WiFi module 701 can establish a wireless link with the first WiFi module 102 of the first wireless terminal 10, the receiving unit 7011 of the second WiFi module 701 can receive the second data on the wireless link, the receiving unit 7011 can transmit the received second data to the decoding unit 7012 of the second WiFi module 701, the decoding unit 7012 can decode the second data to obtain third data, the second WiFi module 701 is electrically connected to the playback module 702, so that the decoding unit 7012 transmits the third data obtained by decoding the second data to the playback module 702, and the playback module 702 can play the audio and video corresponding to the third data. In the second wireless terminal 70 provided in the embodiment of the present application, after receiving the audio and video data sent by the first wireless terminal 10, the decoding unit 7012 directly decodes the audio and video data, reducing the hardware transmission path of the audio and video data in the second wireless terminal, reducing the internal delay of the audio and video data in the second wireless terminal, and thereby improving the transmission efficiency of the audio and video data.

[0080] In one embodiment, the second WiFi module 701 is further configured to control the decoding unit 7012 to decode the second data if it is detected that the second data is normally encoded data.

[0081] Here, the second WiFi module 701 can also detect whether the second data is normally encoded data during the process of receiving the second data. When the second data is normally encoded data, it indicates that there are no data errors, missing frames, and wrong frames in the second data during the encoding and generation process. The second WiFi module 701 controls the decoding unit 7012 to decode the second data, and the third data decoded by the decoding unit 7012 is complete, thereby ensuring that the playback module 702 can play the corresponding audio and video data normally, thereby reducing the display delay of the audio and video data receiving end.

[0082] In one embodiment, the decoding unit 7012 is further configured to determine a resolution corresponding to the second data; and the playing module 702 is further configured to play the audio and video data corresponding to the third data according to the resolution.

[0083] Here, in the process of decoding the second data by the decoding unit 7012, the resolution corresponding to the generated second data can be determined, wherein the resolution corresponding to the second data is the encoding parameter used when the first wireless terminal 10 encodes the first data. In the process of playing the audio and video corresponding to the third data by the playback module 702, the corresponding audio and video are played according to the resolution corresponding to the second data. When the resolution corresponding to the second data changes, the resolution of the played audio and video data will also change accordingly, so that the resolution can be adjusted in real time during the playback of the audio and video data. In actual applications, when the resolution of certain frames is reduced, it will not have a significant impact on the user's visual experience, and the audio and video are displayed in a timely and smooth manner, which reduces the display delay of the audio and video and improves the playback effect of the audio and video.

[0084] In one embodiment, the second WiFi module 701 is further configured to control the receiving unit 7011 to send data abnormality feedback to the sending unit 1022 if it is detected that the second data is encoding abnormality data, so that the first WiFi module 102 re-encodes the first data and sends the second data.

[0085] Here, during the process of the second WiFi module 701 detecting the second data, if the second data is abnormally encoded data, for example, the second data has frame drops or error frames, the corresponding decoded third data will have data errors, resulting in playback failure or playback interruption during the playback process. The receiving unit 7011 will feedback the data abnormality of the second data to the sending unit 1022 of the first wireless terminal 10, so that the first WiFi module 102 of the first wireless terminal 10 re-encodes the corresponding first data and re-sends the second data to the second wireless terminal 70, ensuring that the second data received by the second wireless terminal 70 is normally encoded data, thereby avoiding playback failure or playback interruption during audio and video playback.

[0086] In the above embodiment, the second WiFi module 701 is further configured to control the communication receiving unit 7011 to send data anomaly feedback to the communication sending unit if it is detected that the second data is abnormally encoded data, so that the first WiFi module 102 re-encodes the first data and sends the second data. This enables real-time feedback of frame errors or frame losses, ensuring that the second wireless terminal 70 can receive audio and video data packets with normal encoding data, thereby reducing the display delay of audio and video data during playback.

[0087] The embodiment of the present application also provides a method for sending audio and video data, which is applied to a first wireless terminal 10, wherein the first wireless terminal 10 includes a collection module 101 and a first WiFi module 102. Figure 8Shown, including:

[0088] S801: Collect first data based on the collection module 101; the first data is original audio and video data.

[0089] In the embodiment of the present application, the structural diagram of the first wireless terminal 10 is as follows: Figure 1 As shown, a large amount of raw audio and video data (first data) can be acquired through the acquisition module 101, wherein the first data is raw audio and video data. It can be understood that the acquisition module 101 may include an image acquisition unit and / or a sound acquisition unit, wherein the image acquisition unit is used to sense ambient light and acquire raw image data. The image acquisition unit may be an image sensor, for example, a CCD image sensor or a CMOS image. The sound acquisition unit is used to acquire audio data, and the sound acquisition unit may be, for example, a microphone.

[0090] S802: Encode the first data based on the first encoding unit 1021 in the first WiFi module 102 to generate the second data.

[0091] Here, the large amount of first data collected by the acquisition module 101 needs to be encoded and compressed, thereby reducing the amount of audio and video data that needs to be transmitted and further improving the transmission speed of the audio and video data. The first encoding unit 1021 in the first WiFi module 102 is responsible for encoding the first data to generate second data. In actual applications, the first WiFi module 102 can be electrically connected to the acquisition module 101. After the acquisition module 101 collects the first data, the first data can be transmitted to the first WiFi module 102 via the electrical connection between the acquisition module 101 and the first WiFi module 102, for example, a MIPI interface.

[0092] S803: Send the second data to the second wireless terminal based on the sending unit 1022 in the first WiFi module 102.

[0093] Here, the generated second data needs to be transmitted to the second wireless terminal 70. The second data is transmitted to the second wireless terminal 70 via the transmitting unit 1022 in the first WiFi module 102. In actual applications, a wireless link can be established between the first WiFi module 102 of the first wireless terminal 10 and the second WiFi module 701 of the second wireless terminal 70, thereby enabling the transmission and reception of the second data via the wireless link. In actual applications, the wireless communication method between the first wireless terminal 10 and the second wireless terminal 70 can be any of WiFi, Bluetooth, Zigbee, and UWB. Because the first data collected by the acquisition module 101 is directly transmitted to the first WiFi module 102, the first encoding unit 1021 in the first WiFi module 102 encodes the first data, and the transmitting unit 1022 in the first WiFi module 102 transmits the encoded second data. There is no need to transmit the first data to other hardware units, which greatly reduces the transmission paths of audio and video data between different hardware units, thereby reducing transmission latency at the hardware level.

[0094] In the above embodiment, the encoding of audio and video data can be completed based on the first encoding unit 1021 in the first WiFi module 102, and the audio and video data can be directly sent based on the sending unit 1022 in the first WiFi module 102, thereby reducing the transmission link of the audio and video data in the first wireless terminal 10, thereby shortening the hardware transmission path of the audio and video data in the first wireless terminal 10, reducing the transmission delay of the audio and video data, improving the transmission efficiency of the audio and video data, and avoiding the phenomenon of jamming during audio and video playback.

[0095] In one embodiment, encoding the first data based on the first encoding unit 1021 in the first WiFi module 102 and generating the second data includes:

[0096] If the encoding capability of the first encoding unit 1021 meets the encoding requirement of the first data, the first encoding unit 1021 is started to encode the first data to generate the second data.

[0097] Here, it is understood that different data puts forward different encoding requirements for the encoding capability of the first encoding unit 1021. Therefore, before encoding the first data, it is necessary to determine whether the encoding capability of the first encoding unit 1021 meets the encoding requirements of the first data. If the encoding capability of the first encoding unit 1021 meets the encoding requirements of the first data, then the first encoding unit 1021 can be started to encode the first data to generate second data, thereby obtaining the second data that matches the encoding requirements. Figure 2In some embodiments of the present application, the first wireless terminal 10 further includes a processor 201, which can determine the encoding capability of the first encoding unit 1021 through command parameters. In actual applications, the processor 201 can detect the encoding capability of the first encoding unit 1021 when the first wireless terminal 10 is powered on, wherein the encoding capability of the first encoding unit 1021 can be determined based on indicators such as the maximum amount of data that the first encoding unit 1021 can process, the compression amount, and whether it complies with the audio and video encoding format.

[0098] In one embodiment, the sending of the second data to the second wireless terminal 70 based on the sending unit 1022 in the first WiFi module 102 includes:

[0099] The wireless air interface status is monitored and the air interface status is fed back to the first encoding unit 1021, so that the first encoding unit 1021 determines the encoding bit rate according to the air interface status.

[0100] Here, the second data can be transmitted based on the wireless link established between the transmitting unit 1022 in the first WiFi module 102 and the receiving unit 7011 of the second wireless terminal 70. The air interface status of the wireless link can affect the transmission efficiency of the second data. For example, when the air interface interference is large, the transmission status of the wireless link may be poor, resulting in the accumulation of second data packets and a large transmission delay. To this end, in the first wireless terminal 10, the transmitting unit 1022 can monitor the air interface status of the wireless link and feedback the air interface status to the first encoding unit 1021. The encoding bit rate of the first data is adjusted based on the wireless air interface status so that the encoding bit rate can adapt to the air interface status. For example, if the air interface status of the wireless link is poor, the first encoding unit 1021 can reduce the encoding bit rate of the first data, thereby increasing the encoding speed of the first encoding unit 1021 and avoiding excessive accumulation of second data. If the air interface status of the wireless link is good, the first encoding unit 1021 can increase the encoding bit rate of the first data, thereby improving the transmission efficiency of the audio and video data. In practical applications, the feedback time of the air interface status of the wireless link of the transmitting unit 1022 needs to be determined according to the current transmission rate. For example, if the current transmission rate is 30fps, the corresponding feedback time is 100-200ms. In practical applications, the resolution of the first data encoding process can also be adjusted in real time according to the air interface status of the wireless link. For example, if the air interface status of the wireless link is poor, the resolution of the first data encoding process can be lowered, and the amount of the corresponding second data generated is reduced, which is conducive to improving the transmission efficiency of the second data when the air interface status of the wireless link is poor.

[0101] In one embodiment, the encoding process of the first data based on the first encoding unit 1021 in the first WiFi module 102 to generate the second data further includes:

[0102] Each frame of the first data is divided into at least two slices of data according to the air interface state; and the first data is encoded using a single slice of data as a unit to generate the second data.

[0103] Here, to further improve the transmission and processing efficiency of audio and video data, each frame of data in the first data can be divided into at least two slices of data. The first encoding unit 1021 can encode the first data of a single slice of data in units of single slices, thereby reducing the amount of data that the first encoding unit 1021 needs to process for a single encoding. In actual applications, after the first encoding unit 1021 generates the second data of the single slice of data, the transmitting unit 1022 can transmit the second data of the single slice of data to the second wireless terminal 70, thereby improving the transmission efficiency of the audio and video data. In particular, when the air interface status is poor, the transmission delay of the audio and video data can be reduced. In actual applications, whether each frame of data in the first data is divided into at least two slices of data can be determined according to the air interface status. When the air interface status is in good condition, the amount of audio and video data sent should be increased. Therefore, the first encoding unit 1021 can directly encode the first data during the process of encoding the first data, without further dividing each frame of the first data into multiple slices of data. This can ensure the smooth transmission of audio and video data while improving the single data processing capacity of the first encoding unit 1021.

[0104] The embodiment of the present application also provides a method for receiving audio and video data in a second wireless terminal, wherein the second wireless terminal 70 includes a second WiFi module 701 and a playback module 702. The method is as follows: Figure 9 Shown, including:

[0105] S901 : Receive the second data based on the wireless link established between the receiving unit 7011 of the second WiFi module 701 and the sending unit 1022 of the first WiFi module 102 .

[0106] In the embodiment of the present application, the structural diagram of the second wireless terminal 70 is as follows: Figure 7 As shown, during the transmission of audio and video data, the receiving unit 7011 of the second WiFi module 701 establishes a wireless link with the sending unit 1022 of the first WiFi module 102, and the second data is transmitted on the established wireless link. The second data is sent from the sending unit 1022 and received by the receiving unit 7011, thereby realizing the reception of the second data.

[0107] S902: The decoding unit 7012 of the second WiFi module 701 decodes the second data to obtain third data.

[0108] Here, the received second data is encoded and processed by the original audio and video data, so the second data needs to be decoded to realize the corresponding audio and video playback, wherein the decoding of the second data is completed by the decoding unit 7012 on the second WiFi module 701. In actual application, through electrical connection, the receiving unit 7011 can transmit the received second data to the decoding unit 7012 of the second WiFi module 701, so that the second WiFi module 701 can be responsible for receiving and decoding the second data.

[0109] In one embodiment, the decoding unit 7012 in the second WiFi module 701 decodes the second data to obtain the third data, including: if it is detected that the second data is normally encoded data, controlling the decoding unit 7012 to decode the second data.

[0110] Here, the second data may contain data errors, missing frames, and wrong frames during the encoding process. Before decoding the second data, it is possible to further determine whether the second data is properly encoded data. This can ensure that the third data decoded by the decoding unit 7012 is complete data and will not be stuck during playback, thereby reducing the display delay of the audio and video data receiving end. In actual applications, the second WiFi module 701 can detect whether the second data is properly encoded data during the process of receiving the second data.

[0111] In one embodiment, the decoding unit 7012 based on the second WiFi module 701 decodes the second data to obtain third data, and further includes: if it is detected that the second data is encoding abnormality data, controlling the receiving unit 7011 to send data abnormality feedback to the sending unit 1022, so that the first WiFi module 102 re-encodes the first data and sends the second data.

[0112] Here, if it is detected that the second data is encoded abnormally, for example, the second data has frame drops or error frames, the corresponding decoded third data will have data errors, resulting in playback failure or playback interruption during the playback process. In order to ensure normal playback of the video data, it is necessary to obtain the second data of the normally encoded data, and therefore it is necessary to feedback the data abnormality to the first wireless terminal 10. Since the first wireless terminal 10 and the second wireless terminal 70 establish communication through the sending unit 1022 and the receiving unit 7011, the receiving unit 7011 can be controlled to feedback the data abnormality of the second data to the sending unit 1022 of the first wireless terminal 10, so that the first WiFi module 102 of the first wireless terminal 10 re-encodes the corresponding first data and re-sends the second data to the second wireless terminal 70, ensuring that the second data received by the second wireless terminal 70 is normally encoded data, thereby avoiding playback failure or playback interruption during audio and video playback.

[0113] S903: The playing module 702 plays the corresponding audio and video according to the third data.

[0114] Here, the second WiFi module 701 is electrically connected to the playback module 702, so that the decoding unit 7012 transmits the third data obtained by decoding the second data to the playback module 702, and the playback module 702 can play the audio and video corresponding to the third data. In this embodiment of the present application, the second WiFi module 701 of the second wireless terminal 70 can receive and encode the second data, and the playback module 702 can play the audio and video data, which reduces the hardware transmission path of the audio and video data in the second wireless terminal 70, reduces the internal delay of the audio and video data in the second wireless terminal 70, and thus improves the transmission efficiency of the audio and video data.

[0115] In one embodiment, the decoding unit 7012 in the second WiFi module 701 decodes the second data to obtain the third data, further comprising:

[0116] Determine a resolution corresponding to the second data.

[0117] Here, in the process of decoding the second data by the decoding unit 7012 based on the second WiFi module 701, a resolution corresponding to the second data can be generated, wherein the resolution corresponding to the second data is the encoding parameter used when the first wireless terminal 10 encodes the first data.

[0118] The playing module 702 plays the corresponding audio and video, including: playing the audio data corresponding to the third data according to the resolution.

[0119] Here, when the playback module 702 plays the corresponding audio and video according to the third data, the corresponding audio and video are played according to the resolution corresponding to the second data. When the resolution corresponding to the second data changes, the resolution of the played audio and video data will also change accordingly, thereby enabling real-time resolution adjustment during the playback of the audio and video data. In actual applications, when the resolution of certain frames is reduced, it will not have a significant impact on the user's visual experience, and the audio and video will be displayed in a timely and smooth manner, reducing the display delay of the audio and video and improving the playback quality of the audio and video.

[0120] The embodiment of the present application also provides an audio and video data transmission system 100, such as Figure 10 As shown, it includes a first wireless terminal 10 and a second wireless terminal 70, wherein,

[0121] The first wireless terminal 10 includes a collection module 101 and a first WiFi module 102. The collection module 101 is used to collect first data, which is original audio and video data. The first WiFi module 102 includes a first encoding unit 1021 and a sending unit 1022. The first encoding unit 1021 is used to encode the first data and generate second data. The sending unit 1022 is used to send the second data to the second wireless terminal 70.

[0122] The second wireless terminal 70 is used to receive audio and video data, including a second WiFi module 701 and a playback module 702; the second WiFi module 701 includes a receiving unit 7011 and a decoding unit 7012, the receiving unit 7011 is used to receive the second data; the decoding unit 7012 is used to decode the second data to obtain third data, and the playback module 702 is used to play the audio and video corresponding to the third data.

[0123] In one embodiment, if Figure 11 As shown, the first wireless terminal 10 in the audio and video data transmission system 100 further includes:

[0124] The processor 201 is configured to start the first encoding unit 1021 to encode the first data to generate the second data if the processor 201 determines that the encoding capability of the first encoding unit 1021 meets the encoding requirements of the first data.

[0125] In one instance, the sending unit 1022 is used to establish a wireless link with the receiving unit 7011 in the second wireless terminal 70 to transmit the second data. The sending unit 1022 is also used to monitor the air interface status of the wireless link and feed back the air interface status to the first encoding unit 1021 so that the first encoding unit 1021 determines the encoding bit rate based on the air interface status.

[0126] In one example, the first encoding unit 1021 is further configured to divide each frame of the first data into at least two slices of data according to the air interface state; and perform encoding processing on the first data in units of a single slice of data to generate the second data.

[0127] In one example, the processor 201 and the acquisition module 101 are electrically connected to the first WiFi module 102 respectively.

[0128] In one instance, the acquisition module 101, the processor 201 and the first WiFi module 102 are electrically connected in sequence; the processor 201 includes a second encoding unit 2011; the second encoding unit 2011 is used to encode the first data if the processor 201 determines that the second encoding unit 2011 is to encode the first data.

[0129] In one example, the second WiFi module 701 is further configured to control the decoding unit 7012 to decode the second data if it is detected that the second data is normally encoded data.

[0130] In one example, the decoding unit 7012 is further configured to determine a resolution corresponding to the second data; and the playing module 702 is further configured to play the audio and video corresponding to the third data according to the resolution.

[0131] In one example, if the second WiFi module 701 detects that the second data is encoding abnormal data, it is further configured to control the receiving unit 7011 to send data abnormality feedback to the sending unit 1022, so that the first WiFi module 102 re-encodes the first data and sends the second data.

[0132] In an exemplary embodiment, the present application also provides a storage medium, namely, a computer storage medium, specifically, a computer-readable storage medium, including, for example, a first memory 1003 storing a computer program. The computer program can be executed by a processor 1002 of a terminal to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0133] In the several embodiments provided in this application, it should be understood that the disclosed devices, terminals and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0134] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above-mentioned integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0136] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, ROM, RAM, disks or optical disks, etc. Various media that can store program codes.

[0137] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0138] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for transmitting audio and video data in a first wireless terminal, wherein the first wireless terminal comprises an acquisition module and a first WiFi module, wherein: The method comprises: Collecting first data based on the acquisition module; the first data is original audio and video data; Encoding the first data based on the first encoding unit or the second encoding unit in the first WiFi module to generate second data; wherein the first WiFi module is electrically connected to the acquisition module; Sending the second data to the second wireless terminal based on the sending unit in the first WiFi module; The sending of the second data to the second wireless terminal based on the sending unit in the first WiFi module includes: The sending unit monitors the wireless air interface status and feeds back the air interface status to the first encoding unit, so that the first encoding unit determines the encoding bit rate according to the air interface status.

2. The method according to claim 1, characterized in that The encoding process of the first data based on the first encoding unit in the first WIFI module to generate second data includes: If the encoding capability of the first encoding unit meets the encoding requirement of the first data, the first encoding unit is started to encode the first data to generate the second data.

3. The method according to claim 1, characterized in that The encoding process of the first data based on the first encoding unit in the first WiFi module to generate second data further includes: Each frame of the first data is divided into at least two slices of data according to the air interface state; and the first data is encoded using a single slice of data as a unit to generate the second data.

4. A method for receiving audio and video data in a second wireless terminal, wherein the second wireless terminal includes a second WiFi module and a playback module, characterized in that: include: receiving second data based on a wireless link established between the receiving unit of the second WiFi module and the sending unit of the first WiFi module; The decoding unit based on the second WiFi module decodes the second data to obtain third data; wherein the receiving unit is electrically connected to the decoding unit; According to the third data, the playback module plays the corresponding audio and video; wherein the second WiFi module is electrically connected to the playback module; The decoding unit based on the second WiFi module decodes the second data to obtain third data, and further includes: if it is detected that the second data is encoding abnormality data, controlling the receiving unit to send data abnormality feedback to the sending unit, so that the first WiFi module re-encodes the first data and sends the second data.

5. The method according to claim 4, characterized in that The decoding unit in the second WiFi module decodes the second data to obtain the third data, including: if it is detected that the second data is normally encoded data, controlling the decoding unit to decode the second data.

6. The method according to claim 5, characterized in that Decoding the second data based on the decoding unit in the second WiFi module to obtain third data further includes: determining a resolution corresponding to the second data; The playing module playing the corresponding audio and video includes: playing the audio data corresponding to the third data according to the resolution.

7. A first wireless terminal, used for transmitting audio and video data, characterized in that: include: An acquisition module, configured to acquire first data, where the first data is original audio and video data; as well as a first WiFi module, the first WiFi module comprising a first encoding unit, a second encoding unit, and a sending unit, wherein the first encoding unit or the second encoding unit is configured to encode the first data and generate second data; the sending unit is configured to send the second data to a second wireless terminal; wherein the first WiFi module is electrically connected to the acquisition module; The sending unit is used to establish a wireless link with the receiving unit in the second wireless terminal to transmit the second data. The sending unit is also used to monitor the air interface status of the wireless link and feed back the air interface status to the first encoding unit so that the first encoding unit determines the encoding bit rate according to the air interface status.

8. The first wireless terminal according to claim 7, characterized in that Also includes: The processor is configured to start the first encoding unit to encode the first data to generate the second data if the processor determines that the encoding capability of the first encoding unit meets the encoding requirements of the first data.

9. The first wireless terminal according to claim 7, wherein: The first encoding unit is further configured to divide each frame of the first data into at least two slices of data according to an air interface state; and perform encoding processing on the first data using a single slice of data as a unit to generate the second data.

10. The first wireless terminal according to claim 8, wherein: The processor and the acquisition module are electrically connected to the first WiFi module respectively.

11. The first wireless terminal according to claim 8, characterized in that include: The acquisition module, the processor and the first WiFi module are electrically connected in sequence; the processor includes a second encoding unit; the second encoding unit is used to encode the first data if the processor determines that the second encoding unit is to encode the first data.

12. A second wireless terminal for receiving audio and video data, characterized in that: include: The second WiFi module includes a receiving unit and a decoding unit, wherein the receiving unit is used to receive the second data; The decoding unit is used to decode the second data to obtain third data; wherein the receiving unit is electrically connected to the decoding unit; and a playback module, configured to play the audio and video corresponding to the third data; wherein the second WiFi module is electrically connected to the playback module; The second WiFi module is further configured to control the receiving unit to send data abnormality feedback to the sending unit if it is detected that the second data is encoding abnormality data, so that the first WiFi module re-encodes the first data and sends the second data.

13. The second wireless terminal according to claim 12, characterized in that: The second WiFi module is further configured to control the decoding unit to decode the second data if it is detected that the second data is normally encoded data.

14. The second wireless terminal according to claim 13, characterized in that The decoding unit is further used to determine the resolution corresponding to the second data; the playing module is further used to play the audio and video corresponding to the third data according to the resolution.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data transmission method according to any one of claims 1 to 6 is implemented.

16. An audio and video data transmission system, characterized in that: comprising a first wireless terminal and a second wireless terminal, The first wireless terminal is used to send audio and video data, and the first wireless terminal includes: an acquisition module for acquiring first data, where the first data is original audio and video data; and a first WiFi module, where the first WiFi module includes a first encoding unit, a second encoding unit, and a sending unit, where the first encoding unit or the second encoding unit is used to encode the first data and generate second data; the sending unit is used to send the second data to the second wireless terminal; wherein the first WiFi module is electrically connected to the acquisition module; the sending unit is used to establish a wireless link with a receiving unit in the second wireless terminal to transmit the second data, and the sending unit is further used to monitor the air interface status of the wireless link and feed the air interface status back to the first encoding unit, so that the first encoding unit determines the encoding bit rate according to the air interface status; The second wireless terminal is used to receive audio and video data, and the second wireless terminal includes: a second WiFi module, including a receiving unit and a decoding unit, the receiving unit is used to receive the second data; wherein the receiving unit is electrically connected to the decoding unit; the decoding unit is used to decode the second data to obtain third data; and a playback module is used to play the audio and video corresponding to the third data; wherein the second WiFi module is electrically connected to the playback module; the second WiFi module is also used to control the receiving unit to send data abnormality feedback to the sending unit if it is detected that the second data is encoding abnormality data, so that the first WiFi module re-encodes the first data and sends the second data.

Images