Data transmission method and apparatus

Abstract

This application provides a data transmission method and apparatus. The method includes: dividing a first image into multiple image blocks; determining at least one first image block and at least one second image block from the multiple image blocks based on the throughput of a first channel, wherein the total data volume of the first image block is less than or equal to the minimum throughput of the first channel; transmitting the first image block and the second image block through the first channel, wherein the transmission priority of any first image block is greater than the transmission priority of any second image block. By dividing the image into first and second image blocks, setting the total data volume of the first image block to be less than or equal to the minimum throughput of the first channel, and setting the transmission priority of the first image block to be greater than the transmission priority of the second image block, the successful transmission of the first image block can be effectively guaranteed under the current throughput of the first channel, avoiding the problem of unstable data transmission.

Description

Technical Field

[0001] This application relates to the field of communications, and more particularly to a data transmission method and apparatus. Background Technology

[0002] With the continuous development of communication technology, the requirements for the stability of data transmission are becoming increasingly stringent during the data transmission process through data transmission channels.

[0003] Currently, existing technologies typically compress data before transmitting it through a data transmission channel. When the throughput of the data transmission channel fluctuates, critical data may be lost, requiring retransmission.

[0004] However, data retransmission can cause significant delays in data transmission, resulting in unstable data transmission. Summary of the Invention

[0005] This application provides a data transmission method and apparatus to overcome the problem of unstable data transmission.

[0006] In a first aspect, embodiments of this application provide a data transmission method, including:

[0007] The first image is divided into multiple image blocks. To ensure the stability of data transmission, the multiple image blocks are divided in this application. At least one first image block and at least one second image block are determined from the multiple image blocks according to the throughput of the first channel. The total data volume of the first image block is less than or equal to the minimum throughput of the first channel. The first image block and the second image block are transmitted through the first channel. The transmission priority of any first image block is greater than the transmission priority of any second image block.

[0008] By dividing the first image block into the second image block according to the throughput of the first channel, and by setting the total data volume of the first image block to be less than or equal to the minimum throughput of the first channel, and by setting the transmission priority of any first image block to be greater than the transmission priority of any second image block, the successful transmission of the first image block can be effectively guaranteed under the current throughput of the first channel, thus avoiding the lack of stability in data transmission caused by the retransmission of critical data.

[0009] In one possible implementation, the first image block and the second image block can be stored in different queues for data transmission, that is:

[0010] The method for transmitting the first image block and the second image block through the first channel can be as follows:

[0011] Store the first image block into the first queue;

[0012] Store the second image block into the second queue;

[0013] The image blocks in the first queue are transmitted through the first channel, and the image blocks in the second queue are transmitted after the image blocks in the first queue have been transmitted. The transmission priority of the first queue is greater than that of the second queue.

[0014] By storing the first image block and the second image block in different queues, the orderliness of image block transmission can be guaranteed. At the same time, by setting the priority of the first queue to be higher than that of the second queue, the first image block can be transmitted first, thus ensuring the successful transmission of the first image block.

[0015] In another possible implementation, the first and second image blocks can be stored in the same queue for data transfer, that is:

[0016] The method for transmitting the first image block and the second image block through the first channel can be as follows:

[0017] The first image block and the second image block are stored in a third queue, in which any first image block is located before any second image block;

[0018] Image blocks in the third queue are transmitted sequentially through the first channel.

[0019] Furthermore, before storing the first image block into the third queue, it may be determined whether the remaining storage space of the third queue is less than the total data amount of the first image block. If the remaining storage space of the third queue is less than the total data amount of the first image block, at least one second image block at the tail of the third queue is deleted, wherein the total data amount of the deleted at least one second image block is greater than or equal to the total data amount of the first image block.

[0020] Therefore, when the first image block and the second image block are stored in the same queue, for the same first image, the first image block is guaranteed to be stored before the second image block. When the next first image generates the first image block, the second image block in the queue can be deleted to ensure that the first image block is stored in the third queue first, thereby effectively guaranteeing the priority transmission of the third image block.

[0021] In one possible implementation, the maximum transmission duration of the first image block is greater than the maximum transmission duration of the second image block.

[0022] Maximum transmission durations are set for the first and second image blocks respectively, which can effectively prevent the first and second image blocks from occupying the queue for a long time, resulting in low data transmission efficiency or queue congestion. At the same time, by setting a larger maximum transmission duration for the first image block, the transmission success rate of the first image block can be effectively guaranteed.

[0023] In one possible implementation, transmitting the first image block and the second image block through the first channel can be:

[0024] Obtain at least one first modulation and coding scheme (MCS) order corresponding to the first image block, and obtain at least one second MCS order corresponding to the second image block, wherein any first MCS order is less than any second MCS order;

[0025] The first image block and the second image block are transmitted through the first channel, wherein the MCS order when transmitting the first image block through the first channel is the first MCS order, and the MCS order when transmitting the second image block through the first channel is the second MCS order.

[0026] By setting the order of the first MCS corresponding to the first image block to be less than the order of the second MCS corresponding to the second image block, the successful transmission of the first image block can be effectively guaranteed.

[0027] In one possible implementation, determining at least one first image block and at least one second image block among the plurality of image blocks based on the throughput of the first channel includes:

[0028] A first value R is determined based on the maximum throughput of the first channel and the minimum throughput of the first channel, wherein R is an integer greater than or equal to 1;

[0029] According to R, a third image block is determined in every R image blocks among the plurality of image blocks, and the image blocks other than the third image block are determined as the fourth image block;

[0030] If the total data volume of the third image block is equal to the minimum throughput of the first channel, then the third image block is determined as the first image block, and the fourth image block is determined as the second image block.

[0031] By determining R based on the minimum and maximum throughput of the first channel, and then determining the third and fourth image blocks based on R, it can be ensured that the currently determined third and fourth image blocks meet the transmission quality of the first channel. At the same time, when the total data volume of the third image block is equal to the minimum throughput of the first channel, the third image block can be directly determined as the first image block, which can effectively enhance the simplicity of determining the second image block.

[0032] In another possible implementation, if the total data volume of the third image block is less than the value corresponding to the minimum throughput of the first channel, the method further includes:

[0033] In the fourth image block, a sub-image block is selected, and the third image block and the selected sub-image block are determined as the first image block, wherein the total data volume of the selected sub-image block and the third image block is equal to the minimum throughput of the first channel;

[0034] The image blocks in the fourth image block other than the sub-image blocks are identified as the second image blocks.

[0035] By redistributing image blocks into first and second image blocks based on the division of the third and fourth image blocks and the throughput of the first channel, as many image blocks as possible can be transmitted on the basis of the minimum throughput of the first channel, thereby improving the display effect of the first image on the receiving device. Furthermore, this process can ensure the successful transmission of the selected third image block, thus effectively guaranteeing the stability of data transmission.

[0036] In one possible implementation, determining the third image block among every R image blocks in the plurality of image blocks according to the R includes:

[0037] Based on R, the Kth image block in every R image blocks is determined as the third image block.

[0038] By determining the Kth image block out of every R image blocks as the third image block, the equal spacing of the selected third image blocks in the first image can be effectively guaranteed, thus ensuring the display effect of the first image.

[0039] In one possible implementation, determining the third image block among every R image blocks in the plurality of image blocks according to the R includes:

[0040] Determine the first mean of the pixel values ​​for each R image patch;

[0041] The target image block in each of the R image blocks is determined as the third image block, wherein the target image block is the image block with the smallest difference between the average pixel value and the first average pixel value.

[0042] In another possible implementation, the ratio of the minimum throughput of the first channel to the maximum throughput of the first channel is equal to the ratio of the value 1 to R.

[0043] In one possible implementation, before determining the third image block in every R image blocks of the plurality of image blocks according to the R, the method further includes:

[0044] The plurality of image blocks are subjected to discrete cosine transform processing, and / or quantization processing, and / or sorting processing to obtain processed image blocks;

[0045] Each R processed image blocks are jointly compressed to obtain multiple jointly compressed image blocks.

[0046] By jointly compressing each set of processed image blocks, multiple jointly compressed image blocks are obtained. Encoding and jointly compressing these image blocks effectively reduces the amount of data transmitted, thereby improving data transmission efficiency.

[0047] Secondly, embodiments of this application provide a data transmission apparatus, including:

[0048] The segmentation module is used to divide the first image into multiple image blocks;

[0049] The determining module is configured to determine at least one first image block and at least one second image block among the plurality of image blocks based on the throughput of the first channel, wherein the total data volume of the first image block is less than or equal to the minimum throughput of the first channel;

[0050] A transmission module is configured to transmit the first image block and the second image block through the first channel, wherein the transmission priority of any first image block is greater than the transmission priority of any second image block.

[0051] In one possible implementation, the transmission module is specifically used for:

[0052] Store the first image block into the first queue;

[0053] Store the second image block into the second queue;

[0054] The image blocks in the first queue are transmitted through the first channel, and the image blocks in the second queue are transmitted after the image blocks in the first queue have been transmitted. The transmission priority of the first queue is greater than that of the second queue.

[0055] In one possible implementation, the transmission module is specifically used for:

[0056] The first image block and the second image block are stored in a third queue, in which any first image block is located before any second image block;

[0057] Image blocks in the third queue are transmitted sequentially through the first channel.

[0058] In one possible implementation, the transmission module is further configured to:

[0059] Before storing the first image block into the third queue, if the remaining storage space of the third queue is less than the total data amount of the first image block, then at least one second image block at the tail of the third queue is deleted, wherein the total data amount of the deleted at least one second image block is greater than or equal to the total data amount of the first image block.

[0060] In one possible implementation, the maximum transmission duration of the first image block is greater than the maximum transmission duration of the second image block.

[0061] In one possible implementation, the transmission module is specifically used for:

[0062] Obtain the first modulation and coding scheme (MCS) order corresponding to the first image block, and obtain the second MCS order corresponding to the second image block, wherein the first MCS order is less than the second MCS order;

[0063] The first image block and the second image block are transmitted through the first channel, wherein the MCS order when transmitting the first image block through the first channel is the first MCS order, and the MCS order when transmitting the second image block through the first channel is the second MCS order.

[0064] In one possible implementation, the determining module is specifically used for:

[0065] A first value R is determined based on the maximum throughput of the first channel and the minimum throughput of the first channel, wherein R is an integer greater than or equal to 1;

[0066] According to R, a third image block is determined in every R image blocks among the plurality of image blocks, and the image blocks other than the third image block are determined as the fourth image block;

[0067] If the total data volume of the third image block is equal to the minimum throughput of the first channel, then the third image block is determined as the first image block, and the fourth image block is determined as the second image block.

[0068] In one possible implementation, if the total data volume of the third image block is less than the value corresponding to the minimum throughput of the first channel, the determining module is further configured to:

[0069] In the fourth image block, a sub-image block is selected, and the third image block and the selected sub-image block are determined as the first image block, wherein the total data volume of the selected sub-image block and the third image block is equal to the minimum throughput of the first channel;

[0070] The image blocks in the fourth image block other than the sub-image blocks are identified as the second image blocks.

[0071] In one possible implementation, the determining module is specifically used for:

[0072] Based on R, the Kth image block in every R image blocks is determined as the third image block.

[0073] In one possible implementation, the determining module is specifically used for:

[0074] Determine the first mean of the pixel values ​​for each R image patch;

[0075] The target image block in each of the R image blocks is determined as the third image block, wherein the target image block is the image block with the smallest difference between the average pixel value and the first average pixel value.

[0076] In one possible implementation, the ratio of the minimum throughput of the first channel to the maximum throughput of the first channel is equal to the ratio of the value 1 to R.

[0077] In one possible implementation, the determining module is further configured to:

[0078] Before determining the third image block in each of the R image blocks according to the R, the plurality of image blocks are subjected to discrete cosine transform processing, and / or quantization processing, and / or sorting processing to obtain processed image blocks;

[0079] Each R processed image blocks are jointly compressed to obtain multiple jointly compressed image blocks.

[0080] Thirdly, embodiments of this application provide a data transmission device, including: a memory, a processor, and a radio frequency transceiver, wherein the memory is used to store programs or instructions, and when the programs or instructions are executed by the processor, the device causes the device to perform the method described in the first aspect above and any of the various possible designs of the first aspect.

[0081] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect above and any of the various possible designs of the first aspect.

[0082] Fifthly, embodiments of this application provide a computer program product, the computer program product including instructions, which, when executed, cause a computer to perform the data transmission method described in any of the first aspects above.

[0083] In a sixth aspect, embodiments of this application provide a system-on-a-chip (SoC) or system-on-a-chip (SoC) that can be applied to a terminal device. The SoC or SoC includes: at least one communication interface, at least one processor, and at least one memory. The communication interface, memory, and processor are interconnected via a bus. The processor executes instructions stored in the memory, enabling the terminal device to perform the data transmission method as described in any of the first aspects.

[0084] In the data transmission method and apparatus provided in this embodiment, the first image block and the second image block are divided according to the throughput of the first channel. By setting the total data volume of the first image block to be less than or equal to the minimum throughput of the first channel, and setting the transmission priority of any first image block to be greater than the transmission priority of any second image block, the successful transmission of the first image block can be effectively guaranteed under the current throughput of the first channel, and the instability of data transmission caused by the retransmission of critical data is avoided. Attached Figure Description

[0085] Figure 1 A schematic diagram of a data transmission system provided in an embodiment of the present invention;

[0086] Figure 2 This is a flowchart illustrating a data transmission method in the prior art.

[0087] Figure 3 The flow of the data transmission method provided in this application Figure 1 ;

[0088] Figure 4 Flowchart of the data transmission method provided in the embodiments of this application Figure 2 ;

[0089] Figure 5 This is a schematic diagram of image block division provided in an embodiment of this application;

[0090] Figure 6 A schematic diagram of the third image block provided in the embodiments of this application. Figure 1 ;

[0091] Figure 7A schematic diagram of the determination of the third image block provided in the embodiments of this application. Figure 2 ;

[0092] Figure 8 A schematic diagram of the determination of the third image block provided in the embodiments of this application. Figure 3 ;

[0093] Figure 9 This is a schematic diagram of a first image composed of a portion of image blocks provided in an embodiment of this application;

[0094] Figure 10 This application provides an illustration of image block redistribution in its embodiments. Figure 1 ;

[0095] Figure 11 This is a schematic diagram of image block redistribution provided for embodiments of this application. Figure 2 ;

[0096] Figure 12 Flowchart of the data transmission method provided in the embodiments of this application Figure 3 ;

[0097] Figure 13 A schematic diagram of the first and second queues provided for the embodiments of the application;

[0098] Figure 14 Flowchart of the data transmission method provided in the embodiments of this application Figure 4 ;

[0099] Figure 15 A schematic diagram of the third queue provided for the application embodiment;

[0100] Figure 16 Flowchart of the data transmission method provided in the embodiments of this application Figure 5 ;

[0101] Figure 17 Schematic diagram of the data transmission device provided in the embodiments of this application Figure 1 ;

[0102] Figure 18 Schematic diagram of the data transmission device provided in the embodiments of this application Figure 2 ;

[0103] Figure 19 This is a schematic diagram of the hardware structure of the data transmission device provided in an embodiment of this application. Detailed Implementation

[0104] First, combine Figure 1 The process of data transmission is explained. Figure 1 This is a schematic diagram of a data transmission system provided in an embodiment of the present invention, such as... Figure 1 As shown, the system includes:

[0105] The transmitting device 101 and the receiving device 102 transmit data between each other through a first channel. Specifically, the first channel transmits the data sent by the transmitting device 101 to the receiving device 102 in the form of radio waves, thereby realizing data transmission between the transmitting device 101 and the receiving device 102. It can be understood that the first channel can be a wireless channel, which is an invisible transmission path between the transmitting device and the receiving device in the process of wireless communication. For radio waves, there is no physical connection between the transmitting end and the receiving end, and its propagation path may be more than one.

[0106] In this embodiment, the transmitting device 101 and the receiving device 102 can be, for example, terminal devices or network devices. Therefore, the terminal devices and network devices will be briefly described below:

[0107] Terminal equipment: A terminal device is a device with wireless transceiver capabilities. Terminal equipment can be deployed on land, including indoors or outdoors, handheld, wearable, or vehicle-mounted; it can also be deployed on water (such as on ships); and it can be deployed in the air (such as on airplanes, balloons, and satellites). Terminal equipment can include mobile phones, tablets, computers with wireless transceiver capabilities, virtual reality (VR) terminal equipment, augmented reality (AR) terminal equipment, wireless terminals in industrial control, vehicle-mounted terminal equipment, wireless terminals in self-driving vehicles, wireless terminal equipment in remote medical care, wireless terminal equipment in smart grids, wireless terminal equipment in transportation safety, wireless terminal equipment in smart cities, wireless terminal equipment in smart homes, wearable terminal equipment, etc. The terminal equipment involved in the embodiments of this application may also be referred to as a terminal, user equipment (UE), access terminal equipment, vehicle-mounted terminal, industrial control terminal, UE unit, UE station, mobile station, mobile station, remote station, remote terminal equipment, mobile device, UE terminal equipment, wireless communication equipment, UE agent, or UE device, etc. The terminal equipment may also be fixed or mobile.

[0108] Network equipment: A device with wireless transceiver capabilities. This includes, but is not limited to: Evolutionary Node Bs (eNBs or eNodeBs) in Long Term Evolution (LTE), base stations (gNodeBs or gNBs) or transmission receiving points / transmission reception points (TRPs) in New Radio (NR) technologies, base stations in subsequent evolution systems, access nodes, wireless relay nodes, and wireless backhaul nodes in Wireless Fidelity (WiFi) systems. Base stations can be: macro base stations, micro base stations, pico base stations, small cells, relay stations, or balloon stations, etc. Multiple base stations can support networks using the same technology mentioned above, or they can support networks using different technologies mentioned above. A base station can contain one or more co-located or non-co-located TRPs. Network equipment can also be a radio controller, centralized unit (CU), and / or distributed unit (DU) in a cloud radio access network (CRAN) scenario. Network devices can also be servers, wearable devices, or vehicle-mounted devices, etc. The following explanation uses a base station as an example. The multiple network devices can be the same type of base station or different types of base stations. A base station can communicate with a terminal directly or via a relay station. A terminal can communicate with multiple base stations using different technologies. For example, a terminal can communicate with a base station supporting LTE networks, or with a base station supporting 5G networks, and can also support dual connections with both LTE and 5G base stations, etc.

[0109] In the specific implementation process, the specific implementation of the transmitting device 101 and the receiving device 102 can be selected according to actual needs, as long as the transmitting device 101 has wireless transmitting function and the receiving device 101 has wireless receiving function. This embodiment does not limit the selection of the transmitting device 101 and the receiving device 102.

[0110] exist Figure 1 Based on the system introduced, the following will combine... Figure 2 This paper introduces the existing methods for implementing data transmission. Figure 2 This is a flowchart illustrating a data transmission method in the prior art, such as... Figure 2 As shown:

[0111] First, the transmitting device converts the image data to be transmitted from RGB color mode to YCrCb color mode. RGB color mode is an industry color standard. Currently, all colors displayed on terminal device screens are made by mixing red (R), green (G), and blue (B) light in different proportions. YCrCb color mode is used to optimize the transmission of color video signals, making them backward compatible with older black and white displays. Therefore, converting image data from RGB color mode to YCrCb color mode ensures that the image data can be displayed compatiblely on black and white displays.

[0112] Secondly, the image data after color mode conversion is processed by Discrete Cosine Transform (DCT), and the processed image data is compressed based on H.264 / H.265 to obtain compressed image data.

[0113] The compressed image data is then transmitted through the first channel, specifically using a transmitter based on Orthogonal Frequency Division Multiplexing (OFDM) technology.

[0114] However, the throughput of the first channel may change during data transmission. Specifically, the channel throughput refers to the number of data bits correctly transmitted per unit time. In the implementation of existing technologies, image compression and transmission are completely separate steps, and the changes in channel throughput are not taken into account. When the channel throughput decreases, data loss may occur. When the image data received by the receiving device is lost, phenomena such as screen tearing and blurring may occur, resulting in a poor visual experience for the user.

[0115] When critical data is lost, it needs to be retransmitted to ensure the correct display of image data. However, the retransmission of critical data will cause a large delay in the receiving device receiving the image data, and the image data cannot be guaranteed to be displayed in real time. This results in a lack of stability in data transmission and a poor user experience.

[0116] To address the problems in existing technologies, this application provides a data transmission method to ensure data transmission stability. The following describes the method in conjunction with... Figure 3 This method will be introduced. Figure 3 The flow of the data transmission method provided in this application Figure 1 .

[0117] like Figure 3 As shown, the method includes:

[0118] S301. Divide the first image into multiple image blocks.

[0119] In this embodiment, the data sent by the transmitting device to the receiving device is specifically image data. In one possible implementation, the transmitting device can divide the video data to be transmitted into frames to obtain multiple first images, and transmit the multiple first images sequentially to realize the transmission of video data; or, the transmitting device can also directly send image data, wherein the image data includes at least one first image, and this embodiment does not limit this.

[0120] Specifically, for any given first image, the first image is divided into multiple image blocks. In one possible implementation, for example, the first image can be divided into X×Y image blocks of equal size, where X and Y are integers greater than 1; or, the first image can be divided into multiple image blocks according to a fixed pixel size. This embodiment does not limit the specific implementation method of dividing the image blocks. In actual operation, the image blocks can be divided according to actual needs, as long as the first image can be divided into multiple image blocks of the same size.

[0121] S302. Based on the throughput of the first channel, determine at least one first image block and at least one second image block among multiple image blocks, wherein the total data volume of the first image block is less than or equal to the minimum throughput of the first channel.

[0122] In this embodiment, to ensure that the receiving device can stably receive image data, the transmitting device divides multiple image blocks into a first image block and a second image block. The first image block can be understood as a key block, which is specifically the image block that needs to be successfully transmitted, while the second image block can be understood as a non-key block, which is specifically the image block that is transmitted with best effort. That is, the transmission of the first image block is given priority. When the throughput of the first channel is insufficient to realize the transmission of the second image block, the second image block may not be transmitted at this time.

[0123] Based on the above description, it can be understood that in one possible implementation, at least one first image block and at least one second image block in this embodiment together constitute all the divided image blocks. That is, in this embodiment, multiple image blocks can be divided into first image blocks and second image blocks.

[0124] In this embodiment, the first image block and the second image block are divided based on the throughput of the first channel. Specifically, the transmitting device can obtain the throughput of the first channel at the current moment. It is understood that since the transmission quality of the first channel is constantly changing, the throughput of the first channel at the current moment is usually a range. Therefore, the minimum throughput of the first channel can be further determined. For example, the throughput of the first channel can be 25.5Mb / s (megabits per second) - 10Mb / s, then the minimum throughput of the first channel is 10Mb / s.

[0125] In order to ensure that the first channel can successfully transmit the first image block, this embodiment determines at least one first image block among multiple image blocks. Each image block has the same size. For example, a first image block can be 4 bits. In this embodiment, the size of each image block depends on the way the image blocks are divided in step S301 above. This embodiment does not limit this.

[0126] In this embodiment, the total data volume of the first image block is less than or equal to the minimum throughput of the first channel. At the same time, at least one second image block is also determined. In one possible implementation, after determining the first image block, the image blocks other than the first image block can be determined as the second image blocks. Since the size of each image block in this embodiment is the same and its data volume is also the same, the number of second image blocks can be obtained based on the total data volume of the image blocks other than the first image block and the individual data volume of each image block. Alternatively, the number of second image blocks can be determined directly based on the above-described multiple image blocks. This embodiment does not impose any special restrictions on this.

[0127] In one possible implementation, for example, sampling can be performed at equal intervals in multiple image blocks, and at least one sampled image block can be determined as the first image block, and at least one unsampled image block can be determined as the second image block; or, a preset number of image blocks can be used as a sampling unit, and image blocks can be selected in each sampling unit to obtain at least one first image block, etc. It can be understood that the sum of the number of first image blocks and the number of second image blocks is the total number of divided image blocks.

[0128] This embodiment does not limit the specific division method of the first image block and the second image block. It can be understood that as long as the total data volume of the first image block is less than or equal to the minimum throughput of the first channel, and the sum of the number of the first image block and the number of the second image block is the total number of image blocks, the specific implementation method can be set according to actual needs.

[0129] S303. Transmit a first image block and a second image block through a first channel, wherein the transmission priority of any first image block is greater than the transmission priority of any second image block.

[0130] In this embodiment, by setting the transmission priority of any first image block to be greater than that of any second image block, the first image block can be transmitted first when both the first and second image blocks exist simultaneously. Furthermore, since the total data volume of the first image block is less than or equal to the minimum throughput of the first channel, the successful transmission of the first image block can be guaranteed.

[0131] In this embodiment, the total data volume of the first image block is less than or equal to the minimum throughput of the first channel. Therefore, it can be guaranteed that the first image block can be successfully transmitted within the current throughput of the first channel. As long as the receiving device can successfully receive the first image block, the problem of loss of critical image data caused by fluctuations in the channel throughput can be avoided, thereby ensuring the stability of the data transmission of the first image block.

[0132] In addition, after the transmission of the first image block is completed, this embodiment performs best-effort transmission of the second image block. That is, under the actual throughput of the first channel, as many second image blocks as possible are transmitted. It can be understood that the more second image blocks are transmitted, the clearer the first image displayed by the receiving device will be.

[0133] In an optional embodiment, if the current second image block cannot be transmitted, the second image block can be discarded, or when the preset period arrives, if the first image block does not exist, an attempt can be made to retransmit the second image block.

[0134] In one possible implementation, during the transmission of the first image block and the second image block through the first channel, this embodiment can obtain the first MCS order corresponding to the first image block and the second MCS order corresponding to the second image block. During the transmission of image blocks through the first channel, the MCS order when the first image block is transmitted through the first channel is the first MCS order, and the MCS order when the second image block is transmitted through the first channel is the second MCS order.

[0135] This section first introduces the MCS order. The first channel includes many different data streams, which are used to transmit image blocks. Different data streams correspond to their own MCS orders. The lower the MCS order of a data stream, the better the transmission quality of that data stream. Therefore, in this embodiment, by setting the first MCS order corresponding to the first image block to be less than the order of the second MCS corresponding to the transmission of the second image block, the successful transmission of the first image block can be effectively guaranteed.

[0136] In one possible implementation, a first MCS order table and a second MCS order table can be pre-set. Each of the first and second MCS order tables includes multiple MCS orders. At the same time, any MCS order in the first MCS order table is less than any MCS order in the second MCS order table. After determining the first image block and the second image block, the first MCS order table is determined as the table corresponding to the first image block, and the second MCS order table is determined as the table corresponding to the second image block, thereby obtaining the first MCS order and the second MCS order.

[0137] During data transmission through the first channel, the MCS order is determined by consulting the corresponding table, and image blocks are transmitted according to the MCS order.

[0138] In another possible implementation, a better channel coding scheme can be assigned to the first image block to ensure successful transmission of the first image block.

[0139] The data transmission method provided in this application includes: dividing a first image into multiple image blocks; determining at least one first image block and at least one second image block among the multiple image blocks based on the throughput of a first channel, wherein the total data volume of the first image block is less than or equal to the minimum throughput of the first channel; transmitting the first image block and the second image block through the first channel, wherein the transmission priority of any first image block is greater than the transmission priority of any second image block. By dividing the first image block and the second image block according to the throughput of the first channel, by setting the total data volume of the first image block to be less than or equal to the minimum throughput of the first channel, and by setting the transmission priority of any first image block to be greater than the transmission priority of any second image block, the successful transmission of the first image block can be effectively guaranteed under the current throughput of the first channel, avoiding the instability of data transmission caused by the retransmission of critical data.

[0140] Based on the above embodiments, the following is combined with Figures 4-11 The data transmission method provided in the embodiments of this application will be described in further detail. Figure 4 Flowchart of the data transmission method provided in the embodiments of this application Figure 2 , Figure 5 This is a schematic diagram of image block division provided in an embodiment of this application. Figure 6 A schematic diagram of the third image block provided in the embodiments of this application. Figure 1 , Figure 7 A schematic diagram of the determination of the third image block provided in the embodiments of this application. Figure 2 , Figure 8 A schematic diagram of the determination of the third image block provided in the embodiments of this application. Figure 3 , Figure 9 This is a schematic diagram of a first image composed of a portion of image blocks provided in an embodiment of this application. Figure 10 This application provides an illustration of image block redistribution in its embodiments. Figure 1 , Figure 11 This is a schematic diagram of image block redistribution provided for embodiments of this application. Figure 2 .

[0141] like Figure 4 As shown, the method includes:

[0142] S401. Divide the first image into multiple image blocks.

[0143] The following is combined Figure 5 This section describes one possible implementation of dividing an image into blocks, such as... Figure 5 As shown, there is a first image. Assuming that the first image is a 64×64 pixel image, it can be divided into 64 image blocks according to the 8×8 specification. Since the first image is 64×64 pixels, each image block can include 8×8 pixels.

[0144] Those skilled in the art will understand that Figure 5 This only illustrates one possible way of dividing image blocks. The specific way of dividing image blocks can be selected according to actual needs. For example, the size of the division or the method of division can be expanded according to actual needs. For details, please refer to the introduction in step S301 above, which will not be repeated here.

[0145] S402. Determine the first value R based on the maximum throughput of the first channel and the minimum throughput of the first channel.

[0146] In this embodiment, the first value R is used to determine the third and fourth image blocks in subsequent steps. Several possible implementations of determining the first value R based on the throughput of the first channel are described below:

[0147] In one possible implementation, the ratio of the minimum throughput of the first channel to the maximum throughput of the first channel is equal to the ratio of the value 1 to the first value R.

[0148] In other words, the value of the first value R satisfies the following formula:

[0149] 1: R = Minimum throughput: Maximum throughput (Formula 1)

[0150] In another possible implementation, the ratio of the total data volume of the first image block to the sum of the total data volume of the first image block and the total data volume of the second image block is less than the ratio of the value 1 to the first value R. In other words, the value of the first value satisfies the following formula:

[0151] KeyBlock data size: (KeyBlock data size + NKeyBlock data size) <= 1: Formula 2

[0152] In another possible implementation, the first value is the difference between the ratio of the maximum throughput to the minimum throughput of the first channel. That is, the value of the first value satisfies the following formula three:

[0153] R = UV (Formula 3)

[0154] Where U is the numerator of the ratio of maximum throughput to minimum throughput, and V is the denominator of the ratio of maximum throughput to minimum throughput. In other words, U and V satisfy the following formula:

[0155] Minimum throughput : Maximum throughput = U : V Formula 4

[0156] It is understood that the first value R in this embodiment is used to subsequently determine the third and fourth image blocks. The specific implementation method can be extended according to actual needs based on the above description. As long as the first value is determined based on the maximum and minimum throughput of the first channel and can be used to determine the image blocks, it is acceptable. In this embodiment, when the first value is not an integer, the first value can be rounded. The rounding method can be rounding up, rounding up, rounding down, etc., which are not limited here.

[0157] It should be noted that the first value in this embodiment is determined by the transmitting device based on the throughput of the first channel. Therefore, the first value R in this embodiment will change. Thus, the various possible implementations of R described above are determined under certain channel scenarios and within a certain time range. As time goes by and the channel scenario changes, R will change accordingly. At the same time, the transmitting device also needs to send the determined first value to the receiving device through signaling so that the transmitting device and the receiving device have the same first value synchronously, thereby ensuring that the receiving device can successfully receive and parse the first image.

[0158] S403. Based on the first value, determine the third image block in every R image blocks among the multiple image blocks, and determine the image blocks other than the third image block as the fourth image block.

[0159] In this embodiment, the first value is used to determine the third image block and the fourth image block. First, the third image block and the fourth image block will be explained. In this embodiment, the third image block (must block) is the image block that must be successfully transmitted. The fourth image block (Not Must block) includes the image block that is transmitted with best efforts, and may also include the image block that must be successfully transmitted.

[0160] In one possible implementation, when the data volume of the third image block has met the data volume requirement of the first image block, the image blocks included in the fourth image block are the best-effort image blocks to be transmitted. However, when the data volume of the third image block has not yet met the data volume requirement of the first image block, some image blocks can be selected from the fourth image block as the first image block. These selected image blocks are the image blocks that must be successfully transmitted, thereby enabling the transmission of more image blocks under the current throughput of the first channel.

[0161] The method for determining the third and fourth image blocks is explained below:

[0162] For ease of explanation, R will be used below to refer to the first value. Specifically, R image blocks are taken as a sampling unit, and one or more third image blocks are determined within each R image blocks. The following will combine... Figures 6-8 The possible implementations for determining the third image patch are introduced below:

[0163] In one possible implementation, the Kth image block out of every R image blocks can be designated as the third image block, where K is an integer greater than or equal to 1.

[0164] Taking R as 4 and K as 1 as an example, combined with Figure 6 To clarify, the first image block out of every four image blocks is designated as the third image block. Figure 6 A sampling unit (every 4 image blocks) is marked with a bold solid line. The first image block in every 4 image blocks is marked with a shaded area to indicate that the current image block is the third image block. Meanwhile, all the image blocks other than the third image block are the fourth image blocks, which are marked with blank spaces in the figure.

[0165] It should be noted that, Figure 6 This is merely an exemplary introduction to this implementation method. In actual implementation, for example, four image blocks in one row can be used as a sampling unit, or four image blocks in one column can be used as a sampling unit. The specific implementation of K can also be selected according to actual needs. This embodiment does not limit the division of sampling units or the selection of K.

[0166] In another possible implementation, a first mean of pixel values ​​for each of the R image blocks can be determined, and a target image block in each of the R image blocks can be determined as a third image block, wherein the difference between the mean of pixel values ​​of the target image block and the first mean is minimized.

[0167] Again, taking R as an example of 4, combined with Figure 7 To illustrate, each image block contains multiple pixels, each with its own pixel value. Based on the pixel values ​​of each pixel, the first average pixel value of every four image blocks can be determined. (See [link to relevant documentation]). Figure 7 Suppose that the first mean pixel value of one sampling unit is 4, and the mean pixel values ​​of the four image blocks included in this sampling unit are 9, 5, 20, and 6 respectively. Then, the mean pixel value with the smallest difference from the first mean is 5. The target image block corresponding to the mean pixel value of 5 is then identified as the third image block and marked with a shaded area. All other image blocks are identified as the fourth image block and marked with a blank area. The identified third and fourth image blocks can be defined as follows: Figure 7 As shown, it is understandable that because this embodiment uses the method of calculating the difference, the arrangement of the third image block is irregular.

[0168] In another possible implementation, if the first image being transmitted is a key image or keyframe, meaning that the complete and successful transmission of the first image must be guaranteed, then all image blocks included in the first image are determined as third image blocks, see [link to relevant documentation]. Figure 8 Each image block in the current first image is marked with a shadow to indicate that all image blocks in the current first image are third image blocks.

[0169] Correspondingly, another first image or another video frame can be sacrificed, that is, part or all of another first image or video frame can be used as the fourth image block to ensure the successful transmission of the current image frame.

[0170] Furthermore, this embodiment addresses image data. In another embodiment, if the current transmitted data is voice data or control data, it is understood that the receiving device can only receive the correct information if the voice data or control data is successfully transmitted. Therefore, the voice data or control data must be transmitted successfully and can be used as the third data block (must block), which is equivalent to the third image block in this embodiment.

[0171] Referring to the above description, those skilled in the art will understand that the third image block in this embodiment is an image block that must be successfully transmitted. However, the selection of each third image block is not because the current image block is more important in the first image. This embodiment simply selects uniformly sampled image blocks in the first image as third image blocks because, for an image, when the first channel cannot guarantee the complete transmission of the first image, a portion of the sampled data can be transmitted. The sampled data can still completely represent the information carried by the first image. In this embodiment, for ease of explanation, the number of image blocks in the images in the above examples is very small. In practical applications, when the number of image blocks is large, it will not affect the display of the first image. The following will be combined with... Figure 9 Explanation of this effect:

[0172] See Figure 9 , Figure 9 In the image, 901 represents the original first image, and 902 represents the original first image with some image patches missing. Figure 9 As can be seen, although 902 does not include all the pixels, we can still receive the information included in the first image.

[0173] It should be noted that, Figure 9 The above is merely an illustrative example, and the image block loss effect it exhibits is quite severe. In actual applications, it is very likely that only a small portion of the image blocks will fail to be transmitted successfully. Therefore, the first image received by the receiving device will not have very obvious pixel loss.

[0174] Those skilled in the art will understand that the specific method of dividing the image blocks can be selected according to actual needs and is not limited to the three implementation methods described above, as long as the selected third image block is evenly distributed in the first image.

[0175] S404. Determine whether the total data volume of the third image block is equal to the minimum throughput of the first channel. If yes, execute S405; otherwise, execute S406.

[0176] In this embodiment, the third and fourth image blocks are selected based on R. Therefore, the number of selected third image blocks may be less than the minimum throughput of the first channel or equal to the maximum throughput of the first channel. In this embodiment, the specific number of third image blocks cannot be determined. In order to ensure that as many image blocks as possible can be transmitted under the current throughput of the first channel, in this embodiment, the total data volume of the first image blocks is set to be equal to the minimum throughput of the first channel.

[0177] Specifically, in this embodiment, the third image block is the image block that must be successfully transmitted. Therefore, it is first determined whether the total data volume of the third image block is equal to the minimum throughput of the first channel.

[0178] It is worth noting that the judgment step S404 in this embodiment is written to ensure the readability of the embodiment. In actual implementation, the current judgment step may be omitted, and S405 or S406 may be executed directly.

[0179] S405. The third image block is designated as the first image block, and the fourth image block is designated as the second image block.

[0180] In one possible implementation, if the total data volume of the third image block is equal to the minimum throughput of the first channel, it means that the minimum throughput of the first channel can just guarantee the successful transmission of the third image block. In this case, the third image block is directly determined as the first image block, so that the successful transmission of the first image block can be guaranteed under the current throughput of the first channel.

[0181] Furthermore, in this embodiment, the fourth image block is identified as the second image block.

[0182] S406. Select a sub-image block from the fourth image block, and designate the third image block and the selected sub-image block as the first image block, and designate the image blocks in the fourth image block other than the sub-image block as the second image block. The total data volume of the selected sub-image block and the third image block is equal to the minimum throughput of the first channel.

[0183] If the total data volume of the third image block is not equal to the throughput of the first channel, in one possible implementation, the total data volume of the third image block is less than the throughput of the first channel. This indicates that under the current throughput of the first channel, in addition to ensuring the successful transmission of the third image block, there are still surplus resources to ensure the successful transmission of a portion of the fourth image block. In this case, a sub-image block can be selected from the fourth image block. This portion of the sub-image block selected from the fourth image block is also used as the first image block. Specifically, the sub-image block can be a complete image block or a part of an image block.

[0184] In this embodiment, if the total data volume of the first image block is set to be equal to the minimum throughput of the first channel, then the total data volume of the third image block and the selected sub-image block is equal to the throughput of the first channel.

[0185] In another possible implementation, if the total data volume of the third image block is greater than the minimum throughput of the first channel, it means that at this time, according to the minimum throughput of the first channel, there is no way to guarantee the successful transmission of all third image blocks. In this case, we still determine the third image block as the first image block, so as to ensure that the third image block can be transmitted first in the future, thereby ensuring the transmission of the third image block as much as possible, and effectively improving the transmission success rate of the third image block when the throughput of the first channel is relatively small.

[0186] Understandably, the third image block is transmitted with best effort based on the minimum throughput of the first channel. If the transmission of the third image block times out and is still unsuccessful, the third image block can only be discarded.

[0187] Based on the above introduction, it can be understood that if the total data volume of the third image block is equal to the minimum throughput of the first channel, then the third image block can be directly determined as the first image block. Below, we will illustrate a possible implementation method for determining the first and second image blocks based on the third and fourth image blocks when the total data volume of the third image block is less than the minimum throughput of the first channel, using a specific example:

[0188] For ease of explanation with reference to the accompanying figures, it is assumed that the current throughput of the first channel is in the range of 2Mb / s-9Mb / s, and the size of one image block is 1Mb. This indicates that two first image blocks should be selected at present. It is also assumed that there is currently one third image block and eight fourth image blocks.

[0189] Based on the example above, it can be determined that the total data volume of the third image block is less than the minimum throughput of the first channel. In this case, it is necessary to select a sub-image block from the fourth image block so that the total data volume of the first image block can be equal to the minimum throughput of the first channel.

[0190] In one possible implementation, see Figure 10 An image block comprises 8×8 pixels. After dividing the image into pixel blocks, DCT processing can be performed on each pixel block to obtain the frequency coefficient matrix corresponding to each pixel block. Since an image block comprises 8×8 pixels, the 8×8 frequency coefficient matrix corresponding to that pixel block has 1 DC component and 63 AC components. Figure 5 The numbers shown represent the DC component or AC component.

[0191] Then, the first row of pixels in the fourth image block can be used as the selected sub-image block. For the eight fourth image blocks, the first row of pixels in each of them can be determined as the first image block. This makes the total data volume of the selected sub-image blocks and the third image blocks equal to the minimum throughput of the first channel. The first image block includes the third image block and part of the fourth image block. Figure 10 The first image block is identified by a solid line box.

[0192] Meanwhile, the remaining sub-image blocks that were not selected in the fourth image block are determined as second image blocks, so this embodiment can obtain 7 second image blocks.

[0193] In another possible implementation, see Figure 11 The third image block can be divided into upper and lower parts. The upper part, together with the sub-image blocks of the four fourth image blocks, forms a first image block. The lower part, together with the sub-image blocks of the other four fourth image blocks, forms another first image block. Figure 10 compared to, Figure 11 The selected pixels are exactly the same. The difference lies in the way the first image block is divided. That is, the first image block can be a complete image block, or it can be incomplete or discontinuous. In this embodiment, as long as the total data volume of the first image block is equal to the minimum throughput of the first channel, the specific division method can be selected according to actual needs.

[0194] Based on the above-described example figures, it should be noted that there are many ways to select sub-image blocks in the fourth image block. For example, for a fourth image block, one column can be selected; or, one pixel can be selected in each row; or, the current complete fourth image block can be determined as the first image block. This embodiment does not limit this. That is to say, this embodiment does not specifically limit the selection method of the first image block, as long as the third image block is entirely the first image block and the total data volume of the first image block is equal to the minimum throughput of the first channel. Various possible implementation methods can be extended according to actual needs, which will not be elaborated here.

[0195] It is understandable that when selecting a sub-image block as the first image block in the fourth image block, the more evenly the sub-image blocks are distributed, the better the final image data received by the receiving device will appear.

[0196] In an optional embodiment, the ratio of the total data volume of the first image block to the total data volume of the second image block can be set to satisfy the following formula:

[0197] Total data volume of the first image block : Total data volume of the second image block = Minimum throughput : (Maximum throughput - Minimum throughput) Formula 5

[0198] For example, the above Figure 10 and Figure 11 In the corresponding example, the minimum throughput of the first channel is 2 and the maximum throughput of the first channel is 10. The ratio of the total data volume of the first image block to the total data volume of the second image block can be 2:7. That is to say, on the basis of ensuring the successful transmission of the first image block according to the minimum throughput, the second image block is transmitted with best effort according to the actual channel state of the first channel.

[0199] S407. Transmit a first image block and a second image block through a first channel, wherein the transmission priority of any first image block is greater than the transmission priority of any second image block.

[0200] Specifically, the implementation of S407 is similar to that of S303, and will not be elaborated here.

[0201] The data transmission method provided in this application includes: dividing a first image into multiple image blocks; determining a first value R based on the maximum throughput and minimum throughput of a first channel; determining a third image block from every R image blocks in the multiple image blocks based on the first value, and determining the image blocks other than the third image block as a fourth image block; determining whether the total data volume of the third image block is equal to the minimum throughput of the first channel; if so, determining the third image block as the first image block and the fourth image block as the second image block; if not, selecting a sub-image block from the fourth image block, determining the third image block and the selected sub-image block as the first image block, and determining the image blocks other than the sub-image blocks in the fourth image block as the second image block. The total data volume of the selected sub-image block and the third image block is equal to the minimum throughput of the first channel. Transmitting the first image block and the second image block through the first channel, wherein the transmission priority of any first image block is greater than the transmission priority of any second image block. By first determining the third image block that must be transmitted, and then reallocating the image blocks into the first image block and the second image block based on the division of the third and fourth image blocks and the throughput of the first channel, as many image blocks as possible can be transmitted on the basis of the minimum throughput of the first channel, thereby improving the display effect of the first image on the receiving device. This process can also ensure the successful transmission of the selected third image block, thus effectively ensuring the stability of data transmission.

[0202] Based on the above embodiments, when transmitting image blocks through the first channel, this application can transmit multiple image blocks through two different queues, or it can transmit multiple image blocks through the same queue. The two different implementation methods are described below.

[0203] First, combine Figure 12 and Figure 13 The implementation method of transmitting data through two different queues will be explained. Figure 12 Flowchart of the data transmission method provided in the embodiments of this application Figure 3 , Figure 13 A schematic diagram of the first queue and the second queue provided for the application embodiment.

[0204] like Figure 12 As shown, the method includes:

[0205] S1201. Store the first image block into the first queue.

[0206] S1202, Store the second image block into the second queue.

[0207] In this embodiment, the first image block and the second image block are stored in different queues, see [link to relevant documentation]. Figure 13 The first image block is stored in the first queue, and the second image block is stored in the second queue. Both the first and second queues are buffer queues in the first channel. Transmitting image blocks through queues can ensure the orderliness of image block transmission and avoid chaos.

[0208] S1203. Transmit image blocks in the first queue through the first channel, and after the image blocks in the first queue are transmitted, transmit image blocks in the second queue. The transmission priority of the first queue is greater than the transmission priority of the second queue.

[0209] Specifically, because the transmission priority of the first image block needs to be higher than that of the second image block, this embodiment sets the transmission priority of the first queue to be higher than that of the second queue. Thus, when there is an image block in the first queue, the image block in the first queue is transmitted first through the first channel. Only when the image block in the first queue has been transmitted, that is, when the first queue is empty, is the image block in the second queue transmitted through the first channel. Therefore, it can be understood that the second image block is transmitted in a best-effort manner.

[0210] In one possible implementation, for example, the first queue could be an Enhanced Distributed Channel Access (EDCA) queue.

[0211] The data transmission method provided in this application includes: storing a first image block in a first queue; storing a second image block in a second queue; transmitting the image blocks in the first queue through a first channel; and transmitting the image blocks in the second queue after the transmission of the image blocks in the first queue is completed, wherein the transmission priority of the first queue is higher than the transmission priority of the second queue. By storing the first image block and the second image block in different queues, the orderly transmission of image blocks can be guaranteed. Furthermore, by setting the priority of the first queue to be higher than the priority of the second queue, the first image block can be transmitted with priority, thereby ensuring the successful transmission of the first image block.

[0212] Secondly, combine Figure 14 and Figure 15 The implementation method of transmitting multiple image blocks through the same queue is explained. Figure 14 Flowchart of the data transmission method provided in the embodiments of this application Figure 4 , Figure 15 A schematic diagram of the third queue provided for the application embodiment.

[0213] like Figure 14 As shown, the method includes:

[0214] S1401. Determine whether the remaining storage space of the third queue is less than the total data volume of the first image block. If yes, execute S1402; otherwise, execute S1403.

[0215] Specifically, in this embodiment, the first image block and the second image block are stored in the same queue. To ensure that the first image block is transmitted with priority, it is necessary to ensure that for the same first image, when a first image block exists, the first image block is always at the head of the queue. In other words, the first image block is always in front of the second image block.

[0216] Meanwhile, when the first image block is generated from the first image, it needs to be stored in the third queue. However, before storing, we first need to determine whether the third queue can store the first image block. Specifically, we need to determine whether the remaining storage space of the third queue is less than the total data volume of the first image block.

[0217] S1402, Delete at least one second image block from the tail of the third queue, wherein the total data amount of the deleted at least one second image block is greater than or equal to the total data amount of the first image block.

[0218] If the remaining storage space of the third queue is less than the total data volume of the first image block, it indicates that the first image block cannot be stored in the third queue at present. In order to ensure the effective transmission of the first image block, this embodiment deletes at least one second image block from the tail of the third queue. The specific number of second image blocks deleted depends on the difference between the current remaining storage space and the total data volume of the first image block to be sent. In this embodiment, the total data volume of the at least one deleted second image block is set to be greater than or equal to the total data volume of the first image block.

[0219] In other words, as long as the deletion allows all M first image blocks to be stored in the third queue, it is sufficient.

[0220] It is understandable that, due to the first-in-first-out (FIFO) nature of the queue, the deleted second image block in this embodiment is actually the second image block of the previous first image, see [link / reference]. Figure 15 , Figure 15 Different shadow effects are used to represent the image blocks corresponding to two different first images. By deleting the second image blocks that were not sent in the previous first image, it can be ensured that the first image blocks can be sent first regardless of which first image is being used.

[0221] S1403. Store the first image block and the second image block into the third queue. In the third queue, the first image block is located before the second image block.

[0222] If the remaining storage space of the third queue is not less than the total data volume of the first image block, or if the remaining storage space of the current third queue is sufficient to store the first image block to be sent after deleting at least one second image block at the tail of the queue, then the first image block will be stored in the third queue first. After the first image block is stored, if there is still remaining storage space, then part or all of the second image block will be stored in the third queue.

[0223] See Figure 15 The first image block stored in the third queue is a different image block from the image blocks that already exist in the previous queues.

[0224] The data transmission method provided in this application includes: determining whether the remaining storage space of a third queue is less than the total data volume of a first image block; if so, deleting at least one second image block from the tail of the third queue, wherein the total data volume of the deleted at least one second image block is greater than or equal to the total data volume of the first image block; if not, storing the first image block and the second image block in the third queue, where the first image block is located before the second image block. Thus, when the first image block and the second image block are stored in the same queue, for the same first image, it is guaranteed that the first image block is stored before the second image block. Furthermore, when the next first image generates a first image block, the second image block in the queue can be deleted to ensure that the first image block is preferentially stored in the third queue, thereby effectively guaranteeing the priority transmission of the third image block.

[0225] In the above Figure 12 and Figure 14 Based on the embodiments described, the data transmission method provided in this application also sets a maximum transmission duration for the first image block and the second image block respectively, so as to avoid the first image block and the second image block occupying the queue for a long time, resulting in low data transmission efficiency or queue congestion.

[0226] Understandably, the maximum transmission duration is used to indicate the maximum duration that the current image block transmission can occupy in the queue. It can be understood as a timer. When the timer expires, if the current image block has not been transmitted, the current image block will be removed from the third queue, or a retransmission can be attempted after the preset duration, or the image block can be discarded directly.

[0227] In this embodiment, the maximum transmission time of the first image block is greater than the maximum transmission time of the second image block. By setting the maximum transmission time of the first image block to be larger, the transmission success rate of the first image block can be effectively guaranteed.

[0228] Of course, this means that the first image block may fail to be transmitted. Those skilled in the art will understand that this embodiment only guarantees the successful transmission of the first image block at the throughput level of the first channel. If the first image block fails to be transmitted for a long time due to data corruption, data errors, etc., such problems are unavoidable. In this case, deleting the first image block can effectively improve the transmission efficiency of the remaining image blocks and avoid queue congestion.

[0229] Based on the above embodiments, this embodiment can further perform a series of encoding processes on the image blocks before determining the third and fourth image blocks. The following describes the process in conjunction with... Figure 16 To explain, Figure 16 Flowchart of the data transmission method provided in the embodiments of this application Figure 5 .

[0230] like Figure 16 As shown, the method includes:

[0231] S1601. Perform discrete cosine transform, quantization, and / or sorting on multiple image blocks to obtain the processed image blocks.

[0232] The Discrete Cosine Transform (DCT) is a transform related to the Fourier Transform. It can be used for image compression, converting pixel-domain signals to the frequency domain and exhibiting good decorrelation performance. DCT processing creates favorable conditions for subsequent quantization and Huffman coding processes in image encoding, and the resulting DCT coefficient matrix is ​​obtained after DCT processing.

[0233] Furthermore, this embodiment can also perform quantization processing on multiple image blocks. Quantization processing involves converting the continuous brightness variation range corresponding to a pixel into a single specific digital value. Specifically, the quantization process is essentially an optimization process of the DCT coefficient matrix. It utilizes the characteristic that the human eye is insensitive to high-frequency components to achieve significant data simplification.

[0234] Furthermore, this embodiment can also perform sorting processing on the image. Specifically, the sorting processing is ZigZag sorting. Since the importance of the coefficient matrix is ​​arranged to a certain extent according to the ZigZag pattern, the important elements at the beginning of this coefficient matrix can be extracted through the ZigZag pattern as the features of the first image in the frequency domain, and then used for classification processing, etc., to achieve the effect of dimensionality reduction.

[0235] Those skilled in the art will understand that one or more of the encoding processes described above can be performed, and this embodiment does not limit this.

[0236] In an optional embodiment, when compressing the first image, in addition to using DCT processing, Singular Value Decomposition (SVD) processing can also be used for image compression. Specifically, Singular Value Decomposition of a matrix is ​​a matrix decomposition method. By using Singular Value Decomposition of a matrix, the main information of the matrix can be extracted, thereby restoring information similar to the original matrix with less data than the original matrix, so as to achieve compression of the first image.

[0237] It should be noted that when using SVD processing to compress images, the sorting process described above is not necessary.

[0238] The specific implementation methods of the various encoding processes described above can be found in existing technologies, and will not be elaborated here.

[0239] By encoding each image block, the amount of data can be effectively reduced, thereby improving data transmission efficiency in subsequent data processing.

[0240] S1602. Perform joint compression on each first number of processed image blocks to obtain multiple image blocks after joint compression.

[0241] Specifically, the above embodiments introduce a first quantity R, which, in addition to indicating the sampling unit, can also be used in this embodiment for joint compression of image blocks.

[0242] Specifically, in this embodiment, each R processed image blocks are jointly compressed. The specific processing method for this joint compression can be Huffman coding compression, so as to obtain multiple image blocks after joint compression.

[0243] By jointly compressing multiple image blocks, the compression ratio of the image blocks can be effectively improved, thereby effectively increasing the data transmission rate in subsequent data transmission processes.

[0244] The data transmission method provided in this invention includes: performing discrete cosine transform processing, and / or quantization processing, and / or sorting processing on multiple image blocks to obtain processed image blocks. Joint compression is then performed on every first number of processed image blocks to obtain multiple jointly compressed image blocks. By encoding and jointly compressing the image blocks, the amount of transmitted data can be effectively reduced, thereby improving data transmission efficiency.

[0245] The following is combined Figure 17 The virtual modules included in the data transmission apparatus provided in the embodiments of this application will be described. Figure 17 Schematic diagram of the data transmission device provided in the embodiments of this application Figure 1 .like Figure 17 As shown, the device 170 includes: a division module 1701, a determination module 1702, and a transmission module 1703.

[0246] The partitioning module 1701 is used to divide the first image into multiple image blocks;

[0247] The determining module 1702 is configured to determine at least one first image block and at least one second image block among the plurality of image blocks based on the throughput of the first channel, wherein the total data amount of the first image block is less than or equal to the minimum throughput of the first channel.

[0248] The transmission module 1703 is used to transmit the first image block and the second image block through the first channel, wherein the transmission priority of any first image block is greater than the transmission priority of any second image block.

[0249] In one possible implementation, the transmission module 1703 is specifically used for:

[0250] Store the first image block into the first queue;

[0251] Store the second image block into the second queue;

[0252] The image blocks in the first queue are transmitted through the first channel, and the image blocks in the second queue are transmitted after the image blocks in the first queue have been transmitted. The transmission priority of the first queue is greater than that of the second queue.

[0253] In one possible implementation, the transmission module 1703 is specifically used for:

[0254] The first image block and the second image block are stored in a third queue, in which any first image block is located before any second image block;

[0255] Image blocks in the third queue are transmitted sequentially through the first channel.

[0256] In one possible implementation, the transmission module 1703 is further configured to:

[0257] Before storing the first image block into the third queue, if the remaining storage space of the third queue is less than the total data amount of the first image block, then at least one second image block at the tail of the third queue is deleted, wherein the total data amount of the deleted at least one second image block is greater than or equal to the total data amount of the first image block.

[0258] In one possible implementation, the maximum transmission duration of the first image block is greater than the maximum transmission duration of the second image block.

[0259] In one possible implementation, the transmission module 1703 is specifically used for:

[0260] Obtain the first modulation and coding scheme (MCS) order corresponding to the first image block, and obtain the second MCS order corresponding to the second image block, wherein the first MCS order is less than the second MCS order;

[0261] The first image block and the second image block are transmitted through the first channel, wherein the MCS order when transmitting the first image block through the first channel is the first MCS order, and the MCS order when transmitting the second image block through the first channel is the second MCS order.

[0262] In one possible implementation, the determining module 1702 is specifically used for:

[0263] A first value R is determined based on the maximum throughput of the first channel and the minimum throughput of the first channel, wherein R is an integer greater than or equal to 1;

[0264] According to R, a third image block is determined in every R image blocks among the plurality of image blocks, and the image blocks other than the third image block are determined as the fourth image block;

[0265] If the total data volume of the third image block is equal to the minimum throughput of the first channel, then the third image block is determined as the first image block, and the fourth image block is determined as the second image block.

[0266] In one possible implementation, if the total data volume of the third image block is less than the value corresponding to the minimum throughput of the first channel, then the determining module 1702 is further configured to:

[0267] In the fourth image block, a sub-image block is selected, and the third image block and the selected sub-image block are determined as the first image block, wherein the total data volume of the selected sub-image block and the third image block is equal to the minimum throughput of the first channel;

[0268] The image blocks in the fourth image block other than the sub-image blocks are identified as the second image blocks.

[0269] In one possible implementation, the determining module 1702 is specifically used for:

[0270] Based on R, the Kth image block in every R image blocks is determined as the third image block.

[0271] In one possible implementation, the determining module 1702 is specifically used for:

[0272] Determine the first mean of the pixel values ​​for each R image patch;

[0273] The target image block in each of the R image blocks is determined as the third image block, wherein the target image block is the image block with the smallest difference between the average pixel value and the first average pixel value.

[0274] In one possible implementation, the ratio of the minimum throughput of the first channel to the maximum throughput of the first channel is equal to the ratio of the value 1 to R.

[0275] In one possible implementation, the determining module 1702 is further configured to:

[0276] Before determining the third image block in each of the R image blocks according to the R, the plurality of image blocks are subjected to discrete cosine transform processing, and / or quantization processing, and / or sorting processing to obtain processed image blocks;

[0277] Each R processed image blocks are jointly compressed to obtain multiple jointly compressed image blocks.

[0278] The following is combined Figure 18 The virtual modules included in the data transmission apparatus provided in the embodiments of this application and their corresponding execution flows are described below. Figure 18 Schematic diagram of the data transmission device provided in the embodiments of this application Figure 2 .like Figure 18 As shown, there are a partitioning module 1801, a determination module 1802, and a transmission module 1803.

[0279] The segmentation module 1801 divides the first image into multiple image blocks. Among the possible implementations, there are three possible processing methods:

[0280] Method 1: Sample multiple image blocks to obtain the third and fourth image blocks;

[0281] The second method involves performing DCT / SVD processing, quantization, and sorting on multiple image blocks, then jointly compressing R (the first number) image blocks, and selecting the third and fourth image blocks from the jointly compressed image blocks.

[0282] The third method involves sampling multiple image blocks, followed by DCT processing, quantization, sorting, and joint compression to obtain the third and fourth image blocks.

[0283] In actual implementation, those skilled in the art can choose the required implementation method according to actual needs to obtain the third image block and the fourth image block.

[0284] It should be noted that the second implementation method, which selects the third and fourth image blocks from the jointly compressed image blocks, is similar to the sampling process, and the sampling process described above corresponds to the above-mentioned... Figure 4The implementation of step S403 in the embodiment will not be described in detail here. Furthermore, the DCT / SVD processing, quantization processing, and sorting processing described above can be selected according to actual needs; not all of them need to be executed. For specific implementation details, please refer to the above. Figure 16 The content described in the examples.

[0285] Next, the determining module 1802 needs to determine the first image block and the second image block from multiple image blocks. First, the image blocks are divided into the third image block and the fourth image block. However, since the number of the third image block and the fourth image block may not meet the requirements, in order to ensure that as many image blocks as possible are transmitted on the basis of the minimum throughput of the first channel, this embodiment redistributes the third image block (Must block) and the fourth image block (Not Must block) to obtain the first image block (Key block) and the second image block (Not Key block).

[0286] Next, transmission module 1803 transmits data, see below. Figure 18 Multiple image blocks are transmitted through the transmission queue in the first channel. At the same time, a timer is set for each first image block and each second image block in the transmission queue to indicate the maximum transmission duration of the current image block. Each image block in the queue is wirelessly transmitted through the standard WIFI transmission unit of the transmitting device.

[0287] Next is the receiving device receiving the data. Specifically, the standard WIFI receiving unit of the receiving device receives the data and restores the received image blocks to obtain the third and / or fourth image blocks. Specifically, because the transmitting device has redistributed the image blocks, the receiving device must restore the redistributed image blocks accordingly.

[0288] Furthermore, the receiving device decompresses the restored image blocks according to Huffman coding, and puts the decompressed image blocks into a buffer queue. Each image block in the buffer queue can also have its own timer, which is used to indicate the maximum parsing time of the image block. Next, each image block in the buffer queue is subjected to inverse DCT processing / inverse SVD processing to obtain the original image block, and the display buffer of the first image is performed according to each image block to display the first image on the display screen of the receiving device.

[0289] It should be noted that the receiving device will only perform Huffman coding decompression if the transmitting device performs joint compression using Huffman coding. Similarly, the receiving device will only perform inverse DCT / inverse SVD processing if the transmitting device performs DCT / SVD processing. Therefore, these two processes are not always required, and are thus marked with dashed boxes.

[0290] The data processing apparatus provided in this embodiment adaptively adjusts the division of the first image block and the second image block in the first image according to the changes in the throughput of the first channel, thereby effectively ensuring the successful transmission of key image blocks. When the first channel experiences jitter, the first image received by the receiving device will become blurred, but the user experience is minimal, thus balancing channel delay and channel jitter. Furthermore, in this embodiment, when wireless transmission delay occurs, the receiving device does not need to report the first channel status to the transmitting device for scheduling; instead, the transmitting device adaptively adjusts, effectively reducing data transmission delay. Simultaneously, this embodiment uses a standard WiFi transmitting unit and a standard WiFi receiving unit, transmitting data via the standard WiFi protocol, thus ensuring compatibility with existing WiFi transmission protocols and effectively guaranteeing the applicability of wireless data transmission. Moreover, by jointly compressing multiple image blocks, the data compression ratio can be effectively increased, improving the data transmission volume.

[0291] Figure 19 This is a schematic diagram of the hardware structure of the data transmission device provided in the embodiments of this application, such as... Figure 19 As shown, the data transmission device 190 of this embodiment includes: a memory 1901, a processor 1902, and a radio frequency transceiver 1903. The memory 1901 is used to store programs or instructions.

[0292] In one possible implementation, processor 1902 can achieve... Figures 17-18 The functions of the partitioning module 1701 and the determination module 1702 are implemented by the radio frequency transceiver 1903. Figures 17-18 The functions of the transmission module 1703 are detailed in the descriptions provided in the foregoing method and device embodiments.

[0293] Alternatively, the memory 1902 can be either standalone or integrated with the processor 1901.

[0294] When the memory 1902 is set up independently, the data transmission device also includes a bus 1904 for connecting the memory 1902 and the processor 1901.

[0295] Optionally, the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps in the business processing method embodiments disclosed in this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0296] This application also provides a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the data transmission method performed by the data transmission device described above.

[0297] This application provides a computer program product, which includes instructions that, when executed, cause a computer to perform the aforementioned data transmission method.

[0298] This application provides a system-on-a-chip (SoC) or system-on-a-chip (SoC) that can be applied to a terminal device. The SoC or SoC includes at least one communication interface, at least one processor, and at least one memory. The communication interface, memory, and processor are interconnected via a bus. The processor executes instructions stored in the memory, enabling the terminal device to perform the aforementioned data transmission method.

[0299] All or part of the steps in the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof.

[0300] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0301] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0302] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0303] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of the embodiments of this application and their equivalents, then the embodiments of this application are also intended to include these modifications and variations.

[0304] In the embodiments of this application, the term "comprising" and its variations can refer to a non-limiting inclusion; the term "or" and its variations can refer to "and / or". In the embodiments of this application, the terms "first", "second", etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In the embodiments of this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

Claims

1. A data transmission method, characterized by, include: The first image is divided into multiple image blocks, and the multiple image blocks are of the same size; Based on the throughput of the first channel, at least one first image block and at least one second image block are determined from the plurality of image blocks, wherein the total data volume of the first image block is less than or equal to the minimum throughput of the first channel; The first image block and the second image block are transmitted through the first channel, wherein the transmission priority of any first image block is greater than the transmission priority of any second image block; The step of determining at least one first image block and at least one second image block from the plurality of image blocks based on the throughput of the first channel includes: The ratio of the maximum throughput of the first channel to the minimum throughput of the first channel is determined as a first value R, wherein R is an integer greater than or equal to 1; Determine the first mean of the pixel values ​​for each R image patch; The target image block in each of the R image blocks is determined as the third image block, wherein the target image block is the image block with the smallest difference between the average pixel value and the first average pixel value; The image block other than the third image block is designated as the fourth image block; If the total data volume of the third image block is equal to the minimum throughput of the first channel, then the third image block is determined as the first image block, and the fourth image block is determined as the second image block.

2. The method of claim 1, wherein, The transmission of the first image block and the second image block through the first channel includes: Store the first image block into the first queue; Store the second image block into the second queue; The image blocks in the first queue are transmitted through the first channel, and the image blocks in the second queue are transmitted after the image blocks in the first queue have been transmitted. The transmission priority of the first queue is greater than that of the second queue.

3. The method of claim 1, wherein, The transmission of the first image block and the second image block through the first channel includes: The first image block and the second image block are stored in a third queue, in which any first image block is located before any second image block; Image blocks in the third queue are transmitted sequentially through the first channel.

4. The method of claim 3, wherein, Before storing the first image block to the third queue, the method further includes: If the remaining storage space of the third queue is less than the total data amount of the first image block, then at least one second image block at the tail of the third queue is deleted, wherein the deleted at least one second image block is an unsent second image block of the previous first image, and the total data amount of the deleted at least one second image block is greater than or equal to the total data amount of the first image block.

5. The method of claim 1, wherein, The maximum transmission duration of the first image block is greater than the maximum transmission duration of the second image block.

6. The method according to any one of claims 1-5, characterized in that, The transmission of the first image block and the second image block through the first channel includes: Obtain at least one first modulation and coding scheme (MCS) order corresponding to the first image block, and obtain at least one second MCS order corresponding to the second image block, wherein any first MCS order is less than any second MCS order; The first image block and the second image block are transmitted through the first channel, wherein the MCS order when transmitting the first image block through the first channel is the first MCS order, and the MCS order when transmitting the second image block through the first channel is the second MCS order.

7. The method according to claim 1, characterized in that, If the total data volume of the third image block is less than the value corresponding to the minimum throughput of the first channel, the method further includes: In the fourth image block, a sub-image block is selected, and the third image block and the selected sub-image block are determined as the first image block, wherein the total data volume of the selected sub-image block and the third image block is equal to the minimum throughput of the first channel; The image blocks in the fourth image block other than the sub-image blocks are identified as the second image blocks.

8. The method according to claim 1, characterized in that, Before determining the target image block in each of the R image blocks as the third image block, the method further includes: The plurality of image blocks are subjected to discrete cosine transform processing, and / or quantization processing, and / or sorting processing to obtain processed image blocks; Each R processed image blocks are jointly compressed to obtain multiple jointly compressed image blocks.

9. A data transmission device, characterized in that, include: A segmentation module is used to divide the first image into multiple image blocks, wherein the multiple image blocks are of the same size; The determining module is configured to determine at least one first image block and at least one second image block among the plurality of image blocks based on the throughput of the first channel, wherein the total data volume of the first image block is less than or equal to the minimum throughput of the first channel; A transmission module is configured to transmit the first image block and the second image block through the first channel, wherein the transmission priority of any first image block is greater than the transmission priority of any second image block; The determining module is specifically used for: The ratio of the maximum throughput of the first channel to the minimum throughput of the first channel is determined as a first value R, wherein R is an integer greater than or equal to 1; Determine the first mean of the pixel values ​​for each R image patch; The target image block in each of the R image blocks is determined as the third image block, wherein the target image block is the image block with the smallest difference between the average pixel value and the first average pixel value; The image block other than the third image block is designated as the fourth image block; If the total data volume of the third image block is equal to the minimum throughput of the first channel, then the third image block is determined as the first image block, and the fourth image block is determined as the second image block.

10. The apparatus according to claim 9, characterized in that, The transmission module is specifically used for: Store the first image block into the first queue; Store the second image block into the second queue; The image blocks in the first queue are transmitted through the first channel, and the image blocks in the second queue are transmitted after the image blocks in the first queue have been transmitted. The transmission priority of the first queue is greater than that of the second queue.

11. The apparatus according to claim 9, characterized in that, The transmission module is specifically used for: The first image block and the second image block are stored in a third queue, in which any first image block is located before any second image block; Image blocks in the third queue are transmitted sequentially through the first channel.

12. The apparatus according to claim 11, characterized in that, The transmission module is also used for: Before storing the first image block into the third queue, if the remaining storage space of the third queue is less than the total data amount of the first image block, then at least one second image block at the tail of the third queue is deleted, wherein the deleted at least one second image block is an unsent second image block of the previous first image, and the total data amount of the deleted at least one second image block is greater than or equal to the total data amount of the first image block.

13. The apparatus according to claim 9, characterized in that, The maximum transmission duration of the first image block is greater than the maximum transmission duration of the second image block.

14. The apparatus according to claim 9, characterized in that, The transmission module is specifically used for: Obtain the first modulation and coding scheme (MCS) order corresponding to the first image block, and obtain the second MCS order corresponding to the second image block, wherein the first MCS order is less than the second MCS order; The first image block and the second image block are transmitted through the first channel, wherein the MCS order when transmitting the first image block through the first channel is the first MCS order, and the MCS order when transmitting the second image block through the first channel is the second MCS order.

15. The apparatus according to claim 9, characterized in that, If the total data volume of the third image block is less than the value corresponding to the minimum throughput of the first channel, the determining module is further configured to: In the fourth image block, a sub-image block is selected, and the third image block and the selected sub-image block are determined as the first image block, wherein the total data volume of the selected sub-image block and the third image block is equal to the minimum throughput of the first channel; The image blocks in the fourth image block other than the sub-image blocks are identified as the second image blocks.

16. The apparatus according to claim 9, characterized in that, The determining module is also used for: Before determining the target image block in each of the R image blocks as the third image block, the multiple image blocks are subjected to discrete cosine transform processing, and / or quantization processing, and / or sorting processing to obtain the processed image blocks; Each R processed image blocks are jointly compressed to obtain multiple jointly compressed image blocks.

17. A data transmission device, characterized in that, include: The device includes a memory, a processor, and a radio frequency transceiver, wherein the memory stores programs or instructions that, when executed by the processor, cause the device to perform the method as described in any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 8.

Images