Video frame sending processing method and system, electronic equipment and readable storage medium

Abstract

The application discloses a video frame sending processing method and system, electronic equipment and a readable storage medium. The method comprises the following steps: dividing a video frame into multiple image blocks, wherein the multiple image blocks are respectively sent to different display terminals; each image block is respectively split into at least one data packet, wherein the number of data packets obtained after splitting different image blocks is the same or different; in the frame period of the video frame, the data packets of each image block are alternately sent until all the data packets of the multiple image blocks are sent out in the frame period. The application solves the problem that the display terminal processing is not utilized when the video frame is divided into multiple image blocks and simultaneously sent to multiple display terminals in the prior art, so that the multiple data blocks are dispersed and sent in one frame period, the smooth transmission of the video frame is realized, and the processing of the display terminal is facilitated.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of video transmission, in particular, to a video frame sending processing method and system, an electronic device and a readable storage medium. BACKGROUND

[0002] During the transmission of a video, the video is transmitted according to video frames, and a video frame is a complete image. The video frame is transmitted within a frame period. Assuming that 60 video frames are transmitted within 1 second, 1 second is equal to 1000 milliseconds (ms), and therefore the transmission time of each video frame is 16.66 ms, which is the frame period.

[0003] The video frame is transmitted from a sending end to a display end. In a general case, the video frame is transmitted from one sending end to one display end, and the display end directly displays the video frame after receiving the video frame. In addition to transmitting the video to one display end, the video can also be transmitted to multiple display ends for display. Figure 1 According to the related art, the video sending end corresponds to multiple display ends, as shown in FIG. 1. Figure 1 The video sending end (referred to as a sending end) transmits the video frame to multiple video receiving ends (referred to as receiving ends). Each receiving end is connected to a display end, and the receiving end transmits the received data to the display end. Figure 1 In the example shown in FIG. 1, the receiving end and the display end are separate. In actual applications, the display end can also include the function of the receiving end. In this case, the receiving end and the display end can be collectively referred to as a display end. Figure 1 In the example shown in FIG. 1, the video frame needs to be displayed on multiple display ends (display end 0 to display end n). Therefore, the video frame needs to be divided into multiple image blocks (image block 0 to image block n), and different image blocks need to be transmitted to different video receiving ends (video receiving end 0 to video receiving end n). After receiving the corresponding image block, the video receiving end transmits the received image block to the corresponding display end.

[0004] When transmitting the image block to multiple display ends, the image block obtained by division is usually transmitted to multiple display ends at the same time in the prior art. Since each image block is obtained by dividing the video frame, the size of each image block is smaller than the size of the video frame. This way of transmitting the image block can cause a large amount of data to be transmitted to multiple display ends at the beginning of a frame period, and an idle time can occur at the latter part of the frame period because the data of the video frame has been transmitted. This transmission method is not conducive to the processing of the video frame by the display end because a large amount of data needs to be processed by the display end at the first half of the frame period, and there is no data to be processed at the second half of the frame period. SUMMARY

[0005] Embodiments of the present application provide a video frame sending processing method, system, electronic device and readable storage medium to at least solve the problem that the display end processing is not utilized when the video frame is divided into multiple image blocks and sent to multiple display ends simultaneously.

[0006] According to an aspect of the present application, a video frame sending processing method is provided, comprising: dividing a video frame into multiple image blocks, wherein the multiple image blocks are sent to different display ends respectively; splitting each image block into at least one data packet respectively, wherein the number of data packets obtained after splitting different image blocks is the same or different; and alternately sending data packets of each image block within a frame period of the video frame until all data packets of the multiple image blocks are sent out within the frame period.

[0007] Further, alternately sending data packets of each image block comprises: obtaining a sending time of each data packet according to the amount of data sent out before sending of each data packet and a sending rate, wherein the same sending rate is used for sending each data packet; and starting to send the data packet at the sending time of the data packet using the average rate.

[0008] Further, obtaining a sending time of each data packet according to the amount of data sent out before sending of each data packet and a sending rate comprises: dividing the total amount of data of the video frame by the frame period to obtain the sending rate; determining a time length for sending data packets that have been sent out according to the amount of data of the data packets that have been sent out before the data packet and the sending rate; and adding the time length to a starting time of the frame period to obtain the sending time of the data packet, wherein the first data packet is sent at the starting time of the frame period.

[0009] Further, alternately sending data packets of each image block comprises: alternately sending data packets of each image block according to rounds, wherein in each round, the data packets that can be sent out by each image block are determined in sequence from the first image block to the Nth image block, and the data packets that can be sent out are sent out, and after the data packet sending in the round is completed, data packet sending in the next round is entered; the multiple image blocks comprise the first image block to the Nth image block, and N is the number of the multiple image blocks.

[0010] Further, the determining of the data packets that can be sent in each round for each image block in sequence from the first image block to the Nth image block comprises: for each of the first image block to the Nth image block, determining that only one data packet can be sent from each image block in the round; wherein if all data packets of an image block have been sent, the sending of data packets jumps to the next image block.

[0011] Further, the determining of the data packets that can be sent in each round for each image block in sequence from the first image block to the Nth image block comprises: for each of the first image block to the Nth image block, determining that only one data packet can be sent from each image block in the round; wherein if all data packets of an image block have been sent, the sending of data packets jumps to the next image block.

[0012] Further, the predetermined condition is: (the amount of data that has been sent in the current image block / the amount of data that has been sent in the video frame) <= (the amount of data of the current image block / the amount of data of the video frame).

[0013] Further, the determining of the data packets that can be sent in each round for each image block in sequence from the first image block to the Nth image block comprises: determining the amount of data packets sent in each round according to the total amount of data packets in the plurality of image blocks and the number of the plurality of image blocks, wherein if the quotient of the total amount of data packets and the number of the plurality of image blocks is an integer, the amount of data packets sent in each round is the quotient, and if the quotient is a decimal, the amount of data packets sent in each round is the upward rounding or the downward rounding of the quotient; determining the amount of data packets that can be sent in each round for each image block according to the proportion of the amount of data packets of each image block in the total amount of data packets of the plurality of image blocks.

[0014] Further, the size of each data packet is the same.

[0015] According to another aspect of the present application, a video frame sending processing apparatus is also provided, characterized in that comprising: a segmentation module configured to divide a video frame into a plurality of image blocks, wherein the plurality of image blocks are respectively sent to different display terminals; a division module configured to divide each image block into at least one data packet, wherein the number of data packets obtained after the division of different image blocks is the same or different; and a sending module configured to alternately send the data packets of each image block in the frame period of the video frame until all data packets of the plurality of image blocks are sent out in the frame period.

[0016] According to another aspect of the present application, a video frame sending processing system is also provided, comprising a sending end and a plurality of display ends, wherein the sending end is configured to perform the method steps described above.

[0017] According to another aspect of the present application, an electronic device is also provided, comprising a memory and a processor; wherein the memory is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the method steps described above.

[0018] According to another aspect of the present application, a readable storage medium is also provided, having computer instructions stored thereon, wherein the computer instructions are executed by a processor to implement the method steps described above.

[0019] In the embodiments of the present application, the video frame is divided into a plurality of image blocks, wherein the plurality of image blocks are sent to different display ends respectively; each image block is divided into at least one data packet respectively, wherein the number of data packets obtained after the division of different image blocks is the same or different; within the frame period of the video frame, the data packets of each image block are sent alternately until all the data packets of the plurality of image blocks are sent out within the frame period. The present application solves the problem that the display end cannot process when the video frame is divided into a plurality of image blocks and sent to a plurality of display ends simultaneously in the prior art, so that the plurality of data blocks are dispersed and sent within a frame period, and the smooth transmission of the video frame is realized, which is beneficial to the processing of the display end. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and are used to interpret the illustrative embodiments of the present application and their descriptions, and do not constitute improper limitations to the present application. In the drawings:

[0021] Figure 1 is a schematic diagram of a video sending end corresponding to a plurality of display ends according to the related art;

[0022] Figure 2 is a schematic diagram of video frame block division according to the embodiments of the present application;

[0023] Figure 3 is a flowchart of a video frame sending processing method according to the embodiments of the present application;

[0024] Figure 4 is a structural schematic diagram of a video smooth transmission system according to the embodiments of the present application;

[0025] Figure 5 is a schematic diagram of round-robin scheduling according to different receiving ends according to the embodiments of the present application;

[0026] Figure 6is a schematic diagram of the coarse smoothing processing in Example 1 according to an embodiment of the present application; and

[0027] Figure 7 is a schematic diagram of the fine smoothing processing in Example 2 according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0029] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0030] The video frame sending method in the following embodiments is applicable to various occasions requiring video processing, such as video conference, video live broadcast, video on demand, video call, as long as the video frame needs to be split into multiple image blocks for transmission, the following embodiments are applicable. First, the technical terms involved in the following embodiments will be described.

[0031] DDR: the Chinese name is double data rate synchronous dynamic random access memory, which is one of the memories, and DDR as a kind of memory can be used to cache video data.

[0032] Video frame: can be understood as an image, multiple video frames are played continuously on the time axis to constitute a video, therefore, the video frame can also be called frame image.

[0033] Frame rate: frame rate is a measure for measuring the number of displayed frames. The measurement unit is frames per second (FPS) or "hertz" (Hz), for example, if the frame rate is 60, it means that 60 video frames will be displayed in 1 second.

[0034] Frame period: the time length of a video frame, for example, the frame rate is 60, that is, 60 video frames are transmitted in 1 second, 1 second is equal to 1000 milliseconds (ms), therefore, 1000 milliseconds / 60 = 16.66 milliseconds is the frame period, all data of a video frame needs to be transmitted within a frame period.

[0035] Image block: a video frame as an image is composed of pixel points, the pixel points are arranged in rows and columns, Figure 2 is a schematic diagram of the video frame block according to an embodiment of the present application, in Figure 2 a circle is used to represent a pixel, such asFigure 2 As shown in the figure, the video frame includes a plurality of pixels, and the video frame can be divided into blocks in a row manner or in a column manner, Figure 2 The block division manner shown in the figure is only an example, and any other block division manner can be used, and Figure 2 In the figure, the video frame is divided into image block 1, image block 2, image block 3, and image block 4. It should be noted that the data amount included in each image block is different, and the data amount included in each image block can also be the same. After the video frame is divided into a plurality of image blocks, each image block can be transmitted from the same sending end to different display ends for display. Figure 2

[0036] Smooth processing of video transmission: The smooth processing of video transmission can be applied in a video transmission control system, in which there is a video sending end and a video receiving end. The smooth processing of video transmission needs to ensure that the video sending end disperses the data of a video frame within a certain time to the video receiving end, rather than transmitting all the data of the video frame in the early stage of the time. For example, the data amount of a video frame is 1 MB (megabyte), and the frame period is 16.66 ms (milliseconds). If the smooth processing of video transmission is not performed, it is likely that 1 MB of data will be transmitted in the first 4 milliseconds of the frame period, which will cause the receiving end to need to process a large amount of data in the first 4 milliseconds. However, if the smooth processing of video transmission is performed, 1 MB of data can be transmitted within the frame period of 16.66 ms, which effectively reduces the amount of data that needs to be processed by the receiving end in a unit of time, and also effectively reduces the performance requirements of the subsequent equipment.

[0037] After the video frame is divided into a plurality of image blocks, if all the image blocks are transmitted to the receiving end in parallel, all the data will be transmitted in the early stage of a frame period, and no data will be transmitted in the late stage of the frame period, and the receiving end needs to process a large amount of data in the early stage of the frame period. In another transmission manner, the first image block can be transmitted to the receiving end, and then the second image block is transmitted, and so on, until all the image blocks are transmitted. For this transmission manner, the display end for displaying the first image block will receive a large amount of data in a short time, and then receive less data in the other time of the frame period, and the same is true for the display ends for receiving other image blocks. Therefore, this transmission manner does not solve the problem of relatively smooth reception of image data by each display end in a frame period.

[0038] To solve the above problems, a video frame transmission processing method is provided in the following embodiments,​Figure 3 is a flow chart of a video frame sending processing method according to an embodiment of the present application, as shown in Figure 3 the following will be described in detail. Figure 3 the steps included in

[0039] Step S302, the video frame is divided into a plurality of image blocks, wherein the plurality of image blocks are respectively sent to different display terminals.

[0040] Step S304, each image block is respectively split into at least one data packet, wherein the number of data packets obtained after splitting different image blocks is the same or different.

[0041] In this step, the size of the data packet obtained after splitting each image block can be the same or different, and since the size of each image block can be the same or different, the number of data packets obtained after splitting each image block can also be the same or different.

[0042] Step S306, in the frame period of the video frame, the data packets of each image block are alternately sent until all data packets of the plurality of image blocks are sent out in the frame period.

[0043] In the above step, each image block is split into at least one data packet, and when sending data, the data is sent in units of data packets. Since the data packet is split from the image block, the size of the data packet is smaller than the size of the image block. In this case, for each image block, its data is not continuously sent to the display terminal, and the data packets of each image block are alternately sent to the corresponding display terminal. When receiving the data packet, the display terminal will have an interval (the interval is because the sending terminal is sending data packets to other display terminals at this time), so as to as far as possible to scatter the data packets sent to one display terminal in a frame period. Compared with the processing mode of continuously sending one image block to the display terminal, the above step makes the transmission of the video frame more smooth, and thus solves the problem of not utilizing the display terminal processing in the prior art when the video frame is divided into a plurality of image blocks and simultaneously sent to a plurality of display terminals, so as to scatter a plurality of data blocks in a frame period for sending, thereby realizing the smooth transmission of the video frame and being beneficial to the processing of the display terminal.

[0044] In order to ensure that the data packets are sent as smoothly as possible, in an optional embodiment, the same sending rate can be used to send each data packet, and the sending of each data packet can be continuous, which is conducive to calculating the time for sending each data packet. That is, in the optional embodiment, the alternately sending the data packets of each image block can include the following steps: obtaining the sending time of each data packet according to the amount of data that has been sent before the sending of each data packet and the sending rate, wherein the same sending rate is used to send each data packet; and starting to send each data packet at the sending time corresponding to the data packet using the average rate. By using the same sending rate to send the data packets, the sending end does not need to frequently change the sending rate, which is conducive to the processing of the sending end. For example, the frame period is 16.66 milliseconds, the data packets are sent at the beginning of the frame period, the speed of 100 bits per millisecond is used to send the data packets, the size of the first data packet is 160 bits, and the time for sending the first data packet is 1.6 milliseconds. The second data packet is sent from 1.6 milliseconds, the size of the second data packet is 100 bits, and the time for sending the second data packet is 1 millisecond. The third data packet is sent from 2.6 milliseconds, and the time for sending each data packet can be calculated in the same manner. In the example, the same sending rate is used for the data packets of each image block. Of course, different sending rates can also be used for the data packets of different image blocks. For example, the first image block is relatively large, and the data packets obtained by splitting the first image block are also relatively large. At this time, a relatively fast sending rate can be used according to the size of the data packets of the image block. The second image block is relatively small, and the data packets obtained by splitting the second image block are also relatively small. A relatively slow sending rate can be used, and so on. When calculating the sending time of the next data packet, the size of the data packet that has been sent and the corresponding sending rate can be used to calculate the sending time, and details are not described herein.

[0045] When sending data packets, it is expected that data packets are sent in the entire frame period, that is, the sending of data packets is continuous in the entire frame period, which is the most smooth sending mode. The data amount of a video frame can be calculated according to information such as the resolution of the video frame, and the frame period is known, so the sending rate can be calculated. That is, the sending time corresponding to each data packet can be obtained according to the data amount of the data packets that have been sent before the sending of each data packet and the sending rate, which can include the following steps: dividing the total data amount of the video frame by the frame period to obtain the sending rate; determining the time length for sending the data packets that have been sent according to the data amount of the data packets that have been sent before the data packet and the sending rate; and adding the time length to the starting time of the frame period to obtain the sending time of the data packet, wherein the first data packet is sent at the starting time of the frame period. For example, a 4K resolution video file has a picture resolution of 4096*2160 or 3840*2160 per frame. The following calculation is based on a resolution of 3840*2160. The resolution of 3860*2160 indicates that there are 3860*2160 pixel points, each pixel point has 3 sub-pixels (red, blue, and green), and each sub-pixel occupies 8 bits. Therefore, the data amount of one frame is 3840*2160*8*3 bits. In the case of a frame rate of 60 frames per second (fps), the corresponding frame period is 16.66 milliseconds, and the calculated sending rate is (3840*2160*8*3) / 16.66, with the unit of bit / millisecond. At the starting time of the frame period, the sending of data packets starts, so according to the calculated sending rate and the size of the data packet, the time for sending the data packet can be obtained, which is the time for starting to send the next data packet.

[0046] In the above embodiment, the sending rate can be calculated by the size of the video frame and the frame period. When the data packets are sent by the sending rate, data packets can be sent in the entire frame period, which is a relatively smooth sending mode for the sending end and is beneficial to the processing of the sending end. In the implementation and application, this mode is recommended for sending data packets. After the sending rate is determined, the alternating sending of data packets in the image block can be in many sending modes. In an optional embodiment, the sending can be performed in multiple rounds according to a round-by-round sending mode. Figure 2The video frame is divided into four image blocks. In the first round, a data packet is sent from image block 1, then from image block 2, then from image block 3, and then from image block 4. The second round of sending can be performed from image block 1. It should be noted that in each round, one data packet or multiple data packets can be sent from each image block. Alternatively, in each round, it can be determined whether a data packet needs to be sent from the image block. If there is no data packet to be sent from the image block in the round, the image block can be skipped and the data packet can be sent from the next image block. For example, image block 1 has five data packets, image block 2 has one data packet, image block 3 has two data packets, and image block 4 has three data packets. A rule is defined, which states that if the number of data packets to be sent from the image block is less than the number of data packets remaining in the previous image block, no data packet is sent from the image block in the next round. In the first round, one data packet is sent from image block 1, leaving four data packets. Image block 2 has one data packet, which is less than the four data packets, so no data packet is sent from image block 2 in the first round. Image block 3 has two data packets, which is greater than the one data packet in image block 2, so one data packet is sent from image block 3 in the first round. Image block 4 has three data packets, which is greater than the one data packet remaining in image block 3, so one data packet is sent from image block 4 in the first round. The second round of sending is then performed, using the same method as the first round. After five rounds, all data packets in the image blocks are sent. As can be seen from the example, if a data packet can be sent from an image block in a round, the data packet is sent. If no data packet can be sent from an image block in the round, the next image block is skipped and the data packet is sent from the next image block. In the alternative embodiment, the alternating sending of the data packets of the image blocks can include the following steps: alternating sending the data packets of the image blocks in rounds, wherein in each round, the data packets that can be sent from each image block are determined in order from the first image block to the Nth image block, and the data packets are sent, and after the data packets are sent in the round, the next round of data packet sending is entered; the image blocks include the first image block to the Nth image block, and N is the number of the image blocks.

[0047] In each round of data packet sending, one data packet or multiple data packets can be sent from each image block.

[0048] It should be noted that the data packets of each image block can support more rounds, and it can be specified that each image block can send at most one data packet in each round. For example, image block 1 has 5 data packets, if image block 1 sends 3 data packets in the first round and 2 data packets in the second round, then image block 1 can only support two rounds; if only 1 data packet is sent in a round, then image block 1 can support 5 rounds. In this way, the data packets of image block 1 can be dispersed as much as possible within the entire frame period. That is, in the optional embodiment, the data packets that each image block can send in each round are determined in the order of the first image block to the Nth image block in this round can include the following steps: for each image block in the first image block to the Nth image block, determine that at most one data packet can be sent from each image block in this round; wherein, in the case that the data packets of an image block have all been sent out, skip to the next image block for data packet transmission.

[0049] In the optional embodiment above, each image block can only send one data packet in each round. As a relatively easy implementation, one data packet of each image block can be sent in each round, and then the data packet of the next image block is sent, and so on, until all data packets of all image blocks are sent. For example, a video frame is divided into four image blocks, namely, the first image block, the second image block, the third image block and the fourth image block. The first image block is divided into three data packets, the second image block is divided into five data packets, the third image block is divided into three data packets, and the fourth image block is divided into three data packets. The sending sequence of the data packets is 1.1-2.1-3.1-4.1, where N.M represents the Mth data packet of the Nth image block, and 2.1 represents the first data packet of the second image block. The data packets are sent in the order of the first image block to the fourth image block. After three rounds, the data packets in the first image block, the third image block and the fourth image block are all sent, and therefore the fourth data packet and the fifth data packet of the second image block are sent continuously in the last period of the frame period. In another optional manner, during the process of dividing the image blocks into data packets, if it is desired that each image block has data packets sent in each round, each image block can be divided into the same number of data packets. In this case, since the sizes of the image blocks can be different, the sizes of the data packets of each image block after being divided into the same number of data packets will also be different. Alternatively, different image blocks can be divided into data packets of the same size, in which case different image blocks will include different numbers of data packets. In order to ensure that each image block has data packets sent in each round, the number of data packets sent in each round can be determined according to the total number of data packets in the plurality of image blocks and the number of the plurality of image blocks, where, in the case that the quotient of the total number of data packets and the number of the plurality of image blocks is an integer, the number of data packets sent in each round is the quotient, and in the case that the quotient is a decimal, the number of data packets sent in each round is the upward rounding or downward rounding of the quotient; the number of data packets that each image block can send in this round is determined according to the proportion of the number of data packets of each image block in the total number of data packets of the plurality of image blocks. For example, image block 1 has six data packets, image block 2 has four data packets, image block 3 has two data packets, and image block 4 has two data packets. All data packets can be sent through two rounds, in each round, three data packets of image block 1, two data packets of image block 2, and one data packet of image block 3 and image block 4 are sent, so that all image blocks have data packets sent in each round.

[0050] In another optional embodiment, in order to ensure that the data packets sent to each display end are also distributed in the entire frame period, for each image block, it is determined that only one data packet can be sent in a round; or it is determined whether the image block meets a predetermined condition in the round, if the predetermined condition is met, only one data packet of the image block can be sent in the round, and if the predetermined condition is not met, no data packet of the image block is sent in the round. The following two examples are used to illustrate this, Figure 4 is a structural schematic diagram of a video smooth transmission system according to an embodiment of the present application, and Figure 4 shows the system structure applied in the two examples, as shown in Figure 4 , the sending end reads a frame of video from the memory, performs smooth processing on the frame of video, and transmits the frame of video to receiving ends (receiving end 0 to receiving end n) through a switch chip. Different regions of the same frame of image need to be displayed on different display ends (display end 0 to display end n), that is, the same frame of image is divided into a plurality of image blocks, and each image block is sent to a different receiving end through the switch chip. The sending end needs to ensure that the video data going to each receiving end is evenly scattered in the frame period. The receiving end receives the image blocks from different sending ends, performs superposition processing on the image blocks, and outputs the image blocks to the display end for display.

[0051] For the sending end, the DDR storage unit is responsible for buffering the video frame data by row, and plays a buffering role between the memory reading module and the video segmentation and smooth sending module. The video segmentation and smooth sending module controls the video to be sent smoothly according to the parameters calculated by the smooth processing mechanism, and each row of video data is divided into a plurality of data packets for sending. The memory reading module reads the video data in the memory according to the parameters of the video segmentation and smooth sending module. The smooth mechanism of the video segmentation and smooth sending module can be referred to as a coarse smooth mechanism and a fine smooth mechanism. The coarse smooth mechanism will be introduced in Example 1 below, and the fine smooth mechanism will be introduced in Example 2.

[0052] Example 1

[0053] The image is divided into a plurality of image blocks and sent to different receiving ends, and each row of video data of each image block is divided into a plurality of packets for sending. Figure 5 is a schematic diagram of round scheduling according to different receiving ends according to an embodiment of the present application, as shown in Figure 5 , the round scheduling is performed according to different receiving ends, and only one data packet of the image block is sent each time. The sending time t of the data packet in the frame period is calculated according to the following formula (I) send When the frame period is counted to the time, the memory reading module reads the corresponding data packet from the DDR memory and sends it to the corresponding receiving end.

[0054] t send = (n send_of_total / Ntotal Formula (I)

[0055] Where: t send : The time when the current image is allowed to be sent, n send_of_total : The amount of data that has been sent for the current image, N total : Total data volume of the current image, T: Frame period.

[0056] In formula (i) above, the transmission time = (amount of data transmitted / current amount of image data) * frame period. Figure 6 This is a schematic diagram of coarse smoothing processing according to Example 1 of the embodiments of this application. The following is in conjunction with... Figure 6 The formula (I) will be explained as follows: Figure 6 As shown, taking a video frame size of 1920x4 and a frame period of 16.6ms as an example, the image of this video frame is divided into image block 0 and image block 1 by receiver 0 and receiver 1. The size of image block 0 is 1440x4 and the size of image block 1 is 480x4. The video data of image block 0 and image block 1 are evenly distributed over the frame period of 16.6ms. Assume that each line of image block 0 is divided into 3 data packets for transmission, that is, each data packet sends 480 pixels; each line of image block 1 is transmitted as 1 data packet, that is, each data packet sends 480 pixels; and the packet scheduling is performed in the order of image block 0 -> image block 1 -> image block 0 -> image block 1…

[0057] Image block 0, packet 0: The amount of data sent for the current image is 0. According to formula (I), the transmission time t0 of packet 0 in the frame period is 0 milliseconds (ms).

[0058] Image block 1, packet 0: The amount of data sent in the current image is 480. According to formula (I), the transmission time t1 of packet 0 in block 1 within the frame period is calculated as (480 / (1920*4))*16.6ms, which is (1 / 16)*16.6ms.

[0059] Image block 0, first data packet: The amount of data sent in the current image is 480*2. According to formula (I), the transmission time t2 of the first data packet of block 0 in the frame period is calculated as ((480*2) / (1920*4))*16.6ms, that is, (2 / 16)*16.6ms. ...

[0061] By analogy, the transmission time of each packet within the frame period can be calculated.

[0062] Example 2

[0063] In Example 2, a judgment method is added each time a data packet is sent to determine whether the data packets in the current image block meet a predetermined condition. If the predetermined condition is not met, the data packets in that image block are not sent in that round. The predetermined condition can be formulated as needed. In the above implementation, one predetermined condition was mentioned: if the number of unsent data packets in the current image block is less than the number of remaining data packets in the previous image block, then no data packets are sent from that image block in the next round. In this Example 2, another predetermined condition is proposed: (the amount of data already sent in the current image block / the amount of data already sent in the video frame) <= (the amount of data in the current image block / the amount of data in the video frame). The data packet sending method using this predetermined condition is described below.

[0064] The transmission time t of each packet is calculated according to formula (1). send Then, based on formula (II), the proportion of data already sent is determined in real time. When the amount of data sent for the image block is n... send_of_blobk When formula (ii) is satisfied, it indicates that t send The image block data can be sent at any time; otherwise, t send If the image block data cannot be sent at any time, it is necessary to calculate whether the amount of data sent for the next image block satisfies formula (II) according to the polling scheduling order.

[0065] n send_of_blobk / n send_of_total <=N blobk / N total Formula (II)

[0066] Where: n send_of_blobk : The amount of data sent for the current image patch, n send_of_total : The amount of data that has been sent for the current image, N blobk : The amount of data in the current image patch, N total : The total amount of data in the current image.

[0067] Figure 7 This is a schematic diagram of the fine smoothing process according to Example 2 of the embodiments of this application. The following is in conjunction with... Figure 7 The explanation of formula (ii) is as follows: Figure 3 As shown, similar to Example 1, the video frame is divided into image block 0 and image block 1 by receiver 0 and receiver 1. The data volume of image block 0 accounts for 3 / 4 of the total image data volume, and the data volume of image block 1 accounts for 1 / 4 of the total image data volume. The round-robin packet scheduling is performed in the order of image block 0 -> image block 1 -> image block 0 -> image block 1…

[0068] Image block 0, 0th data packet: the current image has sent data amount of 0, and the image block 0 has sent data amount of 0; according to formula (I), the sending time t0 of the 0th packet of the image block 0 in the frame period is 0 ms, at this time, formula (II) is satisfied, that is, the 0th data packet of the image block 0 is sent at the time t0.

[0069] Image block 1, 0th data packet: the current image has sent data amount of 480, and the block 1 has sent data amount of 0; the sending time t1 of the 0th packet of the block 1 in the frame period is (480 / (1920*4))*16.6 ms, that is, (1 / 16)*16.6 ms, at this time, “0 / 480<=1 / 4” satisfies formula (II), that is, the 0th data packet of the image block 1 is sent at the time t1.

[0070] Image block 0, 1st data packet: the current image has sent data amount of 480*2, and the image block 0 has sent data amount of 480; the sending time t2 of the 1st data packet of the image block 0 in the frame period is ((480*2) / (1920*4))*16.6 ms, that is, (2 / 16)*16.6 ms; at this time, “480 / (480*2)<=3 / 4” satisfies formula (II), that is, the 1st data packet of the image block 0 is sent at the time t2.

[0071] Image block 1, 1st data packet: the current image has sent data amount of 480*3, and the image block 1 has sent data amount of 480; the sending time t3 of the 1st data packet of the image block 1 in the frame period is ((480*3) / (1920*4))*16.6 ms, that is, (3 / 16)*16.6 ms; at this time, “480 / (480*3)>1 / 4” does not satisfy formula (II), that is, the 1st data packet of the image block 1 cannot be sent at the time t2. ...

[0073] By analogy, the sending time of each data packet in the frame period is calculated.

[0074] In actual application, the sending time of each data packet in the frame period can be calculated by the coarse smoothing mechanism in example 1 and the fine smoothing mechanism in example 2, and when the frame period counts to the time, the control read memory module reads the corresponding data packet from the DDR memory, and finally realizes the smooth sending of the image segmentation of the video frame.

[0075] The optional embodiment provides a new processing mode for the smooth transmission of the segmentation of the video frame, which is suitable for the case that the image is transmitted to different receiving ends by blocks at the sending end, and the video data to each receiving end is evenly scattered in the frame period. The optional embodiment solves the problem of not utilizing the display end processing in the prior art when the video frame is divided into multiple image blocks and simultaneously sent to multiple display ends, so as to disperse multiple data blocks in a frame period for transmission, realize the smooth transmission of the video frame, and facilitate the processing of the display end.

[0076] In the embodiment, an electronic device is provided, including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the method in the above embodiment.

[0077] The above program can be run in the processor, or can also be stored in the memory (or called computer readable medium), the computer readable medium includes permanent and non-permanent, removable and non-removable media, and can be realized by any method or technology to store information. Information can be computer readable instructions, data structure, program module or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tape, magnetic tape disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition in this paper, the computer readable medium does not include transitory computer readable medium, such as modulated data signal and carrier wave.

[0078] These computer programs can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flow Figure 1 One flow or multiple flows and / or the functions specified in the block Figure 1 One block or multiple blocks, and different steps can be realized by different modules.

[0079] The embodiment provides the device. The device is a video frame sending processing device, comprising: a splitting module, configured to split a video frame into a plurality of image blocks, wherein the plurality of image blocks are respectively sent to different display terminals; a dividing module, configured to divide each image block into at least one data packet, wherein the number of data packets obtained by dividing different image blocks is the same or different; and a sending module, configured to alternately send data packets of each image block in a frame period of the video frame until all data packets of the plurality of image blocks are sent out in the frame period.

[0080] The system or device is used to realize the functions of the method in the above-mentioned embodiments. Each module in the system or device corresponds to each step in the method, and has been described in the method and will not be described here.

[0081] Optionally, the sending module is configured to obtain a sending time point of each data packet according to the amount of data sent out before sending of each data packet and a sending rate, wherein the same sending rate is used to send each data packet; and the sending module is configured to start sending the data packet at the sending time point of the data packet using the average rate.

[0082] Optionally, the sending module is configured to divide the total amount of data of the video frame by the frame period to obtain the sending rate; determine a time length for sending data packets that have been sent out according to the amount of data of the data packets that have been sent out before the data packet and the sending rate; and add the time length to a starting time point of the frame period to obtain a sending time point of the data packet, wherein the first data packet is sent at the starting time point of the frame period.

[0083] Optionally, the sending module is configured to alternately send data packets of each image block according to rounds, wherein in each round, the data packets that can be sent out of each image block in the round are determined in sequence from the first image block to the Nth image block, and the data packets that can be sent out are sent out, and after the data packet sending in the round is completed, data packet sending in the next round is entered; the plurality of image blocks comprise the first image block to the Nth image block, and N is the number of the plurality of image blocks.

[0084] Optionally, the sending module is configured to determine that in each round, at most one data packet can be sent out from each image block in the first image block to the Nth image block; and in the case that the data packets of one image block have all been sent out, the sending of the data packets of the next image block is jumped to.

[0085] Optionally, the sending module is configured to determine that only one data packet can be sent in a round for each image block, or determine whether the image block meets a predetermined condition in the round, if the predetermined condition is met, only one data packet of the image block can be sent in the round, if the predetermined condition is not met, no data packet of the image block is sent in the round.

[0086] Optionally, the predetermined condition is: (an amount of data already sent in the current image block / an amount of data already sent in the video frame) <= (an amount of data of the current image block / an amount of data of the video frame).

[0087] Optionally, the sending module is configured to determine an amount of data packets sent in each round according to a total amount of data packets in the plurality of image blocks and a number of the plurality of image blocks, wherein, if a quotient of the total amount of data packets and the number of the plurality of image blocks is an integer, the amount of data packets sent in each round is the quotient, if the quotient is a decimal, the amount of data packets sent in each round is an upward rounding or a downward rounding of the quotient, and determine an amount of data packets of each image block that can be sent in the round according to a proportion of an amount of data packets of each image block in the total amount of data packets of the plurality of image blocks.

[0088] The above optional embodiments solve the problem that the prior art does not utilize the processing of the display terminal when a video frame is divided into a plurality of image blocks and sent to a plurality of display terminals at the same time, thereby dispersing the plurality of data blocks into a frame period for sending, and realizing smooth transmission of the video frame, which is beneficial to the processing of the display terminal.

[0089] The above only describes the embodiments of the present application and is not used to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A video frame transmission processing method, characterized by, include: The video frame is divided into multiple image blocks, and the multiple image blocks are sent to different display terminals respectively; Each image patch is split into at least one data packet, wherein the number of data packets obtained after splitting different image patches is the same or different; Within the frame period of the video frame, data packets for each image block are sent alternately until all data packets for the plurality of image blocks have been sent out within the frame period; The alternating transmission of data packets for each image block includes: obtaining the transmission time corresponding to each data packet based on the amount of data already transmitted and the transmission rate before each data packet is transmitted, wherein each data packet is transmitted using the same transmission rate or different transmission rates, and the different transmission rates are determined based on the data packet size of each image block; starting to transmit the data packet at the transmission time corresponding to each data packet using the transmission rate; obtaining the transmission time corresponding to each data packet based on the amount of data already transmitted and the transmission rate before each data packet is transmitted includes: dividing the total data volume of the video frame by the frame period to obtain the transmission rate; determining the duration used to transmit the already transmitted data packets based on the data volume of data packets already transmitted before this data packet and the transmission rate; adding the duration to the start time of the frame period to obtain the transmission time of the data packet, wherein the first data packet is transmitted at the start time of the frame period; The alternating transmission of data packets for each image block includes: alternatingly transmitting data packets for each image block in rounds, wherein in each round, the data packets that can be transmitted for each image block are determined sequentially from the first image block to the Nth image block, and the transmittable data packets are transmitted. After the data packets in this round are transmitted, the transmission of data packets in the next round begins; the plurality of image blocks include the first image block to the Nth image block, N... The number of the plurality of image blocks; wherein, if all data packets of an image block have been sent, the process jumps to the next image block to send data packets; if it is determined that at most one data packet can be sent from each image block in this round, for each image block, it is determined that only one data packet can be sent in this round; or, it is determined whether the image block meets a predetermined condition in this round. If the predetermined condition is met, only one data packet can be sent in this round; if the predetermined condition is not met, no data packet for the image block is sent in this round; the predetermined condition is: (the amount of data already sent in the current image block / the amount of data already sent in the video frame) <= (the amount of data in the current image block / the amount of data in the video frame).

2. The method of claim 1, wherein, In each round, the data packets that each image block can send in this round are determined sequentially from the first image block to the Nth image block. If it is determined that at least one data packet is sent from each image block in this round, the amount of data packets sent in each round is determined based on the total number of data packets in the plurality of image blocks and the number of the plurality of image blocks. Wherein, if the quotient of the total number of data packets and the number of the plurality of image blocks is an integer, the amount of data packets sent in each round is the quotient; if the quotient is a decimal, the amount of data packets sent in each round is the floor up or floor down of the quotient. The number of data packets that each image block can send in this round is determined based on the proportion of the number of data packets of each image block in the total number of data packets of the multiple image blocks.

3. The method of claim 1, wherein, Each of the data packets is the same size.

4. A video frame transmission and processing apparatus, characterized in that, include: A segmentation module is used to divide a video frame into multiple image blocks, wherein the multiple image blocks are sent to different display terminals respectively; The segmentation module is used to split each image block into at least one data packet, wherein the number of data packets obtained after splitting different image blocks may be the same or different. The sending module is used to alternately send data packets of each image block within the frame period of the video frame until all data packets of the multiple image blocks have been sent out within the frame period; The sending module is configured to obtain the sending time corresponding to each data packet based on the amount of data already sent and the sending rate before each data packet is sent, wherein each data packet is sent using the same sending rate or different sending rates, and the different sending rates are determined based on the data packet size of each image block; the data packet is started to be sent using the sending rate at the sending time corresponding to each data packet; obtaining the sending time corresponding to each data packet based on the amount of data already sent and the sending rate before each data packet is sent includes: dividing the total data of the video frame by the frame period to obtain the sending rate; determining the duration used to send the data packets that have already been sent based on the amount of data of the data packets that have been sent before this data packet and the sending rate; and adding the duration to the start time of the frame period to obtain the sending time of the data packet, wherein the first data packet is sent at the start time of the frame period; The sending module is used to alternately send data packets of each image block in rounds. In each round, following the order from the first image block to the Nth image block, the data packets that can be sent for each image block in that round are determined sequentially, and the sendable data packets are sent. After the data packets in that round are sent, the next round of data packet sending begins. The plurality of image blocks includes the first image block to the Nth image block, where N is the number of the plurality of image blocks. If all data packets for an image block have been sent, the module jumps to the next image block for data packet sending. If it is determined that at most one data packet can be sent from each image block in that round, then for each image block, only one data packet can be sent in that round. Alternatively, it is determined whether the image block meets a predetermined condition in that round. If the predetermined condition is met, only one data packet can be sent in that round; if the predetermined condition is not met, no data packet for that image block is sent in that round. The predetermined condition is: (the amount of data already sent in the current image block / the amount of data already sent in the video frame) <= (the amount of data in the current image block / the amount of data in the video frame).

5. A video frame transmission and processing system, characterized in that, It includes a sending end and multiple display ends, wherein the sending end is used to perform the method steps of any one of claims 1 to 3.

6. An electronic device comprising a memory and a processor; wherein, The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the steps of the method according to any one of claims 1 to 3.

7. A readable storage medium having computer instructions stored thereon, wherein, When executed by a processor, the computer instructions implement the steps of the method described in any one of claims 1 to 3.

Images