Image Transmission Method, Device, Storage Medium, and Electronic Device

Abstract

The present invention discloses a method and apparatus for picture transmission, a storage medium, and an electronic device. Among them, the above method includes: during the process of transmitting a video stream to a receiving party through a transmission channel, when there is a target picture to be uploaded to the receiving party, determining whether the transmission channel is transmitting a video frame at the current moment; when the transmission channel is transmitting a first video frame, determining a first transmission idle period after the transmission of the first video frame by the transmission channel, and transmitting the target picture to the receiving party within the first transmission idle period; when the transmission channel is not transmitting a video frame, determining a second transmission idle period after the transmission of a second video frame to be uploaded in the video stream by the transmission channel, and transmitting the target picture to the receiving party within the second transmission idle period.

Description

Technical Field

[0001] The present invention relates to the field of communications, and in particular, to a picture transmission method, an apparatus, a storage medium, and an electronic device. Background Art

[0002] In the video surveillance industry, there are usually both audio and video services and picture capture services. The audio and video stream displays real-time audio and video information, and picture capture and upload are usually used for backup of the pictures of some important events. The video stream is composed of a series of video frames, which are periodically encoded and transmitted at a frame rate (for example, if the frame rate is 25, it means that 25 frames are encoded and transmitted at equal time intervals within one second, that is, a video frame is encoded and transmitted every 40 ms). However, when the video frames are being transmitted, if a picture is captured during an important event, the transmission traffic of the captured picture will be superimposed on the transmission traffic of the video frames to form a peak. In the case of insufficient bandwidth, it will affect the transmission of the video frames, resulting in delays in the video frames. The role of the picture frames is mainly for event backup, and compared with video data, they belong to low-priority data, and a certain delay is allowed during the process of uploading them to the network.

[0003] During the process of transmitting the video stream, if there is a picture to be transmitted, a common solution is to establish a priority queue. The high-priority video frame data is placed in the high-priority queue, and the low-priority picture capture data is placed in the low-priority queue. Each queue has different access parameters during transmission, including congestion window, packet interval, continuous transmission duration, etc., so that the data in the high-priority queue has a higher probability of being transmitted, and the data in the low-priority queue has a lower transmission probability. However, although different data can obtain different transmission weights through the high-priority and low-priority queues, the data in the low-priority queue still occupies a certain amount of transmission bandwidth. In addition, even if there is data in the high-priority queue and the low-priority queue does not send data, if the low-priority data has started to send data before the high-priority queue sends data, this part of the data being sent will also occupy a certain amount of bandwidth and affect the transmission of the data in the high-priority queue.

[0004] In view of the related art, in the process of transmitting the video stream, in the case where a picture to be transmitted suddenly appears, there is no effective solution to the problem that the determined transmission period of the picture is unreasonable.

[0005] Therefore, it is necessary to improve the related art to overcome the defects in the related art. Summary of the Invention

[0006] Embodiments of the present invention provide a picture transmission method, an apparatus, a storage medium, and an electronic device, so as to at least solve the problem that the determined transmission period of the picture is unreasonable in the process of transmitting the video stream when a picture to be transmitted suddenly appears.

[0007] According to one aspect of an embodiment of the present invention, a picture transmission method is provided, including: during the process of transmitting a video stream to a receiving party through a transmission channel, when there is a target picture to be uploaded to the receiving party, determining whether the transmission channel is transmitting a video frame at the current moment; when the transmission channel is transmitting a first video frame, determining a first transmission idle period after the first video frame is transmitted by the transmission channel, and transmitting the target picture to the receiving party within the first transmission idle period; when the transmission channel is not transmitting a video frame, determining a second transmission idle period after a second video frame to be uploaded in the video stream is transmitted by the transmission channel, and transmitting the target picture to the receiving party within the second transmission idle period.

[0008] In an exemplary embodiment, when the target transmission idle period is the first transmission idle period or the second transmission idle period, determining the target transmission idle period includes: determining the transmission duration of a target video frame and the transmission time interval between two adjacent video frames in the video stream, where the target video frame is the first video frame or the second video frame corresponding to the target transmission idle period; determining the target transmission idle period according to the transmission duration of the target video frame and the transmission time interval.

[0009] In an exemplary embodiment, determining the target transmission idle period according to the transmission duration of the target video frame and the transmission time interval includes: obtaining a pre-set safety duration, where within the safety duration before transmitting a video frame, the transmission channel is not allowed to transmit data; subtracting the transmission duration and the safety duration from the transmission time interval to obtain the transmission duration of the target transmission idle period, where the start moment of the target transmission idle period is the end moment of the transmission of the target video frame, and the difference between the end moment of the target transmission idle period and the start moment of the target transmission idle period is the transmission duration of the target transmission idle period.

[0010] In an exemplary embodiment, when the target transmission idle period is the first transmission idle period or the second transmission idle period, transmitting the target picture to the receiving party within the target transmission idle period includes: determining whether the target picture can be transmitted within the target transmission idle period; when it is determined that the target picture can be transmitted within the target transmission idle period, directly transmitting the target picture within the target transmission idle period.

[0011] In an exemplary embodiment, after determining whether the target picture can be transmitted within the target transmission idle period, the method further includes: in the case where it is determined that the target picture cannot be transmitted within the target transmission idle period, splitting the target picture into data to obtain first data and second data; wherein, the data volume of the first data is the maximum data volume allowed to be transmitted within the target transmission idle period; transmitting the first data within the target transmission idle period, and transmitting the second data within a third transmission idle period; wherein, the third transmission idle period is the transmission idle period of the transmission channel after transmitting a third video frame, and the third video frame is a video frame to be uploaded in the video stream after transmitting a target video frame; the target video frame is a first video frame or a second video frame corresponding to the target transmission idle period.

[0012] In an exemplary embodiment, determining whether the target picture can be transmitted within the target transmission idle period includes: determining a first data volume and a transmission duration of a target video frame, and determining a transmission rate of the target video frame according to the first data volume and the transmission duration of the target video frame; wherein, the target video frame is a first video frame or a second video frame corresponding to the target transmission idle period; determining an allowed transmission data volume within the target transmission idle period according to the transmission rate; in the case where the data volume of the target picture is less than or equal to the allowed transmission data volume, determining that the target picture can be transmitted within the target transmission idle period.

[0013] In an exemplary embodiment, transmitting the second data within the third transmission idle period includes: in the case where it is determined that the second data can be transmitted within the third transmission idle period, directly transmitting the second data within the third transmission idle period; in the case where it is determined that the second data cannot be transmitted within the third transmission idle period, splitting the second data into first sub-data and second sub-data; transmitting the first sub-data within the third transmission idle period, and transmitting the second sub-data within a fourth transmission idle period, wherein, the fourth transmission idle period is the transmission idle period of the transmission channel after transmitting a fourth video frame; the fourth video frame is a video frame to be uploaded in the video stream after transmitting the third video frame.

[0014] According to another aspect of the embodiments of the present invention, there is also provided an image transmission device, including: a determination module, configured to determine whether the transmission channel is transmitting a video frame at the current moment when there is a target image to be uploaded to the receiving party during the process of transmitting a video stream through the transmission channel to the receiving party; a first transmission module, configured to determine a first transmission idle period of the first video frame when the transmission channel is transmitting a first video frame, and transmit the target image to the receiving party during the first transmission idle period; a second transmission module, configured to determine a second transmission idle period of a second video frame to be uploaded in the video stream when the transmission channel is not transmitting a video frame, and transmit the target image to the receiving party during the second transmission idle period.

[0015] According to still another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, in which a computer program is stored, and the computer program is configured to execute the above-mentioned image transmission method when running.

[0016] According to still another aspect of the embodiments of the present invention, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the above-mentioned processor executes the above-mentioned image transmission method through the computer program.

[0017] Through the present invention, if the transmission channel is currently transmitting a first video frame, the target image is transmitted during the first transmission idle period after the transmission channel finishes transmitting the first video frame, which can avoid the bandwidth for transmitting the first video frame being occupied by transmitting the target image. If the transmission channel is not transmitting a video frame, the image is not transmitted until the second transmission idle period, which can avoid the bandwidth of the second video frame to be transmitted being occupied by the untransmitted target image. By adopting the above technical solution, the transmission period of the target image is reasonably determined, ensuring that the video frame can be transmitted with low latency. It solves the problem that the determined transmission period of the image is unreasonable when a picture to be transmitted suddenly appears during the process of transmitting a video stream. Description of the Drawings

[0018] The drawings described herein are used to provide a further understanding of the present invention, and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention, and do not constitute an improper limitation to the present invention. In the drawings:

[0019] Figure 1 is a hardware structure block diagram of a computer terminal of the image transmission method according to an embodiment of the present invention;

[0020] Figure 2 is a flowchart (1) of the image transmission method according to an embodiment of the present invention;

[0021] Figure 3 It is a schematic diagram of the picture transmission method according to an embodiment of the present invention;

[0022] Figure 4 It is a flowchart (II) of the picture transmission method according to an embodiment of the present invention;

[0023] Figure 5 It is a structural block diagram of the picture transmission device according to an embodiment of the present invention. Detailed implementation manners

[0024] In order to enable those skilled in the art of the present technology to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0026] The method embodiments provided in the embodiments of the present application can be executed on a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 It is a hardware structural block diagram of the computer terminal of the picture transmission method according to an embodiment of the present invention. As Figure 1 shown, the computer terminal may include one or more ( Figure 1 only one is shown in Figure 1The structure shown is only illustrative and does not limit the structure of the above computer terminal. For example, the computer terminal may further include more or fewer components than those shown in Figure 1 or have different configurations with the same functions as those shown in Figure 1 or more functions than those shown in Figure 1 shown.

[0027] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the picture transmission method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the computer terminal through a network. Examples of the above network include but are not limited to the Internet, enterprise intranets, local area networks, mobile communication networks, and combinations thereof.

[0028] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the computer terminal. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (Radio Frequency, abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0029] Specifically, in this embodiment, a picture transmission method is provided, including but not limited to being applied to an image acquisition device (including but not limited to a camera), Figure 2 is a flowchart (one) of the picture transmission method according to the embodiment of the present invention, and the process includes the following steps:

[0030] Step S202: During the process of transmitting a video stream to a receiving party through a transmission channel, when there is a target picture to be uploaded to the receiving party, determine whether the transmission channel is transmitting a video frame at the current moment;

[0031] As an optional example, the above receiving party includes but is not limited to a server, an image receiving device, and the like.

[0032] It should be noted that the above transmission channel is a channel for the image acquisition device to transmit images to the receiving party, and the above images include but are not limited to: pictures, videos.

[0033] It should be noted that the video stream includes multiple video frames, and each video frame is periodically transmitted to the receiving party.

[0034] Step S204: When the first video frame is being transmitted on the transmission channel, determine the first transmission idle period after the first video frame is transmitted on the transmission channel, and transmit the target picture to the receiving party within the first transmission idle period;

[0035] It should be noted that the above video stream includes the first video frame.

[0036] It should be noted that transmitting the target picture to the receiving party within the first transmission idle period can avoid reducing the transmission bandwidth of the video frame due to simultaneous transmission of the picture and the video frame, resulting in a delay in the transmission of the first video frame.

[0037] It should be noted that since each video frame in the video stream is uploaded periodically, and the start transmission interval between two adjacent video frames (for example, the first video frame and the second video frame) is greater than the transmission time of the current video frame (i.e., the first video frame), after the current video frame is transmitted, there is a period of time when the transmission channel does not transmit video frames before starting to transmit the next video frame, that is, the period when the transmission channel does not transmit video frames is the transmission idle period.

[0038] Step S206: When the transmission channel is not transmitting a video frame, determine the second transmission idle period after the second video frame to be uploaded in the video stream is transmitted on the transmission channel, and transmit the target picture to the receiving party within the second transmission idle period.

[0039] In an exemplary embodiment, when the target transmission idle period is the first transmission idle period or the second transmission idle period, determining the target transmission idle period can be achieved through the following steps S11 - S12:

[0040] Step S11: Determine the transmission duration of the target video frame and the transmission time interval between two adjacent video frames in the video stream, where the target video frame is the first video frame or the second video frame corresponding to the target transmission idle period;

[0041] It should be noted that the transmission time interval between two adjacent video frames in the video stream can be obtained from the transmission period of the video stream. Exemplarily, the transmission period can be obtained based on the frame rate of the video stream. If the frame rate is F, the transmission period is T = 1000 / F milliseconds. By recording the transmission time t of the first video frame, the subsequent video frame transmission times can be deduced as t + T ms, t + 2T ms, t + 3T ms, t + 4T ms. Suppose the frame rate is 10, then the transmission time interval is 100 ms.

[0042] Step S12: Determine the target transmission idle period according to the transmission duration of the target video frame and the transmission time interval.

[0043] As an optional example, the transmission time interval can be subtracted by the transmission duration of the target video frame to obtain the duration of the target transmission idle period, and then the target transmission idle period can be obtained. Among them, the start time of the target transmission idle period is the end time of the transmission of the target video frame, and the difference between the end time of the target transmission idle period and the start time of the target transmission idle period is the transmission duration of the target video frame.

[0044] In an exemplary embodiment, in order not to affect the transmission of video frames in the next cycle, a safety duration during which the transmission channel does not transmit data can be reserved before transmitting the next video frame. Thus, the above step S12 can be implemented through the following steps S121 - S122:

[0045] Step S121: Obtain the pre-set safety duration. During the safety duration before transmitting the video frame, the transmission channel is not allowed to transmit data;

[0046] As an optional example, assume that the transmission time interval between two adjacent video frames in the video stream is 100 ms, then the safety duration can be 10 ms.

[0047] Step S122: Subtract the transmission duration and the safety duration from the transmission time interval to obtain the transmission duration of the target transmission idle period. Among them, the start time of the target transmission idle period is the end time of the transmission of the target video frame, and the difference between the end time of the target transmission idle period and the start time of the target transmission idle period is the transmission duration of the target transmission idle period.

[0048] For example, assume that the transmission duration of the target video frame data volume is 10 m, the transmission time interval of the video stream is 100 milliseconds, and the reserved safety duration is 10 milliseconds. Then the transmission duration of the target transmission idle period is 80 milliseconds.

[0049] In an exemplary embodiment, when the target transmission idle period is the first transmission idle period or the second transmission idle period, transmitting the target picture to the receiving party within the target transmission idle period can be implemented through the following steps S21 - S22:

[0050] Step S21: Determine whether the target picture can be transmitted within the target transmission idle period;

[0051] In an exemplary embodiment, the above step S21 can be implemented through the following steps S211 - S213:

[0052] Step S211: Determine the first data volume and transmission duration of the target video frame, and determine the transmission rate of the target video frame according to the first data volume and transmission duration of the target video frame; wherein, the target video frame is the first video frame or the second video frame corresponding to the target transmission idle period;

[0053] It should be noted that when the target transmission idle period is the first transmission idle period, the target video frame is the first video frame, and when the target transmission idle period is the second transmission idle period, the target video frame is the second video frame.

[0054] It should be noted that the first data volume can be divided by the transmission duration to obtain the transmission rate of the target video frame.

[0055] Step S212: Determine the allowable transmission data volume within the target transmission idle period according to the transmission rate;

[0056] It should be noted that since the transmission rate of the transmission channel is relatively stable within a period of time, the transmission rate of the target video frame can be determined as the transmission rate of the transmission channel for transmitting picture data within the target transmission idle period. Then, the transmission duration corresponding to the target transmission idle period can be multiplied by the transmission rate of the target video frame to obtain the allowable transmission data volume within the target transmission idle period.

[0057] Step S213: When the data volume of the target picture is less than or equal to the allowable transmission data volume, determine that the target picture can be transmitted within the target transmission idle period.

[0058] Step S22: When it is determined that the target picture can be transmitted within the target transmission idle period, directly transmit the target picture within the target transmission idle period.

[0059] As an optional example, transmitting the target picture to the receiving party within the target transmission idle period includes: starting to transmit the target picture at the start time of the target transmission idle period.

[0060] In an exemplary embodiment, after performing step S21, the following steps S23 - S24 are further included:

[0061] Step S23: In the case where it is determined that the target picture cannot be transmitted within the target transmission idle period, split the data of the target picture to obtain first data and second data; wherein, the data volume of the first data is the maximum data volume allowed to be transmitted within the target transmission idle period;

[0062] Step S24: Transmit the first data within the target transmission idle period, and transmit the second data within the third transmission idle period; wherein, the third transmission idle period is the transmission idle period of the transmission channel after transmitting the third video frame, and the third video frame is the video frame to be uploaded in the video stream after transmitting the target video frame; the target video frame is the first video frame or the second video frame corresponding to the target transmission idle period.

[0063] As an alternative example, assume that about 6400KB of data can be sent within the idle target transmission idle period of 80 milliseconds. Therefore, if the data volume of the target picture is less than 6400KB, the target picture can be directly sent after the target video frame is sent, and thus the transmission of the next cycle video frame can be unaffected. If the data volume of the target picture is greater than 6400KB, for example, 7000KB, the remaining 600KB of data is postponed until after the next cycle video frame (the third video frame) is sent and then sent according to the same rule.

[0064] In an exemplary embodiment, transmitting the second data within the third transmission idle period in the above step S24 can be implemented through the following steps S241 - S242:

[0065] Step S241: In the case where it is determined that the second data can be transmitted within the third transmission idle period, directly transmit the second data within the third transmission idle period;

[0066] Step S242: In the case where it is determined that the second data cannot be transmitted within the third transmission idle period, split the second data to obtain first sub - data and second sub - data; transmit the first sub - data within the third transmission idle period, and transmit the second sub - data within the fourth transmission idle period, wherein, the fourth transmission idle period is the transmission idle period of the transmission channel after transmitting the fourth video frame; the fourth video frame is the video frame to be uploaded in the video stream after transmitting the third video frame.

[0067] As an alternative example, the data volume of the first sub-data is the maximum data volume allowed to be transmitted during the third transmission idle period.

[0068] It should be noted that the method of transmitting the second sub-data during the fourth transmission idle period is similar to the method of transmitting the second data during the third transmission idle period.

[0069] Through the above steps, if the transmission channel is currently transmitting the first video frame, the target picture can be transmitted during the first transmission idle period after the transmission channel finishes transmitting the first video frame, which can avoid the target picture occupying the bandwidth for transmitting the first video frame. If the transmission channel is not transmitting a video frame, the picture transmission starts only during the second transmission idle period, which can avoid the incomplete target picture occupying the bandwidth for the to-be-transmitted second video frame. By adopting the above technical solution, the transmission period of the target picture is reasonably determined, ensuring that the video frame can be transmitted with low latency. This solves the problem of unreasonable determination of the picture transmission period when a picture to be transmitted suddenly appears during the transmission of the video stream.

[0070] Obviously, the above-described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. To better understand the above picture transmission method, the following will describe the above process in conjunction with embodiments, but it is not used to limit the technical solutions of the embodiments of the present invention. Specifically:

[0071] As an alternative example, the present application estimates the sending time of the next video frame through the periodicity of video frame sending, and adjusts the sending time of the target picture with low priority, so that the video frame data better occupies the bandwidth, achieving the purpose of transmitting video frame data with low latency.

[0072] As an alternative example, as Figure 3As shown, the encoding and transmission of video frames have periodic characteristics. Assuming the frame rate is F, the encoding and transmission period is T = 1000 / F milliseconds. By recording the transmission time t of video frame 1, the subsequent video frame transmission times can be deduced as t + T ms, t + 2T ms, t + 3T ms, t + 4T ms,.... If there is a screenshot event at time t1 that requires uploading the picture frame, if there is a video frame being sent currently, wait until the video frame transmission is completed before transmitting the picture frame (i.e., the above target picture), to avoid the transmission of low-priority picture frames occupying a certain bandwidth and increasing the delay of video frame transmission. If there is no video frame being transmitted currently, to prevent the transmission of the picture frame from affecting the transmission of the next video frame, do not send the picture frame first. Wait until the video frame of the next cycle is sent and then send the low-priority picture frame, so that the video frame can better occupy the bandwidth and avoid the transmission of low-priority picture frames during the transmission of the video frame, which increases the transmission time of the video frame. In addition, although adjusting the sending time of the picture frame reserves the maximum idle sending duration for the picture frame, because the picture frame data may be too large, the picture frame may still not be sent completely before the transmission of the next cycle's video frame. Therefore, it is necessary to refer to the duration of the currently sent video frame as a reference for the amount of data to be sent. If the amount of data that can be sent in the calculated idle cycle is less than the current picture data volume, the picture frame data needs to be cut and spread over several cycles of video frames for sending.

[0073] For example, assume the frame rate of the video stream is 10 frames per second. Then a video frame is encoded and transmitted every 100 milliseconds. Assume a video frame is sent at t = 5 seconds. Then the next video frame will be encoded and sent at 5 seconds and 100 milliseconds. If a picture frame is generated at 5 seconds and 70 milliseconds, it is cached and not sent first, so that the video frame of the next cycle can fully occupy the bandwidth for data transmission, achieving the lowest transmission time. In addition, when the video frame of the 5 seconds and 100 milliseconds cycle is being sent, count the data volume and transmission duration of the video frame. For example, the video frame data volume is 800 KB and the transmission duration is 10 ms. The transmission of the video frame of this cycle is completed at 5 seconds and 110 milliseconds. And when reserving a safety threshold of 10 milliseconds without sending data before the transmission time of the video frame of the next cycle at 5 seconds and 200 milliseconds, there is a total of 80 milliseconds of idle cycle. Referring to the sending speed of the current video frame (800 KB of data is sent in 10 ms), about 800 KB * 80 ms / 10 ms = 6400 KB of data can be sent within the 80 milliseconds of idle time. Therefore, if the picture frame is less than 6400 K, it can be sent directly after the video frame is sent, without affecting the transmission of the next cycle's video frame. If the picture frame is greater than 6400 K, for example, it is 7000 K, then the remaining 600 K of data is postponed until after the video frame of the next cycle is sent and sent according to the same rule.

[0074] For better understanding, Figure 4 is the flowchart (II) of the picture transmission method according to an embodiment of the present invention. Specifically, it can be summarized into the following steps:

[0075] 1) Obtain the encoding transmission period T = 1000 / F of the video frame according to the frame rate F;

[0076] 2) Prepare to send the low-priority picture frame data at time t1;

[0077] 3) Detect whether there is video data being sent. If so, immediately send the low-priority picture frame data after the current video frame is sent, so as to ensure that the video frame data transmission obtains the optimal bandwidth;

[0078] 4) If there is no video frame data being sent, cache the low-priority picture frame data, and immediately send the low-priority picture frame data after the next video frame is sent, to avoid the transmission of low-priority picture frame data affecting the transmission of video frame data;

[0079] 5) If it is found that there is low-priority picture frame data to be sent during the video frame transmission, then count the amount of data S sent and the sending duration t2 during the video frame transmission;

[0080] 6) According to the amount of data S and the sending duration t2 of the video frame sent before the picture frame is sent, reserve the data security threshold t3 that is not sent, and calculate the maximum amount of data size S1 that the picture frame can be sent = (S * (T - t2 - t3)) / t2;

[0081] 7) If the data of the low-priority picture frame S2 is less than or equal to the maximum amount of data S1 that can be transmitted in the idle period, then transmit all the picture frame data. If the amount of picture frame data S2 is greater than the maximum amount of data S1 that can be transmitted in the idle period, then only transmit the picture frame data amount of size S1 in this idle period, and the remaining picture frame data amount of size S2 - S1 waits for the next cycle of video frame transmission to be completed and then continues to be transmitted according to steps 5) to 7), so as to ensure that the next cycle of video frame data transmission obtains the optimal bandwidth.

[0082] In an exemplary embodiment, the present application adjusts the transmission time of the low-priority data through the characteristics of the periodic transmission of the video frame, so that there is no data being transmitted before the periodic video frame transmission, thus being able to occupy the bandwidth more optimally. And by measuring the amount of data and the duration of the video frame transmission in real time as a reference, the amount of data transmitted by the subsequent low-priority data is restricted, so as to ensure that the next cycle of video frame transmission can occupy the bandwidth more optimally, achieving the goal of low-latency transmission of the video stream.

[0083] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of the various embodiments of the present invention.

[0084] In this embodiment, a picture transmission device is further provided. The device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0085] Figure 5 is a structural block diagram of a picture transmission device according to an embodiment of the present invention. The device includes:

[0086] A determination module 502, configured to determine whether the transmission channel is currently transmitting a video frame when there is a target picture to be uploaded to the receiving party during the process of transmitting a video stream through the transmission channel to the receiving party;

[0087] A first transmission module 504, configured to determine a first transmission idle period after the transmission channel finishes transmitting a first video frame when the transmission channel is transmitting a first video frame, and transmit the target picture to the receiving party during the first transmission idle period;

[0088] A second transmission module 506, configured to determine a second transmission idle period after the transmission channel finishes transmitting a second video frame to be uploaded in the video stream when the transmission channel is not transmitting a video frame, and transmit the target picture to the receiving party during the second transmission idle period.

[0089] With the above device, if the transmission channel is currently transmitting the first video frame, the target picture can be transmitted during the first transmission idle period after the first video frame is transmitted on the transmission channel, which can avoid the bandwidth for transmitting the target picture from occupying the bandwidth for transmitting the first video frame. If the transmission channel is not transmitting a video frame, the picture is only transmitted during the second transmission idle period, which can avoid the incomplete target picture from occupying the bandwidth of the second video frame to be transmitted. By adopting the above technical solution, the transmission period of the target picture is reasonably determined, ensuring that the video frame can be transmitted with low latency. This solves the problem that the determined transmission period of the picture is unreasonable when a picture to be transmitted suddenly appears during the transmission of the video stream.

[0090] In an exemplary embodiment, the above device is further configured to determine the transmission duration of the target video frame and the transmission time interval between two adjacent video frames in the video stream, where the target video frame is the first video frame or the second video frame corresponding to the target transmission idle period; and determine the target transmission idle period according to the transmission duration of the target video frame and the transmission time interval.

[0091] In an exemplary embodiment, the above device is further configured to obtain a pre-set safety duration, where within the safety duration before transmitting the video frame, the transmission channel is not allowed to transmit data; subtract the transmission duration and the safety duration from the transmission time interval to obtain the transmission duration of the target transmission idle period, where the start time of the target transmission idle period is the end time of the transmission of the target video frame, and the difference between the end time of the target transmission idle period and the start time of the target transmission idle period is the transmission duration of the target transmission idle period.

[0092] In an exemplary embodiment, the above device is further configured to determine whether the target picture can be completely transmitted within the target transmission idle period; and directly transmit the target picture within the target transmission idle period when it is determined that the target picture can be completely transmitted within the target transmission idle period.

[0093] In an exemplary embodiment, the above-mentioned device is further configured to, after determining whether the target picture can be transmitted within the target transmission idle period, in the case where it is determined that the target picture cannot be transmitted within the target transmission idle period, split the target picture into data to obtain first data and second data; wherein, the data volume of the first data is the maximum data volume allowed to be transmitted within the target transmission idle period; transmit the first data within the target transmission idle period, and transmit the second data within a third transmission idle period; wherein, the third transmission idle period is the transmission idle period of the transmission channel after transmitting a third video frame, and the third video frame is a video frame to be uploaded in the video stream after transmitting a target video frame; the target video frame is the first video frame or the second video frame corresponding to the target transmission idle period.

[0094] In an exemplary embodiment, the above-mentioned device is further configured to determine a first data volume and a transmission duration of a target video frame, and determine a transmission rate of the target video frame according to the first data volume and the transmission duration of the target video frame; wherein, the target video frame is the first video frame or the second video frame corresponding to the target transmission idle period; determine the allowed transmission data volume within the target transmission idle period according to the transmission rate; in the case where the data volume of the target picture is less than or equal to the allowed transmission data volume, determine that the target picture can be transmitted within the target transmission idle period.

[0095] In an exemplary embodiment, the above-mentioned device is further configured to, in the case where it is determined that the second data can be transmitted within the third transmission idle period, directly transmit the second data within the third transmission idle period; in the case where it is determined that the second data cannot be transmitted within the third transmission idle period, split the second data to obtain first sub-data and second sub-data; transmit the first sub-data within the third transmission idle period, and transmit the second sub-data within a fourth transmission idle period, wherein, the fourth transmission idle period is the transmission idle period of the transmission channel after transmitting a fourth video frame; the fourth video frame is a video frame to be uploaded in the video stream after transmitting the third video frame.

[0096] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0097] Optionally, in this embodiment, the above storage medium may be configured to store a computer program for executing the following steps:

[0098] S1. During the process of transmitting a video stream to a receiver via a transmission channel, when there is a target picture to be uploaded to the receiver, determine whether the transmission channel is transmitting a video frame at the current moment;

[0099] S2. When the transmission channel is transmitting a first video frame, determine the first transmission idle period after the first video frame is transmitted, and transmit the target picture to the receiver within the first transmission idle period;

[0100] S3. When the transmission channel is not transmitting a video frame, determine the second transmission idle period after the second video frame to be uploaded in the video stream is transmitted, and transmit the target picture to the receiver within the second transmission idle period.

[0101] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), external hard drives, magnetic disks, or optical discs that can store computer programs.

[0102] Specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary embodiments, and will not be elaborated herein.

[0103] An embodiment of the present invention further provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0104] Optionally, in this embodiment, the above processor may be configured to execute the following steps through the computer program:

[0105] S1. During the process of transmitting a video stream to a receiver via a transmission channel, when there is a target picture to be uploaded to the receiver, determine whether the transmission channel is transmitting a video frame at the current moment;

[0106] S2. When the transmission channel is transmitting a first video frame, determine the first transmission idle period after the first video frame is transmitted, and transmit the target picture to the receiver within the first transmission idle period;

[0107] S3. When the transmission channel is not transmitting a video frame, determine the second transmission idle period after the second video frame to be uploaded in the video stream is transmitted, and transmit the target picture to the receiver within the second transmission idle period.

[0108] In an exemplary embodiment, the above electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0109] Specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary embodiments, and will not be elaborated herein.

[0110] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. They can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order from here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to be implemented. In this way, the present invention is not limited to any specific combination of hardware and software.

[0111] The above is only the preferred embodiment of the present invention and is not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for image transmission, characterized in that, Including: When there is a target picture to be uploaded to the receiving party during the process of transmitting a video stream through a transmission channel, determining whether the transmission channel is transmitting a video frame at the current moment; When the transmission channel is transmitting a first video frame, determining a first transmission idle period after the first video frame is transmitted by the transmission channel, and transmitting the target picture to the receiving party during the first transmission idle period; When the transmission channel is not transmitting a video frame, determining a second transmission idle period after a second video frame to be uploaded in the video stream is transmitted by the transmission channel, and transmitting the target picture to the receiving party during the second transmission idle period; Wherein, when the target transmission idle period is the first transmission idle period or the second transmission idle period, transmitting the target picture during the target transmission idle period includes: determining whether the target picture can be completely transmitted during the target transmission idle period; when it is determined that the target picture can be completely transmitted during the target transmission idle period, directly transmitting the target picture during the target transmission idle period; when it is determined that the target picture cannot be completely transmitted during the target transmission idle period, splitting the data of the target picture to obtain first data and second data; wherein, the data volume of the first data is the maximum data volume allowed to be transmitted during the target transmission idle period; transmitting the first data during the target transmission idle period, and transmitting the second data during a third transmission idle period; wherein, the third transmission idle period is the transmission idle period after a third video frame is transmitted by the transmission channel, and the third video frame is a video frame to be uploaded in the video stream after a target video frame is transmitted; the target video frame is the first video frame or the second video frame corresponding to the target transmission idle period.

2. The method according to claim 1, characterized in that, When the target transmission idle period is the first transmission idle period or the second transmission idle period, determining the target transmission idle period includes: Determining the transmission duration of a target video frame and the transmission time interval between two adjacent video frames in the video stream, wherein the target video frame is the first video frame or the second video frame corresponding to the target transmission idle period; Determining the target transmission idle period according to the transmission duration of the target video frame and the transmission time interval.

3. The method according to claim 2, wherein Determining the target transmission idle period according to the transmission duration of the target video frame and the transmission time interval includes: Obtaining a pre-set safety duration, wherein within the safety duration before transmitting a video frame, the transmission channel is not allowed to transmit data; Subtracting the transmission duration and the safety duration from the transmission time interval to obtain the transmission duration of the target transmission idle period, wherein the start moment of the target transmission idle period is the end moment of the transmission of the target video frame, and the difference between the end moment and the start moment of the target transmission idle period is the transmission duration of the target transmission idle period.

4. The method according to claim 1, characterized in that, Determine whether the target picture can be transmitted within the target transmission idle period, including: Determine the first data volume and transmission duration of the target video frame, and determine the transmission rate of the target video frame according to the first data volume and transmission duration of the target video frame; wherein, the target video frame is the first video frame or the second video frame corresponding to the target transmission idle period; Determine the allowable transmission data volume within the target transmission idle period according to the transmission rate; In the case that the data volume of the target picture is less than or equal to the allowable transmission data volume, determine that the target picture can be transmitted within the target transmission idle period.

5. The method according to claim 1, characterized in that, Transmit the second data during the third transmission idle period, including: In the case that it is determined that the second data can be transmitted within the third transmission idle period, directly transmit the second data within the third transmission idle period; In the case that it is determined that the second data cannot be transmitted within the third transmission idle period, split the second data to obtain the first sub-data and the second sub-data; transmit the first sub-data within the third transmission idle period, and transmit the second sub-data within the fourth transmission idle period, wherein the fourth transmission idle period is the transmission idle period of the transmission channel after transmitting the fourth video frame; the fourth video frame is the video frame to be uploaded in the video stream after transmitting the third video frame.

6. A picture transmission device, characterized in that, Include: A determination module, configured to determine whether the transmission channel is currently transmitting a video frame in the case that there is a target picture to be uploaded to the receiving party during the process of transmitting the video stream through the transmission channel to the receiving party; A first transmission module, configured to, in the case that the transmission channel is transmitting the first video frame, determine the first transmission idle period after transmitting the first video frame by the transmission channel, and transmit the target picture to the receiving party within the first transmission idle period; A second transmission module, configured to, in the case that the transmission channel is not transmitting a video frame, determine the second transmission idle period after transmitting the second video frame to be uploaded in the video stream by the transmission channel, and transmit the target picture to the receiving party within the second transmission idle period; Wherein, the device is further configured to determine whether the target picture can be transmitted within the target transmission idle period; in the case where it is determined that the target picture can be transmitted within the target transmission idle period, directly transmit the target picture within the target transmission idle period; in the case where it is determined that the target picture cannot be transmitted within the target transmission idle period, split the target picture into data to obtain first data and second data; wherein, the data volume of the first data is the maximum data volume allowed to be transmitted within the target transmission idle period; transmit the first data within the target transmission idle period, and transmit the second data within a third transmission idle period; wherein, the third transmission idle period is the transmission idle period of the transmission channel after transmitting a third video frame, and the third video frame is a video frame to be uploaded in the video stream after transmitting a target video frame; the target video frame is a first video frame or a second video frame corresponding to the target transmission idle period.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when running, executes the method described in any one of claims 1 to 5 above.

8. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to execute the method described in any one of claims 1 to 5 through the computer program.

Images