Video Stream Transmission Control Method and Device, Video System, Equipment, and Storage Medium

Abstract

The present application discloses a video stream transmission control method and apparatus, a video system, an electronic device, and a computer-readable storage medium. The video stream transmission control method includes: grouping a plurality of devices for transmitting video streams to obtain a plurality of device groups, where the video stream includes key frames; generating corresponding control information for the device groups, and adjusting video parameters of the devices based on the control information so that the reception time interval between two key frames received from different devices in the device group and adjacent in time sequence is greater than or equal to an interval threshold. In this way, the present application can improve the problem of key frame overlap to improve the data loss problem, and can improve the real-time performance of the picture.

Description

Technical Field

[0001] This application relates to the technical field of data transmission control, and particularly to a video stream transmission control method and apparatus, a video system, an electronic device, and a computer-readable storage medium. Background Art

[0002] When multiple devices are connected to a client, on the one hand, if the client receives key frames of video data from multiple devices at a certain time point, it is very easy to cause video loss due to exceeding the bandwidth or the receiving buffer being full; on the other hand, for clients with multiple receiving channels such as 128 channels and 256 channels, there is a situation where multiple key frames are received simultaneously at a certain time point. If the client receives multiple key frames at a certain time point, problems such as video stuttering and buffer overflow will occur, resulting in data loss. Summary of the Invention

[0003] The main technical problem to be solved by this application is how to improve the problem of key frame overlap, so as to improve the data loss problem and enhance the real-time performance of the picture.

[0004] To solve the above technical problem, this application provides a video stream transmission control method. The video stream transmission control method includes: grouping multiple devices that send video streams to obtain multiple device groups, where the video stream includes key frames; generating corresponding control information for the device groups, and adjusting the video parameters of the devices based on the control information, so that the receiving time interval between two adjacent key frames received from different devices in the device group is greater than or equal to an interval threshold.

[0005] To solve the above technical problem, this application provides a video system. The video system includes: multiple devices for collecting video data; a client communicatively connected to the devices, for grouping multiple devices that send video streams to obtain multiple device groups, where the video stream includes key frames, and for generating corresponding control information for the device groups, and adjusting the video parameters of the devices based on the control information, so that the receiving time interval between two adjacent key frames received from different devices in the device group is greater than or equal to an interval threshold.

[0006] To solve the above technical problem, this application provides a video stream transmission control apparatus. The video stream transmission control apparatus includes: a device grouping module for grouping multiple devices that send video streams to obtain multiple device groups, where the video stream includes key frames; a parameter adjustment module connected to the device grouping module, for generating corresponding control information for the device groups, and adjusting the video parameters of the devices based on the control information, so that the receiving time interval between two adjacent key frames received from different devices in the device group is greater than or equal to an interval threshold.

[0007] To solve the above technical problems, the present application provides an electronic device. The electronic device includes: a processor and a memory, where program data is stored in the memory, and the processor is configured to execute the program data to implement the above video stream transmission control method.

[0008] To solve the above technical problems, the present application provides a computer-readable storage medium, which stores program data that can implement the above video stream transmission control method when executed by a processor.

[0009] The beneficial effect of the present application is as follows: The video stream transmission control method of the present application first groups multiple devices that send video data to obtain multiple device groups, then generates corresponding control information for each device group, and adjusts the video parameters of the devices based on the control information, so that the reception time interval of key frames received from different devices in the same device group and adjacent in time sequence is greater than or equal to the interval threshold. In this way, the present application can ensure that the key frames received from different devices in each device group do not overlap, can improve problems such as video stuttering and buffer overflow, thereby improving the problem of data loss; and the present application groups multiple devices that send video data and performs video stream transmission control on each device group, which can achieve grouped control and management of device video streams and avoid the problem of poor real-time performance of the screen caused by unified management of all devices. Description of the Drawings

[0010] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0011] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0012] Figure 1 is a schematic flowchart of an embodiment of the video stream transmission control method of the present application;

[0013] Figure 2 is Figure 1 a specific schematic flowchart of step S11 in the embodiment;

[0014] Figure 3 is Figure 1 another specific schematic flowchart of step S11 in the embodiment;

[0015] Figure 4 is Figure 1 A specific process schematic diagram of step S12 in the embodiment;

[0016] Figure 5 A schematic structural diagram of the video system and its video data of the present application;

[0017] Figure 6 Another schematic structural diagram of the video system and its video data of the present application;

[0018] Figure 7 is Figure 1 Another specific process schematic diagram of step S12 in the embodiment;

[0019] Figure 8 is Figure 1 Another specific process schematic diagram of step S12 in the embodiment;

[0020] Figure 9 Another schematic structural diagram of the video system and its video data of the present application;

[0021] Figure 10 is Figure 1 Another specific process schematic diagram of step S12 in the embodiment;

[0022] Figure 11 Another schematic structural diagram of the video system and its video data of the present application;

[0023] Figure 12 A process schematic diagram of another embodiment of the video stream transmission control method of the present application;

[0024] Figure 13 A process schematic diagram of another embodiment of the video stream transmission control method of the present application;

[0025] Figure 14 A schematic structural diagram of an embodiment of the video system of the present application;

[0026] Figure 15 A schematic structural diagram of an embodiment of the video stream transmission control device of the present application;

[0027] Figure 16 A schematic block diagram of the structure of an embodiment of the electronic device of the present application;

[0028] Figure 17 A schematic structural diagram of an embodiment of the computer-readable storage medium of the present application. Detailed implementation manners

[0029] In the following description, specific details such as specific system architectures and technologies are presented for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0030] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0031] It should also be understood that the terms used in this specification of the present application are merely for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification of the present application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0032] It should be further understood that the term "and / or" used in this specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0033] As used in this specification and the appended claims, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" depending on the context.

[0034] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0035] The present application first proposes a video stream transmission control method, as Figure 1 described, Figure 1 is a schematic flowchart of an embodiment of the video stream transmission control method of the present application. The video stream transmission control method of this embodiment specifically includes the following steps:

[0036] Step S11: Group multiple devices that send video streams to obtain multiple device groups, where the video data includes key frames.

[0037] The video stream transmission control method of this embodiment is used for a video system, which includes a client and multiple devices communicatively connected to the client. The multiple devices are used to collect video data and upload the video data to the client in the form of a video stream. Among them, the client can also communicate with the devices that collect video data through network devices such as switches.

[0038] The video picture previewed in the video system is real-time, and high requirements are placed on the smoothness of the picture. This embodiment uses I-frames and P-frames for video stream transmission, which can improve the network adaptability and reduce the decoding cost. Among them, the I-frame is a key frame and belongs to intra-frame compression. It can be understood as the complete retention of the picture of this frame, and only the data of this frame is required for decoding (because it contains the complete picture); the P-frame records the difference from the previous frame, and when decoding, the previously cached picture needs to be superimposed with the difference defined in this frame to generate the final picture. A video stream can be understood as a video sequence of multiple video pictures, and the first frame of this video sequence is always an I-frame.

[0039] The device compresses the collected video data. For example, several frames of pictures are divided into a group. To prevent movement changes, the number of frames should not be too large; the client decodes the received video data to restore the video picture.

[0040] Optionally, this embodiment can implement step S11 through the method shown in Figure 2 This embodiment of the method includes steps S21 to S23.

[0041] Step S21: Obtain the bitrates of multiple devices respectively and sort them in descending order to obtain a bitrate sequence.

[0042] The device uploads the video stream to the client at a certain bitrate. The bitrate refers to the data traffic used by the video data per unit time, also called the bitstream or bitrate; at the same resolution, the larger the bitrate of the video data, the smaller the compression ratio, the higher the picture quality, and the higher the decoding ability required for the playback device.

[0043] The client can determine the bitrate of the device based on the video stream already received from the device; the client can also obtain the preset bitrate of the device, and the device uploads the video stream at the preset bitrate.

[0044] The client first sorts the bitrates of all devices that are connected, that is, upload the video stream to the client, and obtains a bitrate sequence arranged from largest to smallest.

[0045] Step S22: Divide the bitrate sequence into multiple sequentially adjacent subsequences.

[0046] The client divides the bitrate sequence into multiple adjacent subsequences in turn, and the number of each subsequence can be the same or different.

[0047] Step S23: Divide the devices with the same sequence position in the multiple subsequences into the same device group.

[0048] Each bitrate in a subsequence has a sequence position in that subsequence, and the client divides the devices with the same sequence position in the multiple subsequences into the same device group.

[0049] For example, the client first sorts the bitrates of all the connected devices, selects the top n devices with the largest bitrates, and initially divides them into n groups; then it sequentially distributes the next n devices into the n groups that have been divided until all the devices are distributed into these n groups.

[0050] In some application scenarios, when the client detects that a new device is connected, it groups multiple devices.

[0051] Optionally, this embodiment can implement step S11 through the method as Figure 3 shown. The method of this embodiment includes steps S31 to S33.

[0052] Step S31: Obtain the event priorities of multiple devices respectively and sort them in descending order to obtain an event priority sequence.

[0053] Each device is provided with an event bitstream, and the event bitstream is the transmission bitstream of the video data of the event that occurs on the device; different event bitstreams correspond to different event priorities.

[0054] The client can determine the event bitstream of the device based on the video stream already received from the device, so as to determine the event priority of the device; or the client can also obtain the event priority preset by the device, and when the device captures the video data of the event that occurs, it transmits the video stream with the event bitstream corresponding to the preset event priority.

[0055] The client first sorts the event priorities of all the connected devices, that is, all the devices that upload video streams to the client, to obtain a bitrate sequence arranged from largest to smallest in terms of event priority.

[0056] Step S32: Divide the event priority sequence into multiple adjacent subsequences in turn.

[0057] The client divides the event priority sequence into multiple adjacent subsequences in turn, and the number of each subsequence can be the same or different.

[0058] Step S33: Divide the devices with the same sequence position in the multiple subsequences into the same device group.

[0059] The event priorities in each subsequence have sequence positions in that subsequence, and the client groups devices with the same sequence positions in multiple subsequences into the same device group.

[0060] For example, the client first sorts the event priorities of all the accessed devices, selects the top n devices with the highest event priorities, and initially divides them into n groups; then it sequentially assigns the next n devices to the n groups that have been divided previously until all devices are assigned to these n groups.

[0061] In some application scenarios, after the video system has been running for a period of time, the client groups multiple devices.

[0062] Step S12: Generate corresponding control information for the device group and adjust the video parameters of the devices based on the control information so that the reception time interval between two adjacent key frames received from different devices in the device group is greater than or equal to the interval threshold.

[0063] The client performs video stream transmission control for the devices in each device group, and specifically can manage the video parameters and the like of all the devices in the device group.

[0064] The client generates control information corresponding to each device group and sends the control information to the devices in the device group to adjust the video parameters of the devices in the device group based on the control information so that the reception time interval between two adjacent key frames received by the client from different devices in the device group is greater than or equal to the interval threshold.

[0065] This embodiment can ensure that the key frames received from different devices in each device group do not overlap, can improve problems such as video stuttering and buffer overflow, thereby improving the problem of data loss; and this application groups multiple devices that send video data and performs video stream transmission control for each device group, which can achieve grouped control management of device video streams and can avoid the problem of poor real-time performance of the picture caused by unified management of all devices.

[0066] Optionally, this embodiment can implement step S12 through the method as Figure 4 shown, and the method of this embodiment includes step S41.

[0067] Step S41: Generate corresponding control information for the device group and adjust the interval between adjacent key frames in the video stream of the devices and the sending time of the key frames of the video stream based on the control information.

[0068] The control information generated by the client may include control instructions, the sending time of the key frames of the video stream uploaded by the device, and the interval between the key frames in the video stream; wherein, the sending time is the sending time of the first key frame of the video stream, and the sending times corresponding to different devices in the device group are different, so as to stagger the sending times (and receiving times) of the key frames received from different devices and avoid key frame overlap; adjusting the interval between adjacent key frames in the video stream can not only make the overall bit rate stable, but also ensure that the key frames in any adjacent time periods do not overlap. The adjacent time periods refer to the key frame cycle of the same device; the interval between adjacent key frames should be greater than the interval between the sending time of the first key frame and the sending time of the last key frame of all devices in the same time period of the device group.

[0069] For example, as Figure 5 shown, a Network Video Recorder (NVR) communicates with a device group through a router. The device group includes IP cameras (IPCs), specifically including IPC1 - IPC4. The sending times of the first key frames of each device are different, so that the first key frames received by the client from different devices in the same time period do not overlap; and the interval between two adjacent key frames of the same device is a fixed value, and this fixed value is greater than the interval between the sending time of the earliest sent first key frame and the sending time of the latest sent first key frame among IPC1 - IPC4, so that there is no overlap between the key frames in adjacent time periods received by the client.

[0070] Optionally, the method of this embodiment may further include step S42 and step S43.

[0071] Step S42: Determine whether the key frame received from the device is an event key frame.

[0072] The client can determine whether the key frame is an event key frame based on the image features of the received key frame, etc., so as to determine whether the video stream collected by the device is a video stream in which an event occurs, that is, whether an event occurs in the monitoring scenario.

[0073] Step S43: If the key frame is an event key frame, adjust the sending time of the key frame corresponding to the device with the event key frame to before the sending times of the key frames of other devices in the device group, so as to receive the event key frame preferentially.

[0074] In some embodiments, when multiple devices send event key frames, the client can also determine the event priorities of the devices that send event key frames, and determine the order of sending event key frames of the devices that send event key frames according to the high or low event priorities. In this way, the client can receive the event key frames with high event priorities first within the same time period, then receive the event key frames with lower event priorities, and finally receive the ordinary key frames. Among them, the events corresponding to the event key frames with high event priorities have a higher degree of urgency, and the events corresponding to the event key frames with low event priorities have a lower degree of urgency. Therefore, the client can detect the events with high degrees of urgency first for timely processing.

[0075] For example, as Figure 6 shown, if IPC4 and IPC5 both send event key frames (shown as hatched boxes) within the same time period, then adjust the sending times of the key frames of IPC4 and IPC5 to before the sending times of the key frames of IPC1 - IPC13 that send ordinary key frames (shown as non-hatched boxes); and the event priority of IPC4 is higher than that of IPC5, so adjust the sending time of the key frame of IPC4 to before the sending time of the key frame of IPC5.

[0076] In some embodiments, the client can also adjust the interval between each key frame to a unified value through control information to ensure the smoothness of the bit rate.

[0077] In other embodiments, the client can also determine whether the video stream collected by the device is a video stream of an event based on the bit rate of the received video stream. If an event occurs, the device will upload the video stream with an event bit stream, and the event bit stream is greater than the ordinary bit rate.

[0078] In another embodiment, step S12 can also be implemented by the method as Figure 7 shown. The method of this embodiment includes steps S71 to S76.

[0079] Step S71: Generate corresponding control information for the device group, and adjust the interval between adjacent key frames in the video stream of the device and the sending time of the key frames of the video stream based on the control information.

[0080] For the specific implementation manner, refer to the above embodiments.

[0081] Step S72: Determine whether the key frame received from the device is an event key frame.

[0082] For the specific implementation manner, refer to the above embodiments.

[0083] Step S73: If the key frame is an event key frame, adjust the transmission time of the key frame of the device corresponding to the event key frame to be before the transmission times of the key frames of other devices in the device group based on the control information, so as to receive the event key frame preferentially.

[0084] For specific implementation manners, refer to the above embodiments.

[0085] Step S74: If the key frame is an event key frame, obtain the total bit rate of the device group corresponding to the device.

[0086] Step S75: If the total bit rate is greater than the bit rate threshold, determine whether the sum of the total bit rates of other device groups and the bit rate of the device is greater than the bit rate threshold.

[0087] Step S76: If the above sum is greater than the bit rate threshold, control the device to adjust the event bitstream for transmitting the event key frame.

[0088] If the above sum is less than or equal to the bit rate threshold, adjust the device to another device group.

[0089] For example, the bandwidth of the client can be W, then the bandwidth of each device group is W / n. If there are more devices in a certain device group that generate events and the total bit rate of this device group exceeds W / n, query whether there is remaining bandwidth in other device groups. If there is remaining bandwidth, do not adjust the event bit rate of this device group, and the exceeded bit rate of this device group can borrow the remaining bandwidth of other device groups. If there is no remaining bandwidth, the event bitstreams of these devices can be adjusted through control signals according to the event priorities of the devices that generate events in this device group, so that the total bit rate of this device group does not exceed W / n. This event priority is set by the user or preset during device initialization, such as the human form event is higher than the motion detection event, and the smoke detection event is higher than the wandering detection event, etc.

[0090] In another embodiment, step S12 can also be implemented by the method as Figure 8 shown. The method of this embodiment includes steps S81 to S84.

[0091] Step S81: Generate corresponding control information for the device group, and adjust the interval between adjacent key frames in the video stream of the device and the transmission time of the key frames of the video stream based on the control information.

[0092] For specific implementation manners, refer to the above embodiments.

[0093] Step S82: Set a first transmission time and a second transmission time, where the first transmission time is earlier than the second transmission time, and the time interval between the first transmission time and the second transmission time is greater than or equal to the interval threshold.

[0094] Step S83: Adjust the transmission time of the last key frame of a device group within a preset time period to the first transmission time.

[0095] Step S84: Adjust the transmission time of the first key frame of another device group within a preset time period to the second transmission time.

[0096] For example, as Figure 9 shown, the client adjusts the first transmission time of the last key frame of the device groups ICP1 - ICP4 within a preset time period based on the control information to be earlier than the second transmission time of the first key frame of the device groups ICP11 - ICP14 within the preset time period, so that the key frames of the device groups ICP1 - ICP4 and the device groups ICP11 - ICP14 do not overlap; and the interval between the first transmission time and the second transmission time is greater than or equal to the interval threshold to ensure the smoothness of the overall bit rate.

[0097] Similar improvements can be made for the above other embodiments.

[0098] In another embodiment, step S12 can also be implemented by the method as Figure 10 shown. The method of this embodiment includes steps S101 to S106.

[0099] Step S101: Generate corresponding control information for the device group, and adjust the interval between adjacent key frames in the video stream of the device and the transmission time of the key frames of the video stream based on the control information.

[0100] For specific implementation manners, refer to the above embodiments.

[0101] Step S102: Determine whether the key frame received from the device is an event key frame.

[0102] For specific implementation manners, refer to the above embodiments.

[0103] Step S103: If the key frame is an event key frame, adjust the transmission time of the key frame corresponding to the device of the event key frame to before the transmission time of the key frames of other devices in the device group based on the control information, so as to receive the event key frame preferentially.

[0104] For specific implementation manners, refer to the above embodiments.

[0105] Step S104: Set a third transmission time and a fourth transmission time, where the third transmission time is earlier than the fourth transmission time.

[0106] Step S105: Adjust the transmission time of the last event key frame of a device group within a time period to the third transmission time.

[0107] Step S106: Adjust the transmission time of the event key frames of another device group within the time period to the fourth time.

[0108] For example, as Figure 11 shown, based on the control information, the client adjusts the third transmission time of the last event key frame (shown as the slant-filled box) of device groups ICP1 - ICP4 within the time period to be earlier than the fourth transmission time of the first event key frame of device groups ICP11 - ICP14 within the time period, so that the event key frames of device groups ICP1 - ICP4 do not overlap with those of device groups ICP11 - ICP14.

[0109] Similar improvements can be made to the above other embodiments.

[0110] For multiple device groups, within the same time period, the key frames can be completely staggered to avoid key frame overlap, as Figure 8 described in the embodiment; if key frame overlap cannot be avoided, then stagger them to avoid the overlap of event key frames of each group, as Figure 10 described in the embodiment.

[0111] The present application further proposes a video stream transmission control method for another embodiment. As Figure 12 shown, Figure 12 is a schematic flowchart of an embodiment of the video stream transmission control method of the present application. The video stream transmission control method of this embodiment specifically includes the following steps:

[0112] Step S121: Group multiple devices that send video streams to obtain multiple device groups, where the video data includes key frames.

[0113] The specific implementation can refer to the above embodiments.

[0114] Step S122: Generate corresponding control information for the device group and adjust the video parameters of the device based on the control information, so that the reception time interval between two adjacent key frames received from different devices in the device group is greater than or equal to the interval threshold.

[0115] The specific implementation can refer to the above embodiments.

[0116] Step S123: If the video stream of a new device is received, calculate the total bit rate of each device group.

[0117] During the operation of the video system, there may be new devices communicating with the client, and the client needs to assign the new devices to the multiple device groups that have been divided.

[0118] If a new device is connected, the client calculates the total bit rate of each device group respectively, that is, the sum of the bit rates of each device in the device group.

[0119] Step S124: Divide the new device into the device group with the minimum total bitrate.

[0120] The client redistributes the newly connected device into the device group with the minimum total bitrate, improving the smoothness of the total bitrate of each device group and reducing the real-time difference in the pictures of each device group.

[0121] In another embodiment, if the video stream of a new device is received, the client obtains the event priorities of the devices in each device group, and obtains the highest event priority in each device group to obtain multiple highest event priorities, and divides the new device into the device group corresponding to the lowest priority among the multiple highest event priorities. Since the higher the event priority, the greater the bitrate, thus, in this way, the smoothness of the total bitrate of each device group can be improved and the real-time difference in the pictures of each device group can be reduced.

[0122] In another embodiment, as Figure 13 shown, Figure 13 is a schematic flowchart of an embodiment of the video stream transmission control method of the present application. The video stream transmission control method of this embodiment specifically includes the following steps:

[0123] Step S131: Group multiple devices that send video streams to obtain multiple device groups, where the video data includes key frames.

[0124] The specific implementation manner can refer to the above embodiment.

[0125] Step S132: Generate corresponding control information for each device group, and adjust the video parameters of the devices based on the control information, so that the reception time interval between two adjacent key frames received from different devices in the device group is greater than or equal to the interval threshold.

[0126] The specific implementation manner can refer to the above embodiment.

[0127] Step S133: For each device group, in response to a new device joining or a new event key frame being sent, determine whether the reception time interval between two adjacent key frames received from different devices in the device group is less than the interval threshold.

[0128] During the operation of the video system, a new device may join and communicate with the client, or an event may occur in the scene monitored by the joined device, generating a new event key frame. These factors will all affect the change of the overall video stream transmission parameters of the device group, such as the change of the overall bitrate and the reduction of the interval between adjacent key frames. The client needs to determine whether the reception time interval between two adjacent key frames received from different devices in the device group is less than the interval threshold.

[0129] Step S134: If so, regroup the multiple devices.

[0130] If the reception time interval between two adjacent key frames received from different devices in the same device group is less than the interval threshold, the client needs to regroup the connected devices, that is, execute Step S131, in order to control each group to ensure that the key frames received from the same device group do not overlap and the total bit rate of the device group does not exceed the bit rate threshold.

[0131] In other embodiments, after the video system runs for a period of time, the client can perform regrouping according to the event log records during this period to avoid uneven grouping.

[0132] In other embodiments, since the network environments of each device communicating with the client are different, there are network delays and network transmission times. To accurately correct and reduce the overlap probability of the key frames received by the client, the client can separately set the transmission time t of each device, and then analyze the reception time t + t0 of the key frame actually received from this device. The client gives the transmission time of each device as t - t0 to ensure that the reception time of the key frame received by the client is t. In this way, the client can accurately control the reception time of the received key frames and avoid key frame overlap.

[0133] The execution subject of the video stream transmission control method in the above embodiments of the present application is the client, to achieve the grouping control of the video stream transmission of the devices connected to the client.

[0134] The present application further proposes a video system, as Figure 14 shown, Figure 14 is a schematic structural diagram of an embodiment of the video system of the present application. The video system of this embodiment includes: multiple devices 141 and a client 142. Among them, the device 141 is used to collect video data; the client 142 is communicatively connected to the device 141 and is used to group the multiple devices 141 that send video streams to obtain multiple device groups. Among them, the video stream includes key frames, and is used to generate corresponding control information for the device group and adjust the video parameters of the device based on the control information, so that the reception time interval between two adjacent key frames received from different devices 141 in the device group is greater than or equal to the interval threshold.

[0135] The client 142 in this embodiment can be an NVR, and the device can be an IPC. The client 142 divides the multiple devices 141 into two device groups IPC1-IPCm and ICP11-ICP1m.

[0136] Optionally, the video system of this embodiment may further include a network device, such as a switch 143. Each device group communicates with the client 142 through the switch 143.

[0137] The video system of this embodiment is also used to implement the above video transmission control method. For the video transmission control management method of the device group by the client 142, reference can be made to the above embodiment and will not be elaborated here.

[0138] In other embodiments, other video surveillance devices, etc. can be used to replace the NVR, such as a mobile terminal equipped with video software, etc. Also, a depth camera or other image or video acquisition device can be used to replace the IPC.

[0139] In an application scenario, the mobile terminal can be a mobile phone equipped with a video APP. The mobile phone can communicate with the IPC through a cloud server and perform video stream transmission control on the IPC to achieve grouped control management of the IPC video stream.

[0140] In other embodiments, the number of device groups in the video system and the number of devices in each device group are not limited.

[0141] The present application further proposes a video stream transmission control device, as Figure 15 shown, Figure 15 is a schematic structural diagram of an embodiment of the video stream transmission control device of the present application. The video stream transmission control device of this embodiment includes: a device grouping module 151 and a parameter adjustment module 152. Among them, the device grouping module 151 is used to group multiple devices that send video streams to obtain multiple device groups, where the video stream includes key frames; the parameter adjustment module 152 is connected to the device grouping module 151 and is used to generate corresponding control information for the device group and adjust the video parameters of the device based on the control information so that the reception time interval between two adjacent key frames received from different devices in the device group is greater than or equal to the interval threshold.

[0142] The video stream transmission control device of this embodiment is also used to implement the above video transmission control method.

[0143] The video stream transmission control device of this embodiment can implement the above client or cloud server.

[0144] The present application further proposes an electronic device, as Figure 16 shown, Figure 16 is a schematic structural diagram of an embodiment of the electronic device of the present application. The electronic device 100 of this embodiment includes a processor 101, a memory 102 coupled to the processor 101, an input / output device 103, and a bus 104.

[0145] The processor 101, the memory 102, and the input / output device 103 are respectively connected to the bus 104. The memory 102 stores program data, and the processor 101 is used to execute the program data to implement the above video transmission control method.

[0146] In this embodiment, the processor 101 may also be referred to as a CPU (Central Processing Unit). The processor 101 may be an integrated circuit chip with signal processing capabilities. The processor 101 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor 101 may also be any conventional processor, etc.

[0147] The present application further provides a computer-readable storage medium, such as Figure 17 shown Figure 17 is a schematic structural diagram of an embodiment of the computer-readable storage medium of the present application. The computer-readable storage medium 131 stores program data 132 thereon, and when the program data 132 is executed by a processor (not shown in the figure), the above video transmission control method is implemented.

[0148] The computer-readable storage medium 131 in this embodiment may be, but is not limited to, a USB flash drive, an SD card, a PD optical drive, a mobile hard disk, a large-capacity floppy drive, a flash memory, a multimedia memory card, a server, etc.

[0149] The video stream transmission control method of the present application first groups multiple devices that send video data to obtain multiple device groups, then generates corresponding control information for each device group, and adjusts the video parameters of the devices based on the control information, so that the reception time interval of key frames received from different devices in the device group and adjacent in time sequence is greater than or equal to the interval threshold. In this way, the present application can ensure that the key frames received from different devices in each device group do not overlap, can improve problems such as video stuttering and buffer overflow, thereby improving the problem of data loss; and the present application groups multiple devices that send video data and performs video stream transmission control on each device group, which can achieve grouped control management of the device video stream and can avoid the problem of poor real-time performance of the picture caused by unified management of all devices.

[0150] The present application manages the key frames of the devices, making the bit rate more stable. For the client, data processing is also more reasonable, and data loss can be avoided; the client of the present application adjusts and manages the video parameters of each device, which can ensure the video quality and real-time performance of the key pictures of devices with certain high-priority events.

[0151] In addition, when the above functions are implemented in the form of software functions and sold or used as independent products, they can be stored in a storage medium readable by a mobile terminal. That is, the present application also provides a storage device storing program data, and the program data can be executed to implement the methods of the above embodiments. The storage device can be a USB flash drive, an optical disc, a server, etc. That is to say, the present application can be embodied in the form of a software product, which includes several instructions for causing an intelligent terminal to execute all or part of the steps of the methods described in each embodiment.

[0152] In the description of the present application, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0153] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0154] Any process or method description in the flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or more executable instructions for implementing a specific logical function or process. The scope of the preferred embodiments of the present application includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in the reverse order according to the involved functions, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0155] The logic and / or steps represented in the flowchart or otherwise described herein can, for example, be considered as a definitional sequence of executable instructions for implementing a logical function, which can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device (which can be a personal computer, a server, a grid device, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device). For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, as the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing it as appropriate, and then storing it in a computer memory.

[0156] The above are only embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall equally be included in the patent protection scope of the present application.

Claims

1. A video stream transmission control method, characterized in that Including: Grouping multiple devices that send video streams to obtain multiple device groups, where the video stream includes key frames; Generating corresponding control information for the device group and adjusting the video parameters of the device based on the control information so that the reception time interval between two adjacent key frames received from different devices in the device group is greater than or equal to an interval threshold; Wherein, the video stream transmission control method further includes: For each device group, in response to a new device joining or sending a new event key frame, determining whether the reception time interval between two adjacent key frames received from different devices in the device group is less than the interval threshold; If so, re-grouping the multiple devices.

2. The video stream transmission control method according to claim 1, characterized in that The adjusting the video parameters of the device based on the control information includes: Adjusting the interval between adjacent key frames in the video stream of the device and the sending time of the key frames of the video stream based on the control information.

3. The video stream transmission control method according to claim 2, wherein The adjusting the video parameters of the device based on the control information further includes: Determining whether the key frame received from the device is an event key frame; If the key frame is an event key frame, adjusting the sending time of the key frame of the device corresponding to the event key frame to before the sending time of the key frames of other devices in the device group to preferentially receive the event key frame.

4. The video stream transmission control method according to claim 1, characterized in that The generating corresponding control information for the device group and adjusting the video parameters of the device based on the control information further includes: Setting a first sending time and a second sending time, where the first sending time is earlier than the second sending time, and the time interval between the first sending time and the second sending time is greater than or equal to the interval threshold; Adjusting the sending time of the last key frame of a device group within a preset time period to the first sending time; Adjusting the sending time of the first key frame of another device group within the preset time period to the second sending time.

5. The video stream transmission control method according to claim 3, wherein The generating corresponding control information for the device group and adjusting the video parameters of the device based on the control information further includes: Setting a third sending time and a fourth sending time, where the third sending time is earlier than the fourth sending time; Adjusting the sending time of the last event key frame of a device group within a time period to the third sending time; Adjusting the sending time of the event key frames of another device group within the time period to the fourth sending time.

6. The video stream transmission control method according to claim 1, characterized in that The grouping multiple devices that send video streams to obtain multiple device groups includes: Respectively obtaining the bit rates of the multiple devices and arranging them in descending order to obtain a bit rate sequence; Dividing the bit rate sequence into multiple adjacent sub-sequences in sequence; Assigning the devices with the same sequence position in the multiple sub-sequences to the same device group.

7. The video stream transmission control method according to claim 1, wherein The grouping multiple devices that send video streams to obtain multiple device groups includes: Respectively obtaining the event priorities of the multiple devices and arranging them in descending order to obtain an event priority sequence; Divide the event priority sequence into multiple adjacent subsequences in sequence; Group the devices with the same sequence position in the multiple subsequences into the same device group.

8. The video stream transmission control method according to claim 1, wherein Further include: If a video stream of a new device is received, calculate the total bitrate of each device group; Divide the new device into the device group with the minimum total bitrate.

9. The video stream transmission control method according to claim 1, wherein The adjusting the video parameters of the device based on the control information further includes: Determine whether the key frame received from the device is an event key frame; If the key frame is an event key frame, obtain the total bitrate of the device group corresponding to the device; If the total bitrate is greater than the bitrate threshold, determine whether the sum of the total bitrates of the other device groups and the bitrate of the device is greater than the bitrate threshold; If the sum is greater than the bitrate threshold, adjust the event bitstream for the device to send the event key frame.

10. A video system, characterized in that, Include: Multiple devices for collecting video data; A client communicatively connected to the devices for grouping multiple devices that send video streams to obtain multiple device groups, where the video stream includes key frames, and for generating corresponding control information for the device groups and adjusting the video parameters of the devices based on the control information so that the reception time interval between two adjacent key frames received from different devices in the device group is greater than or equal to an interval threshold; Wherein, for each device group, in response to a new device joining or a new event key frame being sent, determine whether the reception time interval between two adjacent key frames received from different devices in the device group is less than the interval threshold; If so, regroup the multiple devices.

11. A video transmission control device, characterized in that, Include: A device grouping module for grouping multiple devices that send video streams to obtain multiple device groups, where the video stream includes key frames; A parameter adjustment module connected to the device grouping module for generating corresponding control information for the device groups and adjusting the video parameters of the devices based on the control information so that the reception time interval between two adjacent key frames received from different devices in the device group is greater than or equal to an interval threshold; Wherein, for each device group, in response to a new device joining or a new event key frame being sent, determine whether the reception time interval between two adjacent key frames received from different devices in the device group is less than the interval threshold; If so, regroup the multiple devices.

12. An electronic device, characterized in that, Include: A processor and a memory, where program data is stored in the memory, and the processor is configured to execute the program data to implement the video stream transmission control method according to any one of claims 1-9.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program data, and when the program data is executed by a processor, it can implement the video stream transmission control method according to any one of claims 1-9.

Images