Video stream processing method and electronic equipment

Abstract

The invention provides a video stream processing method and electronic equipment, and the method comprises the steps: receiving each video stream which comprises a key frame sequence; according to each key frame sequence and a pre-configured priority configuration parameter, evaluating a video stream collision degree, the priority configuration parameter being used for indicating whether video stream processing is performed according to the priority of the video stream in each video stream processing stage, and indicating the processing priority of each video stream in each video stream processing stage; determining whether to perform collision detection processing and collision elimination processing according to the video stream collision degree; and if yes, determining a target video stream according to the priority configuration parameter, and performing collision elimination processing on the target video stream according to the video stream collision degree and the priority configuration parameter, so as to improve the accuracy of evaluating the video stream collision degree and the flexibility of video stream processing.

Description

Technical Field

[0001] This application relates to the field of data transmission technologies, and in particular, to a video stream processing method and an electronic device. Background Art

[0002] In the field of data transmission, a video stream consists of multiple video frames, which mainly include I-frames (key frames) and P-frames (predicted frames). I-frames are used to describe the image background and moving objects, and a complete image can be reconstructed using only the I-frame data during decoding. Since I-frames contain more parameters than P-frames, more bandwidth is required for network transmission.

[0003] When watching multiple video streams over a network, such as in a video conference or multi-channel surveillance videos, multiple video streams are transmitted simultaneously, and it is possible for multiple I-frames to arrive almost at the same time, that is, an I-frame collision occurs. I-frame collisions can cause problems such as network congestion and packet loss, seriously affecting the video quality.

[0004] Currently, it is impossible to accurately determine the severity of video stream collisions. Therefore, the prior art cannot accurately and efficiently evaluate the transmission quality of video streams, and there are certain limitations. Summary of the Invention

[0005] The purpose of this application is to provide a video stream processing method and an electronic device for the deficiencies in the above prior art, so as to solve the problem of the actual need that the prior art cannot accurately and efficiently evaluate the transmission quality of video streams.

[0006] To achieve the above purpose, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, an embodiment of this application provides a video stream processing method, and the method includes:

[0008] Receiving each video stream, each of the video streams respectively includes a key frame sequence;

[0009] Evaluating the degree of video stream collision according to each key frame sequence and pre-configured priority configuration parameters, where the priority configuration parameters are used to indicate whether video streams are processed according to the priorities of the video streams during each video stream processing stage, and the processing priorities of each video stream during each video stream processing stage;

[0010] Determining whether to perform collision detection processing and collision elimination processing according to the degree of video stream collision;

[0011] If so, determining a target video stream according to the priority configuration parameters, and performing collision elimination processing on the target video stream according to the degree of video stream collision and the priority configuration parameters.

[0012] In a second aspect, an embodiment of the present application provides a video stream processing device, which includes:

[0013] A receiving module, configured to receive each video stream, where each of the video streams includes a key frame sequence respectively;

[0014] An evaluation module, configured to evaluate the video stream collision degree according to each of the key frame sequences and pre-configured priority configuration parameters, where the priority configuration parameters are used to indicate whether to process video streams according to the priorities of the video streams in each video stream processing stage, and the processing priorities of each video stream in each video stream processing stage;

[0015] A determination module, configured to determine whether to perform collision detection processing and collision elimination processing according to the video stream collision degree;

[0016] A processing module, configured to, if so, determine a target video stream according to the priority configuration parameters, and perform collision elimination processing on the target video stream according to the video stream collision degree and the priority configuration parameters.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of the video stream processing method described in the first aspect above.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it performs the steps of the video stream processing method described in the first aspect above.

[0019] The beneficial effects of the present application are:

[0020] The present application provides a video stream processing method and an electronic device, which receive multiple video streams in a multi-video stream concurrent scenario, and evaluate the video stream collision degree according to the key frame sequences included in each video stream and the pre-configured priority configuration parameters. Among them, the priority configuration parameter is a switch for whether to process according to the priority in each video stream processing stage, to indicate whether to process the video streams according to the priorities of the video streams in each video stream processing stage, and the processing priorities of each video stream in each video stream processing stage. According to the evaluated video stream collision degree, it is determined whether to perform collision detection processing and collision elimination processing. If so, in the video stream collision detection stage, the target video stream is determined according to the priority configuration parameter, and in the video stream collision elimination stage, the target video stream is subjected to collision elimination processing according to the video stream collision degree and the priority configuration parameter to obtain the processed target video stream. By pre-configuring the priority configuration parameters of each video stream, the video streams participating in this stage can be flexibly selected in each video stream processing stage, greatly improving the flexibility of video stream processing. In each video stream processing stage, the video streams are processed according to the processing priorities of each video stream to protect more critical video streams and prevent their transmission quality from being affected. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and thus should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0022] Figure 1 It is a schematic flowchart of the video stream processing method provided by the embodiment of the present application;

[0023] Figure 2 It is a schematic architecture diagram of each video stream processing stage provided by the embodiment of the present application;

[0024] Figure 3 It is a schematic flowchart of evaluating the video stream collision degree of the video stream processing method provided by the embodiment of the present application;

[0025] Figure 4 It is a schematic flowchart of determining the video streams to be evaluated in the video stream processing method provided by the embodiment of the present application;

[0026] Figure 5 It is another schematic flowchart of evaluating the video stream collision degree of the video stream processing method provided by the embodiment of the present application;

[0027] Figure 6 It is yet another schematic flowchart of evaluating the video stream collision degree of the video stream processing method provided by the embodiment of the present application;

[0028] Figure 7 It is a flowchart showing the process of determining whether to perform collision detection processing and collision elimination processing for the video stream processing method provided by the embodiments of the present application;

[0029] Figure 8 It is a flowchart showing the process of determining the target video stream for the video stream processing method provided by the embodiments of the present application;

[0030] Figure 9 It is a flowchart showing the process of performing collision elimination processing on the target video stream for the video stream processing method provided by the embodiments of the present application;

[0031] Figure 10 It is another flowchart showing the process of performing collision elimination processing on the target video stream for the video stream processing method provided by the embodiments of the present application;

[0032] Figure 11 It is a module structure diagram of the video stream processing device provided by the embodiments of the present application;

[0033] Figure 12 It is a schematic structural diagram of the electronic device provided by the embodiments of the present application. Detailed implementation manners

[0034] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the accompanying drawings in the present application are only for the purposes of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to actual scale. The flowcharts used in the present application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without logical context relationships may be reversed or implemented simultaneously. In addition, those skilled in the art may add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of the present application.

[0035] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application to be protected, but only represents the selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts belong to the scope of protection of the present application.

[0036] It should be noted that the term "including" will be used in the embodiments of the present application to indicate the existence of the features stated thereafter, but does not exclude the addition of other features.

[0037] Currently, it is impossible to accurately and intuitively evaluate the severity of video stream collisions, nor can it solve the network congestion problem caused by I-frame collisions, seriously affecting the transmission quality of video streams.

[0038] Based on the above problems, the embodiments of the present application propose a video stream processing method, which pre-configures the priority configuration parameters of each video stream, and uses the priority configuration parameters of each video stream to indicate whether the priority configuration of the video stream takes effect in each video stream processing stage, and the processing priority of each video stream in each video stream processing stage. Calculate the occupancy rate of the key frames of the video stream on the network bandwidth based on the priority configuration parameters, evaluate the degree of video stream collision, and perform further video stream detection processing and collision elimination processing based on the degree of video stream collision. By configuring the priority configuration parameters of the video stream in each processing stage, video processing can be performed according to the priority of the video stream to ensure the transmission quality of high-priority video streams and maximize the network usage experience of most users.

[0039] Figure 1 It is a schematic flowchart of the video stream processing method provided by the embodiments of the present application. The execution subject of this method can be any electronic device with computing and processing capabilities. As Figure 1 shown, the method includes:

[0040] S101. Receive each video stream, and each video stream respectively includes a key frame sequence.

[0041] Optionally, the electronic device can be communicatively connected to multiple video stream sending sources. In the scenario of concurrent multi-video streams, each video stream sending source communicates with the electronic device and respectively sends the collected or stored video stream to the electronic device.

[0042] Among them, the video stream sending source can be a video stream acquisition device such as a camera or a device storing a video stream, and no specific limitation is made here. The video stream can be a video file for video playback and data transmission in the form of frames, and each video stream respectively includes a key frame sequence, that is, an I-frame sequence.

[0043] S102. Evaluate the degree of video stream collision according to each key frame sequence and the pre-configured priority configuration parameters. The priority configuration parameters are used to indicate whether to perform video stream processing according to the priority of the video stream in each video stream processing stage, and the processing priority of each video stream in each video stream processing stage.

[0044] Optionally, the key frame sequence is a series of I-frames arranged in the video playback order. The key frame sequence, i.e., the I-frame sequence, characterizes the features and changes of the video content. For example, information such as scene switching and object movement can be reflected through the I-frame sequence.

[0045] Based on various factors such as different business scenario requirements, service quality requirements, user preferences, and whether the video stream sending source is a key device, the priority configuration parameters of each video stream are pre-configured. The priority configuration parameters can be used as a switch for whether to process according to the priority in each stage, so as to indicate whether to process the video stream according to the priority of the video stream in each video stream processing stage, and the processing priority of each video stream in each video stream processing stage. Figure 2 The following is a schematic diagram of the architecture of each video stream processing stage provided by the embodiments of the present application. As Figure 2 shown, the video stream processing stage may include a video stream collision evaluation stage, a video stream collision detection stage, a video stream collision elimination stage, and an effect evaluation stage for video stream collision elimination.

[0046] Exemplarily, in a video conference, the video stream of the speaker is processed according to the priority of the video stream in each video stream processing stage, and the processing priority of the video stream of the speaker in each video stream processing stage is higher than that of the video stream of the participants.

[0047] It should be noted that for the same video stream, the priority of the video stream in different video stream processing stages may be the same or different. For example, in a video conference, when a participant is speaking, the processing priority of each video stream of the participant in each video stream processing stage is different.

[0048] Specifically, in the process of configuring the priority configuration parameters, clear priority rules can be formulated, and in the configuration file, the priority configuration parameters are configured for each video stream according to the priority rules. The priority configuration parameters may include the priority switch identifier and the priority identifier of each video stream processing stage. Exemplarily, if the priority switch identifiers in the video stream collision evaluation stage, the video stream collision detection stage, and the video stream collision elimination stage are 1, and the priority switch identifier in the effect evaluation stage for video stream collision elimination is 0, it means that in the video stream collision evaluation stage, the video stream collision detection stage, and the video stream collision elimination stage, the collision evaluation, collision detection, and collision elimination are performed according to the priority of the video stream, and in the effect evaluation stage for video stream collision elimination, the effect evaluation of collision elimination is not performed according to the priority of the video stream. Among them, the priority identifier of each video stream may be a character identifier (such as high and normal). For example, in the security monitoring scenario, the priority identifier of the video stream in the key area (such as the entrance and exit) may be high, and the priority identifier of the video stream in other areas may be normal.

[0049] According to the I-frame sequences and priority configuration parameters of each video stream, the degree of video stream collision is evaluated during the video stream collision evaluation phase. Specifically, first, according to the priority switch flag in the priority configuration parameters, it is determined whether to evaluate the degree of video stream collision according to the priorities of the video streams during the video stream collision evaluation phase. If so, the degree of video stream collision is evaluated according to the priority flags and I-frame sequences of each video stream during the video stream collision evaluation phase. If not, the degree of video stream collision is directly evaluated according to the I-frame sequences of each video stream. Among them, the degree of video stream collision can indicate the impact of the I-frame collision of multiple video streams on the network.

[0050] S103. Determine whether to perform collision detection processing and collision elimination processing according to the degree of video stream collision.

[0051] Optionally, according to the impact of the I-frame collision of multiple video streams indicated by the degree of video stream collision on the network, it is determined whether to perform subsequent video stream collision detection processing and collision elimination processing.

[0052] Among them, if the degree of video stream collision is low, indicating that the impact of the I-frame collision of multiple video streams on the network is limited, there is no need to perform collision detection processing and collision elimination processing. If the degree of video stream collision is high, indicating that the impact of the I-frame collision of multiple video streams on the network is large and it is easy to cause network congestion, collision detection processing and collision elimination processing need to be performed.

[0053] S104. If so, determine the target video stream according to the priority configuration parameters, and perform collision elimination processing on the target video stream according to the degree of video stream collision and the priority configuration parameters.

[0054] Optionally, in the case where collision detection processing and collision elimination processing need to be performed, during the video stream collision detection phase, according to the priority switch flag in the priority configuration parameters, it is determined whether to perform video stream collision detection according to the priorities of each video stream during the video stream collision detection phase. If so, video stream collision detection is performed according to the priority flags and I-frame sequences of each video stream during the video stream collision detection phase to select the target video stream for which collision needs to be eliminated. If not, video stream collision detection is directly performed according to the I-frame sequences of each video stream.

[0055] After selecting the target video stream, enter the video stream collision elimination stage. According to the video stream collision degree and the priority configuration parameters, perform collision elimination processing on the target video stream to obtain the processed target video stream. Specifically, in the video stream collision elimination stage, according to the priority switch identifier in the priority configuration parameters, determine whether to perform video stream collision elimination according to the priorities of each video stream in the video stream collision elimination stage. If so, perform video stream collision elimination on the target video stream according to the priority identifier of the target video stream in the video stream collision elimination stage and the video stream collision degree, so as to obtain the processed target video stream. If not, directly perform video stream collision elimination on the target video stream according to the video stream collision degree. Among them, different video stream collision degrees correspond to different video stream collision elimination processing methods. The specific corresponding relationship and each elimination method will be specifically explained in the subsequent embodiments.

[0056] It should be noted that after obtaining the processed target video stream, enter the effect evaluation stage of video stream collision elimination, and continue to determine whether to perform the effect evaluation of video stream collision elimination according to the priorities of each video stream according to the priority switch identifier in the priority configuration parameters in the effect evaluation stage of video stream collision elimination. If so, perform the effect evaluation of video stream collision elimination according to the priority identifier of each video stream in the effect evaluation stage of video stream collision elimination and the video stream collision degree, so as to decide whether to continue the collision elimination processing of the video stream.

[0057] In this embodiment, multiple video streams are received in a multi-video stream concurrent scenario, and the video stream collision degree is evaluated according to the key frame sequences included in each video stream and the pre-configured priority configuration parameters. Among them, the priority configuration parameter is a switch for whether to process according to the priority in each video stream processing stage, to indicate whether to perform video stream processing according to the priority of the video stream in each video stream processing stage, and the processing priority of each video stream in each video stream processing stage. According to the evaluated video stream collision degree, determine whether to perform collision detection processing and collision elimination processing. If so, in the video stream collision detection stage, determine the target video stream according to the priority configuration parameters, and in the video stream collision elimination stage, perform collision elimination processing on the target video stream according to the video stream collision degree and the priority configuration parameters to obtain the processed target video stream. By pre-configuring the priority configuration parameters of each video stream, the video streams participating in this stage can be flexibly selected in each video stream processing stage, greatly improving the flexibility of video stream processing. In each video stream processing stage, perform video stream processing according to the processing priorities of each video stream to protect more critical video streams and avoid affecting their transmission quality.

[0058] As an alternative implementation, the priority configuration parameters include: a first priority configuration parameter, which is used to indicate whether to perform video stream collision evaluation according to the priorities of video streams during the video stream collision evaluation phase, and the processing priorities of each video stream during the video stream collision evaluation phase.

[0059] Optionally, the priority configuration parameters include a first priority configuration parameter, which is specifically set for the video stream collision evaluation phase and may include a first priority switch identifier and a first priority identifier. It guides and constrains the collision evaluation process from two key aspects to ensure that the evaluation results can more accurately reflect the importance of different video streams during the collision evaluation phase. Among them, the first key aspect is to determine whether to perform video stream collision evaluation according to the priorities of video streams based on the first priority switch identifier during the video stream collision evaluation phase, that is, when the first priority switch identifier is on, the corresponding priority-effective evaluation mode; when the first priority switch identifier is off, the corresponding priority-invalid evaluation mode. The second key aspect is to distinguish the high and low processing priorities of each video stream during the video stream collision evaluation phase through the first priority identifiers of each video stream.

[0060] Next, the process of evaluating the degree of video stream collision according to each key frame sequence and the pre-configured priority configuration parameters will be described in detail.

[0061] Figure 3 The flowchart of evaluating the degree of video stream collision for the video stream processing method provided by the embodiments of this application is as Figure 3 shown. In step S102 above, evaluating the degree of video stream collision according to each key frame sequence and the pre-configured priority configuration parameters includes:

[0062] S301. Determine the video streams to be evaluated according to the first priority configuration parameter.

[0063] Optionally, during the video stream collision evaluation phase, screen out the video streams to be evaluated according to the first priority configuration parameter from each video stream.

[0064] Specifically, read the first priority configuration parameters pre-configured for each video stream, and screen out the video streams to be evaluated from each video stream according to the first priority switch identifier and the first priority identifier in the first priority configuration parameter. Among them, the first priority switch identifier is used to indicate the evaluation mode, including the priority-effective evaluation mode and the priority-invalid evaluation mode. The first priority identifier is used to indicate the high and low processing priorities of each video stream during the video stream collision evaluation phase.

[0065] S302. Evaluate the degree of video stream collision according to the key frame sequences of the video streams to be evaluated.

[0066] Optionally, after determining the video to be evaluated, the degree of video stream collision is evaluated according to the I-frame sequence of the video to be evaluated, so as to determine the impact degree of the I-frame collision of multiple video streams on the network. Among them, the degree of video stream collision is evaluated by analyzing the occupancy of network resources by the I-frame sequence of the video to be evaluated. The occupancy of network resources by the I-frame sequence of the video to be evaluated is positively correlated with the degree of video stream collision.

[0067] Specifically, the higher the occupancy ratio of the I-frame sequence of the video to be evaluated to network resources, the higher the degree of video stream collision, and the greater the impact degree of the I-frame collision of multiple video streams on the network.

[0068] In this embodiment, in the video stream collision evaluation stage, according to the first priority configuration parameter, the video stream to be evaluated is screened out from each video stream. After determining the video to be evaluated, the degree of video stream collision is evaluated according to the occupancy of network resources by the I-frame sequence of the video to be evaluated, so as to determine the impact degree of the I-frame collision of multiple video streams on the network. Among them, the occupancy of network resources by the I-frame sequence of the video to be evaluated is positively correlated with the degree of video stream collision. Determining the video stream to be evaluated based on the first priority configuration parameter improves the flexibility of video stream collision evaluation.

[0069] Next, the process of determining the video stream to be evaluated according to the first priority configuration parameter will be described in detail.

[0070] Figure 4 is a schematic flowchart of determining the video stream to be evaluated for the video stream processing method provided in the embodiment of the present application. As Figure 4 shown, in step S301 above, determining the video stream to be evaluated according to the first priority configuration parameter includes:

[0071] S401. According to the first priority configuration parameter, determine whether the priority configuration in the video stream collision evaluation stage takes effect.

[0072] Optionally, read the first priority switch identifier in the first priority configuration parameter, and determine whether the priority configuration in the video stream collision evaluation stage takes effect according to the first priority switch identifier.

[0073] Specifically, if the first priority switch identifier is on, the indicated evaluation mode is the evaluation mode in which the priority takes effect, and it is determined that the evaluation needs to be performed according to the processing priority of each video stream in the video stream collision evaluation stage. If the first priority switch identifier is off, the indicated evaluation mode is the evaluation mode in which the priority fails, and it is determined that there is no need to evaluate according to the processing priority of each video stream in the video stream collision evaluation stage.

[0074] S402. If so, determine the video streams with normal priority according to the first priority configuration parameter, and use the video streams with normal priority as the video streams to be evaluated.

[0075] Optionally, if it is determined that the priority configuration in the video stream collision evaluation phase takes effect, then in the evaluation mode where the priority takes effect, the video streams to be evaluated are determined according to the first priority identifiers of the respective video streams.

[0076] Specifically, read the first priority identifiers in the first priority configuration parameters of the respective video streams, filter out the video streams with the first priority identifier being ordinary, and use the video streams with the first priority identifier being ordinary as the video streams to be evaluated. That is, exclude the video streams with high priority from the received video streams, and use the video streams with ordinary priority as the video streams to be evaluated.

[0077] That is to say, only the video streams with ordinary priority participate in the evaluation of the video stream collision degree, so that the bandwidth requirements occupied by the video streams with high priority do not need to be considered in the subsequent evaluation process, and the transmission quality of the video streams with high priority is guaranteed.

[0078] S403: Otherwise, use each video stream as the video stream to be evaluated.

[0079] Optionally, if it is determined that the priority configuration in the video stream collision evaluation phase fails, then in the evaluation mode where the priority fails, use each video stream as the video stream to be evaluated. That is to say, all video streams participate in the evaluation of the video stream collision degree.

[0080] In this embodiment, according to whether the video stream collision evaluation is performed according to the priority of the video stream in the video stream collision evaluation phase indicated by the first priority configuration parameter, it is determined whether the priority configuration in the video stream collision evaluation phase takes effect. If so, according to the processing priorities of the respective video streams in the video stream collision evaluation phase indicated by the first priority configuration parameter, use the video streams with ordinary priority as the video streams to be evaluated, so that only the video streams with ordinary priority participate in the evaluation of the video stream collision degree. If the priority configuration in the video stream collision evaluation phase does not take effect, use each video stream as the video stream to be evaluated, so that all video streams participate in the evaluation of the video stream collision degree. Determining the video streams to be evaluated as all video streams or the video streams with ordinary priority based on the first priority configuration parameter improves the flexibility of the video streams to be evaluated, so as to flexibly protect the transmission quality of the video streams with high priority.

[0081] Next, the process of evaluating the video stream collision degree according to the key frame sequence of the video stream to be evaluated will be described in detail.

[0082] Figure 5 This is another flowchart showing the process of evaluating the video stream collision degree for the video stream processing method provided by the embodiment of the present application, as Figure 5As shown, in step S302 above, the degree of video stream collision is evaluated according to the key frame sequence of the video stream to be evaluated, including:

[0083] S501. Determine the total packet accumulation value of the key frames of the video stream to be evaluated according to the key frame sequence of the video stream to be evaluated.

[0084] Optionally, there is an observation window with a preset time length T-win on the electronic device. The video stream to be evaluated is numbered within the observation window at the current time T-win, and a unique corresponding identifier flow_ID is assigned to each video stream to be evaluated to distinguish the data of different video streams to be evaluated.

[0085] Exemplarily, if the electronic device determines N video streams to be evaluated within the observation window at the current time T-win, then the identifiers flow_ID of the N video streams to be evaluated = [1, 2, 3,..., N]. The identifier of the i-th video stream to be evaluated is flow_ID i , where 1 ≤ i ≤ N, and i is any one of the N video streams to be evaluated.

[0086] First, determine the packet accumulation value Data vol (i) in the I-frame of the i-th video stream to be evaluated, and then calculate the total packet accumulation value Data-I of the I-frames of the n video streams to be evaluated within the observation window at the current time T-win based on the following formula:

[0087]

[0088] where Data-I is the total packet accumulation value of the I-frames of the N video streams to be evaluated within the observation window at the current time T-win, i is any one of the N video streams to be evaluated, N is the number of video streams to be evaluated, and Data vol (i) is the packet accumulation value in the I-frame of the i-th video stream to be evaluated.

[0089] The total packet accumulation value Data-I of the I-frames of the N video streams to be evaluated within the observation window at the current time T-win is the result of adding up the packet sizes of all the I-frames of each video stream to be evaluated. This value can be used to evaluate the total bandwidth or storage space occupied by all the I-frames in the video stream to be evaluated.

[0090] S502. Determine the video occupancy rate of the video stream to be evaluated according to the total packet accumulation value of the key frames and the network transmission bandwidth.

[0091] Optionally, obtain the network transmission bandwidth BW, which can represent the transmission capacity of the network environment. Based on the cumulative value Data-I of the total I-frame packets of N video streams to be evaluated within the observation window of the current time T-win and the network transmission bandwidth BW, determine the video occupancy ratio Ratio-I of the N video streams to be evaluated within the observation window of the current time T-win according to the following formula:

[0092]

[0093] Among them, Ratio-I is the cumulative value of the total I-frame packets of N video streams to be evaluated within the observation window of the current time T-win, T-win is the time length of the current observation window, and BW is the network bandwidth.

[0094] Calculate the ratio of the cumulative value Ratio-I of the total I-frame packets of N video streams to be evaluated within the observation window of the current time T-win to the network bandwidth T-win*BW within the observation window of the current time T-win. The relative degree of network bandwidth occupied by the video streams to be evaluated within the observation window of the current time T-win can be reflected by the video occupancy ratio Ratio-I.

[0095] S503. Evaluate the degree of video stream collision according to the video occupancy ratio.

[0096] Optionally, evaluate the degree of video stream collision according to the relative degree of network bandwidth occupied by the video streams to be evaluated reflected by the video occupancy ratio Ratio-I of the N video streams to be evaluated within the observation window of the current time T-win.

[0097] Among them, the higher the video occupancy ratio Ratio-I of the N video streams to be evaluated within the observation window of the current time T-win, the higher the relative degree of network bandwidth occupied by the video streams to be evaluated, and the greater the degree of video stream collision, and the more likely it is to cause network congestion.

[0098] In this embodiment, determine the cumulative value of the packets in the key frames of each video stream to be evaluated, and add up the cumulative values of the packets of all the key frames of each video stream to be evaluated to obtain the total cumulative value of the key-frame packets of the video stream to be evaluated. Calculate the ratio of the total cumulative value of the key-frame packets of the video stream to be evaluated within the observation window of the preset time to the network bandwidth within the observation window of the preset time, and determine the video occupancy ratio of the video stream to be evaluated. Evaluate the degree of video stream collision according to the relative degree of network bandwidth occupied by the video stream to be evaluated reflected by the video occupancy ratio of the video stream to be evaluated. By calculating the video occupancy ratio of the video to be evaluated, the degree of video stream collision is evaluated intuitively and quantitatively.

[0099] Next, the process of evaluating the degree of video stream collision according to the video occupancy ratio will be described in detail.

[0100] Figure 6 Another flowchart for evaluating the video stream collision degree of the video stream processing method provided by the embodiment of the present application is shown as Figure 6 shown. In step S503 above, evaluating the video stream collision degree according to the video occupancy rate includes:

[0101] S601. If the video occupancy rate is less than or equal to the first threshold, determine that the video stream collision degree is the first level.

[0102] Optionally, based on factors such as the transmission performance of the network, the quality requirements of the video service, and the actual application scenario, each threshold is preset to measure the interval where the video occupancy rate is located through each preset threshold, and then the level classification of the video stream collision degree is determined through the interval where the video occupancy rate is located. Among them, the preset thresholds include the first threshold and the second threshold. Exemplarily, the first threshold can be 1, and the second threshold can be 2. Or, the first threshold can be 50%, and the second threshold can be 80%. The present application does not specifically limit the specific values of the first threshold and the second threshold.

[0103] Compare the size relationship between the video occupancy rate Ratio-I of N video streams to be evaluated within the observation window of the current time T-win with the first threshold and the second threshold. If the video occupancy rate Ratio-I of N video streams to be evaluated within the observation window of the current time T-win is less than or equal to the first threshold, determine that the video stream collision degree is the first level, indicating that the proportion of the video service in the network bandwidth of the observation window of the current time T-win is relatively low, that is, the demand for network bandwidth during video concurrency is small, and the impact of I-frame collision on the network is limited.

[0104] S602. If the video occupancy rate is greater than the first threshold and less than or equal to the second threshold, determine that the video stream collision degree is the second level.

[0105] Optionally, if the video occupancy rate Ratio-I of N video streams to be evaluated within the observation window of the current time T-win is greater than the first threshold and less than or equal to the second threshold, determine that the video stream collision degree is the second level, indicating that the proportion of the video service in the network bandwidth of the observation window of the current time T-win is relatively high, that is, the video service is the main consuming service of network bandwidth resources. At this time, the impact of I-frame collision on the network is relatively large.

[0106] S603. If the video occupancy rate is greater than the second threshold, determine that the video stream collision degree is the third level.

[0107] Optionally, if the video occupancy ratio Ratio-I of N video streams to be evaluated within the observation window of the current time T-win is greater than the second threshold, it is determined that the video stream collision degree is the third level, indicating that the video service has almost completely occupied the network bandwidth of the observation window at the current time T-win. At this time, a slight I collision can have a greater impact on the network.

[0108] In this embodiment, by comparing the size relationship between the video occupancy ratio and the first threshold and the second threshold, the level of the video stream collision degree is determined according to the comparison result. If the video occupancy ratio is less than or equal to the first threshold, it is determined that the video stream collision degree is the first level. At this time, the impact of key frame collision on the network is limited. If the video occupancy ratio is greater than the first threshold and less than or equal to the second threshold, it is determined that the video stream collision degree is the second level. At this time, the key frame collision has a greater impact on the network. If the video occupancy ratio is greater than the second threshold, it is determined that the video stream collision degree is the third level. At this time, a slight key frame collision can have a greater impact on the network. The accuracy of video stream collision assessment is improved.

[0109] Next, the process of determining whether to perform collision detection processing and collision elimination processing according to the video stream collision degree will be described in detail.

[0110] Figure 7 It is a flowchart showing the process of determining whether to perform collision detection processing and collision elimination processing for the video stream processing method provided by the embodiment of the present application. As Figure 7 shown, in step S103 above, determining whether to perform collision detection processing and collision elimination processing according to the video stream collision degree includes:

[0111] S701. If the video stream collision degree is the first level, it is determined not to perform collision detection processing and collision elimination processing.

[0112] Optionally, when the video stream collision degree is determined to be the first level, it means that the resource competition situation faced by the video stream to be evaluated during network transmission is in a relatively benign and stable state. In this state, the resources such as network bandwidth occupied by the video stream to be evaluated are within an acceptable range. It is unlikely that obvious interference and conflicts will occur between the video streams to be evaluated due to resource competition. The network can more easily meet the transmission requirements of the video stream and is not prone to abnormal situations such as stuttering and packet loss. Therefore, it can be determined that there is no need to perform special video stream collision detection processing and collision elimination processing.

[0113] S702. If the video stream collision degree is the second level or the third level, it is determined to perform collision detection processing and collision elimination processing.

[0114] Optionally, when the video stream collision degree is at the second level or the third level, it indicates that the video stream to be evaluated faces relatively serious resource competition problems during network transmission, and the network condition can no longer allow the video stream to be transmitted smoothly and stably. Therefore, at this time, it is necessary to carry out video stream collision detection processing and collision elimination processing to ensure the transmission quality of the video stream and the normal operation of the network. Among them, the collision elimination processing methods corresponding to the second level and the third level are different.

[0115] In this embodiment, if the video stream collision degree is at the first level, it means that the network bandwidth occupied by the video stream to be evaluated is limited, and it is determined not to perform collision detection processing and collision elimination processing. If the video stream collision degree is at the second level or the third level, it means that the network bandwidth occupied by the video stream to be evaluated is relatively high, and it is determined to perform collision detection processing and collision elimination processing. When the video stream collision degree is at the first level, it avoids the unnecessary waste of computing resources in the collision detection processing and collision elimination processing.

[0116] As an optional implementation manner, the priority configuration parameter further includes: a second priority configuration parameter, and the second priority configuration parameter is used to indicate whether to perform video stream collision detection according to the priority of the video stream during the video stream collision detection stage, and the processing priority of each video stream during the video stream collision detection stage.

[0117] Optionally, the priority configuration parameter further includes a second priority configuration parameter, which is specifically set for the video stream collision detection stage and may include a second priority switch identifier and a second priority identifier. It guides and constrains the collision detection process from two key aspects to ensure that the evaluation result can more accurately reflect the importance of different video streams during the collision detection stage. Among them, the first key aspect is to determine whether to perform video stream collision detection according to the priority of the video stream during the video stream collision detection stage according to the second priority switch identifier, that is, when the second priority switch identifier is on, it corresponds to the detection mode in which the priority takes effect; when the second priority switch identifier is off, it corresponds to the detection mode in which the priority fails. The second key aspect is to distinguish the high and low processing priorities of each video stream during the video stream collision detection stage through the second priority identifiers of each video stream.

[0118] Next, the process of determining the target video stream according to the priority configuration parameter will be described in detail.

[0119] Figure 8 It is a schematic flowchart of the process of determining the target video stream for the video stream processing method provided by the embodiment of the present application, as Figure 8 shown, determining the target video stream according to the priority configuration parameter in step S104 above includes:

[0120] S801. Determine whether the priority configuration in the video stream collision detection phase takes effect according to the second priority configuration parameter.

[0121] Optionally, read the second priority switch identifier in the second priority configuration parameter, and determine whether the priority configuration in the video stream collision detection phase takes effect according to the second priority switch identifier.

[0122] Specifically, if the second priority switch identifier is on, the indicated detection mode is the detection mode in which the priority takes effect, and it is determined that the detection needs to be performed according to the processing priority of each video stream in the video stream collision detection phase. If the second priority switch identifier is off, the indicated detection mode is the detection mode in which the priority fails, and it is determined that there is no need to perform the detection according to the processing priority of each video stream in the video stream collision detection phase.

[0123] S802. If so, determine the video streams with normal priority according to the second priority configuration parameter, and determine the target video stream according to the video streams with normal priority.

[0124] Optionally, if it is determined that the priority configuration in the video stream collision detection phase takes effect, then in the detection mode in which the priority takes effect, determine the video streams with normal priority according to the second priority identifiers of each video stream. Specifically, read the second priority identifiers in the second priority configuration parameters of each video stream, and filter out the video streams with the second priority identifier being normal. That is to say, only the video streams with normal priority participate in the video stream collision detection, so as to exclude the video streams with high priority in the subsequent collision detection and elimination process, avoid performing collision elimination processing on the video streams with high priority, and further ensure the transmission quality of the video streams with high priority.

[0125] Determine the target video stream from each video stream with normal priority. Among them, the target video stream can be the video stream with the smallest change in the picture scene among each video stream with normal priority, or the video stream with the most collision combinations among each video stream with normal priority, or the video stream with the most collision occurrences among each video stream with normal priority. The present application does not specifically limit the method of selecting the target video stream in the video stream collision detection phase.

[0126] S803. Otherwise, determine the target video stream according to each video stream.

[0127] Optionally, if it is determined that the priority configuration in the video stream collision detection phase fails, then in the detection mode in which the priority fails, determine the target video stream from each video stream. That is to say, all video streams participate in the video stream collision detection.

[0128] Among them, the target video stream can be the video stream with the least change in the picture scene among all video streams, or the video stream with the most collision combinations among all video streams, or the video stream with the most collision occurrences among all video streams. The specific method of selecting the target video stream in the video stream collision detection stage is not limited herein.

[0129] In this embodiment, according to whether video stream collision detection is performed according to the priorities of video streams as indicated by the second priority configuration parameter in the video stream collision detection stage, it is determined whether the priority configuration in the video stream collision detection stage takes effect. If so, according to the processing priorities of each video stream as indicated by the second priority configuration parameter in the video stream collision detection stage, the video streams with normal priorities are determined, so that only the video streams with normal priorities participate in video stream collision detection, and the target video stream is determined from the video streams with normal priorities. If the priority configuration in the video stream collision detection stage does not take effect, the target video stream is determined from all video streams, so that all video streams participate in video stream collision detection. Based on the second priority configuration parameter, it is determined whether the video streams with high priorities participate in video stream collision detection, which improves the flexibility of selecting the target video stream and further flexibly protects the transmission quality of the video streams with high priorities.

[0130] As an optional implementation manner, the priority configuration parameter further includes: a third priority configuration parameter, and the third priority configuration parameter is used to indicate whether video stream collision elimination is performed according to the priorities of video streams in the video stream collision elimination stage, and the processing priorities of each video stream in the video stream collision elimination stage.

[0131] Optionally, the priority configuration parameter further includes a third priority configuration parameter, which is specifically set for the video stream collision elimination stage and may include a third priority switch identifier and a third priority identifier. It guides and constrains the collision elimination process from two key aspects to ensure that it more accurately reflects the importance of video streams in the video stream collision elimination stage. Among them, the first key aspect is to determine whether video stream collision elimination is performed according to the priorities of video streams according to the third priority switch identifier in the video stream collision elimination stage, that is, when the third priority switch identifier is on, the elimination mode with the corresponding priority takes effect; when the third priority switch identifier is off, the elimination mode with the corresponding priority fails. The second key aspect is to distinguish the high and low processing priorities of each video stream in the video stream collision elimination stage through the third priority identifiers of each video stream.

[0132] Next, the process of performing collision elimination processing on the target video stream according to the video stream collision degree and the priority configuration parameter is described in detail.

[0133] Figure 9This is a schematic flowchart of the collision elimination process for the target video stream in the video stream processing method provided by the embodiments of this application. As Figure 9 shown, in step S104 above, according to the video stream collision degree and the priority configuration parameter, the collision elimination process for the target video stream includes:

[0134] S901. According to the third priority configuration parameter, determine whether the priority configuration for video stream collision elimination takes effect, and determine whether the priority of the target video stream is a high priority according to the third priority configuration parameter.

[0135] Optionally, read the third priority switch identifier in the third priority configuration parameter, and according to the third priority switch identifier, determine whether the priority configuration in the video stream collision elimination stage takes effect. And read the third priority identifier in the third priority configuration parameter of the target video stream, and according to the third priority identifier, determine whether the priority of the target video stream is a high priority.

[0136] Specifically, if the third priority switch identifier is on, the indicated elimination mode is the elimination mode in which the priority takes effect, and it is determined that the collision elimination needs to be performed according to the processing priority of each video stream in the video stream collision elimination stage. If the third priority switch identifier is off, the indicated elimination mode is the elimination mode in which the priority fails, and it is determined that there is no need to perform collision elimination according to the processing priority of each video stream in the video stream collision elimination stage. If the third priority identifier of the target video stream is high, it is determined that the priority of the target video stream is a high priority. If the third priority identifier of the target video stream is normal, it is determined that the priority of the target video stream is a normal priority, that is, not a high priority.

[0137] S902. If it takes effect and the priority of the target video stream is a high priority, reconfigure the second priority configuration parameter, and according to the reconfigured second priority configuration parameter, re-determine the target video stream with a normal priority, and perform collision elimination processing on the target video stream according to the video stream collision degree.

[0138] Optionally, if it is determined that the priority configuration in the video stream collision elimination stage takes effect and the target video stream is a video stream with a high priority, since the priority of the target video stream is a high priority, when performing collision elimination processing on the target video stream with a high priority in the subsequent video stream collision elimination stage, it will affect the transmission quality of the target video stream with a high priority, so it is necessary to re-select the target video stream with a normal priority.

[0139] Specifically, the priority of the target video stream is high priority, indicating that the second priority switch identifier in the second priority configuration parameter is off. Then, it is necessary to reconfigure the second priority switch identifier in the second priority configuration parameter to on, filter out the video stream with high priority during the video stream collision detection stage, and re-determine the target video stream with normal priority according to the second priority identifier in the reconfigured second priority configuration parameter. And during the video stream collision evaluation stage, according to the degree of video stream collision, collision elimination processing is performed on the target video stream with normal priority during the video stream collision elimination stage. Protect the video stream with high priority to avoid performing collision elimination processing on the video stream with high priority.

[0140] S903. Otherwise, according to the degree of video stream collision, collision elimination processing is performed on the target video stream.

[0141] Optionally, if it is determined that the priority configuration in the video stream collision elimination stage takes effect and the target video stream is a video stream with normal priority, or if it is determined that the priority configuration in the video stream collision elimination stage does not take effect, then directly according to the degree of video stream collision, collision elimination processing is performed on the target video stream.

[0142] Specifically, if it is determined that the priority configuration in the video stream collision elimination stage takes effect and the target video stream is a video stream with normal priority, then in the elimination mode where the priority takes effect, collision elimination processing is performed on the target video stream according to the degree of video stream collision. If it is determined that the priority configuration in the video stream collision elimination stage does not take effect, then in the elimination mode where the priority fails, collision elimination processing is directly performed on the target video stream according to the degree of video stream collision.

[0143] In this embodiment, it is determined whether the priority configuration in the video stream collision elimination stage takes effect according to whether video stream collision elimination is performed according to the priority of the video stream as indicated by the third priority configuration parameter. The priority of the target video stream is determined to be high priority according to the processing priority of the target video stream in the video stream collision elimination stage as indicated by the third priority configuration parameter of the target video stream. If it takes effect and the priority of the target video stream is high priority, then the second priority configuration parameter is reconfigured, and the target video stream with normal priority is re-determined according to the reconfigured second priority configuration parameter. According to the degree of video stream collision, collision elimination processing is performed on the target video stream. Otherwise, directly according to the degree of video stream collision, collision elimination processing is performed on the target video stream. Avoid performing collision elimination processing on the video stream with high priority.

[0144] Next, the process of performing collision elimination processing on the target video stream according to the degree of video stream collision is described in detail.

[0145] Figure 10Another schematic diagram of the process for performing collision elimination processing on the target video stream in the video stream processing method provided by the embodiments of this application. As Figure 10 shown, in steps S901 and S902 above, according to the video stream collision degree, performing collision elimination processing on the target video stream includes:

[0146] S1001. If the video stream collision degree is the second level, perform first collision elimination processing on the target video stream to adjust the encoding parameters of the target video stream.

[0147] Optionally, when the video stream collision degree is at the second level, it means that the resource competition of the video stream has begun to affect the video transmission quality, but has not reached an extremely serious level. Some minor abnormal phenomena may occur, such as the video occasionally freezing, the frame rate fluctuating slightly, the packet loss rate rising slightly, etc. The first collision elimination processing can be performed on the target video stream to adjust the encoding parameters of the target video stream.

[0148] Specifically, the processor of the electronic device sends the identifier of the target video stream to the encoder deployed on the electronic device. The encoder deployed on the electronic device performs first collision elimination processing on the target video stream corresponding to the identifier of the target video stream at the encoding end based on the identifier of the target video stream to obtain the first processed target video stream. Performing first collision elimination processing on the target video stream can be to adjust the encoding parameters of the target video stream to adjust the occupied bandwidth when transmitting the target video stream. Among them, the encoding parameters of the video stream at least include: resolution, video frame rate, I-frame interval, code rate upper limit, encoding complexity, and image quality. The code rate upper limit is an unadjustable encoding parameter, that is, a fixed value, and the resolution, video frame rate, I-frame interval, encoding complexity, and image quality are adjustable encoding parameters.

[0149] S1002. If the video stream collision degree is the third level, perform second collision elimination processing on the target video stream to close the target video stream and restart the target video stream after a preset duration.

[0150] Optionally, when the video stream collision degree is at the third level, it means that the collision situation of the video stream is very serious and the video transmission quality will be greatly affected. Frequent freezing, long buffering, a large number of packet losses, and even the video screen showing a mosaic or being unable to play normally may occur, seriously affecting the user's viewing experience. At this time, it is necessary to immediately perform second collision elimination processing on the target video stream to close the target video stream and restart the target video stream after a preset duration. By temporarily closing the target video stream to alleviate network congestion.

[0151] Specifically, a restart timer with a preset duration is set for the target video stream, the target video stream is closed, and the restart timer is activated. When the restart timer expires, the video stream sending source of the target video stream is restarted. Among them, the preset duration of the restart timer can be the product of the I-frame period of the target video stream and a preset value, and the preset value can be any non-zero positive integer.

[0152] In this embodiment, if the video stream collision degree is the second level, the first collision elimination process is performed on the target video stream to adjust the encoding parameters of the target video stream so as to adjust the occupied bandwidth when transmitting the target video stream. If the video stream collision degree is the third level, the second collision elimination process is performed on the target video stream to close the target video stream and restart the target video stream after a preset duration. The network congestion is alleviated by immediately and temporarily closing the target video stream. The flexibility, accuracy, and rationality of the video stream collision elimination process are improved.

[0153] It should be noted that the priority configuration parameter further includes: a fourth priority configuration parameter, which is used to indicate whether to evaluate the effect of video stream collision elimination according to the priority of the video stream during the effect evaluation stage of video stream collision elimination, and the processing priority of each video stream during the effect evaluation stage of video stream collision elimination. So as to evaluate the effect of video stream collision elimination according to the fourth priority configuration parameter.

[0154] Based on the same inventive concept, an embodiment of the present application also provides a video stream processing device corresponding to the video stream processing method. Since the principle of solving problems by the device in the embodiment of the present application is similar to the above video stream processing method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0155] Figure 11 For the module structure diagram of the video stream processing device provided by the embodiment of the present application, as Figure 11 shown, the device includes:

[0156] A receiving module 1101, configured to receive each video stream, and each video stream respectively includes a key frame sequence.

[0157] An evaluation module 1102, configured to evaluate the video stream collision degree according to each key frame sequence and the pre-configured priority configuration parameter, where the priority configuration parameter is used to indicate whether to perform video stream processing according to the priority of the video stream during each video stream processing stage, and the processing priority of each video stream during each video stream processing stage.

[0158] A determination module 1103, configured to determine whether to perform collision detection processing and collision elimination processing according to the video stream collision degree.

[0159] A processing module 1104, which, if so, determines a target video stream according to the priority configuration parameter, and performs collision elimination processing on the target video stream according to the video stream collision degree and the priority configuration parameter.

[0160] As an alternative implementation, the priority configuration parameter includes: a first priority configuration parameter, which is used to indicate whether to perform video stream collision evaluation according to the priority of the video stream during the video stream collision evaluation stage, and the processing priority of each video stream during the video stream collision evaluation stage; specifically, the evaluation module 1102 is configured to:

[0161] Determine the video stream to be evaluated according to the first priority configuration parameter.

[0162] Evaluate the video stream collision degree according to the key frame sequence of the video stream to be evaluated.

[0163] As an alternative implementation, the evaluation module 1102 is specifically configured to:

[0164] Determine whether the priority configuration during the video stream collision evaluation stage takes effect according to the first priority configuration parameter.

[0165] If so, determine the video stream with normal priority according to the first priority configuration parameter, and use the video stream with normal priority as the video stream to be evaluated.

[0166] Otherwise, use each video stream as the video stream to be evaluated.

[0167] As an alternative implementation, the evaluation module 1102 is specifically configured to:

[0168] Determine the total key frame packet accumulation value of the video stream to be evaluated according to the key frame sequence of the video stream to be evaluated.

[0169] Determine the video occupancy rate of the video stream to be evaluated according to the total key frame packet accumulation value and the network transmission bandwidth.

[0170] Evaluate the video stream collision degree according to the video occupancy rate.

[0171] As an alternative implementation, the evaluation module 1102 is specifically configured to:

[0172] If the video occupancy rate is less than or equal to the first threshold, determine that the video stream collision degree is the first level.

[0173] If the video occupancy rate is greater than the first threshold and less than or equal to the second threshold, determine that the video stream collision degree is the second level.

[0174] If the video occupancy rate is greater than the second threshold, determine that the video stream collision degree is the third level.

[0175] As an alternative implementation, the determining module 1103 is specifically configured to:

[0176] If the degree of video stream collision is at the first level, it is determined not to perform collision detection processing and collision elimination processing.

[0177] If the degree of video stream collision is at the second level or the third level, it is determined to perform collision detection processing and collision elimination processing.

[0178] As an alternative implementation, the priority configuration parameter further includes: a second priority configuration parameter, which is used to indicate whether to perform video stream collision detection according to the priority of the video stream during the video stream collision detection stage, and the processing priority of each video stream during the video stream collision detection stage; the processing module 1104 is specifically configured to:

[0179] According to the second priority configuration parameter, determine whether the priority configuration during the video stream collision detection stage takes effect.

[0180] If so, determine the video streams with normal priority according to the second priority configuration parameter, and determine the target video stream according to the video streams with normal priority.

[0181] Otherwise, determine the target video stream according to each video stream.

[0182] As an alternative implementation, the priority configuration parameter further includes: a third priority configuration parameter, which is used to indicate whether to perform video stream collision elimination according to the priority of the video stream during the video stream collision elimination stage, and the processing priority of each video stream during the video stream collision elimination stage; the processing module 1104 is specifically configured to:

[0183] According to the third priority configuration parameter, determine whether the priority configuration of video stream collision elimination takes effect, and determine whether the priority of the target video stream is high priority according to the third priority configuration parameter.

[0184] If it takes effect and the priority of the target video stream is high priority, reconfigure the second priority configuration parameter, determine the target video stream with normal priority again according to the reconfigured second priority configuration parameter, and perform collision elimination processing on the target video stream according to the degree of video stream collision.

[0185] Otherwise, perform collision elimination processing on the target video stream according to the degree of video stream collision.

[0186] As an alternative implementation, the processing module 1104 is specifically configured to:

[0187] If the video stream collision level is the second level, perform the first collision elimination process on the target video stream to adjust the encoding parameters of the target video stream.

[0188] If the video stream collision level is the third level, perform the second collision elimination process on the target video stream to turn off the target video stream and restart the target video stream after a preset duration.

[0189] The embodiment of the present application also provides an electronic device, as Figure 12 shown, which is a schematic structural diagram of the electronic device provided by the embodiment of the present application, including: a processor 121, a memory 122, and a bus 123. The memory 122 stores machine-readable instructions executable by the processor 121 (such as, Figure 11 the execution instructions corresponding to the receiving module 1101, the evaluation module 1102, the determination module 1103, and the processing module 1104 in the device in

[0190] When the electronic device runs, the processor 121 communicates with the memory 122 through the bus 123. When the machine-readable instructions are executed by the processor 121, the steps of the video stream processing method in the above embodiment are executed.

[0191] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the method embodiments, and will not be repeated in this application. In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, and the indirect coupling or communication connection of the devices or modules may be in an electrical, mechanical, or other form.

[0192] In addition, each functional unit in various embodiments of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0193] The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application.

Claims

1. A video stream processing method, characterized in that: The method comprises: Receiving each video stream, each of the video streams comprises a key frame sequence; Evaluate the degree of video stream collision according to each of the key frame sequences and a pre-configured priority configuration parameter, wherein the priority configuration parameter is used to indicate whether to perform video stream processing according to the priority of the video stream in each video stream processing stage, and the processing priority of each video stream in each video stream processing stage; Determining whether to perform collision detection and collision elimination according to the collision degree of the video stream; If so, the target video stream is determined according to the priority configuration parameters, and collision elimination processing is performed on the target video stream according to the collision degree of the video stream and the priority configuration parameters.

2. The method according to claim 1, characterized in that The priority configuration parameters include: a first priority configuration parameter, the first priority configuration parameter is used to indicate whether to perform video stream collision evaluation according to the priority of the video stream in the video stream collision evaluation stage, and the processing priority of each video stream in the video stream collision evaluation stage; The step of evaluating the collision degree of the video streams according to the key frame sequences and the pre-configured priority configuration parameters of the video streams includes: Determining a video stream to be evaluated according to the first priority configuration parameter; The collision degree of the video stream is evaluated according to the key frame sequence of the video stream to be evaluated.

3. The method according to claim 2, characterized in that The step of determining the video stream to be evaluated according to the first priority configuration parameter includes: Determining whether the priority configuration in the video stream collision assessment phase is effective according to the first priority configuration parameter; If so, determining a video stream with a normal priority according to the first priority configuration parameter, and using the video stream with the normal priority as the video stream to be evaluated; Otherwise, each of the video streams is used as the video stream to be evaluated.

4. The method according to claim 2, characterized in that: The step of evaluating the collision degree of the video stream according to the key frame sequence of the video stream to be evaluated includes: Determine the total key frame message accumulation value of the video stream to be evaluated according to the key frame sequence of the video stream to be evaluated; Determine the video occupancy rate of the video stream to be evaluated according to the total key frame message accumulation value and the network transmission bandwidth; The video stream collision degree is evaluated according to the video occupancy rate.

5. The method according to claim 4, characterized in that The step of evaluating the video stream collision degree according to the video occupancy rate includes: If the video occupancy rate is less than or equal to a first threshold, determining that the video stream collision degree is a first level; If the video occupancy rate is greater than the first threshold and less than or equal to the second threshold, determining that the video stream collision degree is the second level; If the video occupancy rate is greater than the second threshold, it is determined that the video stream collision degree is a third level.

6. The method according to claim 5, characterized in that The step of determining whether to perform collision detection processing and collision elimination processing according to the collision degree of the video stream comprises: If the collision degree of the video stream is the first level, determining not to perform collision detection processing and collision elimination processing; If the collision degree of the video stream is the second level or the third level, it is determined to perform collision detection processing and collision elimination processing.

7. The method according to claim 1, characterized in that The priority configuration parameter also includes: a second priority configuration parameter, the second priority configuration parameter is used to indicate whether to perform video stream collision detection according to the priority of the video stream in the video stream collision detection stage, and the processing priority of each video stream in the video stream collision detection stage; The step of determining the target video stream according to the priority configuration parameter comprises: Determining whether the priority configuration of the video stream collision detection phase is effective according to the second priority configuration parameter; If so, determining a video stream with a normal priority according to the second priority configuration parameter, and determining the target video stream according to the video stream with the normal priority; Otherwise, the target video stream is determined according to each of the video streams.

8. The method according to claim 1, characterized in that The priority configuration parameter also includes: a third priority configuration parameter, the third priority configuration parameter is used to indicate whether to perform video stream collision elimination according to the priority of the video stream in the video stream collision elimination stage, and the processing priority of each video stream in the video stream collision elimination stage; The performing collision elimination processing on the target video stream according to the collision degree of the video stream and the priority configuration parameter comprises: Determine whether the priority configuration of video stream collision elimination is effective according to the third priority configuration parameter, and determine whether the priority of the target video stream is a high priority according to the third priority configuration parameter; If it is effective, and the priority of the target video stream is high priority, reconfigure the second priority configuration parameters, and re-determine the target video stream with normal priority according to the reconfigured second priority configuration parameters, and perform collision elimination processing on the target video stream according to the collision degree of the video stream; Otherwise, collision elimination processing is performed on the target video stream according to the collision degree of the video stream.

9. The method according to claim 8, characterized in that The performing collision elimination processing on the target video stream according to the collision degree of the video stream comprises: If the collision degree of the video stream is the second level, performing a first collision elimination process on the target video stream to adjust the encoding parameters of the target video stream; If the collision degree of the video stream is the third level, a second collision elimination process is performed on the target video stream to close the target video stream, and the target video stream is restarted after a preset time.

10. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the video stream processing method as described in any one of claims 1 to 9.

Images