Video processing method based on video integrated platform, video integrated platform and equipment

Abstract

The invention provides a video processing method based on a video integrated platform, the video integrated platform and equipment, and the method comprises the steps: determining a first time difference and at least one second time difference through a main control module, transmitting the first time difference to an input module, and transmitting each second time difference to a corresponding decoding module; the first time difference is the time difference of the input module relative to the main control module, and the second time difference is the time difference of the decoding module relative to the main control module; a sending time sequence value of the collected video data is determined through an input module according to the first time difference, and the video data and the corresponding sending time sequence value are sent to each decoding module; and for any decoding module, determining the current to-be-displayed video data through the decoding module according to the second time difference and the sending time sequence value of the video data, and sending and displaying the current to-be-displayed video data. Therefore, synchronous transmission and display of the video data in the video integrated platform can be realized, and the synchronous display effect of the video data is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of video data processing, and in particular to a video processing method based on a video integration platform, a video integration platform and a device. Background Art

[0002] The video integrated platform is a telecom-grade rack-mounted video processing integrated platform product. The video integrated platform usually includes a main control, input board, and decoding output board. The input board is mainly used to collect video data, and the decoding output board is used to receive video data and decode and display the video data.

[0003] However, since the startup time of the master control, multiple input boards, and multiple decoding output boards is not fixed, there is a time difference between the boards, which will cause the problem of asynchronous display when different decoding output boards send and display the same frame of video data. In addition, in the relevant video integrated platform technology, when the network stream and the local stream need to be processed and displayed at the same time, the network stream and the local stream will interfere with each other, which will affect the efficiency of display. Summary of the invention

[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, the present invention provides a video processing method based on a video comprehensive platform, a video comprehensive platform and a device to achieve synchronous transmission and display of video data in the video comprehensive platform and ensure the synchronous display effect of video data.

[0005] The present invention provides a video processing method based on a video integrated platform, wherein the video integrated platform comprises a main control module, an input module and at least one decoding module connected to each other, and the method comprises:

[0006] Determine a first time difference and at least one second time difference through the main control module, and send the first time difference to the input module, and send each second time difference to the corresponding decoding module; the first time difference is the time difference between the input module and the main control module, and the second time difference is the time difference between the decoding module and the main control module;

[0007] Determine the sending timing value of the collected video data according to the first time difference through the input module, and send the video data and the corresponding sending timing value to each of the decoding modules;

[0008] For any decoding module, the decoding module determines the video data to be currently displayed according to the second time difference and the sending timing value of the video data, and sends the video data to be currently displayed for display.

[0009] In some possible implementations, determining the first time difference and at least one second time difference through the main control module includes: obtaining the master control startup duration, the master control clock cycle, the input startup duration, the input clock cycle of the input module, the decoding startup duration and the decoding clock cycle of each decoding module; determining the master control timing value according to the master control startup duration and the master control clock cycle, determining the input timing value according to the input startup duration and the input clock cycle, and determining the first time difference according to the master control timing value and the input timing value; determining the decoding timing value according to the decoding startup duration and the decoding clock cycle, and determining the second time difference according to the master control timing value and the decoding timing value.

[0010] In some possible implementations, determining the sending timing value of the collected video data according to the first time difference through the input module includes: determining the synchronized input timing value according to the first time difference and the input timing value; and determining the sending timing value of the video data according to the synchronized input timing value.

[0011] In some possible implementations, each of the decoding modules includes at least two first cache units, different first cache units are used to cache different frames of video data, and the decoding module determines the current video data to be displayed according to the second time difference and the sending timing value of the video data, including: for any decoding module, determining the synchronized decoding timing value according to the second time difference and the decoding timing value; determining the current video data to be displayed according to the synchronized decoding timing value and the sending timing value, and retrieving the current video data to be displayed from the first cache unit where the current video data to be displayed is located.

[0012] In some possible implementations, determining the video data currently to be displayed based on the synchronized decoding timing value and the sending timing value includes: when the sending timing value satisfies a preset condition, determining the video data corresponding to the sending timing value as the video data currently to be displayed; the preset condition includes that the sending timing value is adjacent to the synchronized decoding timing value, and the sending timing value is less than or equal to the synchronized decoding timing value.

[0013] In some possible implementations, the video data to be currently displayed is determined by the decoding module based on the second time difference and the sending timing value of the video data, including: adjusting the initial phases of each decoding module to be consistent; and determining the video data to be currently displayed by the adjusted decoding module based on the second time difference and the sending timing value of the video data.

[0014] In some possible implementations, each of the decoding modules further includes a second cache unit and a third cache unit, and the method further includes: receiving the video data and the corresponding sending timing value through the second cache unit; and sending the video data and the corresponding sending timing value to the first cache unit for caching through the third cache unit.

[0015] In some possible implementations, the number of the decoding modules is at least two, the video integrated platform further includes a first switch and a second switch, the video data includes network video data and local video data, and the method further includes: receiving the network video data through the first switch, and sending the network video data to any one of the decoding modules, and forwarding the network video data to other decoding modules through the decoding module; sending the local video data obtained by the input module to each of the decoding modules through the second switch.

[0016] The present invention also provides a video integrated platform, comprising: a main control module, an input module and at least one decoding module which are interconnected; the main control module is used to determine a first time difference and at least one second time difference, and send the first time difference to the input module, and send each second time difference to the corresponding decoding module; the first time difference is the time difference of the input module relative to the main control module, and the second time difference is the time difference of the decoding module relative to the main control module; the input module is used to determine the sending timing value of the collected video data according to the first time difference, and send the video data and the corresponding sending timing value to each decoding module; the decoding module is used to determine the video data to be currently displayed according to the second time difference and the sending timing value of the video data, and send the video data to be currently displayed for display.

[0017] In some possible implementations, the number of the decoding modules is at least two, the video integrated platform further includes a first switch and a second switch, the video data includes network video data and local video data; the first switch is used to receive the network video data, and send the network video data to any of the decoding modules, and forward the network video data to other decoding modules through the decoding module; the second switch is used to send the local video data obtained by the input module to each of the decoding modules; the input module includes a video acquisition unit, a first video processing unit and a video sending unit connected in sequence; the decoding module includes a video receiving unit, a second video processing unit, a synchronization control unit and a display unit connected in sequence, and the video receiving unit is connected to the video sending unit; the decoding module also includes a code stream receiving unit, a code stream sending unit and a decoding unit, the code stream receiving unit is connected to the code stream sending unit, the code stream receiving unit is connected to the decoding unit, the decoding unit is connected to the second video processing unit, and the code stream sending unit is connected to the code stream receiving unit of the next decoding module.

[0018] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the video processing method based on the video integration platform as described above is implemented.

[0019] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the video processing method based on the video integration platform as described above is implemented.

[0020] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the video processing method based on any one of the above-mentioned video integration platforms is implemented.

[0021] The video processing method, video comprehensive platform and device based on the video comprehensive platform provided by the present invention determine the second time difference between each decoding module and the main control module by determining the first time difference between the input module and the main control module; the input module then determines the sending timing value of the video data sent by the input module according to the first time difference, and the input module sends the video data and the sending timing value together to each decoding module; each decoding module then synchronizes the processing timing of the decoding module with the main control module through the corresponding second time difference, and determines the current video data to be displayed according to the synchronized processing timing and the sending timing value of the video data, thereby realizing the synchronous processing of the video data by each module in the video comprehensive platform, and then realizing the synchronous sending and display of each decoding module, and ensuring the synchronous display effect of the video data. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A schematic diagram of a scenario of a video processing method based on a video integration platform provided by the present invention;

[0024] Figure 2 One of the flowcharts of the video processing method according to an embodiment of the present invention;

[0025] Figure 3 This is a second flowchart of the video processing method according to an embodiment of the present invention;

[0026] Figure 4 This is a third flowchart of the video processing method according to an embodiment of the present invention;

[0027] Figure 5 A schematic diagram of the structure of an input module and a decoding module according to an embodiment of the present invention;

[0028] Figure 6 This is a fourth flowchart of the video processing method according to an embodiment of the present invention;

[0029] Figure 7 A schematic diagram of a first cache unit of a specific example of the present invention;

[0030] Figure 8a A timing diagram of an initial phase of each decoding module not being adjusted according to a specific example of the present invention;

[0031] Figure 8b A timing diagram after adjusting the initial phase of each decoding module according to a specific example of the present invention;

[0032] Fig. 9 This is a fifth flowchart of the video processing method according to an embodiment of the present invention;

[0033] Fig.10 A timing diagram of a specific example of the present invention;

[0034] Fig.11 A schematic diagram of the structure of a video integration platform according to an embodiment of the present invention;

[0035] Fig.12 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] For ease of understanding, this specification provides a scenario example of a video processing method based on a video integration platform. The scenario example is applied in Figure 1 In the application environment shown, a video integration platform, an IP Camera (IPC) and an external switch are included, and the video integration platform establishes a communication connection with the IP Camera via the external switch.

[0038] In this scenario example, video data includes two types, one is network video data and the other is local video data. The difference is that network video data needs to be obtained from an external device through the Internet, while local video data is video data pre-cached in local memory.

[0039] In this scenario example, the video integrated platform includes a main control module, an input module and a decoding module, and also includes a first switch and a second switch. The main control module can communicate with the input module and the decoding module through the first switch and the second switch.

[0040] Specifically, the video comprehensive platform can have multiple main control modules. When any one of the main control modules fails, the remaining main control modules can back up and display the video data to ensure display stability. In this scenario example, two main control modules are set in the video comprehensive platform, and the two main control modules have the same function and structure.

[0041] In this scenario example, the input module can be an input card, on which an FPGA (Field Programmable Gate Array) chip is provided. The decoding module can be a decoding card, on which a DSP (Digital Signal Processing) chip is provided. A plurality of card slots for inserting cards can be provided in the product corresponding to the video comprehensive platform, and each card slot can be used to insert an input card or a decoding card. When the input card or the decoding card is inserted into the card slot, the input card or the decoding card can establish communication with the main control module. In addition, since the number of input cards and decoding cards inserted into the product corresponding to the video comprehensive platform is not fixed, the number of input modules and the number of decoding modules are also not fixed accordingly. The staff can select the number of input modules and the number of decoding modules according to actual needs.

[0042] In this scenario example, the first switch may be an Ethernet switch, for example, a Gigabit Ethernet Switch (GE SW). The second switch may be a bus switch, for example, a Peripheral Component Interconnect Express Switch (PCIe SW). The first switch and the second switch may be arranged on the same hardware link.

[0043] In this scenario example, the network camera can obtain network video data, and then send the network video data to the video integrated platform through an external switch. The first switch inside the video integrated platform then sends the network video data to the decoding module, and the decoding module decodes the network video data and sends it for display. For details, please refer to Figure 1 The solid arrows indicate the direction of data flow.

[0044] In this scenario example, the input module can obtain local video data from the local device, and then send the local video data to each decoding module through the second switch, and each decoding module decodes and displays the local video data. For details, please refer to Figure 1 The dashed arrows indicate the direction of data flow.

[0045] With reference to the scenario example of the video processing method based on the video integration platform in the aforementioned embodiment, the video processing method based on the video integration platform in an embodiment of the present invention is introduced in detail below.

[0046] Figure 2FIG. 1 is one of the flow charts of the video processing method of an embodiment of the present invention. The video processing method based on the video integrated platform can be applied to the video integrated platform of the above embodiment, and the video integrated platform includes a main control module, an input module and at least one decoding module connected to each other. Figure 2 As shown, the video processing method based on the video integration platform may include the following steps:

[0047] Step 210: Determine a first time difference and at least one second time difference through the main control module, and send the first time difference to the input module, and send each second time difference to the corresponding decoding module; the first time difference is the time difference between the input module and the main control module, and the second time difference is the time difference between the decoding module and the main control module.

[0048] First, it should be noted that this embodiment does not limit the number of main control modules, input modules and decoding modules.

[0049] In this embodiment, the main control module includes a processor CPU. When the main control module, the input module and each decoding module are started, the CPU of the main control module can first obtain the current startup duration of each input module, the startup duration of each decoding module and the startup duration of the main control module itself. The CPU of the main control module then determines the first time difference between the input module and the main control module based on the current startup duration of the main control module and the current startup duration of the input module; the CPU of the main control module determines the second time difference between each decoding module and the main control module based on the current startup duration of the main control module and the current startup duration of each decoding module.

[0050] After obtaining the first time difference and each second time difference, the main control module sends the first time difference to the input module and sends each second time difference to the corresponding decoding module. After receiving the first time difference, the input module can achieve time synchronization with the main control module according to the first time difference. Similarly, after receiving the corresponding second time difference, each decoding module can achieve time synchronization with the main control module according to the second time difference, thereby achieving time synchronization between each decoding module and the input module.

[0051] Step 220: Determine the sending timing value of the collected video data according to the first time difference through the input module, and send the video data and the corresponding sending timing value to each decoding module.

[0052] It should be noted that when the input module processes and sends the collected video data, it processes and sends it according to the internal input timing. For example, the input timing can be the timing of the frame synchronization pulse signal. Usually, the frame synchronization pulse signal is generated every 1 / 30 second (33.3 milliseconds), that is, the input module can send video data every 33.3ms.

[0053] After the input module receives the first time difference sent by the main control module, the input module can synchronize the original input timing according to the first time difference; after synchronization, each time the input module sends a frame of video data to each decoding module, it will send the current synchronized input timing value as the sending timing value to each decoding module.

[0054] Step 230: For any decoding module, the decoding module determines the video data to be currently displayed according to the second time difference and the sending timing value of the video data, and sends the video data to be currently displayed for display.

[0055] It should be noted that when each decoding module processes the video data, it also processes it according to the internal decoding timing.

[0056] After the decoding module receives the second time difference sent by the main control module, the decoding module can synchronize the original decoding timing according to the second time difference; then based on the synchronized decoding timing and the sending timing value of the video data, the decoding module determines the video data to be currently displayed, and sends the video data to be currently displayed to the display screen.

[0057] Figure 3 FIG. 2 is a second flow chart of the video processing method according to an embodiment of the present invention. Figure 3 As shown, in some implementations, determining the first time difference and at least one second time difference by the main control module in step 210 may include the following steps:

[0058] Step 310: Obtain the master control startup duration and master control clock cycle of the master control module, the input startup duration and input clock cycle of the input module, and the decoding startup duration and decoding clock cycle of each decoding module.

[0059] Step 320: Determine a master timing value according to the master startup duration and the master clock cycle, determine an input timing value according to the input startup duration and the input clock cycle, and determine a first time difference according to the master timing value and the input timing value.

[0060] Step 330: Determine a decoding timing value according to the decoding start duration and the decoding clock cycle, and determine a second time difference according to the master timing value and the decoding timing value.

[0061] It should be noted that the input startup duration refers to the startup duration of the input module, and the decoding startup duration refers to the startup duration of the decoding module.

[0062] Specifically, after the main control module, the input module and each decoding module are started, the main control startup duration, the main control clock cycle, the input startup duration, the input clock cycle, the decoding startup duration and the decoding clock cycle of each decoding module can be first obtained through the CPU of the main control module. After obtaining the above parameters, the CPU of the main control module further calculates the first time difference and the second time difference according to the above parameters.

[0063] In this embodiment, the startup duration of each module may be converted into a timing value in units of TICK (time base) according to the startup duration of each module and the clock cycle, thereby converting the startup duration into an integer.

[0064] The following takes a master clock cycle of 1 / 60 second, an input clock cycle of 1 / 30 second, and a decoding clock cycle of 1 / 60 second as an example to describe in detail how the first time difference and the second time difference are calculated.

[0065] It can be understood that if the master control clock cycle is 1 / 60 seconds, it means that the master control module outputs 60 pulses in 1 second. When the master control startup time (in seconds) is obtained, the number of pulses output by the master control module within this time can be determined based on the master control startup time. These pulses are arranged in chronological order to obtain the current master control timing value. Therefore, the master control timing value can be calculated by the following formula:

[0066]

[0067] Wherein, Time Tick is the main control timing value, S is seconds, and NS is nanoseconds. As an example, if the startup time of the current main control module is 90 seconds, then Time Tick = 90*60 = 5400 (Tick), and at this time, the main control timing value of the main control module is 5400 (Tick).

[0068] Similarly, if the input clock cycle is 1 / 30 seconds, it means that the input module outputs 30 pulses in 1 second. When the input startup duration (in seconds) is obtained, the number of pulses output by the input module within this duration can be determined according to the input startup duration. These pulses are arranged in chronological order to obtain the current input timing value. In this embodiment, the input timing value is recorded as Tick_IN. As an example, if the input startup duration of the current input module is 60 seconds, the input timing value Tick_IN = 60*30 = 1800 (Tick).

[0069] After obtaining the master timing value and the input timing value, since the master clock cycle is 1 / 60 second and the input clock cycle is 1 / 30 second, it is also necessary to unify the clock cycles of the master timing value and the input timing value, and then calculate the first time difference. Specifically, the first time difference can be calculated by the following formula:

[0070] Delta Tick_IN=Time Tick-Tick_IN*2

[0071] Delta Tick_IN is the first time difference. As an example, if the current startup time of the main control module is 90 seconds and the current input startup time of the input module is 60 seconds, then Delta Tick_IN = (90*60) - (60*30*2) = 1800 (Ticks).

[0072] It is worth mentioning that the main control module can directly obtain the frame synchronization pulse signal count inside the input module as the input timing value, and further calculate the first time difference, because the input module uses the timing of the frame synchronization pulse signal as the input timing to process and send the video data.

[0073] Similarly, if the decoding clock cycle is 1 / 60 seconds, it means that the input module outputs 60 pulses in 1 second. When the decoding startup duration (in seconds) is obtained, the number of pulses output by the decoding module within this duration can be determined according to the decoding startup duration. These pulses are arranged in chronological order to obtain the current decoding timing value. In this embodiment, the decoding timing value is recorded as Tick_OUT. As an example, if the input startup duration of the current decoding module is 50 seconds, the input timing value Tick_OUT = 50*60 = 3000 (Tick).

[0074] After obtaining the master timing value and the decoding timing value, the second time difference can be calculated by the following formula:

[0075] Delta Tick_OUT=Time Tick-Tick_OUT

[0076] Delta Tick_OUT is the second time difference. As an example, if the current startup duration of the main control module is 90 seconds and the current input startup duration of the decoding module is 50 seconds, then Delta Tick_OUT = (90*60) - (50*60) = 2400 (Ticks).

[0077] It is worth mentioning that the main control module can directly obtain the Vblank (pulse voltage) interrupt count inside the decoding module as the decoding timing value, and further calculate the second time difference. A Vblank interruption means that the decoding module has completed a display transmission, and it also means that the next frame of video data can be displayed. Therefore, there is no need to calculate the decoding timing value here, and the Vblank interrupt count is directly used to represent the decoding timing value.

[0078] Figure 4FIG3 is a flowchart of a video processing method according to an embodiment of the present invention. Figure 4 As shown, in some implementations, determining the sending timing value of the collected video data according to the first time difference by the input module in step 220 may include the following steps:

[0079] Step 410: Determine a synchronized input timing value according to the first time difference and the input timing value.

[0080] Step 420: Determine the sending timing value of the video data according to the synchronized input timing value.

[0081] In order to more clearly explain the process of the input module sending video data and sending timing values ​​to the decoding module, the specific structure of the input module is first introduced below. Figure 5 FIG. 1 is a schematic diagram of the structure of the input module and the decoding module of an embodiment of the present invention. Figure 5 As shown, the input module of this embodiment includes a video acquisition unit, a first video processing unit and a video sending unit which are connected in sequence.

[0082] The video acquisition unit is used to acquire video data and send the video data to the first video processing unit. The first video processing unit will perform pre-processing such as cropping on the video data to obtain YUV data (color-coded data). The first video processing unit will send the obtained YUV data to the video sending unit, and finally the video sending unit will send the YUV data to each decoding module.

[0083] After the main control module calculates the first time difference, the main control module sends the first time difference to the input module, and the video sending unit of the input module records the first time difference Delta Tick_IN. Every time the input module sends a frame of video data to the decoding module, the input timing value of the input module after current synchronization is sent as the sending timing value to each decoding module. As an example, if the main control clock cycle is 1 / 60 seconds and the input clock cycle is 1 / 30 seconds, the synchronized input timing value can be calculated by the following formula:

[0084] Synchronization Tick_IN=Tick_IN*2+Delta Tick_IN

[0085] Among them, Synchronization Tick_IN is the input timing value after synchronization.

[0086] Specifically, the order in which the video sending unit of the input module sends data packets to the decoding module is: frame head packet, video data and frame tail packet, among which the frame tail packet will carry the timestamp when the input module sends the video data. In this embodiment, the timestamp carried in the frame tail packet is the input timing value after the current synchronization (i.e., the sending timing value).

[0087] Considering that the process of the input module processing and sending video data requires a certain amount of time, in some embodiments, a redundancy value can be added on the basis of the current synchronized input timing value to obtain the sending timing value. The size of the redundancy value can be manually set by the staff according to actual needs, and the minimum redundancy value is 1.

[0088] As an example, if the master clock cycle is 1 / 60 seconds, the input clock cycle is 1 / 30 seconds, the current master module startup duration is 90 seconds, and the current input module input startup duration is 60 seconds, the first time difference can be calculated to be 1800 (Ticks), the current input timing value is 1800 (Ticks), and further, the synchronized input timing value Synchronization Tick_IN = 1800*2 + 1800 = 5400 (Ticks) can be calculated. If the redundancy value is set to 1, the sending timing value of the video sending unit of the input module in the frame tail packet is 5401 (Ticks).

[0089] Continue to refer Figure 5 In some embodiments, the decoding module may include a video receiving unit, a second video processing unit, a synchronization control unit, and a display unit connected in sequence. The video receiving unit is connected to the video sending unit of the input module, and the video receiving unit is used to receive the video data and the corresponding sending timing value sent by the video sending unit of the input module. Furthermore, the video receiving unit sends the video data and the corresponding sending timing value to the second video processing unit.

[0090] After the second video processing unit performs processing such as cropping on a frame of video data, the frame of video data is sent to the synchronization control unit. In some embodiments, each decoding module includes at least two first cache units, and different first cache units are used to cache different frames of video data. In this embodiment, each first cache unit is arranged in the synchronization control unit.

[0091] Figure 6 FIG4 is a flowchart of a video processing method according to an embodiment of the present invention. Figure 6 As shown, in some implementations, determining the current video data to be displayed by the decoding module according to the second time difference and the sending timing value of the video data in step 230 may include the following steps:

[0092] Step 610: For any decoding module, determine a synchronized decoding timing value according to the second time difference and the decoding timing value.

[0093] Step 620: Determine the video data to be currently displayed according to the synchronized decoding timing value and the sending timing value, and retrieve the video data to be currently displayed from the first buffer unit where the video data to be currently displayed is located.

[0094] When the decoding module receives the second time difference sent by the main control module, the decoding module can record the second time difference Delta Tick_OUT, and then determine the current synchronized decoding timing value based on the second time difference and the decoding timing value. Taking the main control clock cycle of 1 / 60 seconds and the decoding clock cycle of 1 / 60 seconds as an example, the synchronized decoding timing value can be calculated by the following formula:

[0095] Synchronization Tick_OUT=Tick_OUT+Delta Tick_OUT

[0096] Among them, Synchronization Tick_OUT is the decoding timing value after synchronization.

[0097] As an example, if the master clock cycle is 1 / 60 second, the decoding clock cycle is 1 / 60 second, the current master control module startup time is 90 seconds, and the current input module input startup time is 50 seconds, the first time difference can be calculated to be 2400 (Ticks), and the current decoding timing value is 3000 (Ticks). Further, the synchronized decoding timing value of 5400 (Ticks) can be calculated.

[0098] After each frame of video data is received by the video receiving unit of the decoding module, the video data and the corresponding sending timing value are sent to the second video processing unit for processing; after the second video processing unit completes the processing, the video data and the corresponding sending timing value are sent to the synchronization control unit, and the synchronization control unit stores the processed video data with the sending timing value in a first cache unit.

[0099] Similarly, considering that it takes time for the decoding module to process the video data, in some implementations, a redundancy value may be added to the synchronized decoding timing value, and the minimum redundancy value is 1.

[0100] When the display sending unit of the decoding module is sending for display, the display sending unit determines the decoding timing value after the current synchronization, and finds the qualified video data in each first cache unit according to the decoding timing value after the current synchronization, and uses the qualified video data as the video data to be currently displayed. At this time, the display sending unit retrieves the video data to be currently displayed from the first cache unit where the video data to be currently displayed is located for display.

[0101] In some embodiments, determining the video data to be currently displayed based on the synchronized decoding timing value and the sending timing value in step 620 may include: when the sending timing value satisfies a preset condition, determining the video data corresponding to the sending timing value as the video data to be currently displayed; the preset condition includes that the sending timing value is adjacent to the synchronized decoding timing value, and the sending timing value is less than or equal to the synchronized decoding timing value.

[0102] Specifically, the display unit can find video data adjacent to and less than or equal to the current synchronized decoding timing value in each first cache unit according to the current synchronized decoding timing value, and the video data is the video data to be displayed.

[0103] As an example, Figure 7 As shown, if the number of first cache units is 6, the sending timing values ​​corresponding to the video data stored in these 6 first cache units are: 5400 (Tick), 5402 (Tick), 5404 (Tick), 5406 (Tick), 5408 (Tick) and 5410 (Tick). If the display unit determines that the current synchronized decoding timing value is 5405 (Tick), the video data corresponding to the sending timing value of 5404 (Tick) can be used as the video data to be displayed.

[0104] In some embodiments, the video receiving unit of each decoding module includes a second cache unit and a third cache unit. The video processing method based on the video integrated platform may also include: receiving video data and corresponding sending timing values ​​through the second cache unit; sending the video data and corresponding sending timing values ​​to the first cache unit for caching through the third cache unit.

[0105] Specifically, the video receiving unit is internally provided with a second cache unit and a third cache unit. When the video sending unit of the input module sends a frame of video data to the decoding module, the video receiving unit of the decoding module will store the frame of video data in the second cache unit, and then take out the previous frame of video data from the third cache unit and send it to the second video processing unit. Using two caches for data reception and data transmission can effectively solve the tearing problem caused by one cache and the problem of high latency due to multiple caches.

[0106] In some embodiments, in step 230, the video data to be currently displayed is determined by the decoding module according to the second time difference and the sending timing value of the video data, and it can also include: adjusting the initial phases of each decoding module to be consistent; and determining the video data to be currently displayed by the adjusted decoding module according to the second time difference and the sending timing value of the video data.

[0107] Since the initial phases of the original decoding processing timings may be inconsistent after each decoding module is started, in this embodiment, after each decoding module is started, the initial phases of the original decoding processing timings of each decoding module may be adjusted first to keep the initial phases of the original decoding processing timings of each decoding module consistent, thereby further ensuring the synchronization of display of different decoding modules. Figure 8a A timing diagram showing the initial phase of each decoding module is not adjusted; Figure 8b The timing diagram after adjusting the initial phase of each decoding module is shown.

[0108] refer to Figure 1 In some implementations, the number of decoding modules is at least two, the video integrated platform further includes a first switch and a second switch, and the video data includes network video data and local video data. Figure 5 As shown, the decoding module further includes a code stream receiving unit, a code stream forwarding unit and a decoding unit, wherein the code stream receiving unit is connected to the code stream forwarding unit and the decoding unit respectively, and the decoding unit is connected to the second video processing unit.

[0109] In this embodiment, the first switch may be a GE SW, the second switch may be a PCIe SW, and the first switch and the second switch may be arranged on the same hardware link. The main control module may realize data interaction with each decoding module and the input module through the first switch or the second switch, and similarly, the input module may also realize data interaction with each decoding module through the second switch.

[0110] Fig. 9 FIG5 is a fifth flow chart of the video processing method according to an embodiment of the present invention. Fig. 9 As shown, in some embodiments, the video data includes network video data and local video data. When the network video data and the local video data need to be processed simultaneously and decoded and displayed via multiple decoding modules, the video processing method based on the video integrated platform may further include the following steps:

[0111] Step 910: Receive network video data through the first switch, send the network video data to any decoding module, and forward the network video data to other decoding modules through the decoding module.

[0112] Step 920: Send the local video data acquired by the input module to each decoding module through the second switch.

[0113] Specifically, the first switch can receive network video data sent by the external network camera IPC. After receiving the network video data, the first switch will send the network video data to any decoding module; the code stream receiving unit of the decoding module is used to receive the network video data. After receiving the network video data, the code stream receiving unit will send the network video data to the code stream sending unit; the code stream sending unit directly forwards the network video data to the code stream receiving units of other decoding modules.

[0114] The second switch is used to receive the local video data sent by the video sending unit in the input module, and send the local video data to the video receiving units in each decoding module.

[0115] It should be noted that when the existing video integrated platform needs to process network video data and local video data at the same time and decode and display them through multiple decoding modules, the processing process is as follows: the first switch receives the network video data and sends the network video data to the decoding module; after the decoding module receives the network video data, it performs scaling, cropping and other processing to obtain YNV data; after obtaining the YNV data, due to the large amount of YNV data, the decoding module needs to send the YNV data to other decoding modules through the second switch, so the YNV data sent by the decoding module will occupy the bandwidth of the second switch. At the same time, the second switch also needs to send the local video data to each decoding module, so the local video data and the network video data will interfere with each other, thereby affecting the display of the video data.

[0116] Therefore, in this embodiment, after receiving the network video data, the code stream receiving unit directly sends the network video data to the code stream sending unit; the code stream sending unit then forwards the network video data to the code stream receiving units of other decoding modules through an internal protocol, so that the network video data does not occupy the bandwidth of the second switch, thereby realizing the separate transmission of local video data and network video data, reducing the mutual interference between local video data and network video data, and can effectively improve the transmission efficiency of video data.

[0117] The synchronous display process of the same network video data for multiple decoding modules is as follows: after each decoding module receives the second time difference sent by the main control module, the code stream receiving unit of each decoding module can record the second time difference, and determine the synchronized decoding timing value according to the second time difference and the decoding timing value; after each code stream receiving unit receives a frame of network video data, it will record the current synchronized decoding timing value in the private header of the network video data. When the network video data is displayed, each display unit determines the network video data to be displayed according to the current synchronized decoding timing value.

[0118] For ease of understanding, the video processing method based on the video integrated platform of an embodiment of the present invention is further introduced by taking the case where one channel of local video data is synchronously transmitted and displayed by two decoding modules (decoding module 1 and decoding module 2) as an example.

[0119] First, the acquisition frame rate of the input module is 60 Hz, that is, one frame of local video data is acquired every 16.6 ms (1 / 60 s), and the input clock cycle of the input module is configured to be 1 / 30 s, that is, one frame of local video data is processed or sent every 33.3 ms. In addition, it is assumed that the current input startup time of the input module is 60 s.

[0120] Secondly, the decoding clock cycle of decoding module 1 and decoding module 2 is configured to be 1 / 60 seconds, that is, one frame of local video data is output and processed every 16.6ms, but the startup duration of decoding module 1 and decoding module 2 is different. In this example, the VBlank interrupt count can be directly obtained as the decoding timing value. In addition, it is assumed that the current input startup duration of decoding module 1 is 50s, and the current input startup duration of decoding module 2 is 40s.

[0121] Finally, the master control clock period of the master control module is configured to be 1 / 60 seconds, assuming that the current master control startup time of the master control module is 90 seconds. Fig.10 For a timing diagram of a specific example of the present invention, the acquisition timing, processing timing of the input module and the processing timing of the decoding module can be referred to Fig.10 The timing in .

[0122] Furthermore, the main control module determines that the main control timing value is 5400 (Ticks) based on the main control startup duration of 90s and the main control clock cycle of 1 / 60; determines that the input timing value is 1800 (Ticks) based on the input startup duration of 60s and the input clock cycle of 1 / 30. In this example, the input timing value does not need to be calculated, and the frame synchronization pulse count is directly obtained as the input timing value; according to the decoding startup duration of 50s and the decoding clock cycle of 1 / 60 of decoding module 1, the decoding timing value of decoding module 1 is determined to be 3000 (Ticks); according to the decoding startup duration of 40s and the decoding clock cycle of 1 / 60 of decoding module 2, the decoding timing value of decoding module 2 is determined to be 2400 (Ticks). In this example, the decoding timing value does not need to be calculated, and the VBlank interrupt count of each decoding module is directly obtained as the decoding timing value.

[0123] Furthermore, the main control module determines the first time difference according to the main control timing value and the frame synchronization pulse count; and determines the second time difference according to the main control timing value and the VBlank interrupt count. The main control module then determines the synchronized input timing value according to the first time difference and the frame synchronization pulse count; and determines the synchronized decoding timing value according to the second time difference and the VBlank interrupt count. Specifically, the correspondence between the synchronized input timing value and the frame synchronization pulse count, and the correspondence between the synchronized decoding timing value and the VBlank interrupt count can be referred to. Fig.10 The corresponding relationship in .

[0124] like Fig.10 As shown in the figure, when the input module starts to collect local video data from the 9th frame, the input module triggers the frame synchronization pulse signal at the same time to process the local video data. When the frame synchronization pulse count is 1802 (Tick), the input module sends the 11th frame of local video data and converts the frame synchronization pulse count of 1802 (Tick) into the synchronized input timing value:

[0125] Synchronization Tick_IN=1802*2+1800=5404(Tick)

[0126] After obtaining the synchronized input timing value 5404 (Tick), a redundant value 1 is added to the synchronized input timing value 5404 (Tick), and the sending timing value corresponding to the 11th frame of local video data is obtained as 5405 (Tick). Finally, the input module sends the 11th frame of local video data and the sending timing value 5405 (Tick) to decoding module 1 and decoding module 2 respectively through the video sending unit.

[0127] Considering that it takes a certain amount of time for the decoding module to process video data, a redundancy value of 1 is also required in the timing of the decoding module sending and displaying local video data. Fig.10 , decoding module 1 receives the 11th frame of local video data and sends the timing value 5405 (Tick). When the decoding timing value after the decoding module 1 is synchronized reaches 5406 (Tick), the decoding module 1 can send and display the 11th local video data. Similarly, decoding module 2 receives the 11th frame of local video data and sends the timing value 5405 (Tick). When the decoding timing value after the decoding module 2 is synchronized reaches 5406 (Tick), the decoding module 2 can send and display the 11th local video data.

[0128] Therefore, the decoding module 1 and the decoding module 2 send and display the same frame of local video data at the same time, realizing the synchronous sending and display of each decoding module and ensuring the synchronous display effect of the video data.

[0129] Fig.11 FIG. 1 is a schematic diagram of the structure of a video integrated platform according to an embodiment of the present invention. Fig.11 As shown, the video integrated platform 1100 includes a main control module 1110 , an input module 1120 and at least one decoding module 1130 which are interconnected.

[0130] The main control module 1110 is used to determine a first time difference and at least one second time difference, and send the first time difference to the input module 1120, and send each second time difference to the corresponding decoding module 1130; the first time difference is the time difference between the input module 1120 and the main control module 1110, and the second time difference is the time difference between the decoding module 1130 and the main control module 1110.

[0131] The input module 1120 is used to determine a sending timing value of the collected video data according to the first time difference, and send the video data and the corresponding sending timing value to each decoding module 1130 .

[0132] The decoding module 1130 is used to determine the video data to be currently displayed according to the second time difference and the sending timing value of the video data, and send the video data to be currently displayed for display.

[0133] Therefore, by determining the first time difference between the input module 1120 and the main control module 1110, the second time difference between each decoding module 1130 and the main control module 1110 is determined; the input module 1120 then determines the sending timing value of the video data according to the first time difference, and the input module 1120 sends the video data and the sending timing value together to each decoding module 1130; each decoding module 1130 then synchronizes the processing timing of the decoding module 1130 with the main control module 110 through the corresponding second time difference, and determines the current video data to be displayed according to the synchronized processing timing and the sending timing value of the video data, thereby realizing the synchronous processing of the video data by each module in the video integrated platform 1100, and then realizing the synchronous transmission and display of each decoding module 1130, ensuring the synchronous display effect of the video data.

[0134] In some embodiments, reference Figure 5 and Fig.11 The number of decoding modules is at least two, the video integrated platform also includes a first switch and a second switch, the video data includes network video data and local video data; the first switch is used to receive network video data, and send the network video data to any decoding module, and forward the network video data to other decoding modules through the decoding module; the second switch is used to send the local video data obtained by the input module to each decoding module; the input module includes a video acquisition unit, a first video processing unit and a video sending unit connected in sequence; the decoding module includes a video receiving unit, a second video processing unit, a synchronization control unit and a display unit connected in sequence, and the video receiving unit is connected to the video sending unit; the decoding module also includes a code stream receiving unit, a code stream sending unit and a decoding unit, the code stream receiving unit is connected to the code stream sending unit, the code stream receiving unit is connected to the decoding unit, the decoding unit is connected to the second video processing unit, and the code stream sending unit is connected to the code stream receiving unit of the next decoding module.

[0135] It should be noted that for details not disclosed in the video comprehensive platform of this embodiment, please refer to the details disclosed in the embodiment of the video processing method based on the video comprehensive platform in the embodiments of this specification, and no further details will be given here.

[0136] Fig.12 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.12As shown, the electronic device may include: a processor 1210, a communication interface 1220, a memory 1230 and a communication bus 1240, wherein the processor 1210, the communication interface 1220 and the memory 1230 complete mutual communication through the communication bus 1240. The processor 1210 may call the logic instructions in the memory 1230 to execute the video processing method of the video integrated platform, the method comprising determining a first time difference and at least one second time difference through the main control module, sending the first time difference to the input module, and sending each second time difference to the corresponding decoding module; the first time difference is the time difference of the input module relative to the main control module, and the second time difference is the time difference of the decoding module relative to the main control module; determining the sending timing value of the collected video data according to the first time difference through the input module, and sending the video data and the corresponding sending timing value to each decoding module; for any decoding module, determining the current video data to be displayed through the decoding module according to the second time difference and the sending timing value of the video data, and sending the current video data to be displayed.

[0137] In addition, the logic instructions in the above-mentioned memory 1230 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0138] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the video processing method of the video integrated platform provided by the above methods. The method includes determining a first time difference and at least one second time difference through a main control module, and sending the first time difference to an input module, and sending each second time difference to a corresponding decoding module; the first time difference is the time difference of the input module relative to the main control module, and the second time difference is the time difference of the decoding module relative to the main control module; determining the sending timing value of the collected video data according to the first time difference through the input module, and sending the video data and the corresponding sending timing value to each decoding module; for any decoding module, determining the current video data to be displayed through the decoding module according to the second time difference and the sending timing value of the video data, and sending the current video data to be displayed for display.

[0139] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the video processing method of the video integrated platform provided by the above-mentioned methods. The method includes determining a first time difference and at least one second time difference through a main control module, and sending the first time difference to an input module, and sending each second time difference to a corresponding decoding module; the first time difference is the time difference of the input module relative to the main control module, and the second time difference is the time difference of the decoding module relative to the main control module; determining the sending timing value of the collected video data according to the first time difference through the input module, and sending the video data and the corresponding sending timing value to each decoding module; for any decoding module, determining the current video data to be displayed through the decoding module according to the second time difference and the sending timing value of the video data, and sending the current video data to be displayed for display.

[0140] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0141] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video processing method based on a video integration platform, characterized in that: The video integrated platform includes a main control module, an input module and at least one decoding module connected to each other, and the method includes: Determine a first time difference and at least one second time difference through the main control module, and send the first time difference to the input module, and send each second time difference to a corresponding decoding module; the first time difference is the time difference between the input module and the main control module, and the second time difference is the time difference between the decoding module and the main control module; Determine the sending timing value of the collected video data according to the first time difference through the input module, and send the video data and the corresponding sending timing value to each of the decoding modules; For any decoding module, the decoding module determines the video data to be currently displayed according to the second time difference and the sending timing value of the video data, and sends the video data to be currently displayed for display.

2. The video processing method based on the video integration platform according to claim 1, characterized in that: The determining the first time difference and at least one second time difference by the main control module includes: Obtaining the master control startup duration and the master control clock cycle of the master control module, the input startup duration and the input clock cycle of the input module, and the decoding startup duration and the decoding clock cycle of each decoding module; Determine a master timing value according to the master startup duration and the master clock cycle, determine an input timing value according to the input startup duration and the input clock cycle, and determine the first time difference according to the master timing value and the input timing value; A decoding timing value is determined according to the decoding start duration and the decoding clock cycle, and the second time difference is determined according to the master control timing value and the decoding timing value.

3. The video processing method based on the video integration platform according to claim 2 is characterized in that: The step of determining the sending timing value of the collected video data according to the first time difference through the input module includes: Determine a synchronized input timing value according to the first time difference and the input timing value; The sending timing value of the video data is determined according to the synchronized input timing value.

4. The video processing method based on the video integration platform according to claim 2, characterized in that: Each of the decoding modules includes at least two first cache units, and different first cache units are used to cache different frames of video data. The determining of the current video data to be displayed by the decoding module according to the second time difference and the sending timing value of the video data includes: For any decoding module, determining a synchronized decoding timing value according to the second time difference and the decoding timing value; The video data to be currently displayed is determined according to the synchronized decoding timing value and the sending timing value, and the video data to be currently displayed is retrieved from the first cache unit where the video data to be currently displayed is located.

5. The video processing method based on the video integration platform according to claim 4 is characterized in that: The step of determining the video data to be currently displayed according to the synchronized decoding timing value and the sending timing value comprises: When the sending timing value satisfies a preset condition, the video data corresponding to the sending timing value is determined as the video data currently to be displayed; the preset condition includes that the sending timing value is adjacent to the synchronized decoding timing value, and the sending timing value is less than or equal to the synchronized decoding timing value.

6. The video processing method based on the video integration platform according to any one of claims 1 to 5, characterized in that: The step of determining the video data to be currently displayed according to the second time difference and the sending timing value of the video data by the decoding module includes: Adjusting the initial phases of the decoding modules to be consistent; The video data to be currently displayed is determined by the adjusted decoding module according to the second time difference and the sending timing value of the video data.

7. The video processing method based on the video integration platform according to claim 4 is characterized in that: Each of the decoding modules further includes a second cache unit and a third cache unit, and the method further includes: Receiving the video data and the corresponding sending timing value through the second buffer unit; The video data and the corresponding sending timing value are sent to the first cache unit for cache through the third cache unit.

8. The video processing method based on the video integration platform according to any one of claims 1 to 5, characterized in that: The number of the decoding modules is at least two, the video integrated platform further includes a first switch and a second switch, the video data includes network video data and local video data, and the method further includes: receiving the network video data through the first switch, and sending the network video data to any one of the decoding modules, and forwarding the network video data to other decoding modules through the decoding module; The local video data acquired by the input module is sent to each of the decoding modules through the second switch.

9. A video integrated platform, characterized in that: It includes a main control module, an input module and at least one decoding module which are connected to each other; The main control module is used to determine a first time difference and at least one second time difference, and send the first time difference to the input module, and send each second time difference to a corresponding decoding module; the first time difference is the time difference between the input module and the main control module, and the second time difference is the time difference between the decoding module and the main control module; The input module is used to determine the sending timing value of the collected video data according to the first time difference, and send the video data and the corresponding sending timing value to each of the decoding modules; The decoding module is used to determine the video data to be currently displayed according to the second time difference and the sending timing value of the video data, and send the video data to be currently displayed for display.

10. The video integrated platform according to claim 9, characterized in that: The number of the decoding modules is at least two, the video integrated platform further comprises a first switch and a second switch, and the video data comprises network video data and local video data; The first switch is used to receive the network video data, send the network video data to any one of the decoding modules, and forward the network video data to other decoding modules through the decoding module; The second switch is used to send the local video data acquired by the input module to each of the decoding modules; The input module includes a video acquisition unit, a first video processing unit and a video sending unit connected in sequence; The decoding module comprises a video receiving unit, a second video processing unit, a synchronization control unit and a display sending unit connected in sequence, and the video receiving unit is connected to the video sending unit; The decoding module also includes a code stream receiving unit, a code stream sending unit and a decoding unit, the code stream receiving unit is connected to the code stream sending unit, the code stream receiving unit is connected to the decoding unit, the decoding unit is connected to the second video processing unit, and the code stream sending unit is connected to the code stream receiving unit of the next decoding module.

Images