Method, apparatus and device for processing multi-channel video data

Abstract

The present application relates to the technical field of computer, disclose a kind of multi-channel video data processing method, device and equipment, comprising: obtaining after being preprocessed multi-channel video data;For each channel video data, split with group of pictures (GOP) as unit, obtain multiple GOP packages;After erasure coding processing to multiple GOP packages, according to the generation time of each GOP package in multi-channel, determine the difference between GOP packages in different channels;Based on the maximum allocation principle of GOP package difference, select a target GOP package from each channel respectively and store in the same hard disk.By this way, even in the failure of one or more hard disk, the probability of losing the same GOP package is greatly reduced, the difficulty of data recovery is greatly reduced, and the repair accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and specifically to a method, apparatus, and device for processing multi-channel video data. Background Technology

[0002] Traditional erasure striping storage uses a random write mode, which has the advantage of high write bandwidth. However, in video storage systems, when faced with the problem of data loss beyond the fault domain, the traditional random write mode is very likely to cause the Group of Pictures (GOP) packets on the hard drive to be too correlated. Once the hard drive fails, several consecutive GOP packets may be completely lost, which greatly increases the difficulty of data recovery, reduces the accuracy of repair, and may even make it impossible to guarantee the ability to repair video data. Summary of the Invention

[0003] In view of this, the present invention provides a multi-channel video data processing method, apparatus and device to solve the problem that the traditional method of writing GOP packets to hard disks leads to excessive correlation between GOP packets within the same hard disk.

[0004] In a first aspect, the present invention provides a multi-channel video data processing method, the method comprising:

[0005] Acquire pre-processed multi-channel video data, which is data generated by acquiring data through different channels and then pre-processing it.

[0006] For each channel of video data, it is split into multiple GOP packets based on the Group of Pictures (GOP).

[0007] After erasure coding is performed on multiple GOP packets, the degree of difference between GOP packets in different channels is determined based on the generation time of each GOP packet in the multi-channel system.

[0008] Based on the principle of maximizing the time difference of GOP packets, a target GOP packet is selected from each channel and stored on the same hard drive, where the time difference between target GOP packets in different channels is maximized.

[0009] The multi-channel video data processing method provided by this invention has the following advantages:

[0010] After acquiring the preprocessed multi-channel video data, the video data for each channel is split into Groups of Pictures (GOPs), resulting in multiple GOP packets. Then, erasure coding is performed on these GOP packets. Based on the generation time of each GOP packet in the multi-channel dataset, the difference between GOP packets in different channels is determined. Following the principle of maximizing the allocation of GOP packets based on their difference, a target GOP packet is selected from each channel and stored on the same hard drive. This method significantly reduces the probability of losing the same GOP packet even in the event of one or more hard drive failures, greatly simplifying data recovery and improving repair accuracy.

[0011] In one optional implementation, after erasure coding of multiple GOP packets, the degree of difference between GOP packets in different channels is determined based on the generation time of each GOP packet in the multi-channel system, including:

[0012] Taking the first GOP packet in the first channel of the multi-channel as the benchmark, in the current iteration cycle, each GOP packet is selected from the other channels in the multi-channel except the first channel. Based on the generation time of all the traversed GOP packets and the generation time of the first GOP packet, the difference between different GOP packets in different channels is generated. Here, the first channel is any channel in the multi-channel, and the first GOP packet is any GOP packet in the first channel.

[0013] Specifically, since video images are composed of individual frames, the division of GOP packets follows a specific temporal order. Therefore, determining the differences between GOP packets based on their generation times is a more reasonable and accurate approach.

[0014] In one optional implementation, based on the principle of maximizing GOP dissimilarity allocation, one target GOP packet is selected from each channel and stored on the same hard drive, including:

[0015] After traversing the first GOP packet in the first channel as the reference and obtaining multiple differences, select the first GOP packet group corresponding to the largest difference. The first GOP packet group includes the first GOP packet and a target GOP packet in each channel other than the first channel in the multi-channel.

[0016] All GOP packets in the first GOP packet group are stored on the same hard disk. GOP packets from different groups are stored on different hard disks. The number of hard disks is the sum of the number of GOP packets in any channel and the number of erasure codes in that channel.

[0017] Specifically, after traversing the first GOP packet in the first channel as a reference, multiple differences are obtained. Then, GOP packets from channels other than the first channel that generate the maximum difference with the first GOP packet are selected to form the target GOP packet group. These target GOP packets are stored on the same hard drive. Therefore, even if the hard drive subsequently fails and the GOP packets on that hard drive are lost, there will be little impact. The GOP packets on that hard drive can be recovered using GOP packets generated at the same time on other hard drives, as well as other GOP packets, with higher accuracy.

[0018] In an optional implementation, when a data failure domain loss occurs and the second GOP packet in the first channel of a multi-channel system is lost, while GOP packets generated at the same time as the second GOP packet in other channels besides the first channel are not lost, the method further includes:

[0019] I-frame background information is extracted from a preset number of GOP packets in the first channel that are not lost except for the second GOP packet and have the highest correlation with the second GOP packet, forming an I-frame background information set. The correlation is determined according to the generation time of the GOP packet. The smaller the difference between the generation time of the GOP packet in the first channel and the generation time of the second GOP packet, the higher the correlation.

[0020] Select GOP packets generated at the same time as the second GOP packet from the channels other than the first channel in the multi-channel array to form the second GOP packet group, and extract the I-frame information and P-frame information corresponding to each GOP packet in the second GOP packet group.

[0021] And, from each of the channels other than the first channel, select a preset number of GOP packets that are not lost and have the highest relevance to the second GOP packet group, excluding the second GOP packet group, to form the third GOP packet group;

[0022] Extract the I-frame and P-frame information from each GOP packet in the second GOP packet group;

[0023] Based on the I-frame background information set, the I-frame and P-frame information corresponding to each GOP packet in the second GOP packet group, and the I-frame and P-frame information in each GOP packet in the third GOP packet group, the second GOP packet is recovered.

[0024] Specifically, when a data failure occurs and the second GOP packet in the first channel is lost, while GOP packets in other channels at the same time are not lost, I-frame background information can be extracted from a preset number of GOP packets in the first channel that are not lost and have the highest correlation with the second GOP packet, forming an I-frame background information set. This set is used to generate the I-frame background information for the second GOP packet. Then, from the channels other than the first channel, GOP packets generated at the same time as the second GOP packet are selected to form a second GOP packet group, and the I-frame and P-frame information corresponding to each GOP packet in the second GOP packet group are extracted. Furthermore, from the channels other than the first channel, a preset number of GOP packets that are not lost and have the highest correlation with the second GOP packet, excluding the second GOP packet group, are selected to form a third GOP packet group, and the I-frame and P-frame information of each GOP packet in the second GOP packet group are extracted. By fusing this information, the second GOP packet can be recovered. This is equivalent to combining data information collected from different angles (different channels) at the same time, as well as data information from adjacent time points, to finally complete the recovery of the second GOP packet. The accuracy of the recovery will be relatively higher.

[0025] In one optional implementation, the second GOP packet is recovered based on the I-frame background information set, the I-frame information and P-frame information corresponding to each GOP packet in the second GOP packet group, and the I-frame information and P-frame information in each GOP packet of the third GOP packet group, including:

[0026] Based on the I-frame information of each GOP packet in the second GOP packet group, the foreground information of the second GOP packet is recovered.

[0027] Based on the P-frame information of each GOP packet in the second GOP packet group and the P-frame information of each GOP packet in the third GOP packet group, determine the P-frame differential information.

[0028] The second GOP packet is recovered based on foreground information, I-frame background information set, and P-frame differential information.

[0029] Specifically, based on the I-frame information of each GOP packet in the second GOP packet group, the foreground information of the second GOP packet can be recovered. Based on the P-frame information of each GOP packet in the second GOP packet group, the P-frame difference information between different GOP packets at the same time, as well as GOP packets at adjacent time points, and the second GOP packet can be determined. Finally, by combining the foreground information, the set of I-frame background information, and the P-frame difference information, the second GOP packet can be recovered.

[0030] In one optional implementation, when there are 3 channels, taking the first GOP packet in the first channel of the multi-channel as the benchmark, in the current iteration cycle, each GOP packet is selected from all channels except the first channel. Based on the generation time of all the traversed GOP packets and the generation time of the first GOP packet, the difference between different GOP packets in different channels is generated, expressed by the following formula:

[0031]

[0032] Where d represents the difference between different GOP packets in different channels, and t (1,i) Let t be the generation time of the i-th GOP in the first channel. (2,j) Let t be the generation time of the j-th GOP in the second channel. (3,k) Let i be the generation time of the k-th GOP in the third channel, and let i, j, and k be the identifiers of any GOP packets in the three channels, where i, j, and k are all positive integers.

[0033] In one alternative implementation, the method further includes:

[0034] Based on the principle of maximizing the difference of GOP packets, after selecting a target GOP packet from each channel, the target GOP packets belonging to the same group are compared with the GOP packets already stored in the quasi-storage hard disk. The target GOP packets in the same group are the GOP packets to be stored on the same hard disk selected based on the rule of maximizing the difference of GOP packets.

[0035] When it is determined that the relevance between the target GOP packets in the same group and the GOP packets already stored in the quasi-storage hard disk is less than or equal to a preset relevance threshold, the target GOP packets in the same group are stored in the quasi-storage hard disk.

[0036] Alternatively, if the correlation between any GOP packet in the same group of target GOP packets and the GOP packets already stored in the quasi-storage hard disk is determined to be less than a preset correlation threshold, then replace the storage hard disk for the same group of target GOP packets.

[0037] Specifically, considering that GOP packets acquired at the same time from channels within the same stripe can be allocated to different hard drives for storage, we can also consider allocating related GOP packets from different stripes to different hard drives for storage. This ensures that if a GOP packet is lost on one hard drive, GOP packets from different stripes can be retrieved promptly and accurately. Therefore, based on the principle of maximizing GOP packet difference allocation, after selecting a target GOP packet from each channel, each target GOP packet belonging to the same group is compared with the GOP packets already stored on the quasi-storage hard drive. Only when the correlation between the target GOP packets in the same group and the GOP packets already stored on the quasi-storage hard drive is less than or equal to a preset correlation threshold will the target GOP packets in the same group be stored on the quasi-storage hard drive. Otherwise, the target GOP packets in this group will be re-selected for storage on a suitable hard drive according to the above principle.

[0038] In a second aspect, the present invention provides a multi-channel video data processing apparatus, the apparatus comprising:

[0039] The acquisition module is used to acquire pre-processed multi-channel video data, which is data generated by acquiring data through different channels and then pre-processing it.

[0040] The splitting module is used to split the video data of each channel into multiple GOP packets, based on the group of images (GOP).

[0041] The processing module is used to perform erasure coding on multiple GOP packets, and determine the difference between GOP packets in different channels based on the generation time of each GOP packet in the multi-channel. Based on the principle of maximizing the difference of GOP packets, a target GOP packet is selected from each channel and stored on the same hard disk, where the time difference between the target GOP packets in different channels is the greatest.

[0042] The multi-channel video data processing device provided by this invention has the following advantages:

[0043] After acquiring the preprocessed multi-channel video data, the video data for each channel is split into Groups of Pictures (GOPs), resulting in multiple GOP packets. Then, erasure coding is performed on these GOP packets. Based on the generation time of each GOP packet in the multi-channel dataset, the difference between GOP packets in different channels is determined. Following the principle of maximizing the allocation of GOP packets based on their difference, a target GOP packet is selected from each channel and stored on the same hard drive. This method significantly reduces the probability of losing the same GOP packet even in the event of one or more hard drive failures, greatly simplifying data recovery and improving repair accuracy.

[0044] In one optional implementation, the processing module is specifically used to take the first GOP packet in the first channel of the multi-channel as a reference, and in the current iteration cycle, to traverse and select each GOP packet from the other channels of the multi-channel except the first channel, and generate the difference degree between different GOP packets in different channels based on the generation time of all traversed GOP packets and the generation time of the first GOP packet, wherein the first channel is any channel in the multi-channel, and the first GOP packet is any GOP packet in the first channel.

[0045] Specifically, since video images are composed of individual frames, the division of GOP packets follows a specific temporal order. Therefore, determining the differences between GOP packets based on their generation times is a more reasonable and accurate approach.

[0046] In one optional implementation, the processing module is specifically used to select the first GOP packet group corresponding to the largest difference after traversing the first GOP packet in the first channel as a reference and obtaining multiple differences. The first GOP packet group includes the first GOP packet and a target GOP packet in each channel other than the first channel in the multiple channels.

[0047] All GOP packets in the first GOP packet group are stored on the same hard disk. GOP packets from different groups are stored on different hard disks. The number of hard disks is the sum of the number of GOP packets in any channel and the number of erasure codes in that channel.

[0048] Specifically, after traversing the first GOP packet in the first channel as a reference, multiple differences are obtained. Then, GOP packets from channels other than the first channel that generate the maximum difference with the first GOP packet are selected to form the target GOP packet group. These target GOP packets are stored on the same hard drive. Therefore, even if the hard drive subsequently fails and the GOP packets on that hard drive are lost, there will be little impact. The GOP packets on that hard drive can be recovered using GOP packets generated at the same time on other hard drives, as well as other GOP packets, with higher accuracy.

[0049] In an optional implementation, when a data failure occurs and the second GOP packet in the first channel of a multi-channel system is lost, while the GOP packets generated at the same time as the second GOP packet in other channels besides the first channel are not lost, the processing module is further configured to:

[0050] I-frame background information is extracted from a preset number of GOP packets in the first channel that are not lost except for the second GOP packet and have the highest correlation with the second GOP packet, forming an I-frame background information set. The correlation is determined according to the generation time of the GOP packet. The smaller the difference between the generation time of the GOP packet in the first channel and the generation time of the second GOP packet, the higher the correlation.

[0051] Select GOP packets generated at the same time as the second GOP packet from the channels other than the first channel in the multi-channel array to form the second GOP packet group, and extract the I-frame information and P-frame information corresponding to each GOP packet in the second GOP packet group.

[0052] And, from each of the channels other than the first channel, select a preset number of GOP packets that are not lost and have the highest relevance to the second GOP packet group, excluding the second GOP packet group, to form the third GOP packet group;

[0053] Extract the I-frame and P-frame information from each GOP packet in the second GOP packet group;

[0054] Based on the I-frame background information set, the I-frame and P-frame information corresponding to each GOP packet in the second GOP packet group, and the I-frame and P-frame information in each GOP packet in the third GOP packet group, the second GOP packet is recovered.

[0055] Specifically, when a data failure occurs and the second GOP packet in the first channel is lost, while GOP packets in other channels at the same time are not lost, I-frame background information can be extracted from a preset number of GOP packets in the first channel that are not lost and have the highest correlation with the second GOP packet, forming an I-frame background information set. This set is used to generate the I-frame background information for the second GOP packet. Then, from the channels other than the first channel, GOP packets generated at the same time as the second GOP packet are selected to form a second GOP packet group, and the I-frame and P-frame information corresponding to each GOP packet in the second GOP packet group are extracted. Furthermore, from the channels other than the first channel, a preset number of GOP packets that are not lost and have the highest correlation with the second GOP packet, excluding the second GOP packet group, are selected to form a third GOP packet group, and the I-frame and P-frame information of each GOP packet in the second GOP packet group are extracted. By fusing this information, the second GOP packet can be recovered. This is equivalent to combining data information collected from different angles (different channels) at the same time, as well as data information from adjacent time points, to finally complete the recovery of the second GOP packet. The accuracy of the recovery will be relatively higher.

[0056] In one alternative implementation, the processing module is specifically used for:

[0057] Based on the I-frame information of each GOP packet in the second GOP packet group, the foreground information of the second GOP packet is recovered.

[0058] Based on the P-frame information of each GOP packet in the second GOP packet group and the P-frame information of each GOP packet in the third GOP packet group, determine the P-frame differential information.

[0059] The second GOP packet is recovered based on foreground information, I-frame background information set, and P-frame differential information.

[0060] Specifically, based on the I-frame information of each GOP packet in the second GOP packet group, the foreground information of the second GOP packet can be recovered. Based on the P-frame information of each GOP packet in the second GOP packet group, the P-frame difference information between different GOP packets at the same time, as well as GOP packets at adjacent time points, and the second GOP packet can be determined. Finally, by combining the foreground information, the set of I-frame background information, and the P-frame difference information, the second GOP packet can be recovered.

[0061] In one optional implementation, when there are 3 channels, the processing module is specifically used to generate the difference between different GOP packets in different channels based on the following expression:

[0062]

[0063] Where d represents the difference between different GOP packets in different channels, and t (1,i) Let t be the generation time of the i-th GOP in the first channel. (2,j) Let t be the generation time of the j-th GOP in the second channel. (3,k) Let i be the generation time of the k-th GOP in the third channel, and let i, j, and k be the identifiers of any GOP packets in the three channels, where i, j, and k are all positive integers.

[0064] In an optional implementation, when it is determined that the current stripe is not the first stripe, the processing module is further configured to:

[0065] Based on the principle of maximizing the difference of GOP packets, after selecting a target GOP packet from each channel, the target GOP packets belonging to the same group are compared with the GOP packets already stored in the quasi-storage hard disk. The target GOP packets in the same group are the GOP packets to be stored on the same hard disk selected based on the rule of maximizing the difference of GOP packets.

[0066] When it is determined that the relevance between the target GOP packets in the same group and the GOP packets already stored in the quasi-storage hard disk is less than or equal to a preset relevance threshold, the target GOP packets in the same group are stored in the quasi-storage hard disk.

[0067] Alternatively, if the correlation between any GOP packet in the same group of target GOP packets and the GOP packets already stored in the quasi-storage hard disk is determined to be less than a preset correlation threshold, then replace the storage hard disk for the same group of target GOP packets.

[0068] Specifically, considering that GOP packets acquired at the same time from channels within the same stripe can be allocated to different hard drives for storage, we can also consider allocating related GOP packets from different stripes to different hard drives for storage. This ensures that if a GOP packet is lost on one hard drive, GOP packets from different stripes can be retrieved promptly and accurately. Therefore, based on the principle of maximizing GOP packet difference allocation, after selecting a target GOP packet from each channel, each target GOP packet belonging to the same group is compared with the GOP packets already stored on the quasi-storage hard drive. Only when the correlation between the target GOP packets in the same group and the GOP packets already stored on the quasi-storage hard drive is less than or equal to a preset correlation threshold will the target GOP packets in the same group be stored on the quasi-storage hard drive. Otherwise, the target GOP packets in this group will be re-selected for storage on a suitable hard drive according to the above principle.

[0069] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the multi-channel video data processing method described in the first aspect or any corresponding embodiment thereof.

[0070] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the multi-channel video data processing method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0071] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0072] Figure 1 This is a flowchart illustrating a multi-channel video data processing method provided in an embodiment of the present invention;

[0073] Figure 2 This is a schematic diagram of another multi-channel video data processing method provided in an embodiment of the present invention;

[0074] Figure 3 This is a schematic diagram of another multi-channel video data processing method provided by the present invention;

[0075] Figure 4 This is a schematic diagram of another multi-channel video data processing method provided in an embodiment of the present invention;

[0076] Figure 5 This is a schematic diagram of another multi-channel video data processing method provided by the present invention;

[0077] Figure 6 This is a schematic diagram illustrating data loss in disks 2, 4, and 7 provided by the present invention. It shows the data loss due to exceeding the fault domain.

[0078] Figure 7 This is a schematic diagram illustrating the extraction of adjacent GOPs of lost data using the method provided in this application embodiment of the present invention;

[0079] Figure 8 This is a schematic diagram provided by the present invention, which shows how to extract GOP packets from other disks at the same time as GOP2 when GOP2 in disk 2 is lost and cannot be recovered.

[0080] Figure 9 This is a block diagram illustrating the specific implementation of the multi-channel video data storage provided by the present invention;

[0081] Figure 10 This is a flowchart of the overall process for recovering lost data beyond the fault domain provided by the present invention;

[0082] Figure 11 This is the overall flowchart of the present invention for maximizing the GOP difference between different stripes;

[0083] Figure 12 This is a structural block diagram of a multi-channel video data processing device according to an embodiment of the present invention;

[0084] Figure 13 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0085] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0086] Traditional erasure striping storage employs a random write mode, which offers high write bandwidth. However, in video storage systems, this mode cannot guarantee data recovery when facing out-of-fault domain data loss. Out-of-fault domain refers to faults exceeding the designed redundancy and fault tolerance range, resulting in irrecoverable and permanent data loss. For example, the commonly used k+m RS erasure coding storage method has a fault tolerance of m; when more than m data blocks become unavailable, it constitutes an out-of-fault domain.

[0087] There are methods for repairing single-channel image data in related technologies, but these methods cannot be applied to the problem of repairing multi-channel video data, and no other related literature has been reported.

[0088] To address the aforementioned problems, this invention provides a multi-channel video data processing embodiment. It should be noted that the steps shown in the flowcharts in the accompanying drawings can be executed in a computer system (computer device) including a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0089] This embodiment provides a multi-channel video data processing method, which can be used in the aforementioned terminal devices, such as mobile phones and tablet computers. Figure 1 This is a flowchart illustrating a multi-channel video data processing method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the process includes the following steps:

[0090] Step S101: Obtain the preprocessed multi-channel video data.

[0091] Specifically, multi-channel video data refers to data acquired through different channels and generated after preprocessing. Preprocessing may include operations such as resizing and compression.

[0092] Step S102: For each channel of video data, split it into multiple GOP packets by image group (GOP).

[0093] Step S103: After performing erasure coding on multiple GOP packets, determine the degree of difference between GOP packets in different channels based on the generation time of each GOP packet in the multi-channel system.

[0094] Specifically, by applying erasure coding to the GOP packets in each channel of a multi-channel dataset, erasure stripe data can be generated. Then, based on the generation time of the GOP packets in each erasure stripe of the multi-channel dataset, the degree of difference between GOP packets in different channels can be determined. The longer the interval between generation times, the greater the degree of difference.

[0095] Step S104: Based on the principle of maximizing the allocation of GOP packets with the greatest difference, select one target GOP packet from each channel and store it on the same hard drive.

[0096] Specifically, the GOP packets with the greatest differences across different channels can be grouped together and stored on the same hard drive. For example, there might be three channels, each containing the same data. See Table 1 for details.

[0097] Table 1

[0098]

[0099] Table 1 shows the video data of each channel after being split using 8+2RS erasure coding, and the resulting 8 GOP packets and 2 erasure codes after the erasure operation. Furthermore, each channel includes 8 GOP packets captured at different times, such as GOP1 to GOP8. GOP packets captured at the same time from different channels are from different angles, such as GOP1, GOP1', and GOP". These are different GOP packets generated after multiple channels capture the same object from different angles; although it is the same object, the different angles may contain slightly different information.

[0100] To prevent different GOP packets acquired at the same time from being stored on the same hard drive, it is necessary to select one target GOP packet from each channel and place it on the same hard drive based on the principle of maximizing the time difference between target GOP packets in different channels.

[0101] The degree of difference can be determined based on the generation time of different GOP packages. The greater the interval between generation times, the greater the degree of difference.

[0102] The multi-channel video data processing method provided in this embodiment, after acquiring the preprocessed multi-channel video data, splits the video data of each channel into multiple GOP packets, using Group of Pictures (GOPs) as units. Then, after performing erasure coding on the multiple GOP packets, the difference between GOP packets in different channels is determined based on the generation time of each GOP packet in the multi-channel dataset. According to the principle of maximizing the allocation of GOP packets based on their difference, a target GOP packet is selected from each channel and stored on the same hard drive. This method significantly reduces the probability of losing the same GOP packet even in the event of failure of one or more hard drives, greatly reducing the difficulty of data recovery and improving the accuracy of repair.

[0103] This embodiment provides a multi-channel video data processing method, which can be used in the aforementioned mobile terminals, such as mobile phones and tablet computers. Figure 2 This is a schematic flowchart of another multi-channel video data processing method provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the process includes the following steps:

[0104] Step S201: Obtain the preprocessed multi-channel video data.

[0105] Among them, multi-channel video data is data generated by collecting data through different channels and then preprocessing it.

[0106] Step S202: For each channel of video data, split it into multiple GOP packets by image group (GOP).

[0107] For a detailed description of steps S201 to S202, please refer to [link / reference needed]. Figure 1 Steps S101 to S102 of the illustrated embodiment will not be described again here.

[0108] Step S203: After performing erasure coding on multiple GOP packets, determine the degree of difference between GOP packets in different channels based on the generation time of each GOP packet in the multi-channel system.

[0109] Specifically, after performing erasure coding on multiple GOP packets, when determining the degree of difference between GOP packets in different channels based on the generation time of each GOP packet in multiple channels, this can be achieved in the following ways:

[0110] Step S2031: Based on the first GOP packet in the first channel of the multi-channel, in the current iteration cycle, select each GOP packet from the other channels of the multi-channel except the first channel.

[0111] Step S2032: Based on the generation time of all traversed GOP packets and the generation time of the first GOP packet, generate the difference between different GOP packets in different channels.

[0112] Here, the first channel is any one of the multiple channels, and the first GOP packet is any GOP packet in the first channel.

[0113] Specifically, assuming there are three channels, the video data in each channel, after being divided into GOPs, results in the same number of GOP packets. The degree of difference can be determined based on the generation time of different GOP packets. The closer the generation time, the smaller the difference. In particular, the difference between GOP packets generated at the same time is relatively minimal. Taking the first GOP packet in the first channel as the baseline, we then iterate through each GOP packet in the other two channels. Here, we need to consider not only the difference between the first GOP packet and the GOP packets in each corresponding channel, but also the difference between GOP packets in other different channels. That is, if there are three channels, the difference between the three GOP packets stored on the same hard drive is the greatest. For example, if the first GOP packet is the first one in the first channel, when selecting the GOP packet in the second channel, we can choose the one with the middle generation time, while the GOP packet selected in the third channel can be the one with the last generation time. In this case, the difference between the three will be relatively large. For example, the first GOP packet might be the second one in the first channel, the second channel could select a GOP packet whose generation time is in the middle and immediately following it, and the third channel could select the first GOP packet. This ensures that the differences between GOP packets on each hard drive are relatively large.

[0114] Of course, for a more accurate calculation of the differences between different GOP packages, please refer to the formula below.

[0115] In an optional implementation, taking a channel count of 3 as an example, using the first GOP packet in the first channel as the benchmark, in the current iteration cycle, each GOP packet is selected from all channels except the first channel. Based on the generation time of all traversed GOP packets and the generation time of the first GOP packet, the difference between different GOP packets in different channels is generated, expressed by the following formula:

[0116] d=max∑ i,j,k∈[1,n] |t (1,i) -t (2,j) |+|t (2,j) -t (3,k) |+|t (1,i) -t (3,k) |(Formula 1)

[0117] Where d represents the difference between different GOP packets in different channels, and t (1,i) Let t be the generation time of the i-th GOP in the first channel. (2,j) Let t be the generation time of the j-th GOP in the second channel. (3,k) Let i be the generation time of the k-th GOP in the third channel, and let i, j, and k be the identifiers of any GOP packets in the three channels, where i, j, and k are all positive integers.

[0118] As described in the formula above, we can take the i-th GOP packet in the first channel as the benchmark, then calculate the difference between the i-th GOP packet and any GOP packet in other channels, and then select the GOP packet with the largest difference. This process continues until i, j, and k have all reached n.

[0119] Specifically, since video images are composed of individual frames, the division of GOP packets follows a specific temporal order. Therefore, determining the differences between GOP packets based on their generation times is a more reasonable and accurate approach.

[0120] Step S204: Based on the principle of maximizing the allocation of GOP packets with the greatest difference, select one target GOP packet from each channel and store it on the same hard drive.

[0121] For a detailed description of step S204, please refer to [link / reference needed]. Figure 1 Step S104 of the illustrated embodiment will not be described again here.

[0122] Furthermore, in an optional implementation, based on the principle of maximizing GOP dissimilarity allocation, one target GOP packet is selected from each channel and stored on the same hard drive. This can be achieved in the following way, see details below. Figure 3 As shown, it includes:

[0123] Step S301: After traversing the first GOP packet in the first channel as a reference and obtaining multiple differences, select the first GOP packet group corresponding to the largest difference.

[0124] The first GOP packet group includes the first GOP packet and a target GOP packet in each of the multiple channels except the first channel.

[0125] Specifically, the process of selecting the first GOP packet group, including the first GOP packet, has been described in detail above and will not be repeated here.

[0126] Step S302: Store all GOP packets in the first GOP packet group on the same hard disk.

[0127] Different groups of GOP packets are stored on different hard drives. The number of hard drives is the sum of the number of GOP packets in any channel and the number of erasure codes in that channel.

[0128] Specifically, after traversing the first GOP packet in the first channel as a reference, multiple differences are obtained. Then, GOP packets from channels other than the first channel that generate the maximum difference with the first GOP packet are selected to form the target GOP packet group. These target GOP packets are stored on the same hard drive. Therefore, even if the hard drive subsequently fails and the GOP packets on that hard drive are lost, there will be little impact. The GOP packets on that hard drive can be recovered using GOP packets generated at the same time on other hard drives, as well as other GOP packets, with higher accuracy.

[0129] This embodiment provides a multi-channel video data processing method, which can be used in the aforementioned mobile terminals, such as mobile phones and tablet computers. Figure 4 This is a schematic diagram of another multi-channel video data processing method provided in an embodiment of the present invention, as shown below. Figure 4 As shown, this method mainly considers the specific handling scheme when data exceeds the fault domain and the second GOP packet in the first channel of the multi-channel is lost, while the GOP packets generated at the same time as the second GOP packet in other channels besides the first channel are not lost.

[0130] In this context, "out-of-fault domain" refers to a situation where a fault exceeds the designed redundancy and fault tolerance range, resulting in unrecoverable data. In this application, because 8+2RS erasure coding is used, the loss of more than three GOP packets can be considered an out-of-fault domain occurrence. This application uses the loss of three GOP packets as an example to illustrate how to achieve data out-of-fault domain recovery. Specifically, data out-of-fault domain recovery means that even when the number of failed hard drives exceeds the maximum capacity of the storage system, the original data can still be recovered from the remaining usable data or checksum data.

[0131] The overall implementation idea of ​​this application is to reconstruct the lost data in the fault domain by utilizing the background consistency information within the channel and the foreground correlation information between channels, thereby realizing video data recovery.

[0132] As mentioned earlier, in the multi-channel video information storage stage, this application first encodes each GOP packet of the acquired image according to the k+m erasure coding method; then, the data stripes of each channel are distributed and stored according to the principle of "maximum GOP difference within the disk", thereby reducing the difficulty of data recovery.

[0133] During the data recovery phase of the out-of-fault domain, the process includes the following steps:

[0134] Step S401: Extract I-frame background information from a preset number of GOP packets (excluding the second GOP packet) that are not lost and have the highest correlation with the second GOP packet in the first channel, and form an I-frame background information set.

[0135] The relevance is determined by the generation time of the GOP packets. The smaller the difference between the generation time of the GOP packets in the first channel and the generation time of the second GOP packets, the higher the relevance. It's important to clarify the difference between difference and relevance. Difference refers to the differences between GOP packets in different channels, while relevance refers to the association between different GOP packets in the same channel. The shorter the generation time interval between different GOP packets, the greater the correlation between the content in the two GOP packets. Therefore, the smaller the difference between the generation time of one GOP packet and the generation time of the second GOP packet, the higher the relevance.

[0136] Specifically, I-frame background information is extracted from a preset number of GOP packets in the first channel that are not lost except for the second GOP packet and have the highest correlation with the second GOP packet, forming an I-frame background information set. This I-frame background information extraction can be performed using a pre-trained Gaussian Mixture Model (GMM).

[0137] Step S402: Select GOP packets generated at the same time as the second GOP packet from the channels other than the first channel in the multi-channel array to form the second GOP packet group.

[0138] Taking the GOP packets in Table 1 as an example, as shown in Table 1, assuming that the second GOP packet is GOP2 in the first channel, then the GOP packets generated at the same time as the second GOP packet are selected from the other channels besides the first channel in the multi-channel array, which are GOP2' and GOP2".

[0139] Step S403: Extract the I-frame information and P-frame information corresponding to each GOP packet in the second GOP packet group.

[0140] Specifically, the implementation scheme for extracting the I-frame and P-frame information corresponding to each GOP packet in the second GOP packet group is a relatively mature technology, such as the GMM implementation introduced above, which will not be elaborated on here.

[0141] Step S404: Select a preset number of GOP packets that are not lost and have the highest correlation with the second GOP packets from the channels other than the first channel, excluding the second GOP packet group, to form the third GOP packet group.

[0142] The specific implementation process is similar to step S401 - the process of selecting a preset number of GOP packets from the first channel that are not lost except for the second GOP packet and have the highest correlation with the second GOP packet, and will not be described in detail here.

[0143] Step S405: Extract the I-frame information and P-frame information from each GOP packet in the second GOP packet group.

[0144] The specific implementation can be achieved using existing mature technologies, such as GMM, so it will not be elaborated here.

[0145] Step S406: Based on the I-frame background information set, the I-frame information and P-frame information corresponding to each GOP packet in the second GOP packet group, and the I-frame information and P-frame information in each GOP packet in the third GOP packet group, the second GOP packet is recovered.

[0146] Specifically, when a data failure occurs and the second GOP packet in the first channel is lost, while GOP packets in other channels at the same time are not lost, I-frame background information can be extracted from a preset number of GOP packets in the first channel that are not lost and have the highest correlation with the second GOP packet, forming an I-frame background information set. This set is used to generate the I-frame background information for the second GOP packet. Then, from the channels other than the first channel, GOP packets generated at the same time as the second GOP packet are selected to form a second GOP packet group, and the I-frame and P-frame information corresponding to each GOP packet in the second GOP packet group are extracted. Furthermore, from the channels other than the first channel, a preset number of GOP packets that are not lost and have the highest correlation with the second GOP packet, excluding the second GOP packet group, are selected to form a third GOP packet group, and the I-frame and P-frame information of each GOP packet in the second GOP packet group are extracted. By fusing this information, the second GOP packet can be recovered. This is equivalent to combining data information collected from different angles (different channels) at the same time, as well as data information from adjacent time points, to finally complete the recovery of the second GOP packet. The accuracy of the recovery will be relatively higher.

[0147] In an optional implementation, the second GOP packet is recovered based on the I-frame background information set, the I-frame and P-frame information corresponding to each GOP packet in the second GOP packet group, and the I-frame and P-frame information in each GOP packet of the third GOP packet group. This can be achieved in the following way, see [link to relevant documentation]. Figure 5 As shown, it includes:

[0148] Step S501: Based on the I-frame information of each GOP packet in the second GOP packet group and the I-frame information of each GOP packet in the second GOP packet group, the foreground information of the second GOP packet is restored.

[0149] Specifically, when recovering the foreground information of the second GOP packet based on the I-frame information of each GOP packet in the second GOP packet group, pre-trained Generative Adversarial Networks (GANs) can be used. GANs are then used to reconstruct and recover the foreground information of the second GOP packet.

[0150] Step S502: Determine P-frame differential information based on the P-frame information of each GOP packet in the second GOP packet group and the P-frame information of each GOP packet in the third GOP packet group.

[0151] Similarly, when determining the P-frame differential information based on the P-frame information of each GOP packet in the second GOP packet group and the P-frame information of each GOP packet in the third GOP packet group, a trained convolutional neural network (CNN) can be used to reconstruct and restore the P-frame differential information, thereby recovering the P-frame information of the second GOP packet.

[0152] Step S503: Based on the foreground information, the I-frame background information set, and the P-frame differential information, recover the second GOP packet.

[0153] Finally, the second GOP packet is reconstructed based on foreground information, I-frame background information set, and P-frame differential information.

[0154] Specifically, based on the I-frame information of each GOP packet in the second GOP packet group, the foreground information of the second GOP packet can be recovered. Based on the P-frame information of each GOP packet in the second GOP packet group, the P-frame difference information between different GOP packets at the same time, as well as GOP packets at adjacent time points, and the second GOP packet can be determined. Finally, by combining the foreground information, the set of I-frame background information, and the P-frame difference information, the second GOP packet can be recovered.

[0155] Figure 6 The diagram illustrates 10 disks. Disks 1 through 8 store GOP packets, while disks 9 and 10 store checksum blocks. Data was lost on disks 2, 4, and 7, resulting in data loss beyond the fault domain. Figure 7This diagram illustrates the method for extracting adjacent Groups of Points (GOPs) of lost data according to an embodiment of this application. For example, when GOP2 is lost in disk 2, data from GOP1 and GOP3, which are adjacent to GOP2, is extracted. When GOP4 is lost, data from GOP3 and GOP5, which are adjacent to GOP4, is extracted. Adjacent GOPs have the highest correlation.

[0156] Figure 8 The diagram illustrates that when the GOP2 data in data disk 2 is lost and unrecoverable, relevant foreground information, as well as I-frame and P-frame information related to GOP2, are extracted from GOP2 in channels 2 and 3, respectively. By combining the information of adjacent GOPs within a channel and GOPs from different channels at the same time, and analyzing the characteristics of I-frame and P-frame information, foreground, background, and differential information can be reconstructed, enabling the recovery of data lost beyond the fault domain.

[0157] The specific implementation process has been described in detail above, so it will not be repeated here.

[0158] Further optionally, in one possible implementation, although following the above method, if three disks are lost simultaneously, it is possible to lose three packets at the same time. Although the probability of this happening is very low, once it does occur, a suitable handling method is still needed to resolve it. Therefore, this application embodiment also provides the following method steps, specifically as follows, taking the loss of the third GOP packet in the first channel as an example, and the loss of the third GOP' packet and the third GOP" packet in other channels respectively as examples, the following description takes the recovery of the third GOP packet as an example.

[0159] Step a1: Extract I-frame background information from a preset number of GOP packets (excluding the third GOP packet) that are not lost and have the highest correlation with the third GOP packet in the first channel, and form an I-frame background information set.

[0160] Step a2: Select a preset number of GOP packets that have not been lost from the multi-channel and have the highest relevance to the third GOP packet to form the fourth GOP packet group.

[0161] Step a3: Extract the I-frame and P-frame information from each GOP packet in the fourth GOP packet group;

[0162] Step a4: Based on the I-frame background information set and the I-frame and P-frame information corresponding to each GOP packet in the fourth GOP packet group, recover the second GOP packet.

[0163] Step a4 can be achieved through the following steps:

[0164] Step b1: Based on the I-frame information corresponding to each GOP packet in the fourth GOP packet group, the foreground information of the second GOP packet is recovered;

[0165] Step b2: Determine the P-frame differential information based on the P-frame information corresponding to each GOP packet in the fourth GOP packet group;

[0166] Step b3: Based on the foreground information, the I-frame background information set, and the P-frame differential information, recover the second GOP packet.

[0167] The specific implementation process of steps a1 to a4 and steps b1 to b3 is the same as described above. Figure 6 The corresponding embodiments, and Figure 6 The methods described in the following embodiments are similar, so they will not be elaborated on here.

[0168] It should be noted that since all GOP packets generated at the same time as the third GOP packet have been lost, it is not possible to select GOP packets generated at the same time as the third GOP packet from channels other than the first channel in the multi-channel. Therefore, the content related to this part in steps a1 to a4 and steps b1 to b3 is directly omitted.

[0169] In an optional implementation, considering that GOP packets acquired at the same time from channels within the same stripe can be allocated to different hard drives for storage, it is also possible to allocate related GOP packets from different stripes to different hard drives for storage. This ensures that if a GOP packet on one hard drive is lost, GOP packets from different stripes can be retrieved promptly and accurately. Therefore, the method further includes the following steps, as detailed below:

[0170] Step c1: Based on the principle of maximizing the difference of GOP packets, after selecting a target GOP packet from each channel, each target GOP packet belonging to the same group is compared with the GOP packets already stored in the quasi-storage hard disk.

[0171] Among them, the target GOP packets in the same group are the GOP packets to be stored on the same hard drive selected based on the GOP packet difference maximum allocation rule;

[0172] Step c2: When it is determined that the relevance between the target GOP packets in the same group and the GOP packets already stored in the quasi-storage hard disk is less than or equal to the preset relevance threshold, the target GOP packets in the same group are stored in the quasi-storage hard disk.

[0173] In addition to the generation time mentioned above, the relevance can also be determined by the association between the metadata in the two GOP packages, such as the metadata content itself and the original storage address of the metadata.

[0174] or,

[0175] Step c3: When it is determined that the correlation between any GOP packet in the same group of target GOP packets and the GOP packets already stored in the quasi-storage hard disk is less than a preset correlation threshold, replace the storage hard disk for the target GOP packets in the same group.

[0176] Optionally, to further reduce the global GOP correlation within the disk and improve data reliability, during GOP writing, the write address of the GOP in each stripe is recorded, and the GOP write record in memory is updated to achieve relatively low GOP correlation between stripes within 2*k seconds. In fact, steps c1 to c3 of the aforementioned method can minimize the correlation between GOP packets from different stripes on the same hard drive, meaning the difference is relatively large. However, when determining the correlation, it is also possible to consider determining it based on the current storage address of the GOP in the stripe. If relatively related GOP packets from different stripes are already stored on hard drive A, then GOP packets that have not yet been stored can be considered for storage on hard drive B.

[0177] Figure 9 The diagram below illustrates the specific implementation block diagram for multi-channel video data storage. Figure 10 The diagram illustrates the overall process flow for recovering data lost beyond the fault domain. It should be noted that this application focuses on the case beyond the fault domain, so it does not describe the specific processing flow for the ordinary case where the beyond-fault domain does not occur. In fact, in the ordinary case where the beyond-fault domain does not occur, it can be directly implemented through ordinary error correction coding (EC coding). Figure 11 The middle section illustrates the overall flowchart for maximizing the difference in GOPs between different stripes. Figure 11 The target GOP write address is determined by maximizing the difference between GOP packets between different channels within and between stripes. Figure 11 The storage disk array includes D1, D2, ..., D6, etc., which are the hard disks mentioned in the embodiments of this application. Because Figures 9 to 11 The methods and procedures have been described in detail in the previous text, so they will not be repeated here.

[0178] This embodiment also provides a multi-channel video data processing apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0179] This embodiment provides a multi-channel video data processing device, such as... Figure 12 The module includes: an acquisition module 1201, a splitting module 1202, and a processing module 1203.

[0180] The acquisition module 1201 is used to acquire preprocessed multi-channel video data, which is data generated by acquiring data through different channels and preprocessing it.

[0181] The splitting module 1202 is used to split the video data of each channel into multiple GOP packets by taking the group of images (GOP) as the unit.

[0182] The processing module 1203 is used to perform erasure coding on multiple GOP packets, determine the difference between GOP packets in different channels based on the generation time of each GOP packet in the multi-channel, and select one target GOP packet from each channel and store it on the same hard disk based on the principle of maximizing the difference between GOP packets. Among them, the time difference between the target GOP packets in different channels is the greatest.

[0183] In one optional implementation, the processing module 1203 is specifically used to take the first GOP packet in the first channel of the multi-channel as a reference, and in the current iteration cycle, to traverse and select each GOP packet from the other channels in the multi-channel except the first channel, and generate the difference degree between different GOP packets in different channels based on the generation time of all traversed GOP packets and the generation time of the first GOP packet, wherein the first channel is any channel in the multi-channel, and the first GOP packet is any GOP packet in the first channel.

[0184] In an optional implementation, the processing module 1203 is specifically used to select the first GOP packet group corresponding to the largest difference after traversing the first GOP packet in the first channel as a reference and obtaining multiple differences. The first GOP packet group includes the first GOP packet and a target GOP packet in each channel other than the first channel in the multiple channels.

[0185] All GOP packets in the first GOP packet group are stored on the same hard disk. GOP packets from different groups are stored on different hard disks. The number of hard disks is the sum of the number of GOP packets in any channel and the number of erasure codes in that channel.

[0186] In an optional implementation, when a data failure occurs and the second GOP packet in the first channel of a multi-channel system is lost, but the GOP packets generated at the same time as the second GOP packet in other channels besides the first channel are not lost, the processing module 1203 is further configured to:

[0187] I-frame background information is extracted from a preset number of GOP packets in the first channel that are not lost except for the second GOP packet and have the highest correlation with the second GOP packet, forming an I-frame background information set. The correlation is determined according to the generation time of the GOP packet. The smaller the difference between the generation time of the GOP packet in the first channel and the generation time of the second GOP packet, the higher the correlation.

[0188] Select GOP packets generated at the same time as the second GOP packet from the channels other than the first channel in the multi-channel array to form the second GOP packet group, and extract the I-frame information and P-frame information corresponding to each GOP packet in the second GOP packet group.

[0189] And, from each of the channels other than the first channel, select a preset number of GOP packets that are not lost and have the highest relevance to the second GOP packet group, excluding the second GOP packet group, to form the third GOP packet group;

[0190] Extract the I-frame and P-frame information from each GOP packet in the second GOP packet group;

[0191] Based on the I-frame background information set, the I-frame and P-frame information corresponding to each GOP packet in the second GOP packet group, and the I-frame and P-frame information in each GOP packet in the third GOP packet group, the second GOP packet is recovered.

[0192] In one optional implementation, the processing module 1203 is specifically used for:

[0193] Based on the I-frame information of each GOP packet in the second GOP packet group, the foreground information of the second GOP packet is recovered.

[0194] Based on the P-frame information of each GOP packet in the second GOP packet group and the P-frame information of each GOP packet in the third GOP packet group, determine the P-frame differential information.

[0195] The second GOP packet is recovered based on foreground information, I-frame background information set, and P-frame differential information.

[0196] In an optional implementation, when there are 3 channels, the processing module 1203 is specifically used to generate the difference between different GOP packets in different channels based on the following expression:

[0197] d=max∑ i,j,k∈[1,n] |t (1,i) -t (2,j) |+|t (2,j) -t (3,k) |+|t (1,i) -t (3,k)|(Formula 2)

[0198] Where d represents the difference between different GOP packets in different channels, and t (1,i) Let t be the generation time of the i-th GOP in the first channel. (2,j) Let t be the generation time of the j-th GOP in the second channel. (3,k) Let i be the generation time of the k-th GOP in the third channel, where i, j, and k are the identifiers of any GOP packet in the three channels, and i, j, and k are all positive integers.

[0199] In an optional implementation, when it is determined that the current stripe is not the first stripe, the processing module 1203 is further configured to:

[0200] Based on the principle of maximizing the difference of GOP packets, after selecting a target GOP packet from each channel, the target GOP packets belonging to the same group are compared with the GOP packets already stored in the quasi-storage hard disk. The target GOP packets in the same group are the GOP packets to be stored on the same hard disk selected based on the rule of maximizing the difference of GOP packets.

[0201] When it is determined that the relevance between the target GOP packets in the same group and the GOP packets already stored in the quasi-storage hard disk is less than or equal to the preset relevance threshold, the target GOP packets in the same group are stored in the quasi-storage hard disk.

[0202] Alternatively, when it is determined that the correlation between the target GOP packets in the same group and the GOP packets already stored in the quasi-storage hard disk is less than a preset correlation threshold, the target GOP packets in the same group are replaced with other storage hard disks.

[0203] In this embodiment, the multi-channel video data processing device is presented in the form of a functional module. Here, a module refers to an application-specific integrated circuit (ASIC), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0204] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0205] This invention provides a multi-channel video data processing apparatus. After acquiring preprocessed multi-channel video data, it splits the video data of each channel into multiple Group of Pictures (GOPs) to obtain multiple GOP packets. Then, after erasure coding processing of the multiple GOP packets, the difference between GOP packets in different channels is determined based on the generation time of each GOP packet in the multi-channel dataset. According to the principle of maximizing the allocation of GOP packets based on their difference, a target GOP packet is selected from each channel and stored on the same hard drive. This method significantly reduces the probability of losing the same GOP packet even in the event of failure of one or more hard drives, greatly reducing the difficulty of data recovery and improving the accuracy of repair.

[0206] This invention also provides a computer device having the above-described features. Figure 12 The multi-channel video data processing device shown.

[0207] Please see Figure 13 , Figure 13 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 13 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 13 Take a processor 10 as an example.

[0208] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0209] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0210] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device as shown by a landing page for an app. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transient memory, such as at least one hard disk storage device, flash memory device, or other non-transient solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0211] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0212] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 13 Taking the example of a connection between China and Israel via a bus.

[0213] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0214] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0215] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A multi-channel video data processing method, characterized in that, The method includes: Acquire preprocessed multi-channel video data, wherein the multi-channel video data is data acquired through different channels and generated after preprocessing; For each channel of video data, it is split into multiple GOP packets based on the Group of Pictures (GOP). After performing erasure coding on multiple GOP packets, the degree of difference between GOP packets in different channels is determined based on the generation time of each GOP packet in multiple channels. Based on the principle of maximizing the difference in GOP packets, a target GOP packet is selected from each channel and stored on the same hard drive, where the time difference between the target GOP packets in different channels is maximized. When a data failure occurs and the second GOP packet in the first channel of the multi-channel system is lost, but the GOP packets generated at the same time as the second GOP packet in other channels besides the first channel are not lost, the method further includes: I-frame background information is extracted from a preset number of GOP packets in the first channel that are not lost except for the second GOP packet and have the highest correlation with the second GOP packet, forming an I-frame background information set. The correlation is determined based on the generation time of the GOP packet. The smaller the difference between the generation time of the GOP packet in the first channel and the generation time of the second GOP packet, the higher the correlation. Select GOP packets generated at the same time as the second GOP packet from the channels other than the first channel in the multi-channel, form the second GOP packet group, and extract the I-frame information and P-frame information corresponding to each GOP packet in the second GOP packet group. In addition, a preset number of GOP packets that are not lost and have the highest relevance to the second GOP packet are selected from other channels besides the first channel to form a third GOP packet group; Extract the I-frame and P-frame information from each GOP packet in the second GOP packet group; Based on the I-frame background information set, the I-frame and P-frame information corresponding to each GOP packet in the second GOP packet group, and the I-frame and P-frame information in each GOP packet in the third GOP packet group, the second GOP packet is recovered.

2. The method according to claim 1, characterized in that, After performing erasure coding on multiple GOP packets, the degree of difference between GOP packets in different channels is determined based on the generation time of each GOP packet in the multi-channel system, including: Based on the first GOP packet in the first channel of the multi-channel, in the current iteration cycle, each GOP packet is selected from the other channels of the multi-channel except the first channel. Based on the generation time of all the traversed GOP packets and the generation time of the first GOP packet, the difference between different GOP packets in different channels is generated. Here, the first channel is any one of the multi-channel, and the first GOP packet is any GOP packet in the first channel.

3. The method according to claim 2, characterized in that, The principle of maximizing the allocation of GOP packets based on their differences, which involves selecting one target GOP packet from each channel and storing it on the same hard drive, includes: After traversing the first GOP packet in the first channel as a reference and obtaining multiple differences, the first GOP packet group corresponding to the largest difference is selected. The first GOP packet group includes the first GOP packet and a target GOP packet in each of the multiple channels except the first channel. All GOP packets in the first GOP packet group are stored on the same hard disk, wherein GOP packets from different groups are stored on different hard disks, and the number of hard disks is the sum of the number of GOP packets in any channel and the number of erasure codes in that channel.

4. The method according to claim 3, characterized in that, The step of recovering the second GOP packet based on the I-frame background information set, the I-frame information and P-frame information corresponding to each GOP packet in the second GOP packet group, and the I-frame information and P-frame information in each GOP packet of the third GOP packet group includes: Based on the I-frame information of each GOP packet in the second GOP packet group and the I-frame information of each GOP packet in the third GOP packet group, the foreground information of the second GOP packet is recovered. Based on the P-frame information of each GOP packet in the second GOP packet group and the P-frame information of each GOP packet in the third GOP packet group, the P-frame differential information is determined. The second GOP packet is recovered based on the foreground information, the I-frame background information set, and the P-frame differential information.

5. The method according to claim 2 or 3, characterized in that, When there are 3 channels, the first GOP packet in the first channel of the multi-channel is used as the reference. In the current iteration cycle, each GOP packet is selected from all channels except the first channel. Based on the generation time of all traversed GOP packets and the generation time of the first GOP packet, the difference between different GOP packets in different channels is generated, which is expressed by the following formula: Where d represents the degree of difference between different GOP packets in different channels. Let be the generation time of the i-th GOP in the first channel. Let j be the generation time of the j-th GOP in the second channel. Let i be the generation time of the k-th GOP in the third channel, where i, j, and k are the identifiers of any GOP packet in the three channels, and i, j, and k are all positive integers.

6. The method according to any one of claims 1-3 or the method thereof, characterized in that, The method further includes: Based on the principle of maximizing the difference of GOP packets, after selecting a target GOP packet from each channel, the target GOP packets belonging to the same group are compared with the GOP packets already stored in the quasi-storage hard disk. The target GOP packets in the same group are the GOP packets to be stored on the same hard disk selected based on the principle of maximizing the difference of GOP packets. When it is determined that the relevance between the target GOP packets in the same group and the GOP packets already stored in the quasi-storage hard disk is less than or equal to a preset relevance threshold, the target GOP packets in the same group are stored in the quasi-storage hard disk. Alternatively, when it is determined that the correlation between any GOP package in the same group of target GOP packages and the GOP packages already stored in the quasi-storage hard disk is less than the preset correlation threshold, the target GOP packages in the same group are replaced with other storage hard disks.

7. A multi-channel video data processing device, characterized in that, The device includes: The acquisition module is used to acquire preprocessed multi-channel video data, which is data generated by acquiring data through different channels and preprocessing it. The splitting module is used to split the video data of each channel into multiple GOP packets, based on the group of images (GOP). The processing module is used to perform erasure coding on multiple GOP packets, determine the difference between GOP packets in different channels based on the generation time of each GOP packet in multiple channels, and select one target GOP packet from each channel and store it on the same hard disk based on the principle of maximizing the difference between GOP packets. The target GOP packets in different channels have the greatest time difference. When a data failure occurs and the second GOP packet in the first channel of the multi-channel is lost, but the GOP packets generated at the same time as the second GOP packet in other channels besides the first channel are not lost, the processing module is further configured to extract I-frame background information from a preset number of GOP packets in the first channel that are not lost and have the highest correlation with the second GOP packet, forming an I-frame background information set. The correlation is determined based on the generation time of the GOP packet. The smaller the difference between the generation time of the GOP packet in the first channel and the generation time of the second GOP packet, the higher the correlation. Select GOP packets generated at the same time as the second GOP packet from the channels other than the first channel in the multi-channel, form the second GOP packet group, and extract the I-frame information and P-frame information corresponding to each GOP packet in the second GOP packet group. In addition, a preset number of GOP packets that are not lost and have the highest relevance to the second GOP packet are selected from other channels besides the first channel to form a third GOP packet group; Extract the I-frame and P-frame information from each GOP packet in the second GOP packet group; Based on the I-frame background information set, the I-frame and P-frame information corresponding to each GOP packet in the second GOP packet group, and the I-frame and P-frame information in each GOP packet in the third GOP packet group, the second GOP packet is recovered.

8. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the multi-channel video data processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the multi-channel video data processing method according to any one of claims 1 to 6.

Images