A video data fault tolerance method, device, equipment and storage medium

Abstract

The present application relates to the field of computer technologies, and discloses a video data fault tolerance method, apparatus, device, and storage medium, including: defining a first erasure mode and a second erasure mode according to service requirements; wherein, the first erasure mode is used for scenarios where conventional erasure code redundancy technologies can perform fault tolerance processing, and the second erasure mode is used for scenarios where conventional erasure code redundancy technologies cannot perform fault tolerance processing; determining the number of faulty nodes in a distributed storage system storing target video data; performing mode switching between the first erasure mode and the second erasure mode according to the number of faulty nodes, so as to process the read video data using the switched erasure mode and send it to service software. By defining different erasure modes and realizing switching between different erasure modes in different scenarios according to the number of faulty nodes associated with the scenario, the present application ensures the continuity of video services while improving storage utilization rate.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a video data fault tolerance method, apparatus, device and storage medium. Background Art

[0002] With the development of computer technology, more and more data is stored in servers, and a single server can no longer meet the requirements for storing data. Therefore, a distributed storage system has been developed. In the face of a large amount of data storage, the reliability of data storage has to be considered. The commonly used erasure code technology is used to make a certain redundancy of data to increase the reliability of the system. The erasure redundancy rule technology divides the stored data into K data blocks, calculates M parity blocks through an encoding algorithm, and uses any K data blocks and parity blocks among the K+M data blocks. As long as the sum of the read data blocks and parity blocks is greater than or equal to K, the original data can be restored. By setting the erasure rule of the erasure code, the size of the storage space can be controlled, which is beneficial to improving the storage utilization rate. For example, when the ratio of data blocks to parity blocks in the erasure rule is 4 to 2, the available capacity of the storage system is two-thirds of the total capacity, and the data is guaranteed to be safe without more than two node failures.

[0003] Especially in the application of the video industry in a large amount of distributed storage, video customers generally choose the erasure redundancy rule to improve the storage utilization rate. However, in a large-scale cluster, if the erasure ratio is set low, for example, the set erasure ratio is K+2, where M is 2, although the obtained utilization rate is relatively high, when the sum of the failed data blocks and parity blocks is greater than 2, some data will fail to be read and the entire service will be interrupted. Most services in the video industry, such as video surveillance, etc., have relatively high requirements for the continuity of the service and relatively low requirements for individual page frame drops, which do not affect the overall viewing. However, the current erasure rules for distributed storage, aiming at the strong consistency of data, determine that the overall service is interrupted if one data block cannot be read.

[0004] Therefore, the above technical problems need to be solved by those skilled in the art urgently. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a video data fault tolerance method, apparatus, device and storage medium, which can ensure the continuity of video services while improving the storage utilization rate. The specific solutions are as follows:

[0006] The first aspect of the present application provides a video data fault tolerance method, including:

[0007] Define a first erasure mode and a second erasure mode according to service requirements; wherein, the first erasure mode is used for scenarios where conventional erasure code redundancy technology can perform fault tolerance processing, and the second erasure mode is used for scenarios where conventional erasure code redundancy technology cannot perform fault tolerance processing;

[0008] Determine the number of faulty nodes in the distributed storage system storing the target video data; wherein, there is an association relationship between the number of faulty nodes and the scenario;

[0009] Perform mode switching between the first erasure mode and the second erasure mode according to the number of faulty nodes, so as to process the read video data using the switched erasure mode and send it to the business software.

[0010] Optionally, before determining the number of faulty nodes in the distributed storage system storing the target video data, it further includes:

[0011] Judge whether all data chunks of the target video data can be read from the distributed storage system;

[0012] If all data chunks can be read, it is determined that there are no faulty nodes, and all the read data chunks are directly sent to the business software;

[0013] If all data chunks cannot be read, it is determined that there are faulty nodes, and the readable data chunks are read to execute the step of determining the number of faulty nodes.

[0014] Optionally, after reading the readable data chunks, it further includes:

[0015] Read the readable parity chunks from the distributed storage system, and determine the first sum of the number of readable data chunks and the number of readable parity chunks;

[0016] Calculate the difference between the first sum and the second sum of the number of all data chunks and all parity chunks corresponding to the target video data, and determine this difference as the number of faulty nodes.

[0017] Optionally, performing mode switching between the first erasure mode and the second erasure mode according to the number of faulty nodes includes:

[0018] If the number of faulty nodes is less than the number of all parity chunks, switch to the first erasure mode through the switch setting method;

[0019] Correspondingly, processing the read video data using the switched erasure mode and sending it to the business software includes:

[0020] Restore the target video data based on the readable data chunks and readable parity chunks using the first erasure mode, and send the restored target video data to the business software.

[0021] Optionally, the mode switching between the first erasure mode and the second erasure mode according to the number of faulty nodes includes:

[0022] If the number of faulty nodes is greater than the total number of parity chunks, switch to the second erasure mode by means of switch setting;

[0023] Correspondingly, the processing of the read video data using the switched erasure mode and sending it to the business software includes:

[0024] Determine whether the number of faulty nodes is less than the preset erasure redundancy. If so, perform a filling operation on the corresponding video data content, and send the filled video data content to the business software, so that the business software displays the filled video data content coherently.

[0025] Optionally, after determining whether the number of faulty nodes is less than the preset erasure redundancy, it further includes:

[0026] If not, output an error message to the business software to interrupt the business process of the business software.

[0027] Optionally, the preset erasure redundancy is a value between the total number of parity chunks and the second number.

[0028] The second aspect of the present application provides a video data fault tolerance device, including:

[0029] A mode definition module for defining a first erasure mode and a second erasure mode according to service requirements; wherein, the first erasure mode is used for scenarios where conventional erasure code redundancy technology can perform fault tolerance processing, and the second erasure mode is used for scenarios where conventional erasure code redundancy technology cannot perform fault tolerance processing;

[0030] A faulty node determination module for determining the number of faulty nodes in a distributed storage system storing target video data; wherein, there is an association relationship between the number of faulty nodes and the scenario;

[0031] A fault tolerance processing module for switching the mode between the first erasure mode and the second erasure mode according to the number of faulty nodes, so as to perform fault tolerance processing on the read video data using the switched erasure mode and send it to the business software.

[0032] A third aspect of the present application provides an electronic device, which includes a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the foregoing video data fault tolerance method.

[0033] A fourth aspect of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are loaded and executed by a processor, the foregoing video data fault tolerance method is implemented.

[0034] In the present application, a first erasure mode and a second erasure mode are defined according to service requirements; wherein, the first erasure mode is used for scenarios where conventional erasure code redundancy technology can perform fault tolerance processing, and the second erasure mode is used for scenarios where conventional erasure code redundancy technology cannot perform fault tolerance processing; then the number of faulty nodes in the distributed storage system storing the target video data is determined; finally, a mode switch is made between the first erasure mode and the second erasure mode according to the number of faulty nodes, so as to process the read video data using the switched erasure mode and send it to the service software. It can be seen that by defining different erasure modes, the present application switches different erasure modes in different scenarios according to the number of faulty nodes associated with the scenario, thereby ensuring the continuity of video services while improving storage utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0036] Figure 1 It is a flowchart of a video data fault tolerance method provided by the present application;

[0037] Figure 2 It is a flowchart of a specific video data fault tolerance method provided by the present application;

[0038] Figure 3 It is a flowchart of a specific method for calculating the number of faulty nodes provided by the present application;

[0039] Figure 4 It is a flowchart of a specific mode switching method provided by the present application;

[0040] Figure 5 It is a schematic diagram of a specific video data fault tolerance method provided by the present application;

[0041] Figure 6 Schematic diagram of a video data fault tolerance device provided by this application;

[0042] Figure 7 Structural diagram of a video data fault tolerance electronic device provided by this application. Specific implementation manners

[0043] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0044] Existing video clients generally choose erasure redundancy rules to improve storage utilization. However, in a large-scale cluster, if the erasure ratio is set relatively low, for example, the set erasure ratio is K + 2, where M is 2. Although the obtained utilization rate is relatively high, when the sum of the faulty data blocks and parity blocks is greater than 2, some data will fail to be read and the entire service will be interrupted. Most services in the video industry, such as video surveillance, etc., have relatively high requirements for service continuity and relatively low requirements for frame drops on individual pages, which do not affect the overall viewing. However, the current erasure rules for distributed storage, for the strong consistency of data, determine that the overall service is interrupted if one data block cannot be read. To address the above technical deficiencies, this application provides a video data fault tolerance solution. By defining different erasure modes and switching different erasure modes according to the number of faulty nodes associated with the scenario in different scenarios, the continuity of video services can be ensured while improving storage utilization.

[0045] Figure 1 Flowchart of a video data fault tolerance method provided by an embodiment of this application. See Figure 1 As shown, the video data fault tolerance method includes:

[0046] S11: Define a first erasure mode and a second erasure mode according to service requirements; wherein, the first erasure mode is used for scenarios where conventional erasure code redundancy technology can perform fault tolerance processing, and the second erasure mode is used for scenarios where conventional erasure code redundancy technology cannot perform fault tolerance processing.

[0047] In this embodiment, different erasure modes are defined first. In this embodiment, a first erasure mode and a second erasure mode are mainly defined according to service requirements. Among them, the first erasure mode is used for scenarios where the conventional erasure code redundancy technology can perform fault tolerance processing, and the second erasure mode is used for scenarios where the conventional erasure code redundancy technology cannot perform fault tolerance processing. It can also be understood that the first erasure mode is a conventional erasure mode, and the second erasure mode is a custom intelligent erasure mode with an increased redundancy. Among them, the conventional erasure mode is based on the erasure redundancy rule technology, which divides the stored data into K data blocks, calculates M parity blocks through an encoding algorithm, and uses any K of the K+M data blocks and parity blocks. As long as the sum of the read data blocks and parity blocks is greater than or equal to K, the original data can be restored.

[0048] S12: Determine the number of faulty nodes in the distributed storage system storing the target video data; among them, there is an association relationship between the number of faulty nodes and the scenario.

[0049] In this embodiment, the number of faulty nodes in the distributed storage system storing the target video data is determined. Among them, there is an association relationship between the number of faulty nodes and the scenario. It can be understood that when the target video data is stored, it will be first allocated to a certain number of placement groups (PGs), and then stored in different object storage devices (OSDs). Generally, one disk corresponds to one OSD. In this way, a video file will be divided into multiple erasure stripes, and each stripe is allocated in K+M OSDs, and each OSD is on a different node.

[0050] S13: Perform mode switching between the first erasure mode and the second erasure mode according to the number of faulty nodes, so as to process the read video data using the switched erasure mode and send it to the service software.

[0051] In this embodiment, mode switching is performed between the first erasure mode and the second erasure mode according to the number of faulty nodes, so as to process the read video data using the switched erasure mode and send it to the service software. Since there is an association relationship between the number of faulty nodes and the scenario, mode switching can be performed between the first erasure mode and the second erasure mode according to the number of faulty nodes. For example, if the number of faulty nodes is less than M, select the scenario where the conventional erasure code redundancy technology can perform fault tolerance processing, that is, switch to the first erasure mode; if the number of faulty nodes is not less than M, select the scenario where the conventional erasure code redundancy technology cannot perform fault tolerance processing, that is, switch to the second erasure mode.

[0052] In addition, in this embodiment, a switch module is provided. The switch module ensures that the stored video reading is switched between the first erasure mode and the second erasure mode. Turning on the switch of the switch module on the corresponding interface can initiate a smooth mode switch between the first erasure mode and the second erasure mode.

[0053] It can be seen that the embodiment of the present application first defines the first erasure mode and the second erasure mode according to service requirements. Among them, the first erasure mode is used for scenarios where conventional erasure code redundancy technology can perform fault tolerance processing, and the second erasure mode is used for scenarios where conventional erasure code redundancy technology cannot perform fault tolerance processing. Then, the number of faulty nodes in the distributed storage system storing the target video data is determined. Finally, the mode is switched between the first erasure mode and the second erasure mode according to the number of faulty nodes, so as to process the read video data using the switched erasure mode and send it to the service software. By defining different erasure modes and switching different erasure modes in different scenarios according to the number of faulty nodes associated with the scenario, the embodiment of the present application improves the storage utilization rate while ensuring the continuity of the video service.

[0054] Figure 2 It is a flowchart of a specific video data fault tolerance method provided by the embodiment of the present application. Refer to Figure 2 As shown, the video data fault tolerance method includes:

[0055] S21: Determine whether all data chunks of the target video data can be read from the distributed storage system.

[0056] In this embodiment, first, the service software will read the target video data from the distributed storage system. When the distributed storage system receives the request, the reading module will immediately start reading data and read the data chunks of each data shard. At this time, it is only necessary to determine whether all data chunks of the target video data can be read from the distributed storage system to determine whether there are faulty nodes.

[0057] S22: If all data chunks can be read, it is determined that there are no faulty nodes, and the read all data chunks are directly sent to the service software.

[0058] In this embodiment, if all data chunks can be read, it is determined that there are no faulty nodes, and the read all data chunks are directly sent to the service software. That is, if the cluster is healthy, there are no node failures, the reading module operates normally, and the data service runs normally according to the standard process. The algorithm module will perform checksum and statistics. If all data blocks can be read, it will directly reply to the service software, and the service software will display the data normally.

[0059] S23: If all data chunks cannot be read, determine that there are faulty nodes, and read the readable data chunks to perform the step of determining the number of faulty nodes.

[0060] In this embodiment, when reading data, all data blocks will be read preferentially. If the number of data blocks is insufficient, that is, if all data chunks cannot be read, determine that there are faulty nodes, and read the readable data chunks to perform the step of determining the number of faulty nodes. The algorithm module reads the parity blocks, and then counts all the data blocks and parity blocks that can be read. Determine the number of faulty nodes based on all the data blocks and parity blocks that can be read, which specifically includes the following steps (as Figure 3 shown):

[0061] S231: Read the readable parity blocks from the distributed storage system, and determine the first sum of quantities of the readable data chunks and the readable parity blocks.

[0062] S232: Calculate the difference between the first sum of quantities and the second sum of quantities of all the data chunks and all the parity blocks corresponding to the target video data, and determine this difference as the number of faulty nodes.

[0063] S233: Calculate the difference between the first sum of quantities and the second sum of quantities of all the data chunks and all the parity blocks corresponding to the target video data, and determine this difference as the number of faulty nodes.

[0064] In this embodiment, if the algorithm module counts that the number of data blocks does not match the actual number, read the readable data chunks, then read the readable parity blocks from the distributed storage system, and determine the first sum of quantities of the readable data chunks and the readable parity blocks. Specifically, count the number of readable data blocks K ’ and the number of readable parity blocks M ’ and their sum L ’ (L ’ = K ’ + M ’ ). Then calculate the difference between the first sum of quantities and the second sum of quantities of all the data chunks and all the parity blocks corresponding to the target video data, and determine this difference as the number of faulty nodes. The second sum of quantities of all the data blocks K and all the parity blocks M is the erasure degree L corresponding to the first erasure mode. Compare L ’ with the stored erasure degree L (L = K + M), and obtain the difference H = L - L ’ . H is the number of faulty nodes.

[0065] S24: Perform mode switching between the first erasure mode and the second erasure mode according to the number of faulty nodes, so as to process the read video data using the switched erasure mode and send it to the service software.

[0066] In this embodiment, the mode is switched between the first erasure mode and the second erasure mode according to the number of faulty nodes, so as to process the read video data by using the switched erasure mode and send it to the service software. The algorithm module makes further operations by comparing the values of H and M, and specifically includes the following steps ( Figure 4 ):

[0067] S241: If the number of faulty nodes is less than the total number of parity blocks, switch to the first erasure mode by means of switch setting.

[0068] S242: Use the first erasure mode to recover the target video data based on the readable data chunks and readable parity blocks, and send the recovered target video data to the service software.

[0069] In this embodiment, if the number of faulty nodes is less than the total number of parity blocks, switch to the first erasure mode by means of switch setting. At this time, use the first erasure mode to recover the target video data based on the readable data chunks and readable parity blocks, and send the recovered target video data to the service software. Specifically, if the number of faulty nodes is less than or equal to M, according to the erasure principle, the reading module calculates the correct data through the existing read data blocks and parity blocks and returns it to the service software to make the service run normally. That is, when H ≤ M, the reading module calculates the correct data according to the read data blocks and parity blocks and returns it to the service software to make the service run normally.

[0070] S243: If the number of faulty nodes is greater than the total number of parity blocks, switch to the second erasure mode by means of switch setting.

[0071] S244: Determine whether the number of faulty nodes is less than the preset erasure redundancy. If so, perform a filling operation on the corresponding video data content, and send the filled video data content to the service software, so that the service software displays the filled video data content coherently.

[0072] In this embodiment, if the number of faulty nodes is greater than the total number of parity blocks, switch to the second erasure mode by means of switch setting. At this time, determine whether the number of faulty nodes is less than the preset erasure redundancy. Specifically, if the number of faulty nodes is greater than M, switch to the second erasure mode. In this mode, further determine whether the number of faulty nodes is less than (or equal to) the preset erasure redundancy. The above preset erasure redundancy is the intelligent erasure redundancy S corresponding to the second erasure mode.

[0073] It can be understood that the preset erasure redundancy is a value between the total number of parity blocks and the second number, that is, M ≤ S ≤ L. As shown before, in this embodiment, a switch module will be set up, and the switch module ensures that the storage reading can be switched between conventional erasure and intelligent erasure fault tolerance. After turning on the switch button, it is necessary to fill in the intelligent erasure redundancy S, and this value will be restricted in the interface, and it needs to be greater than M and less than the value of K + M. Finally, a command is sent to the storage system to enable the intelligent erasure function. The process example is as follows: The first step is to turn on the intelligent erasure function through the switch setting; the second step is to configure the intelligent erasure redundancy and send the configuration to the background. Assume that the current storage erasure rule is 4 + 2, the data block K is 4, the parity block M is 2, and the configured intelligent erasure redundancy S is 4; the third step is to read the data and return the data to the business software. The setting of the preset erasure redundancy can, according to the requirements of the video industry, indirectly expand the erasure redundancy while ensuring the unchanged storage utilization rate, and ensure business continuity.

[0074] In this embodiment, if the number of faulty nodes is less than the preset erasure redundancy, the corresponding video data content is filled up, and the filled-up video data content is sent to the business software, so that the business software can continuously display the filled-up video data content. During the data reading process by the business software, the existing data blocks and parity blocks will be read. If the sum of the readable data blocks K ’ and the readable parity blocks M ’ and the difference H between the sum of K + M and the sum of the readable data blocks and readable parity blocks is greater than M and less than or equal to S, no correct data calculation is performed, and the data content is directly filled up with 0 characters and transmitted to the business software. A certain part of the frame displayed by the business software will be in the form of garbled characters, such as mosaics, so as to maintain business continuity and not affect the overall video viewing. Specifically, when M < H ≤ S, the data content is directly filled up with 0 characters and transmitted to the business software.

[0075] S245: If not, an error message is output to the business software to interrupt the business process of the business software.

[0076] In this embodiment, if the number of faulty nodes is not less than the preset erasure redundancy, an error message is output to the business software to interrupt the business process of the business software. That is, if the number of faulty nodes is greater than the custom intelligent erasure redundancy S, during data reading, the algorithm will make a comparison. When the H value is greater than S, an error will be directly reported to the business software, the data reading fails, and the business is interrupted. Figure 5 The above shows the entire process of the above process.

[0077] See Figure 6 As shown, the embodiment of the present application also correspondingly discloses a video data fault tolerance device, including:

[0078] A mode definition module 11 is configured to define a first erasure mode and a second erasure mode according to service requirements. Among them, the first erasure mode is used for scenarios where conventional erasure code redundancy technology can perform fault tolerance processing, and the second erasure mode is used for scenarios where conventional erasure code redundancy technology cannot perform fault tolerance processing.

[0079] A fault node determination module 12 is configured to determine the number of fault nodes in a distributed storage system storing target video data. Among them, there is an association relationship between the number of fault nodes and the scenario.

[0080] A fault tolerance processing module 13 is configured to perform mode switching between the first erasure mode and the second erasure mode according to the number of fault nodes, so as to perform fault tolerance processing on the read video data using the switched erasure mode and send it to the service software.

[0081] In this embodiment, the mode definition module 11 first defines different erasure modes. This embodiment mainly defines a first erasure mode and a second erasure mode according to service requirements. Among them, the first erasure mode is used for scenarios where conventional erasure code redundancy technology can perform fault tolerance processing, and the second erasure mode is used for scenarios where conventional erasure code redundancy technology cannot perform fault tolerance processing. It can also be understood that the first erasure mode is a conventional erasure mode, and the second erasure mode is a custom intelligent erasure mode with an increased redundancy. Among them, the conventional erasure mode is based on the erasure redundancy rule technology, which divides the stored data into K data blocks, calculates M parity blocks through an encoding algorithm, and uses any K of the K + M data blocks and parity blocks. As long as the sum of the read data blocks and parity blocks is greater than or equal to K, the original data can be restored.

[0082] In this embodiment, the fault node determination module 12 determines the number of fault nodes in a distributed storage system storing target video data. Among them, there is an association relationship between the number of fault nodes and the scenario. It can be understood that when the target video data is stored, it will first be allocated to a certain number of placement groups (PGs), and then stored in different object storage devices (OSDs). Generally, one disk corresponds to one OSD. In this way, a video file will be divided into multiple erasure stripes, and each stripe is allocated in K + M OSDs, and each OSD is on a different node.

[0083] In this embodiment, the fault tolerance processing module 13 switches between the first erasure mode and the second erasure mode according to the number of faulty nodes, so as to process the read video data using the switched erasure mode and send it to the service software. Since there is an association relationship between the number of faulty nodes and the scenario, it is possible to switch between the first erasure mode and the second erasure mode according to the number of faulty nodes. For example, if the number of faulty nodes is less than M, a scenario where the conventional erasure code redundancy technology can perform fault tolerance processing is selected, that is, switched to the first erasure mode; if the number of faulty nodes is not less than M, a scenario where the conventional erasure code redundancy technology cannot perform fault tolerance processing is selected, that is, switched to the second erasure mode.

[0084] In addition, this embodiment sets a switch module. The switch module ensures that the stored video reading is switched between the first erasure mode and the second erasure mode. By turning on the switch of the switch module on the corresponding interface, a smooth mode switch between the first erasure mode and the second erasure mode can be started.

[0085] It can be seen that the embodiment of the present application first defines the first erasure mode and the second erasure mode according to service requirements; where the first erasure mode is used for scenarios where the conventional erasure code redundancy technology can perform fault tolerance processing, and the second erasure mode is used for scenarios where the conventional erasure code redundancy technology cannot perform fault tolerance processing; then determines the number of faulty nodes in the distributed storage system storing the target video data; finally, switches between the first erasure mode and the second erasure mode according to the number of faulty nodes, so as to process the read video data using the switched erasure mode and send it to the service software. The embodiment of the present application defines different erasure modes, and switches different erasure modes in different scenarios according to the number of faulty nodes associated with the scenario, thereby ensuring the continuity of video services while improving storage utilization.

[0086] In some specific embodiments, the video data fault tolerance device further includes:

[0087] A judgment module, configured to judge whether all data chunks of the target video data can be read from the distributed storage system;

[0088] A first determination module, configured to, if all data chunks can be read, determine that there are no faulty nodes and directly send the read all data chunks to the service software;

[0089] A second determination module, configured to, if not all data chunks can be read, determine that there are faulty nodes, and read the readable data chunks to perform the step of determining the number of faulty nodes.

[0090] In some specific embodiments, the second determination module specifically includes:

[0091] A first reading unit, configured to determine that there is a faulty node if all data chunks cannot be read, and read the readable data chunks;

[0092] A second reading unit, configured to read readable parity chunks from the distributed storage system and determine the first sum of the number of readable data chunks and the number of readable parity chunks;

[0093] A calculation unit, configured to calculate the difference between the first sum and the second sum of the number of all data chunks and all parity chunks corresponding to the target video data, and determine the number of faulty nodes as this difference.

[0094] In some specific embodiments, the fault tolerance processing module 13 specifically includes:

[0095] A first switching unit, configured to switch to the first erasure mode by means of switch setting if the number of faulty nodes is less than the number of all parity chunks;

[0096] A recovery unit, configured to recover the target video data based on the readable data chunks and the readable parity chunks by using the first erasure mode, and send the recovered target video data to the service software;

[0097] A second switching unit, configured to switch to the second erasure mode by means of switch setting if the number of faulty nodes is greater than the number of all parity chunks;

[0098] A judgment unit, configured to judge whether the number of faulty nodes is less than a preset erasure redundancy;

[0099] A filling unit, configured to, if so, perform a filling operation on the corresponding video data content, and send the filled video data content to the service software, so that the service software displays the filled video data content continuously;

[0100] An error reporting unit, configured to, if not, output an error message to the service software to interrupt the service process of the service software.

[0101] Furthermore, an embodiment of the present application further provides an electronic device. Figure 7 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure cannot be considered as any limitation to the scope of use of the present application.

[0102] Figure 7Schematic diagram of the structure of an electronic device 20 provided by an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the video data fault tolerance method disclosed in any of the foregoing embodiments.

[0103] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and specific limitations are not imposed here; the input / output interface 25 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application needs, and no specific limitations are made here.

[0104] In addition, as a carrier for resource storage, the memory 22 can be a read-only memory, a random access memory, a disk, or an optical disc, etc. The resources stored thereon can include an operating system 221, a computer program 222, and data 223, etc., and the storage method can be short-term storage or permanent storage.

[0105] Among them, the operating system 221 is used to manage and control each hardware device on the electronic device 20 and the computer program 222 to implement the operation and processing of a large amount of data 223 in the memory 22 by the processor 21. It can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the video data fault tolerance method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks. The data 223 can include the number of faulty nodes collected by the electronic device 20.

[0106] Furthermore, an embodiment of the present application also discloses a storage medium in which a computer program is stored. When the computer program is loaded and executed by a processor, the steps of the video data fault tolerance method disclosed in any of the foregoing embodiments are implemented.

[0107] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0108] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the said element.

[0109] The above has introduced in detail the video data fault tolerance method, device, equipment and storage medium provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A video data fault tolerance method, characterized in that, Including: Define a first erasure mode and a second erasure mode according to service requirements; wherein, the first erasure mode is used for scenarios where conventional erasure code redundancy technology can perform fault tolerance processing, and the second erasure mode is used for scenarios where conventional erasure code redundancy technology cannot perform fault tolerance processing; Determine the number of faulty nodes in the distributed storage system storing the target video data; wherein, there is an association relationship between the number of faulty nodes and the scenario; Perform mode switching between the first erasure mode and the second erasure mode according to the number of faulty nodes, so as to process the read video data using the switched erasure mode and send it to the business software; Wherein, if the number of faulty nodes is less than the total number of parity blocks, switch to the first erasure mode, and recover the target video data based on the readable data chunks and readable parity blocks; If the number of faulty nodes is greater than the total number of parity blocks, switch to the second erasure mode, and determine whether the number of faulty nodes is less than the preset erasure redundancy. If so, perform a filling operation on the corresponding video data content; the preset erasure redundancy is a preset value between the total number of parity blocks and the second number sum.

2. The video data fault tolerance method according to claim 1, wherein Before determining the number of faulty nodes in the distributed storage system storing the target video data, it further includes: Judge whether all data chunks of the target video data can be read from the distributed storage system; If all data chunks can be read, determine that there are no faulty nodes, and directly send all the read data chunks to the business software; If all data chunks cannot be read, determine that there are faulty nodes, and read the readable data chunks to execute the step of determining the number of faulty nodes.

3. The video data fault tolerance method according to claim 2, wherein After reading the readable data chunks, it further includes: Read the readable parity blocks from the distributed storage system, and determine the first number sum of the readable data chunks and the readable parity blocks; Calculate the difference between the first number sum and the second number sum of all data chunks and all parity blocks corresponding to the target video data, and determine this difference as the number of faulty nodes.

4. The video data fault tolerance method according to claim 3, wherein The performing mode switching between the first erasure mode and the second erasure mode according to the number of faulty nodes includes: If the number of faulty nodes is less than the total number of parity blocks, switch to the first erasure mode by means of switch setting; Correspondingly, the processing the read video data using the switched erasure mode and sending it to the business software includes: Use the first erasure mode to recover the target video data based on the readable data chunks and the readable parity blocks, and send the recovered target video data to the business software.

5. The video data fault tolerance method according to claim 3, wherein, The performing mode switching between the first erasure mode and the second erasure mode according to the number of faulty nodes includes: If the number of faulty nodes is greater than the total number of parity blocks, switch to the second erasure mode by means of switch setting; Correspondingly, the processing the read video data using the switched erasure mode and sending it to the business software includes: Determine whether the number of faulty nodes is less than the preset erasure redundancy. If so, perform a complement operation on the corresponding video data content and send the complemented video data content to the business software, so that the business software displays the complemented video data content coherently.

6. The video data fault tolerance method according to claim 5, characterized in that After determining whether the number of faulty nodes is less than the preset erasure redundancy, it further includes: If not, output an error message to the business software to interrupt the business process of the business software.

7. The video data fault tolerance method according to claim 5 or 6, characterized in that The preset erasure redundancy is a value between the total number of parity blocks and the second quantity sum.

8. A video data fault tolerance device, characterized in that, It includes: A mode definition module for defining a first erasure mode and a second erasure mode according to business requirements; wherein, the first erasure mode is used for scenarios where the conventional erasure code redundancy technology can perform fault tolerance processing, and the second erasure mode is used for scenarios where the conventional erasure code redundancy technology cannot perform fault tolerance processing; A faulty node determination module for determining the number of faulty nodes in a distributed storage system storing target video data; wherein there is an association relationship between the number of faulty nodes and the scenario; A fault tolerance processing module for switching between the first erasure mode and the second erasure mode according to the number of faulty nodes, so as to perform fault tolerance processing on the read video data using the switched erasure mode and send it to the business software; Among them, the video data fault tolerance device is further used for: if the number of faulty nodes is less than the total number of parity blocks, switch to the first erasure mode, and recover the target video data based on the readable data chunks and the readable parity blocks; if the number of faulty nodes is greater than the total number of parity blocks, switch to the second erasure mode, and determine whether the number of faulty nodes is less than the preset erasure redundancy. If so, perform a complement operation on the corresponding video data content; the preset erasure redundancy is a preset value between the total number of parity blocks and the second quantity sum.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the video data fault tolerance method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, For storing computer-executable instructions, when the computer-executable instructions are loaded and executed by the processor, the video data fault tolerance method according to any one of claims 1 to 7 is implemented.

Images