A distributed method and system for audio and video data storage and recovery.

Abstract

This invention relates to the field of distributed storage technology, specifically to a distributed audio and video data storage and recovery method and system. The method includes the following steps: based on the audio and video data stream, decomposing the image and analyzing audio changes, extracting and standardizing features, adjusting encoding and grouping compression based on the features, inserting redundant information, allocating data based on node status, establishing mapping information, retrieving and repairing data, and obtaining the audio and video data repair result. This invention establishes a content- and change-oriented compressed redundancy structure through multi-dimensional feature extraction and sequence standardization, refines data block attributes, dynamically adjusts node allocation and resource mapping, constructs a multi-level index and caching optimization mechanism, enhances the flexibility and completeness of data retrieval, improves the targeting and collaborative repair capabilities of the recovery process, ensures seamless connection and complete restoration of audio and video content, and adapts to dynamic fluctuations in nodes and changes in storage resources in a distributed environment.

Description

Technical Field

[0001] This invention relates to the field of distributed storage technology, and in particular to a distributed audio and video data storage and recovery method and system. Background Technology

[0002] Distributed storage involves technologies for sharding, replicating, and coordinating the storage and access of data across multiple physical nodes or devices. It is widely used in large-scale data processing environments, particularly for ensuring high availability, consistency, and efficient access to data. It forms the foundational supporting technology for building cloud computing platforms, edge computing architectures, and massive data application systems. Traditional distributed audio and video data storage and recovery methods involve sharding audio and video data according to time or content characteristics and distributing it across multiple nodes. Recovery is achieved through pre-defined data redundancy strategies in the event of node failure or data loss.

[0003] Existing technologies employing data sharding and redundant storage methods often suffer from low retrieval efficiency, broken connections between segments, and issues such as insufficient integrity, disordered timing, and increased latency during content recovery. This is due to factors like coarse data block attributes, simple mapping mechanisms, and single-layer index structures, especially in scenarios involving node resource allocation and fault response. For example, when distributed surveillance videos experience storage node anomalies or expansion adjustments, some audio and video content is difficult to recover in a timely manner, affecting the continuity of subsequent applications and the efficiency of data utilization. Summary of the Invention

[0004] To address the technical problems existing in the prior art, embodiments of the present invention provide a distributed audio and video data storage and recovery method and system. The technical solution is as follows: On the one hand, a distributed audio and video data storage and recovery method is provided, including the following steps: S1: Based on the audio and video data stream to be processed, the video image is decomposed frame by frame, the spatial features of each frame and the feature differences between adjacent frames are extracted, the temporal changes of the audio signal are analyzed, the feature parameters are sorted and normalized, and the key morphological feature set is obtained. S2: Based on the key morphological feature set, group the audio and video content, adjust the encoding parameters, distinguish key frames from ordinary frames, select the compression method according to the block content, and insert redundant encoding information to obtain a compressed redundant dataset. S3: Based on the compressed redundant dataset, collect the feature labels and access frequency of each data block, analyze the available space and status of each node, and allocate the data blocks to the nodes according to the labels and features to obtain the node distribution mapping information. S4: Based on the node distribution mapping information, extract the unique identifier, node location and redundancy identifier of each data block, establish a data index structure, periodically detect node and data changes, synchronously adjust the index and optimize the cache hierarchy, and obtain the metadata index structure. S5: Based on the metadata index structure, retrieve the data blocks and redundant fragments of the target node, verify the data integrity, and repair the missing fragments by splicing together the existing fragments to obtain the audio and video data repair results.

[0005] On the other hand, the key morphological feature set includes spatial distribution features, audio component features, and temporal alignment features; the compressed redundant dataset includes compressed content units, redundancy check units, and grouping attribute information; the node distribution mapping information includes node allocation relationships, storage topology information, and resource distribution data; the metadata index structure includes index entries, mapping tables, and cache level labels; and the audio and video data repair results include complete video segments, complete audio segments, and recovered data identifiers.

[0006] On the other hand, the specific steps for obtaining the key morphological feature set are as follows: S101: Based on the audio and video data stream to be processed, analyze the image content of each frame in the audio and video data stream, traverse the pixel array sequentially through a sliding window, and process the structural parameters of each local region using convolution based on the boundary shape, texture distribution and brightness change parameters of the region to obtain the frame space feature sequence. S102: Based on the frame space feature sequence, according to the spectral distribution and energy change characteristics of the audio, the parameters of adjacent frames are compared item by item, and the changes in the spectral distribution within the audio segments are analyzed. All differences are normalized and arranged to obtain the temporal variation parameter sequence. S103: Based on the time-varying parameter sequence, determine the order of frame data arrangement, and combine the difference item sorting results to jointly reorganize the frame and audio features at each time point, aggregate the spatial structure and audio component parameters at each time point, and obtain the key morphological feature set.

[0007] On the other hand, the specific steps for obtaining the compressed redundant dataset are as follows: S201: Based on the key morphological feature set, analyze the temporal evolution of each spatial feature and audio feature, and adjust the encoding parameters according to the magnitude of change and content correlation by comparing the continuous changes of feature parameters. Merge similar features into the corresponding parameter configuration sequence to obtain the feature encoding configuration set. S202: Based on the feature encoding configuration set, determine the feature sequence corresponding to each parameter, divide the frame blocks according to the frame type and content change features, and jointly compare the parameters and features of adjacent frames to obtain the content grouping structure; S203: Based on the content grouping structure, filter the structural attributes of each group of data, compare the content characteristics within the group, determine the corresponding compression path, and insert redundant coded segments into each group of data to obtain a compressed redundant dataset.

[0008] On the other hand, the steps for obtaining the node distribution mapping information are as follows: S301: Based on the compressed redundant dataset, analyze the structural description information and access behavior records of each data block, and by identifying content type parameters and access behavior characteristics, summarize the access patterns of the data blocks to obtain a block access behavior feature set. S302: Based on the block access behavior feature set, determine the running status monitoring data of the distributed nodes, compare the task response latency and storage space utilization of each node, match the characteristics of each data block with the node status, and allocate data blocks in sequence according to content activity and node load to obtain distributed block allocation data. S303: Based on the distributed block allocation data, according to the allocation result of each data block, associate the storage node identifier, and combine the synchronization distribution requirements of redundant coded segments, integrate the node positions of the original data blocks and redundant segments to obtain node distribution mapping information.

[0009] On the other hand, the steps for obtaining the metadata index structure are as follows: S401: Based on the node distribution mapping information, analyze the unique identifier, node location and redundancy identifier of each data block, and determine the correspondence between data blocks and nodes by summarizing the identifier parameters of each type, and obtain the index mapping entry table; S402: Based on the index mapping entry table, determine the storage status of the distributed nodes, compare the online status of each node with the data block synchronization status, check the differences between the node record parameters and the index content, perform synchronization adjustment on the asynchronous parts, correct the associated entries, and obtain the index synchronization entry set; S403: Based on the index synchronization entry set, combined with the access frequency of data blocks, high-frequency access entries are filtered, the cache level order is adjusted, and the cache management configuration is associated with the index entries to obtain the metadata index structure.

[0010] On the other hand, the specific steps for obtaining the audio and video data restoration results are as follows: S501: Based on the metadata index structure, analyze the mapping relationship between data blocks and nodes, determine the correspondence between data blocks and node positions in the index entries, and sequentially schedule each node to retrieve the original data blocks and redundant fragments to obtain the node retrieval fragment set; S502: Based on the node, retrieve the fragment set, compare the original data block with the redundant fragment, identify the integrity differences of the data content between the fragments, locate the missing fragments, call the acquired fragment parameters, splice and integrate the missing content, correct the data content, and obtain the fragment splicing and repair set. S503: Based on the fragment splicing and repair set, filter the repaired audio and video fragments, combine them with the identifier parameters generated after data repair, integrate all audio and video data fragments and recovery identifiers, and obtain the audio and video data repair result.

[0011] On the other hand, the spatial features of each frame are a set of attributes that describe the structural contour, texture distribution, object boundaries, and brightness contrast of each frame image. The feature differences refer to the changes in spatial and temporal attributes between consecutive frames or segments.

[0012] On the other hand, the compression method refers to the compression configuration allocated to different blocks, including compression level, fidelity strategy or data processing flow, and the redundant coding information is a data fragment generated on the basis of compressed data.

[0013] On the other hand, a distributed audio and video data storage and recovery system is provided, which is applied to a distributed audio and video data storage and recovery method, including: The feature extraction module decomposes the video image frame by frame based on the audio and video data stream to be processed, extracts the spatial features of each frame and the feature differences between adjacent frames, analyzes the temporal changes of the audio signal, organizes the feature parameters and performs normalization, and obtains the key morphological feature set. The compression redundancy module groups the audio and video content based on the key morphological feature set, adjusts the encoding parameters, distinguishes key frames from ordinary frames, selects the compression method according to the block content, and inserts redundant encoding information to obtain a compressed redundancy dataset. Based on the compressed redundant dataset, the node mapping module collects the feature labels and access frequency of each data block, analyzes the available space and status of each node, and allocates the data blocks to the nodes according to the labels and features to obtain node distribution mapping information. Based on the node distribution mapping information, the index caching module extracts the unique identifier, node location and redundancy identifier of each data block, establishes a data index structure, periodically detects node and data changes, synchronously adjusts the index and optimizes the cache hierarchy, and obtains the metadata index structure. Based on the metadata index structure, the retrieval and repair module retrieves data blocks and redundant fragments of the target node, verifies data integrity, and repairs missing fragments by splicing together existing fragments to obtain audio and video data repair results.

[0014] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: By extracting features from multiple dimensions and standardizing sequences, a compressed and redundant structure oriented towards content and change is established. Data block attributes are refined, node allocation and resource mapping are dynamically adjusted, and a multi-level index and caching optimization mechanism is constructed to enhance the flexibility and completeness of data retrieval, improve the targeting and collaborative repair capabilities of the recovery process, ensure seamless connection and complete restoration of audio and video content, and adapt to the dynamic fluctuations of nodes and changes in storage resources in a distributed environment. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of the main steps of the present invention; Figure 2 This is a flowchart of steps S1 of the present invention; Figure 3 This is a flowchart of steps S2 of the present invention; Figure 4 This is a flowchart of steps S3 of the present invention; Figure 5 This is a flowchart of step S4 of the present invention; Figure 6 This is a flowchart of steps S5 of the present invention; Figure 7 This is a system block diagram of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0018] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0019] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0020] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0021] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0022] This invention provides a distributed audio and video data storage and recovery method, such as... Figure 1 As shown, it includes the following steps: S1: Based on the audio and video data stream to be processed, the video image is decomposed frame by frame, and the spatial features of each frame are extracted by convolution. At the same time, the changes of the audio signal on the time line are analyzed, the feature differences between each frame and adjacent frames are compared, all feature parameters are sorted and the order is normalized to obtain the key morphological feature set. S2: Based on the key morphological feature set, the coding parameters are adjusted according to the feature priority order to locate key frames and ordinary frame segments. The data is divided into multiple blocks according to the temporal changes of audio and video content. Corresponding compression paths are selected for different blocks, and redundant coding segments are inserted into each group of data to obtain a compressed redundant dataset. S3: Based on the compressed redundant dataset, extract the feature labels and access frequency records of each data block, analyze the node status information, compare the available space of the nodes, allocate data blocks to each node according to the data block labels and access characteristics, establish the correspondence structure between data blocks and node identifiers item by item, and obtain the node distribution mapping information. S4: Based on node distribution mapping information, extract the unique identifier, node location and redundancy identifier of each data block, build an index structure, periodically check the node storage status and data changes, and adjust the index structure synchronously if differences are found. At the same time, optimize the cache hierarchy according to the access frequency to obtain the metadata index structure. S5: Based on the metadata index structure, query the corresponding information of data blocks and nodes, sequentially schedule nodes to retrieve original data blocks and redundant segments, compare the integrity of data content, and splice and repair missing parts according to existing segment parameters to obtain audio and video data repair results.

[0023] The key morphological feature set includes spatial distribution features, audio component features, and temporal alignment features. The compressed redundant dataset includes compressed content units, redundancy check units, and grouping attribute information. The node distribution mapping information includes node allocation relationships, storage topology information, and resource distribution data. The metadata index structure includes index entries, mapping tables, and cache level labels. The audio and video data restoration results include complete video segments, complete audio segments, and restored data identifiers.

[0024] In S1, convolution refers to applying sliding window processing to video images frame by frame to extract spatial information such as shape, texture, and color of local regions; each frame's spatial features are a set of attributes describing the structural contour, texture distribution, object boundaries, brightness and darkness contrast of each frame image; changes on the timeline refer to the dynamic parameter changes of audio signals such as frequency, energy, and rhythm over time; feature differences refer to the numerical changes of spatial and temporal attributes between consecutive frames or segments, used to measure the degree of content change; feature parameters are numerical indicators used to express the structure and changes of image and audio content, such as spectral distribution and energy value.

[0025] In S2, feature priority order ranks the feature parameters output from the previous stage according to their importance, determining the order of processing and storage; encoding parameters refer to the relevant technical parameters set in the audio and video compression stage, including bitrate, resolution, frame type allocation, etc., such as H.265, H.264, etc.; temporal changes refer to the dynamic changes of audio and video content during playback, such as scene switching, action rhythm, and pitch fluctuations; multiple chunks refer to dividing data into continuous data units based on content and temporal boundaries, with each unit being a chunk; compression path refers to the compression scheme allocated to different chunks, including compression level, fidelity strategy, and data processing flow; redundant coded segments are additional data segments generated on top of the compressed data, used for subsequent fault tolerance and recovery, including checksums, backup blocks, etc.

[0026] In S3, feature tags are attribute labels attached to each data block, identifying its content type, importance level, access activity, and other characteristics; access frequency records refer to the statistics of the frequency or number of times each data block is accessed, reflecting its usage activity in actual applications; node status information refers to the real-time operating status of distributed storage nodes, including availability, response speed, load status, etc.; node available space refers to the storage capacity currently available for allocation on the storage node, determining the allocation location of new data blocks; and the corresponding structure refers to the data distribution structure table formed by establishing a one-to-one mapping relationship between data blocks and nodes.

[0027] In S4, a unique identifier refers to the data number or identification code assigned to each data block for unique identification and tracking; node location refers to the physical address or logical location description of the distributed node that actually stores the data block; redundancy identifier refers to a special mark used to identify whether a data block is a redundant fragment, erasure block, or backup unit; synchronized index structure adjustment refers to the real-time synchronization and correction of the stored index content when a change in the state of a node or data block is detected; cache hierarchy refers to the multi-level cache areas set up in the distributed system according to the data access frequency, such as high-frequency cache and ordinary cache.

[0028] In S5, the corresponding information refers to the association entries between data blocks and storage nodes recorded in the metadata index structure; the sequential scheduling node refers to the distributed nodes that are instructed to participate in data retrieval and fragment recovery in sequence according to the predetermined recovery order; the redundant fragment refers to the data unit stored separately from the original data to support data recovery and error verification; the existing fragment parameters are the data block content and its technical parameters obtained in the current recovery process, which are used as the basis for reconstructing missing data.

[0029] like Figure 2 As shown, the specific steps for obtaining the key morphological feature set are as follows: S101: Based on the audio and video data stream to be processed, analyze the image content of each frame in the audio and video data stream, traverse the pixel array sequentially through a sliding window, and process the structural parameters of each local region using convolution based on the boundary shape, texture distribution and brightness change parameters of the region to obtain the frame space feature sequence. The input data is divided into a sequence of consecutive image frames. Each frame, after being loaded into memory, is treated as a two-dimensional pixel matrix composed of multiple color channels. A fixed-size sliding window, 5×5 pixels, is then used to cover the entire image, moving one pixel at a time. At an image resolution of 1280×720, this generates approximately 900,000 window regions. The color values ​​of pixels within each window region are extracted. By statistically analyzing the average color value, its range of variation, and the color difference with neighboring pixels, the brightness distribution and boundary changes of the region are evaluated. Furthermore, by calculating the brightness difference between the left and right pixels and the vertical brightness difference within each window region, the presence of image edge features is determined. For example, in a… In a video frame showing a person walking, when the edge of the person passes through a window area, the pixel color difference on both sides of the window increases significantly. This difference can be used to determine the object boundary. Then, image filtering is used to calculate the edge contrast in the horizontal and vertical directions, which are combined to form the local structural features of the area. The structural parameters of all window areas are arranged in order from left to right and from top to bottom, forming the spatial feature sequence of the current frame image. The length of each frame's spatial feature sequence is related to the image size and window movement settings. In the current example, it is approximately 1.2 million local feature points. Each feature point contains 9 image attribute description values, such as brightness value, color difference, texture contrast, etc. All frame spatial features are stored sequentially as the basic data for time series analysis.

[0030] S102: Based on the frame space feature sequence, according to the spectral distribution and energy change characteristics of the audio, the parameters of adjacent frames are compared item by item. At the same time, the changes in the spectral distribution within the audio segments are analyzed, and all differences are normalized and arranged to obtain the temporal variation parameter sequence. Based on the spatial feature sequence of each frame, the audio stream is read at the synchronization moment. The audio is divided into several segments, each corresponding to the time of an image frame. For example, if the video is 25 frames per second, then the audio is divided into segments every 40 milliseconds. Each audio segment is converted into energy distribution data. The spatial features of the images between adjacent frames are compared item by item, and the differences in image brightness, texture, and boundary values ​​of the same area are calculated. The differences are then calculated item by item with the previous frame and combined with the frequency energy change difference of the corresponding audio segment. The two types of differences are merged into a set of fused difference data. To avoid the influence of different units on the results, each set of differences is uniformly processed to keep its values ​​within the same range. Then, the fused difference data of each frame is organized on the time axis. For example, a 5-second video contains 125 frames, which can yield 124 sets of fused difference data. The dimension of each set of data is equal to the total number of differences in the image region and the audio segment. The data can clearly reflect the intensity and area of ​​change of the audio and video content at each time point, providing a reference for subsequent keyframe recognition and temporal structure reconstruction.

[0031] S103: Based on the time-varying parameter sequence, determine the order of frame data, combine the difference item sorting results to jointly reorganize the frame and audio features at each time point, aggregate the spatial structure and audio component parameters at each time point, and obtain the key morphological feature set. Based on the previously obtained temporal difference data sets, the changes in image and audio at each time point are analyzed sequentially. If more than 70% of the difference items in a set of difference data exceed the standard reference value, the current frame is marked as a frame with significant changes and enters the keyframe candidate list. Subsequently, the image structure features and audio spectrum features corresponding to the frame are obtained, and the features of the frames before and after it are added for fusion and recombination. For example, if the current frame 25 is marked as a keyframe, then all image and audio features of frames 23 to 27 are read simultaneously, and the features are averaged item by item to remove local abnormal interference points, forming a stable keyframe feature description block. Finally, the features corresponding to all keyframes that meet the conditions are combined into a key morphological feature set. For example, in an 8-second video clip, if frames 45, 86, and 121 are identified as keyframes, then the key morphological feature set of this segment consists of three sets of image structure and audio energy description data, which are used to provide a basis for priority protection and strategy selection of keyframes during subsequent data compression.

[0032] like Figure 3 As shown, the specific steps for obtaining the compressed redundant dataset are as follows: S201: Based on the key morphological feature set, analyze the temporal evolution of each spatial feature and audio feature. By comparing the continuous changes of feature parameters, adjust the encoding parameters according to the magnitude of change and the correlation with the content. Merge similar features into the corresponding parameter configuration sequence to obtain the feature encoding configuration set. Image spatial features and audio spectral features are matched and organized along the time axis to construct a multi-dimensional feature set at each time point. Then, the change trajectory of each frame's features in consecutive time points is analyzed sequentially. Parameters such as boundary gradient, texture complexity, color contrast, and brightness distribution of each frame are called, and the differences between these parameters and the corresponding positions in adjacent frames are calculated. Parameters with differences greater than a set change benchmark are marked as changed parameters. This benchmark is set as the 75th percentile of the change amplitude in previous samples. For example, for a set of image brightness parameters, a change where the brightness difference between adjacent frames exceeds 30 is considered a significant change. Simultaneously, the fluctuation amplitude of energy in each frequency band of the audio spectrum is analyzed. If the change amplitude of energy values ​​between consecutive frames exceeds a reference threshold of 5, it is considered an audio change point. The image and audio change information are combined by frame index and jointly judged. If three or more image and audio parameters change simultaneously in the current frame, it is marked as a content change frame. Then, a classification operation is performed on all marked frames. If multiple frames have similar combination of changes, they are included in the same type of coding configuration category. In actual video, if the movement of a person causes changes in the image edge and is accompanied by an increase in the audio rhythm, the image boundary gradient and the high-frequency energy of the spectrum will be enhanced synchronously. Such frames are classified into motion configurations. Then, a set of coding parameters is set for each configuration. The configuration with large boundary gradient fluctuations corresponds to a higher inter-frame prediction weight, the configuration with high texture complexity corresponds to a higher bit retention rate, and the configuration with large changes in audio energy corresponds to a lower audio track compression ratio. Through the above classification and parameter selection, the change types of all frames are assigned to different parameter configuration sequences, forming a set of coding configurations merged according to content features.

[0033] S202: Based on the feature encoding configuration set, determine the feature sequence corresponding to each parameter, divide the frame blocks according to the frame type and content change features, and jointly compare the parameters and features of adjacent frames to obtain the content grouping structure; The feature sequence corresponding to each parameter configuration is read, and its structural consistency is determined. The determination process involves sequentially searching for frames with the same parameter configuration marker in the frame sequence and checking whether adjacent frames are continuous in terms of content features. For example, similar image edge directions, continuous textures, brightness distribution differences within 10, and audio spectrum morphology changes less than 5 in the frequency dimension. If the above conditions are met, they are determined to be the same content segment. Subsequently, the frames that meet the conditions are divided into frame blocks in sequence. In a demonstration video, if frames 31 to 45 are all moving content and have the same parameter configuration, they are classified into a motion block. Then, taking each block as a unit, any two frames within it are compared... All image and audio parameters are jointly compared, and the degree of change is recorded item by item. The comparison results are stored in a numerical matrix. Each matrix element represents the degree of difference between two frames in the corresponding parameter. If the average difference in the matrix is ​​lower than a certain unified standard value, the current block is marked as a stable content segment; otherwise, it is considered a transitional content segment. In actual processing, this standard value is set to 20. If the average change value of a block is 18, it is determined to be a content consistent block; if it is 26, it is considered a content jump block. All frame blocks are divided according to their content characteristics. All content consistent blocks and content jump blocks are put into different grouping sequences, thus completing the establishment of the content grouping structure.

[0034] S203: Based on the content grouping structure, filter the structural attributes of each group of data, compare the content characteristics within the group, determine the corresponding compression path, and insert redundant coded segments into each group of data to obtain a compressed redundant dataset. The system sequentially reads the frames and their corresponding parameters contained in each block, analyzes the variation characteristics of image and audio within the block, and averages the spatial distribution attributes of all frames, such as edge density, texture coverage, brightness standard deviation, and audio frequency band energy density, to construct a structural attribute description vector for each group. Then, it compares each group's structural attributes with a preset compression level reference table. The comparison logic is as follows: if the edge density exceeds 80%, the texture coverage is higher than 60%, and the audio frequency band energy density fluctuation range exceeds a set interval, a low compression level compression path is selected; if the edge density is lower than 40%, the texture coverage is lower than 30%, and the energy fluctuation range is less than 5, a high compression level compression path is selected. The compression path is set in three levels: high compression (low quality), medium compression (balanced), and low compression (high quality). For example, a dialogue segment chunk with simple image texture, static background, uniform speech rate, and small audio variation will choose a high compression path, while an action scene chunk will choose a low compression path. Then, redundant coded data is added to the end of the compressed data of each chunk. This redundant segment generates a check segment based on the average features of the group and the compression parameters. The length of the check segment accounts for 10% of the compressed block data. At the same time, an information block with check bits is generated for each group of segments. This information block is used for subsequent data verification and splicing recovery, constructing a compressed redundant dataset consisting of compressed content, check data, and group labels.

[0035] like Figure 4 As shown, the specific steps for obtaining node distribution mapping information are as follows: S301: Based on the compressed redundant dataset, analyze the structural description information and access behavior records of each data block, and summarize the access patterns of the data blocks by identifying content type parameters and access behavior characteristics to obtain the block access behavior feature set. The structural description information attached to each data block is extracted. This information includes the range of image feature parameters, audio energy distribution statistics, compression configuration type marker, content block tag, and compression level identifier. Next, the access behavior records are parsed. These records are the access request logs for each data block within a specified time window. The log content includes access time, request frequency, request source node identifier, and access method type, such as read, modify, or redundancy check. Then, based on the content type parameter in the structural description information, the content category tag corresponding to the current data block is identified, such as dialogue, fast motion, or static background. The number of accesses within the corresponding time interval in the access log is counted and statistically analyzed, and the access frequency is divided into low, medium, and high frequency categories. The system is divided into three levels: medium, high, and medium, corresponding to average access times of less than 5, 5 to 20, and more than 20 per hour, respectively. In a real-world environment, if a block containing close-up shots of a person is requested to play by multiple users simultaneously, and its access record shows 36 accesses within one hour, it is marked as a high-frequency block. Further, indicators such as the average access interval, the number of consecutive accesses, and the distribution of access sources are extracted and combined with content category tags to construct access behavior characteristics. For example, if the average consecutive access interval of this block is only 90 seconds and the access source IPs are concentrated in a certain type of client, it can be classified as a high-frequency, short-interval access pattern. The above steps are repeated to perform corresponding analysis and statistical classification of the structure and behavior of all data blocks, forming a block access behavior characteristic set.

[0036] S302: Based on the block access behavior feature set, determine the running status monitoring data of distributed nodes, compare the task response latency and storage space utilization of each node, match the characteristics of each data block with the node status, and allocate data blocks in sequence according to content activity and node load to obtain distributed block allocation data. The system loads operational status monitoring data for all available storage nodes in the current distributed system. This data includes node CPU utilization, I / O response latency, remaining allocatable storage capacity, network bandwidth utilization, and the task processing queue length of each node over the past hour. Then, it extracts the response latency and storage space utilization for each node. Latency values ​​are calculated in milliseconds and categorized into three levels: less than 50ms, 50-150ms, and greater than 150ms. Space utilization is divided into three levels: less than 40%, 40%-80%, and greater than 80%. Next, it performs node availability assessment. Nodes with a response latency below 150ms and a space utilization of less than 80% are considered allocatable. Finally, it analyzes each data block in the access behavior feature set, based on its access frequency level and operation type ratio. Content activity is scored based on the access interval value. For example, high-frequency blocks are assigned 3 points, medium-frequency blocks are assigned 2 points, and low-frequency blocks are assigned 1 point. Then, the node processing capability score is read. The score is calculated based on the response latency and the current load. For example, a node with a response latency of 45ms and a queue length of less than 5 is scored 5 points. High-activity blocks are prioritized for matching to high-scoring nodes. During the matching process, all data blocks are sorted from high to low access activity, and all nodes are sorted from high to low node scores. Then, the first n high-activity data blocks are sequentially allocated to the first n nodes in the score, ensuring that the allocation of each node does not exceed its available space threshold. Through traversal and loop allocation, all blocks are mapped to nodes to generate a corresponding table structure, resulting in a set of distributed block allocation data that identifies the node to which each data block belongs.

[0037] S303: Based on distributed block allocation data, according to the allocation result of each data block, the storage node identifier is associated, and combined with the synchronization distribution requirements of redundant coded fragments, the node positions of the original data blocks and redundant fragments are integrated to obtain node distribution mapping information; Read the allocation result entry for each data block, extract the block number and corresponding storage node identifier, and establish synchronization distribution rules for the original data blocks and redundant fragments. The rules stipulate that each original block must be stored on the primary node, while its corresponding redundant fragments should be distributed on standby nodes under different network topology paths. This ensures that primary and standby data are not concentrated in the same physical area or the same fault domain. For example, if data block number A001 is allocated to node N05, its redundant fragments, marked A001-R1 ​​and A001-R2, are allocated to nodes N12 and N18 respectively. Nodes N12 and N18 must be located in different subnets at the network layer, or deployed in physically isolated racks or areas. When generating the node allocation structure, it is necessary to determine whether there is address overlap or logical position conflict between each group of original blocks and redundant fragments. If a conflict is found, the node positions of the redundant fragments are readjusted. The adjustment logic is to reselect nodes with low response time and low bandwidth utilization from the remaining available nodes for priority allocation. During the data structure generation stage, a complete entry is generated for each data block. The entry records the main block position, the positions of all redundant fragment nodes and the corresponding node identifiers, and also marks its content category and access feature tags. After being sorted in ascending order by block number, the entries are uniformly summarized and stored in the node mapping table to obtain node distribution mapping information with complete structure, complete entries and compliant redundancy distribution.

[0038] like Figure 5 As shown, the specific steps for obtaining the metadata index structure are as follows: S401: Based on node distribution mapping information, analyze the unique identifier, node location and redundancy identifier of each data block, and determine the correspondence between data blocks and nodes by summarizing the identifier parameters of each type, and obtain the index mapping entry table; Each data block is read sequentially by its unique identifier. This identifier is a structured string encoded according to temporal and content characteristics during data block generation. The prefix corresponds to the content group to which the block belongs, and the suffix represents the timestamp or frame number. For example, the identifier "V05_000123" represents the compressed data block of frame 123 in video group 5. Then, the node location corresponding to this identifier is extracted. This location consists of the node's logical address and actual deployment location. The logical address is like "Node_08," and the deployment location includes information such as rack location and subnet number. Next, the redundancy identifier is read. This field indicates whether the current data block is an original data block or a redundant encoded fragment. The redundancy block identifier... The first method uses formats such as "R1" and "R2". After summarizing the three types of parameters, the corresponding relationship judgment is performed. During the judgment process, the data block number is used as the index key value to establish a one-to-one mapping between the original block and the node. At the same time, it checks whether there are multiple nodes corresponding to the same number. If multiple positions are found to record the same number, the redundant identification field needs to be read. If there are more than two entries with the "original" identification, it is identified as a mapping conflict. The conflict point needs to be recorded and an anomaly is marked. Then, based on the amount of data stored in each node and its set of numbers, a reverse index entry is constructed. The node is used as the primary key to record the set of data blocks it manages, and an index mapping entry table is generated.

[0039] S402: Based on the index mapping entry table, determine the storage status of distributed nodes, compare the online status of each node with the data block synchronization status, check the differences between node record parameters and index content, perform synchronization adjustment for asynchronous parts, correct related entries, and obtain the index synchronization entry set; Perform status read operations on all distributed nodes to check their online status. The criterion is whether the node's heartbeat records are continuous within a specified time window. If there are more than 3 consecutive interruptions, the node is marked as offline. Then, compare the stored data list in the node's local record item by item with the contents of the index mapping entry table. The comparison steps are: reading each data block number corresponding to the node from the index and searching for a completely matching entry in the node's local record. If there is a matching number but inconsistent redundancy markers, or if a data block is missing, it is marked as a synchronization anomaly. For all anomaly entries... The system records the entry number, missing type, missing content location, and timestamp information. Then, it performs a synchronization adjustment operation on the abnormal entry. Specifically, if the original block is missing and there is a redundant block, the original block is recovered from the redundant block and rewritten to the node. If the redundant block is missing, the redundant fragment is rewritten to other available nodes based on the current node status and availability. Redundancy recovery is based on the most recent synchronization time, and only data blocks that have changed after that time point are rewritten. The status flag of the corrected entry is updated and synchronized to the index mapping entry table. All entries that have undergone synchronization processing are constructed into an index synchronization entry set.

[0040] S403: Based on the index synchronization entry set, combined with the access frequency of data blocks, high-frequency access entries are filtered, the cache level order is adjusted, and the cache management configuration is associated with the index entries to obtain the metadata index structure; The system calculates the access frequency of each data block in the entry set. This calculation is based on the access logs recorded in the index structure. It reads the time and source node of each access record, aggregates the access frequency within a specified time window, and sets the aggregation period to 24 hours. The frequency statistics are counted based on the number of times each data block appears. All data blocks are sorted in descending order of access frequency. Entries with an access frequency exceeding 50 times within 24 hours are classified as high-frequency access entries. This high-frequency access data is then linked with the original cache management strategy to determine the current cache level. If the current cache is in the ordinary cache area and the entry is high-frequency, it is adjusted to the high-frequency cache area. During the cache level adjustment process, it is necessary to determine whether the current capacity utilization of the target cache area exceeds the set limit. If it is full, a cache eviction policy is executed, removing low-priority entries according to the least recently used principle before adjustment. The adjustment results are then written to the cache management configuration table, which records information such as the number, cache level identifier, most recently accessed time, and current node number of each cache entry. At the same time, the cache level field in the index entry is updated to be consistent with the actual cache strategy, generating a metadata index structure.

[0041] like Figure 6 As shown, the specific steps for obtaining the audio and video data restoration results are as follows: S501: Based on the metadata index structure, analyze the mapping relationship between data blocks and nodes, determine the correspondence between data blocks and node positions in the index entries, and sequentially schedule each node to retrieve the original data blocks and redundant fragments to obtain the node retrieval fragment set; Extract the data block number field, corresponding node location field, and redundancy identifier field from each index entry. Map the node location field to a combination of physical address and logical node number. Determine whether the data block is an original block or a redundant fragment from the redundancy identifier field. For example, if the data block number in an index entry is "VID_00234", the node number is "Node_11", and the redundancy identifier is "R0", it is determined to be an original block; if it is "R1" or "R2", it corresponds to a redundant fragment. Then, determine the sequence position of the block in the audio / video structure based on the number prefix and frame number. Sort the data blocks in ascending order by data number to form a scheduling task list. For each item in the scheduling list, query the online status record of the corresponding node. If the status is online, send a retrieval request to that node. It includes data number, frame position and content category identifier. Each node returns the corresponding data block or redundant fragment after responding. If there is packet loss or response timeout, it will retry three times within a certain period of time. If the retry fails, it will be recorded as a retrieval failure. The scheduling is performed in the order of index sorting throughout the process. Starting from the first number, it will traverse to the last record. If multiple redundant fragments are encountered with the same data number, the fragments need to be batch scheduled according to the priority queue. The priority setting is based on the node response speed and network bandwidth. For example, nodes with a response time of less than 30ms and bandwidth usage of less than 40% are given priority to be scheduled. All successfully retrieved data blocks and redundant fragments are collected into a unified structure to form a node retrieval fragment set classified by node source, data number and redundancy status.

[0042] S502: Based on node retrieval of fragment sets, compare the original data blocks with redundant fragments, identify the integrity differences of data content between fragments, locate the position of missing fragments, call the acquired fragment parameters, splice and integrate the missing content, correct the data content, and obtain the fragment splicing and repair set; The original blocks and redundant segments are categorized and organized according to their data numbers. Within each group, the content summary information of the original block and all redundant segments is read. This summary information includes data length, intra-frame byte alignment structure, hash checksum, audio segment statistics, and image texture parameter summary. Then, the original blocks and each redundant segment are compared. The comparison process involves checking the starting displacement of each segment, the alignment of image region blocks, and the length of the audio frame. If any field contains a null value, a broken format, or fails verification, it is considered an incomplete data segment. For data groups with discrepancies, the location of the missing segment is first identified, determining whether it is a missing image frame tail, a skipped frame in the middle, or a truncated audio segment. For example, the data with ID VID_00234... The original block image data is 1024 bytes long, while the data returned from the node is 896 bytes long, indicating that 128 bytes are missing at the end. At this point, the tail structure segment is extracted from the corresponding redundant fragment and inserted into the end of the original data to form a reconstructed frame. Then, the data start time of the inserted segment is moved forward to the missing time point according to the original timestamp and parameter record, and the image pixel arrangement order is matched to ensure that the reconstructed image frame is consistent with the structure of the preceding and following frames. If multiple redundant fragments are involved in the stitching process, they are stitched one by one in chronological order. Before insertion, the inter-frame difference is judged. Those with a difference of less than 50 are retained, and those with a difference of less than 50 are discarded. After all the missing fragments are stitched and integrated, a complete frame data is constructed and saved as a repaired state, forming a fragment stitching and repair set.

[0043] S503: Based on the fragment splicing and repair set, filter the repaired audio and video fragments, combine the identifier parameters generated after data repair, integrate all audio and video data fragments and recovery identifiers, and obtain the audio and video data repair result; The repair identifier field is read and all repaired entries are filtered. Entries marked "SUCCESS" or "RECOVERED" in the repair status field are considered valid data. The image frames and audio segments corresponding to each data entry are recombined. The recombining process is performed in chronological order according to the video content. Image data is arranged by frame number from smallest to largest, and audio data is concatenated continuously by timestamp. The data header information is updated using the identifier parameters contained in each repair entry. These identifier parameters include the repair source, redundant segment number, data integrity score, and recovery timestamp. For example, VID_00234 is under repair. If the recovery time in the recovery record is "12:02:43" and the score is 98%, then this identifier record is inserted at the beginning of the data segment to track the recovery source and determine whether the data is the original or spliced ​​version. Then, all image frames are spliced ​​into a complete video segment at a configuration of 30 frames per second, and audio segments within each second are merged into continuous audio segments. The video segments and audio segments are interleaved according to the frame index structure and written to a unified output buffer to generate a new set of complete audio and video data segments. A recovery identifier field is attached to the beginning of each data segment. The entire set of recovered data is uniformly named and archived as the audio and video data repair result.

[0044] like Figure 7 As shown, a distributed audio and video data storage and recovery system includes: The feature extraction module decomposes the video image frame by frame based on the audio and video data stream to be processed, extracts the spatial features of each frame and the feature differences between adjacent frames, analyzes the temporal changes of the audio signal, organizes the feature parameters and performs normalization, and obtains the key morphological feature set. The compression redundancy module groups audio and video content based on key morphological feature sets, adjusts encoding parameters, distinguishes key frames from ordinary frames, selects compression methods according to block content, and inserts redundant encoding information to obtain a compressed redundancy dataset. The node mapping module is based on a compressed redundant dataset. It collects the feature labels and access frequency of each data block, analyzes the available space and status of each node, and allocates data blocks to nodes according to labels and features to obtain node distribution mapping information. Based on node distribution mapping information, the index caching module extracts the unique identifier, node location and redundancy identifier of each data block, establishes a data index structure, periodically detects node and data changes, and synchronously adjusts the index and optimizes the cache hierarchy to obtain the metadata index structure. The retrieval and repair module is based on the metadata index structure. It retrieves data blocks and redundant fragments of the target node, verifies data integrity, and repairs missing fragments by splicing together existing fragments to obtain the audio and video data repair results.

[0045] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A distributed method for storing and recovering audio and video data, characterized in that, The method includes: S1: Based on the audio and video data stream to be processed, the video image is decomposed frame by frame, the spatial features of each frame and the feature differences between adjacent frames are extracted, the temporal changes of the audio signal are analyzed, the feature parameters are sorted and normalized, and the key morphological feature set is obtained. S2: Based on the key morphological feature set, group the audio and video content, adjust the encoding parameters, distinguish key frames from ordinary frames, select the compression method according to the block content, and insert redundant encoding information to obtain a compressed redundant dataset. S3: Based on the compressed redundant dataset, collect the feature labels and access frequency of each data block, analyze the available space and status of each node, and allocate the data blocks to the nodes according to the labels and features to obtain the node distribution mapping information. S4: Based on the node distribution mapping information, extract the unique identifier, node location and redundancy identifier of each data block, establish a data index structure, periodically detect node and data changes, synchronously adjust the index and optimize the cache hierarchy, and obtain the metadata index structure. S5: Based on the metadata index structure, retrieve the data blocks and redundant fragments of the target node, verify the data integrity, and repair the missing fragments by splicing together the existing fragments to obtain the audio and video data repair results.

2. The distributed audio and video data storage and recovery method according to claim 1, characterized in that, The key morphological feature set includes spatial distribution features, audio component features, and temporal alignment features; the compressed redundant dataset includes compressed content units, redundancy check units, and grouping attribute information; the node distribution mapping information includes node allocation relationships, storage topology information, and resource distribution data; the metadata index structure includes index entries, mapping tables, and cache level labels; and the audio and video data repair results include complete video segments, complete audio segments, and recovered data identifiers.

3. The distributed audio and video data storage and recovery method according to claim 1, characterized in that, The specific steps for obtaining the key morphological feature set are as follows: S101: Based on the audio and video data stream to be processed, analyze the image content of each frame in the audio and video data stream, traverse the pixel array sequentially through a sliding window, and process the structural parameters of each local region using convolution based on the boundary shape, texture distribution and brightness change parameters of the region to obtain the frame space feature sequence. S102: Based on the frame space feature sequence, according to the spectral distribution and energy change characteristics of the audio, the parameters of adjacent frames are compared item by item, and the changes in the spectral distribution within the audio segments are analyzed. All differences are normalized and arranged to obtain the temporal variation parameter sequence. S103: Based on the time-varying parameter sequence, determine the order of frame data arrangement, and combine the difference item sorting results to jointly reorganize the frame and audio features at each time point, aggregate the spatial structure and audio component parameters at each time point, and obtain the key morphological feature set.

4. The distributed audio and video data storage and recovery method according to claim 1, characterized in that, The specific steps for obtaining the compressed redundant dataset are as follows: S201: Based on the key morphological feature set, analyze the temporal evolution of each spatial feature and audio feature, and adjust the encoding parameters according to the magnitude of change and content correlation by comparing the continuous changes of feature parameters. Merge similar features into the corresponding parameter configuration sequence to obtain the feature encoding configuration set. S202: Based on the feature encoding configuration set, determine the feature sequence corresponding to each parameter, divide the frame blocks according to the frame type and content change features, and jointly compare the parameters and features of adjacent frames to obtain the content grouping structure; S203: Based on the content grouping structure, filter the structural attributes of each group of data, compare the content characteristics within the group, determine the corresponding compression path, and insert redundant coded segments into each group of data to obtain a compressed redundant dataset.

5. The distributed audio and video data storage and recovery method according to claim 1, characterized in that, The specific steps for obtaining the node distribution mapping information are as follows: S301: Based on the compressed redundant dataset, analyze the structural description information and access behavior records of each data block, and by identifying content type parameters and access behavior characteristics, summarize the access patterns of the data blocks to obtain a block access behavior feature set. S302: Based on the block access behavior feature set, determine the running status monitoring data of the distributed nodes, compare the task response latency and storage space utilization of each node, match the characteristics of each data block with the node status, and allocate data blocks in sequence according to content activity and node load to obtain distributed block allocation data. S303: Based on the distributed block allocation data, according to the allocation result of each data block, associate the storage node identifier, and combine the synchronization distribution requirements of redundant coded segments, integrate the node positions of the original data blocks and redundant segments to obtain node distribution mapping information.

6. The distributed audio and video data storage and recovery method according to claim 1, characterized in that, The specific steps for obtaining the metadata index structure are as follows: S401: Based on the node distribution mapping information, analyze the unique identifier, node location and redundancy identifier of each data block, and determine the correspondence between data blocks and nodes by summarizing the identifier parameters of each type, and obtain the index mapping entry table; S402: Based on the index mapping entry table, determine the storage status of the distributed nodes, compare the online status of each node with the data block synchronization status, check the differences between the node record parameters and the index content, perform synchronization adjustment on the asynchronous parts, correct the associated entries, and obtain the index synchronization entry set; S403: Based on the index synchronization entry set, combined with the access frequency of data blocks, high-frequency access entries are filtered, the cache level order is adjusted, and the cache management configuration is associated with the index entries to obtain the metadata index structure.

7. The distributed audio and video data storage and recovery method according to claim 1, characterized in that, The specific steps for obtaining the audio and video data restoration results are as follows: S501: Based on the metadata index structure, analyze the mapping relationship between data blocks and nodes, determine the correspondence between data blocks and node positions in the index entries, and sequentially schedule each node to retrieve the original data blocks and redundant fragments to obtain the node retrieval fragment set; S5 02: Based on the node, retrieve the fragment set, compare the original data block with the redundant fragment, identify the integrity differences of the data content between the fragments, locate the missing fragments, call the acquired fragment parameters, splice and integrate the missing content, correct the data content, and obtain the fragment splicing and repair set; S503: Based on the fragment splicing and repair set, filter the repaired audio and video fragments, combine them with the identifier parameters generated after data repair, integrate all audio and video data fragments and recovery identifiers, and obtain the audio and video data repair result.

8. The distributed audio and video data storage and recovery method according to claim 1, characterized in that, The spatial features of each frame are a set of attributes that describe the structural contours, texture distribution, object boundaries, and brightness contrast of each frame image. The feature differences refer to the changes in spatial and temporal attributes between consecutive frames or segments.

9. The distributed audio and video data storage and recovery method according to claim 1, characterized in that, The compression method refers to the compression configuration allocated to different blocks, including compression level, fidelity strategy or data processing flow. The redundant encoding information is a data fragment generated on the basis of compressed data.

10. A distributed audio and video data storage and recovery system, the system being used to implement the distributed audio and video data storage and recovery method as described in any one of claims 1-9, characterized in that, The system includes: The feature extraction module decomposes the video image frame by frame based on the audio and video data stream to be processed, extracts the spatial features of each frame and the feature differences between adjacent frames, analyzes the temporal changes of the audio signal, organizes the feature parameters and normalizes them to obtain the key morphological feature set. The compression redundancy module groups the audio and video content based on the key morphological feature set, adjusts the encoding parameters, distinguishes between key frames and ordinary frames, selects the compression method according to the block content, and inserts redundant encoding information to obtain a compressed redundancy dataset. Based on the compressed redundant dataset, the node mapping module collects the feature labels and access frequency of each data block, analyzes the available space and status of each node, and allocates the data blocks to the nodes according to the labels and features to obtain node distribution mapping information. Based on the node distribution mapping information, the index caching module extracts the unique identifier, node location and redundancy identifier of each data block, establishes a data index structure, periodically detects node and data changes, synchronously adjusts the index and optimizes the cache hierarchy, and obtains the metadata index structure. Based on the metadata index structure, the retrieval and repair module retrieves data blocks and redundant fragments of the target node, verifies data integrity, and repairs missing fragments by splicing together existing fragments to obtain audio and video data repair results.

Images