Video data storage and reading method and device

Abstract

The invention relates to the technical field of video storage and reading, in particular to a video data storage and reading method and equipment. The method comprises the following steps: acquiring and analyzing cloud storage video transmission historical data, extracting peak period data and calculating a peak single-user video transmission quantity; predicting the congestion probability of the video transmission network according to the data, and further predicting the load increase of the cloud storage system; through analyzing load increase of a cloud storage system, identifying load abnormity of a server, detecting video transmission response speed attenuation and evaluating attenuation of video storage reading performance, video data storage reading abnormity analysis is carried out according to attenuation data, and a cloud storage video transmission system is optimized. According to the invention, through optimization of the video storage reading method, the accuracy of video storage reading abnormity prediction is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video storage and reading, and in particular to a method and device for video data storage and reading. Background Art

[0002] Video files are usually stored as objects in cloud storage systems, and each video file is accessed through a unique identifier (such as a URL or object ID). Object storage is highly scalable and can efficiently store large-scale, massive video data, supporting the upload, download, and access of large files. In order to meet the storage needs of massive video data, distributed storage technology is adopted. Video data is divided into multiple small blocks and distributed on multiple storage nodes, which are coordinated and managed through network connections. For large-capacity video files, video data is usually fragmented and transmitted in a streaming manner. The fragmentation technology can divide video files into small blocks, increase the transmission speed, and support breakpoint resumption. When reading the video, the client can access the video clips sequentially or randomly as needed. However, a method and device for storing and reading video data has the problem of inaccurate prediction of transmission network congestion anomalies and inaccurate prediction of video transmission server anomalies. Summary of the invention

[0003] Based on this, it is necessary to provide a video data storage and reading method and device to solve at least one of the above technical problems.

[0004] To achieve the above object, a video data storage and reading method comprises the following steps:

[0005] Step S1: Obtain cloud storage video transmission history data; extract the peak period of video data storage and reading according to the cloud storage video transmission history data to obtain video transmission peak period data; calculate the peak single-user video transmission volume according to the video transmission peak period data to obtain peak single-user video transmission volume data;

[0006] Step S2: predicting the probability of video transmission network congestion according to the peak single-user video transmission volume data to obtain video transmission network congestion probability data; predicting the cloud storage system load growth according to the video transmission network congestion probability data to obtain cloud storage system load growth data;

[0007] Step S3: Perform server load anomaly analysis on the cloud storage system load growth data to obtain server load anomaly data; perform video transmission response speed attenuation detection based on the server load anomaly data to obtain video transmission response speed attenuation data; perform video storage read performance attenuation detection based on the video transmission response speed attenuation data and the server load anomaly data to obtain video storage read performance attenuation data;

[0008] Step S4: performing video data storage reading anomaly analysis according to the video storage reading performance attenuation data to obtain video data storage reading anomaly data; optimizing the cloud storage video transmission system for the video data storage reading anomaly data to obtain cloud storage video transmission system optimization data.

[0009] The present invention can accurately extract the peak time period of video transmission, and then calculate the peak single-user video transmission volume. This provides basic data for subsequent prediction and optimization, helps the system identify potential time periods of traffic surge, and prevents network congestion. Based on the analysis of the peak single-user transmission volume, the congestion probability of the video transmission network can be predicted, and the load growth of the cloud storage system can be further calculated, so as to identify the system load pressure in advance and avoid the occurrence of overload. On the basis of load growth, through the abnormal analysis of server load, it is possible to timely find the situation of excessive server resource consumption or uneven load, and then perform video transmission response speed attenuation detection, identify performance bottlenecks, and ensure system stability. Combined with load abnormality data, further video storage reading performance attenuation detection is performed, which helps to find the root cause of storage performance degradation, thereby providing a basis for system optimization. Through abnormal data analysis, problems occurring in the storage reading process can be accurately identified, and corresponding optimization measures are implemented to improve the overall performance of the cloud storage video transmission system and ensure the efficiency and reliability of video transmission. This method not only improves the prediction accuracy of the system, but also realizes efficient monitoring and intelligent optimization in the video storage reading process, providing sustainable technical support for large-scale video data transmission. Therefore, the present invention optimizes a traditional video data storage and reading method and device, solves the problem of inaccurate prediction of transmission network congestion anomalies and inaccurate prediction of video transmission server anomalies in the traditional video data storage and reading method and device, and improves the accuracy of transmission network congestion anomaly prediction and video transmission server anomaly prediction.

[0010] Preferably, step S1 comprises the following steps:

[0011] Step S11: Obtain cloud storage video transmission history data;

[0012] Step S12: performing noise reduction processing on the cloud storage video transmission history data to obtain the historical video transmission noise reduction data;

[0013] Step S13: extracting the peak period of video data storage and reading according to the historical video transmission noise reduction data to obtain the video transmission peak period data;

[0014] Step S14: Calculate the peak single-user video transmission volume according to the video transmission peak time period data to obtain the peak single-user video transmission volume data.

[0015] The present invention can first accurately extract the peak time period of video transmission through a comprehensive analysis of the historical data of cloud storage video transmission, and then calculate the peak single-user video transmission volume. This provides basic data for subsequent prediction and optimization, helps the system identify potential time periods of traffic surge, and prevents network congestion. Based on the analysis of the peak single-user transmission volume, the congestion probability of the video transmission network can be predicted, and the load growth of the cloud storage system can be further calculated, so as to identify the system load pressure in advance and avoid the occurrence of overload. On the basis of load growth, through the abnormal analysis of server load, it is possible to timely find that the server resource consumption is too high or the load is uneven, and then perform video transmission response speed attenuation detection, identify performance bottlenecks, and ensure system stability. Combined with load abnormality data, further video storage read performance attenuation detection is performed, which helps to find the root cause of storage performance degradation, thereby providing a basis for system optimization. Finally, through abnormal data analysis, problems occurring in the storage reading process can be accurately identified, and corresponding optimization measures can be implemented to improve the overall performance of the cloud storage video transmission system and ensure the efficiency and reliability of video transmission. This method not only improves the prediction accuracy of the system, but also realizes efficient monitoring and intelligent optimization in the video storage reading process, providing sustainable technical support for large-scale video data transmission.

[0016] Preferably, step S14 comprises the following steps:

[0017] Step S141: Counting the number of online users during the peak period according to the video transmission peak period data to obtain the number of online users during the peak period;

[0018] Step S142: Perform video storage volume statistics on the video transmission peak time period data to obtain peak video storage volume data;

[0019] Step S143: performing video reading volume statistics on the video transmission peak time period data to obtain peak video reading volume data;

[0020] Step S144: calculating the video reading volume of a single user according to the number of online users during the peak period and the peak video reading volume data, and obtaining the video reading volume data of a single user;

[0021] Step S145 calculates the single-user video storage capacity according to the number of online users during the peak period and the peak video storage capacity data to obtain the single-user video storage capacity data;

[0022] Step S146: Calculate the peak single-user video transmission volume based on the single-user video storage volume data and the single-user video reading volume data to obtain the peak single-user video transmission volume data.

[0023] The present invention can clarify the scale of high concurrent access by counting the number of online users during peak hours, and help identify the pressure points of the system when the traffic surges. Statistics on video storage and video reading help understand the demand for storage and reading resources in the system during peak hours, thereby evaluating the carrying capacity of the system capacity. These data are crucial for optimizing cloud storage resource allocation, and can guide how to arrange storage and bandwidth resources more reasonably to avoid bottlenecks. Further calculating the video reading and storage volume of a single user can accurately quantify the specific needs of each user during peak hours, and provide a more detailed performance optimization reference. Finally, by comprehensively analyzing the storage and reading volume of a single user and calculating the peak single-user video transmission volume, the overall performance of the system under high concurrency can be evaluated and potential overload risks can be identified. This series of steps helps to predict the system load in advance from the perspective of user behavior, and provides comprehensive data support for optimizing the cloud storage system, thereby ensuring high efficiency and stability in the process of video data transmission and storage.

[0024] Preferably, step S2 comprises the following steps:

[0025] Step S21: evaluating the high concurrent requests of the cloud storage system according to the peak single-user video transmission volume data to obtain the high concurrent request data of the cloud storage system;

[0026] Step S22: performing bandwidth bottleneck detection according to the high concurrent request data of the cloud storage system to obtain video transmission bandwidth bottleneck data;

[0027] Step S23: predicting the video transmission network congestion probability based on the cloud storage system high-concurrency request data and the video transmission bandwidth bottleneck data to obtain video transmission network congestion probability data;

[0028] Step S24: Predict the cloud storage system load growth based on the video transmission network congestion probability data and the video transmission bandwidth bottleneck data to obtain the cloud storage system load growth data.

[0029] The present invention evaluates the high concurrent requests of the cloud storage system based on the peak single-user video transmission volume data, and can identify the concentrated time periods of a large number of requests in advance, thereby providing a basis for subsequent bandwidth bottleneck detection. Next, through bandwidth bottleneck detection, the bandwidth limitation in transmission is accurately captured, helping the system to optimize resource allocation and avoid video transmission delay or interruption caused by bandwidth bottleneck. In addition, by combining high concurrent request data with bandwidth bottleneck data, the probability of video transmission network congestion is predicted, and the network congestion risk can be identified in advance to avoid jamming or packet loss during video transmission. Based on the congestion probability and bandwidth bottleneck prediction data, the cloud storage system load growth prediction can be carried out, and the load change trend of the system under high concurrency can be accurately estimated, providing data support for timely adjustment of resource allocation and optimization of load balancing, thereby improving the responsiveness and stability of the system in high concurrent video transmission scenarios. This series of steps not only improves the efficiency of cloud storage video transmission, but also provides an important early warning mechanism for future traffic surges and system optimization.

[0030] Preferably, step S22 includes the following steps:

[0031] Step S221: performing request volatility analysis based on the high-concurrency request data of the cloud storage system to obtain request volatility data of the cloud storage system;

[0032] Step S222: performing request frequency increase calculation on the cloud storage system request volatility data to obtain video transmission request frequency increase data;

[0033] Step S223: Calculate the bandwidth occupancy rate growth according to the video transmission request frequency increase data to obtain the bandwidth occupancy rate growth data of the cloud storage system;

[0034] Step S224: performing multi-stream concurrent statistics according to the high-concurrency request data of the cloud storage system to obtain multi-stream concurrent data of the cloud storage system;

[0035] Step S225: Perform bandwidth bottleneck detection based on the multi-stream concurrent data of the cloud storage system and the bandwidth occupancy growth data of the cloud storage system to obtain video transmission bandwidth bottleneck data.

[0036] The present invention performs request volatility analysis to identify the fluctuation pattern of the request volume, provide the system with more accurate traffic prediction, and help identify potential load peaks. Next, by calculating the increase in request frequency, the growth trend of video transmission requests can be quantified, thereby more accurately predicting the bandwidth pressure that the system will face in the future. Based on these frequency growth data, the growth of bandwidth occupancy rate is further calculated, which can reveal the use of bandwidth resources in advance, ensure that the system can adjust resources in time, and avoid transmission delays or bottleneck problems. At the same time, through multi-stream concurrent statistics, the load situation when the system processes multiple video streams at the same time can be accurately captured, providing data support for bandwidth management and optimization, and combining multi-stream concurrent data and bandwidth occupancy rate growth data for bandwidth bottleneck detection, which can fully identify the existing bandwidth bottleneck problems, such as expanding bandwidth or adjusting traffic distribution, thereby maintaining the smooth operation of the system. Overall, these steps effectively improve the system's ability to respond to high-concurrency video requests, and ensure the smoothness of video transmission and the efficiency of cloud storage.

[0037] Preferably, step S23 includes the following steps:

[0038] Step S231: performing statistics on the number of requests surge according to the high-concurrency request data of the cloud storage system to obtain data on the number of video transmission requests surge;

[0039] Step S232: performing a centralized detection of the request density according to the data on the surge in the number of video transmission requests, and obtaining centralized data on the density of video transmission requests;

[0040] Step S233: performing transmission throughput overload prediction on the concentrated data of video transmission request density and the data on the surge in the number of video transmission requests to obtain transmission throughput overload data of the cloud storage system;

[0041] Step S234: predicting network delay increase based on the cloud storage system transmission throughput overload data and the video transmission bandwidth bottleneck data to obtain network delay increase data;

[0042] Step S235: performing video transmission request backlog prediction according to the cloud storage system transmission throughput overload data and the video transmission bandwidth bottleneck data to obtain video transmission request backlog data;

[0043] Step S236: Predict the probability of video transmission network congestion based on the video transmission request backlog data and the network delay increase data to obtain video transmission network congestion probability data.

[0044] The present invention can identify the peak period of sudden increase in the number of requests in advance by counting the surge in the number of requests, help the system better predict traffic changes, and avoid system overload caused by sudden requests. Then, the centralized detection of request density can accurately locate the request concentration period and area, provide data support for subsequent traffic scheduling, thereby avoiding uneven allocation of system resources or the generation of bottlenecks. By combining the surge in the number of requests and the dense request density, the overload prediction of the transmission throughput is carried out, and the throughput overload problem of the system can be identified in advance, and corresponding optimization measures such as load balancing or bandwidth expansion are taken to avoid transmission delays or failures caused by overload. Based on the throughput overload data and bandwidth bottleneck information, the network delay increase prediction is carried out, which is helpful to judge the growth trend of network delay, help operators or technical personnel to take measures in time, and reduce the impact of delay on video transmission quality. Further, by combining the transmission throughput overload and bandwidth bottleneck data to carry out request backlog prediction, the risk of request backlog can be identified in advance, video request accumulation can be avoided, and the user experience is affected. Finally, the prediction of network congestion probability based on the combination of request backlog and delay increase data helps to comprehensively evaluate the probability of system congestion, help the system to schedule and optimize resources in advance, and ensure the smooth operation of video transmission. These steps effectively improve the system's predictive ability, enabling it to make timely adjustments and optimizations under high concurrency conditions, ensuring the smoothness and efficiency of the video transmission process.

[0045] Preferably, step S3 comprises the following steps:

[0046] Step S31: performing server load anomaly analysis on the cloud storage system load growth data to obtain server load anomaly data;

[0047] Step S32: Perform video transmission response speed attenuation detection according to the server load abnormality data to obtain video transmission response speed attenuation data;

[0048] Step S33: performing storage read operation failure prediction on the server load abnormality data to obtain server storage read operation failure data;

[0049] Step S34: performing a video file data loss assessment on the server storage read operation failure data to obtain cloud storage video file loss data;

[0050] Step S35: Perform video storage read performance attenuation detection according to the cloud storage video file loss data and the video transmission response speed attenuation data to obtain video storage read performance attenuation data.

[0051] The present invention can timely discover abnormal situations that occur in servers under high load conditions by performing server load anomaly analysis on load growth data, and provide early warning for subsequent performance optimization. Next, video transmission response speed attenuation detection is performed based on server load anomaly data, which helps to evaluate the response time change of video transmission when the server load is too heavy, thereby predicting the performance degradation and ensuring that the response speed will not significantly decay when the load increases. Further, by analyzing the server load anomaly data, the storage read operation failure is predicted, and storage problems such as disk failure or data read and write conflicts can be discovered early, avoiding video data loss or transmission interruption due to storage operation failure. Based on this, the prediction of video file loss in combination with storage read operation failure data can accurately identify the potential risk of video file loss in the system, and timely take measures such as data backup and recovery to ensure the security and integrity of video files. By combining video file loss data and transmission response speed attenuation data to perform storage read performance attenuation detection, the performance change of the cloud storage system under high load can be comprehensively evaluated, the decline trend of storage read performance can be predicted, and guidance can be provided for optimizing storage system performance. Overall, this series of steps not only helps to improve the stability and responsiveness of the cloud storage system, but also ensures the security of video data and reduces the risk of service interruption due to excessive system load or storage failure.

[0052] Preferably, step S31 includes the following steps:

[0053] Step S311: Calculate the growth of storage resource consumption according to the cloud storage system load growth data to obtain storage resource consumption growth data;

[0054] Step S312: performing server load balancing failure prediction according to the storage resource consumption growth data and the cloud storage system load growth data to obtain server load balancing failure data;

[0055] Step S313: Perform server overload detection according to the server load balancing failure data to obtain server overload data;

[0056] Step S314: performing server continuous cumulative load analysis on the server overload data to obtain server continuous cumulative load data;

[0057] Step S315: performing server crash frequency growth prediction on the server continuous accumulated load data and server load balancing failure data to obtain server crash frequency growth data;

[0058] Step S316: Perform server load anomaly analysis based on the server crash frequency growth data and the server overload data to obtain server load anomaly data.

[0059] The present invention can accurately understand the changing trend of cloud storage system resource usage by calculating the growth of storage resource consumption, and provide a basis for system expansion or optimization of resource configuration. Next, the server load balancing failure prediction is performed in combination with the storage resource consumption and load growth data, and the load balancing failure problem can be identified in advance when the server load is too heavy, so as to prevent the server from being overloaded due to failure to effectively distribute the load. Further, by performing overload detection on the server load balancing failure data, it is possible to timely find out which servers in the system exceed their carrying capacity, thereby avoiding failures or delays caused by server overload. Continuous cumulative load analysis of overloaded servers can provide an in-depth understanding of the accumulation of loads and help technicians identify which servers are at risk of crashing under long-term loads. Combined with these analysis data, the growth prediction of server crash frequency can be performed, which servers can be evaluated in advance which servers will frequently crash in the future, and necessary measures can be taken to reduce the frequency of system downtime. By analyzing the crash frequency growth and overload data, server load abnormality analysis is performed, which helps to fully identify the causes of load abnormalities and provide data support for taking optimization and adjustment strategies, thereby reducing service interruptions and performance degradation caused by load abnormalities. Overall, this series of steps provides a comprehensive load management and optimization strategy for the cloud storage system, ensuring that the system can efficiently and stably handle large-scale video data storage and reading needs.

[0060] Preferably, step S35 includes the following steps:

[0061] Step S351: collecting the encoding format loss data according to the cloud storage video file loss data to obtain the cloud storage video encoding format loss data;

[0062] Step S352: Perform video file damage detection according to the cloud storage video encoding format loss data to obtain video file damage data;

[0063] Step S353: Predicting video definition attenuation according to the video file damage data to obtain video definition attenuation data;

[0064] Step S354: performing video playback screen freeze detection according to the video file damage data and the video transmission response speed attenuation data to obtain video playback screen freeze data;

[0065] Step S355: Perform video storage read performance degradation detection based on the video playback screen freeze data and the video clarity degradation data to obtain video storage read performance degradation data.

[0066] The present invention collects the loss of coding format based on the loss data of cloud storage video files, can identify and capture the loss of coding format caused by storage or transmission errors, and provide preliminary diagnosis for subsequent data recovery and format conversion. Next, by detecting the video file damage of the loss data of the coding format, it can be timely discovered whether the video file in the storage cannot be read normally due to damage, reducing the situation that the user cannot play or the file is damaged during the playback process. Based on the video file damage data, the video clarity attenuation prediction can evaluate the risk of video quality attenuation, judge in advance the image quality degradation of the video during playback, and ensure that the system can take appropriate repair measures to ensure clarity. Further, by combining the video file damage data and the video transmission response speed attenuation data for screen freeze detection, it can accurately predict whether the screen freeze phenomenon caused by data damage or transmission delay will occur during video playback, thereby optimizing the video transmission path or buffering mechanism, reducing the impact of freeze on user experience, and combining the screen freeze and clarity attenuation data for video storage read performance attenuation detection, which helps to fully understand the reading performance of the video storage system under high load or abnormal state, timely discover potential performance degradation problems, and take effective optimization measures to ensure the smoothness of video transmission and file quality. These steps ensure the stability and quality of video files during storage, transmission, and playback, provide comprehensive technical guarantees, and reduce the risk of service interruption or degradation of user experience.

[0067] The present invention also provides a video data storage and reading device, which is used to execute the video data storage and reading method as described above, and the video data storage and reading device comprises:

[0068] Video transmission volume calculation module: obtain cloud storage video transmission history data; extract video data storage and reading peak time period according to cloud storage video transmission history data, and obtain video transmission peak time period data; calculate peak single-user video transmission volume according to video transmission peak time period data, and obtain peak single-user video transmission volume data;

[0069] Storage system load growth prediction module: predicts the probability of video transmission network congestion based on the peak single-user video transmission volume data to obtain video transmission network congestion probability data; predicts the cloud storage system load growth based on the video transmission network congestion probability data to obtain cloud storage system load growth data;

[0070] Video storage read performance attenuation detection module: perform server load anomaly analysis on the cloud storage system load growth data to obtain server load anomaly data; perform video transmission response speed attenuation detection based on the server load anomaly data to obtain video transmission response speed attenuation data; perform video storage read performance attenuation detection based on the video transmission response speed attenuation data and server load anomaly data to obtain video storage read performance attenuation data;

[0071] Cloud storage video transmission system optimization module: perform video data storage reading anomaly analysis based on video storage reading performance attenuation data to obtain video data storage reading anomaly data; optimize the cloud storage video transmission system based on the video data storage reading anomaly data to obtain cloud storage video transmission system optimization data.

[0072] The present invention is that the peak time period of video transmission can be accurately extracted, and then the peak single-user video transmission volume can be calculated. This provides basic data for subsequent prediction and optimization, helps the system identify potential time periods of traffic surge, and prevents network congestion. Based on the analysis of the peak single-user transmission volume, the congestion probability of the video transmission network can be predicted, and the load growth of the cloud storage system can be further calculated, so as to identify the system load pressure in advance and avoid the occurrence of overload. On the basis of load growth, through the abnormal analysis of server load, it is possible to timely find that the server resource consumption is too high or the load is uneven, and then perform video transmission response speed attenuation detection, identify performance bottlenecks, and ensure system stability. Combined with load abnormality data, further video storage read performance attenuation detection is performed, which helps to find the root cause of storage performance degradation, thereby providing a basis for system optimization. Finally, through abnormal data analysis, problems occurring in the storage reading process can be accurately identified, and corresponding optimization measures can be implemented to improve the overall performance of the cloud storage video transmission system and ensure the efficiency and reliability of video transmission. This method not only improves the prediction accuracy of the system, but also realizes efficient monitoring and intelligent optimization in the video storage reading process, providing sustainable technical support for large-scale video data transmission. Therefore, the present invention optimizes a traditional video data storage and reading method and device, solves the problem of inaccurate prediction of transmission network congestion anomalies and inaccurate prediction of video transmission server anomalies in the traditional video data storage and reading method and device, and improves the accuracy of transmission network congestion anomaly prediction and video transmission server anomaly prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A schematic diagram of the steps of a video data storage and reading method;

[0074] Figure 2 for Figure 1Detailed implementation steps of step S2 in the flowchart;

[0075] Figure 3 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0076] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0077] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0078] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different network and / or processor methods and / or microcontroller methods.

[0079] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0080] To achieve this, please refer to Figures 1 to 3 , a video data storage and reading method, comprising the following steps:

[0081] Step S1: Obtain cloud storage video transmission history data; extract the peak period of video data storage and reading according to the cloud storage video transmission history data to obtain video transmission peak period data; calculate the peak single-user video transmission volume according to the video transmission peak period data to obtain peak single-user video transmission volume data;

[0082] Step S2: predicting the probability of video transmission network congestion according to the peak single-user video transmission volume data to obtain video transmission network congestion probability data; predicting the cloud storage system load growth according to the video transmission network congestion probability data to obtain cloud storage system load growth data;

[0083] Step S3: Perform server load anomaly analysis on the cloud storage system load growth data to obtain server load anomaly data; perform video transmission response speed attenuation detection based on the server load anomaly data to obtain video transmission response speed attenuation data; perform video storage read performance attenuation detection based on the video transmission response speed attenuation data and the server load anomaly data to obtain video storage read performance attenuation data;

[0084] Step S4: performing video data storage reading anomaly analysis according to the video storage reading performance attenuation data to obtain video data storage reading anomaly data; optimizing the cloud storage video transmission system for the video data storage reading anomaly data to obtain cloud storage video transmission system optimization data.

[0085] In the embodiment of the present invention, reference Figure 1 FIG. 1 is a schematic diagram of a process flow of a video data storage and reading method of the present invention. In this example, the video data storage and reading method comprises the following steps:

[0086] Step S1: Obtain cloud storage video transmission history data; extract the peak period of video data storage and reading according to the cloud storage video transmission history data to obtain video transmission peak period data; calculate the peak single-user video transmission volume according to the video transmission peak period data to obtain peak single-user video transmission volume data;

[0087] In an embodiment of the present invention, the video transmission history data of the cloud storage system is obtained. The historical data includes key indicators such as user requests, data transmission volume, request delay, bandwidth occupancy, etc. of the cloud storage system in different time periods. The way to obtain historical data is usually through log analysis tools, such as log collection systems (such as ELK Stack) or through API interfaces to obtain the operation data of the cloud platform. After the data is extracted, the time series data of the video transmission is screened to extract the peak transmission period, that is, the period when the user's video request volume and transmission data volume reach the highest in certain specific time periods. This process identifies the time window with the highest data transmission volume (such as 18:00 to 22:00 every day, or special holidays, etc.) by performing time series analysis on the historical data. Next, based on the peak period data, the video transmission volume of a single user during the peak period is calculated. By aggregating the video transmission volume during the peak period and dividing it according to each user, the average video transmission volume of each user during the peak period is calculated.

[0088] Step S2: predicting the probability of video transmission network congestion according to the peak single-user video transmission volume data to obtain video transmission network congestion probability data; predicting the cloud storage system load growth according to the video transmission network congestion probability data to obtain cloud storage system load growth data;

[0089] In an embodiment of the present invention, based on the peak single-user video transmission volume data, the probability of video transmission network congestion is predicted. This prediction is calculated based on the relationship between the single video transmission volume of the user during the peak period and the current network bandwidth. In this step, the probability of network congestion at a certain point in the future needs to be estimated by combining factors such as bandwidth usage, video transmission load data, and network delay with actual traffic data. The adopted technology includes bandwidth capacity and load prediction analysis, and commonly used methods include linear regression and time series analysis. On this basis, the growth of the cloud storage system load is further predicted based on the congestion probability data of the video transmission network. The key to predicting the load growth of the cloud storage system is to identify the increase in the system load after the congestion period. The system load growth prediction needs to rely on historical load data and network status changes to calculate the load increments in different time periods. The load growth trend is usually determined by fitting historical load changes.

[0090] Step S3: Perform server load anomaly analysis on the cloud storage system load growth data to obtain server load anomaly data; perform video transmission response speed attenuation detection based on the server load anomaly data to obtain video transmission response speed attenuation data; perform video storage read performance attenuation detection based on the video transmission response speed attenuation data and the server load anomaly data to obtain video storage read performance attenuation data;

[0091] In the embodiment of the present invention, when the server load anomaly analysis is performed on the cloud storage system load growth data, the main purpose is to detect the mutation of the historical load data and identify the abnormal conditions such as overload and overload in the system. The specific manifestation of the load anomaly can be that the CPU or memory usage of the server increases abnormally within a certain period of time, or that the disk I / O frequently encounters bottlenecks. In specific implementation, it can be judged by setting a threshold value. When the load of the server exceeds a preset limit, it is determined to be a load anomaly. At this time, the server load anomaly data can be collected and processed by system monitoring tools (such as Prometheus, Zabbix, etc.), and further the attenuation detection of the video transmission response speed is performed. If the load anomaly is detected, the response speed of the video transmission will be affected, the response time will increase, and the video loading speed will slow down. Therefore, by collecting performance indicators such as response time, the video transmission delay is detected using performance analysis tools (such as Wireshark, tcpdump) to identify the attenuation trend. Combined with the load anomaly data and the response speed data, the video storage read performance attenuation detection can be further performed. If the response delay caused by the server load anomaly is detected, it is further inferred that the read performance of the video storage is reduced, resulting in a backlog of data requests or even packet loss. At this point, it is necessary to predict performance degradation by analyzing indicators such as the storage system's read request queue length and average read latency.

[0092] Step S4: performing video data storage reading anomaly analysis according to the video storage reading performance attenuation data to obtain video data storage reading anomaly data; optimizing the cloud storage video transmission system for the video data storage reading anomaly data to obtain cloud storage video transmission system optimization data.

[0093] In an embodiment of the present invention, it is necessary to aggregate the reading logs of video data to analyze the time periods and causes of abnormal situations. If the storage read performance decays, it will manifest as video file reading failures, long response delays, and other phenomena. The analysis of storage read abnormal data can rely on log analysis tools (such as Splunk) to automatically identify the time period when the problem occurs and calculate the corresponding abnormality rate. For the detected anomalies, targeted optimization measures are taken to identify the source of performance bottlenecks, such as insufficient network bandwidth, storage device performance problems, etc. By implementing the optimization of the cloud storage video transmission system, the system performance is optimized by methods such as load balancing, cache optimization, and storage tiered management, which can effectively alleviate the decay of video storage read performance under high load conditions. The optimization plan also includes dynamically adjusting the allocation of storage resources, automatically adjusting the storage location of data, caching strategies, or expanding storage resources based on the system load.

[0094] Preferably, step S1 comprises the following steps:

[0095] Step S11: Obtain cloud storage video transmission history data;

[0096] Step S12: performing noise reduction processing on the cloud storage video transmission history data to obtain the historical video transmission noise reduction data;

[0097] Step S13: extracting the peak period of video data storage and reading according to the historical video transmission noise reduction data to obtain the video transmission peak period data;

[0098] Step S14: Calculate the peak single-user video transmission volume according to the video transmission peak time period data to obtain the peak single-user video transmission volume data.

[0099] In an embodiment of the present invention, historical video transmission data is obtained by calling the API interface of the cloud storage platform or from the storage log system. These historical data include user request information, file size of transmission, transmission timestamp, delay during video playback, bandwidth occupancy, and real-time fluctuations of network bandwidth. In order to ensure the integrity and timeliness of the data, historical data with sufficient duration should be selected, covering at least 1 month or longer. The data collection tool can use tools such as Elasticsearch, Logstash, Kibana, etc. or customized data collection scripts to ensure that the historical data is comprehensive and not missing. After obtaining the data, the data is stored in a distributed storage system, such as Hadoop HDFS or a cloud-based database, and the obtained historical video transmission data is subjected to noise reduction processing, and standard signal processing methods are used to reduce the interference of existing data anomalies or noise on subsequent analysis results. In specific implementation, the transmission data is smoothed, and a moving average method or a low-pass filter can be used to eliminate short-term fluctuations and retain long-term trend data. If the data contains obvious outliers, such as abnormally high or low single transmission data, it can be processed by IQR (interquartile range method) or Z-Score standardization method to identify and delete these outliers. To ensure the processing effect, the noise reduction process needs to process each item of the time series data independently to retain the overall trend and periodic fluctuations of the data. Use the pandas library in Python or a specialized data processing tool (such as Apache Spark) to batch process the data, and smooth the transmission data of each period to obtain the "historical video transmission noise reduction data". Based on the processed historical video transmission noise reduction data, extract the peak time period for video data storage and reading, and define the standard for the peak time period. In actual operation, the threshold can be set according to the trend of the transmission volume. For example, when the transmission volume in a certain period exceeds 95% of the total network transmission volume, the period is regarded as the peak period. At this time, the time series analysis method is used to cluster the noise-reduced data according to date and time to identify the time period when the transmission volume increases sharply. In order to improve the accuracy, combined with the volatility of data traffic, the sliding window method can be used to segment the data and calculate the total transmission volume of each period at different time granularities such as hourly and minutely. By comparing the transmission volume of each time period, the data of the peak period is automatically marked. This process uses Python's NumPy, pandas library or Hadoop-based distributed processing framework to perform data segmentation and analysis, and extract the time period data representing the peak of video transmission. After extracting the video transmission peak period data, the peak single-user video transmission volume is calculated, and for each peak period, the request data of each user active in the period is analyzed. In specific implementation, users can be grouped according to their IP address or device ID, and the total video transmission volume of each user during the peak period is calculated.The transmission volume can include indicators such as the byte size and reading time of the video file. Then, calculate the average video transmission volume for each user. If there are many users in a period of time, you can first calculate the total transmission volume of all users, and then divide it by the number of users to get the average transmission volume of a single user during the peak period. In this process, you need to pay attention to the accuracy of the data, especially for users with multiple concurrent requests, double counting should be avoided. Through data analysis platforms (such as Hive, Presto, etc.) or custom calculation scripts, you can calculate the user's transmission volume in parallel in a distributed environment, and finally obtain the peak single-user video transmission volume data.

[0100] Preferably, step S14 comprises the following steps:

[0101] Step S141: Counting the number of online users during the peak period according to the video transmission peak period data to obtain the number of online users during the peak period;

[0102] Step S142: Perform video storage volume statistics on the video transmission peak time period data to obtain peak video storage volume data;

[0103] Step S143: performing video reading volume statistics on the video transmission peak time period data to obtain peak video reading volume data;

[0104] Step S144: calculating the video reading volume of a single user according to the number of online users during the peak period and the peak video reading volume data, and obtaining the video reading volume data of a single user;

[0105] Step S145 calculates the single-user video storage capacity according to the number of online users during the peak period and the peak video storage capacity data to obtain the single-user video storage capacity data;

[0106] Step S146: Calculate the peak single-user video transmission volume based on the single-user video storage volume data and the single-user video reading volume data to obtain the peak single-user video transmission volume data.

[0107] In the embodiment of the present invention, the data of the peak period of video transmission is first analyzed in detail to extract the online information of users in each period. Specifically, each user who is active during the peak period of video transmission is identified through the data log system. The user activity can be identified by the IP address, device ID, or the login status of the user account to ensure that each active user is accurately counted. The login and logout time of each user is recorded using a timestamp, and the online time of the user during the peak period is determined based on this information. Then, by counting the number of users who are online at the same time in each period, the number of users online in each peak period is obtained. This process can use a distributed computing framework (such as Apache Spark) for parallel computing to quickly process large-scale data sets to ensure the efficiency and accuracy of data statistics. Finally, the number of online users during the peak period is obtained, the size and storage time of each video file are determined, and then the storage capacity of each video is calculated one by one according to the data during the peak period. Specifically, the system needs to obtain detailed information of the video files transmitted during the peak period from the storage system, including file name, storage location, file size, storage time, etc. Based on this information, the total file size in each period is summarized to obtain the video storage capacity data during the peak period. At this time, a distributed data processing framework, such as Hadoop MapReduce or Spark, is used to quickly summarize and calculate the storage volume in large-scale data sets. In this process, it is also necessary to ensure that the video file storage records during the peak period are not missed. Through data deduplication and integrity check, the video reading volume statistics are performed for the data during the peak period of video transmission. This process requires identifying the reading of all video files during each peak period, including the number of times each user reads the video file and the size of each read. Through log analysis, the reading records of each user for the video file are obtained, and the recorded content includes the reading time, video file ID, the read file size, and the reading duration. By counting these reading records, the total video reading volume in each period is obtained. In order to improve efficiency, a streaming processing framework (such as Apache Flink) can be used to process real-time data streams and perform efficient reading volume statistics in real-time systems. By summarizing the total amount of all video reads, the total video reading volume data during the peak period is finally obtained. According to the number of online users and peak video reading data during peak hours, the video reading volume of a single user is calculated, and the number of online users during the peak hours obtained in step S141 is used as the denominator; and the peak video reading volume data obtained in step S143 is used as the numerator. The video reading volume of a single user during each peak hour is equal to the video reading volume during that hour divided by the number of online users. For example, assuming that the video reading volume is 10TB during a certain period of time, and there are 5,000 online users during that period of time, the video reading volume of a single user is 10TB / 5,000 people = 2GB / user.The calculations in this process can be implemented by writing custom scripts based on statistical tools (such as Python, R, etc.). To ensure the accuracy of the calculation, the data of each peak period can be filtered and verified to avoid the influence of outliers on the calculation results. According to the number of online users obtained in step S141, combined with the peak video storage data counted in step S142, the storage capacity of a single user during the peak period is calculated. The specific calculation method is: divide the total video storage capacity of the period by the number of online users. For example, if the video storage capacity is 50TB in a certain period of time, and the number of online users in the period is 5000, the video storage capacity of a single user is 50TB / 5000=10GB / user. In order to improve the accuracy of the calculation, the storage capacity data can be obtained through database query tools (such as SQL, Hive, etc.), and relevant division calculations can be performed. The single-user video storage data finally obtained is combined with the single-user video storage data and the single-user video reading data to calculate the peak single-user video transmission volume. The single-user video reading volume and the single-user video storage volume data obtained in step S144 and step S145 are weighted averaged to obtain the transmission volume of each user during the peak period. Specifically, the single-user video transmission volume is equal to the sum of the reading volume and storage volume of the user. For example, if the single-user video reading volume of a user is 2GB and the single-user video storage volume is 10GB, the single-user video transmission volume of the user is 2GB+10G=12GB. This process is obtained through a simple addition operation. For large-scale data sets, the data of each user can be parallelized through a batch processing framework (such as Apache Spark) to ensure the accuracy of the results and the efficiency of the calculation. The peak single-user video transmission volume data is finally obtained.

[0108] Preferably, step S2 comprises the following steps:

[0109] Step S21: evaluating the high concurrent requests of the cloud storage system according to the peak single-user video transmission volume data to obtain the high concurrent request data of the cloud storage system;

[0110] Step S22: performing bandwidth bottleneck detection according to the high concurrent request data of the cloud storage system to obtain video transmission bandwidth bottleneck data;

[0111] Step S23: predicting the video transmission network congestion probability based on the cloud storage system high-concurrency request data and the video transmission bandwidth bottleneck data to obtain video transmission network congestion probability data;

[0112] Step S24: Predict the cloud storage system load growth based on the video transmission network congestion probability data and the video transmission bandwidth bottleneck data to obtain the cloud storage system load growth data.

[0113] As an example of the present invention, refer to Figure 2As shown, in this example, step S2 includes:

[0114] Step S21: evaluating the high concurrent requests of the cloud storage system according to the peak single-user video transmission volume data to obtain the high concurrent request data of the cloud storage system;

[0115] In an embodiment of the present invention, peak single-user video transmission volume data is collected and analyzed to determine the transmission volume of each user during the peak period. Based on this data, the number of concurrent requests during the peak period is calculated. The number of concurrent requests is usually determined by the request frequency of video data, the number of users, and the video transmission volume of each user. The number of simultaneous requests in each period can be calculated by analyzing the request timestamps and user identification information recorded in the log file. Use efficient log analysis tools, such as Elasticsearch or Logstash, to automatically extract and count request data, thereby obtaining the number of high concurrent requests for the cloud storage system.

[0116] Step S22: performing bandwidth bottleneck detection according to the high concurrent request data of the cloud storage system to obtain video transmission bandwidth bottleneck data;

[0117] In an embodiment of the present invention, the bandwidth bottleneck in the video transmission process is identified and determined. The bandwidth bottleneck is usually caused by the number of concurrent requests exceeding the carrying capacity of the network bandwidth, so it is necessary to monitor the bandwidth usage of the network link. Use a network traffic monitoring tool (such as Wireshark, NetFlow or SNMP monitoring system) to obtain the bandwidth usage of each time period in the network, especially the peak time period. By calculating the bandwidth usage per second during the video transmission process and comparing it with the total bandwidth of the cloud storage system, it can be determined which time periods have bandwidth bottlenecks. If the bandwidth usage of certain time periods is close to or exceeds the maximum bandwidth limit of the system, it means that there is a bandwidth bottleneck in this time period.

[0118] Step S23: predicting the video transmission network congestion probability based on the cloud storage system high-concurrency request data and the video transmission bandwidth bottleneck data to obtain video transmission network congestion probability data;

[0119] In an embodiment of the present invention, the probability of video transmission network congestion is predicted according to the high concurrent request data of the cloud storage system and the video transmission bandwidth bottleneck data. The core of this step is to estimate the probability of network congestion in the future period of time by analyzing the existing concurrent request data and bandwidth bottleneck. In combination with the peak concurrent request data and bandwidth bottleneck data, statistical methods or time series analysis methods (such as ARIMA models) can be used to predict the data. For example, the relationship between the historical bandwidth usage and the request volume can be used to model and predict the probability of bandwidth overload. Based on this model, the probability of video transmission network congestion under different load conditions is obtained. In addition, the hotspot nodes in the network can be further analyzed in combination with the flow prediction algorithm and the network topology structure, and the congestion situation of each node in the future is estimated. The video transmission network congestion probability data finally obtained by all these analysis results.

[0120] Step S24: Predict the cloud storage system load growth based on the video transmission network congestion probability data and the video transmission bandwidth bottleneck data to obtain the cloud storage system load growth data.

[0121] In an embodiment of the present invention, a prediction model for the load growth of a cloud storage system is established by combining the probability data of video transmission network congestion with the data of bandwidth bottleneck. The model mainly considers two factors: one is the limitation of network transmission bandwidth, and the other is the growth of video transmission demand. Specifically, based on the previous network congestion probability data, the carrying capacity of the current bandwidth and transmission load can be evaluated, and how the load of the cloud storage system will grow during future peak hours can be predicted. Through mathematical modeling (such as linear regression or exponential smoothing methods), combined with historical load growth trends, future bandwidth requirements and system loads can be predicted. At the same time, using bandwidth bottleneck data as input parameters, the bandwidth requirements under different loads can be accurately estimated. This process can be processed using data mining tools (such as scikit-learn in Python) to obtain accurate load growth prediction data.

[0122] Preferably, step S22 includes the following steps:

[0123] Step S221: performing request volatility analysis based on the high-concurrency request data of the cloud storage system to obtain request volatility data of the cloud storage system;

[0124] Step S222: performing request frequency increase calculation on the cloud storage system request volatility data to obtain video transmission request frequency increase data;

[0125] Step S223: Calculate the bandwidth occupancy rate growth according to the video transmission request frequency increase data to obtain the bandwidth occupancy rate growth data of the cloud storage system;

[0126] Step S224: performing multi-stream concurrent statistics according to the high-concurrency request data of the cloud storage system to obtain multi-stream concurrent data of the cloud storage system;

[0127] Step S225: Perform bandwidth bottleneck detection based on the multi-stream concurrent data of the cloud storage system and the bandwidth occupancy growth data of the cloud storage system to obtain video transmission bandwidth bottleneck data.

[0128] In an embodiment of the present invention, the cloud storage system records the number of user requests, request timestamps, and request types in each time period. By performing time series analysis on these data, the fluctuation pattern of the request volume can be identified. For example, the historical data is divided into time windows by hours or minutes, and the request volume in each window is summed and the standard deviation is calculated. By comparing the request volume in different time periods, the volatility of system requests can be identified. If the number of requests increases significantly in a certain period, it indicates that there is a fluctuation in the request volume in this period. The volatility analysis can be performed through programming languages ​​(such as the Pandas library in Python) to process data, and the fluctuation data can be displayed through visualization tools (such as Matplotlib or Excel charts) to help identify high-volume periods. The increment of the request volume in each time period is extracted from the obtained request volatility data, and then the increase in frequency is calculated. For example, by comparing the difference in the number of requests in adjacent time periods, the increment of the request frequency in each time period is obtained. If the increase in the request volume in a certain period exceeds a certain threshold, it can be determined as a period of increased frequency. In order to more accurately calculate the increase in request frequency, the difference method of time series can be used to calculate the change in request frequency between the previous and next time periods. This calculation can be implemented through programming tools such as NumPy or Pandas libraries in Python. Through further statistical analysis, the time period with a sharp increase in request frequency can be identified. Based on the size of each video transmission request and the data of the increase in request frequency, the system's bandwidth demand during this period can be estimated. For example, for each request, the size of the data (such as the resolution and frame rate of the video) can be recorded, and then the bandwidth occupied by each request can be calculated. By multiplying the bandwidth demand in each period by the frequency increment, the growth data of the bandwidth occupancy rate can be obtained. This process needs to be evaluated by calculating the bandwidth demand, latency requirements and actual bandwidth usage of the request. In order to calculate the specific value of bandwidth occupancy, network bandwidth monitoring tools (such as Wireshark, Nagios, etc.) can be used to collect real-time data, and the growth trend of bandwidth usage can be further calculated through data analysis tools. For high concurrent requests in each period, the number of concurrent requests of all requests in that period is counted. By analyzing the maximum number of requests that the cloud storage system can process simultaneously in a certain period, the concurrent processing capacity of the system can be determined. The specific method is to perform a timing analysis on the requests in each time period, calculate the maximum number of requests in each time period, record the situation of multiple streams requesting at the same time, and identify the intensive time period of concurrent requests. If the number of concurrent requests is large, it indicates that the time period faces a large processing pressure. Furthermore, the concurrency of the requests can be analyzed through the flow control mechanism to determine whether there are multiple request streams occupying the bandwidth concurrently. Combined with the multi-stream concurrency data obtained in step S224 and the bandwidth occupancy rate growth data obtained in step S223, bandwidth bottleneck analysis can be performed.Specifically, by counting the concurrent data of multiple streams, the bandwidth usage of multiple stream requests in each time period is determined, and the data is compared with the bandwidth usage growth data. If multiple request streams occur at the same time and the bandwidth demand grows too fast, it can be determined as a period of bandwidth bottleneck. In order to detect bottlenecks, the bandwidth usage can be compared with the maximum bandwidth carrying capacity. If the bandwidth usage exceeds the set threshold, it indicates that the system is about to or has already encountered a bandwidth bottleneck. In addition, the existence of transmission bottlenecks can be further detected by real-time monitoring of the relationship between bandwidth utilization, bandwidth usage and network delay data.

[0129] Preferably, step S23 includes the following steps:

[0130] Step S231: performing statistics on the number of requests surge according to the high-concurrency request data of the cloud storage system to obtain data on the number of video transmission requests surge;

[0131] Step S232: performing a centralized detection of the request density according to the data on the surge in the number of video transmission requests, and obtaining centralized data on the density of video transmission requests;

[0132] Step S233: performing transmission throughput overload prediction on the concentrated data of video transmission request density and the data on the surge in the number of video transmission requests to obtain transmission throughput overload data of the cloud storage system;

[0133] Step S234: predicting network delay increase based on the cloud storage system transmission throughput overload data and the video transmission bandwidth bottleneck data to obtain network delay increase data;

[0134] Step S235: performing video transmission request backlog prediction according to the cloud storage system transmission throughput overload data and the video transmission bandwidth bottleneck data to obtain video transmission request backlog data;

[0135] Step S236: Predict the probability of video transmission network congestion based on the video transmission request backlog data and the network delay increase data to obtain video transmission network congestion probability data.

[0136] In an embodiment of the present invention, a surge in the number of requests is counted based on the high concurrent request data of the cloud storage system, and the high concurrent request data of each time period is collected, including the number of video requests in each time period and the timestamp of each request. These request data are statistically analyzed to calculate the change trend of the number of requests. In a specific operation, the request data is divided into multiple time windows by hours or minutes, and the difference between the number of requests in each time window and the historical average level is compared. When the number of requests in a certain period is significantly higher than the historical average, it indicates that a surge in the number of requests has occurred in that period. This process can calculate the fluctuation of the number of requests in each period through a time series analysis method in a programming language, such as the Pandas library in Python, and display the period of abnormal fluctuation through a chart. The surge in the number of requests data obtained from step S231 is combined with the distribution of requests in each period to further analyze the density concentration of the requests. The density concentration reflects whether the requests are too concentrated in a specific period and region. To this end, it is necessary to perform a cluster analysis on the request distribution of each period to identify the period and region where the requests are highly concentrated. For example, the data with a surge in the number of requests is grouped according to the geographical area or request type where the request occurs, and the concentration of requests in different areas or types is calculated. For periods with highly concentrated request frequency and density, the periods with high request concentration can be identified by calculating the clustering coefficient or using the density peak method. Through density concentration detection, the periods that cause excessive bandwidth or server load can be identified in advance. By combining the data of the surge in the number of requests with the data of the concentration of request density, it is analyzed which periods have too fast request growth and concentrated request distribution. On this basis, by calculating the total number of requests in each period and the bandwidth required for each request, it is predicted whether the bandwidth and transmission throughput of the system can meet the demand. For periods with a sharp increase in the number of requests and high concentration, it is necessary to evaluate whether the system can effectively carry these requests. In specific operations, real-time traffic prediction can be performed on the request data, and compared with the historical throughput data of the system to check whether the current system throughput is close to the limit. If the throughput of certain periods is expected to exceed the maximum load of the system, the risk of throughput overload can be predicted. The transmission throughput overload data is combined with the bandwidth bottleneck data to analyze the trend of network delay when the throughput approaches or exceeds the maximum capacity of the system. By monitoring the bandwidth utilization and throughput data, when the system approaches the bandwidth bottleneck, the network delay will increase significantly. In order to predict the change of delay, the queuing theory model can be used to analyze the delay change of the request during the transmission process, or the relationship between the actual bandwidth usage and the delay can be fitted and analyzed. Specifically, when the bandwidth usage reaches a certain critical value, the increase of network delay can be predicted based on historical data or real-time monitoring data. Network performance monitoring tools such as PingPlotter or Traceroute can be used to monitor the delay changes in real time and discover potential delay problems in advance.Combined with the throughput overload data and bandwidth bottleneck data, the queue situation of request processing is analyzed. If the system bandwidth reaches a bottleneck, video requests will be backlogged due to insufficient processing capacity. By monitoring the queue length, processing speed and request processing time of the system, it is possible to predict which time periods or areas will have backlogs. When calculating the number of backlog requests, factors such as the arrival frequency of requests, server response time and processing speed are considered. For the period when the system throughput is close to the limit, the time period and the amount of requests for backlog can be calculated through monitoring data. Combined with the video transmission request backlog data obtained from step S235 and the network delay increase data obtained from step S234, the probability of video transmission network congestion is predicted. When the request backlog reaches a certain level and the network delay continues to increase, the system will face the risk of network congestion. To this end, by establishing a relationship model between network throughput, delay and request backlog, the correlation between request backlog and network congestion is analyzed. Through regression analysis of historical data, the probability of network congestion in a specific period or a specific request mode is predicted. For example, if the network delay continues to grow in certain time periods and the request backlog increases significantly, the probability of network congestion in that period can be inferred. This prediction model can be used for real-time prediction and early warning through traffic analysis tools in network monitoring systems, such as SolarWinds or Zabbix.

[0137] Preferably, step S3 comprises the following steps:

[0138] Step S31: performing server load anomaly analysis on the cloud storage system load growth data to obtain server load anomaly data;

[0139] Step S32: Perform video transmission response speed attenuation detection according to the server load abnormality data to obtain video transmission response speed attenuation data;

[0140] Step S33: performing storage read operation failure prediction on the server load abnormality data to obtain server storage read operation failure data;

[0141] Step S34: performing a video file data loss assessment on the server storage read operation failure data to obtain cloud storage video file loss data;

[0142] Step S35: Perform video storage read performance attenuation detection according to the cloud storage video file loss data and the video transmission response speed attenuation data to obtain video storage read performance attenuation data.

[0143] As an example of the present invention, refer to Figure 3 As shown, in this example, step S3 includes:

[0144] Step S31: performing server load anomaly analysis on the cloud storage system load growth data to obtain server load anomaly data;

[0145] In an embodiment of the present invention, server load data of the cloud storage system in different time periods are collected, including information such as CPU usage, memory occupancy, disk read and write speed, and network bandwidth. Time series analysis is performed on these data to identify patterns of abnormal loads. For example, by setting a load threshold, when the load in a certain period exceeds the normal range, it is considered abnormal. In specific operations, open source tools such as Prometheus can be used for data collection and monitoring, combined with Grafana for data visualization, to help intuitively display load fluctuations. By analyzing the load change trend, the period of abnormal load can be identified.

[0146] Step S32: Perform video transmission response speed attenuation detection according to the server load abnormality data to obtain video transmission response speed attenuation data;

[0147] In an embodiment of the present invention, based on the abnormal server load data obtained in step S31, the change in server response time is further analyzed. Specifically, it is analyzed whether the response time of the video transmission request is delayed when the server load reaches an abnormal level. When the server load is abnormal, the processing speed is reduced, thereby affecting the transmission speed of the video data. This step can evaluate the degree of attenuation of the response speed by comparing the response time under normal load and abnormal load. Use network performance monitoring tools such as Wireshark, PingPlotter, etc. to monitor the delay of video transmission in real time. For video transmission requests during abnormal load, if the response time increases significantly, it is marked as video transmission response speed attenuation.

[0148] Step S33: performing storage read operation failure prediction on the server load abnormality data to obtain server storage read operation failure data;

[0149] In an embodiment of the present invention, the failure of storage read operations of a server under abnormal load is analyzed. By collecting the read success rate and failure rate of the storage system under normal load and abnormal load, the probability of storage read operation failure under abnormal load is calculated. This process can be implemented by comparing the storage operation logs under normal and abnormal loads. The success and failure status of each read request is recorded in the log of the storage system. By calculating the proportion of failed requests, when the load is abnormal, the failure rate of storage read operations will increase, thereby predicting the occurrence of storage read operation failures.

[0150] Step S34: Predicting video file data loss for the failed data of the server storage read operation to obtain cloud storage video file loss data;

[0151] In an embodiment of the present invention, by collecting and analyzing relevant data of storage read operation failures, situations that cause video file loss during the reading process are identified. Storage read failures cause some video files to be unable to be correctly read or damaged, thereby causing data loss. By analyzing failed read requests, the probability of video file loss in a storage system under abnormal load conditions is calculated. For example, when the storage read failure rate reaches a certain threshold, it can be inferred that some data has not been successfully stored or read, resulting in video file loss. The risk of file loss can be evaluated in combination with the backup mechanism and fault tolerance mechanism of the storage system.

[0152] Step S35: Perform video storage read performance attenuation detection according to the cloud storage video file loss data and the video transmission response speed attenuation data to obtain video storage read performance attenuation data.

[0153] In an embodiment of the present invention, the video file loss data is combined with the response speed attenuation data to analyze the performance degradation of video storage reading under abnormal load, data loss and response delay. If the video file loss rate increases and the response speed decreases during the abnormal load, the overall storage reading performance will be attenuated. This analysis can be performed by continuously monitoring the real-time performance data of the video storage system, combining the lost files and delayed transmission requests, and calculating the performance degradation ratio of the video storage system under abnormal load conditions. Server and storage performance monitoring is performed through tools such as Nagios, Zabbix, etc., and combined with the analysis results, the trend of storage performance degradation can be discovered in a timely manner, and early warning of storage system failures can be issued.

[0154] Preferably, step S31 includes the following steps:

[0155] Step S311: Calculate the growth of storage resource consumption according to the cloud storage system load growth data to obtain storage resource consumption growth data;

[0156] Step S312: performing server load balancing failure prediction according to the storage resource consumption growth data and the cloud storage system load growth data to obtain server load balancing failure data;

[0157] Step S313: Perform server overload detection according to the server load balancing failure data to obtain server overload data;

[0158] Step S314: performing server continuous cumulative load analysis on the server overload data to obtain server continuous cumulative load data;

[0159] Step S315: performing server crash frequency growth prediction on the server continuous accumulated load data and server load balancing failure data to obtain server crash frequency growth data;

[0160] Step S316: Perform server load anomaly analysis based on the server crash frequency growth data and the server overload data to obtain server load anomaly data.

[0161] In an embodiment of the present invention, the load growth data of each storage node in the cloud storage system is collected, especially the indicators related to the consumption of storage resources, including the utilization rate of disk space, I / O load, cache usage, etc. Based on these data, the consumption growth of storage resources is calculated by comparing the changes in storage resources in different time periods. For example, if the storage resource consumption growth exceeds a predetermined threshold in a certain time period, it can be inferred that the storage consumption growth caused by the increase in system load. The server load balancing failure prediction is performed based on the storage resource consumption growth data and the cloud storage system load growth data. According to the storage resource consumption growth data obtained in the previous step, combined with the load growth trend of the system, the effectiveness of the server load balancing mechanism is evaluated. By analyzing the relationship between load growth and storage resource consumption, it is found that when the load reaches a certain critical point, the server load balancing fails. For example, when the increase in storage resource consumption causes some nodes to be overloaded, the load balancing strategy fails to effectively distribute the load, which will cause some servers to be overloaded. To this end, historical data is analyzed and the signs of load balancing failure are identified by calculating the ratio of the load growth rate to the resource consumption growth rate. According to the load balancing failure data in step S312, the load status of the server is further analyzed. Load balancing failure often causes some servers to bear too high a load, thus triggering an overload phenomenon. Overload detection monitors the server's CPU usage, memory usage, network bandwidth, and disk I / O performance indicators to identify whether the server is in an overloaded state. When the server load exceeds the set safety threshold, it will be judged as overloaded. Use monitoring tools (such as Prometheus, Grafana, Collectd, etc.) to monitor the various performance indicators of the server in real time, and regularly compare them with the set thresholds to ensure that overload phenomena can be identified in a timely manner. By analyzing the server's load data in a continuous time period, you can identify the pattern of continuous load increase. For example, if the load of a server continues to increase without being processed and optimized in a timely manner, its stability will decrease and eventually crash. By collecting various server load data (such as CPU load, memory usage, storage I / O, etc.) and calculating the cumulative load, analyze its long-term change trend. Use time series analysis tools (such as Grafana, InfluxDB, etc.) to process these data graphically and observe the load trend to identify whether there is a continuous load growth. Combined with the server load trend analyzed in step S314 and the load balancing failure data found in step S312, calculate the increase in the frequency of server crashes. The increase in server crash frequency is usually related to load growth and load balancing failure. When the load continues to increase and the load balancing mechanism cannot be adjusted in time, the server crash frequency will increase significantly. To this end, by establishing a crash frequency model, combined with historical load balancing failure records and load data, the future crash frequency is predicted.The model can use statistical analysis methods (such as regression analysis) or prediction algorithms based on historical data. By comparing the crash frequency in different time periods, the probability of server crashes under different load conditions is predicted. The probability of server load anomalies will be evaluated, especially when the load continues to grow, load balancing fails, and the server is overloaded. By analyzing the crash data and overload data at different load levels, a prediction of server load anomalies is obtained. To achieve this analysis, statistical methods (such as standard deviation, outlier detection algorithms, etc.) can be used to identify the threshold of load anomalies. When the load exceeds a certain threshold, the server load is considered abnormal, and the system can issue a warning signal or automatically take load balancing optimization measures. Data can be collected and analyzed by load monitoring tools (such as Nagios, Zabbix, Prometheus) to ensure timely response when load anomalies occur.

[0162] Preferably, step S35 includes the following steps:

[0163] Step S351: collecting the encoding format loss data according to the cloud storage video file loss data to obtain the cloud storage video encoding format loss data;

[0164] Step S352: Perform video file damage detection according to the cloud storage video encoding format loss data to obtain video file damage data;

[0165] Step S353: Predicting video definition attenuation according to the video file damage data to obtain video definition attenuation data;

[0166] Step S354: performing video playback screen freeze detection according to the video file damage data and the video transmission response speed attenuation data to obtain video playback screen freeze data;

[0167] Step S355: Perform video storage read performance degradation detection based on the video playback screen freeze data and the video clarity degradation data to obtain video storage read performance degradation data.

[0168] In an embodiment of the present invention, metadata of all video files in a cloud storage system is collected, and information such as the storage location, size, format, and encoding type of the files is analyzed. By comparing the actual storage of the video file with the expected stored data, the lost encoding format information is identified. This process can be performed by scanning the header information of the video file. In a specific implementation, the FFmpeg tool can be used to read the metadata of the video file, extract the encoding format information, and compare it with the metadata in the database. When it is found that the encoding format information is missing or inconsistent, it is marked as encoding format missing data. It is determined whether the relevant video file is damaged. Video file damage detection can be performed by checking the integrity of the video file and whether the decoding is normal. In this process, a video analysis tool (such as FFmpeg) is used to perform a decoding test on the video. If an error or frame loss occurs during the decoding process, it can be determined that the video file is damaged. By comparing the checksum (such as MD5, SHA-256, etc.) of the damaged video file and comparing it with the checksum during storage, it is further confirmed whether the file is damaged. If the file is damaged, the damaged data, including the degree of damage, location, and affected area, is recorded, the information of the damaged area is obtained through the damage detection step, and the impact of the damaged location on the video quality is analyzed. Image processing algorithms (such as image sharpness analysis and contrast measurement) are used to evaluate the impact of damaged video areas. For severely damaged video files, the picture is blurred or artifacts appear. Image quality evaluation methods such as SSIM (Structural Similarity Index) or PSNR (Peak Signal-to-Noise Ratio) can be used to predict the impact of damage on video clarity. Based on these evaluation results, the degree of video clarity attenuation is calculated and clarity attenuation data is generated. Based on the video file damage detection results, it is determined whether the video file has a potential problem of playback jamming. Secondly, combined with the attenuation data of the video transmission response speed, the delay of video playback is analyzed. When the network bandwidth is insufficient or the video data is not transmitted in time, the video playback will experience a jamming phenomenon. By comparing the degree of damage to the video file and the attenuation of the transmission response speed, it is detected whether there is a jamming phenomenon during video playback. Video jamming detection can be achieved by monitoring the frame rate, delay and packet loss rate during video playback. Using tools such as Wireshark or TCPDump to analyze the video data packets in real time, detect whether there is a jamming caused by data packet loss or network delay. The attenuation of video storage read performance is usually manifested as jamming and image quality degradation during video playback. In this step, the performance of the storage system when reading video data is analyzed in combination with the clarity attenuation data and the jamming detection data in steps S353 and S354. By monitoring indicators such as the reading speed, cache hit rate, and I / O load of video files, it is detected whether the storage system has performance attenuation. If the storage reading performance is attenuated, it will cause jamming and deterioration of image quality during video playback.You can use tools such as iostat and vmstat to monitor the I / O performance of storage devices, and combine the system throughput and latency to determine the performance changes of video storage reads. At the same time, you can verify the attenuation of storage read performance through real-time feedback and log information of video playback.

[0169] The present invention also provides a video data storage and reading device, which is used to execute the video data storage and reading method as described above, and the video data storage and reading device comprises:

[0170] Video transmission volume calculation module: obtain cloud storage video transmission history data; extract video data storage and reading peak time period according to cloud storage video transmission history data, and obtain video transmission peak time period data; calculate peak single-user video transmission volume according to video transmission peak time period data, and obtain peak single-user video transmission volume data;

[0171] Storage system load growth prediction module: predicts the probability of video transmission network congestion based on the peak single-user video transmission volume data to obtain video transmission network congestion probability data; predicts the cloud storage system load growth based on the video transmission network congestion probability data to obtain cloud storage system load growth data;

[0172] Video storage read performance attenuation detection module: perform server load anomaly analysis on the cloud storage system load growth data to obtain server load anomaly data; perform video transmission response speed attenuation detection based on the server load anomaly data to obtain video transmission response speed attenuation data; perform video storage read performance attenuation detection based on the video transmission response speed attenuation data and server load anomaly data to obtain video storage read performance attenuation data;

[0173] Cloud storage video transmission system optimization module: perform video data storage reading anomaly analysis based on video storage reading performance attenuation data to obtain video data storage reading anomaly data; optimize the cloud storage video transmission system based on the video data storage reading anomaly data to obtain cloud storage video transmission system optimization data.

[0174] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for storing and reading video data, characterized in that: The following steps are involved: Step S1: Obtain cloud storage video transmission history data; extract the peak period of video data storage and reading according to the cloud storage video transmission history data to obtain video transmission peak period data; calculate the peak single-user video transmission volume according to the video transmission peak period data to obtain peak single-user video transmission volume data; Step S2: predicting the probability of video transmission network congestion according to the peak single-user video transmission volume data to obtain video transmission network congestion probability data; predicting the cloud storage system load growth according to the video transmission network congestion probability data to obtain cloud storage system load growth data; Step S3: Perform server load anomaly analysis on the cloud storage system load growth data to obtain server load anomaly data; perform video transmission response speed attenuation detection based on the server load anomaly data to obtain video transmission response speed attenuation data; Perform video storage read performance attenuation detection based on video transmission response speed attenuation data and server load abnormality data to obtain video storage read performance attenuation data; Step S4: performing video data storage reading abnormality analysis according to the video storage reading performance attenuation data to obtain video data storage reading abnormality data; The cloud storage video transmission system is optimized by reading abnormal data of video data storage to obtain the optimized data of cloud storage video transmission system.

2. The video data storage and reading method according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: Obtain cloud storage video transmission history data; Step S12: performing noise reduction processing on the cloud storage video transmission history data to obtain the historical video transmission noise reduction data; Step S13: extracting the peak period of video data storage and reading according to the historical video transmission noise reduction data to obtain the video transmission peak period data; Step S14: Calculate the peak single-user video transmission volume according to the video transmission peak time period data to obtain the peak single-user video transmission volume data.

3. The video data storage and reading method according to claim 2, characterized in that: Step S14 includes the following steps: Step S141: Counting the number of online users during the peak period according to the video transmission peak period data to obtain the number of online users during the peak period; Step S142: Perform video storage volume statistics on the video transmission peak time period data to obtain peak video storage volume data; Step S143: performing video reading volume statistics on the video transmission peak time period data to obtain peak video reading volume data; Step S144: calculating the video reading volume of a single user according to the number of online users during the peak period and the peak video reading volume data, and obtaining the video reading volume data of a single user; Step S145 calculates the video storage capacity of a single user according to the number of online users during the peak period and the peak video storage capacity data to obtain the video storage capacity data of a single user; Step S146: Calculate the peak single-user video transmission volume based on the single-user video storage volume data and the single-user video reading volume data to obtain the peak single-user video transmission volume data.

4. The video data storage and reading method according to claim 1, characterized in that: Step S2 includes the following steps: Step S21: evaluating the high concurrent requests of the cloud storage system according to the peak single-user video transmission volume data to obtain the high concurrent request data of the cloud storage system; Step S22: performing bandwidth bottleneck detection according to the high concurrent request data of the cloud storage system to obtain video transmission bandwidth bottleneck data; Step S23: predicting the video transmission network congestion probability based on the cloud storage system high-concurrency request data and the video transmission bandwidth bottleneck data to obtain video transmission network congestion probability data; Step S24: Predict the cloud storage system load growth based on the video transmission network congestion probability data and the video transmission bandwidth bottleneck data to obtain the cloud storage system load growth data.

5. The video data storage and reading method according to claim 4, characterized in that: Step S22 includes the following steps: Step S221: performing request volatility analysis based on the high-concurrency request data of the cloud storage system to obtain request volatility data of the cloud storage system; Step S222: performing request frequency increase calculation on the cloud storage system request volatility data to obtain video transmission request frequency increase data; Step S223: Calculate the bandwidth occupancy rate growth according to the video transmission request frequency increase data to obtain the bandwidth occupancy rate growth data of the cloud storage system; Step S224: performing multi-stream concurrent statistics according to the high-concurrency request data of the cloud storage system to obtain multi-stream concurrent data of the cloud storage system; Step S225: Perform bandwidth bottleneck detection based on the multi-stream concurrent data of the cloud storage system and the bandwidth occupancy growth data of the cloud storage system to obtain video transmission bandwidth bottleneck data.

6. The video data storage and reading method according to claim 4, characterized in that: Step S23 includes the following steps: Step S231: performing statistics on the number of requests surge according to the high-concurrency request data of the cloud storage system to obtain data on the number of video transmission requests surge; Step S232: performing a centralized detection of the request density according to the data on the surge in the number of video transmission requests, and obtaining centralized data on the density of video transmission requests; Step S233: performing transmission throughput overload prediction on the concentrated data of video transmission request density and the data on the surge in the number of video transmission requests to obtain transmission throughput overload data of the cloud storage system; Step S234: predicting network delay increase based on the cloud storage system transmission throughput overload data and the video transmission bandwidth bottleneck data to obtain network delay increase data; Step S235: performing video transmission request backlog prediction according to the cloud storage system transmission throughput overload data and the video transmission bandwidth bottleneck data to obtain video transmission request backlog data; Step S236: Predict the probability of video transmission network congestion based on the video transmission request backlog data and the network delay increase data to obtain video transmission network congestion probability data.

7. The video data storage and reading method according to claim 1, characterized in that: Step S3 includes the following steps: Step S31: performing server load anomaly analysis on the cloud storage system load growth data to obtain server load anomaly data; Step S32: Perform video transmission response speed attenuation detection according to the server load abnormality data to obtain video transmission response speed attenuation data; Step S33: performing storage read operation failure prediction on the server load abnormality data to obtain server storage read operation failure data; Step S34: performing a video file data loss assessment on the server storage read operation failure data to obtain cloud storage video file loss data; Step S35: Perform video storage read performance attenuation detection according to the cloud storage video file loss data and the video transmission response speed attenuation data to obtain video storage read performance attenuation data.

8. The video data storage and reading method according to claim 7, characterized in that: Step S31 The following steps are involved: Step S311: Calculate the growth of storage resource consumption according to the cloud storage system load growth data to obtain storage resource consumption growth data; Step S312: performing server load balancing failure prediction according to the storage resource consumption growth data and the cloud storage system load growth data to obtain server load balancing failure data; Step S313: Perform server overload detection according to the server load balancing failure data to obtain server overload data; Step S314: performing server continuous cumulative load analysis on the server overload data to obtain server continuous cumulative load data; Step S315: performing server crash frequency growth prediction on the server continuous accumulated load data and server load balancing failure data to obtain server crash frequency growth data; Step S316: Perform server load anomaly analysis based on the server crash frequency growth data and the server overload data to obtain server load anomaly data.

9. The video data storage and reading method according to claim 7, characterized in that: Step S35 includes the following steps: Step S351: collecting the encoding format loss data according to the cloud storage video file loss data to obtain the cloud storage video encoding format loss data; Step S352: Perform video file damage detection according to the cloud storage video encoding format loss data to obtain video file damage data; Step S353: Predicting video definition attenuation according to the video file damage data to obtain video definition attenuation data; Step S354: performing video playback screen freeze detection according to the video file damage data and the video transmission response speed attenuation data to obtain video playback screen freeze data; Step S355: Perform video storage read performance degradation detection based on the video playback screen freeze data and the video clarity degradation data to obtain video storage read performance degradation data.

10. A video data storage and reading device, characterized in that: For executing the video data storage reading method according to claim 1, the video data storage reading device comprises: Video transmission volume calculation module: obtain cloud storage video transmission history data; extract video data storage and reading peak time period according to cloud storage video transmission history data, and obtain video transmission peak time period data; calculate peak single-user video transmission volume according to video transmission peak time period data, and obtain peak single-user video transmission volume data; Storage system load growth prediction module: predicts the probability of video transmission network congestion based on the peak single-user video transmission volume data to obtain video transmission network congestion probability data; predicts the cloud storage system load growth based on the video transmission network congestion probability data to obtain cloud storage system load growth data; Video storage read performance attenuation detection module: perform server load anomaly analysis on the cloud storage system load growth data to obtain server load anomaly data; perform video transmission response speed attenuation detection based on the server load anomaly data to obtain video transmission response speed attenuation data; perform video storage read performance attenuation detection based on the video transmission response speed attenuation data and server load anomaly data to obtain video storage read performance attenuation data; Cloud storage video transmission system optimization module: perform video data storage reading anomaly analysis based on video storage reading performance attenuation data to obtain video data storage reading anomaly data; optimize the cloud storage video transmission system based on the video data storage reading anomaly data to obtain cloud storage video transmission system optimization data.

Images