A video acquisition and storage method and system based on distributed architecture

Abstract

The present invention relates to the technical field of video acquisition and storage, and in particular to a video acquisition and storage method and system based on a distributed architecture, comprising: generating video data based on video information acquired in each area; transmitting the video data of each area to a corresponding storage node; identifying a data accumulation time point; calibrating each storage node based on a data receiving speed of each storage node; determining whether the storage of the video data is qualified based on statistical calibration results of the storage nodes and abnormal parameters, and scheduling the video data based on a data reception difference when determining storage abnormalities for each video data; and monitoring the process of storing the video, thereby improving the storage efficiency of the video data.

Description

Technical Field

[0001] The present invention relates to the technical field of video acquisition and storage, and in particular to a video acquisition and storage method and system based on a distributed architecture. Background Art

[0002] With the widespread adoption of video surveillance, video recording, and other application scenarios, the demand for video acquisition and storage is growing. In large-scale video acquisition and storage systems, the amount of data is huge and the sources are diverse. Traditional centralized architectures face many challenges in processing this massive amount of video data:

[0003] Chinese patent application publication number: CN103152437A, discloses a distributed video surveillance cloud storage system, including two storage devices, each of which has platform software embedded in it. The encoder converts the analog signal collected by the camera into a digital signal and directly enters the storage device without passing through the video server. At the same time, multiple storage devices control the entire monitoring system through the platform software; the platform software consists of two parts: one is some service programs running in a distributed manner on the storage devices, and the other is a client program used by users on Windows; it can be seen that the above technical solution has the following problems: it does not consider monitoring the process of storing video, which affects the storage efficiency of video data. Summary of the Invention

[0004] To this end, the present invention provides a video acquisition and storage method and system based on a distributed architecture to overcome the problem in the prior art that the process of storing videos is not considered to be monitored, thereby affecting the storage efficiency of video data.

[0005] In one aspect, the present invention provides a video acquisition and storage method based on a distributed architecture, comprising:

[0006] Generate video data based on the video information collected in each area, divide the video information converted into an encoded format into several video segments, and generate a hash value and a timestamp corresponding to each video segment;

[0007] Based on the preset transmission time point corresponding to each area and the amount of temporarily stored video data, the video data of each area is transmitted to the corresponding storage node;

[0008] Identifying a data accumulation time point based on the amount of data to be transmitted obtained at several time nodes;

[0009] When a data accumulation time point is identified, each storage node is calibrated based on its data receiving speed to classify a single storage node as a slow node or a fast node;

[0010] Determine whether the video data storage is qualified based on the statistical calibration results and abnormal parameters of the storage node, including:

[0011] determining storage anomalies for each video data and scheduling the video data based on a difference in data reception;

[0012] Alternatively, it is determined that the storage of each video data is qualified, and the current storage parameters are continuously used to complete the storage of each data.

[0013] Furthermore, the process of transmitting the video data of each region to the corresponding storage node includes:

[0014] For a single area, when the single area reaches the transmission time node corresponding to the area and the data volume of the temporarily stored video data reaches a preset critical data volume, the temporarily stored video data of the area is transmitted to the corresponding storage node.

[0015] Furthermore, the process of identifying the data accumulation time point based on the acquired amount of data to be transmitted at each time node includes:

[0016] Draw a time domain curve of the amount of data to be transmitted based on the amount of data to be transmitted at each time node;

[0017] Identify the peak values ​​in the time domain curve of the amount of data to be transmitted, and determine the time corresponding to each peak value as the data accumulation time point.

[0018] Furthermore, the process of determining whether the storage of video data is qualified based on the statistical calibration results of the storage nodes includes:

[0019] The ratio of the number of calculated speed-pass nodes to the total number of storage nodes is recorded as the operation rate;

[0020] If the operation rate is less than or equal to the first preset operation rate, determining that storage of each video data is abnormal, and scheduling the video data based on the data reception difference;

[0021] If the operation rate is less than or equal to the second preset operation rate and greater than the first preset operation rate, determining whether the storage of the video data is qualified based on the abnormal parameter;

[0022] If the operation rate is greater than the second preset operation rate, it is determined that the storage of each video data is qualified, and the current storage parameters are continuously used to complete the storage of each data.

[0023] Furthermore, the process of determining whether the storage of the video data is qualified based on the abnormal parameters includes:

[0024] For a single hysteresis node, each video data within a preset verification time is obtained, a hash value of each video data obtained is calculated, the recalculated hash value is compared with the original hash value of the video data, and video data with inconsistent hash value comparison is determined as abnormal data;

[0025] The ratio of the number of hysteresis nodes with abnormal data in each hysteresis node to the total number of hysteresis nodes is recorded as the abnormal parameter;

[0026] If the abnormal parameter is less than or equal to the preset abnormal parameter, adjusting the bit rate of the video data generated in the process of generating the video data to a corresponding value based on the abnormal parameter;

[0027] If the abnormal parameter is greater than the preset abnormal parameter, it is determined that the storage of each video data is abnormal, and the video data is scheduled based on the data reception difference;

[0028] The bit rate of the video data generated in the process of generating the video data is adjusted to a corresponding value based on the abnormal parameter, wherein:

[0029] The reduction in bit rate is proportional to the abnormal parameter.

[0030] Furthermore, the process of scheduling video data based on the data reception difference includes:

[0031] Obtain the storage data volume of the video data stored in each storage node within a preset verification time, calculate the variance of each storage data volume, and obtain the reception difference amount;

[0032] If the received difference is less than or equal to the preset received difference, an abnormal alarm message is issued for each sluggish node;

[0033] If the reception difference is greater than a preset reception difference, the video data is scheduled based on the smoothness of each storage node.

[0034] Furthermore, the process of scheduling video data based on the smoothness of each storage node includes:

[0035] Based on the storage data volume of a single fast-pass node obtained during each preset verification time, a storage data volume time domain curve is drawn, and the difference between the maximum and minimum values ​​in the calculated storage data volume time domain curve is recorded as the smoothness;

[0036] The smoothness of each fast-pass node is arranged in descending order, a preset number of fast-pass nodes are selected in turn as nodes to be coordinated, and the amount of video data received by the nodes to be coordinated is adjusted.

[0037] Furthermore, when the adjustment of the data volume of the video data received by each node to be coordinated is completed, determining whether to adjust the storage parameter of the video data based on the data fluctuation parameter includes:

[0038] Calculate the derivative of the acquired time domain curve of the amount of data to be transmitted at the current time node to obtain the data fluctuation parameter;

[0039] When the data fluctuation parameter is less than or equal to the preset data fluctuation parameter, the current storage parameter is continuously used to complete the storage of each data;

[0040] When the data fluctuation parameter is greater than a preset data fluctuation parameter, the number of storage nodes is increased to a corresponding value based on the data fluctuation parameter.

[0041] Furthermore, the number of storage nodes is increased to a corresponding value based on the data fluctuation parameter, wherein,

[0042] The increase in the number of storage nodes is proportional to the data fluctuation parameter.

[0043] On the other hand, the present invention further provides a video acquisition and storage system using the above-mentioned video acquisition and storage method based on a distributed architecture, comprising:

[0044] A video acquisition module, comprising a plurality of video acquisition units for respectively acquiring video information in each area; the video acquisition units comprising an encoding mechanism for converting the video information into an encoding format, and a generation mechanism for respectively generating original hash values ​​based on the divided video segments;

[0045] a data transmission module connected to the video acquisition module and configured to transmit the video data of each video acquisition unit when the video data stored in the video acquisition unit reaches a preset critical data volume and reaches a preset transmission time point corresponding to the video acquisition unit; a data storage module connected to the data transmission module, comprising a plurality of storage nodes for respectively storing corresponding video data;

[0046] a data identification module, connected to the data transmission module and the data storage module, respectively, for identifying a data accumulation time point based on the amount of data to be transmitted of the data transmission module at each time point, and, upon identifying the data accumulation time point, calibrating each storage node based on its data receiving speed;

[0047] a data discrimination module, connected to the data transmission module, the data storage module, and the data identification module, respectively, for determining whether the storage of the video data is qualified based on the statistical calibration results of the storage nodes and the abnormal parameters, and scheduling the video data based on the data reception difference amount when it is determined that the storage of each video data is abnormal;

[0048] The data statistics module is connected to the data storage module and is used to record the abnormal alarm information for each hysteresis node issued by the data identification module.

[0049] Compared with the existing technology, the beneficial effect of the present invention is that based on the preset transmission time point corresponding to each area and the amount of temporarily stored video data, the video data of each area is transmitted to the corresponding storage node respectively, and the timing of data transmission is reasonably arranged according to the actual situation of different areas, thereby avoiding the blindness and confusion of data during the transmission process, effectively reducing the problems of data accumulation and transmission delay, and improving the efficiency and stability of data transmission.

[0050] Furthermore, by drawing a time domain curve of the amount of data to be transmitted based on the amount of data to be transmitted at each time node, and identifying the peak value in the curve as the data accumulation time point, it is possible to intuitively and accurately discover the data accumulation situation that may occur during the data transmission process, so as to provide early warning of data accumulation problems and provide sufficient time for subsequent corresponding measures, which helps to ensure the smoothness of data transmission and further improve the storage efficiency of video data.

[0051] Furthermore, each storage node is calibrated based on its data receiving speed, and the storage nodes are divided into slow nodes and fast-pass nodes, so that the performance status of each storage node can be clearly understood. By calculating the proportion of the number of fast-pass nodes to the total number of nodes and the abnormal parameters, a comprehensive assessment of whether the storage of video data is qualified can be made. The multi-dimensional evaluation method is more detailed and accurate, and potential problems in the storage process can be discovered in a timely manner. The operating rate represents the operating status of each storage node when data is accumulated. When the operating rate is less than or equal to the first preset operating rate, a large number of storage nodes are operating abnormally, resulting in risks in data storage. In this case, the video data is scheduled to further improve the storage efficiency of the video data.

[0052] Furthermore, when the abnormal parameter is less than or equal to the preset abnormal parameter, there is a small amount of video data that cannot be received normally. It is determined that the packet loss rate has increased due to insufficient network bandwidth, and the current network cannot support high-bitrate video transmission. At this time, the target bit rate of the video is lowered to ensure stable data transmission, further improving the storage efficiency of the video data.

[0053] Furthermore, when storage anomalies are detected, video data is scheduled based on the data reception variance. The variance of the data stored at each storage node is calculated to determine the reception variance, and appropriate measures are taken based on the specific situation. The data reception variance represents the variance in the amount of data stored at each storage node during operation. When the reception variance exceeds a preset value, this indicates that the storage nodes are not fully coordinated due to the large differences in the video data stored, affecting storage efficiency. In this case, video data is scheduled based on the smoothness of the storage nodes to further optimize data storage allocation. Smoothness represents the data reception capacity of a single fast-passing node; greater smoothness increases the fast-passing node's ability to receive video data. When the reception variance is less than a preset value, the storage nodes operate in a coordinated manner. In this case, data accumulation due to the presence of an abnormal storage node results in an anomaly alarm. This effectively balances the load of each storage node, improving the stability and reliability of the entire video data storage. By adjusting the video data bitrate based on the anomaly parameter and the number of storage nodes based on the data fluctuation parameter, storage parameters can be dynamically adjusted based on actual conditions to ensure video data storage quality and optimize system performance.

[0054] Furthermore, after completing the adjustment of the data volume of the video data received by the coordination node, the transmission situation is continuously monitored to determine the data fluctuation parameter. The data fluctuation parameter characterizes the transmission situation of the adjusted data. When the data fluctuation parameter is greater than the preset data fluctuation parameter, in this case, only coordinating the storage nodes cannot improve the data accumulation situation. At this time, the number of storage nodes is increased to the corresponding value based on the data fluctuation parameter to achieve high scalability, ensure the smoothness of data transmission, and further improve the storage efficiency of video data. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flowchart of the steps of the video acquisition and storage method based on a distributed architecture according to an embodiment of the present invention;

[0056] Figure 2 This is a module block diagram of a video acquisition and storage system based on a distributed architecture according to an embodiment of the present invention;

[0057] Figure 3 A logic decision diagram for calibrating each storage node based on the data receiving speed of each storage node according to an embodiment of the present invention;

[0058] Figure 4 This is a logic decision diagram for determining whether storage of video data is qualified based on abnormal parameters according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.

[0060] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0061] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside", and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0062] Furthermore, it should be noted that, in the description of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0063] See also Figure 1 As shown in FIG, which is a flowchart of the steps of a video acquisition and storage method based on a distributed architecture according to an embodiment of the present invention, the video acquisition and storage method according to the present invention includes:

[0064] S1, generating video data based on the video information collected in each area;

[0065] S2, based on the preset transmission time point corresponding to each area and the amount of temporarily stored video data, the video data of each area is transmitted to the corresponding storage node;

[0066] S3, identifying a data accumulation time point based on the amount of data to be transmitted at each time node;

[0067] S4, when a data accumulation time point is identified, each storage node is calibrated based on its data receiving speed;

[0068] S5, based on the statistical calibration results of the storage nodes and the abnormal parameters, determining whether the storage of the video data is qualified includes:

[0069] determining storage anomalies for each video data and scheduling the video data based on a difference in data reception;

[0070] Alternatively, it is determined that the storage of each video data is qualified, and the current storage parameters are continuously used to complete the storage of each data.

[0071] Specifically, each area is preset with a corresponding preset transmission time point, so that the video information of each area is transmitted in different time periods to ensure data transmission efficiency.

[0072] Based on the preset transmission time points corresponding to each area and the amount of temporarily stored video data, the video data of each area is transmitted to the corresponding storage node respectively. According to the actual situation of different areas, the timing of data transmission is reasonably arranged to avoid the blindness and confusion of data during the transmission process, effectively reduce the problems of data accumulation and transmission delay, and improve the efficiency and stability of data transmission.

[0073] Specifically, for a single area, transmission is performed when the transmission time node is reached and the data volume reaches the preset critical data volume. This can make full use of network resources, avoid bandwidth waste caused by frequent transmission of too small data volumes, and avoid the problem of transmission timeout due to excessive data volumes, further improving the storage efficiency of video data.

[0074] It is understood that actual implementation personnel can determine the preset transmission time points corresponding to each region based on actual conditions or based on regular changes in the amount of video data collected in different regions in historical data. In this embodiment, the amount of video data collected for a single region during different time periods is obtained. By analyzing historical data, the changing trends of data volume in each region and the time when transmission peaks occur are statistically analyzed. Based on this statistical data, the time when transmission peaks occur is determined as the preset transmission time point for the corresponding region, allowing data transmission to proceed when data volume is relatively stable.

[0075] See also Figure 2 As shown in FIG, which is a module block diagram of a video acquisition and storage system based on a distributed architecture according to an embodiment of the present invention, the video acquisition and storage system according to the present invention includes:

[0076] A video acquisition module, comprising a plurality of video acquisition units for respectively acquiring video information in each area; the video acquisition units comprising an encoding mechanism for converting the video information into an encoding format, and a generation mechanism for respectively generating original hash values ​​based on the divided video segments;

[0077] a data transmission module connected to the video acquisition module and configured to transmit the video data of each video acquisition unit when the video data stored in the video acquisition unit reaches a preset critical data volume and reaches a preset transmission time point corresponding to the video acquisition unit; a data storage module connected to the data transmission module, comprising a plurality of storage nodes for respectively storing corresponding video data;

[0078] a data identification module, connected to the data transmission module and the data storage module, respectively, for identifying a data accumulation time point based on the amount of data to be transmitted of the data transmission module at each time point, and, upon identifying the data accumulation time point, calibrating each storage node based on its data receiving speed;

[0079] a data discrimination module, connected to the data transmission module, the data storage module, and the data identification module, respectively, for determining whether the storage of the video data is qualified based on the statistical calibration results of the storage nodes and the abnormal parameters, and scheduling the video data based on the data reception difference amount when it is determined that the storage of each video data is abnormal;

[0080] The data statistics module is connected to the data storage module and is used to record the abnormal alarm information for each hysteresis node issued by the data identification module.

[0081] Specifically, the video data includes each video segment converted into an encoded format, an original hash value corresponding to each video segment, and a timestamp corresponding to the original hash value;

[0082] Specifically, there is no limitation on the specific structure of the generating mechanism, which can be any logical component. It can be understood that the generating mechanism can divide the encoded video into several video segments based on a fixed duration, generate a unique hash value for each video segment through a hash algorithm, and record the timestamp of the video segment corresponding to each hash value in the entire video. This will not be elaborated here.

[0083] Specifically, a single video acquisition unit includes several video acquisition devices, each of which can achieve high refresh rate video acquisition of 340 frames per second. During the acquisition process, the acquisition device can encode the video in real time and convert the video into an encoding format to reduce the amount of data and improve transmission efficiency.

[0084] Specifically, after receiving the video data, the data storage module determines the specific storage node where the data should be stored based on the hash algorithm.

[0085] Specifically, each video data is stored on different nodes in chronological order. In addition, a metadata index is established for each video data to record the storage location, timestamp, and video encoding format of the video data for subsequent query and retrieval.

[0086] Specifically, this solution distributes video data to various storage nodes for storage, and at the same time, designs data redundancy and fault tolerance mechanisms to ensure data security and reliability.

[0087] Specifically, the process of generating video data includes dividing the video information converted into an encoded format into a number of video segments, generating a hash value for each video segment and a timestamp corresponding to each video segment;

[0088] The process of transferring video data from each region to the corresponding storage node includes:

[0089] For a single area, when the single area reaches the transmission time node corresponding to the area and the data volume of the temporarily stored video data reaches a preset critical data volume, the temporarily stored video data of the area is transmitted to the corresponding storage node.

[0090] Specifically, the process of identifying the data accumulation time point based on the acquired amount of data to be transmitted at each time node includes:

[0091] Draw a time domain curve of the amount of data to be transmitted based on the amount of data to be transmitted at each time node;

[0092] Identify each peak value in the time domain curve of the amount of data to be transmitted, and determine each peak value as the data accumulation time point;

[0093] See also Figure 3 As shown in FIG. , it is a logic decision diagram for calibrating each storage node based on the data receiving speed of each storage node according to an embodiment of the present invention. The process of calibrating each storage node based on the data receiving speed of each storage node according to the present invention includes:

[0094] If the data receiving speed is less than or equal to the preset data receiving speed, the single storage node is marked as a slow node;

[0095] If the data receiving speed is greater than the preset data receiving speed, the single storage node will be marked as a fast-pass node.

[0096] It is understandable that actual implementers can determine the preset data receiving speed based on actual conditions or based on the average of the data receiving speeds of each storage node in historical data. Preferably, the preset data receiving speed is selected within the range of [400Mbps, 800Mbps].

[0097] In this embodiment, optionally, the time nodes of the acquired data volume to be transmitted at each time node can be selected as the corresponding time nodes when the video data stored in the video acquisition unit reaches a preset critical data volume, or can be time nodes with equal interval duration; the data volume to be transmitted is the data volume of the video data that remains and has not completed the transmission task when the data transmission module transmits the video data of each video acquisition unit.

[0098] Specifically, by drawing a time domain curve of the amount of data to be transmitted based on the amount of data to be transmitted at each time node, and identifying the peak value in the curve as the data accumulation time point, it is possible to intuitively and accurately discover the data accumulation situation that may occur during the data transmission process, so as to provide early warning of data accumulation problems, provide sufficient time for subsequent corresponding measures, help ensure the smoothness of data transmission, and further improve the storage efficiency of video data.

[0099] Specifically, the process of determining whether the storage of video data is qualified based on the statistical calibration results of the storage nodes includes:

[0100] The ratio of the number of calculated fast-pass nodes to the total number of nodes is recorded as the operation rate;

[0101] If the operation rate is less than or equal to the first preset operation rate, determining that storage of each video data is abnormal, and scheduling the video data based on the data reception difference;

[0102] If the operation rate is less than or equal to the second preset operation rate and greater than the first preset operation rate, determining whether the storage of the video data is qualified based on the abnormal parameter;

[0103] If the operation rate is greater than the second preset operation rate, it is determined that the storage of each video data is qualified, and the current storage parameters are continuously used to complete the storage of each data;

[0104] See also Figure 4 As shown in FIG. , it is a logic determination diagram for determining whether the storage of video data is qualified based on abnormal parameters according to an embodiment of the present invention. The process of determining whether the storage of video data is qualified based on abnormal parameters according to the present invention includes:

[0105] For a single hysteresis node, each video data within a preset verification time is obtained, a hash value of each video data obtained is calculated, the recalculated hash value is compared with the original hash value of the video data, and video data with inconsistent hash value comparison is determined as abnormal data;

[0106] The ratio of the number of hysteresis nodes with abnormal data in each hysteresis node to the total number of hysteresis nodes is recorded as the abnormal parameter;

[0107] If the abnormal parameter is less than or equal to the preset abnormal parameter, adjusting the bit rate of the video data generated in the process of generating the video data to a corresponding value based on the abnormal parameter;

[0108] If the abnormal parameter is greater than the preset abnormal parameter, it is determined that the storage of each video data is abnormal, and the video data is scheduled based on the data reception difference.

[0109] Specifically, the first preset operating rate is selected within the interval [0.41, 0.62], and the second preset operating rate is selected within the interval [0.78, 0.9].

[0110] Specifically, the preset abnormal parameter C0 is selected within the interval [0.1, 0.26].

[0111] Specifically, each storage node is calibrated based on its data receiving speed, and the storage nodes are divided into slow nodes and fast-pass nodes, so that the performance status of each storage node can be clearly understood. By calculating the ratio of the number of fast-pass nodes to the total number of nodes, the operating rate and abnormal parameters, a comprehensive assessment of whether the storage of video data is qualified can be made. The multi-dimensional evaluation method is more detailed and accurate, and potential problems in the storage process can be discovered in a timely manner. The operating rate represents the operating status of each storage node when data is accumulated. When the operating rate is less than or equal to the first preset operating rate, a large number of storage nodes are operating abnormally, resulting in risks in data storage. In this case, the video data is scheduled to further improve the storage efficiency of the video data.

[0112] Specifically, when the abnormal parameter is less than or equal to the preset abnormal parameter, there is a small amount of video data that cannot be received normally. It is determined that the packet loss rate has increased due to insufficient network bandwidth, and the current network cannot support high-bitrate video transmission. At this time, the target bit rate of the video is lowered to ensure stable data transmission, further improving the storage efficiency of the video data.

[0113] Specifically, the bit rate of the video data generated during the video data generation process is adjusted to a corresponding value based on the abnormal parameter, wherein:

[0114] The reduction in bit rate is proportional to the abnormal parameter.

[0115] In this embodiment, optionally,

[0116] Comparing the abnormal parameter with a first preset abnormal parameter comparison threshold and a second preset abnormal parameter comparison threshold;

[0117] When the abnormal parameter is less than or equal to the first preset abnormal parameter comparison threshold, the bit rate of the generated video data is adjusted to 0.91 times the initial bit rate;

[0118] When the abnormal parameter is less than or equal to the second preset abnormal parameter comparison threshold and greater than the first preset abnormal parameter comparison threshold, adjusting the bit rate of the generated video data to 0.83 times the initial bit rate;

[0119] When the abnormal parameter is greater than a second preset abnormal parameter comparison threshold, the bit rate of the generated video data is adjusted to 0.7 times the initial bit rate;

[0120] The first preset abnormal parameter comparison threshold is 0.52C0, and the second preset abnormal parameter comparison threshold is 0.88C0.

[0121] Specifically, the process of scheduling video data based on the data reception difference includes:

[0122] Obtain the storage data volume of the video data stored in each storage node within a preset verification time, calculate the variance of each storage data volume, and obtain the reception difference amount;

[0123] If the received difference is less than or equal to the preset received difference, an abnormal alarm message is issued for each sluggish node;

[0124] If the reception difference is greater than a preset reception difference, the video data is scheduled based on the smoothness of each storage node.

[0125] Specifically, the preset reception difference is in the interval [2P0 2 ,4P0 2 ], P0 is the average value of each storage data amount.

[0126] Specifically, upon receiving abnormal alert information for each slow node, each storage node begins self-checking. This includes using the SMART (Self-Monitoring, Analysis and Reporting Technology) tool to obtain various hard disk parameters, including hard disk temperature, read and write error counts, and remaining lifespan, to check the health of the storage device's hard disks. If the hard disk temperature is too high or there are a large number of read and write errors, these errors are recorded and reported to the data statistics module.

[0127] Specifically, when storage anomalies are determined, video data is scheduled based on the data reception difference. The reception difference is obtained by calculating the variance of the amount of data stored in each storage node, and corresponding measures are taken according to different situations. The data reception difference characterizes the difference in the amount of data stored in each storage node during operation. When the reception difference is greater than the preset value, in this case, due to the large difference in the video data stored in each storage node, the storage nodes fail to fully coordinate their operation, affecting the storage effect. At this time, video data is scheduled based on the smoothness of the storage node to further optimize data storage allocation. The smoothness characterizes the data reception capability of a single fast-pass node. The greater the smoothness, the greater the ability of the fast-pass node to receive video data. When the reception difference is less than the preset value, the storage nodes operate reasonably in coordination. At this time, due to the existence of abnormal storage nodes, data accumulates, and an abnormal alarm message is issued in this case. This effectively balances the load of each storage node and improves the stability and reliability of the entire video data storage.

[0128] The bit rate of video data is adjusted based on abnormal parameters, and the number of storage nodes is adjusted based on data fluctuation parameters. The storage parameters can be dynamically adjusted according to actual conditions to ensure the storage quality of video data and the performance optimization of the system.

[0129] Specifically, the process of scheduling video data based on the smoothness of each storage node includes:

[0130] Based on the storage data volume of a single fast-pass node obtained during each preset verification time, a storage data volume time domain curve is drawn, and the difference between the maximum and minimum values ​​in the calculated storage data volume time domain curve is recorded as the smoothness;

[0131] The smoothness of each fast-pass node is arranged in descending order, a preset number of fast-pass nodes are selected in turn as nodes to be coordinated, and the amount of video data received by the nodes to be coordinated is adjusted.

[0132] In this embodiment, optionally,

[0133] The process of adjusting the amount of video data received by the node to be coordinated includes:

[0134] The data ratio of the video data received by the node to be coordinated is adjusted to 1.1 times the initial data ratio.

[0135] The data ratio is the ratio of the amount of video data received by a single storage node to the total amount of video data transmitted by the data transmission module.

[0136] Specifically, when the adjustment of the data volume of the video data received by each node to be coordinated is completed, determining whether to adjust the storage parameter of the video data based on the data fluctuation parameter includes:

[0137] Calculate the derivative of the acquired time domain curve of the amount of data to be transmitted at the current time node to obtain the data fluctuation parameter;

[0138] When the data fluctuation parameter is less than or equal to the preset data fluctuation parameter, the current storage parameter is continuously used to complete the storage of each data;

[0139] When the data fluctuation parameter is greater than a preset data fluctuation parameter, the number of storage nodes is increased to a corresponding value based on the data fluctuation parameter.

[0140] Specifically, the preset data fluctuation parameter B0 is selected within the interval [0.51L0, 0.62L0], and L0 is the average value of the derivatives of each time node in the time domain curve of the data volume to be transmitted.

[0141] Specifically, after completing the adjustment of the amount of video data received by the coordinated node, the transmission situation is continuously monitored to determine the data fluctuation parameter. The data fluctuation parameter characterizes the transmission situation of the adjusted data. When the data fluctuation parameter is greater than the preset data fluctuation parameter, in this case, only coordinating the storage nodes cannot improve the data accumulation situation. At this time, the number of storage nodes is increased to the corresponding value based on the data fluctuation parameter to achieve high scalability, ensure the smoothness of data transmission, and further improve the storage efficiency of video data.

[0142] Specifically, the number of storage nodes is increased to a corresponding value based on the data fluctuation parameter, where:

[0143] The increase in the number of storage nodes is proportional to the data fluctuation parameter.

[0144] In this embodiment, optionally,

[0145] Comparing the data fluctuation parameter with a first preset fluctuation comparison threshold and a second preset fluctuation comparison threshold;

[0146] When the data fluctuation parameter is less than or equal to the first preset fluctuation comparison threshold, the number of storage nodes is increased to 1.1 times the initial number of storage nodes;

[0147] When the data fluctuation parameter is less than or equal to the second preset fluctuation comparison threshold and greater than the first preset fluctuation comparison threshold, the number of storage nodes is increased to 1.15 times the initial number of storage nodes;

[0148] When the data fluctuation parameter is greater than a second preset fluctuation comparison threshold, the number of storage nodes is increased to 1.2 times the initial number of storage nodes;

[0149] The first preset fluctuation comparison threshold is 1.35B0, and the second preset fluctuation comparison threshold is 1.66B0.

[0150] Specifically, after adding storage nodes, the video data is evenly distributed to the newly added nodes and existing nodes based on the storage capacity and current load of each storage node. In this embodiment, the remaining available storage capacity of each node is calculated, and the video data to be stored is distributed to each storage node according to the ratio of the remaining available storage capacity of each storage node.

[0151] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

[0152] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A video acquisition and storage method based on a distributed architecture, characterized in that: include: Generate video data based on the video information collected in each area, divide the video information converted into an encoded format into several video segments, and generate a hash value and a timestamp corresponding to each video segment; Based on the preset transmission time point corresponding to each area and the amount of temporarily stored video data, the video data of each area is transmitted to the corresponding storage node; Identifying a data accumulation time point based on the amount of data to be transmitted obtained at several time nodes; When a data accumulation time point is identified, each storage node is calibrated based on its data receiving speed to classify a single storage node as a slow node or a fast node; Determine whether the video data storage is qualified based on the statistical calibration results and abnormal parameters of the storage node, including: The ratio of the number of calculated speed-pass nodes to the total number of storage nodes is recorded as the operation rate; If the operation rate is less than or equal to the first preset operation rate, determining that storage of each video data is abnormal, and scheduling the video data based on the data reception difference; If the operation rate is less than or equal to the second preset operation rate and greater than the first preset operation rate, determining whether the storage of the video data is qualified based on the abnormal parameter; If the operation rate is greater than the second preset operation rate, it is determined that the storage of each video data is qualified, and the current storage parameters are continuously used to complete the storage of each data; The process of determining whether the storage of video data is qualified based on abnormal parameters includes: For a single hysteresis node, each video data within a preset verification time is obtained, a hash value of each video data obtained is calculated, the recalculated hash value is compared with the original hash value of the video data, and video data with inconsistent hash value comparison is determined as abnormal data; The ratio of the number of hysteresis nodes with abnormal data in each hysteresis node to the total number of hysteresis nodes is recorded as the abnormal parameter; If the abnormal parameter is less than or equal to the preset abnormal parameter, adjusting the bit rate of the video data generated in the process of generating the video data to a corresponding value based on the abnormal parameter; If the abnormal parameter is greater than the preset abnormal parameter, it is determined that the storage of each video data is abnormal, and the video data is scheduled based on the data reception difference.

2. The video acquisition and storage method based on a distributed architecture according to claim 1, characterized in that: The process of transferring video data from each region to the corresponding storage node includes: For a single area, when the single area reaches the transmission time node corresponding to the area and the data volume of the temporarily stored video data reaches a preset critical data volume, the temporarily stored video data of the area is transmitted to the corresponding storage node.

3. The video acquisition and storage method based on distributed architecture according to claim 2, characterized in that: The process of identifying the data accumulation time point based on the acquired amount of data to be transmitted at each time node includes: Draw a time domain curve of the amount of data to be transmitted based on the amount of data to be transmitted at each time node; Identify the peak values ​​in the time domain curve of the amount of data to be transmitted, and determine the time corresponding to each peak value as the data accumulation time point.

4. The video acquisition and storage method based on a distributed architecture according to claim 3, characterized in that: The bit rate of the video data generated in the process of generating the video data is adjusted to a corresponding value based on the abnormal parameter, wherein: The reduction in bit rate is proportional to the abnormal parameter.

5. The video acquisition and storage method based on distributed architecture according to claim 4 is characterized in that: The process of scheduling video data based on the difference in data reception includes: Obtain the storage data volume of the video data stored in each storage node within a preset verification time, calculate the variance of each storage data volume, and obtain the reception difference amount; If the received difference is less than or equal to the preset received difference, an abnormal alarm message is issued for each sluggish node; If the reception difference is greater than a preset reception difference, the video data is scheduled based on the smoothness of each storage node.

6. The video acquisition and storage method based on distributed architecture according to claim 5, characterized in that: The process of scheduling video data based on the smoothness of each storage node includes: Based on the storage data volume of a single fast-pass node obtained during each preset verification time, a storage data volume time domain curve is drawn, and the difference between the maximum and minimum values ​​in the calculated storage data volume time domain curve is recorded as the smoothness; The smoothness of each fast-pass node is arranged in descending order, a preset number of fast-pass nodes are selected in turn as nodes to be coordinated, and the amount of video data received by the nodes to be coordinated is adjusted.

7. The video acquisition and storage method based on a distributed architecture according to claim 6, characterized in that: When the adjustment of the data amount of the video data received by each node to be coordinated is completed, determining whether to adjust the storage parameter of the video data based on the data fluctuation parameter includes: Calculate the derivative of the acquired time domain curve of the amount of data to be transmitted at the current time node to obtain the data fluctuation parameter; When the data fluctuation parameter is less than or equal to the preset data fluctuation parameter, the current storage parameter is continuously used to complete the storage of each data; When the data fluctuation parameter is greater than a preset data fluctuation parameter, the number of storage nodes is increased to a corresponding value based on the data fluctuation parameter.

8. The video acquisition and storage method based on distributed architecture according to claim 7, characterized in that: The number of storage nodes is increased to the corresponding value based on the data fluctuation parameter, where: The increase in the number of storage nodes is proportional to the data fluctuation parameter.

9. A video acquisition and storage system using the video acquisition and storage method based on a distributed architecture according to any one of claims 1 to 8, characterized in that: include: A video acquisition module, which includes a plurality of video acquisition units for respectively acquiring video information in each area; The video acquisition unit includes an encoding mechanism for converting video information into an encoding format, and a generation mechanism for generating original hash values ​​based on the divided video segments. a data transmission module connected to the video acquisition module, configured to transmit the video data of each video acquisition unit when the video data stored in the video acquisition unit reaches a preset critical data amount and reaches a preset transmission time point corresponding to a single video acquisition unit; A data storage module, connected to the data transmission module, comprising a plurality of storage nodes for respectively storing corresponding video data; a data identification module, connected to the data transmission module and the data storage module, respectively, for identifying a data accumulation time point based on the amount of data to be transmitted of the data transmission module at each time point, and, upon identifying the data accumulation time point, calibrating each storage node based on its data receiving speed; a data discrimination module, which is connected to the data transmission module, the data storage module and the data identification module respectively, and is used to determine whether the storage of video data is qualified based on the calibration results and abnormal parameters of the statistical storage nodes, and to schedule video data based on the difference in data reception; The data statistics module is connected to the data storage module and is used to record the abnormal alarm information for each hysteresis node issued by the data identification module.

Images