A method for security video storage planning and positioning reading and writing based on spatial and temporal balance

Abstract

The present invention belongs to the field of security, and discloses a method for planning and positioning reading and writing of security video storage based on space and time balance. The video storage system implemented based on the method consists of two parts: generating a cluster map and a storage work process, wherein one storage work process corresponds to one hard disk, completing the work of data reading and writing and collecting available space. The present invention has multiple advantages. First, it makes the system design more flexible and eliminates the restriction that all hard disk capacity requirements are consistent. Secondly, the system can achieve nearly linear horizontal expansion, solving the problem that the storage system capacity expansion is limited by the hard disk with the smallest available capacity, so that the storage capacity of the system can grow linearly with the increase of hard disks, greatly improving the scalability of the system. Finally, the system improves the space utilization of the hard disk, avoiding the situation where data cannot be written when some hard disks still have available space, thereby significantly improving the overall performance of the system.

Description

Technical Field

[0001] The invention belongs to the field of security, and in particular is a method for storage planning and positioning reading and writing of security videos based on spatial and temporal balance. Background Art

[0002] When designing a video storage solution for a security system, it is necessary to consider the need to dynamically expand the storage space of the storage system during use. The traditional method of using camera devices as the minimum unit for allocating storage space can easily lead to unbalanced space utilization of each hard disk in the storage system, and does not take into account the different storage space sizes of each hard disk in the storage system and the unbalanced space utilization of each hard disk when the storage system is expanded. These problems can easily lead to excessive differences in space utilization of each hard disk in the storage system, resulting in a waste of storage space. Therefore, it is necessary to choose a more reasonable storage solution to improve storage space utilization and avoid waste. Summary of the invention

[0003] The purpose of the present invention is to provide a method for planning and positioning reading and writing of security video storage based on spatial and temporal balance, so as to solve the problems raised in the above-mentioned background technology.

[0004] In order to achieve the above-mentioned purpose, the present invention provides the following technical solutions: a method for space- and time-balanced security video storage planning, the video storage system implemented based on the method consists of two parts: generating a cluster map and a storage work process, wherein one storage work process corresponds to one hard disk, completing the work of data reading and writing and free space collection, and the specific steps of generating a cluster map are as follows:

[0005] S1: Data storage is divided into stages to facilitate management and access to data;

[0006] S2: Calculate the total available space based on the available storage space of each hard disk in the current storage stage to understand the storage status of the entire cluster;

[0007] S3: Divide the 32-bit integer space into 16383 parts, and finely divide the integer space to prepare for subsequent operations;

[0008] S4: Generates a unique ID for each integer space allocated to each hard disk, ensuring that each space can be accurately identified when storing and accessing data;

[0009] S5: Through step S4, a unique ID has been generated for each of the 16383 integer spaces divided into the 32-bit integer space, providing a convenient and accurate operating basis for data storage and management.

[0010] Preferably,

[0011] The data storage is divided into stages. When the system is initialized, hard disks are added, and expired data in the hard disk space is cleaned up, a new storage stage will be generated and saved in the cluster map. Each storage stage records the time when it is generated. When generating a storage stage, the cluster map will collect the available storage space information of each hard disk, conveniently manage different stages of data storage, and ensure the rational use of hard disk space in the cluster.

[0012] Preferably,

[0013] The cluster storage space analysis is to understand the storage status of the entire cluster by calculating the available storage space of each hard disk in the cluster, and determine the proportion of the data slices allocated to each hard disk in the total data slices generated in this stage.

[0014] Preferably,

[0015] The fine division of the integer space requires that the 32-bit integer space be divided into 16383 parts, and then the number of parts that each hard disk can get in the integer space is calculated according to the proportion of each hard disk obtained in the previous step. The purpose is to ensure the balanced distribution of data storage and provide an accurate basis for subsequent operations. This fine division method helps to improve the overall performance and stability of the system.

[0016] Preferably,

[0017] The unique ID identification space appends the hard disk ID to the serial number represented by the integer number, and the serial number of the first integer space obtained by each hard disk starts counting from 0, and the subsequent integer spaces increase in increments of 1.

[0018] Preferably,

[0019] The S5 is to sort the unique IDs according to the serial number from small to large according to step S4. We have successfully evenly distributed the integer space occupied by the hard disk on the 32-bit integer space. Each of the 16383 integer spaces has a global digital number, and the numbering starts from 0. This numbering method makes data storage and access more convenient and accurate, and also optimizes the utilization of storage space.

[0020] A method for positioning reading and writing based on the storage planning method,

[0021] The specific steps of the storage work process to locate the hard disk where the data is located are as follows:

[0022] A1: Generate a unique identifier for the data request;

[0023] A2: Calculate the MD5 value (128 bits) of the unique identifier of the generated data request;

[0024] A3: Divide the calculated MD5 value into three 32-bit integers and calculate the average value.

[0025] Preferably,

[0026] The unique identifier for generating a data request requires a write data and a read data request. The unique identifier for the data request is generated based on the unique identifier of the camera device and the specific time period in which the data is generated. This means that each time data is written, a unique identifier is created based on the camera device and a specific time period to ensure the accuracy and completeness of the data. At the same time, under a read data request, a unique identifier for the data request is generated based on the unique identifier of the camera device and the time period in which the data to be read is generated.

[0027] Preferably,

[0028] The generated data uniquely identifies the requested data and calculates the MD5 value (128 bits) We can then convert the MD5 hash value into an integer and split it into 3 parts.

[0029] Preferably,

[0030] The calculated MD5 value is divided into three 32-bit integers and the average is calculated. The average is divided by 16383 to determine which integer space the data is distributed in in the 32-bit integer space, and the unique ID of the hard disk is found based on the unique ID of the integer space.

[0031] The beneficial effects of the present invention are as follows:

[0032] The present invention has multiple advantages through the security video storage planning and positioning reading and writing method. First, it makes the system design more flexible and eliminates the restriction that all hard disk capacities must be consistent. Second, the system can achieve nearly linear horizontal expansion, solving the problem that the storage system capacity expansion is limited by the hard disk with the smallest available capacity, so that the storage capacity of the system can increase linearly with the increase of hard disks, greatly improving the scalability of the system. Finally, the system improves the space utilization of the hard disk, avoiding the situation where data cannot be written when some hard disks still have available space, thereby significantly improving the overall performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flow chart for generating a cluster map for the present invention;

[0034] Figure 2 A flowchart of the present invention for locating which hard disk the data is located on;

[0035] Figure 3 The service structure diagram of the method implemented in the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] like Figures 1 to 3 As shown, an embodiment of the present invention provides a method for space and time balanced security video storage planning. The video storage system implemented based on the method consists of two parts: generating a cluster map and a storage work process. One storage work process corresponds to one hard disk, and completes the work of data reading and writing and free space collection. The specific steps of generating a cluster map are as follows:

[0038] S1: Data storage is divided into stages to facilitate management and access to data;

[0039] S2: Calculate the total available space based on the available storage space of each hard disk in the current storage stage to understand the storage status of the entire cluster;

[0040] S3: Divide the 32-bit integer space into 16383 parts, and finely divide the integer space to prepare for subsequent operations;

[0041] S4: Generates a unique ID for each integer space allocated to each hard disk, ensuring that each space can be accurately identified when storing and accessing data;

[0042] S5: Through step S4, a unique ID has been generated for each of the 16383 integer spaces divided into the 32-bit integer space, providing a convenient and accurate operating basis for data storage and management.

[0043] Among them, the data storage is divided into stages. When the system is initialized, a hard disk is added, and expired data in the hard disk space is cleaned up, a new storage stage will be generated and saved in the cluster map. Each storage stage records the time when it is generated. When generating a storage stage, the cluster map will collect the available storage space information of each hard disk, conveniently manage the different stages of data storage, and ensure the rational use of hard disk space in the cluster.

[0044] The cluster storage space analysis is to understand the storage status of the entire cluster by calculating the available storage space of each hard disk in the cluster, and determine the proportion of the data slices allocated to each hard disk in the total data slices generated in this stage.

[0045] Among them, the fine division of the integer space requires that the 32-bit integer space be finely divided into 16383 parts, and then the number of parts that each hard disk can get in the integer space is calculated according to the proportion of each hard disk obtained in the previous step. The purpose is to ensure the balanced distribution of data storage and provide an accurate basis for subsequent operations. This fine division method helps to improve the overall performance and stability of the system.

[0046] The unique ID identification space appends the hard disk ID to the serial number represented by the integer number, and the serial number of the first integer space obtained by each hard disk starts from 0, while the subsequent integer spaces increase in increments of 1.

[0047] Among them, S5 is to sort the unique IDs according to the serial number from small to large according to step S4. We have successfully evenly distributed the integer space occupied by the hard disk on the 32-bit integer space. Each of the 16383 integer spaces has a global digital number, and the numbering starts from 0. This numbering method makes data storage and access more convenient and accurate, and also optimizes the utilization of storage space.

[0048] The method for positioning reading and writing based on the storage planning method, the specific steps of the storage work process locating the hard disk where the data is located are as follows:

[0049] A1: Generate a unique identifier for the data request;

[0050] A2: Calculate the MD5 value (128 bits) of the unique identifier of the generated data request;

[0051] A3: Divide the calculated MD5 value into three 32-bit integers and calculate the average value.

[0052] Among them, the unique identifier for generating a data request requires a write data and a read data request, and the unique identifier for the data request is generated based on the unique identifier of the camera device and the specific time period when the data is generated. This means that each time data is written, a unique identifier is created based on the camera device and a specific time period to ensure the accuracy and completeness of the data. At the same time, under a read data request, a unique identifier for the data request is generated based on the unique identifier of the camera device and the time period when the data to be read is generated.

[0053] Among them, the unique identifier of the generated data request calculates the MD5 value (128 bits). In this embodiment, the MD5 value is calculated using C language. In practical applications, Python language can also be used for implementation. If Python programming language is used, Python has a built-in hashlib library that can easily generate MD5 hash values. Then, we can convert the MD5 hash value into an integer and divide it into 3 parts.

[0054] The calculated MD5 value is divided into three 32-bit integers and the average value is calculated. The average value is divided by 16383 to determine which integer space the data is distributed in in the 32-bit integer space, and the unique ID of the hard disk is found based on the unique ID of the integer space.

[0055] Figure 3 As shown, a method for planning and positioning reading and writing of video storage based on spatial and temporal balance is implemented, which requires the cooperation of the cluster map process, the management process and the read-write process. The cluster map process collects the available space of each hard disk in a certain period of time in the cluster environment in real time, thereby forming the basis for the method to schedule read and write requests to the corresponding hard disk. Each hard disk corresponds to a read-write process, which is specifically responsible for processing the read and write requests scheduled to the hard disk. When the cluster changes (such as adding a hard disk, cleaning up expired data), it is also responsible for collecting the available space status of the hard disk in the current period and reporting it to the management process under the physical host where the hard disk is located. The management process is responsible for collecting the available space status of the hard disk under each physical host and reporting it to the cluster map process. At the same time, the management process is also responsible for processing the hard disk configuration commands from the cluster map process. For example, when a command to add a hard disk is received, a corresponding read-write process is dynamically created for the hard disk.

[0056] At the deployment level, the cluster map process needs to occupy a dedicated physical host. The processes under each other physical host in the cluster are composed of a management process and several read-write processes. The number of read-write processes corresponds to the number of hard disks configured on the physical host.

[0057] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0058] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for planning security video storage based on spatial and temporal balance, characterized by: The video storage system implemented based on this method consists of two parts: generating a cluster map and a storage work process. One storage work process corresponds to one hard disk and completes the work of data reading and writing and free space collection. The specific steps of generating a cluster map are as follows: S1: Data storage is divided into stages; the data storage is divided into stages based on system initialization, adding hard disks, and clearing expired data in hard disk space, generating a new storage stage and saving it to the cluster map. Each storage stage records the time of generation. When generating a storage stage, the cluster map collects the available storage space information of each hard disk; S2: Calculate the total available space based on the available storage space of each hard disk in the current storage stage; the condition for cluster storage space analysis is to calculate the available storage space of each hard disk in the cluster and determine the proportion of the data shards allocated to each hard disk in the total data shards generated in this stage; S3: Divide the 32-bit integer space into 16383 parts: According to the proportion of each hard disk obtained in the previous step, calculate the number of parts that each hard disk can get in the integer space; S4: Generate a unique ID for each integer space allocated to each hard disk, ensuring that each space can be accurately identified when storing and accessing data; the unique ID identification space requires the hard disk ID to be appended to the serial number represented by the integer number. The serial number of the first integer space obtained by each hard disk starts from 0, and the subsequent integer spaces are incremented by 1 in sequence; S5: through step S4, a unique ID has been generated for each of the 16383 integer spaces divided into the 32-bit integer space; said S5 is to sort the unique IDs from small to large according to the sequence number according to S4; The method based on space and time balanced security video storage planning needs to be implemented by the cooperation of the cluster map process, the management process and the read-write process. The cluster map process collects the available space of each hard disk in a certain period of time in the cluster environment in real time, thereby forming a basis for scheduling read-write requests to the corresponding hard disk. Each hard disk corresponds to a read-write process, which is specifically responsible for processing the read-write requests scheduled to the hard disk. When the cluster changes, it is also responsible for collecting the available space status of the hard disk in the current period and reporting it to the management process under the physical host where the hard disk is located. The management process is responsible for collecting the available space status of the hard disk under each physical host and reporting it to the cluster map process. At the same time, the management process is also responsible for processing the hard disk configuration command from the cluster map process; The specific steps for the storage work process to locate the hard disk where the data is located are as follows: A1: Generate a unique identifier for the data request; The prerequisite for generating the unique identifier of the data request is that a write data and a read data request are required. The unique identifier of the data request is generated according to the unique identifier of the camera device and the specific time period in which the data is generated. Each time data is written, a unique identifier is created based on the camera device and the specific time period to ensure the accuracy and integrity of the data. At the same time, under the read data request, the unique identifier of the data request is generated based on the unique identifier of the camera device and the time period in which the data to be read is generated; A2: Calculate the 128-bit MD5 value of the unique identifier of the generated data request; The unique identifier of the generated data request is calculated as a 128-bit MD5 value, and then the 128-bit MD5 hash value is converted into an integer and divided into 3 parts; A3: Divide the calculated 128-bit MD5 value into three 32-bit integers and calculate the average value; The A3 is obtained by dividing the calculated 128-bit MD5 value into three 32-bit integers and finding the average value. At the same time, the average value is divided by 16383 to find out which integer space the data is distributed in in the 32-bit integer space, and the unique ID of the hard disk is found based on the unique ID of the integer space.

Images