A video slice index database storage method

Abstract

The present invention relates to the technical field of database sharding, and specifically provides a sharding method for video slice index database storage, aiming to solve the problem that the sharding method in the prior art cannot meet the sharding demand for video index storage. The method provided by the present invention includes: the resource allocation end receives a request from the video slice index storage end, and checks the memory; if the database instance has been allocated, it is returned to the video slice index storage end; if it has not been allocated, the allocator is requested to allocate; the allocator locks the database instance with the most remaining resources in the resource database and associates it with the device, and updates the remaining resources of the database instance; the resource allocation end caches the allocated database instance information and device information and returns it to the video slice index storage end. The present invention meets the sharding demand for video index storage, and can not only support the storage of video slice indexes of devices with different packages, but also achieves a relative balance in the resources of each of the database instances.

Description

Technical Field

[0001] The present invention belongs to the technical field of database sharding, and in particular relates to a sharding method for storing a video slice index database. Background Art

[0002] With the increasing number of devices connected to the security platform, the storage cost of video slice index is also an important evaluation indicator, especially for cloud storage devices that record all day according to the package; the video slice index is the description information for a binary video file, generally also called video metadata; the video slice index includes information such as the size, start time, end time and storage location of the video file. Database storage is a relatively mature storage solution that is relatively friendly to development and operation and maintenance, and can also implement complex business scenarios. Considering that video slice indexes are generally massive, database-based video slice index storage must be sharded. The existing database sharding method is to distribute the same data table to multiple database instances according to a certain rule of one or more combined fields.

[0003] The storage of video slice indexes is different from the storage of other services. The characteristics of the video slice index storage are:

[0004] 1. The storage package lengths of devices are different, so the amount of stored data will vary with the number of devices.

[0005] 2. The same data table records in different database instances should have the same data volume as much as possible. Balance cannot be achieved based on the number of devices.

[0006] 3. Databases with different configurations can be used.

[0007] Based on the above-mentioned video slice index storage characteristics, the existing database partitioning method cannot meet the database partitioning requirements of video slice index storage. Summary of the Invention

[0008] The present invention provides a library sharding method for video slice index database storage, aiming to solve the problem that the library sharding method in the prior art cannot meet the library sharding requirements for video index storage.

[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0010] In a first aspect, the present invention provides a method for storing a video slice index database in a database partitioning manner, comprising:

[0011] The device sends storage request information of the video slice index of the device to the video slice index storage end;

[0012] The video slice index storage end receives the storage request information and sends a request information for allocating a database instance to the device to the resource allocation end;

[0013] The resource allocation end receives a request for allocating a database instance to the device; the resource allocation end checks a memory to determine whether a database instance has been allocated to the device;

[0014] If it has been allocated, the resource allocation end returns the allocated database instance information and device information to the video slice index storage end; the video slice index storage end stores the video slice index of the device in the allocated database instance using the returned allocated database instance information and device information;

[0015] If not, the resource allocation end sends a request message for allocating a database instance to the device to the allocator; the allocator allocates a database instance to the device through the resource management database; the resource allocation end caches the database instance information and the device information allocated by the allocator into a memory, and returns the allocated database instance information and device information to the video slice index storage end; the video slice index storage end stores the video slice index of the device in the allocated database instance using the returned allocated database instance information and device information;

[0016] The specific steps of allocating a database instance to the device through a resource management database include:

[0017] The allocator queries the resource management database for the database instance with the most remaining resources according to the database instance resource table in the resource management database;

[0018] The allocator locks the database instance with the most remaining resources in the resource management database;

[0019] The allocator associates the request information with the locked database instance information to obtain the database instance resource allocation information of the device; and caches the database instance resource allocation information of the device in the database instance resource allocation table in the resource management database;

[0020] The allocator updates the database instance resource table using the allocated database instance resources;

[0021] The allocator checks whether there is dirty data in the locked database instance; if there is no dirty data, the allocator completes the allocation of the database instance to the device; if there is dirty data, the allocator reallocates the database instance to the device.

[0022] A further solution: the request information received by the resource allocation end includes the device number and the number of days of the device package.

[0023] A further solution: the database instance resource table includes the number of remaining resources for storage in each of the database instances in the resource management database.

[0024] Based on the above solution, the database instance resource table clearly records the resources of each database instance, making it convenient for the allocator to query.

[0025] A further solution: the number of days of the device's package is equal to the number of resources of the database instance occupied by the device; the allocator updates the remaining number of resources of the database instance in the database instance resource table by subtracting the number of resources occupied by the device from the remaining number of allocated database instances.

[0026] Based on the above solution, the number of remaining resources of the database instance in the database instance resource table is updated so that the allocator can accurately query the database instance with the most remaining resources in the resource management database.

[0027] The number of days of the package of the device is equal to the number of resources of the database instance occupied by the device, so that the utilization of the database instance resources is more reasonable.

[0028] Further solution: the database instance resource allocation table includes the device number, the number of days of the device package, the allocated database instance information and the number of remaining resources of the allocated database instance.

[0029] Based on the above solution, operation and maintenance personnel can quickly and clearly understand the association information between the device and the database instance by querying the database instance resource allocation table.

[0030] A further solution is: when the package days of the device are invalid or expired, the resources of the database instance occupied by the device are reclaimed.

[0031] Based on the above solution, the resources of the database instance occupied by the device whose package days have expired or expired are recovered, and the recovered database instance resources are reused, thereby saving the resources of the database instance.

[0032] A further solution: when the resources of each database instance in the resource management database exceed the set threshold, the resource management database receives a reminder from Feishu, then a new resource management database is added; and the newly added resource management database is initialized.

[0033] Based on the above solution, through the Feishu reminder, the operation and maintenance personnel can expand the capacity in time, avoiding the excessive amount of data in the resource management database causing the operation speed of the resource management database to decrease;

[0034] Initialization processing is performed on the newly added resource management database to prepare for the allocator to allocate a database instance to the device. The newly added resource management database will not affect the video slice index stored in the old resource management database.

[0035] Further solution: the threshold value set by the database instance is 80%.

[0036] Based on the above scheme, the threshold value set at 80% is the optimal resource storage state of the database instance; if it is less than 80%, the resources of the database instance will be wasted, and if it is greater than 80%, the resources of the database instance will be overused, resulting in an excessive amount of data stored in the database instance, thereby reducing the query speed of the database instance.

[0037] In a second aspect, the present invention provides an electronic device, comprising:

[0038] at least one memory for storing a program;

[0039] at least one processor, configured to execute the program stored in the memory;

[0040] an input device for inputting programs into the memory;

[0041] an output device, configured to display a result of the processor executing the program stored in the memory;

[0042] Among them, when the program stored in the memory is executed, the processor is used to execute a video slice index database storage partitioning method as described in the first aspect.

[0043] In a third aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to implement a library partitioning method for storing a video slice index database as described in the first aspect.

[0044] The beneficial effects of the present invention are:

[0045] The present invention enables each database instance in the resource management database to achieve relative balance; since the package days of each device are different, when the allocator allocates database instances to the devices, the allocator queries the database instance with the largest remaining resources in the resource management database based on the database instance resource table in the resource management database. After the database instance with the largest remaining resources stores the video slice index, the allocator will update the remaining resources of the database instance in the database instance resource table in real time, so that the resources of each database instance can be reasonably used, thereby achieving a relatively balanced amount of video slice indexes stored in each database instance.

[0046] In addition, taking into account the problem of concurrent allocation, the allocator locks the database instance with the most remaining resources to avoid allocating the same database instance when allocating database instances to other devices, resulting in excessive use of the database instance resources and destroying the balance of the video slice index storage between each database instance.

[0047] The present invention is applicable to resource management databases with different configurations. In the early stage, since the number of video slice indexes to be stored is small, a low-configuration resource management database can be applied, saving the cost of the resource management database. In the later stage, as the storage capacity of the video slice index increases, the configuration of the newly added resource management database can be specified, which is more convenient.

[0048] The beneficial effects of the present invention also include that the resource allocation end caches the database instance information and the device information allocated by the allocator into the memory. Since the speed of accessing the memory is relatively fast and can be controlled within 5ms, when the device subsequently repeatedly requests, the database instance information and the device information allocated by the allocator can be quickly returned to the video slice index storage end, which can meet the needs of massive video slice index storage services.

[0049] In summary, the video slice index database storage sub-library method of the present invention meets the video index storage sub-library requirements, effectively achieves the stability of the video slice index storage business, and not only supports the storage of video slice indexes of devices with different packages, but also makes the resources of each database instance in the resource management database relatively balanced, saving the cost of the resource management database. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 It is a flow chart of a library partitioning method for storing a video slice index database according to the present invention;

[0052] Figure 2 It is a structural diagram of a video slice index database storage method of the present invention;

[0053] Figure 3 It is a structural diagram of a video slice index storage device of a video slice index database storage method of the present invention;

[0054] Figure 4 It is a structural schematic diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0056] Example 1:

[0057] like Figure 1 or Figure 2 As shown, this embodiment provides a method for storing a video slice index database in a database partitioning manner, including:

[0058] The device sends storage request information of the video slice index of the device to the video slice index storage end;

[0059] The video slice index storage end receives the storage request information and sends a request information for allocating a database instance to the device to the resource allocation end;

[0060] The resource allocation end receives a request for allocating a database instance to the device; the resource allocation end checks a memory to determine whether a database instance has been allocated to the device;

[0061] If it has been allocated, the resource allocation end returns the allocated database instance information and device information to the video slice index storage end; the video slice index storage end stores the video slice index of the device in the allocated database instance using the returned allocated database instance information and device information;

[0062] If not, the resource allocation end sends a request message for allocating a database instance to the device to the allocator; the allocator allocates a database instance to the device through the resource management database; the resource allocation end caches the database instance information and the device information allocated by the allocator into a memory, and returns the allocated database instance information and device information to the video slice index storage end; the video slice index storage end stores the video slice index of the device in the allocated database instance using the returned allocated database instance information and device information;

[0063] The specific steps of allocating a database instance to the device through a resource management database include:

[0064] The allocator queries the resource management database for the database instance with the most remaining resources according to the database instance resource table in the resource management database;

[0065] The allocator locks the database instance with the most remaining resources in the resource management database;

[0066] The allocator associates the request information with the locked database instance information to obtain the database instance resource allocation information of the device; and caches the database instance resource allocation information of the device in the database instance resource allocation table in the resource management database;

[0067] The allocator updates the database instance resource table using the allocated database instance resources;

[0068] The allocator checks whether there is dirty data in the locked database instance; if there is no dirty data, the allocator completes the allocation of the database instance to the device; if there is dirty data, the allocator reallocates the database instance to the device.

[0069] Specifically, the video of the device is segmented into 30-second intervals, and each segmented 30-second video is called a slice; and the video slice index includes information such as the size, start time, end time, and storage location of a slice.

[0070] The database instance information includes database instance identification information (eg, number or label, etc.), which is used to identify different database instances.

[0071] The device information includes information such as the device number and the number of days of the device package.

[0072] The database instance resource is the storage space available for the database instance. For example, if a device's daily video cloud storage capacity is represented as one resource, and the video duration is sliced ​​into 30-second slices, then the number of video slice indexes that can be stored in one resource is 2880 (24 * 60 * 60 / 30 = 2880). Assuming the maximum storage capacity of a database instance is 5 million, then a database instance has approximately 1800 resources (5,000,000 / 2880 = 1800).

[0073] The allocator may allocate a database instance to the device not only through a resource management database, but also through Redis (memory database).

[0074] The allocator locks the database instance with the most remaining resources in the resource management database; the allocator may allocate database instances to multiple devices at the same time, which is called concurrent allocation; when the allocator allocates a database instance to one of the devices and queries the database instance with the most remaining resources, the allocator locks the queried database instance with the most remaining resources to prevent the allocator from querying the same database instance with the most remaining resources when allocating database instances to other devices, causing the resources in this database instance with the most remaining resources to be overused.

[0075] Among them, as an improved solution, specifically:

[0076] The request information received by the resource allocation terminal includes the device number and the number of days of the device package.

[0077] Specifically, the number of days in the package determines which videos shot during that day can be stored in the database instance of the resource management database. For example, if a device has a 7-day package, then videos shot by that device within those 7 days can be stored in the database instance of the resource management database. Videos shot beyond 7 days (e.g., on the 8th day) cannot be stored in the database instance of the resource management database.

[0078] The database instance resource table includes the number of remaining resources for storage in each database instance in the resource management database.

[0079] like Figure 3 As shown, the number of days of the package of the device is equal to the number of resources of the database instance occupied by the device; the allocator updates the remaining number of resources of the database instance in the database instance resource table by subtracting the number of resources occupied by the device from the remaining number of allocated database instances.

[0080] Specifically, assuming that the package duration of a device is 7 days, 7 resources of the database instance need to be occupied; if a database instance has about 1800 resources, the number of remaining resources of the database instance is 1793 (1800-7=1793).

[0081] The database instance resource allocation table includes the device number, the number of days of the device package, the allocated database instance information, and the remaining number of resources of the allocated database instance.

[0082] Specifically, such as Figure 3 As shown, there are three devices and three database instances; device 1, the package duration is 7 days; device 2, the package duration is 10 days; device 3, the package duration is 20 days; database instance 1, the remaining resources are 1800; database instance 2, the remaining resources are 1800; database instance 3, the remaining resources are 1800; the allocator allocates database instance 1 to device 1, the allocator allocates database instance 3 to device 2, and the allocator allocates database instance 2 to device 3.

[0083] The database instance resource allocation table includes: Device 1 [7 days]: Example 1 [Remaining resources 1793]; Device 2 [10 days]: Example 3 [Remaining resources 1790]; Device 3 [20 days]: Example 2 [Remaining resources 1780].

[0084] When the package days of the device are invalid or expired, the resources of the database instance occupied by the device are reclaimed.

[0085] For example, if device 1's package lasts 7 days, the allocated resources of database instance 1 are originally 1000, and after storing the video slice index of device 1, the remaining resources are 993. When device 1's package expires or expires, database instance 1 reclaims 7 resources, and the remaining resources of database instance 1 are now 1000.

[0086] When the resources of each database instance in the resource management database exceed the set threshold, the resource management database receives a reminder from Feishu, and then adds a new resource management database; and initializes the newly added resource management database.

[0087] The threshold value set in the database instance is 80%.

[0088] Specifically, Feishu Reminder can set the time, frequency, and reminder items, so that Feishu Reminder will periodically remind you at a fixed time. The operation and maintenance personnel of the resource management database can also create a threshold for database instance resources (for example, 80%) on Feishu Reminder. When the resource utilization rate of each database instance in the resource management database exceeds 80%, the Feishu Reminder will remind the operation and maintenance personnel to add a new resource management database.

[0089] For example, a resource management database can be configured with 128, 256, or 512 different database instances. Assuming a resource management database is configured with 256 different database instances, and each database instance has approximately 1,800 resources (5,000,000 / 2,880 = 1,800), then the resource management database has approximately 460,800 resources (256 * 1,800 = 460,800). The resource management database can store approximately 66,000 devices (460,800 / 7) with a 7-day data plan.

[0090] Example 2:

[0091] This embodiment provides an electronic device, including:

[0092] at least one memory for storing a program;

[0093] at least one processor, configured to execute the program stored in the memory;

[0094] an input device for inputting programs into the memory;

[0095] an output device, configured to display a result of the processor executing the program stored in the memory;

[0096] Among them, when the program stored in the memory is executed, the processor is used to execute a video slice index database storage partitioning method as described in Example 1.

[0097] Specifically, the input device may be a keyboard, a mouse, a camera, etc., and in this embodiment, the input device is a keyboard; the output device may be a speaker, a display, or a printer, etc., and in this embodiment, the input device is a display.

[0098] The memory is also called internal memory, which is the storage space that the processor can directly address. The memory includes random access memory (RAM), read-only memory (ROM), advanced cache and external memory.

[0099] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The central processing unit includes a controller (CU) and an arithmetic logic unit (ALU); the controller is the command system of the point-in-time device and completes the command work of the electronic device; the arithmetic logic unit is used to perform arithmetic operations and logical operations.

[0100] like Figure 4 As shown, the relationship between the memory, the processor, the input device and the output device is:

[0101] The input device inputs a program, and the input program is stored in the memory. The processor retrieves the program from the memory and executes the program. The output device displays the result of the processor executing the program.

[0102] Example 3:

[0103] This embodiment provides a computer-readable storage medium, which stores computer instructions. The computer instructions enable the computer to implement a video slice index database storage method as described in Example 1.

[0104] Specifically, when the computer instructions are executed on a computer, a library partitioning method for storing a video slice index database according to the above-mentioned embodiment 1 is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, or microwave) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive).

[0105] The present invention is not limited to the above-mentioned optional implementation modes. Anyone can derive other forms of products under the inspiration of the present invention. However, no matter what changes are made in the shape or structure, any technical solution that falls within the scope defined by the claims of the present invention falls within the scope of protection of the present invention.

Claims

1. A video slice index database storage method, characterized in that: include: The device sends storage request information of the video slice index of the device to the video slice index storage end; The video slice index storage end receives the storage request information and sends a request information for allocating a database instance to the device to the resource allocation end; The resource allocation end receives a request for allocating a database instance to the device; the resource allocation end checks a memory to determine whether a database instance has been allocated to the device; If it has been allocated, the resource allocation end returns the allocated database instance information and device information to the video slice index storage end; The video slice index storage terminal stores the video slice index of the device in the allocated database instance through the returned allocated database instance information and device information; If not, the resource allocator sends a request message for allocating a database instance to the device to the allocator; The allocator allocates a database instance to the device through a resource management database; the resource allocator caches the database instance information and the device information allocated by the allocator into a memory, and returns the allocated database instance information and device information to the video slice index storage end; The video slice index storage terminal stores the video slice index of the device in the allocated database instance through the returned allocated database instance information and device information; The specific steps of allocating a database instance to the device through a resource management database include: The allocator queries the resource management database for the database instance with the most remaining resources according to the database instance resource table in the resource management database; The allocator locks the database instance with the most remaining resources in the resource management database; The allocator associates the request information with the locked database instance information to obtain the database instance resource allocation information of the device; and caches the database instance resource allocation information of the device in the database instance resource allocation table in the resource management database; The allocator updates the database instance resource table using the allocated database instance resources; The allocator checks whether there is dirty data in the locked database instance; if there is no dirty data, the allocator completes the allocation of the database instance to the device; if there is dirty data, the allocator reallocates the database instance to the device.

2. The method for storing a video slice index database according to claim 1, wherein: The request information received by the resource allocation terminal includes the device number and the number of days of the device package.

3. The method for storing a video slice index database according to claim 2, characterized in that: The database instance resource table includes the number of remaining resources for storage in each database instance in the resource management database.

4. The method for storing a video slice index database according to claim 3, wherein: The number of days of the package of the device is equal to the number of resources of the database instance occupied by the device; the allocator updates the remaining number of resources of the database instance in the database instance resource table by subtracting the number of resources occupied by the device from the remaining number of allocated database instances.

5. The method for storing a video slice index database according to claim 4, characterized in that: The database instance resource allocation table includes the device number, the number of days of the device package, the allocated database instance information, and the remaining number of resources of the allocated database instance.

6. The video slice index database storage sub-database method according to claim 5, characterized in that: When the package days of the device are invalid or expired, the resources of the database instance occupied by the device are reclaimed.

7. The method for storing a video slice index database according to claim 1, wherein: When the resources of each database instance in the resource management database exceed the set threshold, the resource management database receives a reminder from Feishu, and then adds a new resource management database; and initializes the newly added resource management database.

8. The video slice index database storage sub-database method according to claim 7, characterized in that: The threshold value set in the database instance is 80%.

9. An electronic device, characterized in that: include: at least one memory for storing a program; at least one processor, configured to execute the program stored in the memory; an input device for inputting programs into the memory; an output device, configured to display a result of the processor executing the program stored in the memory; Wherein, when the program stored in the memory is executed, the processor is used to execute a library partitioning method for storing a video slice index database as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to implement a video slice index database storage method as described in any one of claims 1-8.

Images