Database control management method and device, electronic equipment and storage medium
By dynamically loading and cleaning multimodal data in the database, the problems of degradation in database performance and time-consuming cleaning of historical data are solved, and efficient data management and query performance are achieved.
Patent Information
- Application Number
- CN202411426339.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2025-02-11
AI Technical Summary
The prior art is difficult to effectively manage and clean up the large amount of memory space occupied by multimodal data, resulting in a degradation of database query writing performance, and the cleaning of historical search data is time-consuming and cannot meet the retrieval needs of cold data.
A database control management method is proposed, by obtaining multimodal data and storing it to the database according to preset policies, dynamically loading the target storage data to the temporary database in response to the data retrieval operation, and dynamically cleaning the temporary data according to the data storage period and the current time Data in the database.
It realizes storage, retrieval and dynamic cleaning of multimodal data, improves the database control and management efficiency, optimizes storage and retrieval efficiency, improves data query performance, and reduces manual intervention and error rates.
Smart Images
Figure CN119415576A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of database management, and in particular to a database control management method, device, electronic device and storage medium. Background Art
[0002] At present, in many fields and industries such as AI and big data, for large-scale multimodal data such as high-dimensional vector data and structured data, the retrieval of multimodal data requires a very large amount of memory space, and after multiple retrievals, a large amount of historical retrieval data often accumulates in the database, affecting the subsequent query and write performance of the database. At the same time, the cleaning of historical retrieval data is very time-consuming and cannot meet the retrieval of cold data in the database. Summary of the invention
[0003] The main purpose of the embodiments of the present application is to propose a database control management method, device, electronic device and storage medium, which can realize the storage, retrieval and dynamic cleaning of multimodal data and improve the control and management efficiency of the database.
[0004] On the one hand, an embodiment of the present application provides a database control management method, the method comprising the following steps: Acquire multimodal data, and store the multimodal data in a database according to a preset data storage strategy; In response to a data retrieval operation, obtaining a data retrieval condition and a data storage time range; Determine corresponding target storage data from the database according to the data storage time range and the data retrieval condition; Dynamically loading the target storage data into a temporary database, and connecting to the temporary database to perform data retrieval and display of the target storage data; The data storage period and current storage time corresponding to the target storage data are obtained, and the target storage data in the temporary database is dynamically cleared according to the data storage period and the current storage time.
[0005] In some embodiments, the method further comprises: The temporary storage time limit and the current time corresponding to the temporary database are obtained, and the current dynamic cleaning time is determined according to the current time and the temporary storage time limit. The current dynamic cleaning time is expressed by the following formula: ; in, is the current dynamic cleanup time, is the current time, For temporary storage period; Execute a scheduled cleaning task, obtain the data loading time corresponding to each temporary storage data in the temporary database according to a preset scheduled cleaning time interval, determine several expired storage data in the temporary database from multiple temporary storage data according to the data loading time corresponding to each temporary storage data and the current dynamic cleaning time, and dynamically clean up all the expired storage data in the temporary database.
[0006] In some embodiments, the database includes a plurality of sub-databases, and the acquiring of multimodal data, according to a preset data storage strategy, storing the multimodal data in the database, specifically includes: According to a plurality of preset data classifications, the multimodal data is divided to determine a number of data to be stored under each of the data classifications; The sub-databases corresponding to each of the data classifications are connected, and according to a preset strategy of partitioning tables by database, the data to be stored under each of the data classifications is stored in the corresponding sub-databases.
[0007] In some embodiments, the method further comprises: Acquire a preset capacity control cleaning strategy, determine the storage data to be cleaned in the temporary database according to the capacity control cleaning strategy, and dynamically clean the storage data to be cleaned in the temporary database; In response to the data cleaning operation, determining a data cleaning time range; According to the data cleaning time range, the designated cleaning data in the temporary database is determined, and the designated cleaning data in the temporary database is dynamically cleaned.
[0008] In some embodiments, dynamically loading the target storage data into a temporary database and connecting to the temporary database to perform data retrieval and display of the target storage data specifically includes: Get the preset partition time dimension; Obtain the database storage time corresponding to the target storage data, determine the storage partition of the target storage data in the temporary database according to the partition time dimension and the database storage time, and dynamically load the target storage data into the storage partition.
[0009] In some embodiments, obtaining a preset capacity control cleaning strategy, determining the storage data to be cleaned in the temporary database according to the capacity control cleaning strategy, and dynamically cleaning the storage data to be cleaned in the temporary database specifically includes: Obtaining a first capacity control threshold and a second capacity control threshold; According to a preset capacity control time interval, the current data capacity of the temporary database is determined, and a capacity control judgment is performed according to the current data capacity, the first capacity control threshold, and the second capacity control threshold; when it is determined that the temporary database meets the preset capacity control conditions, capacity control is performed on the temporary database according to a preset capacity control clearing strategy.
[0010] In some embodiments, the determining of the current data capacity of the temporary database according to a preset capacity control time interval, performing capacity control judgment according to the current data capacity, the first capacity control threshold, and the second capacity control threshold, and when it is determined that the temporary database meets the preset capacity control condition, performing capacity control on the temporary database according to a preset capacity control clearing strategy, specifically includes: When the current data capacity is greater than the first capacity control threshold but less than the second capacity control threshold, determining cold data in the temporary database, and dynamically clearing the cold data in the temporary database, wherein the cold data is data in the temporary database whose data access frequency is lower than a preset frequency threshold or whose data loading time exceeds a given storage time; When the current data capacity is greater than the second capacity control threshold, the scheduled cleaning task is executed, the current dynamic cleaning time is updated, and according to the current dynamic cleaning time, the corresponding multiple expired storage data are determined from the temporary database, and the multiple expired storage data in the temporary database are dynamically cleaned, wherein the data capacity of all the expired storage data is greater than the capacity difference, and the capacity difference is the difference between the current data capacity of the temporary database and the first capacity control threshold.
[0011] On the other hand, an embodiment of the present application provides a database control management device, the device comprising: The first module is used to obtain multimodal data and store the multimodal data in a database according to a preset data storage strategy; The second module is used to obtain data retrieval conditions and data storage time range in response to the data retrieval operation; A third module is used to determine corresponding target storage data from the database according to the data storage time range and the data retrieval condition; The fourth module is used to dynamically load the target storage data into a temporary database, and connect to the temporary database to perform data retrieval and display of the target storage data; The fifth module is used to obtain the data storage period and current storage time corresponding to the target storage data, and dynamically clean up the target storage data in the temporary database according to the data storage period and the current storage time.
[0012] On the other hand, an embodiment of the present application provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned database control management method when executing the computer program.
[0013] On the other hand, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the database control management method described above is implemented.
[0014] The embodiments of the present application include at least the following beneficial effects: the present application provides a database control management method, device, electronic device and storage medium, which obtains data retrieval conditions and data storage time range, determines the corresponding target storage data from the database, dynamically loads the target storage data into a temporary database, connects to the temporary database to perform data retrieval and display of the target storage data, obtains the data storage period and current storage time corresponding to the target storage data, and dynamically cleans up the target storage data in the temporary database according to the data storage period and current storage time. The present application can realize the storage, retrieval and dynamic cleaning of multimodal data, improve the control management efficiency of the database, optimize the storage efficiency and retrieval efficiency of the database, and improve the data query performance of the database. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 is a flow chart of a database control management method provided by an embodiment of the present application; Figure 2 is a schematic diagram of the structure of an unstructured vector database in an embodiment of the present application; Figure 3 Flow chart of step S104 in the embodiment of the present application; Figure 4 It is a schematic diagram of the structure of the database in the embodiment of the present application; Figure 5 is another flow chart of a database control management method provided by an embodiment of the present application; Figure 6 is another flow chart of a database control management method provided by an embodiment of the present application; Figure 7 is a flowchart of step S501 in an embodiment of the present application; Figure 8 is a flowchart of step S602 in an embodiment of the present application; Fig. 9 It is a structural diagram of a database control management device provided in an embodiment of the present application; Fig.10 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0016] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.
[0017] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".
[0018] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.
[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0020] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.
[0021] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.
[0022] 1) Column storage: refers to column storage databases. Column storage databases use column-related storage architecture to store data, and are mainly suitable for batch data processing and instant query. Including MongoDB and ClickHouse databases.
[0023] 2) Vector database: a database system used to store, manage, query and retrieve vectors. This article mainly refers to Milvus and self-developed vector databases.
[0024] 3) Rapid database cleanup: Rapid database cleanup is a multi-step process that ensures that the database is always in optimal condition. By clarifying the cleanup goals, cleaning up useless data, optimizing the database structure, using tools and scripts, and regularly monitoring and maintaining, the performance and maintainability of the database can be effectively improved.
[0025] 4) Multimodal application: A deep learning model that can simultaneously process and understand multiple types of data (such as text, images, audio, video, etc.) to realize intelligent applications in various fields.
[0026] 5) MongoDB: MongoDB is a database based on distributed file storage [1]. It is written in C++ and is designed to provide a scalable, high-performance data storage solution for web applications. The data structure it supports is very loose, in a bson format similar to json, so it can store relatively complex data types.
[0027] 6) ClickHouse: ClickHouse is a column-based database management system for online analytical processing (OLAP), supporting flexible MPP architecture and linear expansion. Its advantages include column-based storage, vectorized computing, high compression ratio and high performance.
[0028] 7) Milvus: Milvus is a vector database that can be used to store, index and manage massive vectors generated by deep neural network learning and other machine learning models.
[0029] With the rapid development of AI large model applications and big data technology, column-based and vector databases have been widely used, covering almost many fields and industries such as AI and big data. For large-scale multimodal data such as high-dimensional vector data and structured data, the retrieval of multimodal data requires a very large memory space, and after multiple retrievals, a large amount of historical retrieval data often accumulates in the database, affecting the query and write performance of the subsequent database. At the same time, the cleaning of historical retrieval data is very time-consuming and cannot meet the retrieval of cold data in the database.
[0030] Column-based and vector databases have become the core infrastructure for storing multimodal large models. As time goes by, a large amount of historical dynamic library vectors and structured attribute data often accumulate in the database, which not only occupies storage resources but also may affect the query and write performance of the data. Currently, the cleaning and retrieval of column-based and vector databases have the following problems: 1. The vector database may not meet different data cleaning requirements, resulting in slow cleaning speed; 2. When performing large-scale data cleansing, the read and write performance of column-based storage structured data is poor, which affects batch business writing and retrieval, thus affecting normal business.
[0031] Based on this, the embodiments of the present application propose a database control management method, device, electronic device and storage medium, which can improve the cleanliness and data quality of vector and column-stored databases, flexibly manage the life cycle of column-stored data and vector data, optimize storage efficiency, accelerate query performance, store retrieval data and historical cold data separately, reduce database capacity, integrate with historical data in real time, improve query efficiency, improve the level of automated cleaning of temporary databases, reduce manual intervention and reduce error rates, improve dynamic cleaning efficiency and accuracy, and support predictive cleaning of temporary databases to achieve capacity control, thereby improving database control management efficiency.
[0032] Reference Figure 1 , Figure 1 This is an optional flowchart of a database control management method provided in an embodiment of the present application. The method may include but is not limited to steps S101 to S105: Step S101, acquiring multimodal data, and storing the multimodal data in a database according to a preset data storage strategy; Step S102, in response to the data retrieval operation, obtaining data retrieval conditions and data storage time range; Step S103, determining corresponding target storage data from the database according to the data storage time range and data retrieval conditions; Step S104, dynamically loading the target storage data into a temporary database, connecting to the temporary database to perform data retrieval and display of the target storage data; Step S105, obtaining the data storage period and current storage time corresponding to the target storage data, and dynamically clearing the target storage data in the temporary database according to the data storage period and current storage time.
[0033] In some embodiments, the data retrieval conditions may include data retrieval key fields, data retrieval types, etc. According to the data retrieval conditions and the selected data storage time range, the data in the database is retrieved, the target storage data that meets the data retrieval conditions and the data storage time range is determined, the target storage data is dynamically loaded from the database to a temporary database, a temporary database is constructed in the database, and the temporary database is used to retrieve and display the target storage data, thereby reducing the read and write performance pressure of the database.
[0034] In some embodiments, the database includes multiple sub-databases, and step S101 may include but is not limited to steps S201 to S202: Step S201, dividing the multimodal data according to a plurality of preset data categories, and determining a number of data to be stored under each data category; Step S202, connecting the sub-databases corresponding to each data classification, and storing the data to be stored under each data classification into the corresponding sub-database according to the preset table division by database strategy.
[0035] In step S201 of some embodiments, optionally, the multimodal data is divided to obtain multiple unstructured vector data and multiple structured data. The database includes multiple sub-databases such as an unstructured vector database and a structured database. The unstructured vector database, such as the Milvus database, can be used to store multiple unstructured vector data. The structured database, such as MongoDB and ClickHouse database, can be used to store multiple structured data.
[0036] In step S202 of some embodiments, the multimodal data is stored in separate libraries and tables according to a library-table partitioning strategy. Specifically, the multimodal data includes a data source identifier. According to the data source identifier and data classification, the data to be stored under each data classification is stored in the corresponding sub-database.
[0037] Exemplarily, assume that there are different users, and each user captures multimodal data including videos, images, and texts through several camera devices. Each user has a unique user number, and each camera device also has a unique device number. First, the multimodal data is classified to obtain multiple unstructured vector data and multiple structured data. Each unstructured vector data has a corresponding user number and device number (such as the above-mentioned data source identifier). According to the user number and device number of the unstructured vector data, the corresponding target storage data table is determined in the unstructured vector database, and the unstructured vector data is stored in the target storage data table. Each structured data also has a corresponding user number and device number (such as the above-mentioned data source identifier). According to the user number and device number of the structured data, the corresponding target storage data table is determined in the structured database, and the structured data is stored in the target storage data table. Figure 2 , Figure 2 It is an optional structural diagram of the unstructured vector database in the embodiment of the present application. Taking the unstructured vector data storage as an example, it is assumed that there are N users, each user has M camera devices, and user 1 captures multimodal data through M camera devices with device numbers 11 to 1M. Then, storage data table 1 of user 1 is constructed in the unstructured vector database. Storage data table 1 of user 1 is used to store the table structure information of M sub-storage data tables. The sub-storage data tables are used to store the unstructured vector data captured by the camera devices. For example, sub-storage data table 1M is used to store the unstructured vector data captured by the camera device with device number 1M under user 1, and so on.
[0038] In some embodiments, reference Figure 3 , Figure 3 An optional flowchart of step S104 in the embodiment of the present application, step S104 may include but is not limited to steps S301 to S302: Step S301, obtaining a preset partition time dimension; Step S302, obtaining the database storage time corresponding to the target storage data, determining the storage partition of the target storage data in the temporary database according to the partition time dimension and the database storage time, and dynamically loading the target storage data into the storage partition.
[0039] In some embodiments, the partition time dimension may include but is not limited to year, month, week, day, etc., and can be set according to actual needs without limitation. Optionally, based on the preset partition time dimension and in combination with the database storage time range of the target storage data, multiple storage partitions are established in the temporary database. The database storage time is the time when the target storage data is stored in the database. The database storage time range is determined by the earliest database storage time and the latest database storage time in the target storage data. Then, based on the database storage time corresponding to the target storage data, the target storage time is stored in the storage partition corresponding to its database storage time.
[0040] In some embodiments, reference Figure 4 , Figure 4 It is an optional structural diagram of the database in the embodiment of the present application, wherein the database includes an unstructured vector database and a structured database, and a temporary database A1 to AN corresponding to the unstructured vector database is constructed in the database, and the temporary database A1 to AN is used to dynamically load multiple unstructured vector data transmitted from the unstructured vector database, and multiple temporary databases B1 to BN corresponding to the unstructured vector database are constructed in the database, and the temporary databases B1 to BN are used to dynamically load multiple structured data transmitted from the structured database.
[0041] In some embodiments, reference Figure 5 , Figure 5 This is another optional flow chart of a database control management method provided in an embodiment of the present application. The method may also include but is not limited to steps S401 to S402: Step S401, obtaining the temporary storage time limit and current time corresponding to the temporary database, and determining the current dynamic cleanup time according to the current time and the temporary storage time limit; Step S402, execute the scheduled cleaning task, obtain the data loading time corresponding to each temporary storage data in the temporary database according to the preset scheduled cleaning time interval, determine several expired storage data in the temporary database from multiple temporary storage data according to the data loading time corresponding to each temporary storage data and the current dynamic cleaning time, and dynamically clean up all expired storage data in the temporary database.
[0042] In step S401 of some embodiments, the current dynamic cleaning time is expressed by the following formula: ; in, is the current dynamic cleanup time, is the current time, Temporary storage period.
[0043] In step S402 of some embodiments, a scheduled cleanup task is executed, and the temporary database is scanned regularly according to the scheduled cleanup time interval, and multiple expired stored data in the temporary database whose data loading time is earlier than the current dynamic cleanup time are dynamically cleaned, thereby effectively reducing the storage pressure of the temporary database.
[0044] In some embodiments, reference Figure 6 , Figure 6 This is another optional flow chart of a database control management method provided by an embodiment of the present application. The method may also include but is not limited to steps S501 to S503: Step S501, obtaining a preset capacity control cleaning strategy, determining the storage data to be cleaned in the temporary database according to the capacity control cleaning strategy, and dynamically cleaning the storage data to be cleaned in the temporary database; Step S502, in response to the data cleaning operation, determining a data cleaning time range; Step S503, determining designated cleaning data in the temporary database according to the data cleaning time range, and dynamically cleaning the designated cleaning data in the temporary database.
[0045] In some embodiments, in response to a data cleaning operation, a data cleaning time range is determined, and based on the data cleaning time range, specified cleaning data in a temporary database whose database storage time is within the data cleaning time range is dynamically cleaned. Exemplarily, multiple temporary databases corresponding to the structured database MongoDB are dynamically cleaned, and a data id generation rule is obtained: temporary database number + data cleaning time range + cleaning task increment number. According to the data id generation rule, a user inputs a corresponding temporary database number and a data cleaning time range to generate a corresponding target data ID. Based on the temporary database number contained in the target data ID, a target temporary database is determined from multiple temporary databases. Based on the data cleaning time range contained in the target data ID, specified cleaning data in the target temporary database whose database storage time is within the data cleaning time range is dynamically cleaned.
[0046] In some embodiments, reference Figure 7 , Figure 7 This is an optional flowchart of step S501 in the embodiment of the present application. Step S501 may include but is not limited to steps S601 to S602: Step S601, obtaining a first capacity control threshold and a second capacity control threshold; Step S602, according to the preset capacity control time interval, determine the current data capacity of the temporary database, and perform capacity control judgment according to the current data capacity, the first capacity control threshold and the second capacity control threshold. When it is determined that the temporary database meets the preset capacity control conditions, capacity control is performed on the temporary database according to the preset capacity control clearing strategy.
[0047] In some embodiments, since the hardware resources of some database management projects are fixed and the resources cannot be elastically expanded, the dynamic data growing in the temporary database every day is not fixed. It is necessary to predictively clean up the data that is about to expire in the temporary database and control the current data storage capacity of the temporary database to reduce the storage pressure of the temporary database and maintain a certain read and write performance.
[0048] In some embodiments, reference Figure 8 , Figure 8 This is an optional flowchart of step S602 in the embodiment of the present application. Step S602 may include but is not limited to steps S701 to S702: Step S701, when the current data capacity is greater than the first capacity control threshold but less than the second capacity control threshold, determining the cold data in the temporary database, and dynamically cleaning the cold data in the temporary database, wherein the cold data is data in the temporary database whose data access frequency is lower than a preset frequency threshold or whose data loading time exceeds a given storage time; Step S702, when the current data capacity is greater than the second capacity control threshold, execute the scheduled cleaning task, update the current dynamic cleaning time, determine the corresponding multiple expired storage data from the temporary database according to the current dynamic cleaning time, and dynamically clean up the multiple expired storage data in the temporary database, wherein the data capacity of all expired storage data is greater than the capacity difference, and the capacity difference is the difference between the current data capacity of the temporary database and the first capacity control threshold.
[0049] In some embodiments, when the current data capacity is greater than the first capacity control threshold but less than the second capacity control threshold, the storage read and write performance of the temporary database is lower than when the current data capacity is less than the first capacity control threshold, and data reading and writing are slow. When the current data capacity is greater than the second capacity control threshold, the storage read and write performance of the temporary database is greatly reduced, and data reading and writing are extremely slow, which is not conducive to retrieval and display.
[0050] Exemplarily, the first capacity control threshold is set to 80% of the total data storage capacity of the temporary database, the second capacity control threshold is set to 95% of the total data storage capacity of the temporary database, and the capacity control time interval is 1 hour, that is, the capacity control of the temporary database is performed once every hour. When the current data capacity of the temporary database is greater than 80% but less than 95% of the total data storage capacity, the cold data in the temporary database is dynamically cleaned up. Specifically, according to the data loading time of each target storage data in the temporary database, that is, the time when the target storage data is loaded into the temporary database, each target storage data is sorted in descending order of time to obtain a preset standard capacity threshold. For example, if it is set to 60%, when the current data capacity of the temporary database is less than the standard capacity threshold, the storage read and write performance of the temporary database is relatively poor. Good, the data reading and writing speed is fast, according to the target storage data and the standard capacity threshold after descending sorting, the target storage data at the standard capacity threshold node is determined, and the data loading time of the target storage data is determined as the given storage time, and other target storage data whose data loading time exceeds the given storage time are determined as cold data, that is, the target storage data whose data loading time is earlier than the given storage time, and the cold data of the temporary database is dynamically cleaned up; when the current data capacity of the temporary database is greater than 95% of the total data storage capacity, the temporary storage time limit in the temporary database is updated multiple times to update the current dynamic cleaning time, and according to the current dynamic cleaning time, the scheduled cleaning task is executed to dynamically clean up all expired storage data in the temporary database until the current data capacity of the temporary database is less than the standard capacity threshold.
[0051] Reference Fig. 9 , Fig. 9 : is an optional structural diagram of a database control management device provided in an embodiment of the present application. The device is used to implement the above-mentioned database control management method. The device may include: The first module is used to obtain multimodal data and store the multimodal data in a database according to a preset data storage strategy; The second module is used to obtain data retrieval conditions and data storage time range in response to the data retrieval operation; The third module is used to determine the corresponding target storage data from the database according to the data storage time range and the data retrieval condition; The fourth module is used to dynamically load the target storage data into the temporary database and connect to the temporary database to perform data retrieval and display of the target storage data; The fifth module is used to obtain the data storage period and current storage time corresponding to the target storage data, and dynamically clean up the target storage data in the temporary database according to the data storage period and current storage time.
[0052] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0053] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above database control management method when executing the computer program. The electronic device can be any smart terminal including a tablet computer.
[0054] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0055] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes: The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application; The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the database control management method of the embodiment of this application; Input / output interface 903, used to implement information input and output; Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.); A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904); The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .
[0056] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned database control management method is implemented.
[0057] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0058] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0059] The embodiments of the present application provide a database control management method, device, electronic device, and storage medium, which obtain data retrieval conditions and data storage time range, determine the corresponding target storage data from the database, dynamically load the target storage data into a temporary database, connect to the temporary database to perform data retrieval and display of the target storage data, obtain the data storage period and current storage time corresponding to the target storage data, and dynamically clean up the target storage data in the temporary database according to the data storage period and current storage time. The embodiments of the present application can realize the storage, retrieval, and dynamic cleaning of multimodal data, improve the control management efficiency of the database, optimize the storage efficiency and retrieval efficiency of the database, and improve the data query performance of the database.
[0060] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0061] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0062] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0063] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0064] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0065] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0066] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0067] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0068] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0069] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in an assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed ASIC for this purpose.
[0070] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.
[0071] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.
Claims
1. A database control management method, characterized in that: The method comprises the following steps: Acquire multimodal data, and store the multimodal data in a database according to a preset data storage strategy; In response to a data retrieval operation, obtaining a data retrieval condition and a data storage time range; Determine corresponding target storage data from the database according to the data storage time range and the data retrieval condition; Dynamically loading the target storage data into a temporary database, and connecting to the temporary database to perform data retrieval and display of the target storage data; The data storage period and current storage time corresponding to the target storage data are obtained, and the target storage data in the temporary database is dynamically cleared according to the data storage period and the current storage time.
2. The database control management method according to claim 1, characterized in that: The method further comprises: The temporary storage time limit and the current time corresponding to the temporary database are obtained, and the current dynamic cleaning time is determined according to the current time and the temporary storage time limit. The current dynamic cleaning time is expressed by the following formula: ； in, is the current dynamic cleanup time, is the current time, For temporary storage period; Execute a scheduled cleaning task, obtain the data loading time corresponding to each temporary storage data in the temporary database according to a preset scheduled cleaning time interval, determine several expired storage data in the temporary database from multiple temporary storage data according to the data loading time corresponding to each temporary storage data and the current dynamic cleaning time, and dynamically clean up all the expired storage data in the temporary database.
3. The database control management method according to claim 1, characterized in that: The database includes a plurality of sub-databases, and the acquiring of multimodal data, according to a preset data storage strategy, storing the multimodal data in the database, specifically includes: According to a plurality of preset data classifications, the multimodal data is divided to determine a number of data to be stored under each of the data classifications; The sub-databases corresponding to each of the data classifications are connected, and according to a preset strategy of partitioning tables by database, the data to be stored under each of the data classifications is stored in the corresponding sub-databases.
4. The database control management method according to claim 1, characterized in that: The method further comprises: Acquire a preset capacity control cleaning strategy, determine the storage data to be cleaned in the temporary database according to the capacity control cleaning strategy, and dynamically clean the storage data to be cleaned in the temporary database; In response to the data cleaning operation, determining a data cleaning time range; According to the data cleaning time range, the designated cleaning data in the temporary database is determined, and the designated cleaning data in the temporary database is dynamically cleaned.
5. The database control management method according to claim 1, characterized in that: The dynamically loading the target storage data into a temporary database and connecting to the temporary database to perform data retrieval and display of the target storage data specifically includes: Get the preset partition time dimension; Obtain the database storage time corresponding to the target storage data, determine the storage partition of the target storage data in the temporary database according to the partition time dimension and the database storage time, and dynamically load the target storage data into the storage partition.
6. The database control management method according to claim 2, characterized in that: The obtaining of a preset capacity control cleaning strategy, determining the storage data to be cleaned in the temporary database according to the capacity control cleaning strategy, and dynamically cleaning the storage data to be cleaned in the temporary database specifically includes: Obtaining a first capacity control threshold and a second capacity control threshold; According to a preset capacity control time interval, the current data capacity of the temporary database is determined, and a capacity control judgment is performed according to the current data capacity, the first capacity control threshold, and the second capacity control threshold; when it is determined that the temporary database meets the preset capacity control conditions, capacity control is performed on the temporary database according to a preset capacity control clearing strategy.
7. The database control management method according to claim 6, characterized in that: The step of determining the current data capacity of the temporary database according to a preset capacity control time interval, performing capacity control judgment according to the current data capacity, the first capacity control threshold, and the second capacity control threshold, and performing capacity control on the temporary database according to a preset capacity control clearing strategy when it is determined that the temporary database meets the preset capacity control condition, specifically includes: When the current data capacity is greater than the first capacity control threshold but less than the second capacity control threshold, determining cold data in the temporary database, and dynamically clearing the cold data in the temporary database, wherein the cold data is data in the temporary database whose data access frequency is lower than a preset frequency threshold or whose data loading time exceeds a given storage time; When the current data capacity is greater than the second capacity control threshold, the scheduled cleaning task is executed, the current dynamic cleaning time is updated, and according to the current dynamic cleaning time, the corresponding multiple expired storage data are determined from the temporary database, and the multiple expired storage data in the temporary database are dynamically cleaned, wherein the data capacity of all the expired storage data is greater than the capacity difference, and the capacity difference is the difference between the current data capacity of the temporary database and the first capacity control threshold.
8. A database control management device, characterized in that: The device comprises: The first module is used to obtain multimodal data and store the multimodal data in a database according to a preset data storage strategy; The second module is used to obtain data retrieval conditions and data storage time range in response to the data retrieval operation; A third module is used to determine corresponding target storage data from the database according to the data storage time range and the data retrieval condition; The fourth module is used to dynamically load the target storage data into a temporary database, and connect to the temporary database to perform data retrieval and display of the target storage data; The fifth module is used to obtain the data storage period and current storage time corresponding to the target storage data, and dynamically clean up the target storage data in the temporary database according to the data storage period and the current storage time.
9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the database control management method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the database control management method according to any one of claims 1 to 7 is implemented.