A database data processing method, device and computer readable storage medium
By using frequency and citation frequency to assess data importance, the data processing problem when the database capacity reaches its limit is solved, enabling differentiated data retention and new data entry, thus ensuring the preservation of important data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG DAHUA TECH CO LTD
- Filing Date
- 2022-05-24
- Publication Date
- 2026-06-05
AI Technical Summary
When the database capacity reaches its limit, existing technologies struggle to effectively distinguish between important and unimportant data, resulting in new data failing to be properly stored or old data being indiscriminately deleted, potentially leading to the loss of important data.
By using two metrics, frequency of use and frequency of citation, the importance of target data can be assessed. Combining these metrics, low-importance data can be deleted when database capacity is limited, ensuring that new data can be successfully added to the database.
This technology enables the selective retention of data based on its importance and relevance, even with limited database capacity. This avoids the loss of important data and ensures that new data can be successfully imported into the database.
Smart Images
Figure CN115203182B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data analysis technology, and in particular to a database data processing method, device, and computer-readable storage medium. Background Technology
[0002] With the development of computer network technology, and in today's era of analog and digital high-definition development, intelligence is becoming an increasingly important trend. This is accompanied by massive data analysis and processing, and databases are undoubtedly the best choice for storing such massive amounts of data.
[0003] Due to limitations such as the performance of monitoring equipment, the database capacity has an upper limit. Once the data inserted into the database accumulates to a certain extent, it will eventually reach this upper limit. At this point, either the data insertion operation must be stopped and subsequent new data cannot be saved, or the oldest data should be cleared first according to the insertion order before the new data is inserted to ensure that the newly generated useful data can be saved. Summary of the Invention
[0004] The main technical problem solved by this invention is to provide a database data processing method, device and computer-readable storage medium that can selectively retain important data when the database capacity is insufficient.
[0005] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is: to provide a database data processing method, which includes: obtaining the usage frequency and citation frequency of target data in the database, wherein the usage frequency is the number of times the target data is used, and the citation frequency is the number of times the target data is referenced by related data; when the database needs to delete data, the target data with low importance is deleted by combining the usage frequency and citation frequency, wherein the importance of the target data is proportional to the usage frequency and / or citation frequency.
[0006] Specifically, the process of deleting target data with low importance by combining citation frequency and usage frequency includes: determining whether the citation frequency of the target data is the preset citation frequency; if the citation frequency is not the preset citation frequency, then retaining the target data; if the citation frequency is the preset citation frequency, then determining whether the usage frequency of the target data is less than the preset usage frequency; if the usage frequency of the target data is less than the preset usage frequency, then deleting the target data.
[0007] The process of deleting target data with low importance by combining citation frequency and usage frequency includes: determining whether the citation frequency of the target data is the preset citation frequency; if the citation frequency is not the preset citation frequency, then retaining the target data; if the citation frequency is the preset citation frequency, then obtaining all target data with the preset citation frequency, sorting the usage frequency of the target data, and deleting a preset number of target data with a low usage frequency.
[0008] The database data processing method also includes: if at least two target data have the same frequency of use, sort the target data according to the time of writing to the database, and delete the target data that was written to the database earlier.
[0009] Specifically, the process of deleting low-importance target data by combining citation frequency and usage frequency includes: determining whether the usage frequency of the target data is the same; if the usage frequencies are different, sorting the usage frequencies and deleting a predetermined number of target data with lower usage frequencies; if the usage frequencies are the same, determining whether the citation frequencies are the same; if the citation frequencies are different, sorting the citation frequencies and deleting a predetermined number of target data with lower citation frequencies; and if the citation frequencies are the same, sorting the target data by the time it was written to the database and prioritizing the deletion of target data written to the database earlier.
[0010] The process of obtaining the usage frequency and reference frequency of target data in the database includes: the initial values of reference frequency and usage frequency are zero; when the target data is written to the database or used, the usage frequency of the target data is incremented by one each time it is used; when the target data is referenced by related data, the reference frequency of the target data is incremented by one each time it is referenced.
[0011] Among them, obtaining the reference frequency of target data in the database includes: when the associated data referencing the target data is deleted, the usage frequency and reference frequency of the target data are reduced by one.
[0012] The target data is facial data, and the associated data is license plate data. The frequency of use and reference of the target data in the database are obtained as follows: when facial data is written to the database or when facial data is used, the frequency of use of facial data is incremented by one each time it is used; when license plate data is written to the database, the frequency of use of license plate data is incremented by one; when license plate data is associated with facial data, the frequency of use and reference of facial data are incremented by one.
[0013] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is to provide a database data processing device, which includes a processor for executing the above-mentioned database data processing method.
[0014] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is to provide a computer-readable storage medium for storing instruction / program data, which can be executed to implement the above-mentioned database data processing method.
[0015] The beneficial effects of this invention are as follows: Unlike existing technologies, this invention proposes two indicators: usage frequency and citation frequency. By statistically storing the usage frequency and citation count of data when it is entered into the database, and considering the limited database capacity, by comparing and judging the usage frequency and citation count of each target data in the database, it is possible to know the activity frequency of the target within the control area and the number of times the target data is reused in other modules. This allows for the determination of the attention and importance of the target data. Based on the importance of the data, unimportant data is deleted to clear space for unimportant data and ensure that new data can be entered into the database normally. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating one embodiment of the database data processing method of this application;
[0017] Figure 2 This is a flowchart illustrating another embodiment of the database data processing method of this application;
[0018] Figure 3 This is a flowchart illustrating a specific implementation of the database data processing method of this application;
[0019] Figure 4 This is a schematic diagram of the structure of the database data processing device in the embodiments of this application;
[0020] Figure 5 This is a schematic diagram of the structure of the database data processing device in the embodiments of this application;
[0021] Figure 6 This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of this application. Detailed Implementation
[0022] To make the objectives, technical solutions, and effects of the present invention clearer and more explicit, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.
[0023] This application provides a database data processing method that proposes two indicators: usage frequency and citation frequency. By statistically storing the usage frequency and citation count of data when it is entered into the database, and considering the limited database capacity, by comparing and judging the usage frequency and citation count of each target data in the database, it is possible to know the activity frequency of the target within the control area and the number of times the target data is reused in other modules. This allows for the determination of the attention and importance of the target data. Based on the importance of the data, unimportant data is deleted to clear space for unimportant data and ensure that new data can be entered into the database normally.
[0024] Please see Figure 1 , Figure 1This is a flowchart illustrating one embodiment of the database data processing method of this application. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that result. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, this embodiment includes:
[0025] S110: Obtain the usage frequency and citation frequency of the target data in the database.
[0026] To obtain the usage information of target data in the database, this application uses usage frequency and citation frequency to evaluate the usage of target data, where usage frequency is the number of times the target data is used and citation frequency is the number of times the target data is cited by related data.
[0027] S130: When the database needs to delete data, combine usage frequency and citation frequency to delete target data with low importance.
[0028] Databases have a capacity limit; when the amount of data accumulates to a certain level, data deletion is necessary. Based on the usage of the target data, less important target data is deleted. The importance of target data is directly proportional to its usage frequency and / or citation frequency. A certain number of target data items with low usage and / or citation frequencies are selected for deletion.
[0029] In this implementation, by statistically storing the frequency of data usage and the reference count when data is entered into the database, and considering the limited database capacity, the frequency of activity of the target within the controlled area and the number of times the target data is reused in other modules can be determined by comparing and judging the frequency of use and the reference count of each target data in the database. This allows the focus and importance of the target data to be determined, and unimportant data can be deleted according to the importance of the data to clear unimportant data space and ensure that new data can be entered into the database normally.
[0030] Please see Figure 2 , Figure 2 This is a flowchart illustrating another embodiment of the database data processing method of this application. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that outcome. Figure 2 The illustrated process sequence is limited. For example... Figure 2 As shown, this embodiment includes:
[0031] S210: Obtain the usage frequency and citation frequency of the target data in the database.
[0032] To assess the usage of target data in the database, this application utilizes usage frequency and citation frequency. Usage frequency refers to the number of times the target data is used, i.e., the number of times it is used and hit within the database. This reflects the activity frequency of the target data within the monitored or controlled area; a higher usage frequency indicates more frequent target activity and greater importance of the target data. Citation frequency refers to the number of times the target data is referenced by related data, i.e., the number of times the target data is used in conjunction with other modules outside the database. Citation frequency reflects the level of attention the target data receives within the monitored or controlled area, indicating that more than one function or module requires the target data, and that it is also used in other module databases. A higher citation frequency indicates a greater correlation, wider scope, and greater utility of the target data, making it more important. The initial values for target citation frequency and usage frequency in the database are zero. When the target data is first detected and written to the database, the usage frequency is incremented by one. Each time the target data is used, the usage frequency is incremented by one; each time the target data is referenced by related data, the citation frequency is incremented by one. Specifically, when target data is referenced by other data, both the usage frequency and the reference frequency of that data are incremented by one.
[0033] S230: Determine if the database capacity meets the conditions for deleting data.
[0034] Databases have a capacity limit; when the amount of data in the database accumulates to a certain level, data needs to be deleted. Therefore, it is necessary to first determine whether the database capacity meets the conditions for data deletion. In one implementation, when the database capacity is full, data deletion is selected. In another implementation, a database capacity warning value is established, and a configuration switch is provided to enable or disable the intelligent deletion strategy when the warning value is reached. When the currently stored data in the database reaches the warning value, the intelligent deletion strategy configuration option is enabled, thus meeting the data deletion conditions.
[0035] S250: Combine usage frequency and citation frequency to delete target data with low importance.
[0036] Based on the usage of the target data, delete target data with lower importance. The importance of target data is directly proportional to its usage frequency and / or citation frequency. Select and delete a certain number of target data items with low usage and / or citation frequencies.
[0037] In one implementation, the reference frequency of the target data is first determined, followed by the usage frequency. Specifically, it is determined whether the reference frequency of the target data is a preset reference frequency; if the reference frequency is not the preset reference frequency, the target data is retained; if the reference frequency is the preset reference frequency, it is then determined whether the usage frequency of the target data is less than a preset usage frequency; if the usage frequency is less than the preset usage frequency, the target data is deleted. In one specific implementation, the preset reference frequency is zero. If the reference frequency is not zero, it indicates that the target data has associated data and is currently being used, so the target data is retained. If the reference frequency is zero, it indicates that the target data is not currently associated with other data, therefore, its usage frequency is then determined.
[0038] In another implementation, the reference frequency of the target data is determined first, followed by the usage frequency. Specifically, it is determined whether the reference frequency of the target data is a preset reference frequency; if the reference frequency is not a preset reference frequency, the target data is retained; if the reference frequency is a preset reference frequency, all target data with a preset reference frequency are obtained, and the usage frequencies of the target data are sorted, deleting a preset number of target data with lower usage frequencies. If at least two target data have the same usage frequency during the sorting process, the target data with the same usage frequency are sorted according to the time they were written to the database, and the target data written to the database earlier is deleted first.
[0039] In another implementation, the usage frequency of the target data is first determined, followed by the reference frequency. Specifically, it is determined whether the usage frequencies of the target data are the same. If the usage frequencies are different, the usage frequencies are sorted, and a predetermined number of target data with lower usage frequencies are deleted. If the usage frequencies are the same, the reference frequencies are then determined. If the reference frequencies are different, the reference frequencies are sorted, and a predetermined number of target data with lower reference frequencies are deleted. If the reference frequencies are the same, the target data with the same reference frequency are sorted by the time they were written to the database, and the target data written to the database earlier is deleted first.
[0040] S270: When the associated data referencing the target data is deleted, the usage frequency and reference frequency of the target data are reduced by one.
[0041] When target data is referenced by other related data, both the usage frequency and the reference frequency are incremented by one. When the related data referencing the target data is deleted, the target data loses that related data, and both the usage frequency and the reference frequency of the target data are decremented by one.
[0042] In this implementation, by statistically storing the frequency of data usage and the citation count when data is entered into the database, and considering the limited database capacity, the frequency of activity of each target data within the controlled area and the number of times the target data is reused in other modules can be determined by comparing and judging the frequency of use and citation count of each target data in the database. This allows for the assessment of the target data's attention and importance. Based on the data's importance, less important data is deleted to clear space for unimportant data, ensuring that new data can be entered into the database normally. Simultaneously, strategies and methods are implemented to set database capacity warning values and automatically enable intelligent overwrite to ensure that new data can be entered into the database normally.
[0043] In one specific implementation, the entrance and exit checkpoint equipment in residential communities or large shopping malls supports both facial recognition and license plate recognition. Data detected by both the facial recognition and license plate recognition modules is added to and stored in their respective databases. License plate recognition also needs to detect the driver and front passenger, and bind this information to the identified vehicle information to prevent vehicle theft or to quickly locate the vehicle owner in the event of theft. Please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a flowchart illustrating a specific implementation of the database data processing method of this application. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that result. Figure 3 The illustrated process sequence is limited. For example... Figure 3 As shown, this embodiment includes:
[0044] First, the target data is input for detection, and its usage frequency and citation frequency are counted. In this implementation, the target data includes license plate data and facial data. Initially, the usage frequency and citation frequency of both license plate data and facial data are zero. When facial data is written to the database or used, its usage frequency is incremented by one each time it is used; when license plate data is written to the database, its usage frequency is incremented by one, and when license plate data is associated with facial data, both the usage frequency and citation frequency of facial data are incremented by one.
[0045] Specifically, when a vehicle owner enters or exits through the entrance / exit for the first time without driving, their facial information is detected and added to the database, and the usage frequency of the facial data is incremented by one. Subsequent detections of the same person entering or exiting the entrance / exit accumulate the number of times that person appears and update the usage frequency of the facial data in the database. When a vehicle owner drives through the entrance / exit, vehicle and driver information are recognized. The detected driver's facial information is then compared with facial data already added to the facial database. If the driver is in the database, the usage frequency and citation frequency of that facial data are both incremented by one. If the driver's facial information is not in the database, it is added as newly detected facial data, and both the usage and citation frequencies are set to one and updated in the facial database. When a vehicle enters before, the citation frequency of the same facial information associated with it does not increase.
[0046] The system monitors the database in real time to see if the data reaches the warning threshold. A database capacity warning threshold is established. If the current data stored in the database has not reached the warning threshold, the target data ingestion operation continues. When the current data stored in the database reaches the capacity warning threshold, it determines whether to enable the intelligent overwrite strategy. If the intelligent overwrite strategy is enabled, the target data is deleted using a pre-defined deletion strategy. If the intelligent overwrite strategy is not enabled, it checks if the database capacity is full. If the database capacity is full and the data cannot be overwritten, data ingestion is stopped. If the database data can be overwritten and the overwrite strategy is intelligent overwrite, the target data is deleted using a pre-defined deletion strategy. If the overwrite strategy is not intelligent overwrite, the target data is deleted based on its ingestion time, prioritizing the earlier ingested data.
[0047] In this implementation, the intelligent overlay strategy first obtains the usage frequency and citation frequency of face data and license plate data in the database. It then determines whether the usage frequencies of the target data are the same. If the usage frequencies are different, the target data with the lowest usage frequency is deleted. If the usage frequencies are the same, it further determines whether the citation frequencies are the same. If the citation frequencies are different, the target data with the lowest citation frequency is deleted. If the citation frequencies are the same, the target data that was entered into the database earlier is deleted first, based on the data's entry time.
[0048] Specifically, when license plate data is deleted from the license plate database, the corresponding driver's facial data must be deleted simultaneously. This means the usage frequency and citation frequency of the facial data linked to the license plate information in the facial database are decremented by one and updated in the facial database. However, when facial data is deleted from the facial database, the usage frequency and citation frequency of the license plate data remain unaffected.
[0049] In this embodiment, this application sets a data capacity warning value and provides a method for whether to enable intelligent overwriting when the warning value is reached. This clears non-critical data space at necessary times to ensure that new data can be normally stored. The database full-capacity handling strategy proposed in this application adds an intelligent overwriting method. When the database capacity is full, it analyzes and statistically analyzes the number of face data matching hits and the number of times face data is referenced in other databases or related functions. Based on the statistical results, it optimizes the data deletion logic to ensure that valuable data is stored in the database to the maximum extent. This avoids situations where choosing a no-insertion strategy results in data not being able to be stored, or choosing an overwriting strategy results in data being indiscriminately overwritten, potentially leading to the loss of some important data. This maximizes the storage time of important data in the database.
[0050] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a database data processing device according to an embodiment of this application. In this embodiment, the database data processing device includes an acquisition module 41 and a deletion module 42.
[0051] The acquisition module 41 is used to acquire the usage frequency and citation frequency of target data in the database. The usage frequency is the number of times the target data is used, and the citation frequency is the number of times the target data is referenced by related data. The deletion module 42 is used to delete target data with low importance when the database needs to delete data, combining the usage frequency and citation frequency. The importance of target data is directly proportional to the usage frequency and / or citation frequency. This database data processing device is used to statistically save the usage frequency and citation count of data when data is entered into the database. When the database capacity is limited, by comparing and judging the usage frequency and citation count of each target data in the database, it is possible to know the activity frequency of the target within the control area and the number of times the target data is repeatedly used in other modules. Thus, the attention and importance of the target data can be determined, and unimportant data can be deleted according to the importance of the data to clear unimportant data space and ensure that new data can be entered into the database normally.
[0052] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a database data processing device according to an embodiment of this application. In this embodiment, the database data processing device 51 includes a processor 52.
[0053] Processor 52 can also be referred to as a CPU (Central Processing Unit). Processor 52 may be an integrated circuit chip with signal processing capabilities. Processor 52 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor, or processor 52 can be any conventional processor.
[0054] The database data processing device 51 may further include a memory (not shown) for storing instructions and data required for the processor 52 to run.
[0055] The processor 52 is used to execute instructions to implement the methods provided by any embodiment and any non-conflicting combination of the database data processing methods of this application described above.
[0056] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of this application. The computer-readable storage medium 61 in this embodiment stores instruction / program data 62. When executed, this instruction / program data 62 implements the methods provided by any embodiment of the database data processing method of this application and any non-conflicting combination thereof. The instruction / program data 62 can be formed into a program file and stored in the aforementioned storage medium 61 in the form of a software product, so that a computer device (which may be a personal computer, server, or network device, etc.) or processor can execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium 61 includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or terminal devices such as computers, servers, mobile phones, and tablets.
[0057] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.
[0058] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0059] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.
Claims
1. A database data processing method, characterized in that, The method includes: Obtain the usage frequency and reference frequency of target data in the database, where the usage frequency is the number of times the target data is used and the reference frequency is the number of times the target data is referenced by associated data; When the database needs to delete data, it combines the usage frequency and the reference frequency to delete the target data with low importance. The importance of the target data is proportional to the usage frequency and the reference frequency. The step of combining the citation frequency and the usage frequency to delete the target data with low importance includes: Determine whether the usage frequency of the target data is the same. If the usage frequency of the target data is different, sort the usage frequency of the target data and delete a preset number of the target data with the smaller usage frequency. If the usage frequency of the target data is the same, then it is determined whether the reference frequency of the target data is the same. If the reference frequency of the target data is different, then the reference frequency of the target data is sorted and a preset number of target data with smaller reference frequencies are deleted. If the reference frequency of the target data is the same, the target data is sorted according to the time it was written into the database, and the target data that was written into the database earlier is deleted first.
2. The database data processing method according to claim 1, characterized in that, The step of combining the citation frequency and the usage frequency to delete the target data with low importance includes: Determine whether the citation frequency of the target data is zero; If the reference frequency is not zero, then the target data is retained; If the reference frequency is zero, then it is determined whether the usage frequency of the target data is less than the preset usage frequency. If the usage frequency of the target data is less than the preset usage frequency, then the target data is deleted.
3. The database data processing method according to claim 1, characterized in that, The step of combining the citation frequency and the usage frequency to delete the target data with low importance includes: Determine whether the citation frequency of the target data is zero; If the reference frequency is not zero, then the target data is retained; If the reference frequency is zero, then all target data with a reference frequency of zero are obtained, the usage frequency of the target data is sorted, and a preset number of target data with a smaller usage frequency are deleted.
4. The database data processing method according to claim 3, characterized in that, The method further includes: If at least two target data have the same usage frequency, the target data are sorted according to the time they were written into the database, and the target data written into the database earlier is deleted first.
5. The database data processing method according to claim 1, characterized in that, The acquisition of the usage frequency and citation frequency of the target data in the database includes: The initial values for the citation frequency and the usage frequency are zero; When the target data is written to the database or when the target data is used, the usage frequency of the target data is incremented by one each time it is used. When the target data is referenced by the associated data, the reference frequency of the target data is incremented by one each time it is referenced.
6. The database data processing method according to claim 5, characterized in that, The frequency of references to the target data in the database includes: When the associated data referencing the target data is deleted, the usage frequency and the reference frequency of the target data are decreased by one.
7. The database data processing method according to claim 5, characterized in that, The target data is facial data, the associated data is license plate data, and obtaining the usage frequency and citation frequency of the target data in the database includes: When the face data is written to the database or when the face data is used, the usage frequency of the face data is incremented by one each time it is used. When the license plate data is written to the database, the usage frequency of the license plate data is incremented by one, the license plate data is associated with the face data, and the usage frequency and reference frequency of the face data are incremented by one.
8. A database data processing device, characterized in that, Includes a processor, the processor being configured to execute instructions to implement the database data processing method as described in any one of claims 1-7.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store instruction / program data that can be executed to implement the database data processing method as described in any one of claims 1-7.