A database upgrading method, device and system

Abstract

The application discloses a database upgrading method, which converts the to-be-upgraded data version in a first state in a database into a to-be-upgraded data version in a second state in response to an upgrading request, the to-be-upgraded data version in the second state comprises: a current version updated by a target version comprising at least one alternative version, a preferred version selected from the alternative version, and a to-be-upgraded data version in a third state converted from the to-be-upgraded data version in the second state, wherein the to-be-upgraded data version in the third state comprises: a current version updated by the preferred version as the target version, and after the conversion is completed, the to-be-upgraded data version in the third state is converted into a data version of upgraded data in the first state, and the data version of the upgraded data in the first state comprises: a current version updated by the preferred version as the target version. The application realizes smooth database upgrading.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular, to a method for upgrading a database. Background Technology

[0002] Databases serve as the data carriers for various business operations. As these operations grow, the amount of data in each operation increases significantly, leading to ever-growing database sizes and making smooth upgrades difficult. For example, in target detection operations based on feature databases, updating the massive amounts of feature data in the database typically takes a considerable amount of time, such as several days. If feature data updates cause operational disruptions, it will negatively impact the user experience. Summary of the Invention

[0003] This invention provides a database upgrade method to achieve seamless database upgrades.

[0004] This invention provides a database upgrade method, the method comprising:

[0005] In response to the upgrade request, the data version to be upgraded in the database in the first state is converted to the data version to be upgraded in the second state. The data version to be upgraded in the second state includes: the current version updated using a target version that includes at least one alternative version.

[0006] Select a preferred version from the candidate versions, and convert the data version to be upgraded in the second state to the data version to be upgraded in the third state. The data version to be upgraded in the third state includes: the current version updated with the preferred version as the target version.

[0007] After the conversion from the data version to be upgraded in the second state to the data version to be upgraded in the third state is completed, the data version to be upgraded in the third state is converted into the data version of the upgraded data in the first state. The data version of the upgraded data in the first state includes: the current version updated with the preferred version as the target version;

[0008] in,

[0009] The first state is the normal state before or after the upgrade, while the second and third states are intermediate states during the upgrade process.

[0010] Preferably, the method further includes:

[0011] During the upgrade of the data to be upgraded, in response to business requests from clients, business processing is performed according to the current version of the database corresponding to the data requested by the business request;

[0012] The step of selecting a preferred version from the candidate versions includes: selecting the candidate version with the highest affinity as the preferred version based on the affinity between the candidate versions and the business, wherein affinity is used to characterize the degree of matching between the candidate version and the business.

[0013] Preferably, the data version to be upgraded in the first state also includes: an existing version that was consistent with the current version before responding to the upgrade request.

[0014] The data version to be upgraded in the second state also includes: an existing version that is consistent with the current version.

[0015] The data version to be upgraded in the third state also includes: an existing version that is inconsistent with the current version.

[0016] The method further includes:

[0017] In response to the version rollback operation, if the data version to be upgraded is in the third state, the data version to be upgraded will be converted to the data version to be upgraded in the second state; if the data version to be upgraded is in the second state, the data version to be upgraded will be converted to the data version to be upgraded in the first state.

[0018] Preferably, the first state, the second state, and the third state are identified by a version attribute variable.

[0019] If the attribute variable of the first version equals the attribute variable of the second version, and the attribute variable of the second version equals the attribute variable of the third version, then it is the first state.

[0020] If the first version attribute variable equals the second version attribute variable, and the second version attribute variable is one of the version attribute variables in the third version attribute variable, then it is the second state.

[0021] If the attribute variable of the first version is not equal to the attribute variable of the second version, and the attribute variable of the second version is equal to the attribute variable of the third version, then it is in the third state.

[0022] in,

[0023] The first version attribute variable is used to represent the existing version, the second version attribute variable is used to represent the current version, and the third version attribute variable is used to represent the target version.

[0024] Preferably, the data to be upgraded is feature data to be upgraded, and the affinity is determined as follows:

[0025] For each alternative version

[0026] The similarity between the business feature data requested in the business request and the feature data in the alternative version is calculated, and those with similarity scores greater than a set similarity threshold are selected.

[0027] For each selected similarity, the product of that similarity and its corresponding evaluation value is calculated. The evaluation value is determined based on the difference between a set threshold and the similarity, and the threshold is related to the feature extraction model used for feature extraction.

[0028] Sum all the product results to obtain the affinity of the alternative version.

[0029] Preferably, the data to be upgraded is existing feature data to be upgraded.

[0030] The response following the upgrade request includes:

[0031] Get version change information,

[0032] Based on the version change information, determine the target version.

[0033] Determine whether there is existing feature data to be upgraded in the target version. If not, use the feature extraction model corresponding to the target version to extract features from the source data of the existing feature data, persist the extracted feature data to the storage medium, obtain the upgraded existing feature data, and record the feature data that failed to be extracted.

[0034] Preferably, the data to be upgraded is incremental feature data to be upgraded.

[0035] The response following the upgrade request includes:

[0036] Obtain the version information of the incremental feature data to be upgraded, where the version information is identified by the version attribute variable;

[0037] Based on the version information, identify whether it is in the second or third state;

[0038] If it is the second state, then the feature extraction models corresponding to each candidate version in the target version are used to extract features from the source data of the incremental feature data, and the extracted feature data is persisted to the storage medium to obtain the incremental feature data of each candidate version.

[0039] If it is the third state, then the feature extraction model corresponding to the preferred version in the target version is used to extract features from the source data of the incremental feature data, and the extracted feature data is persisted to the storage medium to obtain the incremental feature data of the target version.

[0040] Record the feature data that failed to be extracted;

[0041] After the upgrade is complete, retain the incremental feature data of the target version and delete the incremental feature data of the alternative versions.

[0042] Preferably, in response to a retrieval request from the client,

[0043] Based on the data requested in the search request, obtain the version information and upgrade status of the database to be searched corresponding to the requested data.

[0044] Using the feature extraction model corresponding to the version information of the database to be retrieved, features are extracted from the requested data to obtain the first feature data.

[0045] The similarity between the first feature data and each feature data in the database to be retrieved is calculated to obtain the first similarity between the first feature data and each feature data in the database to be retrieved.

[0046] If the database to be retrieved contains feature data indicating an upgrade failure, then the feature extraction model corresponding to the existing version is used to extract features from the source data of the upgrade failure feature data to obtain the second feature data.

[0047] The similarity between the first feature data and each second feature data is calculated to obtain the second similarity between the first feature data and each feature data that failed to upgrade in the database to be retrieved.

[0048] Sort the first and second similarity scores according to their magnitude.

[0049] The feature data in the database with the highest similarity is determined as the search result.

[0050] Preferably, the business request is an index request, which includes at least one of a query operation, an add operation, and a delete operation.

[0051] The response to a business request from the client involves performing business processing based on the current version of the database corresponding to the data requested in the business request, including:

[0052] In response to an indexing request from the client,

[0053] Based on the data requested in the index request, obtain the version information and upgrade status of the database to be operated on corresponding to the requested data.

[0054] Process according to the version information and upgrade status, and according to the operation type.

[0055] Preferably, the step of processing according to the operation type based on version information and upgrade status includes:

[0056] If the operation type is a query operation, the query will be performed based on the current version of the requested data.

[0057] If the operation type is a delete operation and / or an add operation, then

[0058] In the first state, no data deletion operation is performed.

[0059] In the second state, delete the data corresponding to the requested data from all alternative versions corresponding to the requested data, and add the requested data to all alternative versions corresponding to the data.

[0060] In the third state, delete the data corresponding to the requested data in the existing version corresponding to the requested data, as well as the data corresponding to the requested data in the preferred version corresponding to the requested data that is inconsistent with the current version, and add the requested data to the preferred version corresponding to the data.

[0061] Preferably, the requested data is feature data.

[0062] The step of adding the requested data to all alternative versions corresponding to the data includes:

[0063] For each alternative version

[0064] Using the feature extraction model corresponding to the alternative version, perform feature extraction on the source data of the requested data, and add the extracted feature data to the alternative version;

[0065] The step of adding the requested data to the preferred version of the data includes:

[0066] Using the feature extraction model corresponding to the preferred version, feature extraction is performed on the source data of the requested data, and the extracted feature data is added to the preferred version.

[0067] The present invention provides a database upgrade apparatus, which includes: a memory, a processor, and a storage medium for storing the database. The memory stores a computer program, and the processor is configured to execute the computer program to implement the steps of any of the database upgrade methods.

[0068] This invention provides a database-based system, which includes:

[0069] The aforementioned database upgrade device, and

[0070] Control module for database operation and maintenance

[0071] Business modules used for business interactions with external clients.

[0072] An index module is used to index data in a database and process business operations from business modules based on the indexes.

[0073] The database upgrade method provided in this application avoids the need to interrupt business services due to version inconsistencies during the upgrade process by maintaining and managing the version in the intermediate state during the upgrade process. It achieves smooth database upgrades, and realizes uninterrupted and seamless upgrades. The use of multiple alternative versions and preferred versions during the upgrade process helps to ensure the accuracy of business operations and facilitates canary upgrades (partial upgrades). Attached Figure Description

[0074] Figure 1 A schematic flowchart of a database upgrade method according to an embodiment of this application.

[0075] Figure 2 This is a schematic diagram illustrating the changes in the feature data version state corresponding to changes in three version attribute variables.

[0076] Figure 3 This is a schematic diagram of a database and a database-based business application system as described in this application.

[0077] Figure 4 This is a schematic flowchart illustrating the upgrade process for an embodiment of this application.

[0078] Figure 5 This is a schematic diagram illustrating an upgrade method for upgrading existing feature data.

[0079] Figure 6 This is a schematic diagram of an upgrade method for upgrading incremental feature data.

[0080] Figure 7 This is a schematic diagram illustrating the processing flow of an index request.

[0081] Figure 8 This is a flowchart illustrating a process for retrieving data using a database.

[0082] Figure 9 This is a schematic diagram of a database upgrade device according to an embodiment of this application. Detailed Implementation

[0083] To make the objectives, technical means, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings.

[0084] Before providing a further detailed description of the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention will be explained, and the nouns and terms involved in the embodiments of the present invention shall be interpreted as follows.

[0085] 1) A database, in short, can be viewed as an electronic storage system where users can perform operations such as adding, querying, updating, deleting, and modifying data in the files. A "database" is a collection of data stored together in a certain way, which can be shared by multiple users, has the lowest possible redundancy, and is independent of applications.

[0086] 2) Uninterrupted service means that external search, CRUD (Create, Read, Update, Delete) services remain uninterrupted. During database updates, users will not experience any noticeable service unavailability.

[0087] 3) A data record is a set of complete related information corresponding to a row of information in a data source.

[0088] 4) Rollback, also known as rollback, refers to the deletion of updates performed by transactions that were partially completed. To ensure the integrity of the database after an application, database, or system error, rollback is used to restore the program or data to its previous correct state.

[0089] 5) Persistence is a mechanism for converting program data between a persistent state and a transient state. In simple terms, it is the persistence of transient data (e.g., data in memory, which cannot be permanently saved) into persistent data (e.g., data persisted to a database, which can be stored for a long time).

[0090] 6) In response to, used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which it depends are met, one or more operations performed may be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0091] See Figure 1 As shown, Figure 1 This is a schematic flowchart of a database upgrade method according to an embodiment of this application. The method includes:

[0092] Step 101: In response to the upgrade request, the system enters an intermediate state during the upgrade process, converting the data version to be upgraded from the first state in the database to the second state.

[0093] In the second state, the data version to be upgraded includes: the current version updated using a target version that includes at least one alternative version.

[0094] It should be understood that upgrade requests can come from external clients independent of the database, or from internal applications or processes such as the database's own operation and maintenance management modules or database-based business modules.

[0095] The data to be upgraded can be in the form of a database, such as a database formed by a collection of data with the same business domain meaning, such as a biometric database or a license plate feature database. It can also be in the form of a data record, or a set of data records specified by the user. In this case, the smallest unit can also be understood as a database.

[0096] Step 102: Select a preferred version from the candidate versions, and convert the data version to be upgraded in the second state to the data version to be upgraded in the third state.

[0097] The data version to be upgraded in the third state includes: the current version updated with the preferred version as the target version.

[0098] In this step, if there is only one alternative version, then that alternative version is the preferred version;

[0099] If there are multiple alternative versions, the alternative version with the highest affinity with the business is selected as the preferred version based on the affinity between the alternative versions and the business. Affinity is used to characterize the degree of matching between the alternative version and the business. This degree of matching can be understood as the degree of relevance, or as a quantitative assessment result of the ability to support the business.

[0100] Step 103: After the conversion from the data version to be upgraded in the second state to the data version to be upgraded in the third state is completed, that is, after step 102 is completed, it means that the intermediate state in the upgrade process is about to end. The data version to be upgraded in the third state is converted to the data version of the upgraded data in the first state.

[0101] In the first state, the upgraded data version includes the current version updated with the preferred version as the target version.

[0102] Of the states mentioned above, the first state is the normal state before or after the upgrade, while the second and third states are intermediate states during the upgrade process.

[0103] Because multiple versions of data exist simultaneously during the upgrade process, the data records are version compatible, which is beneficial for data consistency before and after the upgrade. Furthermore, during the upgrade of the data to be upgraded, in response to business requests from clients, business processing is performed based on the current database version corresponding to the requested data. This ensures both the accuracy of business operations during the instantaneous state of the upgrade process and enables uninterrupted data upgrades. For database users, the upgrade operation is completely imperceptible, thus achieving a seamless data upgrade.

[0104] To enable rollback, the data version to be upgraded in the first state also includes: the existing version that was consistent with the current version before responding to the upgrade request.

[0105] The data versions to be upgraded in the second state also include: existing versions that are consistent with the current version.

[0106] The data versions to be upgraded in the third state also include: existing versions that are inconsistent with the current version.

[0107] The method further includes:

[0108] In response to the version rollback operation, if the data version to be upgraded is in the third state, the data version to be upgraded will be converted to the data version to be upgraded in the second state; if the data version to be upgraded is in the second state, the data version to be upgraded will be converted to the data version to be upgraded in the first state.

[0109] It should be understood that version rollback operations can originate from external clients independent of the database, or from internal applications or processes such as the database's own operation and maintenance management modules or database-based business modules.

[0110] To facilitate understanding of this application, the following explanation uses the smooth upgrade of feature data as an example. It should be understood that this application is not limited to feature data, but can also include other broad types of data.

[0111] In this embodiment, three version attribute variables are used to identify three different states of the feature data version during the upgrade process. The three version attribute variables are represented as follows:

[0112] The first version attribute variable, denoted as forward, represents the version that has been used before, that is, the existing version, corresponding to the first version value, such as v1;

[0113] The second version attribute variable, denoted as current, represents the version currently in use, which corresponds to the second version value, such as v2.

[0114] The third version attribute variable, denoted as target, represents the version to be upgraded to, which is the target version. It corresponds to one or a group of third version values. For example, a group of third version values ​​is [v3.1, v3.2, v3.3]. Thus, the target version can include multiple alternative versions.

[0115] See Figure 2 As shown, Figure 2 This is a schematic diagram illustrating the changes in the feature data version state corresponding to changes in three version attribute variables.

[0116] Before the upgrade is performed, or after the upgrade is completed, the current state is the first state, which can be understood as the normal state or steady state.

[0117] When an upgrade operation is performed, the state transitions from the first state to the second state. This second state can be understood as the first intermediate state or the first transient state in the upgrade process. In response to the current upgrade operation, the version of the feature data to be upgraded transitions from the first state to the second state. In this state: forward is assigned a value equal to current, meaning that the existing version is consistent with the current version; then, current is assigned a value equal to each of the candidate versions in target, meaning that the current version is all the candidate versions in the target version, and there can be one or more candidate versions.

[0118] If a version switch operation is performed, the current upgrade state transitions from the second state to the third state. This third state can be understood as the second intermediate state or the second transient state in the upgrade process. In response to the current version switch operation, the version of the feature data to be upgraded transitions from the second state to the third state. In this state: forward is assigned a value that is not equal to current, that is, the existing version is not the current version and is inconsistent with the current version; current is assigned a value that is the preferred version in target, that is, the current version is a preferred version among the target versions.

[0119] After the upgrade of the feature data version to be upgraded is completed, the current upgrade state changes from the third state to the first state, and the feature data version to be upgraded changes from the third state to the first state. In this state: forward is assigned the value equal to current, that is, the existing version is updated with the current version. Since the current version is the current version in the third state, this version is the preferred version among the target versions. In this way, the feature data version to be upgraded is upgraded to this preferred version, thus completing the upgrade; target is assigned the value to current, that is, the current version is updated with the preferred version.

[0120] If a version rollback operation is performed while the current upgrade is in the third state, the feature data version to be upgraded will be converted from the third state to the second state in response to the current version rollback operation. At this time, the feature data version to be upgraded meets the version characteristics of the second state, that is, the existing version is consistent with the current version, and the current version is one of the candidate versions of the target version.

[0121] If a version rollback operation is performed while the current upgrade is in the second state, the feature data version to be upgraded will be converted from the second state to the first state in response to the current version rollback operation. At this time, the feature data version to be upgraded meets the version characteristics of the first state, that is, the existing version is consistent with the current version, and the current version is consistent with the target version.

[0122] By switching the values ​​of the three version attribute variables, the data version upgrade and rollback operations can be completed. In this embodiment, given the large amount of feature data, for example, a single facial image frame may correspond to a large number of feature data records. These feature data records form the feature data set of the facial image frame, which is equivalent to a small feature library. Therefore, the version attribute variables are bound to the database used to store the feature data set to be upgraded, rather than to a specific feature data record. The upgrade or rollback switching operation is unrelated to the number of feature data records, and the atomicity of the switching is better.

[0123] When the target version contains only one alternative version, that alternative version is the preferred version. When the target version includes multiple alternative versions, an optimal version can be selected from the multiple alternative versions according to a selection strategy. In this embodiment, the optimal version can be selected as follows: calculate the affinity between each alternative version in the target version and the business, and select the version with the highest affinity based on the affinity.

[0124] As an example, when the current upgrade is in the second state and a business request is triggered, the current version is used to respond to the business request, while all alternative versions of the target version that are inconsistent with the current version are also used to respond to the business request for processing. This processing can be offline to avoid delays to the current business.

[0125] For example, if the current business request is facial recognition, then based on each candidate version in the target version, the image to be recognized is used to extract features using the feature extraction model corresponding to that candidate version. The extracted features are then compared with the features in the candidate version to calculate similarity. Similarity values ​​greater than a set similarity threshold are selected. For each selected similarity, the product of the similarity and the corresponding evaluation value is calculated. The evaluation value is determined based on the difference between the set threshold and the similarity. The threshold is related to the feature extraction model used for feature extraction. All product results are summed to obtain the affinity of the candidate version.

[0126] Affinity calculation formula:

[0127] in,

[0128] k: Similarity between two features, with a value in the range [0,1].

[0129] Tr: Similarity threshold, which is set to 0.7 in this embodiment.

[0130] s k The evaluation value corresponding to similarity k is quantified using a percentage system in this embodiment, and is expressed mathematically as follows:

[0131] s k =(rk)100

[0132] When the similarity is below the set threshold r, the evaluation value is negative; otherwise, it is positive.

[0133] The threshold r is related to the feature model used to extract features. In this embodiment, the value is r = 0.8, and the value of k is 0.05.

[0134] See Figure 3 As shown, Figure 3 This is a schematic diagram of a database and a database-based business application system according to an embodiment of this application. The system includes: a control module for database operation and maintenance, an upgrade module for database upgrades, a storage medium for storing database data, a business module for business interaction with external clients, and an index module for indexing data in the database and responding to business operations from the business module based on the index.

[0135] As an example:

[0136] The control module receives database maintenance requests from clients, such as version queries or version change requests. In response to these requests, it controls the upgrade module to perform database upgrades, controls the index module to manage indexes (e.g., creating or deleting index directories), persists feature data version change information to storage media, and interacts with business modules to share feature data version change information, such as broadcasting version change events to business modules.

[0137] Under the control of the control module, the upgrade module upgrades the database, including upgrading existing data and incremental data, and interacts with the business module to exchange incremental data information, such as receiving incremental data change notifications from the business module.

[0138] The business module receives business requests from clients, such as retrieval requests. In response to these requests, it instructs the indexing module to perform indexing and retrieval operations, performs read and write operations with the storage medium, and interacts with the upgrade module to exchange incremental data change information.

[0139] See Figure 4 As shown, Figure 4 This is a schematic flowchart illustrating an upgrade process for an embodiment of this application. The upgrade method includes:

[0140] Step 401: The control module receives a version information query request from the client. The query request includes: querying feature data version information and / or available version information.

[0141] Step 402: In response to the query request, the control module queries the feature data version information and / or all available version information in the storage medium based on the request information in the query request.

[0142] Step 403: The control module receives a version change request for the feature data to be upgraded from the client. This request includes: changing version information, for example, changing the version value of the third-version attribute variable `target` to trigger an upgrade. The version value can be one or more values.

[0143] Step 404: In response to the change request, the control module modifies the version value of the third version attribute variable `target` based on the version change information in the change request.

[0144] Step 405: The control module sends a request to the index module to create an index directory.

[0145] Step 406: In response to the request to create an index directory, the index module creates an index directory for each target version corresponding to the third version attribute variable target. This index directory is used to index the feature data of the corresponding target version.

[0146] After the index module completes the creation of the index directory, it returns the created index directory information to the control module, which then persists the index directory information and target version information to the storage medium and broadcasts the version change event of the feature data to the business module. After the business module listens for the version change event, it queries the changed version information and index directory information from the storage medium and loads them into memory for business processing.

[0147] Step 407: The control module sends an upgrade request to the upgrade module to initiate the upgrade operation of the feature data to be upgraded.

[0148] Step 408: The upgrade module responds to the upgrade request and enters the upgrade process;

[0149] For existing feature data to be upgraded, the upgrade module retrieves the current version information of the feature data from the storage medium. Then, based on the current version information, it batch-retrieves the corresponding feature data to be upgraded, as well as the source data corresponding to the feature data to be upgraded, such as the original image corresponding to the image feature data, from the storage medium.

[0150] Step 409: The upgrade module sends the source data and target version information to the feature extraction unit in the business module to obtain the feature data corresponding to each target version. For example, the original image and target version information are sent to the feature extraction unit.

[0151] Step 410: The feature extraction unit uses the feature extraction model corresponding to the target version to extract features from the source data and returns the feature extraction results to the upgrade module.

[0152] Step 411: Based on the feature extraction results, the upgrade module records the source data where feature extraction failed and the feature data of the target version where feature extraction succeeded.

[0153] In step 412, the upgrade module sends the target version information of successfully extracted features and the feature data of the target version to the index module.

[0154] For example, after the feature data of a frame of facial image of user 1 is upgraded to the target version, the user information, the target version information, and the feature data of the target version are sent to the index module.

[0155] Step 413: The indexing module indexes the target version information and the target version's feature data into the corresponding index directory. That is, it establishes a logical correspondence between the index directory and the target version information and the target version's feature data, so as to index and store the target version information and the target version's feature data.

[0156] In the above process:

[0157] When the client modifies the version value of the second version attribute variable `current` to one of the version values ​​of the third version attribute variable `target` through the control module, it transitions to the third state, which is called the upgrade state switch.

[0158] When the client modifies the version value of the first version attribute variable `forward` to the version value of the second version attribute variable `current` through the control module, it is in a normal state. At this point, the gray-scale upgrade and version selection have been completed, and all alternative versions in the target version have been deleted. This is called an upgrade complete.

[0159] For version rollback, the rollback process is the same as the upgrade process in normal state; rollback in upgrade state means modifying the version value of the third version attribute variable target and the version value of the second version attribute variable current to make them consistent with the version value of the first version attribute variable forward.

[0160] See Figure 5 As shown, Figure 5 This diagram illustrates an upgrade method for existing feature data. Since the existing feature data is not real-time data, the upgrade module performs the upgrade offline. The upgrade template executes the following steps:

[0161] Step 501: Query the version change information of the existing feature data to be upgraded from the storage medium to determine the target version.

[0162] In this step, when there are multiple version values ​​for the target version, the preferred version is selected based on affinity and determined as the target version.

[0163] Step 502: Query the data records of the existing feature data to be upgraded and the source data of the existing feature data to be upgraded in batches from the storage medium;

[0164] Step 503, for each data record of the existing feature data to be upgraded:

[0165] Retrieve the corresponding data record in the target version from the storage medium;

[0166] If the query is successful, it means that the corresponding data record exists in the target version, and it also means that the target version of that data record has been upgraded.

[0167] If the query fails, it means that the corresponding data record does not exist in the target version. In this case, the feature extraction model corresponding to the target version is used to extract features from the source data corresponding to the data record. For example, the neural network model corresponding to the target version can be used for feature extraction.

[0168] If feature extraction is successful, the extracted feature data will be persisted to the storage medium as a data record in the target version.

[0169] If feature extraction fails, the source data and / or data record will be marked, for example, recorded in the feature extraction failure record table, so that the features extracted by the feature extraction model corresponding to the existing version and the feature extraction model corresponding to the target version can be processed for compatibility based on the source data of the data record. When the business module performs the search, the search comparison is performed based on the mark. That is to say, even if there is no corresponding feature data, the search comparison will still be performed when searching.

[0170] Repeat step 503 until every data record of the existing feature data to be upgraded has been upgraded to the target version. This completes the upgrade of the existing feature data.

[0171] Taking the upgrade of a user's facial image frame's base feature data as an example, the user's base database includes several data records used to characterize the user's facial features. The current version of the base database is version 1, and the target version is version 2. The upgrade module queries the storage medium for the version change information of the base database, queries the storage medium for the several data records included in version 1 base database, and the source image corresponding to the base database; it queries the storage medium for whether the data records in version 2 base database exist in version 1 base database. If they exist, the base database has been upgraded; otherwise, the feature extraction model corresponding to version 2 is used to extract features from the source image and perform feature attribute analysis, such as gender and age group. If feature extraction is successful, the extracted features are persisted to the storage medium, and the feature attribute information in version 1 is updated; if feature extraction fails, the user's source image or the unextracted features are recorded.

[0172] Specifically, the feature attribute information in updated version 1 may include:

[0173] For each feature attribute, calculate the confidence level of that feature attribute in all target versions, and retain the highest confidence level;

[0174] The feature attribute corresponding to the highest confidence level is compared with the feature attribute corresponding to version 1. If they are the same, no update is made. If they are not the same, the feature attribute is updated when the highest confidence level reaches the set confidence threshold and is higher than the confidence level of the feature attribute corresponding to the existing version.

[0175] In the process of upgrading existing feature data, since the target version is already determined and there is no need to switch versions, a seamless upgrade can be achieved.

[0176] See Figure 6 As shown, Figure 6 This is a schematic diagram illustrating an upgrade method for incremental feature data. Since incremental data is typically used in real-time, during the upgrade process, the existing version, the current version, and the target version coexist. Therefore, the upgrade module may enter intermediate states during the upgrade process, i.e., there may be a second state and a third state. The upgrade template executes the following steps:

[0177] Step 601: The upgrade template receives the incremental data change event sent by the client through the business module. The upgrade module responds to the change event by querying the version attribute variable corresponding to the incremental data in the change event. The version attribute variable includes three variables: forward, current, and target, in order to obtain the version information corresponding to the incremental data and also obtain the source data corresponding to the incremental data.

[0178] For example, the client initiates a registration request to the business module. In response to the registration request, the business module queries the version information and the database to which the requested incremental feature data belongs, based on the client information, writes it to the database, and sends an incremental data change event to the upgrade module.

[0179] Step 602: Based on the version attribute variable, determine whether the database to which the incremental feature data belongs is currently in an upgrade state. Here, the upgrade state may be the second state or the third state. The difference between the two states lies in the feature extraction model used. When the upgrade state is the second state, the feature extraction model corresponding to each alternative version in the target version is used. When the upgrade state is the third state, the feature extraction model corresponding to the preferred version in the target version is used.

[0180] If so, proceed to step 603.

[0181] Otherwise, the upgrade process will end.

[0182] Step 603: Use the feature extraction model corresponding to the current version of the database to which the incremental feature data belongs to extract features from the source data.

[0183] In this step, if the current upgrade status is the second status, then since the current version variable in the database is assigned to multiple alternative versions in the target version, feature extraction is performed on the source data using each alternative version.

[0184] If the current upgrade status is in the third state, then since the current version variable in the database is assigned the preferred version from the target version, the preferred version is used to extract features from the source data.

[0185] Step 604: Determine whether feature extraction was successful.

[0186] If successful, the extracted feature data is persisted to the storage medium to obtain the feature data of the target version, and then step 605 is executed.

[0187] Otherwise, record the incremental feature data and / or source data, and then end.

[0188] Step 605: Update the feature attributes of the current version.

[0189] Taking the upgrade of a newly added user's facial image frame as an example, the upgrade module responds to the incremental data registration event by querying the version attribute variable of the database corresponding to the user from the storage medium. Based on the version attribute variable, the current upgrade status can be determined. If the current upgrade status is the second status, then based on the information of each candidate version, the feature extraction model corresponding to each candidate version is used to extract features from the facial image frame. If the current upgrade status is the third status, then based on the information of the preferred version, the feature extraction model corresponding to the preferred version is used to extract features from the facial image frame. The extracted features are persisted to the storage medium, and the feature attribute information in the current version is updated. If feature extraction fails, the user's image or the unextracted features are recorded.

[0190] In the process of upgrading incremental feature data, different target versions are used for different upgrade states. This allows for different target version strategies during the upgrade process while ensuring the accuracy of business operations during instantaneous state switching. The coexistence of existing and current versions during the upgrade process enables seamless upgrades without service interruption.

[0191] See Figure 7 As shown, Figure 7 This is a schematic diagram illustrating the processing flow of an index request. It includes the following steps:

[0192] Step 701: The client sends an index request to the business module to perform operations on the feature data. This index request includes three operations: query, add, and delete.

[0193] Step 702: The index module receives an index request from the business module. Based on the request information in the index request, the control module queries the version information and upgrade status corresponding to the requested data.

[0194] Step 703: The indexing module responds accordingly based on the different request types.

[0195] Query operation: Respond to the query request with the current version corresponding to the requested data.

[0196] Adding an operation generally involves adding feature data to the database. The processing differs depending on the upgrade status. In a normal state, the index module writes the added feature data into the current version of the database. In an upgrade state, the index module instructs the upgrade module to write the added feature data into the current version of the database and the index directory.

[0197] Deletion operation: Generally, the feature data is deleted from the database or the database is deleted; or, the feature data of the current version is deleted, and then the deletion operation of feature data of other versions in the upgrade state is handed over to the upgrade module for processing.

[0198] Step 704: The index module sends the added or deleted feature data information to the upgrade module;

[0199] Step 705: The upgrade module queries the storage medium for the corresponding version information through the control module based on the feature data information;

[0200] Step 706: The upgrade module performs corresponding processing based on the version status:

[0201] Normal state (first state): No action is taken;

[0202] Second state: When adding, the incremental feature data is handled in the same way as in the upgrade process; when deleting, the feature data corresponding to the requested feature data in all alternative versions is deleted.

[0203] Third state: When adding, the existing version is used to perform the same operation as the upgrade process on the feature data. When deleting, the feature data corresponding to the requested feature data in the existing version and the feature data in the target version that is not equal to the current version and corresponds to the requested feature data are deleted. Here, the upgrade module deletes the data in the storage medium and also needs to send a deletion operation to the index module to delete the index directory or the corresponding index data.

[0204] See Figure 8 As shown, Figure 8 A flowchart illustrating a database retrieval process, including:

[0205] Step 801: The client sends a search request to the business module. This search request includes: the object to be searched and the database to be searched.

[0206] Step 802: The index module receives a retrieval request from the business module. Based on the request information in the retrieval request, it queries the version information and upgrade status of the database to be retrieved, as well as whether there are any records of upgrade failures for the current version in the database to be retrieved.

[0207] For example, if the feature data of the current version of user 1's facial image is missing due to feature extraction failure, then the user or the facial image will be recorded.

[0208] Step 803: The index module loads the feature data corresponding to the database to be retrieved from the index directory into memory.

[0209] Step 804: The indexing module uses the source data of the requested object to be retrieved, as well as the current version and target version (which may include existing versions), to initiate feature extraction to the feature extraction module. For example, it uses the requested original image, current version information, and target version information (which may include existing version information) to initiate feature extraction to obtain the feature data of the object to be retrieved.

[0210] Step 805: The feature extraction module uses the feature extraction model corresponding to the version to extract features from the object to be retrieved, and obtains the first feature data.

[0211] Step 806: The indexing module uses each feature data in the database to be retrieved from the index directory to calculate the similarity with the first feature data, thereby obtaining the first similarity value for each feature data.

[0212] If the database to be retrieved contains feature data of upgrade failure, the feature extraction model corresponding to the existing version is used to extract features from the source data of the upgrade failure feature data to obtain the second feature data. Then, the first feature data and the second feature data are compared to obtain the second similarity value of the upgrade failure feature data, so as to achieve data compatibility compensation and improve the fault tolerance during the upgrade process.

[0213] Sort the first similarity value and the second similarity value from highest to lowest.

[0214] The database feature data corresponding to the maximum similarity value is used as the retrieval result and fed back to the indexing module.

[0215] Step 807: The index module returns the search results to the control module.

[0216] Step 808: The control module records the search results of the database to be searched, and calculates and updates the affinity of each version.

[0217] This embodiment combines feature library upgrades with feature data, and upgrades the data by using version-corresponding feature extraction models to extract features. For example, for image feature data upgrades, it avoids the problem of missing new version data of some original data records after the upgrade, due to the existence of various low-quality visual images in actual scenarios and the different adaptability of different models to image quality.

[0218] See Figure 9 As shown, Figure 9 This is a schematic diagram of a database upgrade device according to an embodiment of this application. The device includes: a memory, a processor, and a storage medium for storing a database. The memory stores a computer program, and the processor is configured to execute the computer program to implement the steps of any of the database upgrade methods described above.

[0219] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0220] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0221] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the database upgrade methods described above.

[0222] For the device / network-side equipment / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and relevant parts can be referred to in the description of the method embodiments.

[0223] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0224] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A database upgrade method, characterized in that, The method includes: In response to the upgrade request, the data version to be upgraded in the database in a first state is converted to a data version to be upgraded in a second state, wherein the data version to be upgraded in the second state includes: the current version updated using a target version that includes at least one alternative version. Select a preferred version from the candidate versions, and convert the data version to be upgraded in the second state to the data version to be upgraded in the third state. The data version to be upgraded in the third state includes: the current version updated with the preferred version as the target version. After the conversion from the data version to be upgraded in the second state to the data version to be upgraded in the third state is completed, the data version to be upgraded in the third state is converted into the data version of the upgraded data in the first state. The data version of the upgraded data in the first state includes: the current version updated with the preferred version as the target version; in, The first state represents the normal state before or after the upgrade; the second and third states represent intermediate states during the upgrade process. The first, second, and third states are identified by the version attribute variable. If the attribute variable of the first version equals the attribute variable of the second version, and the attribute variable of the second version equals the attribute variable of the third version, then it is the first state. If the first version attribute variable equals the second version attribute variable, and the second version attribute variable is one of the version attribute variables in the third version attribute variable, then it is the second state. If the attribute variable of the first version is not equal to the attribute variable of the second version, and the attribute variable of the second version is equal to the attribute variable of the third version, then it is in the third state. The first version attribute variable is used to represent the existing version, the second version attribute variable is used to represent the current version, and the third version attribute variable is used to represent the target version. The step of selecting a preferred version from the candidate versions includes: If there is only one alternative version, then that alternative version is the preferred version; If there are multiple alternative versions, the alternative version with the highest affinity with the business is selected as the preferred version. Affinity is used to characterize the degree of matching between the alternative version and the business.

2. The method as described in claim 1, characterized in that, The method further includes: During the upgrade of the data to be upgraded, in response to business requests from clients, business processing is performed based on the current version of the database corresponding to the data requested by the business request.

3. The method as described in claim 1, characterized in that, The data version to be upgraded in the first state also includes: the existing version that was consistent with the current version before responding to the upgrade request. The data version to be upgraded in the second state also includes: an existing version that is consistent with the current version. The data version to be upgraded in the third state also includes: an existing version that is inconsistent with the current version. The method further includes: In response to the version rollback operation, if the data version to be upgraded is in the third state, the data version to be upgraded will be converted to the data version to be upgraded in the second state; if the data version to be upgraded is in the second state, the data version to be upgraded will be converted to the data version to be upgraded in the first state.

4. The method as described in claim 2, characterized in that, The data to be upgraded is feature data to be upgraded, and the affinity is determined as follows: For each alternative version The similarity between the business feature data requested in the business request and the feature data in the alternative version is calculated, and those with similarity scores greater than a set similarity threshold are selected. For each selected similarity, the product of that similarity and its corresponding evaluation value is calculated. The evaluation value is determined based on the difference between a set threshold and the similarity, and the threshold is related to the feature extraction model used for feature extraction. Sum all the product results to obtain the affinity of the alternative version.

5. The method as described in claim 4, characterized in that, The data to be upgraded refers to existing feature data that needs to be upgraded. The response following the upgrade request includes: Get version change information, Based on the version change information, determine the target version. Determine whether there is existing feature data to be upgraded in the target version. If not, use the feature extraction model corresponding to the target version to extract features from the source data of the existing feature data, persist the extracted feature data to the storage medium, obtain the upgraded existing feature data, and record the feature data that failed to be extracted.

6. The method as described in claim 4, characterized in that, The data to be upgraded is the incremental feature data to be upgraded. The response following the upgrade request includes: Obtain the version information of the incremental feature data to be upgraded, where the version information is identified by the version attribute variable; Based on the version information, identify whether it is in the second or third state; If it is the second state, then the feature extraction models corresponding to each candidate version in the target version are used to extract features from the source data of the incremental feature data, and the extracted feature data is persisted to the storage medium to obtain the incremental feature data of each candidate version. If it is the third state, then the feature extraction model corresponding to the preferred version in the target version is used to extract features from the source data of the incremental feature data, and the extracted feature data is persisted to the storage medium to obtain the incremental feature data of the target version. Record the feature data that failed to be extracted; After the upgrade is complete, retain the incremental feature data of the target version and delete the incremental feature data of the alternative versions.

7. The method as described in claim 4, characterized in that, The service request is a retrieval request. The response to a business request from the client involves performing business processing based on the current version of the database corresponding to the data requested in the business request, including: In response to a retrieval request from the client, Based on the data requested in the search request, obtain the version information and upgrade status of the database to be searched corresponding to the requested data. Using the feature extraction model corresponding to the version information of the database to be retrieved, features are extracted from the requested data to obtain the first feature data. The similarity between the first feature data and each feature data in the database to be retrieved is calculated to obtain the first similarity between the first feature data and each feature data in the database to be retrieved. If the database to be retrieved contains feature data indicating an upgrade failure, then the feature extraction model corresponding to the existing version is used to extract features from the source data of the upgrade failure feature data to obtain the second feature data. The similarity between the first feature data and each second feature data is calculated to obtain the second similarity between the first feature data and each feature data that failed to upgrade in the database to be retrieved. Sort the first and second similarity scores according to their magnitude. The feature data in the database with the highest similarity is determined as the search result.

8. The method as described in claim 2, characterized in that, The business request is an index request, which includes at least one of a query operation, an add operation, and a delete operation. The response to a business request from the client involves performing business processing based on the current version of the database corresponding to the data requested in the business request, including: In response to an indexing request from the client, Based on the data requested in the index request, obtain the version information and upgrade status of the database to be operated on corresponding to the requested data. Process according to the version information and upgrade status, and according to the operation type.

9. The method as described in claim 8, characterized in that, The process based on version information and upgrade status, according to operation type, includes: If the operation type is a query operation, the query will be performed based on the current version of the requested data. If the operation type is a delete operation and / or an add operation, then In the first state, no data deletion operation is performed. In the second state, delete the data corresponding to the requested data from all alternative versions corresponding to the requested data, and add the requested data to all alternative versions corresponding to the data. In the third state, delete the data corresponding to the requested data in the existing version corresponding to the requested data, as well as the data corresponding to the requested data in the preferred version corresponding to the requested data that is inconsistent with the current version, and add the requested data to the preferred version corresponding to the data.

10. The method as described in claim 9, characterized in that, The requested data is feature data. The step of adding the requested data to all alternative versions corresponding to the data includes: For each alternative version Using the feature extraction model corresponding to the alternative version, perform feature extraction on the source data of the requested data, and add the extracted feature data to the alternative version; The step of adding the requested data to the preferred version of the data includes: Using the feature extraction model corresponding to the preferred version, feature extraction is performed on the source data of the requested data, and the extracted feature data is added to the preferred version.

11. A database upgrade device, characterized in that, The apparatus includes: a memory, a processor, and a storage medium for storing a database, the memory storing a computer program, and the processor being configured to execute the computer program to implement the steps of the database upgrade method as described in any one of claims 1 to 10.

12. A database-based system, characterized in that, The system includes: The database upgrade apparatus as described in claim 11, and Control module for database operation and maintenance Business modules used for business interactions with external clients. An index module is used to index data in a database and process business operations from business modules based on the indexes.

Images