Data retrieval method and device, equipment and storage medium

Abstract

The invention provides a data retrieval method and device, equipment and a storage medium, which are used for solving the problem of relatively low retrieval efficiency in a data retrieval process. The method at least comprises the following steps: when the retrieval service is started, acquiring pre-stored synchronous data from a key value memory area; wherein the synchronous data is synchronously pre-stored in the key value memory area from a retrieval database according to the storage time of the corresponding original data in the retrieval database; when a data retrieval instruction is received, based on the storage time and the instruction receiving time of the data retrieval instruction, obtaining newly added data which are not synchronized and pre-stored in the key value memory area from the retrieval database; and based on the data retrieval instruction, performing data retrieval in the synchronous data and the newly added data to obtain a retrieval result. By simplifying the data acquisition process, the retrieval efficiency of data retrieval can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a data retrieval method and device, equipment and a storage medium. BACKGROUND

[0002] With the continuous development of technology, more and more devices can perform data retrieval to obtain retrieval results, so that subsequent processing can be performed based on the retrieval results.

[0003] For example, in the field of biometric identification, after obtaining the biometric features of a target object, data retrieval can be performed based on a feature database to determine whether there is a reference feature in the feature database that matches the biometric features of the target object, so that the retrieval result can be obtained.

[0004] When the retrieval result indicates that there is a reference feature in the feature database that matches the biometric features of the target object, it means that the target object is an authorized object with specified permissions, so the relevant processing related to the specified permissions can be continued. When the retrieval result indicates that there is no reference feature in the feature database that matches the biometric features of the target object, it means that the target object is an abnormal object without specified permissions, so the interception processing can be performed, etc.

[0005] In related technologies, the method of data retrieval is usually to wait for receiving a data retrieval instruction when starting a retrieval service each time, to obtain full retrieval data from a retrieval database and store it in memory when receiving the data retrieval instruction, and to perform data retrieval based on the data retrieval instruction and the full retrieval data obtained from the memory to obtain a retrieval result.

[0006] However, since the amount of data of the full retrieval data stored in the retrieval database is increasingly large, the process of obtaining the full retrieval data from the retrieval database and storing it in the memory each time the retrieval service is started consumes a large amount of time cost, resulting in low retrieval efficiency of the data retrieval process. In the environment where the service requirements of the retrieval service are continuously improving, the device cannot provide fast retrieval service.

[0007] It can be seen that the retrieval efficiency of the data retrieval process in related technologies is low. SUMMARY

[0008] The embodiments of the present application provide a data retrieval method, device, equipment and storage medium to solve the problem of low retrieval efficiency of the data retrieval process.

[0009] In a first aspect, a data retrieval method is provided, applied to a service node configured with a retrieval service, comprising:

[0010] The pre-stored synchronization data is obtained from the key-value memory area when the retrieval service starts, wherein the synchronization data is pre-stored from the retrieval database to the key-value memory area according to storage time of corresponding original data in the retrieval database;

[0011] When the data retrieval instruction is received, the newly added data which is not pre-stored in the key-value memory area is obtained from the retrieval database based on the storage time and instruction receiving time of the data retrieval instruction;

[0012] Based on the data retrieval instruction, data retrieval is performed in the synchronization data and the newly added data, and a retrieval result is obtained.

[0013] In a second aspect, a data retrieval device is provided, which is applied to a service node configured with a retrieval service, and includes:

[0014] The obtaining module is configured to obtain pre-stored synchronization data from the key-value memory area when the retrieval service starts, wherein the synchronization data is pre-stored from the retrieval database to the key-value memory area according to storage time of corresponding original data in the retrieval database;

[0015] The processing module is configured to obtain newly added data which is not pre-stored in the key-value memory area from the retrieval database based on the storage time and instruction receiving time of the data retrieval instruction when the data retrieval instruction is received;

[0016] The processing module is further configured to perform data retrieval in the synchronization data and the newly added data based on the data retrieval instruction, and obtain a retrieval result.

[0017] Optionally, the processing module is specifically configured to:

[0018] In response to a service configuration operation for the service node, the retrieval service is configured for the service node;

[0019] When the retrieval service is initially started, the first data request is sent to the key-value memory area by using the created update thread, wherein the first data request indicates that pre-stored synchronization data is obtained from the key-value memory area;

[0020] The synchronization data returned by the key-value memory area is received by using the update thread.

[0021] Optionally, the processing module is further configured to:

[0022] after the data retrieval instruction is received, based on the storage time and an instruction receiving time of the data retrieval instruction, obtaining, from the retrieval database, newly added data that is not synchronized and pre-stored in the key-value memory area, and based on a preset data sharing strategy, using the update thread to establish, in the shared memory area, a mapping relationship between respective physical storage addresses of the synchronized data and the newly added data and virtual storage addresses stored in the shared memory area;

[0023] after detecting that the retrieval service is closed, storing the newly added data in the key-value memory area and taking the synchronized data and the newly added data as historical synchronized data;

[0024] after detecting that the retrieval service is restarted, using a re-created update thread to reload the pre-stored synchronized data in the key-value memory area based on the mapping relationship stored in the shared memory area, to perform data retrieval.

[0025] Optionally, the processing module is further configured to:

[0026] after the retrieval service is configured for the service node in response to the service configuration operation for the service node, and when the retrieval service is not detected to be initially started or is detected to be closed, using a created daemon process to synchronize, in a preset time period, storage data stored in the retrieval database to the key-value memory area.

[0027] Optionally, the key-value memory area comprises a plurality of feature partitions, and data stored in different feature partitions has different data features.

[0028] Therefore, the processing module is specifically configured to:

[0029] for each period, the following operations are performed: determining respective data features of each storage sub-data stored in the period in the retrieval database;

[0030] for each storage sub-data, the following operations are performed:

[0031] when it is determined that there is a target partition in the plurality of feature partitions that matches the data feature of the storage sub-data, synchronizing the storage sub-data to the target partition;

[0032] when it is determined that there is no target partition in the plurality of feature partitions that matches the data feature of the storage sub-data, newly creating the target partition in the key-value memory area and synchronizing the storage sub-data to the target partition.

[0033] Optionally, when the stored sub-data is palmprint data, between data features corresponding to each of two feature partitions in the plurality of feature partitions, there is at least one of the following feature differences: a number of principal lines contained in a principal line of the palmprint, a number of triangular regions formed by the principal line, a pattern of the principal line, and a size range to which a size of the palmprint belongs; wherein the pattern of the principal line includes at least one of a ring shape, a spiral shape, and an arc shape.

[0034] Optionally, the processing module is specifically configured to:

[0035] When the retrieval service is started, a data retrieval instruction is received; wherein the data retrieval instruction indicates that data with a similarity to the to-be-retrieved data satisfying a similarity condition is retrieved;

[0036] From the plurality of feature partitions, at least one to-be-retrieved partition matching a data feature of the to-be-retrieved data is determined;

[0037] From the at least one to-be-retrieved partition, pre-stored synchronization data is obtained.

[0038] Optionally, the processing module is specifically configured to:

[0039] When it is detected that the retrieval service is restarted after being closed, the created update thread is used to load pre-stored synchronization data in the key-value memory area based on a mapping relationship pre-stored in the shared memory area;

[0040] The mapping relationship is established in the shared memory area by the retrieval service at a historical time based on a preset data sharing strategy; the mapping relationship represents a correspondence relationship between a physical storage address of historical synchronization data and a virtual storage address stored in the shared memory area; the historical synchronization data includes synchronization data and new data obtained by the retrieval service after being started at the historical time.

[0041] Optionally, the processing module is specifically configured to:

[0042] The update thread is used to send a second data request to the retrieval database; wherein the second data request indicates that new data stored in the retrieval database in a time period between the storage time and the instruction receiving time is obtained from the retrieval database;

[0043] The update thread is used to receive the new data returned by the retrieval database.

[0044] Optionally, the service node includes a graphics processing unit, and a processing data amount of the graphics processing unit when processing in parallel is greater than a preset data amount.

[0045] The processing module is specifically configured to:

[0046] invoke a data interaction interface of the graphics processing unit to copy the synchronization data and the new data in a display memory of the graphics processing unit;

[0047] adopt the graphics processing unit to perform data retrieval in the synchronization data and the new data based on the data retrieval instruction, and obtain a retrieval result.

[0048] Optionally, the data retrieval instruction indicates to retrieve data with a similarity to the to-be-retrieved data satisfying a similarity condition.

[0049] The processing module is specifically configured to:

[0050] determine respective data features of each of synchronization sub-data contained in the synchronization data and respective data features of each of new sub-data contained in the new data;

[0051] based on the respective data features of each of the synchronization sub-data and the respective data features of each of the new sub-data, select, from the synchronization sub-data and the new sub-data, each target sub-data matching the to-be-retrieved data;

[0052] invoke a data interaction interface of the graphics processing unit to copy the synchronization data and the new data in a display memory of the graphics processing unit;

[0053] Optionally, the data retrieval instruction indicates to retrieve data with a similarity to the to-be-retrieved data satisfying a similarity condition.

[0054] The processing module is specifically configured to:

[0055] obtain to-be-retrieved data indicated by the data retrieval instruction;

[0056] determine similarities between each of synchronization sub-data contained in the synchronization data and each of new sub-data contained in the new data and the to-be-retrieved data, and obtain respective data similarities;

[0057] based on the obtained respective data similarities, select, from the synchronization sub-data and the new sub-data, a plurality of retrieval sub-data satisfying a similarity condition indicated by the data retrieval instruction as the retrieval result.

[0058] In a third aspect, a computer program product is provided, including a computer program which, when executed by a processor, implements the method according to the first aspect.

[0059] In a fourth aspect, a computer device is provided, including:

[0060] a memory configured to store program instructions;

[0061] a processor configured to invoke program instructions stored in the memory and perform the method according to the first aspect.

[0062] In a fifth aspect, a computer-readable storage medium is provided, which stores computer-executable instructions for causing a computer to perform the method according to the first aspect.

[0063] In the embodiments of the present application, the original data stored in the retrieval database is pre-synchronized from the retrieval database to the key-value memory area in the memory, so that when the retrieval service is started, the pre-stored synchronized data can be directly obtained from the key-value memory area, without the need to read the data from the retrieval database when receiving the data retrieval instruction, and then write the read data into the memory, which simplifies the data obtaining process and improves the data obtaining efficiency, thereby improving the retrieval efficiency of data retrieval.

[0064] Further, since a large amount of data is stored in the retrieval database, a large amount of data stored in the retrieval database is pre-synchronized to the key-value memory area, and only a small amount of newly added data not pre-stored in the key-value memory area is obtained from the retrieval database when receiving the data retrieval instruction, which reduces the amount of data to be obtained when receiving the data retrieval instruction, improves the data obtaining efficiency, and thus improves the retrieval efficiency of data retrieval.

[0065] Further, not only the data stored in the retrieval database is obtained through the key-value memory area in the memory, but also the data is obtained through the retrieval database itself, so that the data for data retrieval is the full amount of data stored in the retrieval database in real time, avoiding the situation of missing detection, thereby ensuring the accuracy of data retrieval. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1A A scenario diagram of the data retrieval method provided by the embodiments of the present application;

[0067] Figure 1B A principle diagram of the data retrieval method in the related art;

[0068] Figure 1C An application scenario of the data retrieval method provided by the embodiments of the present application;

[0069] Figure 2 A flowchart of the data retrieval method provided by the embodiments of the present application;

[0070] Figure 3A A principle diagram of the data retrieval method provided by the embodiments of the present application;

[0071] Figure 3B A schematic diagram of the principle of the data retrieval method provided in the embodiments of this application. Figure Two ;

[0072] Figure 4A A schematic diagram three illustrating the principle of the data retrieval method provided in this application embodiment;

[0073] Figure 4B A schematic diagram four illustrating the principle of the data retrieval method provided in this application embodiment;

[0074] Figure 4C A schematic diagram five illustrating the principle of the data retrieval method provided in this application embodiment;

[0075] Figure 5A A schematic diagram six illustrating the principle of a data retrieval method provided in an embodiment of this application;

[0076] Figure 5B A schematic diagram seven illustrating the principle of a data retrieval method provided in this application embodiment;

[0077] Figure 6A A schematic diagram of the principle of the data retrieval method provided in the embodiments of this application. Figure Eight ;

[0078] Figure 6B A schematic diagram of the principle of the data retrieval method provided in the embodiments of this application. Figure Nine ;

[0079] Figure 6C A schematic diagram ten illustrating the principle of a data retrieval method provided in this application embodiment;

[0080] Figure 7A 11. A schematic diagram illustrating the principle of the data retrieval method provided in this application embodiment;

[0081] Figure 7B A flowchart illustrating a data retrieval method provided in this application embodiment. Figure Two ;

[0082] Figure 7C A flowchart illustrating a data retrieval method provided in this application embodiment is shown in Figure 3.

[0083] Figure 8 A schematic diagram of the structure of a data retrieval device provided in the embodiments of this application;

[0084] Figure 9 A schematic diagram of the structure of the data retrieval device provided in the embodiments of this application. Figure Two . Detailed Implementation

[0085] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0086] Some terms in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0087] (1) Graphics Processing Unit (GPU):

[0088] GPU is optimized for performing complex mathematical and geometric calculations. Compared with a traditional CPU (Central Processing Unit), GPU contains more cores and can execute thousands of threads at the same time, which is very suitable for processing highly parallel tasks.

[0089] The internal structure of GPU usually includes a large number of Stream Processors, texture units, Raster Operation Pipelines (ROPs), etc., which are designed to achieve higher throughput when processing graphics and parallel computing tasks.

[0090] (2) Memory Mapping (mmap):

[0091] mmap is a POSIX standard system call in Unix and Unix-like systems, and its main function is to map a file or other object (such as a device) to the address space of a process. Through mmap, the operating system creates a direct mapping of file content to memory, so that the process can directly access file data as if it were accessing regular memory, without the need for traditional read / write system calls.

[0092] It should be noted that in the embodiments of the present application, when the above embodiments of the present application are applied to specific products or technologies, the user's permission or consent is required, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0093] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as processing circuitry or memory) or a combination thereof. Similarly, one processor (or multiple processors or memory) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the functions of the module or unit.

[0094] The application field of the data retrieval method provided in the embodiments of the present application is briefly introduced as follows.

[0095] With the continuous development of science and technology, more and more devices can perform data retrieval to obtain retrieval results, so that subsequent processing can be performed based on the retrieval results.

[0096] For example, in the field of biometric recognition, after obtaining the biometric feature of a target object, data retrieval can be performed based on a feature database to determine whether there is a reference feature in the feature database that matches the biometric feature of the target object, so that the retrieval result can be obtained.

[0097] When the retrieval result indicates that there is a reference feature in the feature database that matches the biometric feature of the target object, it means that the target object is an authorized object with specified permissions, and therefore the relevant processing related to the specified permissions can be continued. When the retrieval result indicates that there is no reference feature in the feature database that matches the biometric feature of the target object, it means that the target object is an abnormal object without the specified permissions, and therefore the interception processing can be performed.

[0098] For another example, in the field of target search, please refer to Figure 1A After the service node starts the retrieval service, the client sends a data retrieval instruction to the service node, which indicates that in the reference target database, N reference targets with the highest similarity to the to-be-retrieved feature are to be retrieved. The service node receives the data retrieval instruction sent by the client, sends a data acquisition instruction to the reference target database, and the data acquisition instruction is used to acquire the full amount of data stored in the reference target database. The reference target database receives the data acquisition instruction sent by the service node, and sends the stored full amount of data to the service node. The service node receives the full amount of data sent by the reference target database, and stores the received full amount of data in the memory. Based on the full amount of data stored in the memory, the service node uses the CPU to sequentially determine the similarity between each reference feature in the full amount of data and the to-be-retrieved feature indicated by the data retrieval instruction. Then, the N reference targets with the largest similarity are selected as the retrieval result according to the similarity size. The service node sends the retrieval result to the client, and the client receives the retrieval result sent by the service node and obtains the N reference targets, thereby completing the data retrieval.

[0099] In the related art, the method of data retrieval is generally to wait for receiving a data retrieval instruction each time the retrieval service is started, to acquire the full amount of retrieval data from the retrieval database and store it in the memory when the data retrieval instruction is received, and to perform data retrieval based on the data retrieval instruction and the full amount of retrieval data acquired in the memory to obtain the retrieval result.

[0100] For example, please refer to Figure 1BAfter starting the retrieval service each time, the service node waits to receive a data retrieval instruction. The client sends the data retrieval instruction to the service node, and the service node establishes a working thread to receive the data retrieval instruction sent by the client. The service node creates an update thread to send a data acquisition instruction to the reference feature database. The reference feature database receives the data acquisition instruction sent by the update thread, reads the full-amount data in the reference feature database, and sends the full-amount data to the update thread. The update thread receives the full-amount data sent by the reference feature database and sends the obtained full-amount data to the memory. The working thread sends the data retrieval instruction to the memory, and the service node uses the CPU to perform data retrieval on the stored full-amount data in the memory based on the data retrieval instruction, to obtain a retrieval result. The CPU returns the retrieval result to the working thread, the working thread receives the returned retrieval result, and sends the retrieval result to the client. The client receives the retrieval result sent by the working thread.

[0101] However, as the amount of data stored in the retrieval database is increasingly large, the process of obtaining the full-amount retrieval data from the retrieval database and storing it in the memory each time the retrieval service is started needs to consume a large amount of time cost. The data acquisition process of ten-thousand-level data amount often needs tens of minutes. If a service bottleneck occurs, the restart and expansion of the retrieval service cannot be completed in time, the retrieval service cannot be provided externally, the normal use of the retrieval service is affected, and the retrieval efficiency of the data retrieval process is low. Under the environment of continuously improving service requirements of the retrieval service, the device cannot provide fast retrieval service.

[0102] It can be seen that the retrieval efficiency of the data retrieval process under the related technology is low.

[0103] To solve the problem of low retrieval efficiency of the data retrieval process, a data retrieval method is provided. The method is applied to a service node configured with a retrieval service. In the method, when the retrieval service is started, pre-stored synchronization data is obtained from a key-value memory area. The synchronization data is pre-stored in the key-value memory area from the retrieval database according to the storage time of the corresponding original data in the retrieval database. When a data retrieval instruction is received, based on the storage time and the instruction receiving time of the data retrieval instruction, new data that is not pre-stored in the key-value memory area is obtained from the retrieval database. Based on the data retrieval instruction, data retrieval is performed in the synchronization data and the new data, and a retrieval result is obtained.

[0104] In the embodiment of the present application, the original data stored in the retrieval database is pre-synchronized from the retrieval database to the key-value memory area in the memory, so that when the retrieval service is started, the pre-stored synchronized data can be directly obtained from the key-value memory area, without the need to read the data from the retrieval database when receiving the data retrieval instruction, and then write the read data into the memory, which simplifies the data obtaining process and improves the data obtaining efficiency, thereby improving the retrieval efficiency of data retrieval.

[0105] Further, since a large amount of data is stored in the retrieval database, a large amount of data stored in the retrieval database is pre-synchronized to the key-value memory area, and only when the data retrieval instruction is received, the small amount of newly added data not pre-stored in the key-value memory area is obtained from the retrieval database, thereby reducing the amount of data to be obtained when receiving the data retrieval instruction, improving the data obtaining efficiency, and thereby improving the retrieval efficiency of data retrieval.

[0106] Further, not only the data stored in the retrieval database is obtained through the key-value memory area in the memory, but also the data is obtained through the retrieval database itself, so that the data for data retrieval is the full amount of data stored in the retrieval database in real time, avoiding the situation of missing detection, thereby ensuring the accuracy of data retrieval.

[0107] The application scenario of the data retrieval method provided by the present application will be described below.

[0108] Please refer to Figure 1C , an application scenario of the data retrieval method provided by the present application. The application scenario includes a server 101 and a client 102, and the server 101 and the client 102 can communicate with each other. The communication mode can be wired communication technology, such as connection of network cable or serial cable; or wireless communication technology, such as Bluetooth or wireless fidelity (WIFI) technology, without limitation.

[0109] The client 102 generally refers to a device that can send a data retrieval instruction or receive a retrieval result, such as a terminal device, a third-party application accessible by the terminal device, or a webpage accessible by the terminal device. The server 101 generally refers to a device that can execute a data retrieval method, such as a terminal device or a server.

[0110] The terminal device includes but is not limited to a mobile phone, a computer, a smart medical device, a smart home appliance, a vehicle-mounted terminal, an aircraft, etc. The server includes but is not limited to a cloud server, a local server, or an associated third-party server, etc. The client 101 and the server 102 can both use cloud computing to reduce the occupation of local computing resources; similarly, cloud storage can also be used to reduce the occupation of local storage resources.

[0111] As an embodiment, the server 101 and the client 102 can be the same device, or can be different devices, or can be different devices that share some modules, etc., and the specific implementation is not limited.

[0112] Figure 1C Taking the client 102 as a palmprint scanning device in a palmprint payment scenario as an example, the client 102 obtains palmprint data by scanning a palmprint, and the client 102 can send a data retrieval instruction carrying the palmprint data to the server 101. The server 101 receives the data retrieval instruction sent by the client 102, performs data retrieval based on the palmprint data in the data retrieval instruction, and obtains a retrieval result. The retrieval result represents whether there is a payment account matched with the palmprint data; if there is, the palmprint payment is performed using the payment account; if there is not, a payment failure is prompted.

[0113] The data retrieval method provided in the embodiments of the present application will be described in detail below based on Figure 1C , based on a service node configured with a retrieval service. Please refer to Figure 2 , for a flowchart of the data retrieval method provided in the embodiments of the present application.

[0114] S201, when the retrieval service is started, pre-stored synchronization data is obtained from a key-value memory area.

[0115] The retrieval service can be started at a specified time, or can be triggered to start, such as being triggered to start in response to a start operation for the retrieval service, etc., and the specific implementation is not limited, and the embodiments of the present application are introduced in the form of being triggered to start.

[0116] The synchronization data is pre-stored in the key-value memory area before the retrieval service is triggered to start. According to the storage time of the original data corresponding to the synchronization data in the retrieval database, the data before the retrieval service is triggered to start is pre-stored from the retrieval database to the key-value memory area. Thus, when the retrieval service is triggered to start, the service node can directly obtain the pre-stored synchronization data from the key-value memory area to perform data retrieval, without reading out the data stored in the retrieval database and writing into the memory, which simplifies the data obtaining process and improves the data obtaining efficiency, thereby improving the retrieval efficiency of data retrieval.

[0117] The data stored in the database can be biometric data such as fingerprints, palm prints, facial images, or voices, or other data that can be used for retrieval, such as articles, videos, comments, map routes, etc., without limitation.

[0118] The retrieval service is triggered to start, which can be the first time the retrieval service is triggered to start on the service node, or the retrieval service is triggered to start again after being triggered to stop on the node, etc. The data retrieval method provided by the embodiments of the application is introduced below in these two cases.

[0119] The retrieval service is triggered to start for the first time:

[0120] For example, in response to a service configuration operation for the service node, the retrieval service is configured for the service node. After the retrieval service is configured, it is detected that the retrieval service is triggered to start, so it can be determined that the retrieval service is triggered to start for the first time.

[0121] For another example, the service node can also count the number of times the retrieval service is triggered to start, so when the number of starts is determined to be 0, it can be determined that the retrieval service is triggered to start for the first time.

[0122] When the retrieval service is triggered to start for the first time, the service node can use the created update thread to send a first data request to the key-value memory area. The first data request indicates that the pre-stored synchronization data is obtained from the key-value memory area. The update thread receives the synchronization data returned by the key-value memory area.

[0123] The key-value memory area is an area in memory that stores data in a simple data format and has high throughput performance. The simple data format is a data format that occupies less storage resource, such as the data format of a key-value pair. Therefore, when the retrieval service is started for the first time, the pre-stored synchronization data can be obtained from the key-value memory area. With the high throughput performance of the key-value memory area, the synchronization data can be quickly pulled, i.e. the original data stored in the retrieval database before the retrieval service is started for the first time can be quickly obtained. Compared with the process of obtaining data from the retrieval database, it is more efficient, thereby improving the retrieval efficiency of data retrieval.

[0124] For example, please refer to Figure 3A After the service node is configured with the retrieval service, the stored data in the retrieval database can be pre-synchronized to the key-value memory area. After the retrieval service is configured, it can be triggered to start as the first start of the retrieval service. After the retrieval service is started for the first time, the service node creates an update thread. The update thread is used to obtain pre-stored synchronization data from the key-value memory area, and the key-value memory area returns the synchronization data to the update thread.

[0125] As an embodiment, after the retrieval service is configured for the service node in response to the service configuration operation for the service node, if it is detected that the retrieval service is triggered to start for the first time, it indicates that the retrieval service has not been started; or if it is detected that the retrieval service is triggered to stop, it indicates that the retrieval service has been started and is currently stopped. Then, the service node can use the created daemon process to synchronize the storage data stored in the retrieval database to the key-value memory area in each period with a preset time length as a cycle.

[0126] By performing the offline synchronization when the retrieval service is not started, the process of pre-storing the synchronization data does not affect any performance of the retrieval service, so that the service node can provide all computing resources to the retrieval service when the retrieval service is started, and the retrieval efficiency of data retrieval is improved to a certain extent.

[0127] For example, refer to Figure 3B For example, the storage data stored in the retrieval database with the largest time interval between the storage time and the current time is the storage data stored at 17:52 with a preset time length of 5 minutes. After the retrieval service is configured for the service node, before it is detected that the retrieval service is triggered to start for the first time, the service node can start a daemon process in the background on the script machine. The daemon process starts from 17:52, and 5 minutes is a cycle. First, the storage data with the storage time between 17:52 and 17:57 is read from the retrieval database, and then the read storage data between 17:52 and 17:57 is written into the key-value memory area. Then, the storage data between 17:57 and 18:02 is read, and then the read storage data between 17:57 and 18:02 is written into the key-value memory area. In this way, the process is continued until the interval between the storage time of the latest read storage data and the current time is less than 5 minutes.

[0128] After the retrieval service is triggered to start for the first time, the daemon process can pause the reading and writing process, and after the retrieval service is triggered to stop, the reading and writing process is restarted to continue the above reading and writing process based on the storage time of the corresponding original data in the retrieval database and the latest synchronization data stored in the key-value memory area. In this way, the storage data stored in the retrieval database is continuously synchronized to the key-value memory area.

[0129] The daemon process started in the script machine can be shared by other service nodes, or multiple daemon processes can be created in the script machine for multiple service nodes to call.

[0130] As an embodiment, in the key-value memory area, a plurality of feature partitions can be set, and the data stored in different feature partitions has different data features. When synchronizing the stored data stored in the retrieval database to the pre-stored key-value memory area offline, the data can be synchronized to the corresponding feature partition according to the data feature of the currently synchronized data, and the large library of the retrieval database is converted into a plurality of sub-libraries in the key-value memory area for storage. Then, when data retrieval is performed, the retrieval can be performed on a certain sub-library or certain sub-libraries, and the retrieval efficiency of data retrieval is further improved.

[0131] The data synchronization process in a cycle is introduced below, and the data synchronization processes in other cycles are similar and will not be repeated here.

[0132] Since the stored data is the general term of the data stored in the retrieval database, a large amount of data is stored in the retrieval database, and therefore, the stored data includes a plurality of storage sub-data.

[0133] Then, in a cycle, each storage sub-data stored in the retrieval database and stored in this cycle can be obtained, and the data feature of each storage sub-data is obtained.

[0134] The data feature includes, for example, the value of a specified field in the data, the identification result of a target in an image when the data includes the image, the length of a video when the data includes the video, the data source, the data type, and the like, and the specific setting can be made according to the use scenario, which is not limited here.

[0135] The synchronization process of one storage sub-data is introduced below, and the synchronization processes of other storage sub-data are similar and will not be repeated here.

[0136] After obtaining the data feature of the storage sub-data, it can be determined whether there is a target partition in each feature partition included in the key-value memory area, which matches the data feature of the storage sub-data. The matching can be completely the same, or the data feature of the target partition can include the data feature of the storage sub-data, or the similarity between the data feature of the target partition and the data feature of the storage sub-data is greater than a preset value, and the like, which is not limited.

[0137] When there is a target partition that matches the data feature of the storage sub-data, the storage sub-data is synchronized to the target partition. When it is determined that there is no target partition in the plurality of feature partitions that matches the data feature of the storage sub-data, the storage sub-data can be stored in a featureless partition in the key-value memory area, and the data stored in the featureless partition has no limitation on the data feature; or a target partition can be newly created in the key-value memory area, and the storage sub-data is synchronized to the target partition, and the like, which is not limited.

[0138] Please refer to Figure 4AThe key-value memory area includes a plurality of feature partitions, respectively, a first feature partition, a second feature partition, a third feature partition, and an Nth feature partition. The data stored in the first feature partition has a data feature of being less than 14 years old; the data stored in the second feature partition has a data feature of being taller than 160 cm; the data stored in the third feature partition has a data feature of being good at playing badminton; and the data stored in the Nth feature partition has a data feature of being located in a suburb. After obtaining the data feature of the stored sub-data, if the data feature is determined to be less than 14 years old, the first feature partition can be taken as a target partition, and the stored sub-data can be stored in the first feature partition. If the data feature of the stored sub-data satisfies a plurality of feature partitions, the stored sub-data can be stored in the feature partition with a smaller data amount.

[0139] As an embodiment, when the stored sub-data is palmprint data, in the plurality of feature partitions, at least one feature difference exists between the respective data features of each two feature partitions, including the number of main lines in a palmprint, the number of triangular regions formed by the main lines, the pattern of the main lines, and the size range of the palmprint size. The pattern of the main lines includes at least one of a ring shape, a spiral shape, and an arc shape, and can also include other shapes, which are not limited in particular. Thus, the large database storing a large amount of data is converted into a plurality of sub-databases storing a small amount of data according to the data features in the palmprint recognition scenario, which not only reduces the amount of data when traversing the data, but also allows multiple sub-databases to be traversed in parallel, thereby improving the retrieval efficiency.

[0140] For example, the data feature corresponding to the first feature partition is that the palmprint contains at most 3 main lines, the main lines form at least one triangular region, the pattern of the main lines is spiral, and the palmprint size is within 8 cm x 8 cm; the data feature corresponding to the second feature partition is that the palmprint contains at least 4 main lines, the main lines form at least one triangular region, the pattern of the main lines is non-spiral, and the palmprint size is within 8 cm x 8 cm; the data feature corresponding to the third feature partition is that the palmprint contains at most 3 main lines, the main lines do not form a triangular region, the pattern of the main lines is non-spiral, and the palmprint size is within 8 cm x 8 cm; and the data feature corresponding to the fourth feature partition is that the palmprint contains at least 4 main lines, the main lines do not form a triangular region, the pattern of the main lines is spiral, and the palmprint size is within 8 cm x 8 cm but within 12 cm x 12 cm, and the like, which are not listed one by one.

[0141] Then, for a stored sub-data, if the data feature of the stored sub-data includes that the palmprint contains 4 main lines, the palmprint does not contain a triangular region, the pattern of the main lines is spiral, and the palmprint size is 8 cm x 10 cm, the fourth feature partition can be determined as a target partition, and the stored sub-data can be synchronized to the target partition.

[0142] As an embodiment, if the key-value memory area is divided into multiple feature partitions, when obtaining the synchronization data from the key-value memory area, the synchronization data can be obtained only in the feature partitions that match the data features of the to-be-retrieved data indicated by the received data retrieval instruction, without the need to obtain the full amount of data stored in the key-value memory area, further reducing the amount of data to be processed and improving the data retrieval efficiency.

[0143] When the retrieval service is triggered to start, the service node can start receiving the data retrieval instruction. The data retrieval instruction can indicate: retrieving data similar to the to-be-retrieved data that meets a similarity condition; the data retrieval instruction can also indicate: retrieving data with a specified feature, such as time, data source, or data content, without limitation.

[0144] Before obtaining the synchronization data, if the data retrieval instruction is received, the service node can determine at least one to-be-retrieved partition that matches the data features of the to-be-retrieved data from the multiple feature partitions. The pre-stored synchronization data is obtained from the at least one to-be-retrieved partition.

[0145] For example, in the palm print payment field, please refer to Figure 4B , the key-value memory area is divided into N feature partitions, including a first feature partition, a second feature partition,..., and an Nth feature partition. The data stored in the first feature partition has a data feature of payment data of accounts with an age of less than 14; the data stored in the second feature partition has a data feature of payment data of accounts with a gender of male; the data stored in the third feature partition has a data feature of payment data of accounts with a gender of female; and the data stored in the Nth feature partition has a data feature of payment data of a primary account in a plurality of associated accounts.

[0146] When the data retrieval instruction is obtained, the data retrieval instruction can indicate: retrieving payment data similar to the palm print picture that meets a similarity condition. Then, at least one to-be-retrieved partition that matches the data features of the palm print picture can be determined from the N feature partitions. For example, the data features of the palm print picture are male palm print and age greater than 14. Then, the second feature partition and the Nth feature partition can be selected as two to-be-retrieved partitions from the N feature partitions. The pre-stored synchronization data is obtained from the two to-be-retrieved partitions.

[0147] As an embodiment, the service node can establish a mapping relationship between the physical storage address of the synchronization data in the shared memory area in the memory and the virtual storage address stored in the shared memory area based on the data sharing strategy. Then, when the retrieval service is detected to be triggered to restart subsequently, the service node can use the newly created update thread to reload the pre-stored synchronization data in the key-value memory area based on the mapping relationship stored in the shared memory area for data retrieval.

[0148] The data sharing strategy is not particularly limited, for example, a memory mapping (mmap) strategy or a shared memory strategy. Taking the mmap strategy as an example, through the mmap strategy, the process in which the update thread is located requests the service node to map the file content or device data into the virtual memory thereof. After the mapping is completed, the process can directly operate the memory through a pointer, just like operating a regular memory, without the need to read and write the file through a call of a read and write operation. The mmap strategy reduces the number of data copying, because the data can be directly loaded into the memory and accessed by the process, without being copied from a kernel buffer to an object space, reduces the call overhead, and improves the data retrieval efficiency.

[0149] Further, after obtaining the synchronization data, if the data retrieval instruction is received, the service node can obtain the newly added data that is not synchronized and pre-stored in the key-value memory region from the retrieval database based on the storage time of the corresponding original data of the synchronization data in the retrieval database and the instruction receiving time of the data retrieval instruction. Details are described below. Then, after the service node receives the data retrieval instruction, obtains the newly added data that is not synchronized and pre-stored in the key-value memory region from the retrieval database based on the storage time and the instruction receiving time of the data retrieval instruction, the service node can establish, based on the preset data sharing strategy, a mapping relationship between the physical storage addresses of the synchronization data and the newly added data in the shared memory region and the virtual storage addresses stored in the shared memory region by using the update thread.

[0150] Through the shared memory region, the multiple processes created by the service node can share the data stored in the shared memory region, that is, share the mapping relationship described above. Then, after detecting that the retrieval service is triggered to be closed, the newly added data is stored in the key-value memory region, and the synchronization data and the newly added data are taken as historical synchronization data. Thus, when detecting that the retrieval service is triggered to be restarted, the update thread that is re-created is used to reload the pre-stored synchronization data in the key-value memory region based on the mapping relationship stored in the shared memory region, so as to perform data retrieval, further reduce the amount of data that needs to be obtained from the retrieval database after the retrieval service is restarted, and improve the data retrieval efficiency.

[0151] Please refer to Figure 4C When the retrieval service in the service node is initially started, the update thread obtains the pre-stored synchronization data from the key-value memory region, obtains the storage time of the corresponding original data of the synchronization data in the retrieval database and the instruction receiving time of the received data retrieval instruction from the retrieval database, and stores the newly added data in the retrieval database. In the shared memory region, the mapping relationship between the physical storage addresses of the synchronization data and the newly added data and the virtual storage addresses stored in the shared memory region is established.

[0152] When the search service is restarted, the current synchronization data is loaded based on the mapping relationship stored in the shared memory area, the synchronization data loaded at the time of the restart includes the synchronization data obtained by the search service at the time of the initial start and the new data. Then, the storage time of the original data corresponding to the synchronization data in the search database is obtained from the search database, and the storage time of the new data in the search database is obtained between the instruction receiving time of the data search instruction received after the search service is restarted and the instruction receiving time of the data search instruction received after the search service is restarted. The new data is different from the new data obtained by the search service at the time of the initial start.

[0153] The search service is triggered to restart:

[0154] When it is detected that the search service is triggered to restart after being closed, the search service is restarted, and the service node can use the created update thread to load the pre-stored synchronization data in the key-value memory area based on the pre-stored mapping relationship in the shared memory area. The mapping relationship is established in the shared memory area by the search service at a historical time based on a preset data sharing strategy. The historical time can be any time of starting the search service before the current restart of the search service, for example, it can be the time of starting the search service last time before the current restart of the search service; for another example, it can be the time of starting the search service for the first time, etc., which is not limited.

[0155] The mapping relationship represents the correspondence between the physical storage address of the historical synchronization data and the virtual storage address stored in the shared memory area. The historical synchronization data includes the synchronization data obtained by the search service after being triggered to start at a historical time and the new data. For details, please refer to the foregoing description.

[0156] The process in which the data sharing strategy update thread is located requests the service node to map the file content or device data to its virtual memory. After the mapping is completed, the process can directly operate the memory through a pointer, just like operating a regular memory, without the need to read and write the file through the call of a read-write operation, reducing the number of data copying, because the data can be directly loaded into the memory and accessed by the process, without the need to be copied from the kernel buffer to the object space, reducing the calling overhead and improving the data search efficiency.

[0157] Please refer to Figure 5A When it is detected that the search service is triggered to restart after being closed, the service node creates an update thread. The service node uses the update thread to batch-pull the synchronization data based on the pre-stored mapping relationship in the shared memory area until all the synchronization data indicated by the mapping relationship is obtained. Then, the update thread is used to batch-obtain the new data from the search database until all the new data stored in the search database is obtained.

[0158] As an embodiment, when obtaining the new data from the retrieval database, the updating thread can continue to send a second data request to the retrieval database. The second data request indicates that the new data stored in the retrieval database in the time period between the storage time and the instruction receiving time is obtained from the retrieval database. The updating thread receives the new data returned by the retrieval database. Thus, the retrieval database only needs to obtain a small amount of data, which greatly improves the efficiency of obtaining data and improves the retrieval efficiency of data retrieval.

[0159] Please refer to Figure 5B The updating thread sends a second data request to the retrieval database. The retrieval database returns a batch of new data obtained first, another batch of new data obtained second, and the last batch of new data obtained last, and sends all the new data stored in the retrieval database to the updating thread. Compared with the method of obtaining full data from the retrieval database, the data transmission efficiency is higher.

[0160] S202, when receiving the data retrieval instruction, obtaining the new data not pre-stored in the key-value memory area from the retrieval database based on the storage time and the instruction receiving time of the data retrieval instruction.

[0161] When the data retrieval instruction is received, the instruction receiving time can be recorded. Since the synchronization data has been obtained, the new data stored in the retrieval database between the storage time of the synchronization data and the instruction receiving time can be obtained, so as to ensure the real-time data of the data retrieval. The process of obtaining the new data can be referred to the foregoing description, which will not be repeated here.

[0162] S203, based on the data retrieval instruction, performing data retrieval in the synchronization data and the new data to obtain a retrieval result.

[0163] After obtaining the synchronization data and the new data, the data retrieval instruction can be received, and the data retrieval can be performed in the synchronization data and the new data based on the data retrieval instruction to obtain a retrieval result. For example, the target data satisfying the similarity condition between the data retrieval instruction and the data retrieval instruction is searched in the synchronization data and the new data.

[0164] As an embodiment, the service node can further comprise a graphics processing unit (GPU) which has a larger processing data amount in parallel processing than a preset data amount. The preset data amount can be, for example, a processing data amount of a central processing unit (CPU) of the service node in parallel processing, and is not limited in particular. Then, after obtaining the synchronization data and the new data, the service node can copy the synchronization data and the new data in the video memory of the GPU, and perform data retrieval using the parallel processing capability of the GPU to improve the retrieval efficiency in data retrieval.

[0165] The service node calls the data interaction interface of the GPU to copy the synchronization data and the new data in the video memory of the GPU. The GPU performs data retrieval in the synchronization data and the new data based on the data retrieval instruction to obtain a retrieval result.

[0166] Please refer to Figure 6A The update thread can copy the obtained synchronization data in the GPU video memory each time the synchronization data is obtained, and copy the obtained new data in the GPU video memory each time the new data is obtained. When the synchronization data and the new data are all copied in the GPU video memory, the GPU can perform data retrieval in the synchronization data and the new data based on the data retrieval instruction to obtain a retrieval result.

[0167] As an embodiment, if the data retrieval instruction indicates that data with a similarity to the to-be-retrieved data satisfying a similarity condition is to be retrieved, the data features of each synchronization sub-data contained in the synchronization data and the data features of each new sub-data contained in the new data can be determined before the data is copied in the GPU video memory. Based on the data features of each synchronization sub-data and the data features of each new sub-data, each target sub-data matching the to-be-retrieved data is selected from each synchronization sub-data and each new sub-data. The data interaction interface of the GPU is called to copy each target sub-data in the video memory of the GPU. By reducing the data amount of the data to be copied in the GPU video memory, the retrieval efficiency in data retrieval is improved.

[0168] Please refer to Figure 6BThe CPU is used to determine the data characteristics of each synchronization sub-data contained in the synchronization data obtained by the update thread, and determine the data characteristics of each new sub-data contained in the new data obtained by the update thread. The CPU is used to select each target sub-data matching the data to be searched from each synchronization sub-data and each new sub-data. Finally, the data interaction interface of the GPU is called to copy each target sub-data to the video memory of the GPU, and data searching is performed in each target sub-data to obtain a search result. The process of obtaining each target sub-data can also be processed by the GPU, and the specific process is not limited.

[0169] As an embodiment, if the data searching instruction indicates that data with a similarity to the data to be searched satisfying a similarity condition is searched. When data searching is performed, the data to be searched indicated by the data searching instruction can be obtained. The similarity between each synchronization sub-data contained in the synchronization data and each new sub-data contained in the new data and the data to be searched is determined to obtain the corresponding data similarity. Based on the obtained data similarity, multiple search sub-data satisfying the similarity condition indicated by the data searching instruction are selected from each synchronization sub-data and each new sub-data as the search result.

[0170] For example, please refer to Figure 6C The service node receives the data searching instruction through the worker thread, and the worker thread calls the device interface to send the data searching instruction to the GPU, indicating that the GPU is used to determine the similarity between each synchronization sub-data contained in the synchronization data and each new sub-data contained in the new data and the data to be searched, and obtain the corresponding data similarity. After the GPU determines each data similarity, the synchronization sub-data and the new sub-data are sorted in descending order of each data similarity, and the target sub-data ranked in the top K is selected.

[0171] The GPU sends the obtained K target sub-data to the CPU. The CPU obtains the reference data related to the K target sub-data in the CPU memory based on the received K target sub-data. The worker thread is used to return the K target sub-data and the reference data corresponding to each target sub-data as the search result.

[0172] Next, the data searching method provided by the application embodiment is exemplarily introduced taking the palm print payment scene as an example. Please refer to Figure 7A and Figure 7B .

[0173] S701, the service node creates a daemon process in the script machine, and periodically obtains stored data from the search database and synchronizes it to the key value memory area in the memory of the service node to form synchronization data.

[0174] The service node can use a daemon process to obtain the newly stored data in the retrieval database from the retrieval database every 5 minutes after the retrieval service is configured and before the retrieval service is started for the first time, and synchronize the obtained stored data to the key-value memory area.

[0175] For example, after the retrieval service is configured, the retrieval database starts to store data, and after 5 minutes, refer to Figure 7C .

[0176] S71, synchronizing the stored data stored in the retrieval database in the past 5 minutes to the key-value memory area in batches;

[0177] S72, when it is determined that the stored data stored in the retrieval database in the past 5 minutes is not synchronized in a batch, continuing to synchronize the stored data in batches until the stored data is completely synchronized to the key-value memory area;

[0178] S73, when the stored data is completely synchronized to the key-value memory area, continuing to synchronize the stored data stored in the retrieval database in the next 5 minutes to the key-value memory area.

[0179] In this way, until the retrieval service is started for the first time, the synchronization is paused, and when the retrieval service is triggered to be closed, the synchronization is continued to be started; or after the retrieval service is started, the synchronization is also continued, which is not limited herein.

[0180] S702, when the service node starts the retrieval service for the first time, using the created update thread to pull the pre-stored synchronization data from the key-value memory area.

[0181] Through the pre-stored synchronization data in the key-value memory area, the update thread can obtain the stored data in the retrieval database before the last 5 minutes. The service node can record the latest storage time of the stored data in the retrieval database before the last 5 minutes.

[0182] S703, the service node obtains the newly stored data in the retrieval database between the recorded latest storage time and the instruction receiving time when the data retrieval instruction is received.

[0183] Since the time between the latest storage time and the instruction receiving time is less than 5 minutes, the time is relatively short, and the stored data stored in the retrieval database is relatively small, so the data amount obtained from the retrieval database is greatly reduced, the data obtaining efficiency is improved, and the data retrieval efficiency is improved.

[0184] S704, the service node uses the update thread to store the obtained synchronization data and the newly stored data as historical synchronization data, and stores the mapping relationship between the physical storage address and the virtual storage address of the historical synchronization data in the shared memory area.

[0185] S705, the service node copies the obtained synchronization data and the new data in the GPU memory by using the update thread.

[0186] S706, the service node determines the similarity between the data to be retrieved indicated by the data retrieval instruction and the obtained synchronization data and the new data by using the GPU in parallel. A plurality of target sub-data with the maximum similarity are selected from the synchronization data and the new data as the retrieval result.

[0187] S707, the service node triggers the closing of the retrieval service and restarts the retrieval service.

[0188] S708, the service node loads the historical synchronization data as the synchronization data obtained after restarting the retrieval service by using the newly created update thread based on the mapping relationship pre-stored in the shared memory area.

[0189] Through the historical synchronization data indicated by the mapping relationship pre-stored in the shared memory area, the update thread can obtain the storage data used in the last data retrieval in the retrieval database. The service node can update the recorded latest storage time based on the storage time of the storage data used in the last data retrieval in the retrieval database.

[0190] Based on the currently obtained synchronization data, S703-S708 are executed.

[0191] Through the data retrieval method provided by the embodiments of the present application, when the retrieval service is restarted, the service node only needs 25 seconds from receiving the data retrieval instruction to giving the retrieval result, which greatly improves the data retrieval efficiency and reduces the impact on the service node caused by restarting the retrieval service compared with 15 minutes in the related art. Similarly, when the retrieval service is expanded, the service node only needs at most 60 seconds from receiving the data retrieval instruction to giving the retrieval result, which greatly improves the data retrieval efficiency and reduces the impact on the service node caused by expanding the retrieval service compared with 60 minutes in the related art. Similarly, when the retrieval service performs data retrieval processing, the time for the service node to give the retrieval result is in the order of seconds, which greatly improves the data retrieval efficiency and improves the processing efficiency during data retrieval processing compared with the order of seconds in the related art.

[0192] Based on the same inventive concept, the embodiments of the present application provide a data retrieval device, which can realize the functions corresponding to the foregoing data retrieval method. Please refer to Figure 8 The device comprises an acquisition module 801 and a processing module 802, wherein:

[0193] The acquisition module 801 is configured to acquire pre-stored synchronization data from the key-value memory area when the retrieval service starts; wherein the synchronization data is pre-stored from the retrieval database according to the storage time of the corresponding original data in the retrieval database;

[0194] The processing module 802 is configured to acquire, when the data retrieval instruction is received, new data that is not pre-stored in the key-value memory area from the retrieval database based on the storage time and the instruction receiving time of the data retrieval instruction.

[0195] The processing module 802 is further configured to perform data retrieval in the synchronization data and the new data based on the data retrieval instruction, and obtain a retrieval result.

[0196] In a possible embodiment, the processing module 802 is specifically configured to:

[0197] In response to a service configuration operation for the service node, configure the retrieval service for the service node;

[0198] When it is detected that the retrieval service is started for the first time, the created update thread is used to send a first data request to the key-value memory area; wherein the first data request indicates that pre-stored synchronization data is acquired from the key-value memory area.

[0199] The update thread is used to receive the synchronization data returned by the key-value memory area.

[0200] In a possible embodiment, the processing module 802 is further configured to:

[0201] After the new data that is not pre-stored in the key-value memory area is acquired from the retrieval database based on the storage time and the instruction receiving time of the data retrieval instruction when the data retrieval instruction is received, the update thread is used to establish, based on a pre-set data sharing strategy, a mapping relationship between the respective physical storage addresses of the synchronization data and the new data in the shared memory area and the virtual storage addresses stored in the shared memory area.

[0202] After it is detected that the retrieval service is closed, the new data is stored in the key-value memory area, and the synchronization data and the new data are taken as historical synchronization data.

[0203] When it is detected that the retrieval service is restarted, the update thread that is re-created is used to reload the pre-stored synchronization data in the key-value memory area based on the mapping relationship stored in the shared memory area, so as to perform data retrieval.

[0204] In a possible embodiment, the processing module 802 is further configured to:

[0205] After the retrieval service is configured for the service node in response to the service configuration operation for the service node, the daemon process is adopted to synchronize the stored data stored in the retrieval database to the key-value memory area in each period with a preset time length as a cycle, when the retrieval service is detected to be initially started or is detected to be closed.

[0206] In a possible embodiment, the key-value memory area comprises a plurality of feature partitions, and data stored in different feature partitions have different data features.

[0207] The processing module 802 is specifically configured to:

[0208] For each cycle, the following operations are performed respectively: determining the data features of each storage sub-data stored in the retrieval database in the cycle;

[0209] For each storage sub-data, the following operations are performed respectively:

[0210] When it is determined that there is a target partition in the plurality of feature partitions that matches the data feature of the storage sub-data, the storage sub-data is synchronized to the target partition;

[0211] When it is determined that there is no target partition in the plurality of feature partitions that matches the data feature of the storage sub-data, a target partition is newly created in the key-value memory area, and the storage sub-data is synchronized to the target partition.

[0212] In a possible embodiment, when the storage sub-data is palmprint data, each two feature partitions in the plurality of feature partitions have at least one feature difference between the respective data features corresponding to the two feature partitions, and the at least one feature difference is at least one of the following: a number of main lines included in the palmprint, a number of triangular regions formed by the main lines, a pattern of the main lines, and a size range to which a size of the palmprint belongs; and the pattern of the main lines includes at least one of the following: a ring shape, a spiral shape, and an arc shape.

[0213] In a possible embodiment, the processing module 802 is specifically configured to:

[0214] When the retrieval service is started, a data retrieval instruction is received; and the data retrieval instruction indicates data that satisfies a similarity condition with the to-be-retrieved data in terms of similarity;

[0215] From the plurality of feature partitions, at least one to-be-retrieved partition that matches a data feature of the to-be-retrieved data is determined;

[0216] The pre-stored synchronization data is obtained from the at least one to-be-retrieved partition.

[0217] In a possible embodiment, the processing module 802 is specifically configured to:

[0218] When it is detected that the retrieval service is restarted after being closed, the created update thread loads the pre-stored synchronization data in the key-value memory area based on the pre-stored mapping relationship in the shared memory area;

[0219] The mapping relationship is established in the shared memory area by the retrieval service based on a preset data sharing strategy when the retrieval service is started at a historical time. The mapping relationship represents a correspondence between a physical storage address of historical synchronization data and a virtual storage address stored in the shared memory area. The historical synchronization data includes synchronization data and new data obtained by the retrieval service after being started at the historical time.

[0220] In a possible embodiment, the processing module 802 is specifically configured to:

[0221] The update thread is used to send a second data request to the retrieval database. The second data request indicates that the new data stored in the retrieval database in a time period between the storage time and the instruction receiving time is obtained.

[0222] The update thread is used to receive the new data returned by the retrieval database.

[0223] In a possible embodiment, the service node includes a graphics processing unit. The processing data amount of the graphics processing unit when processing in parallel is greater than a preset data amount.

[0224] The processing module 802 is specifically configured to:

[0225] The data interaction interface of the graphics processing unit is called to copy the synchronization data and the new data in the video memory of the graphics processing unit.

[0226] The graphics processing unit is used to perform data retrieval in the synchronization data and the new data based on the data retrieval instruction, to obtain a retrieval result.

[0227] In a possible embodiment, the data retrieval instruction indicates that data similar to the to-be-retrieved data satisfies a similarity condition.

[0228] The processing module 802 is specifically configured to:

[0229] The data characteristics of each synchronization sub-data included in the synchronization data and the data characteristics of each new sub-data included in the new data are respectively determined.

[0230] Based on the data characteristics of each synchronization sub-data and the data characteristics of each new sub-data, each target sub-data matched with the to-be-retrieved data is selected from each synchronization sub-data and each new sub-data.

[0231] The data interaction interface of the graphics processing unit is called to copy each target sub-data in the video memory of the graphics processing unit.

[0232] In a possible implementation, the data retrieval instruction indicates that data with a similarity to the to-be-retrieved data satisfying a similarity condition is to be retrieved.

[0233] The processing module 802 is specifically configured to:

[0234] Obtain the to-be-retrieved data indicated by the data retrieval instruction.

[0235] Determine the similarity between each synchronization sub-data contained in the synchronization data and each new sub-data contained in the new data and the to-be-retrieved data respectively, to obtain corresponding data similarities.

[0236] Based on the obtained data similarities, select multiple retrieval sub-data from the synchronization sub-data and the new sub-data, which satisfy the similarity condition indicated by the data retrieval instruction, as the retrieval result.

[0237] For more details, please refer to Figure 9 A computer device 900 provided in an embodiment of the present application, for example, can be a client 101 or a server 102 in Figure 1C The current version and the historical version of the data storage program and the application software corresponding to the data storage program can be installed on the computer device 900, which includes a processor 980 and a memory 920. In some embodiments, the computer device 900 can include a display unit 940, which includes a display panel 941, for displaying a user interaction interface and the like.

[0238] In a possible implementation, the display panel 941 can be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0239] The processor 980 is configured to read a computer program and then execute the method defined by the computer program. For example, the processor 980 reads a data storage program or a file, so as to run the data storage program on the computer device 900 and display a corresponding interface on the display unit 940. The processor 980 can include one or more general-purpose processors and also can include one or more digital signal processors (DSPs) for performing related operations to implement the technical solutions provided in the embodiments of the present application.

[0240] The memory 920 generally includes internal memory and external memory. The internal memory can be random access memory (RAM), read only memory (ROM), cache memory (CACHE), etc. The external memory can be a hard disk, an optical disk, a USB disk, a floppy disk, a tape drive, etc. The memory 920 is configured to store computer programs and other data. The computer programs include application programs corresponding to respective clients, etc. The other data can include data generated after the operating system or the application programs are executed, including system data (e.g., configuration parameters of the operating system) and user data. In the embodiments of the present application, the computer programs are stored in the memory 920, and the processor 980 executes the computer programs stored in the memory 920 to implement any one of the methods discussed above with reference to the drawings.

[0241] The display unit 940 is configured to receive inputted digital information, character information or contact touch operation / non-contact gesture, and generate signal input related to user settings and function control of the computer device 900, etc. Specifically, in the embodiments of the present application, the display unit 940 can include a display panel 941. The display panel 941, for example, a touch screen, can collect touch operations (such as user operations on or near the display panel 941 using fingers, styluses or any suitable objects or accessories) of the user and drive corresponding connection devices according to pre-set programs.

[0242] In a possible embodiment, the display panel 941 can include two parts, a touch detection device and a touch controller. The touch detection device detects the touch position of the player and detects the signals caused by the touch operation, and transmits the signals to the touch controller. The touch controller receives the touch information from the touch detection device, converts the touch information into touch coordinates, and sends the touch coordinates to the processor 980. The touch controller can also receive commands from the processor 980 and execute the commands.

[0243] The display panel 941 can be implemented in various types, such as resistive, capacitive, infrared and surface acoustic wave, etc. In addition to the display unit 940, in some embodiments, the computer device 900 can further include an input unit 930. The input unit 930 can include an image input device 931 and other input devices 932. The other input devices can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, on / off keys, etc.), a trackball, a mouse, a joystick, etc.

[0244] In addition to the above, the computer device 900 can further include a power supply 990 for supplying power to other modules, an audio circuit 960, a near field communication module 970, and an RF circuit 910. The computer device 900 can further include one or more sensors 950, such as an acceleration sensor, a light sensor, a pressure sensor, etc. The audio circuit 960 specifically includes a speaker 961 and a microphone 962, etc., for example, the computer device 900 can collect the user's voice through the microphone 962, perform corresponding operations, etc.

[0245] As an embodiment, the number of processors 980 can be one or more, and the processor 980 and the memory 920 can be coupled or relatively independent.

[0246] As an embodiment, Figure 9 The processor 980 in the computer device 900 can be configured to implement the functions of the acquisition module 801 and the processing module 802 in the computer device 900. Figure 8

[0247] As an embodiment, Figure 9 The processor 980 in the computer device 900 can be configured to implement the functions of the server or the terminal device corresponding to the functions discussed above.

[0248] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by a computer program, and the aforementioned computer program can be stored in a computer readable storage medium, and when the computer program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0249] Alternatively, the integrated units of the present application can be stored in a computer readable storage medium if they are realized in the form of software function modules and sold or used as independent products. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products, for example, through a computer program product stored in a storage medium, including a computer program for causing a computer device to execute all or part of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a mobile storage device, a ROM, a RAM, a magnetic disk or an optical disk, and various media that can store program codes.

[0250] ​Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A data retrieval method, characterized in that, Applied to service nodes configured with retrieval services, including: When the retrieval service starts, pre-stored synchronized data is retrieved from the key-value memory area; wherein, the synchronized data is pre-stored from the retrieval database to the key-value memory area according to the storage time of the corresponding original data in the retrieval database. Upon receiving a data retrieval instruction, based on the storage time and the instruction reception time of the data retrieval instruction, the newly added data that was not synchronously pre-stored in the key-value memory area is retrieved from the retrieval database; Based on the data retrieval instructions, data retrieval is performed in the synchronized data and the newly added data to obtain retrieval results.

2. The method according to claim 1, characterized in that, The step of retrieving pre-stored synchronized data from the key-value memory area when the retrieval service starts includes: In response to a service configuration operation for the service node, configure the retrieval service for the service node; When the retrieval service is detected to be started for the first time, a first data request is sent to the key-value memory area using a created update thread; wherein, the first data request indicates: to retrieve pre-stored synchronization data from the key-value memory area; The update thread receives the synchronization data returned by the key-value memory region.

3. The method according to claim 2, characterized in that, Upon receiving a data retrieval instruction, after retrieving newly added data from the retrieval database that was not synchronously pre-stored in the key-value memory area based on the storage time and the instruction reception time of the data retrieval instruction, the process further includes: Based on the preset data sharing strategy, the update thread is used to establish a mapping relationship between the physical storage addresses of the synchronized data and the newly added data in the shared memory area and the virtual storage addresses stored in the shared memory area. After detecting that the retrieval service is closed, the newly added data is stored in the key-value memory area, and the synchronized data and the newly added data are used as historical synchronized data; When the retrieval service is detected to have restarted, a newly created update thread is used to reload the pre-stored synchronization data in the key-value memory area based on the mapping relationship stored in the shared memory area, in order to perform data retrieval.

4. The method according to claim 2, characterized in that, After configuring the retrieval service for the service node in response to the service configuration operation for the service node, the method further includes: If the initial startup of the search service is not detected or the search service is detected to be closed, a daemon process is created to synchronize the stored data stored in the search database in each cycle to the key-value memory area at a preset time interval.

5. The method according to claim 4, characterized in that, The key-value memory area contains multiple feature partitions, and the data stored in different feature partitions have different data characteristics. The step of sequentially synchronizing the stored data stored in the retrieval database in each period to the key-value memory area includes: For each of the aforementioned periods, the following operations are performed: determine the data characteristics of each storage sub-data stored in the retrieval database during that period; For each of the stored sub-data, perform the following operations respectively: When it is determined that there is a target partition among the plurality of feature partitions that matches the data features of the stored sub-data, the stored sub-data is synchronized to the target partition; If it is determined that there is no target partition among the plurality of feature partitions that matches the data characteristics of the stored sub-data, the target partition is created in the key-value memory area, and the stored sub-data is synchronized to the target partition.

6. The method according to claim 5, characterized in that, When the stored sub-data is palmprint data, among the multiple feature partitions, there are at least one difference between the data features corresponding to each pair of feature partitions: the number of main lines included in the palmprint, the number of triangular areas formed by the main lines, the pattern of the main lines, and the size range to which the palmprint size belongs; wherein, the pattern of the main lines includes at least one of the following: ring, spiral, and arc.

7. The method according to claim 5, characterized in that, The step of retrieving pre-stored synchronized data from the key-value memory area when the retrieval service starts includes: When the search service is started, a data retrieval instruction is received; wherein the data retrieval instruction indicates: to retrieve data whose similarity to the data to be retrieved meets the similarity condition; From the plurality of feature partitions, determine at least one partition that matches the data features of the data to be retrieved; Retrieve pre-stored synchronization data from the at least one partition to be retrieved.

8. The method according to claim 1, characterized in that, The step of retrieving pre-stored synchronized data from the key-value memory area when the retrieval service starts includes: When the retrieval service is detected to be restarted after being closed, the created update thread loads the synchronized data pre-stored in the key-value memory area based on the mapping relationship pre-stored in the shared memory area; The mapping relationship is established in the shared memory area based on a preset data sharing strategy when the retrieval service starts at a historical time. The mapping relationship represents the correspondence between the physical storage address of the historical synchronized data and the virtual storage address stored in the shared memory area. The historical synchronized data includes synchronized data and newly added data obtained by the retrieval service after it starts at a historical time.

9. The method according to claim 8, characterized in that, The step of retrieving newly added data from the retrieval database that was not pre-stored in the key-value memory area based on the storage time and the instruction reception time of the data retrieval instruction includes: The update thread is used to send a second data request to the retrieval database; wherein the second data request indicates: to retrieve the newly added data stored in the retrieval database during the time period between the storage time and the instruction reception time; The update thread is used to receive the new data returned by the retrieval database.

10. The method according to any one of claims 1 to 9, characterized in that, The service node includes a graphics processing unit, and the amount of data processed by the graphics processing unit in parallel is greater than the preset amount of data. The step of retrieving data from the synchronized data and the newly added data based on the data retrieval instruction to obtain retrieval results includes: The data interaction interface of the graphics processing unit is invoked to copy the synchronized data and the newly added data into the video memory of the graphics processing unit; Using the graphics processing unit, based on the data retrieval instruction, data retrieval is performed in the synchronized data and the newly added data to obtain retrieval results.

11. The method according to claim 10, characterized in that, The data retrieval instruction indicates: retrieve data whose similarity to the data to be retrieved meets the similarity condition; The step of calling the data interaction interface of the graphics processing unit to copy the synchronization data and the new data into the video memory of the graphics processing unit includes: The data characteristics of each synchronized sub-data contained in the synchronized data are determined, and the data characteristics of each newly added sub-data contained in the newly added data are determined. Based on the data characteristics of each synchronized sub-data and each newly added sub-data, target sub-data that matches the data to be retrieved is selected from the synchronized sub-data and the newly added sub-data. The data interaction interface of the graphics processing unit is invoked to copy each target sub-data into the video memory of the graphics processing unit.

12. The method according to claim 10, characterized in that, The data retrieval instruction indicates: retrieve data whose similarity to the data to be retrieved meets the similarity condition; The step of using the graphics processing unit to perform data retrieval based on the data retrieval instruction in the synchronized data and the newly added data to obtain retrieval results includes: Obtain the data to be retrieved as indicated by the data retrieval instruction; The similarity between each synchronized sub-data contained in the synchronized data and each newly added sub-data contained in the newly added data and the data to be retrieved is determined to obtain the corresponding data similarity. Based on the obtained data similarity, multiple retrieval sub-data that satisfy the similarity conditions indicated by the data retrieval instruction are selected from the synchronized sub-data and the newly added sub-data, and are used as the retrieval results.

13. A data retrieval device, characterized in that, Applied to service nodes configured with retrieval services, including: Acquisition module: used to acquire pre-stored synchronized data from the key-value memory area when the retrieval service starts; wherein, the synchronized data is pre-stored from the retrieval database to the key-value memory area according to the storage time of the corresponding original data in the retrieval database; Processing module: When a data retrieval instruction is received, based on the storage time and the instruction reception time of the data retrieval instruction, it retrieves newly added data that is not synchronously pre-existing in the key-value memory area from the retrieval database; The processing module is also used to: perform data retrieval in the synchronized data and the newly added data based on the data retrieval instruction, and obtain retrieval results.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1 to 12.

15. A computer device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method as described in any one of claims 1 to 12 according to the obtained program instructions.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the method as described in any one of claims 1 to 12.

Images