Data processing method, device and equipment

Abstract

An embodiment of the present application discloses a data processing method, which obtains the data to be inserted and stores the data to be inserted in a first storage space when a data insertion request is received. The first storage space is only used to store the newly inserted data. When a query request is received while the first index map is constructed based on the data in the first storage space, a query operation is performed based on the first query request and the second index map to obtain a first query result. The second index map is constructed based on the data in the second storage space, and the second storage space is used to store the original data. That is, the embodiment of the present application uses multiple storage spaces to perform data insertion operations and data query operations respectively. When the first index map is constructed based on the first storage space, the second storage space will not be locked. Therefore, when the first index map is constructed, the second index map and the second storage space can still be used to respond to query requests, thereby avoiding the problem of query performance degradation due to locking the updated index map.

Description

Technical Field

[0001] The present application relates to the field of computer processing technology, and in particular to a data processing method, apparatus and device. Background Art

[0002] Graph index-based retrieval systems are widely used due to their high performance and high recall. However, graph indexes themselves have limited support for data updates. One update method involves scheduled rebuilds or incremental builds, but these methods incur significant latency. Another approach involves real-time updates, but to prevent query errors, operations that respond to query requests must be locked. This means that queries cannot be responded to while updates are in progress, resulting in reduced query performance and a negative user experience. Summary of the Invention

[0003] In view of this, embodiments of the present application provide a data processing method, apparatus, and device to avoid performance loss caused by locking when updating a graph index.

[0004] To achieve the above objectives, the technical solutions provided in the embodiments of the present application are as follows:

[0005] In a first aspect of an embodiment of the present application, a data processing method is provided, which may include:

[0006] In response to receiving a data insertion request, obtaining data to be inserted, and storing the data to be inserted in a first storage space, where the first storage space is used to store the data to be inserted;

[0007] In response to receiving a first query request when constructing a first index map based on data in the first storage space, a first query result is determined based on the first query request and a second index map, where the second index map is constructed based on the original data in the second storage space.

[0008] In a second aspect of an embodiment of the present application, a data processing device is provided, which may include:

[0009] An acquiring unit, configured to acquire the data to be inserted in response to receiving a data insertion request, and store the data to be inserted in a first storage space, where the first storage space is used to store the data to be inserted;

[0010] A determination unit is used to determine a first query result based on the first query request and a second index map in response to receiving a first query request when constructing a first index map based on data in the first storage space, where the second index map is constructed based on the original data in the second storage space.

[0011] In a third aspect of the embodiments of the present application, an electronic device is provided, the device comprising: a processor and a memory;

[0012] The memory is used to store instructions or computer programs;

[0013] The processor is used to execute the instructions or computer programs in the memory, so that the electronic device performs the data processing method described in the first aspect.

[0014] In a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, comprising instructions, which, when executed on a computer, enable the computer to execute the data processing method described in the first aspect above.

[0015] It can be seen that the embodiments of the present application have the following beneficial effects:

[0016] The technical solution provided by the embodiment of the present application obtains the data to be inserted when a data insertion request is received, and stores the data to be inserted in the first storage space, and the first storage space is only used to store the newly inserted data. When a query request is received when the first index map is constructed according to the data in the first storage space, a query operation can be performed according to the first query request and the second index map to obtain a first query result. Among them, the second index map is constructed based on the data in the second storage space, and the second storage space is used to store the original data. That is, the embodiment of the present application uses multiple storage spaces to perform data insertion operations and data query operations respectively. When the first index map is constructed according to the first storage space, the second storage space will not be locked, so that when the first index map is constructed, the second index map and the second storage space can still be used to respond to query requests, thereby avoiding the problem of query performance degradation due to locking of the updated index map. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart of a data processing method provided in an embodiment of the present application;

[0018] Figure 2 A structural diagram of a data processing device provided in an embodiment of the present application;

[0019] Figure 3 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] To make the above-mentioned objects, features, and advantages of the present application more clearly understood, the following further describes the embodiments of the present application in detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present application and are not intended to limit the present application. In addition, it should be noted that, for ease of description, the drawings only show portions relevant to the present application, not all structures.

[0021] To facilitate understanding of the technical solution of this application, the technical terms involved in this application will be explained below.

[0022] In vector retrieval systems, operations such as searching database data are typically performed using graph indexes. Specifically, a graph index is pre-built for the database data. When operations such as searching, updating, adding, or deleting data are required, the corresponding data can be found and the relevant operations performed based on the data structure established by the graph index. In practical applications, when new data is inserted into the database, the graph index must be updated in real time to ensure that the newly inserted data can be found. To ensure smooth execution of the update operation, the database is locked during the graph index update, resulting in an inability to respond to query requests in a timely manner and reduced query performance.

[0023] Based on this, an embodiment of the present application provides a data processing method that adopts a multi-cache space approach, so that data query and data addition can be performed in different cache spaces, avoiding the performance loss problem caused by locking.

[0024] To facilitate understanding of the technical solutions provided in the embodiments of the present application, they will be described below with reference to the accompanying drawings.

[0025] See also Figure 1 , which is a flow chart of a data processing method provided by an embodiment of the present application. The embodiment of the present application is applicable to situations where a query request needs to be responded to when building a graph index. The method can be executed by a data processing device, which can be implemented in the form of software and / or hardware, such as Figure 1 As shown, the method may include:

[0026] S101: In response to receiving a data insertion request, obtaining data to be inserted, and storing the data to be inserted into a first storage space.

[0027] In this embodiment, the data processing device can receive database operation requests in real time. When the operation request is a data insertion request, the device obtains the data to be inserted and stores the data in the first storage space. The first storage space is only used to store newly inserted data within a preset time period. In other words, the data in the first storage space is time-sensitive and becomes invalid after being stored for a certain period of time.

[0028] Based on the above explanation, to ensure that the inserted data can be queried, the acquired data to be inserted must also be inserted into the original database, i.e., the second storage space. This second storage space stores not only the original data but also the newly inserted data. For example, at time t1, the original data in the original database (the second storage space) consists of data a1-a10, and the first storage space is empty. At time t1, new data b1 is inserted. Data b1 is then inserted into the first and second storage spaces, respectively. At this point, the data in the first storage space is b1, and the data in the second storage space is a1-a10 and b1.

[0029] S102: In response to receiving a first query request when constructing a first index map according to data in a first storage space, determining a first query result according to the first query request and a second index map.

[0030] After the data to be inserted is stored in the first storage space, to ensure that the newly inserted data can be retrieved, a first index graph is periodically constructed based on the data in the first storage space. When a first query request is received while the first index graph is being constructed based on the data in the first storage space, a query operation can be executed based on the first query request and the second index graph to obtain a first query result.

[0031] It is understood that since the first index map is constructed based on the data in the first storage space, there is no need to lock the second storage space. Therefore, when a first query request is received while the first index map is being constructed, the second storage space and the second index map can still be used to promptly respond to the query request. The first query request may include a data identifier of the data to be queried, and the data to be queried is retrieved from the second storage space based on the data identifier and the second index map.

[0032] Among them, the first storage space can maintain the data inserted within a window time, and if it times out, part of the data in the first storage space will be invalid. Therefore, the first index map will be constructed based on the data in the first storage space at regular intervals. Specifically, in response to the arrival of the construction cycle, the first index map is constructed based on the data in the first storage space. Further, in response to the window time timeout, the expired data in the first storage space is invalidated. Among them, invalidating the expired data can mean deleting the data from the first storage space. For example, the first storage space saves the data b1, b2 and b3 inserted within the first window time, and constructs the first index map b1 based on the data b1, b2 and b3; newly inserts data b4-b7 within the second window time. At this time, the data b1-b3 expires, and the first index map B2 is constructed based on the data b4-b7.

[0033] It can be seen that when a data insertion request is received, the data to be inserted is obtained and stored in the first storage space, and the first storage space is only used to store the newly inserted data. When a query request is received while the first index map is constructed based on the data in the first storage space, a query operation can be performed based on the first query request and the second index map to obtain a first query result. Among them, the second index map is constructed based on the data in the second storage space, and the second storage space is used to store the original data. That is, the embodiment of the present application uses multiple storage spaces to perform data insertion operations and data query operations respectively. When the first index map is constructed based on the first storage space, the second storage space will not be locked, so that when the first index map is constructed, the second index map and the second storage space can still be used to respond to query requests, thereby avoiding the problem of query performance degradation due to locking the updated index map.

[0034] Considering that the first storage space is only used to store newly inserted data and that data is time-sensitive, to ensure data stability, newly inserted data must also be stored in the second storage space. The second index map is periodically updated based on the data in the second storage space to obtain an updated second index map. This ensures that newly inserted data can be found when responding to query requests based on the second index map. The second storage space is used to store all data, including both original data and newly inserted data.

[0035] In a specific implementation, when a second query request is received before the first index map is constructed and before the second index map is constructed, the query result is determined based on the second query request, the first index map, and the second index map. That is, when the second query request is received, query operations will be performed based on the first index map and the second index map, respectively. Specifically, a third query result is obtained based on the first index map and the first storage space; a fourth query result is obtained based on the second index map and the second storage space; and the third query result and the fourth query result are merged to obtain the second query result. That is, when the data processing device receives the second query request and does not execute the index map construction operation, it will perform data query based on the first index map and the second index map at the same time, thereby ensuring that the newly inserted data can be queried in a timely manner, thereby improving query efficiency.

[0036] In some application scenarios, there is a situation where data needs to be deleted from the database. Usually, the data needs to be deleted from the second storage space and the second index map needs to be rebuilt. To avoid the inability to respond to query requests due to the deletion operation, this embodiment provides a solution. Specifically, in response to receiving a deletion request, the data identifier of the data to be deleted is stored in the third storage space. That is, when data needs to be deleted from the second storage space, the data identifier of the data to be deleted is saved in the third storage space, and there is no need to perform a deletion operation on the second storage space, thereby not rebuilding the second index map. When the first query request is executed, a fifth query result is obtained based on the first query request, the second index map, and the second storage space; the data located in the third storage space in the fifth query result is removed to obtain the first query result. That is, when the data identifier included in the fifth query result appears in the third storage space, indicating that the data corresponding to the data identifier is data that needs to be deleted, the data is removed from the fifth query result to obtain the final query result, i.e., the first query result.

[0037] Similarly, when determining the second query result according to the second query request, the first index map and the second index map, it is necessary to remove the data in the second query result located in the third storage space to obtain the final query result.

[0038] Based on the above method embodiment, an embodiment of the present application provides a data processing device, which will be described below with reference to the accompanying drawings.

[0039] See also Figure 2 , which is a structural diagram of a data processing device provided in an embodiment of the present application, such as Figure 2 As shown, the apparatus may include: an acquiring unit 201 and a determining unit 202 .

[0040] An acquiring unit 201 is configured to acquire data to be inserted in response to receiving a data insertion request, and store the data to be inserted in a first storage space, where the first storage space is used to store the data to be inserted;

[0041] The determination unit 202 is used to determine a first query result based on the first query request and the second index map in response to receiving a first query request when constructing a first index map based on the data in the first storage space, where the second index map is constructed based on the original data in the second storage space.

[0042] In a specific implementation, the device further includes: a storage unit and an update unit;

[0043] The storage unit is used to store the data to be inserted into the second storage space, and the second storage space is used to store the original data and the data to be inserted;

[0044] The updating unit is configured to update the second index map according to the data in the second storage space to obtain an updated second index map.

[0045] In a specific implementation, the determining unit 202 is further configured to, in response to receiving a second query request, determine a second query result according to the second query request, the first index map, and the second index map.

[0046] In a specific implementation, the determination unit 202 is specifically used to obtain a third query result based on the first index map and the first storage space; obtain a fourth query result based on the second index map and the second storage space; and merge the third query result and the fourth query result to obtain a second query result.

[0047] In a specific implementation, the determination unit 202 is specifically used to obtain a fifth query result based on the first query request, the second index map and the second storage space; remove the data located in the third storage space in the fifth query result to obtain the first query result, and the third storage space is used to store data that needs to be deleted.

[0048] In a specific implementation, the device further includes: a storage unit;

[0049] The storage unit is configured to store a data identifier of the data to be deleted in the third storage space in response to receiving a deletion request.

[0050] In a specific implementation, the apparatus further includes: a configuration unit;

[0051] The configuration unit is configured to invalidate the expired data in the first storage space in response to a window timeout.

[0052] It should be noted that the implementation of each unit in this embodiment can refer to the relevant description in the above method embodiment, and this embodiment will not be repeated here.

[0053] Reference below Figure 3, which shows a schematic structural diagram of an electronic device 300 suitable for implementing an embodiment of the present application. The terminal device in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (portable Android devices), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs (televisions) and desktop computers. Figure 3 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0054] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. Various programs and data required for the operation of the electronic device 300 are also stored in the RAM 303. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0055] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0056] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of the embodiment of the present application are performed.

[0057] The electronic device provided in the embodiment of the present application and the data processing method provided in the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0058] An embodiment of the present application provides a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the data processing method as described in any of the above embodiments is implemented.

[0059] It should be noted that the computer-readable medium mentioned above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0060] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0061] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0062] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the data processing method.

[0063] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0064] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0065] The units involved in the embodiments described in this application may be implemented by software or hardware. In some cases, the name of a unit / module does not constitute a limitation of the unit itself. For example, a voice data acquisition module may also be described as a "data acquisition module."

[0066] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0067] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0068] According to one or more embodiments of the present application, a data processing method is provided, the method comprising:

[0069] In response to receiving a data insertion request, obtaining data to be inserted, and storing the data to be inserted in a first storage space, where the first storage space is used to store the data to be inserted;

[0070] In response to receiving a first query request when constructing a first index map based on data in the first storage space, a first query result is determined based on the first query request and a second index map, where the second index map is constructed based on the original data in the second storage space.

[0071] According to one or more embodiments of the present application, the method further includes:

[0072] storing the data to be inserted into the second storage space, where the second storage space is used to store the original data and the data to be inserted;

[0073] The second index map is updated according to the data in the second storage space to obtain an updated second index map.

[0074] According to one or more embodiments of the present application, the method further includes:

[0075] In response to receiving a second query request, a second query result is determined according to the second query request, the first index map, and the second index map.

[0076] According to one or more embodiments of the present application, determining a second query result based on the second query request, the first index map, and the second index map includes:

[0077] Obtain a third query result based on the first index map and the first storage space;

[0078] Obtain a fourth query result based on the second index map and the second storage space;

[0079] The third query result and the fourth query result are combined to obtain a second query result.

[0080] According to one or more embodiments of the present application, determining a first query result according to the first query request and the second index map includes:

[0081] Obtain a fifth query result according to the first query request, the second index map, and the second storage space;

[0082] The data in the third storage space of the fifth query result is removed to obtain the first query result, and the third storage space is used to store data that needs to be deleted.

[0083] According to one or more embodiments of the present application, the method further includes:

[0084] In response to receiving the deletion request, a data identifier of the data to be deleted is stored in the third storage space.

[0085] According to one or more embodiments of the present application, the method further includes:

[0086] In response to the window time expiring, the expired data in the first storage space is invalidated.

[0087] According to one or more embodiments of the present application, a data processing device is provided, which may include:

[0088] An acquiring unit, configured to acquire the data to be inserted in response to receiving a data insertion request, and store the data to be inserted in a first storage space, where the first storage space is used to store the data to be inserted;

[0089] A determination unit is used to determine a first query result based on the first query request and a second index map in response to receiving a first query request when constructing a first index map based on data in the first storage space, where the second index map is constructed based on the original data in the second storage space.

[0090] According to one or more embodiments of the present application, the apparatus further includes: a storage unit and an update unit;

[0091] The storage unit is used to store the data to be inserted into the second storage space, and the second storage space is used to store the original data and the data to be inserted;

[0092] The updating unit is configured to update the second index map according to the data in the second storage space to obtain an updated second index map.

[0093] According to one or more embodiments of the present application, the determining unit is further configured to, in response to receiving a second query request, determine a second query result based on the second query request, the first index map, and the second index map.

[0094] According to one or more embodiments of the present application, the determination unit is specifically used to obtain a third query result based on the first index map and the first storage space; obtain a fourth query result based on the second index map and the second storage space; and merge the third query result and the fourth query result to obtain a second query result.

[0095] According to one or more embodiments of the present application, the determination unit is specifically used to obtain a fifth query result based on the first query request, the second index map and the second storage space; remove the data located in the third storage space in the fifth query result to obtain the first query result, and the third storage space is used to store data that needs to be deleted.

[0096] According to one or more embodiments of the present application, the apparatus further includes: a storage unit;

[0097] The storage unit is configured to store a data identifier of the data to be deleted in the third storage space in response to receiving a deletion request.

[0098] According to one or more embodiments of the present application, the apparatus further includes: a configuration unit;

[0099] The configuration unit is configured to invalidate the expired data in the first storage space in response to a window timeout.

[0100] According to one or more embodiments of the present application, an electronic device is provided, the device including: a processor and a memory;

[0101] The memory is used to store instructions or computer programs;

[0102] The processor is configured to execute the instructions or computer program in the memory so that the electronic device executes the data processing method.

[0103] According to one or more embodiments of the present application, a computer-readable storage medium is provided, comprising instructions, which, when executed on a computer, enable the computer to execute the above-mentioned data processing method.

[0104] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0105] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0106] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0107] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0108] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method, characterized in that: The method comprises: In response to receiving a data insertion request, obtaining data to be inserted, and storing the data to be inserted in a first storage space, where the first storage space is used to store the data to be inserted; In response to receiving a first query request while constructing a first index map based on data in the first storage space, determining a first query result based on the first query request and a second index map constructed based on original data in the second storage space; storing the data to be inserted into the second storage space, where the second storage space is used to store the original data and the data to be inserted; updating the second index map according to the data in the second storage space to obtain an updated second index map; In response to receiving a second query request, a second query result is determined according to the second query request, the first index map, and the second index map.

2. The method according to claim 1, characterized in that The determining a second query result according to the second query request, the first index map, and the second index map includes: Obtain a third query result based on the first index map and the first storage space; Obtain a fourth query result based on the second index map and the second storage space; The third query result and the fourth query result are combined to obtain a second query result.

3. The method according to claim 1 or 2, characterized in that Determining a first query result according to the first query request and the second index map includes: Obtain a fifth query result according to the first query request, the second index map, and the second storage space; The data in the third storage space of the fifth query result is removed to obtain the first query result, and the third storage space is used to store data that needs to be deleted.

4. The method according to claim 3, characterized in that The method further comprises: In response to receiving the deletion request, a data identifier of the data to be deleted is stored in the third storage space.

5. The method according to claim 1, wherein The method further comprises: In response to the window time expiring, the expired data in the first storage space is invalidated.

6. A data processing device, characterized in that: The device comprises: An acquiring unit, configured to acquire the data to be inserted in response to receiving a data insertion request, and store the data to be inserted in a first storage space, where the first storage space is used to store the data to be inserted; a determining unit configured to, in response to receiving a first query request when constructing a first index map based on data in the first storage space, determine a first query result based on the first query request and a second index map constructed based on original data in the second storage space; A storage unit, configured to store the data to be inserted into the second storage space, wherein the second storage space is configured to store the original data and the data to be inserted; an updating unit, configured to update the second index map according to the data in the second storage space to obtain an updated second index map; The determining unit is further configured to, in response to receiving a second query request, determine a second query result based on the second query request, the first index map, and the second index map.

7. An electronic device, comprising: processor and memory; The memory is used to store instructions or computer programs; The processor is configured to execute the instructions or computer program in the memory so that the electronic device executes the data processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the data processing method according to any one of claims 1 to 5.

Images