Data processing method, electronic device, system, storage medium and program product

Abstract

Embodiments of the present application provide a data processing method, electronic equipment, system, storage medium and program product, relating to the technical field of artificial intelligence, the method comprising: creating a call-in memory page for target data related to to-be-processed data, the target data being located in a physical persistent storage space; adding the call-in memory page to a memory page set of a virtual storage space, and storing the target data to a target cache area corresponding to the call-in memory page in a physical cache space; and generating a processing result of the to-be-processed data according to the target data in the target cache area. In the technical solution of the embodiments of the present application, flexible access and efficient management of memory are achieved, and the accuracy of the processing result can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a data processing method, electronic device, system, storage medium and program product that can be applied to the field of intelligent question answering technology. Background Technology

[0002] With the continuous development of artificial intelligence technology, intelligent agents are being widely used. Users may raise new questions or follow up on past issues while using intelligent agents. Therefore, memory management of intelligent agents is becoming increasingly important. However, current memory management strategies cannot effectively manage historical context over long time intervals, leading to the forgetting of relevant information and the loss of contextual logic, thus reducing the performance and accuracy of the intelligent agent's responses. Summary of the Invention

[0003] This application provides a data processing method, electronic device, system, storage medium, and program product to alleviate or solve one or more technical problems existing in the prior art.

[0004] In a first aspect, embodiments of this application provide a data processing method, comprising: creating a loading memory page for target data related to the data to be processed, wherein the target data is located in a physical persistent storage space; adding the loading memory page to a set of memory pages in a virtual storage space, and storing the target data in a target cache area corresponding to the loading memory page in a physical cache space; and generating a processing result for the data to be processed based on the target data in the target cache area.

[0005] Secondly, embodiments of this application provide a data processing method, comprising: responding to a data processing instruction, sending data to be processed to a server, so that the server creates a loading memory page for target data related to the data to be processed; adding the loading memory page to a memory page set in a virtual storage space; storing the target data in a target cache area corresponding to the loading memory page in a physical cache space; and generating a processing result of the data to be processed based on the target data in the target cache area, wherein the target data is located in a physical persistent storage space; and displaying the processing result sent by the server.

[0006] Thirdly, embodiments of this application provide a data processing system, including: a client and a server, wherein the client is used to execute the method provided in the second aspect above, and the server is used to execute the method provided in the first aspect above.

[0007] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the methods of embodiments of this application when executing the computer program.

[0008] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method of any one of the embodiments of this application.

[0009] Sixthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements any of the methods described in the embodiments of this application.

[0010] In the technical solution of this application embodiment, data is stored in layers, namely, in a physical persistent storage space and a physical cache space. A loading memory page is created for the target data related to the data to be processed in the physical persistent storage space. This loading memory page is added to the memory page set in the virtual storage space, and the target data is stored in the target cache area corresponding to the loading memory page in the physical cache space. Therefore, based on the virtual storage space, the scheduling of target data from the physical persistent storage space to the physical cache space is realized. Furthermore, the target data in the physical cache space can be effectively managed based on the loading memory page. That is, by dynamically scheduling data in the two layers of storage, flexible access and efficient management of memory are achieved. Moreover, the processing result is generated based on the scheduled target data related to the data to be processed, avoiding interference from invalid data and improving the accuracy of the processing result.

[0011] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0012] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0013] Figure 1 This paper illustrates a first architecture diagram of the data processing system provided in an embodiment of this application;

[0014] Figure 2 A second architectural schematic diagram of the data processing system provided in an embodiment of this application is shown;

[0015] Figure 3 A first flowchart of the data processing method provided in an embodiment of this application is shown;

[0016] Figure 4 A first schematic diagram of the data processing method provided in this application embodiment is shown;

[0017] Figure 5 A second schematic diagram of the data processing method provided in this application embodiment is shown;

[0018] Figure 6 A second flowchart of the data processing method provided in an embodiment of this application is shown;

[0019] Figure 7 This paper shows a first block diagram of the data processing apparatus provided in an embodiment of the present application;

[0020] Figure 8 A second block diagram of the data processing apparatus provided in an embodiment of this application is shown;

[0021] Figure 9 A block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0022] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0023] First, the terms used in the embodiments of this application are explained as follows:

[0024] Intelligent agent: A software or hardware entity that can perceive the environment, make autonomous decisions, and execute actions to achieve specific goals, and has the ability to learn autonomously, make autonomous decisions, remember, and process information.

[0025] Memory management: the management of data storage, retrieval, updating, and deletion processes to support agents in using data efficiently.

[0026] Layered memory: Memory (i.e., data) is stored in layers in physical persistent storage space and physical cache space to improve management efficiency and process historical data and real-time interactive information separately.

[0027] Memory page: The smallest unit of virtual storage space management. Data storage location management is achieved through dynamic scheduling of memory pages.

[0028] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0029] With the rapid development of artificial intelligence technology, the memory management capabilities of intelligent agents are becoming increasingly important. Related memory management strategies include full memory, knowledge graphs, and vector databases. Full memory involves stuffing all historical dialogues into the agent's context window; however, processing long contexts is computationally complex and limited by the length of the context window, making it unsuitable for long-term interactions. Knowledge graphs use graph structures to store highly structured relational knowledge, but graph structures are costly to build, and updates typically require batch processing, making it difficult to support real-time, fine-grained memory updates in agent interactions. Vector databases convert data into vectors and rely on similarity for retrieval; however, each retrieval may recall a large number of semantically relevant but dialogue-irrelevant memories, not only crowding the context window but also interfering with result generation. Therefore, current memory management strategies cannot effectively manage memory, thus reducing the agent's performance and response accuracy.

[0030] In view of this, embodiments of this application provide a data processing method, electronic device, system, storage medium, and program product, aiming to achieve flexible access and efficient management of memory, thereby improving the accuracy of processing results. A detailed description follows.

[0031] Figure 1 This application provides an embodiment of a data processing system, which is illustrated in the following schematic diagram. Figure 1 As shown, the system architecture includes a hierarchical memory structure, as well as a client and server for communication connections.

[0032] The hierarchical memory structure includes physical persistent storage and physical cache space. Physical persistent storage can be configured on the server or on a storage device. It can take the form of a disk, hard drive, database, etc. Physical persistent storage is used to store historical memory (also known as cold memory or cold data), such as user profiles and historical conversation data; in other words, it is a long-term data warehouse. Physical cache space can be the server's memory, used to store currently active memory (also known as hot memory or hot data), such as recent conversation snippets and temporary tasks.

[0033] The client can be a mobile phone, tablet computer, desktop computer, laptop, smart wearable device, vehicle terminal, or other terminal device. Figure 1The example shown uses a desktop computer as the client. The client can run a smart question-and-answer application, which can be a standalone application (App), a mini-program embedded in other applications, or a web application. Users can use this smart question-and-answer application to edit data to be processed and input data processing commands. Correspondingly, the client responds to the data processing commands, sends the data to be processed to the server, and displays the processing result upon receiving it from the server.

[0034] The server can be a physical server or a cloud server for cloud computing, etc. Figure 1 The following diagram illustrates the process using a physical server as the server-side example. The server establishes a communication connection with the client. Upon receiving data to be processed from the client, and in response to the lack of a matching memory page in the virtual storage space's memory page set, the server creates a loading memory page for the target data related to the data to be processed in the physical persistent storage space. This loading memory page is then added to the virtual storage space's memory page set, and the target data is stored in the target cache area corresponding to the loading memory page in the physical cache space. This enables the scheduling of target data from the physical persistent storage space to the physical cache space, i.e., the retrieval of the target data. Finally, based on the target data in the target cache area, the processing result for the data to be processed is generated.

[0035] The above describes the architecture of the data processing system from the physical deployment perspective. The following section explains the architecture from the logical functional perspective, such as... Figure 2 As shown, the architecture can include an interaction layer, a memory management layer, a recall strategy layer, and a function call and security layer. The interaction layer includes an input unit and a generation unit. The input unit receives user-inputted data to be processed, and the generation unit generates the processing results. The memory management layer includes physical persistent storage, physical cache space, and a virtual memory manager (VMM). The virtual memory manager dynamically schedules data between the physical persistent storage and physical cache space, handles the loading and unloading of memory pages, and maintains related mapping data. The recall strategy layer includes an extraction unit and an evaluation system. The extraction unit extracts the semantic vector of the data to be processed, and the evaluation system determines the target data related to the data to be processed in the physical persistent storage according to a preset evaluation method, i.e., determines the target data to be recalled. The function call and security layer includes an interface unit and a security unit. The interface unit provides interfaces for adding, deleting, modifying, and querying memory pages, and the security unit performs client authentication and desensitizes sensitive data.

[0036] As can be seen, by implementing layered storage of memory—that is, storing data in both physical persistent storage and physical cache space—and based on virtual storage space and the data to be processed, the target data can be scheduled from physical persistent storage space to physical cache space. Furthermore, the target data can be effectively managed in the physical cache space based on the loaded memory pages. In other words, by dynamically scheduling data across the two storage layers, flexible access and efficient management of memory are achieved. Moreover, generating processing results based on the scheduled target data related to the data to be processed avoids interference from invalid data, thus improving the accuracy of the processing results.

[0037] It should be noted that the application scenarios or examples provided in the embodiments of this application are for ease of understanding, and the embodiments of this application do not specifically limit the application of the technical solutions. In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0038] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0039] Figure 3 A flowchart of a data processing method according to an embodiment of this application is shown. Figure 3 The method shown can be used by Figure 1 Server-side execution, such as Figure 3 As shown, the method may include steps S301, S302 and S303.

[0040] Step S301: Create a loading memory page for the target data related to the data to be processed. The target data is located in the physical persistent storage space.

[0041] In some implementations, the client, in response to a user's data processing instruction, can send a data processing request to the server. The server retrieves the data to be processed from this request. If it determines that the target data related to the data to be processed is located in physical persistent storage, it can call a memory page creation interface to create a loading memory page for the target data in order to schedule the target data from physical persistent storage to physical cache space (i.e., recall the target data). The data processing request may also include information such as the client's identifier. The target data may include historical data to be processed and the historical processing results of that historical data. The historical data to be processed in the target data can be data similar to the data to be processed. The data to be processed can be in the form of text, audio, images, etc. If the data to be processed is audio, it can first be converted into text data, and subsequent processing can be performed based on the text data.

[0042] To ensure data security, in some implementations, when the server receives a data processing request from the client, it can first verify the identity information contained in the data processing request to ensure the authenticity of the client. The identity information can be a client identifier or verification information generated according to a preset method, etc., and this application does not specifically limit its scope.

[0043] Step S302: Add the loaded memory page to the memory page set in the virtual storage space, and store the target data in the target cache area corresponding to the loaded memory page in the physical cache space.

[0044] In some implementations, the physical persistent storage space stores all data. When any data in the physical persistent storage space is recalled, although the data still exists in the physical persistent storage space, its popularity type changes from cold data to hot data. That is, after recalling the target data, the popularity type of the target data changes from cold data to hot data, and the target data needs to be managed in the physical cache space. In order to effectively manage the target data in the physical cache space, in some implementations, the memory page created for the target data is added to the memory page set of the virtual storage space, and the target data is stored in the target cache area corresponding to the memory page in the physical cache space. Thus, when the target data needs to be accessed, the target cache area can be quickly located based on the memory page, realizing effective access to the target data.

[0045] Step S303: Generate the processing result of the data to be processed based on the target data in the target cache area.

[0046] In some implementations, input data can be determined based on target data, data to be processed, and preset prompt word templates. This input data is then input into the data processing model to obtain the processing result of the data to be processed.

[0047] In other implementations, the target data includes historical data to be processed and historical processing results. The historical processing results can be determined as the processing results of the data to be processed, or the historical processing results can be converted into processing results in a preset format.

[0048] In the technical solution of this application embodiment, data is stored in layers, namely, in a physical persistent storage space and a physical cache space. A loading memory page is created for the target data related to the data to be processed in the physical persistent storage space. This loading memory page is added to the memory page set in the virtual storage space, and the target data is stored in the target cache area corresponding to the loading memory page in the physical cache space. Therefore, based on the virtual storage space and the data to be processed, the target data is scheduled from the physical persistent storage space to the physical cache space. Furthermore, the target data in the physical cache space can be effectively managed based on the loading memory page. That is, by dynamically scheduling data in the two storage layers, flexible access and efficient management of memory are achieved. Moreover, the processing result is generated based on the scheduled target data related to the data to be processed, avoiding interference from invalid data and improving the accuracy of the processing result.

[0049] Since the access speed of the physical cache space is higher than that of the physical persistent storage space, in order to improve the response speed of the data to be processed, before creating a memory page for the target data related to the data to be processed, the process may also include: in response to the fact that the target data is not stored in the physical cache space, retrieving the target data from the physical persistent storage space.

[0050] In some implementations, after the server obtains the data to be processed from the data processing request sent by the client, it can determine whether the target data related to the data to be processed exists in the physical cache space. If the target data is not stored in the physical cache space, the target data is obtained from the physical persistent storage space.

[0051] Furthermore, the method may also include: if the target data is stored in the physical cache space, generating the processing result of the data to be processed based on the target data stored in the physical cache space.

[0052] Therefore, we first determine whether the target data exists in the physical cache space, and if the target data is not stored in the physical cache space, we retrieve the target data from the physical persistent storage space. This not only ensures the efficiency of retrieving the target data and thus improves the efficiency of generating the processing results, but also realizes the ordered access of hierarchical memory.

[0053] To quickly and accurately determine whether target data is stored in the physical cache space, some implementations effectively manage the data stored in the physical cache space based on virtual memory. Specifically, the physical cache space includes cache regions associated with any memory page in the memory page set. Accordingly, determining whether target data related to the data to be processed exists in the physical cache space can include: extracting a first semantic vector of the data to be processed; determining the similarity between at least one second semantic vector in the mapping table and the first semantic vector; and determining that no target data is stored in the physical cache space if each similarity is less than a similarity threshold. Here, at least one second semantic vector corresponds one-to-one with at least one memory page in the memory page set, and each second semantic vector is the semantic vector of the data stored in the cache region associated with the corresponding memory page.

[0054] Since the physical cache space includes a cache region associated with any memory page, and that cache region contains data, it is possible to determine whether the set of memory pages contains a memory page that matches the data to be processed, and if the set of memory pages does not contain a memory page that matches the data to be processed, it is determined that the target data is not stored in the physical cache space.

[0055] In some implementations, an extraction tool can be invoked to extract a first semantic vector from the data to be processed. Then, at least one second semantic vector from the mapping table is matched one by one with the first semantic vector. Specifically, for any second semantic vector in the mapping table, the similarity between the second semantic vector and the first semantic vector is determined, and it is determined whether the similarity is less than a similarity threshold. If the similarity is less than the similarity threshold, it is determined that the memory page corresponding to the second semantic vector does not match the data to be processed. If the memory pages corresponding to all second semantic vectors do not match the data to be processed, it is determined that the memory page set does not contain a memory page that matches the data to be processed, i.e., it is determined that the target data is not stored in the physical cache space.

[0056] In other implementations, the similarity between each second semantic vector and the first semantic vector in the mapping table can be determined first. Then, based on each similarity, it can be determined whether the memory page set contains a memory page that matches the data to be processed. That is, for any second semantic vector in the mapping table, the similarity between that second semantic vector and the first semantic vector can be determined. After obtaining the similarity corresponding to each second semantic vector, it is determined whether each similarity is less than a similarity threshold. If all similarities are less than the similarity threshold, it is determined that the memory page set does not contain a memory page that matches the data to be processed, that is, it is determined that the target data is not stored in the physical cache space.

[0057] Therefore, by associating cache regions in the physical cache space with memory pages in the virtual storage space, and maintaining a second semantic vector of the data stored in the cache region corresponding to each memory page in the mapping table, it is possible to quickly and accurately measure whether the target data exists in the physical cache space based on the second semantic vector and the first semantic vector of the data to be processed.

[0058] Furthermore, after obtaining the target data to be processed from the physical persistent storage space, in order to facilitate quick access to the target data later, in some implementations, the method further includes: assigning a page number to the memory page of the target data and extracting the second semantic vector of the target data; establishing a mapping relationship between the page number, the second semantic vector of the target data and the physical address of the target cache area; and adding the mapping relationship to the mapping table.

[0059] Optionally, after creating a load memory page for the target data, a target cache region associated with the load memory page can be allocated from the physical cache space, and a page number can be assigned to the load memory page according to the page number allocation rule. Then, the extraction tool is called to extract the second semantic vector of the target data. Alternatively, after creating a load memory page for the target data, the allocation operation of the target cache region and page number is performed simultaneously with the extraction operation of the second semantic vector. Alternatively, after obtaining the target data from the physical persistent storage space, the extraction tool is called to extract the second semantic vector of the target data; and after creating a load memory page for the target data, a target cache region associated with the load memory page is allocated from the physical cache space, and a page number is assigned to the load memory page according to the page number allocation rule. Furthermore, a mapping relationship is established between the allocated page number, the extracted second semantic vector, and the physical address of the allocated target cache region, and this mapping relationship is added to a mapping table. For example, the page number allocation rule is sequential numbering.

[0060] Therefore, after the target data is scheduled from the physical persistent storage space to the physical cache space, that is, after the target data is recalled, by establishing a mapping relationship between the page number of the target data's loading memory page, the second semantic vector of the target data, and the physical address of the target cache region associated with the loading memory page and storing it in the mapping table, the target data can be quickly and accurately located based on the mapping relationship in the mapping table when accessing the target data in the future, so as to achieve efficient management and effective access to the target data.

[0061] To simplify memory management and improve data access performance, in some implementations, the storage capacity of each memory page in the memory page set is the same. Accordingly, retrieving target data from physical persistent storage may include: retrieving the dataset corresponding to the client's identifier from the physical persistent storage; and retrieving target data from the dataset that is related to the data to be processed and matches the storage capacity.

[0062] In some implementations, the data processing request sent by the client may include a client identifier, and each historical data point is stored in physical persistent storage space corresponding to its respective client identifier. Accordingly, for any identifier in the physical persistent storage space, the identifier can be compared with the client identifier in the data processing request. If the comparison results match, the data corresponding to that identifier is determined as data in the client's dataset.

[0063] In other implementations, a storage area can be allocated for each client in the physical persistent storage space, and the corresponding client's identifier and dataset can be stored in any storage area. Accordingly, for the identifier stored in any storage area in the physical persistent storage space, the identifier can be compared with the client's identifier in the data processing request. If the comparison result matches, the dataset stored in the storage area where the identifier is located is determined as the client's dataset, that is, the dataset corresponding to the client's identifier.

[0064] After obtaining the dataset corresponding to the client's identifier, target data that is related to the data to be processed and matches the storage capacity can be retrieved from that dataset. Here, target data that matches the storage capacity means that the capacity occupied by the target data is the same as the storage capacity.

[0065] Since different users have different question formats and preferences, obtaining the dataset corresponding to the client's identifier and extracting the target data from that dataset ensures that the target data is relevant to the task at hand. This also allows the target data to better represent user preferences, thereby improving user satisfaction with the processing results generated based on the target data. Furthermore, obtaining target data that matches the storage capacity of the memory pages not only avoids memory page fragmentation issues but also retrieves more data, thus improving the hit rate of subsequent physical cache space.

[0066] In some implementations, the dataset corresponding to the client's identifier includes multiple data pairs, each data pair including associated historical data to be processed and historical processing results. Accordingly, the aforementioned acquisition of target data related to the data to be processed and matching the storage capacity from the dataset may include: when the total amount of data in the dataset is greater than the storage capacity, determining the association score between the historical data to be processed and the data to be processed for any historical data to be processed, the association score being used to characterize the degree of correlation between the historical data to be processed and the data to be processed; and determining the data corresponding to the storage capacity in the data pair corresponding to the target association score as the target data according to the chronological order of the data in the dataset, the target association score being greater than other association scores.

[0067] Considering that the user's pending data may be an appendage of historical pending data, in order to recall the context of the pending data and improve the accuracy of the processing results, when the total amount of data in the dataset exceeds the storage capacity of the memory page, for any historical pending data in the dataset, an association score is determined between the historical pending data and the pending data. A higher association score indicates a higher degree of correlation between the historical pending data and the pending data, meaning the historical pending data and the pending data are more similar, and the probability that the historical pending data is the context of the pending data is higher. To recall historical pending data that is more relevant to the pending data, a target association score can be determined among the association scores. The target association score is higher than other association scores; that is, when the association scores are sorted from largest to smallest, the top N association scores are determined as the target association scores, where N is a positive integer, and the value of N is related to the amount of data in the data pair corresponding to the top-ranked association scores. To ensure the completeness of the recalled target data, storage capacity data can be obtained from the data pairs corresponding to the target association scores according to the data's chronological order, and this obtained data is determined as the target data.

[0068] For example, the storage capacity of the memory page is 4k, the total amount of data in the dataset is 10k, the data size of the data pair corresponding to the highest association score of 0.88 is 2.5k, the data size of the data pair corresponding to the association score of 0.72 is 2k, and the number of data pairs corresponding to the association score of 0.70 is 1.6k. Since 2.5k + 2k = 4.5k exceeds the storage capacity of the memory page, N is determined to be 2, that is, the target association scores are 0.88 and 0.72. According to the order of the data, the first 1.5k of the data in the data pair corresponding to the association score of 0.88 and the data pair corresponding to the association score of 0.72 are determined as the target data.

[0069] Therefore, by determining the correlation score and then identifying the target data that matches the storage capacity of the memory page based on the correlation score, it is possible to ensure a high correlation between the target data and the data to be processed, while also avoiding the fragmentation problem of the memory page.

[0070] To accurately measure the correlation between the data to be processed and historical data to be processed, some implementations determine the association score comprehensively from multiple dimensions. Specifically, determining the association score between historical data to be processed and the data to be processed may include: determining the similarity between the historical data to be processed and the data to be processed; determining the number of times the historical data to be processed was stored within a preset time period based on the storage records of the historical data to be processed stored in the physical cache space; determining the time interval between the last storage time of the historical data to be processed and the current time based on the storage records; and weighting the similarity, the number of storage times, and the time interval to obtain the association score.

[0071] In some implementations, the physical persistent storage space can store the first semantic vector of each historical data to be processed. Correspondingly, the similarity between the first semantic vector of the historical data to be processed and the first semantic vector of the data to be processed can be calculated and used as the similarity between the historical data to be processed and the data to be processed. From the maintained storage records, at least one storage record containing the data identifier of the historical data to be processed is obtained. Each storage record may also include the storage time of the corresponding data being stored in the physical cache space. Target storage times within a preset time period are determined, and the number of target storage times is counted, which is used as the number of times the historical data to be processed is stored within the preset time period. Furthermore, the latest storage time among the target storage times is determined as the last storage time according to the chronological order. The time interval between the last storage time and the current time is determined. Based on the weights corresponding to similarity, number of storage times, and time interval, the similarity, number of storage times, and time interval are weighted to obtain an association score.

[0072] Among them, similarity is used to characterize the degree of similarity between historical data to be processed and the data to be processed from the content dimension; the number of times it is stored is used to characterize the frequency of access to historical data to be processed from the access dimension; and the time interval is used to characterize the time decay of accessing historical data to be processed from the time dimension. The preset time period can be a preset historical duration with the current time as the end time.

[0073] In other words, the association score = α•similarity + β•number of entries + γ•time interval, where α, β, and γ are preset weights, or α is generated based on the first semantic vector of the data to be processed (e.g., generated by calling a weight generation model), and β and γ are preset weights. The values ​​of α, β, and γ can be set as needed in practical applications, and this application does not impose specific limitations on them.

[0074] For example, if the preset time period is a historical 48-hour period ending at the current time, and the current time is 8:00 AM on September 20, 2025, then the preset time period is from 8:00 AM on September 18, 2025 to 8:00 AM on September 20, 2025. The storage times within the period from 8:00 AM on September 18, 2025 to 8:00 AM on September 20, 2025 are determined as the target storage times, and the latest storage time within the target storage times, 11:00 PM on September 19, 2025, is determined as the last storage time. Therefore, the time interval between the last storage time and the current time is determined to be 9 hours.

[0075] Therefore, by comprehensively determining the correlation score from the dimensions of content, access, and time, the accuracy of the correlation score is improved. The correlation score can accurately represent the degree of correlation between historical data to be processed and the data to be processed, thereby ensuring the accuracy of the target data obtained.

[0076] In other implementations, to improve the rate of determining the association score, the target similarity between any historical data to be processed and the data to be processed can be determined for any historical data to be processed in the dataset; the association score corresponding to the target similarity can be obtained from the correspondence between similarity and association score, and the obtained association score can be determined as the association score between the historical data to be processed and the data to be processed.

[0077] It should be noted that the method for determining the correlation score is not limited to the one described above, and can be set as needed in practical applications.

[0078] To facilitate subsequent calculation of the correlation score, after storing the target data in the target cache area corresponding to the loaded memory page in the physical cache space, the method further includes: generating a storage record for the target data based on the data identifier of the target data, the storage time of the target data in the target cache area, etc., and saving the storage record.

[0079] Furthermore, since the current session will be stored in the physical cache space, in order to avoid wasting storage resources by storing a large amount of similar data, in some embodiments, after generating the processing result of the data to be processed, the method may further include: updating the data pair corresponding to the first association score in the target cache area to the target data pair, wherein the target data pair includes the data to be processed and the processing result, the first association score is greater than the second association score in the target association score, and the second association score is the association score in the target association score other than the first association score.

[0080] Since the historical data to be processed in the data pair corresponding to the maximum association score has a high similarity to the data to be processed, and the historical processing results in the data pair corresponding to the maximum association score also have a high similarity to the generated processing results, in some implementations, to avoid storing too much similar data, the first association score can be the maximum association score among the target association scores. That is, the data pair corresponding to the maximum association score in the target cache area can be updated to the target data pair including the data to be processed and the processing results.

[0081] In other words, by updating the data pairs corresponding to the maximum correlation scores in the target cache region to target data pairs that include the data to be processed and the processing results, a large amount of similar data is avoided in the physical cache space, thus saving storage resources.

[0082] Similarly, when receiving data to be processed, if the target data is stored in the physical cache space, after generating the processing result of the data to be processed, the method may also include: updating the target data in the physical cache space to the data to be processed and the processing result, thereby avoiding storing a large amount of similar data in the physical cache space and saving storage resources.

[0083] Furthermore, when updating the target data in the physical cache space to the data to be processed and the processing result, if the first data volume of the data to be processed and the processing result is greater than the second data volume of the target data, a new memory page can be created, and a cache area corresponding to the new memory page can be allocated in the physical cache space; the second data volume of the data to be processed and the processing result is stored in the location where the target data is located, and the other data of the data to be processed and the processing result are stored in the cache area corresponding to the new memory page.

[0084] To facilitate the management of data loading and unloading from the physical cache space, in some implementations, such as Figure 4 As shown, a linked list is used to maintain each memory page. That is, at least one memory page in the memory page set corresponds one-to-one with at least one data node in the linked list, and the first data node in this set is the head of the linked list. Meanwhile, to ensure data security, the physical cache space includes sub-storage spaces corresponding to the data to be processed. These sub-storage spaces can be allocated to the client or to the current session.

[0085] Correspondingly, when there is available capacity in the sub-storage space, the aforementioned method of adding the loaded memory page to the memory page set in the virtual storage space includes: adding the data node corresponding to the loaded memory page to the head of the linked list, thereby adding the loaded memory page to the memory page set. Before storing the target data in the target cache area corresponding to the loaded memory page in the physical cache space, the method further includes: allocating the target cache area corresponding to the loaded memory page from the sub-storage space.

[0086] like Figure 4 As shown, the linked list can be a doubly linked list, which includes a head node, a tail node, and at least one data node. The head node is a fixed starting marker, and its next pointer points to the first data node, i.e., the head of the list. The first data node (i.e., the head) corresponds to the Most Recently Used (MRU) memory page. The tail node is a fixed ending marker, and its pre pointer points to the last data node, which corresponds to the Least Recently Used (LRU) memory page. After creating the loaded memory page, it can be determined whether there is available capacity in the sub-storage space corresponding to the data to be processed. If there is available capacity, the data node corresponding to the loaded memory page is created and added to the head of the linked list to indicate that the loaded memory page has been added to the memory page set. Furthermore, the target cache region corresponding to the loaded memory page is allocated from the available capacity of the sub-storage space. It should be noted that... Figure 4This is for illustrative purposes only and not intended to limit the scope of the mapping table. The mapping table may also include information such as the physical address of the cache region corresponding to the memory page. The page number is not limited to sequential numbering and may also include skipping numbers.

[0087] Therefore, when the sub-storage space corresponding to the data to be processed has available capacity, by adding the data node corresponding to the loaded memory page to the table header, it naturally indicates that the corresponding target data is the most recently accessed data, and also realizes the efficient addition of the loaded memory page to the memory page set. By allocating a target cache area in the sub-storage space, it ensures that the target data is stored in the corresponding sub-storage space, achieving effective data isolation and ensuring data security.

[0088] Furthermore, if there is no available capacity in the sub-storage space corresponding to the data to be processed, the aforementioned method of adding the loaded memory page to the set of memory pages in the virtual storage space includes: determining the loaded memory page in the set of memory pages, swapping the loaded memory page and the loaded memory page, and adding the loaded memory page to the set of memory pages. Before storing the target data in the target cache area corresponding to the loaded memory page in the physical cache space, the method further includes: determining the cache area corresponding to the loaded memory page in the sub-storage space as the target cache area.

[0089] In other words, if the sub-storage space corresponding to the data to be processed has no available capacity, some data stored in the sub-storage space needs to be retrieved, and the storage resources occupied by the retrieved data need to be allocated to the target data. In some implementations, to ensure that the data in the sub-storage space is the most frequently used data, the least recently accessed data in the sub-storage space can be retrieved. This involves identifying multiple target memory pages corresponding to the sub-storage space, identifying the target data nodes in the linked list corresponding to these multiple target memory pages, identifying the memory page corresponding to the target data node closest to the tail node as the retrieved memory page, and swapping the retrieved and retrieved memory pages. Furthermore, the cache area corresponding to the retrieved memory page in the sub-storage space is identified as the target cache area.

[0090] Therefore, when there is no available capacity in the sub-storage space corresponding to the data to be processed, by determining the memory page to be retrieved, swapping the retrieved memory page with the memory page to be retrieved, and determining the cache area corresponding to the retrieved memory page in the physical cache space as the target cache area, effective data management is achieved while ensuring effective data isolation, i.e., ensuring data security.

[0091] Considering that in practical applications, data in the physical cache space may be updated multiple times in a short period of time, in order to ensure the comprehensive retention of historical data, such as Figure 4As shown, data can not only be loaded into the physical cache space from the physical persistent storage space, but also loaded out of the physical cache space from the physical persistent storage space. That is, in some implementations, the aforementioned swapping of the loaded and unloaded memory pages can include: if the data corresponding to the unloaded memory page is not stored in the physical persistent storage space, retrieving the data corresponding to the unloaded memory page from the physical cache space; storing the data corresponding to the unloaded memory page in the physical persistent storage space; deleting the data node corresponding to the unloaded memory page from the linked list and adding the data node corresponding to the loaded memory page to the head of the linked list, thereby removing the unloaded memory page from the memory page set and adding the loaded memory page to the memory page set.

[0092] In some implementations, for any data identifier among the data corresponding to the retrieved memory page, it can be determined whether the physical persistent storage space contains that data identifier. If the data identifier is not stored, the data corresponding to that data identifier is retrieved from the physical cache space and stored in the physical persistent storage space. After storing all the data not stored in the physical persistent storage space, the data node corresponding to the retrieved memory page can be deleted from the linked list, and the data node corresponding to the loaded memory page can be added to the head of the linked list. This removes the retrieved memory page from the memory page set and adds the loaded memory page to the memory page set.

[0093] Therefore, during the swapping of memory pages, the data that needs to be synchronized to the physical persistent storage space is first stored in the physical persistent storage space, and then the memory pages are deleted and added. This avoids data loss and forgetting, ensures that the physical persistent storage space contains all user data, and achieves effective data management.

[0094] like Figure 5 As shown in the foregoing description, when the server obtains the data to be processed and determines that the target data of the data to be processed is located in the persistent physical storage space, it determines the association score between the data to be processed and the corresponding historical data to be processed by calculating the similarity, storage times and time intervals. Based on the association score and the capacity of the sub-storage space, the server can add and remove memory pages, that is, realize the swapping in and swapping out of memory pages, thereby realizing the effective management of each historical data and the effective management of recently accessed data.

[0095] Furthermore, considering that in practical applications, the data stored in the physical cache space or physical persistent storage space may contain users' private data, in order to ensure the security of this private data, in some implementations, the private data can first be encrypted using the client's corresponding encryption key to obtain ciphertext, and then the ciphertext can be stored in the corresponding physical cache space or physical persistent storage space. The encryption key can be assigned to the client during registration. Thus, by encrypting the private data, user data security is ensured.

[0096] Corresponding to the application scenarios of the methods provided in the embodiments of this application, the embodiments of this application also provide a data processing method, which can be applied to... Figure 1 The client shown is as follows: Figure 6 As shown, the method may include the following steps S601 and S602:

[0097] Step S601: In response to the data processing instruction, send the data to be processed to the server so that the server creates a loading memory page for the target data related to the data to be processed, adds the loading memory page to the memory page set of the virtual storage space, stores the target data in the target cache area corresponding to the loading memory page in the physical cache space, and generates the processing result of the data to be processed based on the target data in the target cache area, wherein the target data is located in the physical persistent storage space.

[0098] Step S602: Display the processing result sent by the server.

[0099] In some implementations, in response to a user-inputted data processing command, the client may send a data processing request to the server. The server retrieves the data to be processed from the data processing request, generates the processing result of the data to be processed in the aforementioned manner, and sends the processing result to the client. After receiving the processing result, the client displays the processing result.

[0100] In the technical solution of this application embodiment, data is stored in layers, namely, in physical persistent storage space and physical cache space. The server creates a loading memory page for the target data related to the data to be processed in the physical persistent storage space, adds this loading memory page to the memory page set in the virtual storage space, and stores the target data in the target cache area corresponding to the loading memory page in the physical cache space. Thus, based on the virtual storage space, the scheduling of target data from the physical persistent storage space to the physical cache space is realized, and the target data in the physical cache space can be effectively managed based on the loading memory page. That is, by dynamically scheduling data in the two layers of storage, flexible access and efficient management of memory are achieved. Furthermore, the processing result is generated based on the scheduled target data related to the data to be processed, avoiding interference from invalid data and improving the accuracy of the processing result.

[0101] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide a data processing apparatus, which can be applied to... Figure 1 The server shown is as follows: Figure 7 As shown, the device includes:

[0102] A creation module 701 is used to create a loading memory page for target data related to the data to be processed, wherein the target data is located in physical persistent storage space;

[0103] The addition module 702 is used to add the loaded memory page to the memory page set of the virtual storage space, and to store the target data in the target cache area corresponding to the loaded memory page in the physical cache space;

[0104] The generation module 703 is used to generate the processing result of the data to be processed based on the target data in the target cache area.

[0105] In some embodiments, the apparatus further includes an acquisition module for acquiring the target data from the physical persistent storage space in response to the physical cache space not storing the target data.

[0106] In some embodiments, the physical cache space includes a cache region associated with any memory page in the memory page set. The apparatus further includes a determining module for extracting a first semantic vector of the data to be processed; determining the similarity between at least one second semantic vector in the mapping table and the first semantic vector, wherein the at least one second semantic vector corresponds one-to-one with at least one memory page in the memory page set, and each second semantic vector is a semantic vector of data stored in the cache region associated with the corresponding memory page; and determining that the target data is not stored in the physical cache space if each similarity is less than a similarity threshold.

[0107] In some embodiments, the apparatus further includes a first update module, configured to assign a page number to the loaded memory page and extract a second semantic vector of the target data; establish a mapping relationship between the page number, the second semantic vector of the target data, and the physical address of the target cache region; and add the mapping relationship to the mapping table.

[0108] In some implementations, the data to be processed comes from the client, and the memory pages in the memory page set have the same storage capacity. The acquisition module is specifically used to: acquire the dataset corresponding to the client's identifier from the physical persistent storage space; and acquire target data that is related to the data to be processed and matches the storage capacity from the dataset.

[0109] In some implementations, the dataset includes multiple data pairs, each data pair including associated historical unprocessed data and historical processing results. The acquisition module is further specifically used to: when the total amount of data in the dataset is greater than the storage capacity, for any historical unprocessed data, determine the association score between the historical unprocessed data and the data to be processed, wherein the association score is used to characterize the degree of correlation between the historical unprocessed data and the data to be processed; according to the chronological order of the data in the dataset, determine the data corresponding to the storage capacity in the data pair corresponding to the target association score as the target data, wherein the target association score is greater than other association scores.

[0110] In some implementations, the acquisition module is further specifically used for: determining the similarity between the historical data to be processed and the data to be processed; determining the number of times the historical data to be processed was stored in the physical cache space according to the storage record of the historical data to be processed; determining the time interval between the last storage time of the historical data to be processed and the current time according to the storage record; and weighting the similarity, the number of storage times and the time interval to obtain the association score.

[0111] In some embodiments, the apparatus further includes a second update module, configured to update the data pair corresponding to the first association score in the target cache region to a target data pair after generating the processing result of the data to be processed, wherein the target data pair includes the data to be processed and the processing result, the first association score is greater than the second association score in the target association score, and the second association score is the association score in the target association score other than the first association score.

[0112] In some implementations, the physical cache space includes a sub-storage space corresponding to the data to be processed, at least one memory page in the memory page set corresponds one-to-one with at least one data node in the linked list, and the device further includes an allocation module, which, when there is available capacity in the sub-storage space:

[0113] The adding module 702 is specifically used to add the data node corresponding to the loaded memory page to the head of the linked list, so as to add the loaded memory page to the memory page set;

[0114] The allocation module is used to allocate a target cache region corresponding to the loaded memory page from the sub-storage space.

[0115] In some implementations, the physical cache space includes a sub-storage space corresponding to the data to be processed, in the case that the sub-storage space has no available capacity:

[0116] The adding module 702 is specifically used to determine the memory page to be retrieved in the memory page set, and to swap the memory page to be retrieved and the memory page to be retrieved, so as to add the memory page to the memory page set.

[0117] The allocation module is used to determine the cache area corresponding to the retrieved memory page in the sub-storage space as the target cache area.

[0118] In some implementations, at least one memory page in the memory page set corresponds one-to-one with at least one data node in the linked list, and the adding module 702 is further specifically used for:

[0119] If the data corresponding to the retrieved memory page is not stored in the physical persistent storage space, the data corresponding to the retrieved memory page is obtained from the physical cache space.

[0120] The data corresponding to the retrieved memory page is stored in the physical persistent storage space;

[0121] The data node corresponding to the retrieved memory page is deleted from the linked list, and the data node corresponding to the retrieved memory page is added to the head of the linked list, so as to remove the retrieved memory page from the memory page set and add the retrieved memory page to the memory page set.

[0122] In some embodiments, the apparatus further includes a third update module, configured to create a new memory page and allocate a cache area corresponding to the new memory page in the physical cache space when the target data is located in the physical cache space and the first data amount of the data to be processed and the processing result is greater than the second data amount of the target data; store the data of the data to be processed and the second data amount of the processing result in the location where the target data is located, and store the data of the data to be processed and the other data in the processing result in the cache area corresponding to the new memory page.

[0123] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0124] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide a data processing apparatus, which can be applied to... Figure 1 The client shown is as follows: Figure 8 As shown, the device includes:

[0125] The sending module 801 is used to send data to be processed to the server in response to a data processing instruction, so that the server creates a loading memory page for the target data related to the data to be processed, adds the loading memory page to the memory page set of the virtual storage space, stores the target data in the target cache area corresponding to the loading memory page in the physical cache space, and generates the processing result of the data to be processed based on the target data in the target cache area, wherein the target data is located in the physical persistent storage space;

[0126] Display module 802 is used to display the processing result sent by the server.

[0127] The functions of each module in the devices of this application embodiment can be found in the corresponding descriptions of the methods described above, and they have corresponding beneficial effects, which will not be repeated here. Furthermore, the device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components illustrated as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application solution according to actual needs.

[0128] Figure 9 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 9As shown, the electronic device includes a memory 901 and a processor 902. The memory 901 stores a computer program that can run on the processor 902. When the processor 902 executes the computer program, it implements the method described in the above embodiments. The number of memories 901 and processors 902 can be one or more. In a specific implementation, the electronic device may also include a communication interface 903 for communicating with external devices and exchanging data.

[0129] In practical implementation, if the memory 901, processor 902, and communication interface 903 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0130] Optionally, in a specific implementation, if the memory 901, processor 902, and communication interface 903 are integrated on a single chip, then the memory 901, processor 902, and communication interface 903 can communicate with each other through an internal interface.

[0131] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0132] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in this application.

[0133] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0134] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0135] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0136] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0137] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0138] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0139] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0140] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0141] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0142] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0143] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0144] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, comprising: extracting a first semantic vector of to-be-processed data; determining a similarity between at least one second semantic vector in a mapping table and the first semantic vector, respectively, wherein the at least one second semantic vector corresponds to at least one memory page in a memory page set of a virtual storage space, and any second semantic vector is a semantic vector of data stored in a cache area associated with a physical cache space of a corresponding memory page; in a case where each similarity is less than a similarity threshold, determining that target data related to the to-be-processed data is not stored in the physical cache space, and obtaining the target data from a physical persistent storage space; creating a recalled memory page for the target data, adding the recalled memory page to the memory page set, and storing the target data to a target cache area corresponding to the recalled memory page in the physical cache space; generating a processing result of the to-be-processed data according to the target data in the target cache area.

2. The method of claim 1, wherein, The method further comprises: allocating a page number to the recalled memory page, and extracting a second semantic vector of the target data; establishing a mapping relationship between the page number, the second semantic vector of the target data, and a physical address of the target cache area; adding the mapping relationship to the mapping table.

3. The method of claim 1, wherein, The to-be-processed data comes from a client, the storage capacity of each memory page in the memory page set is the same, and the obtaining of the target data from the physical persistent storage space comprises: obtaining a data set corresponding to an identifier of the client from the physical persistent storage space; obtaining target data related to the to-be-processed data and matching the storage capacity from the data set.

4. The method of claim 3, wherein, The data set comprises a plurality of data pairs, any data pair comprises associated historical to-be-processed data and a historical processing result, and the obtaining of the target data related to the to-be-processed data and matching the storage capacity from the data set comprises: in a case where the total amount of data in the data set is greater than the storage capacity, determining, for any historical to-be-processed data, an association score of the historical to-be-processed data and the to-be-processed data, wherein the association score represents the degree of correlation between the historical to-be-processed data and the to-be-processed data; determining, in a data pair corresponding to a target association score in a sequence of data in the data set, data corresponding to the storage capacity as target data, the target association score being greater than other association scores.

5. The method of claim 4, wherein, The determining of the association score of the historical to-be-processed data and the to-be-processed data comprises: determining a similarity between the historical to-be-processed data and the to-be-processed data; determining, according to a storage record of the historical to-be-processed data stored in the physical cache space, a storage frequency of the historical to-be-processed data within a preset time period; determining, according to the storage record, a time interval between a last storage time of the historical to-be-processed data and a current time; weighting the similarity, the storage frequency, and the time interval to obtain the association score.

6. The method of claim 4, wherein, After the generating of the processing result of the to-be-processed data, the method further comprises: updating a data pair corresponding to a first correlation score in the target cache area as a target data pair, wherein the target data pair comprises the to-be-processed data and the processing result, the first correlation score is greater than a second correlation score in the target correlation scores, and the second correlation score is a correlation score other than the first correlation score in the target correlation scores.

7. The method of any one of claims 1 to 6, wherein, The physical cache space comprises a sub-storage space corresponding to the to-be-processed data, and at least one memory page in the memory page set corresponds to at least one data node in the linked list. In a case where there is available capacity in the sub-storage space: The adding of the called-in memory page to the memory page set comprises adding a data node corresponding to the called-in memory page to a head of the linked list to add the called-in memory page to the memory page set. Before the storing of the target data to the target cache area corresponding to the called-in memory page in the physical cache space, the method further comprises: allocating a target cache area corresponding to the called-in memory page from the sub-storage space.

8. The method of any one of claims 1 to 6, wherein, The physical cache space comprises a sub-storage space corresponding to the to-be-processed data, and in a case where there is no available capacity in the sub-storage space: The adding of the called-in memory page to the memory page set comprises determining a called-out memory page in the memory page set, and swapping the called-in memory page and the called-out memory page to add the called-in memory page to the memory page set. Before the storing of the target data to the target cache area corresponding to the called-in memory page in the physical cache space, the method further comprises: determining, as the target cache area, a cache area corresponding to the called-out memory page in the sub-storage space.

9. The method of claim 8, wherein, At least one memory page in the memory page set corresponds to at least one data node in the linked list, and the swapping of the called-in memory page and the called-out memory page comprises: In a case where the data corresponding to the called-out memory page is not stored in the physical persistent storage space, acquiring the data corresponding to the called-out memory page from the physical cache space; storing the data corresponding to the called-out memory page to the physical persistent storage space; removing the data node corresponding to the called-out memory page from the linked list and adding a data node corresponding to the called-in memory page to a head of the linked list to remove the called-out memory page from the memory page set and add the called-in memory page to the memory page set.

10. The method of claim 1, wherein, The method further comprises: In a case where the target data is located in the physical cache space, and a first data amount of the to-be-processed data and the processing result is greater than a second data amount of the target data, creating a new memory page and allocating a cache area corresponding to the new memory page in the physical cache space; storing a second data amount of data in the to-be-processed data and the processing result to a location where the target data is located, and storing other data in the to-be-processed data and the processing result to a cache area corresponding to the new memory page.

11. A data processing method, comprising: In response to the data processing instruction, the client sends the to-be-processed data to the server, so that the server generates a processing result of the to-be-processed data, the processing result is generated according to target data stored in a target cache area of a physical cache space, the target data is stored in the target cache area from a physical persistent storage space after adding corresponding memory pages to a memory page set of a virtual storage space, the target data is obtained from the physical persistent storage space in a case where a similarity between a first semantic vector of the to-be-processed data and at least one second semantic vector in a mapping table is less than a similarity threshold, the at least one second semantic vector corresponds to at least one memory page in the memory page set in a one-to-one manner, and any second semantic vector is a semantic vector of data stored in a cache area associated with the physical cache space by the corresponding memory page; displaying the processing result sent by the server.

12. A data processing system comprising: The client is configured to execute the method in claim 11, and the server is configured to execute the method in any one of claims 1 to 10. 13.An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements the method in any one of claims 1 to 11 when executing the computer program. 14.A computer-readable storage medium having a computer program stored therein, wherein the computer program implements the method in any one of claims 1 to 11 when executed by a processor. 15.A computer program product comprising a computer program, wherein the computer program implements the method in any one of claims 1 to 11 when executed by a processor.

Images