Data reading method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a data reading method and device, electronic equipment and storage medium, and belong to the technical field of data caching. The method comprises: obtaining an access request for target data; in response to the access request, when a target index address of the target data exists in a preset buffer pool, obtaining target description data of the target data based on the target index address pointing to a corresponding position in a preset data pool, and returning the target description data, the buffer pool storing index addresses of multiple data, and the multiple index addresses being arranged in a sorting order, and the data pool storing description data corresponding to each index address; when the target index address is located in a hot data area in the buffer pool, or the target index address is located in a cold data area in the buffer pool and an existing time meets a preset condition, the target index address is transferred to a head position of the hot data area to update the sorting order of the index addresses in the buffer pool. The present application can improve the reading efficiency of data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data caching, and in particular to a data reading method and device, electronic equipment and a storage medium. BACKGROUND

[0002] In a computer device, data reading includes a process in which a central processing unit (CPU) obtains data from a storage medium (such as a disk, memory, etc.) and loads the data into memory.

[0003] Generally, a cache is also provided in the computer device. In the related art, the data read is placed in the cache, and since the cache can exchange data with the CPU prior to the storage medium, the data reading operation is facilitated. However, such a data reading method requires constant addition, transfer or elimination operations of data pages, especially in the case of increasing data volume, which increases the load of the CPU and thus reduces the data reading efficiency. SUMMARY

[0004] The main purpose of the embodiments of the present application is to provide a data reading method, device, electronic equipment and storage medium, which can improve the data reading efficiency.

[0005] To achieve the above purpose, a first aspect of the embodiments of the present application provides a data reading method, which comprises:

[0006] obtaining an access request for target data;

[0007] In response to the access request, when a target index address of the target data exists in a preset buffer pool, obtaining target description data of the target data based on the target index address pointing to a corresponding position in a preset data pool, and returning the target description data, wherein the buffer pool stores a plurality of index addresses of data, and the plurality of index addresses are arranged in an address sorting order, and the data pool stores description data corresponding to each index address;

[0008] When the target index address is located in a hot data area in the buffer pool, or the target index address is located in a cold data area in the buffer pool and the existing time meets a preset condition, the target index address is transferred to a head position of the hot data area to update the sorting order of the index addresses in the buffer pool.

[0009] In some embodiments, each index address in the data pool is provided with corresponding active information, and the active information is used to indicate the probability of the corresponding data corresponding to the index address being read;

[0010] After returning the target description data, the method further comprises:

[0011] when the target index address is located in the cold data area of the buffer pool and the existence time meets the preset condition, determining corresponding access control information and change time information according to the target description data;

[0012] determining target active information corresponding to the target index address, calculating a first ratio between the access control information and the target active information, calculating a second ratio between the change time information and the target active information, and updating the target active information according to the first ratio and the second ratio;

[0013] According to the ordering sequence of the updated target active information and other active information, the target index address is transferred to a first position of the hot data area.

[0014] In some embodiments, after returning the target description data, further comprising:

[0015] when the target index address is located in the cold data area of the buffer pool and the existence time meets the preset condition, determining corresponding access control information and change time information according to the target description data;

[0016] determining target active information corresponding to the target index address, calculating a first ratio between the access control information and the target active information, calculating a second ratio between the change time information and the target active information, and updating the target active information according to the first ratio and the second ratio;

[0017] According to the ordering sequence of the updated target active information and other active information, the target index address is transferred to a first position of the hot data area.

[0018] In some embodiments, after obtaining the access request for the target data, further comprising:

[0019] In response to the access request, when the target index address of the target data does not exist in the preset buffer pool, obtaining target description data of the target data based on the target index address pointing to a corresponding position in the storage medium, and returning the target description data, wherein the storage medium stores addresses of a plurality of data;

[0020] determining a space occupation value of the target data, and calculating a numerical difference between the space occupation value and a value of each free position in a data pool, wherein each free position in the data pool includes a corresponding free position value;

[0021] If there is a free position with a numerical difference less than a preset space threshold, the target index address and the target description data are added to the free position;

[0022] determining target active information according to the target description data, and adding the target index address to a third position of the cold data area according to the target active information.

[0023] In some embodiments, determining target active information according to the target description data comprises:

[0024] determining data attribute information and prediction access information of the target data;

[0025] calculating a first sub-value according to the data attribute information and a preset first weight value, and calculating a second sub-value according to the prediction access information and a preset second weight value;

[0026] performing addition processing on the first sub-value and the second sub-value to obtain target active information corresponding to the target data.

[0027] In some embodiments, after calculating the numerical difference between the space occupancy value and the value of each free position in the data pool, further comprising:

[0028] If there is no free position with a numerical difference less than a preset space threshold, moving any target description data between the two adjacent free positions to one of the free positions;

[0029] until the two free positions are merged, obtaining an updated free position.

[0030] In some embodiments, after determining the target active information according to the target description data and adding the target index address to the third position of the cold data area according to the target active information, further comprising:

[0031] determining the index address in the last region of the cold data area and the description data corresponding to the index address in the data pool.

[0032] To achieve the above-mentioned purposes, a second aspect of the embodiments of the present application proposes a data reading device, the device comprising:

[0033] an acquisition module configured to acquire an access request for target data;

[0034] a data reading module configured to, in response to the access request, when there is a target index address of the target data in a preset buffer pool, acquire target description data of the target data based on the target index address pointing to a corresponding position in a preset data pool, and return the target description data, wherein the buffer pool stores a plurality of index addresses of data, and the plurality of index addresses are arranged in a sorting order, and the data pool stores description data corresponding to each index address;

[0035] an updating module configured to, when the target index address is located in a hot data area of the buffer pool, or the target index address is located in a cold data area of the buffer pool and the existing time meets a preset condition, move the target index address to a head position of the hot data area to update the sorting order of the index addresses in the buffer pool.

[0036] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the data reading method of the first aspect when executing the computer program.

[0037] To achieve the above object, a fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the data reading method of the first aspect.

[0038] The data reading method, device, electronic device and storage medium provided by the present application, the method comprises the following steps: obtaining an access request for target data; in response to the access request, when a target index address of the target data exists in a preset buffer pool, obtaining target description data of the target data based on the target index address pointing to a corresponding position in a preset data pool, and returning the target description data, wherein the buffer pool stores index addresses of a plurality of data, and the plurality of index addresses are arranged in a sorting order, and the data pool stores description data corresponding to each index address; when the target index address is located in a hot data area in the buffer pool, or the target index address is located in a cold data area in the buffer pool and the existing time meets a preset condition, the target index address is transferred to a head position of the hot data area to update the sorting order of the index addresses in the buffer pool. In the embodiments of the present application, only index addresses are stored in the cache pool. Since the data content of the index address is less and the occupied space is small, the CPU can easily find the corresponding index address from the buffer pool, and at the same time, the update of the positions of the index addresses in the buffer pool can be realized based on less computing resources, so that the efficiency of subsequent data reading is improved on the basis of ensuring the efficiency of current data reading. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is a device structure schematic diagram of the data reading device provided by the embodiments of the present application;

[0040] Figure 2 is an optional flowchart of the data reading method provided by the embodiments of the present application;

[0041] Figure 3 is an optional data pool schematic diagram of the data reading method provided by the embodiments of the present application;

[0042] Figure 4 is an optional functional module schematic diagram of the data reading device provided by the embodiments of the present application;

[0043] Figure 5 is a hardware structure schematic diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0044] In order to make the purposes, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0045] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical sequence is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the sequence in the flowchart. The terms "first", "second", etc. in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0047] First, the meanings of several terms involved in the present application are analyzed:

[0048] Uniform Resource Locator (URL), URL specifies the location of the resource that the client program wants to access, which contains protocol (such as HTTP or HTTPS), host name, path, etc. information, and is used to uniquely identify and locate resources.

[0049] Request Method, Request Method specifies the operation that the client wants the server to perform on the resource. Common request methods include: resource acquisition method (GET method), when using the GET method, the client requests to acquire the representation of the resource specified by the target URL, and the server returns the resource as the response body; resource acquisition method (POST method), submit data to the server to create a new resource or perform some operation, when using the POST method, the client sends the data to the target URL in the request body, and the server performs the corresponding operation according to the content of the request body and returns the result.

[0050] Application Programming Interface (API) is a set of rules and conventions that define the interaction between software components. It allows different software systems to communicate and interact with each other, so that they can be connected, share data and functions. API includes various types, such as RESTful API and Simple Object Access Protocol Interface (SOAP).

[0051] The intelligent toilet is a sanitary appliance integrating intelligent technology and high-tech equipment, aiming to provide a more comfortable, convenient and sanitary use experience.

[0052] In a computer device, data reading includes the process of a central processing unit program obtaining data from a storage medium (such as a disk, memory, etc.) and loading it into memory.

[0053] Generally, a cache is also provided in the computer device. In the related art, the read data is placed in the cache due to the cache location advantage and high hit rate, etc. Since the cache can exchange data with the CPU prior to the storage medium, it facilitates subsequent data reading operations. However, such a data reading method needs to constantly add, transfer or eliminate data pages, especially in the case of increasing data volume, which increases the load of the CPU, and thus reduces the data reading efficiency.

[0054] Therefore, the present application provides a data reading method, device, electronic device and storage medium, which can improve the data reading efficiency. First, the data reading device in the embodiment of the present application is introduced, as shown in Figure 1 Figure 1 is a device structure diagram of the data reading device provided by the embodiment of the present application. The data reading device (for convenience of description, hereinafter can be referred to as "device") is provided with a buffer pool and a data pool. When the data in the storage medium is read, the data is written into the buffer pool of the device. The buffer pool stores the index address corresponding to the data, and the data pool stores the index address and the corresponding description data. When a target data such as description data 10 is to be read, the buffer pool is first traversed. If the index address 10 corresponding to the description data 10 can be found in the buffer pool, the corresponding description data 10 is directly found from the data pool according to the index address 10. If the corresponding index address 10 is not found in the buffer pool, the description data 10 is read from the storage medium. It can be understood that only the index address corresponding to the data is stored in the buffer pool. When the corresponding index address is read, the corresponding data is found from the data pool based on the index address. In this way, it is not necessary to search for data from the storage medium with large data volume every time when data is read, and the storage space occupied by the index address stored in the buffer pool is small. Therefore, the data reading speed is greatly improved.

[0055] ​It needs to be first explained that in the embodiments of the present application, when it is necessary to read data from the buffer pool, the data pool or the storage medium, the user's permission or consent will be obtained first, and the collection, use and processing of the read data will comply with relevant laws, regulations and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the separate permission or separate consent of the user will be obtained first, and after obtaining the separate permission or separate consent of the user, the necessary target description data for enabling the embodiments of the present application to normally operate is obtained, for example, in the embodiments of the present application, the user-related account information needs to be read from the data pool, and before obtaining the access request for the target data, the consent of the relevant user will be obtained first. Only when the user's permission or consent is obtained, the target description data required by the embodiments of the present application will be obtained.

[0056] Next, the data reading method in the embodiments of the present application is described in detail through the following embodiments.

[0057] In the embodiments of the present application, the data reading device is described from the dimension of the data reading device. The data reading device can be integrated in a computer device, such as a server. As shown in FIG. 1, Figure 2 Figure 2 is an optional flowchart of the data reading method provided by the embodiments of the present application, Figure 2 The method in the embodiments of the present application can include but is not limited to the following steps 101 to 103, wherein the specific process of the data reading device when executing the data reading method is as follows. It needs to be first explained that the order of steps 101 to 103 in the embodiments is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or increased according to actual needs. Figure 2

[0058] Step 101, obtaining an access request for target data.

[0059] The following describes step 101 in detail.

[0060] Among them, the CPU will first obtain the access request from the program, and the access request indicates the target data to be read. The access request is used to indicate the identifier, index address and type, size and structure of the data to be read. The data operation type indicated by the access request can be reading, writing, updating or deleting. It can be understood that no matter which type, the target data needs to be read first, and then the subsequent operation is performed. Therefore, the embodiments of the present application need to obtain the access request first, so as to perform the data reading operation on the target data based on the access request.

[0061] ​​The access request can be in different formats according to different computer or server hardware devices actually operated or different interface requirements indicated by programs. For example, the access request can be in a HyperText Transfer Protocol (HTTP) request format, an HTTP request usually includes a request method (such as GET, POST), a target URL, a request header, and an optional request body; a Structured Query Language (SQL) format, an SQL query format usually includes a SELECT statement and a condition for specifying the target data required for the query; an API call format, the API call format depends on the API and protocol used, which can be a RESTful API, SOAP, etc.; a Remote Procedure Call (RPC) format, in a distributed system, a Remote Procedure Call (RPC) can be used to request access to the target data. The RPC call format depends on the RPC framework and protocol used.

[0062] The target data can be various types of information or resources, and its specific form depends on the application scenario and actual requirements. Common target data types include: text data, such as articles, documents, logs, chat records, and other text-formatted data; image data, such as photos, charts, paintings, and other image-formatted data; audio data, such as music, voice recordings, sound effects, and other sound-formatted data; video data, such as movies, TV shows, videos, and other video-formatted data, etc. According to the data type, storage method, and application scenario of the data, a unique index address in the storage medium can be generated, and the index address is written into the buffer pool, and the description data and the corresponding index address are written into the data pool.

[0063] Further, data is usually stored in units of data pages, and when a certain description data is accessed, the access probability of other data in the data page where the description data is located is high. Therefore, the target index address corresponding to the target data can be the index address of the data page, and at this time, the data stored in the data pool includes all the data on the data page.

[0064] The storage medium is set in the computer or server, and the storage medium can be a hard disk, a solid state disk, a memory chip, or other hardware devices or materials capable of storing various types of data, for storing all data related to the program. Compared with the buffer pool and the data pool, the storage medium has larger storage capacity, slower read-write speed, and longer data storage persistence. Therefore, while ensuring sufficient space to store the required data, the use of the buffer pool and the data pool can improve the data read efficiency.

[0065] In response to the access request, when the target index address of the target data exists in the preset buffer pool, the target description data of the target data is acquired based on the target index address pointing to the corresponding position in the preset data pool, and the target description data is returned, wherein the buffer pool stores index addresses of a plurality of data, and the plurality of index addresses are arranged in a sorting order, and the data pool stores description data corresponding to each index address.

[0066] The step 102 is described in detail below.

[0067] For the convenience of data addressing and access, the data stored in the storage medium includes two aspects of address and specific content (description data). When the data in the storage medium is accessed, the data is referred to as target data, the address corresponding to the target data is referred to as target address, and the specific content of the target data is referred to as target description data.

[0068] The buffer pool and the data pool are arranged on a medium with faster reading speed, such as a cache. Since the cache is usually located at a position close to the physical distance between the data user (such as CPU) and the data source (such as memory or disk), and the reading speed of the medium used by the cache is usually faster, the speed of reading data from the cache is usually faster than that of reading data from the storage medium.

[0069] Further, the buffer pool and the data pool jointly form a high-speed storage area, and the target index address corresponding to the target data is stored in the buffer pool, and the target description data and the corresponding target index address are stored in the data pool. The target index address in the buffer pool corresponds to the target index address in the data pool, that is, the target index address in the data pool can be found through the target index address in the buffer pool. For example, the target index address of a certain target data A is address A, and the target description data is description data A. When the target data A is successfully accessed, address A needs to be stored in the buffer pool, and (address A + description data A) needs to be stored in the data pool. It can be understood that the reading of data involves the updating of a plurality of information data in the cache. Compared with storing the specific content and the corresponding address of the data in the cache area, only the index address is stored in the buffer pool in the embodiment of the application. Since the data content of the index address is less and occupies a small space, the CPU can easily find the corresponding index address from the buffer pool, and at the same time, the updating of the positions of the index addresses in the buffer pool can be realized based on less computing resources. In this way, the efficiency of subsequent data reading is improved on the basis of ensuring the efficiency of current data reading.

[0070] The index addresses in the buffer pool are sorted in a certain order, for example, the index addresses can be sorted in ascending order of the addresses. Since the addresses of adjacent data are also adjacent, sorting the index addresses in order can help the CPU hit the required read data in a shorter time, thereby improving the data read efficiency.

[0071] In step 103, when the target index address is located in the hot data area in the buffer pool, or the target index address is located in the cold data area of the buffer pool and the existing time meets the preset condition, the target index address is moved to the head position of the hot data area to update the sorting order of the index addresses in the buffer pool.

[0072] The step 103 is described in detail below.

[0073] The buffer pool includes two areas, one is a hot data area for storing data with a high access frequency, and the other is a cold data area for storing data with a low access frequency. In order to ensure the accuracy of data access, the device will update the arrangement order of the target index address in the buffer pool at a preset interval. If the target data is read from the hot data area, it means that the access frequency of the target data itself is relatively high. On this basis, the CPU performs data access again, which means that the probability of subsequent CPU access is still relatively high. Therefore, the target index address corresponding to the target data is moved to the head position of the hot data area to speed up the data read efficiency of the CPU.

[0074] Further, if the target index address is read from the cold data area, it means that the access probability of the corresponding target data is not high. On this basis, the CPU accesses the target index address, and the existence time of the target index address in the cold data area is determined to determine the movement of the target index address. If the existence time is greater than the preset time, it means that the CPU may have read the data in the data page where the target data is located for multiple times, resulting in a longer residence time of the target index address in the cold data area. Therefore, the target index address can be moved to the head position of the hot data area to improve the data read efficiency. If the existence time is not greater than the preset time, it means that the CPU has only read one or several data in the data page. Therefore, the residence time of the data page is short, which means that the probability of subsequent CPU access to other data in the data page is not high. Therefore, no operation is performed on the target index address.

[0075] Exemplarily, the preset time can be 1s. Since a data page usually has many contents accessible within 1s, if the residence time of the target index address in the cold data area exceeds 1s, it indicates that the CPU has accessed the same data or different data of the data page for multiple times. At this time, the target index address can be moved to the head position of the hot data area, so as to avoid a large number of new target index addresses from being loaded into the cold data area, thereby avoiding the frequently accessed data from being evicted.

[0076] It should be noted that the preset time can be set according to actual conditions, and the embodiments of the present application do not make specific limitations.

[0077] It can be understood that, by setting the double cache area, the embodiments of the present application exchange time for space. In this way, the target index data stored in the buffer pool corresponds to the frequently accessed data, and is further divided into more frequently accessed (hot data area) and less frequently accessed (cold data area). When the target data corresponding to the target index address is accessed, the position of the target index address is further adjusted according to the position and existence time of the target index address, so as to improve the hit rate of the target data when the CPU sequentially traverses the buffer pool, thereby accelerating the data reading efficiency.

[0078] In some embodiments, after the target description data is returned, the following steps 201 to 203 are further included:

[0079] Step 201, when the target index address is located in the cold data area of the buffer pool and the existence time satisfies the preset condition, the corresponding access control information and change time information are determined according to the target description data.

[0080] The step 201 is described in detail as follows.

[0081] In some embodiments, in order to further improve the data reading efficiency, each index address in the data pool is further provided with corresponding active information, and the active information is used to indicate the probability of the corresponding data being read.

[0082] In some embodiments, when the target index address is located in the cold data area of the buffer pool and the existence time satisfies the preset condition, in addition to moving the target index address to the head position of the hot data area, the moving position of the target index address can also be determined according to the active information corresponding to each target data. The active information is obtained by pre-setting. When the CPU accesses a target data, the active information corresponding to the target data can be updated according to the target description information of the target data. Then, the moving position of the target index address is determined according to the size of the updated active information and other active information in the hot data area.

[0083] Further, the access control information is used to indicate the access right of the target data, the access right is higher, and the access control information score is lower, indicating that the target data is accessed with a high probability of not many times; the change time information is used to indicate the last access timestamp of the target data, and the change time information score is higher when the current timestamp and the last access timestamp are closer, indicating that the target data is accessed with a higher frequency.

[0084] In step 202, the target active information corresponding to the target index address is determined, the first ratio between the access control information and the target active information is calculated, the second ratio between the change time information and the target active information is calculated, and the target active information is updated according to the first ratio and the second ratio.

[0085] The step 202 is described in detail below.

[0086] In some embodiments, the active information is updated by using the determined access control information and change time information. Specifically, the ratio of the access control information to the target active information can be taken as the first ratio, the ratio of the change time information to the target active information can be taken as the second ratio, and the updated target active information can be obtained according to the sum of the original target active information, the first ratio and the second ratio.

[0087] Further, different weight values can be set for the first ratio and the second ratio, and the updated first ratio and the second ratio can be calculated under the constraint of the weight values, and then the updated target active information is obtained. The specific value of the weight value can be set according to the actual situation, and the embodiments of the present application do not make specific restrictions.

[0088] In step 203, the target index address is moved to the first position of the hot data area according to the sorting order of the updated target active information and other active information.

[0089] Further, after obtaining the updated target active information, the size of the target active information and other active information in the hot data area is compared to determine the first position of the target index address in the hot data area after moving. The first position can be the head position of the hot data area, or it can be other positions, which needs to be determined according to the actual value of the target active information.

[0090] Or, when the active information represents an active level, the corresponding target index address can be placed in the head position of the corresponding active level, so that the moving position of the target index address is determined after traversing all index addresses in the buffer pool, thereby improving the efficiency of the sorting order of the index addresses in the buffer pool. For example, the hot data area includes three active information levels A1, B1 and C1, and the cold data area includes three active information levels A2, B2 and C2. When the residence time of the target data located in the cold data area A2 is greater than 1s, the target data is moved to the head position of the hot data area A1.

[0091] In some embodiments, after returning the target description data, the following steps 301 to 303 are further included:

[0092] Step 301, when the target index address is located in the cold data area of the buffer pool and the existence time does not meet the preset condition, determining the corresponding data quality information according to the target description data.

[0093] The following describes step 301 in detail.

[0094] In some embodiments, if the target index address is located in the cold data area of the buffer pool and the existence time does not meet the preset condition, in addition to not doing anything to the target index address, the access probability of the target data in the future can also be predicted in the case of setting active information. To achieve this purpose, simple data quality information can be determined according to the target description data, so as to simply predict the read probability of the target data in the future according to the data quality information.

[0095] Further, the data quality information is used to indicate the quality of the target data. For example, the data quality information is high when the data proportion is large, indicating that the data content is complete and rich. Since the data with small data proportion usually indicates the core content of the data, the CPU tends to access the data with smaller data proportion, that is, the data quality information with high data quality information has a lower data quality information score, indicating that the probability of subsequent access is not high.

[0096] Step 302, determining the target active information corresponding to the target index address, calculating a third ratio between the data quality information and the target active information, and updating the target active information according to the third ratio.

[0097] The following describes step 302 in detail.

[0098] Further, the determined data quality information is used to update the active information. Specifically, the ratio of the data quality information can be taken as the third ratio, and the updated target active information can be obtained according to the sum of the original target active information and the third ratio.

[0099] Further, a corresponding weight value can be set for the third ratio, and the updated third ratio can be calculated under the constraint of the weight value, and then the updated target active information is obtained. The specific value of the weight value can be set according to actual conditions, and the embodiments of the present application do not make specific limitations.

[0100] In step 303, the target index address is moved to the second position of the hot data area according to the ranking order of the updated target active information and other active information.

[0101] The step 303 is described in detail below.

[0102] Further, after obtaining the updated target active information, the size of the target active information and the other active information in the hot data area is compared to determine the second position of the target index address in the hot data area after moving. The second position can be the head position of the hot data area, or it can be other positions in the hot data area. The actual value of the target active information needs to be determined.

[0103] Further, such a target index address moving method can be applied to a scenario where the access request interval is long, for example, a smart toilet. The interval between the time when the CPU receives the instruction issued by the user is long. At this time, the target index address position in the buffer pool can be updated according to the target description information to improve the update accuracy of the target index address position in the buffer pool, so as to improve the data reading efficiency when the next data access comes.

[0104] Further, in actual conditions, the target index address moving method can be selected according to specific conditions. One with higher data reading efficiency can be selected, or the two target index address moving methods can be used alternately to improve the overall data reading efficiency.

[0105] It can be understood that if the currently accessed data is in the cold data area, in addition to determining whether to place the corresponding index address to the head position of the hot data area according to the existence time, the index address position in the hot data area can also be determined according to the active information determined in advance and updated in real time. In this way, the subsequent situation that the data is no longer accessed is avoided, and the position is frequently traversed, or the excessive resource waste caused by copying and pasting each index address due to the update of the buffer pool. Based on the compromise solution, the data originally in the hot data area is always in the forefront of the traversal, and the newly accessed data is in a reasonable position in the hot data area. If the data is no longer accessed in the future, it can be eliminated at a faster speed. If the data is still accessed in the future, it can also be hit at a faster speed.

[0106] In some embodiments, after obtaining the access request for the target data, the following steps 401-404 are further included:

[0107] Step 401, in response to the access request, when the target index address of the target data does not exist in the preset buffer pool, the target description data of the target data is obtained based on the target index address pointing to the corresponding location in the storage medium, and the target description data is returned, wherein the storage medium stores addresses of a plurality of data.

[0108] Step 402, determine the space occupation value of the target data, and calculate the numerical difference between the space occupation value and the value of each free location in the data pool, wherein each free location in the data pool includes a corresponding free location value.

[0109] Step 403, if there is a free location with a numerical difference less than a preset space threshold, the target index address and the target description data are added to the free location.

[0110] Step 404, determine the target active information according to the target description data, and add the target index address to the third location of the cold data area according to the target active information.

[0111] The steps 401-404 are described in detail below.

[0112] In some embodiments, if the target index address corresponding to the target data to be read is not in the buffer pool, the target data is read from the storage medium storing complete data according to the target index address, and the target index address corresponding to the target data is stored in the buffer pool. The target index address corresponding to the target data and the target description data are stored in the data pool.

[0113] Further, the target index addresses in the buffer pool are usually arranged in order of address size or according to active information, and the target index addresses in the data pool have a mapping relationship with the target index addresses in the buffer pool, that is, the target index addresses in the data pool do not need to be arranged in order of size. Such an arrangement causes the free locations left after deleting the corresponding target index address and target description data in the data pool when the target index address in the buffer pool is deleted are often not continuous. Therefore, after completing the insertion of the target index address in the buffer pool, the space occupation value of the target data needs to be determined in order to determine the addition position of the target data in the data pool, wherein the space occupation value is determined according to the size of the target index address and the size of the target description data.

[0114] Further, the space occupation value is compared with the free position value corresponding to the free position in the data pool, to obtain a numerical difference between the two, to determine that the free position corresponding to the numerical difference less than and less than the preset space threshold is satisfied, and the target index address and the target description data are stored in the free position.

[0115] Further, if the target index address corresponding to the target data is not in the buffer pool before the current access, the target index address can be stored in the head position of the cold data area after the data access is completed, or the target active information corresponding to the target data is first determined, and then the target index address corresponding to the target data is determined in the third position in the buffer pool according to the size relationship between the target active information and other active information in the buffer pool, wherein the third position can be the head position of the cold data area, or can be other positions of the cold data area. The actual value of the target active information needs to be determined.

[0116] In some embodiments, after calculating the numerical difference between the space occupation value and the value of each free position in the data pool, the following steps 501 to step 502 are further included:

[0117] Step 501, if there is no free position whose numerical difference is less than the preset space threshold, any target description data between the two adjacent free positions is moved to one of the free positions.

[0118] Step 502, until the two free positions are combined, to obtain the updated free position.

[0119] The steps 501 to step 502 are described in detail below.

[0120] In some embodiments, if there is no free position whose numerical difference is less than the preset space threshold, for example, as shown in Figure 3 , the target description data between the two adjacent free positions is moved to one of the free positions. Figure 3 is an optional data pool schematic diagram of the data reading method provided by the embodiments of the present application. The occupation space of the target data B (target index address B+target description data B) to be stored at present is greater than the free position 1, the free position 2 and the free position 3, that is, the space of each free position in the current data pool is too small to meet the storage requirement of the target data B, therefore, the remaining free positions need to be combined through fragmentation arrangement to meet the storage requirement of the new target data.

[0121] Exemplarily, as shown in Figure 3As shown, (index address 10 + data 10) between the idle location 1 and the idle location 2 can be moved to the direction of the idle location 2, so that the position of the idle location 1 is constantly increased until the updated idle location 1 = idle location 1 + idle location 2, and the updated idle location is obtained. Wherein, the target data stored in the data pool can only contain target description data, in which case the target index address stored in the buffer pool indicates the corresponding target description data in the data pool by means of a pointer.

[0122] In some embodiments, the target active information is determined according to the target description data, including the following steps 601 to 603:

[0123] Step 601, determine the data attribute information and the predicted access information of the target data.

[0124] Step 602, calculate a first sub-value according to the data attribute information and a preset first weight value, and calculate a second sub-value according to the predicted access information and a preset second weight value.

[0125] Step 603, add the first sub-value and the second sub-value to obtain the target active information corresponding to the target data.

[0126] The steps 601 to 603 are described in detail below.

[0127] Wherein, the active information is set before the data is stored in the high-speed storage area, and the active information is set based on comprehensive consideration of multiple dimensions, and the active information corresponding to the target data is called target active information. In the embodiment of the application, the setting of the active information considers two aspects of data attribute information and predicted access information, wherein the data attribute information is used to indicate the characteristics of the data itself, to represent the importance of the data itself; the predicted access information is used to indicate the probability that the data will be accessed in the future.

[0128] Further, the active information can be a specific active value, or an active level. The data attribute information and the predicted access information can both be represented by scores, and the data attribute score and the predicted access score are determined according to the description data corresponding to the data. For example, a batch of data includes core data and edge data, the core data can be the user's name, gender, etc., and the edge data can be the user's note information, then it can be determined that the data attribute score corresponding to the core data is greater than the edge data; and according to experience, the access frequency of the core data is higher than that of the edge data, so the predicted access score of the core data is greater than that of the edge data.

[0129] Further, the data attribute score can also be determined according to the data type, data source, data destination, etc., and the predicted access score can also be determined according to the historical access times and the last access time, that is, the specific contents of the data attribute information and the predicted access information can be set according to the actual situation, and the embodiments of the present application do not make specific limitations thereon.

[0130] Further, the first weight value corresponds to the data attribute information, and the second weight value corresponds to the predicted access information, and the first weight value can be set to be higher than the second weight value, which indicates that in the process of determining the active information, the characteristics of the data itself are more important than the predicted results obtained by experience judgment.

[0131] It should be noted that the first weight value and the second weight value can be set according to the actual situation, and the second weight value can also be higher than the first weight value, so as to emphasize the importance of the predicted results under experience judgment, and the embodiments of the present application do not make specific limitations thereon.

[0132] Further, the embodiments of the present application obtain the active information by weighting calculation of the features corresponding to the description data from multiple dimensions, and when the data is accessed and stored in the buffer pool, the data can be sorted according to the size of the active information. In this way, the data with higher access probability can be arranged in front of the data with lower access probability, and when the next access request arrives, the CPU can find the corresponding target data with a higher hit rate.

[0133] In some embodiments, after the target active information is updated according to the target description data, and the target index address is added to the third position of the cold data area according to the target active information, the following steps are further included:

[0134] The index address in the last region of the cold data area is determined, and the index address corresponding to the description data in the data pool.

[0135] In some embodiments, the space of the buffer pool is usually limited to avoid unlimited growth of data, and when the data in the buffer pool is full and new index addresses need to be added, the index addresses that are not frequently accessed need to be deleted from the buffer pool. The last region of the cold data area of the buffer pool is set as the data elimination area, and the data capacity of the last region can be one data or multiple data, and when the index addresses in the buffer pool are updated at a predetermined interval, all index data in the last region are deleted. And the index addresses corresponding to the data pool and the corresponding description data are deleted, so as to store data with higher access frequency.

[0136] Further, the active information is set in advance when data enters the buffer area, which is deeply considered from multiple aspects, while the update of the active information after the data is accessed is some simple information directly obtained according to the description data corresponding to the accessed data, and the update of the active information is based on the real-time obtained information, which are two different aspects.

[0137] It should be noted that the amount of data that can be accommodated by the last region can be set according to actual conditions, and the embodiments of the present application do not make specific limitations thereto.

[0138] As shown in Figure 4 , Fig. 1 is a schematic diagram of an optional function module of a data reading device provided by an embodiment of the present application, wherein the data reading device can include: Figure 4 The obtaining module 801 is configured to obtain an access request for target data.

[0139] The data reading module 802 is configured to, in response to the access request, when a target index address of the target data exists in the preset buffer pool, acquire target description data of the target data based on the target index address pointing to a corresponding position in the preset data pool, and return the target description data, wherein the buffer pool stores index addresses of multiple data, and the multiple index addresses are arranged in a sorting order, and the data pool stores description data corresponding to each index address.

[0140] The updating module 803 is configured to, when the target index address is located in a hot data area of the buffer pool, or the target index address is located in a cold data area of the buffer pool and the existing time meets a preset condition, transfer the target index address to a head position of the hot data area, so as to update the sorting order of the index addresses in the buffer pool.

[0141]

[0142] ​The data reading method, device, electronic device and storage medium provided in the application, the method comprises the following steps: obtaining an access request for target data; in response to the access request, when a target index address of the target data exists in a preset buffer pool, obtaining target description data of the target data based on the target index address pointing to a corresponding position in a preset data pool, and returning the target description data, wherein the buffer pool stores index addresses of multiple data, and the multiple index addresses are arranged in a sorting order, and the data pool stores description data corresponding to each index address; when the target index address is located in a hot data area in the buffer pool, or the target index address is located in a cold data area in the buffer pool and the existing time meets a preset condition, the target index address is transferred to a head position of the hot data area to update the sorting order of the index addresses in the buffer pool. In the embodiment of the application, only index addresses are stored in the cache pool. Since the data content of the index address is less and the occupied space is small, the CPU can easily find the corresponding index address from the buffer pool, and at the same time, the update of the positions of the index addresses in the buffer pool can be realized based on less computing resources, so that the efficiency of subsequent data reading is improved on the basis of ensuring the efficiency of current data reading.

[0143] The specific implementation of the data reading device is basically the same as the specific embodiment of the above-mentioned data reading method, and will not be repeated here.

[0144] The embodiment of the application further provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned data reading method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer and the like.

[0145] As shown in Figure 5 , the hardware structure of the electronic device provided in the embodiment of the application is shown in Figure 5 , which comprises:

[0146] The processor 901 can be realized in the mode of a general CPU (Central Processing Unit, central processor), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute related programs to realize the technical solutions provided in the embodiment of the application;

[0147] The memory 902 can be implemented in the form of a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a Random Access Memory (RAM), etc. The memory 902 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 902 and are called and executed by the processor 901 to perform the data reading method of the embodiments of the present application.

[0148] The input / output interface 903 is configured to realize information input and output.

[0149] The communication interface 904 is configured to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, a WI-FI, a Bluetooth, etc.).

[0150] The bus 905 is configured to transmit information between various components (for example, the processor 901, the memory 902, the input / output interface 903, and the communication interface 904) of the device.

[0151] The processor 901, the memory 902, the input / output interface 903, and the communication interface 904 are connected to each other through the bus 905 to realize the communication connection between the device.

[0152] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the above-mentioned data reading method.

[0153] The memory is a non-transitory computer readable storage medium, which can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory and can also include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0154] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0155] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation to the embodiments of the present application, and can include more or fewer steps than the figures, or combine certain steps, or different steps.

[0156] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, that is, can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0157] Those skilled in the art can understand that all or some steps in the above disclosed method, functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0158] The terms "first", "second", "third", "fourth" and the like in the description of the present application and the above-mentioned figures (if any) are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0159] It should be understood that in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the association between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0160] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the above units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0161] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0162] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0163] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that makes a contribution or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0164] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, but this does not limit the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A data reading method, characterized in that: The method comprises: Obtain access requests for target data; In response to the access request, when the target index address of the target data exists in the preset buffer pool, target description data of the target data is obtained based on the corresponding position in the preset data pool pointed to by the target index address, and the target description data is returned, wherein the buffer pool stores the index addresses of multiple data, and the multiple index addresses are arranged in a sorted order, and the data pool stores the description data corresponding to each index address; When the target index address is located in a hot data area in the buffer pool, or when the target index address is located in a cold data area in the buffer pool and the existence time satisfies a preset condition, the target index address is transferred to the head position of the hot data area to update the sorting order of the index addresses in the buffer pool; Each of the index addresses in the data pool is provided with corresponding activity information, where the activity information is used to indicate the probability that the data corresponding to the corresponding index address is read; After returning the target description data, the method further includes: When the target index address is located in the cold data area of ​​the buffer pool and the existence time meets the preset conditions, determining the corresponding access control information and change time information according to the target description data; determining target activity information corresponding to the target index address, calculating a first ratio between the access control information and the target activity information, calculating a second ratio between the change time information and the target activity information, and updating the target activity information according to the first ratio and the second ratio; Transferring the target index address to the first position of the hot data area according to the updated sorting order of the target active information and other active information; After obtaining the access request for the target data, the method further includes: In response to the access request, when the target index address of the target data does not exist in a preset buffer pool, obtaining target description data of the target data based on a corresponding position in a storage medium pointed to by the target index address, and returning the target description data, wherein the storage medium stores addresses of multiple data; Determining a space occupation value of the target data, and calculating a numerical difference between the space occupation value and each free position value in the data pool, wherein each free position in the data pool includes a corresponding free position value; If there is an idle position where the numerical difference is less than a preset space threshold, adding the target index address and the target description data to the idle position; The target activity information is determined according to the target description data, and the target index address is added to the third position of the cold data area according to the target activity information.

2. The method according to claim 1, characterized in that After returning the target description data, the method further includes: When the target index address is located in the cold data area of ​​the buffer pool and the existence time does not meet the preset condition, determining corresponding data quality information according to the target description data; determining the target activity information corresponding to the target index address, calculating a third ratio between the data quality information and the target activity information, and updating the target activity information according to the third ratio; The target index address is transferred to a second position of the hot data area according to the updated sorting order of the target active information and other active information.

3. The method according to claim 1, characterized in that The determining the target activity information according to the target description data includes: determining data attribute information and predicted access information of the target data; A first sub-value is calculated based on the data attribute information and a preset first weight value, and a second sub-value is calculated based on the predicted access information and a preset second weight value; The first sub-value and the second sub-value are added to obtain the activity information corresponding to the target data.

4. The method according to claim 1, wherein After calculating the numerical difference between the space occupancy value and the values ​​of each free position in the data pool, the method further includes: If there is no free position with the numerical difference smaller than the preset space threshold, any target description data between two adjacent free positions is moved toward one of the free positions; Until the two idle positions are merged, the updated idle position is obtained.

5. The method according to claim 1, wherein After determining the target activity information according to the target description data and adding the target index address to the third position of the cold data area according to the target activity information, the method further includes: Determine to clear the index address in the last area of ​​the cold data area and the description data corresponding to the index address in the data pool.

6. A data reading device, characterized in that: The device comprises: An acquisition module, used for acquiring an access request for target data; a data reading module, configured to, in response to the access request, obtain target description data of the target data based on a corresponding position in a preset data pool pointed to by the target index address when the target index address of the target data exists in a preset buffer pool, and return the target description data, wherein the buffer pool stores index addresses of a plurality of data, and the plurality of index addresses are arranged in a sorted order, and the data pool stores description data corresponding to each index address; an updating module, configured to, when the target index address is located in a hot data area in the buffer pool, or when the target index address is located in a cold data area in the buffer pool and the existence time satisfies a preset condition, transfer the target index address to the head position of the hot data area, so as to update the sorting order of the index addresses in the buffer pool; Each of the index addresses in the data pool is provided with corresponding activity information, where the activity information is used to indicate the probability that the data corresponding to the corresponding index address is read; After returning the target description data, the method further includes: When the target index address is located in the cold data area of ​​the buffer pool and the existence time meets the preset conditions, determining the corresponding access control information and change time information according to the target description data; determining target activity information corresponding to the target index address, calculating a first ratio between the access control information and the target activity information, calculating a second ratio between the change time information and the target activity information, and updating the target activity information according to the first ratio and the second ratio; Transferring the target index address to the first position of the hot data area according to the updated sorting order of the target active information and other active information; After obtaining the access request for the target data, the method further includes: In response to the access request, when the target index address of the target data does not exist in a preset buffer pool, obtaining target description data of the target data based on a corresponding position in a storage medium pointed to by the target index address, and returning the target description data, wherein the storage medium stores addresses of multiple data; Determining a space occupation value of the target data, and calculating a numerical difference between the space occupation value and each free position value in the data pool, wherein each free position in the data pool includes a corresponding free position value; If there is an idle position where the numerical difference is less than a preset space threshold, adding the target index address and the target description data to the idle position; The target activity information is determined according to the target description data, and the target index address is added to the third position of the cold data area according to the target activity information.

7. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the data reading method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the data reading method according to any one of claims 1 to 5 is implemented.

Images