Data processing method and apparatus

Abstract

The application discloses a data processing method and device. The method comprises the following steps: in response to receiving an access request, determining an access address of a device sending the access request; in response to the fact that the first storage structure does not store the historical access frequency of the access address, determining a first storage position corresponding to the access address in the second storage structure, wherein the second storage structure stores the historical access frequency of the access address; updating the historical access frequency of the access address to obtain the current access frequency of the access address; and storing the current access frequency of the access address to the first storage position. The application solves the technical problem of low storage accuracy of the related art in storing the access frequency.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a data processing method and apparatus. Background Technology

[0002] With the development of internet technology and the interconnection of everything, data volume is exploding, making the storage of access frequency a challenge. In existing technologies, the required storage space varies depending on the size of the access frequency. When the access frequency involves a large amount of data, multiple storage spaces may be needed, requiring the access frequency to be split, which can lead to inaccurate storage results.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a data processing method and apparatus to at least solve the technical problem of low storage accuracy when storing access frequency in related technologies.

[0005] According to one aspect of the present invention, a data processing method is provided, comprising: in response to receiving an access request, determining an access address of the device sending the access request; in response to the fact that a first storage structure does not store the historical access frequency of the access address, determining a first storage location in a second storage structure corresponding to the access address, wherein the second storage structure stores the historical access frequency of the access address; updating the historical access frequency of the access address to obtain the current access frequency of the access address; and storing the current access frequency of the access address in the first storage location.

[0006] Optionally, the second storage structure includes: multiple layers of first storage space, the capacity of the multiple layers of first storage space forming a preset ratio, and determining the first storage location corresponding to the access address in the second storage structure includes: traversing the multiple layers of first storage space to determine at least one first storage space storing historical access frequencies; obtaining the first remaining capacity of the first storage space located at the lowest layer of the at least one first storage space; in response to the first remaining capacity being greater than or equal to the first preset capacity, determining the position of the first storage space at the lowest layer in the second storage structure as the first storage location; in response to the first remaining capacity being less than the first preset capacity, determining the position of the first storage space at the second layer in the second storage structure as the first storage location, wherein the level of the first storage space at the second layer is higher than the level of the first storage space at the lowest layer.

[0007] Optionally, after determining that the location of the second storage space in the second storage structure is the first storage location, the method includes: determining the second storage location in the third storage structure corresponding to the first storage location; updating the target data stored in the storage space corresponding to the second storage location, wherein the target data is used to characterize whether data is stored in the storage space corresponding to the first storage location.

[0008] Optionally, the third storage structure includes: a multi-layer second storage space, which corresponds one-to-one with the storage space of a preset layer in the multi-layer first storage space.

[0009] Optionally, the historical access frequency of the access address is updated to obtain the current access frequency of the access address, including: obtaining the access frequency of the access address within the current time period; obtaining the sum of the historical access frequency and the access frequency within the current time period to obtain the current access frequency of the access address.

[0010] Optionally, after storing the current access frequency of the access address to the first storage location, the method further includes: comparing the current access frequency with a preset target threshold; and storing the current access frequency to the first storage structure in response to the current access frequency being greater than or equal to the preset target threshold.

[0011] Optionally, in response to the fact that the first storage structure stores the historical access frequency of the access address, the method further includes: updating the historical access frequency of the access address to obtain the current access frequency of the access address; and storing the current access frequency in the first storage structure.

[0012] Optionally, storing the current access frequency in the first storage structure includes: determining whether the second remaining capacity of the first storage structure is greater than or equal to the second preset capacity; in response to the second remaining capacity being greater than or equal to the second preset capacity, storing the current access frequency in the first storage structure; in response to the second remaining capacity being less than the second preset capacity, deleting the minimum access frequency stored in the first storage structure and storing the current access frequency in the first storage structure.

[0013] Optionally, storing the current access frequency in the first storage structure includes: obtaining the quotient of the current access frequency and the historical access frequency to obtain a frequency quotient; in response to the frequency quotient being greater than or equal to a first preset quotient, storing the access address, historical access frequency, current access frequency, and the time period corresponding to the current access frequency in the first storage structure; in response to the frequency quotient being less than a second preset quotient, obtaining the difference between the current time period and the historical time period corresponding to the historical access frequency to obtain a time period difference; and in response to the time period difference being less than a preset length, deleting the historical access frequency from the first storage space and storing the current access frequency in the second storage structure.

[0014] Optionally, the method further includes: in response to receiving a query request, determining the query address contained in the query request; determining the third storage location corresponding to the query address in the third storage structure; obtaining the data stored in the storage space corresponding to the third storage location in the third storage structure to obtain the target storage location; obtaining the data stored in the storage space corresponding to the target storage location in the second storage structure to obtain the target access frequency corresponding to the query address; and outputting the target access frequency.

[0015] Optionally, when the target storage location includes multiple storage locations, obtaining the data stored in the storage space corresponding to the target storage location in the second storage structure to obtain the target access frequency corresponding to the query address includes: obtaining multiple data stored in the storage space corresponding to multiple storage locations; obtaining the sum of multiple data to obtain the target access frequency.

[0016] According to another aspect of the present invention, a data processing apparatus is also provided, comprising: a first determining module, configured to determine an access address of a device sending an access request in response to receiving an access request; a second determining module, configured to determine a first storage location in a second storage structure corresponding to the access address in response to the absence of historical access frequencies of the access address stored in a first storage structure, wherein the second storage structure stores historical access frequencies of the access address; a first updating module, configured to update the historical access frequencies of the access address to obtain the current access frequency of the access address; and a first storage module, configured to store the current access frequency of the access address in the first storage location.

[0017] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform any of the above-described data processing methods.

[0018] According to another aspect of the present invention, a processor is also provided, which is used to run a program, wherein the program executes any of the above-described data processing methods during runtime.

[0019] In this embodiment of the invention, the method involves: responding to a received access request, determining the access address of the device sending the access request; responding to the fact that the historical access frequency of the access address is not stored in the first storage structure, determining the first storage location corresponding to the access address in the second storage structure, wherein the historical access frequency of the access address is stored in the second storage structure; updating the historical access frequency of the access address to obtain the current access frequency of the access address; and storing the current access frequency of the access address in the first storage location. It is noteworthy that the access frequency is stored by dynamically determining the storage capacity of the storage structure, achieving the goal of accurately storing the access frequency, thereby improving the storage accuracy of access frequency storage and solving the technical problem of low storage accuracy in related technologies. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0021] Figure 1 This is a flowchart of a data processing method according to an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram of an optional storage structure according to an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of the structure of a data processing device according to an embodiment of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] Example 1

[0027] According to an embodiment of the present invention, an embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0028] Figure 1 This is a flowchart of a data processing method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0029] Step S102: In response to receiving an access request, determine the access address of the device that sent the access request.

[0030] The aforementioned access request could be a request sent by a vehicle to the server to access data stored on the server, a request sent by a mobile terminal to the server to access data stored on the server, or a request sent by another server to the server to access data stored on the server.

[0031] The access address of the device sending the access request mentioned above can be an address used to uniquely identify different devices, such as an Internet Protocol (IP) address, a physical address (MAC) address, etc. In this embodiment of the invention, an IP address will be used as an example for explanation.

[0032] Since access requests typically contain the vehicle's IP address, in one alternative embodiment, after the server receives an access request from a vehicle, it can obtain the vehicle's IP address carried in the access request by parsing the request.

[0033] In another optional embodiment, after the server receives an access request from the mobile terminal, it can parse the access request to obtain the IP address of the mobile terminal carried in the access request. It should be noted that the mobile terminal may include, but is not limited to, mobile phones, laptops, and tablets.

[0034] In another optional embodiment, after the server receives an access request from another server, it can parse the access request to obtain the IP address of the other server carried in the access request. It should be noted that the server here can be a local server or a cloud server, but is not limited to these.

[0035] Step S104: In response to the fact that the historical access frequency of the access address is not stored in the first storage structure, determine the first storage location corresponding to the access address in the second storage structure, wherein the historical access frequency of the access address is stored in the second storage structure.

[0036] The aforementioned first storage structure can be a structure in the current server used to store the access frequency of access addresses. The access frequency of all access addresses stored in the first storage structure is greater than or equal to a threshold. When the access frequency exceeds the threshold, it indicates that the access frequency is a large number of data points, and the access frequency can be stored in the first storage structure for faster subsequent lookup and storage.

[0037] The aforementioned second storage structure can be a structure within the current server that filters and stores the access frequency of access addresses. This second storage structure stores the historical access frequencies of access addresses. When the access frequency is greater than or equal to a threshold, the access frequency is stored in the first storage structure, and the access frequency in the second storage structure is deleted. This not only frees up storage capacity in the second storage space but also allows for the classification of access frequencies with small and large data volumes, enabling accurate storage and fast retrieval of access frequencies.

[0038] It should be noted that the threshold can be set according to the user's needs. In this embodiment, 1000 is used as an example for explanation, but it is not limited to this.

[0039] In this embodiment of the invention, the first storage structure can directly store the access frequency of a large amount of data without splitting it into multiple access frequencies of smaller amounts of data. The second storage structure can directly store the access frequency of smaller amounts of data. By setting two different storage structures when storing access frequencies, access frequencies of different amounts of data can be accurately stored, thereby improving the storage accuracy of access frequencies.

[0040] The first storage location mentioned above can be the location in the second storage structure where the access frequency of the access address is stored.

[0041] In one optional embodiment, after determining the access address, it can first be determined whether the historical access frequency of the access address is stored in the first storage structure; in response to the fact that the historical access frequency of the access address is not stored in the first storage structure, the first storage location for storing the access frequency of the access address in the second storage structure can be determined.

[0042] Optionally, the second storage structure includes: multiple layers of first storage space, the capacity of the multiple layers of first storage space forming a preset ratio, and determining the first storage location corresponding to the access address in the second storage structure includes: traversing the multiple layers of first storage space to determine at least one first storage space storing historical access frequencies; obtaining the first remaining capacity of the first storage space located at the lowest layer of the at least one first storage space; in response to the first remaining capacity being greater than or equal to the first preset capacity, determining the position of the first storage space at the lowest layer in the second storage structure as the first storage location; in response to the first remaining capacity being less than the first preset capacity, determining the position of the first storage space at the second layer in the second storage structure as the first storage location, wherein the level of the first storage space at the second layer is higher than the level of the first storage space at the lowest layer.

[0043] The first storage space of the lowest layer mentioned above can be the space in the second storage structure that can store the access frequency of the access address first. In this embodiment, L1 is used as an example for explanation.

[0044] The levels of the aforementioned multi-layer first storage space can be preset; further, the levels of the multi-layer first storage space can be preset by the user; further, the levels of the multi-layer first storage space can be preset by the user according to the actual storage situation. In this embodiment, four layers are used as an example for explanation, but it is not limited to this, and it can also be three layers, five layers, etc.

[0045] The capacity of the aforementioned multi-layer first storage space can be set by the user; further, the capacity of the multi-layer storage space can be preset by the user; even further, the capacity of the multi-layer storage space can be set by the user based on the actual storage situation. In this embodiment, 2 bytes (bit) is used as an example for explanation, but it is not limited to this, and can also be 3 bits, 4 bits, etc. Each layer of storage space contains the capacity of multiple first storage spaces, and each capacity has a corresponding storage location, i.e., the first storage location.

[0046] The aforementioned preset ratio can be pre-set; further, the aforementioned preset ratio can be pre-set by the user; further still, the aforementioned preset ratio can be set by the user according to the actual storage situation. In this embodiment, a 4:5 ratio is used as an example, where the preset ratio means that the usage of the first storage space of the higher level is one-quarter of the usage of the first storage space of the lower level. It should be noted that the preset ratio is not limited to 4:5, and can be any preset ratio.

[0047] The aforementioned first remaining capacity may be the actual remaining capacity in the first storage space that can store the access frequency.

[0048] The aforementioned first preset capacity can be a fixed capacity set by the user in advance, or a dynamic capacity determined based on the current access frequency that can fully store the access frequency.

[0049] The first storage space of the second layer mentioned above can be the second space in the second storage structure that can store the access frequency of the access address. In this embodiment, L2 is used as an example for explanation. It should be noted that the layer of L2 is higher than the layer of L1.

[0050] In one optional embodiment, the second storage structure includes multiple layers of first storage space, wherein the first storage space is used to store the access frequency of the access address. The second storage structure may have multiple layers of first storage space, and the capacity of each first storage space may be 2 bits. The capacity of each layer of first storage space forms a preset ratio, such as 4:5, but not limited to this.

[0051] In another alternative embodiment, firstly, multiple layers of first storage space can be traversed to determine at least one first storage space in the second storage structure that stores historical access frequencies. Secondly, the first remaining capacity of the first storage space located at the lowest layer of at least one first storage space can be obtained. In response to the first remaining capacity being greater than or equal to a first preset capacity, it indicates that the first storage space at the lowest layer can completely store the access frequencies, and the position of the first storage space at the lowest layer in the second storage structure can be determined as the first storage location.

[0052] For example, Figure 2 This is a schematic diagram of an optional storage structure according to an embodiment of the present invention, such as... Figure 2As shown, in response to the fact that the historical access frequency of the access address is not stored in the storage structure (i.e., the first storage structure), the L1 in the filter (i.e., the second storage structure) can be traversed first by a preset function. In response to the fact that the first remaining capacity in L1 is greater than or equal to the first preset capacity, the position of the first storage space of L1 in the second storage structure is determined as the first storage location, wherein the historical access frequency of the access address is stored in the second storage structure.

[0053] In another optional embodiment, in response to the first remaining capacity being less than a first preset capacity, indicating that the lowest-level first storage space cannot fully store the access frequency, the position of the second-level first storage space in the second storage structure can be determined as the first storage location. Here, the level of the second-level first storage space is higher than the level of the lowest-level first storage space.

[0054] For example, such as Figure 2 As shown, in response to the first remaining capacity in L1 being less than the first preset capacity after traversing L1, it is possible to continue traversing whether the first remaining capacity of L2 is greater than or equal to the first preset capacity. In response to the first remaining capacity in L2 being greater than or equal to the first preset capacity, the position of the first storage space of L2 in the second storage structure is determined as the first storage location.

[0055] If the first remaining capacity in L2 is less than the first preset capacity after traversing L2, then the first remaining capacity in L3 can be traversed to see if it is greater than or equal to the first preset capacity. If the first remaining capacity in L3 is greater than or equal to the first preset capacity, then the position of the first storage space in L3 in the second storage structure is determined as the first storage location.

[0056] If the first remaining capacity in L3 is less than the first preset capacity after traversing L3, then the traversal of higher-level L can continue until the access frequency is completely stored.

[0057] It should be noted that the aforementioned preset function can be any function capable of traversing the storage space of the second storage structure. In this embodiment, a hash function is used as an example for explanation.

[0058] Optionally, after determining the location of the first storage space of the second layer in the second storage structure as the first storage location, the method includes: determining the second storage location in the third storage structure corresponding to the first storage location; updating the target data stored in the storage space corresponding to the second storage location, wherein the target data is used to characterize whether data is stored in the storage space corresponding to the first storage location.

[0059] Optionally, the third storage structure includes: a multi-layer second storage space, which corresponds one-to-one with the storage space of a preset layer in the multi-layer first storage space.

[0060] The aforementioned third storage structure can be a storage structure capable of recording the access frequency occupying the first storage location. It should be noted that by setting up the third storage structure, the location of the storage space storing the access frequency can be recorded. This ensures that when storing the access frequency of the same access address subsequently, a preset function can quickly and accurately find the historical access frequency of the same access address and update the historical access frequency, thereby improving the storage accuracy of the access frequency.

[0061] It should be noted that by setting up a third storage structure, when the access frequency of large amounts of data is stored in a higher level L, the preset function can directly store the access frequency of the same access address through the first storage location recorded by the second storage location, without having to traverse each storage space in the second storage structure, thereby improving the efficiency and accuracy of storing access frequency.

[0062] The second storage location mentioned above can be a storage location in the third storage structure that corresponds to the first storage location. The second storage location can record the access frequency of the first storage location. In this embodiment, R is used as an example for explanation.

[0063] When the access frequency occupies the first storage location, the third storage structure can record the occupancy status in the second storage location. For example, when the first storage location is occupied, the second storage location can be recorded as 1, indicating that the access frequency has occupied the corresponding first storage location. Conversely, when the second storage location is recorded as 0, it indicates that the access frequency has not occupied the corresponding first storage location.

[0064] The target data mentioned above can be data that records the access frequency occupying the first storage location. For example, when the access frequency only occupies the first storage location of the lowest level first storage space, the value of the second storage location in R1 is recorded as 0, indicating that the access frequency only occupies the first storage location of the lowest level. When the value of the second storage location in R1 is recorded as 1, it indicates that the access frequency occupies the first storage location of both the lowest and second levels.

[0065] The aforementioned multi-layered second storage space can be a storage space in the third storage structure that records the frequency of access to occupy the first storage location.

[0066] There is a one-to-one correspondence between the multi-level second storage space and the multi-level first storage space. Specifically, R1 can correspond to L2, R2 to L3, and R3 to L4, but this is not limited to these cases. The level of R2 is higher than that of R1.

[0067] In one optional embodiment, when the access frequency occupies multiple storage locations in L1 of the second storage structure, the corresponding historical access frequency can be obtained by traversing L1 through a preset function, and the historical access frequency can be updated.

[0068] In another alternative embodiment, such as Figure 2 As shown, when the access frequency occupies multiple first storage locations in L2 of the second storage structure, the specific locations of the multiple first storage locations occupied by the access frequency can be recorded by R1 corresponding to L2 in the record structure (i.e., the third storage structure), and the data of the second storage location corresponding to the first storage location is marked as 1, which indicates which first storage locations in the second storage structure the access frequency occupies.

[0069] In another alternative embodiment, such as Figure 2 As shown, when the access frequency occupies multiple first storage locations in L3 of the second storage structure, the specific locations of the multiple first storage locations occupied by the access frequency can be recorded through R2 in the third storage structure corresponding to L3, and the data of the second storage location corresponding to the first storage location is marked as 1. At the same time, R1 also records the specific location of L2 occupied by the access frequency, which indicates which first storage locations in the second storage structure the access frequency occupies.

[0070] In another alternative embodiment, after determining the first storage location, a second storage location corresponding to the first storage location in the third storage structure can be determined, wherein the second storage location can record the access frequency occupying the first storage location.

[0071] In another optional embodiment, after the second storage location is determined, the target data stored in the storage space corresponding to the second storage location can be updated, wherein the target data is used to characterize whether there is data stored in the storage space corresponding to the first storage location.

[0072] In another optional embodiment, after determining the second storage location, the target data in the second storage location can be updated using a preset function. For example, when the second storage location is determined to be storage location L2 of the second layer, the target data in the storage space corresponding to R1 can be updated using a preset function. For example, the target data in the corresponding second storage location in R1 can be updated to 1, indicating that the access frequency occupies the first storage location of L2 in the second storage structure.

[0073] It should be noted that the preset function can be any function that can update the target data. In this embodiment, a hash function is used as an example, but it is not limited to this.

[0074] Step S106: Update the historical access frequency of the access address to obtain the current access frequency of the access address.

[0075] Optionally, the historical access frequency can be updated by: obtaining the access frequency of the access address within the current time period; obtaining the sum of the historical access frequency and the access frequency within the current time period to obtain the current access frequency of the access address.

[0076] The aforementioned current time period can be the time period during which the device is currently sending access requests. The specific duration is not limited in this implementation and can be set by the user according to their needs.

[0077] In one optional embodiment, the access frequency of the access address within the current time period can be obtained first, the historical access frequency corresponding to the same access address in the second storage structure can be obtained second, and the historical access frequency can be added to the access frequency within the current time period to obtain the current access frequency of the access address.

[0078] Step S108: Store the current access frequency of the access address in the first storage location.

[0079] In one alternative embodiment, after obtaining the current access frequency, the current access frequency can be stored in the first storage location in the second storage structure.

[0080] In this embodiment, the method involves: responding to a received access request, determining the access address of the device sending the access request; responding to the fact that the historical access frequency of the access address is not stored in the first storage structure, determining the first storage location corresponding to the access address in the second storage structure, wherein the historical access frequency of the access address is stored in the second storage structure; updating the historical access frequency of the access address to obtain the current access frequency of the access address; and storing the current access frequency of the access address in the first storage location. It is noteworthy that the access frequency is stored by dynamically determining the storage capacity of the storage structure, achieving the goal of accurately storing the access frequency, thereby improving the storage accuracy of access frequency storage and solving the technical problem of low storage accuracy in related technologies.

[0081] Optionally, after storing the current access frequency of the access address to the first storage location, the method further includes: comparing the current access frequency with a preset target threshold; and storing the current access frequency to the first storage structure in response to the current access frequency being greater than or equal to the preset target threshold.

[0082] The aforementioned preset target threshold can be pre-set; optionally, the preset target threshold can be pre-set by the user; optionally, the preset target threshold can be set by the user based on the storage situation of the access frequency. In this embodiment, 1000 is used as an example for explanation, but it is not limited to this.

[0083] In one optional embodiment, after obtaining the current access frequency of the access address, the current access frequency can be compared with a preset target threshold. If the current access frequency is greater than or equal to the target preset threshold, it indicates that the access frequency is a large data volume access frequency. Therefore, the current access frequency can be directly stored in the first storage structure without traversing the access frequency of small data volumes in the second storage structure, so that the current server can quickly store the access frequency of the same access address.

[0084] Optionally, in response to the fact that the first storage structure stores the historical access frequency of the access address, the method further includes: updating the historical access frequency of the access address to obtain the current access frequency of the access address; and storing the current access frequency in the first storage structure.

[0085] In one optional embodiment, in response to the fact that the first storage structure stores the historical access frequency of the access address, the access frequency of the current time period can be directly added to the historical access frequency to obtain the current access frequency of the access address.

[0086] In another alternative embodiment, after obtaining the current access frequency of the access address, the current access frequency can be stored in the first storage structure.

[0087] Optionally, storing the current access frequency in the first storage structure includes: determining whether the second remaining capacity of the first storage structure is greater than or equal to the second preset capacity; in response to the second remaining capacity being greater than or equal to the second preset capacity, storing the current access frequency in the first storage structure; in response to the second remaining capacity being less than the second preset capacity, deleting the minimum access frequency stored in the first storage structure and storing the current access frequency in the first storage structure.

[0088] The aforementioned second remaining capacity can be the capacity to fully store the current access frequency. The specific value is not limited, and users can set it according to their own needs.

[0089] The aforementioned second preset capacity can be the minimum capacity that can completely store the current access frequency. If it is less than the second preset capacity, it means that the current access frequency cannot be completely stored.

[0090] In one optional embodiment, before storing the current access frequency to the first storage structure, it can be determined whether the second remaining capacity in the first storage structure is greater than or equal to the second preset capacity. If the second remaining capacity is greater than or equal to the second preset capacity, it indicates that the first storage structure can completely store the current access frequency, and then the current access frequency can be directly stored to the first storage structure.

[0091] In another alternative embodiment, in response to the second remaining capacity being less than the second preset capacity, indicating that the first storage structure cannot fully store the current access frequency, the minimum access frequency stored in the first storage structure can be deleted, and then the current access frequency can be stored in the first storage structure.

[0092] Optionally, storing the current access frequency in the first storage structure includes: obtaining the quotient of the current access frequency and the historical access frequency to obtain a frequency quotient; in response to the frequency quotient being greater than or equal to a first preset quotient, storing the access address, historical access frequency, current access frequency, and the time period corresponding to the current access frequency in the first storage structure; in response to the frequency quotient being less than a second preset quotient, obtaining the difference between the current time period and the historical time period corresponding to the historical access frequency to obtain a time period difference; and in response to the time period difference being less than a preset length, deleting the historical access frequency from the first storage space and storing the current access frequency in the second storage structure.

[0093] The current access frequency can be Ccur, the historical access frequency can be Cpre, and the frequency quotient can be Ccur / Cpre.

[0094] The first preset quotient mentioned above can be 2, but it is not limited to this. The specific value can be set by the user according to their needs.

[0095] The access address mentioned above can be a key; the time period corresponding to the current access frequency can be Win1.

[0096] The second preset quotient mentioned above can be 1 / 2, but it is not limited to this. The specific value can be set by the user according to their needs.

[0097] The historical access frequency mentioned above corresponds to the historical time period of Win2; the time period difference can be Win1-Win2.

[0098] The preset length mentioned above can be the maximum length, maxLength.

[0099] In one optional embodiment, before storing the current access frequency to the first storage structure, the quotient of the current access frequency and the historical access frequency can be obtained first, resulting in a frequency quotient Ccur / Cpre. In response to Ccur / Cpre being greater than or equal to a first preset quotient of 2, the frequency quotient can be...<key,Cpre,Ccur,Win1> Stored in the first storage structure.

[0100] In another optional embodiment, in response to the frequency quotient Ccur / Cpre being less than the second preset quotient 1 / 2, the difference Ccur-Win2 between the current time period and the historical time period corresponding to the historical access frequency can be obtained. In response to Ccur-Win2 being less than the preset length (maxLength), the historical access frequency can be deleted from the first storage space and the current access frequency can be stored in the second storage structure.

[0101] Optionally, the method further includes: in response to receiving a query request, determining the query address contained in the query request; determining the third storage location corresponding to the query address in the third storage structure; obtaining the data stored in the storage space corresponding to the third storage location in the third storage structure to obtain the target storage location; obtaining the data stored in the storage space corresponding to the target storage location in the second storage structure to obtain the target access frequency corresponding to the query address; and outputting the target access frequency.

[0102] The query request mentioned above could be a query request issued by a user to query the access frequency of a certain device.

[0103] The query address mentioned above can be the access address of the device that the query request wants to query, or it can be an address used to uniquely identify the device, such as an IP address, a physical address (MAC address), etc. In this embodiment of the invention, the query address is of the same type as the access address stored in the storage structure.

[0104] The aforementioned third storage location can be a location in a third storage structure, which stores the access frequency of the access address that is the same as the query address in the second storage structure. In this embodiment, K is used as an example for explanation.

[0105] The target storage location mentioned above can be a location in the second storage structure, which stores the access frequency of the access address that is the same as the query address.

[0106] The target access frequency mentioned above can be the total access frequency of access addresses that are the same as the query address.

[0107] In one optional embodiment, upon receiving a query request, the query address corresponding to the query request can be determined. Then, a third storage location in the third storage structure can be determined by a preset function. The third storage location records the storage location of the access address that is the same as the query address. After determining the third storage location, the data of the corresponding storage location recorded in the third storage location can be obtained to obtain the target storage location. The target storage location stores the access frequency of the access address that is the same as the query address.

[0108] In another optional embodiment, after determining the target storage location, the data stored in the storage space corresponding to the target storage location in the second storage structure can be obtained through a preset function, the access frequency of the access address corresponding to the query address is obtained, i.e. the target access frequency, and the target access frequency is output.

[0109] Optionally, when the target storage location includes multiple storage locations, obtaining the data stored in the storage space corresponding to the target storage location in the second storage structure to obtain the target access frequency corresponding to the query address includes: obtaining multiple data stored in the storage space corresponding to multiple storage locations; obtaining the sum of multiple data to obtain the target access frequency.

[0110] In an optional embodiment, when the target storage location contains multiple storage locations, multiple data stored in the storage space corresponding to the multiple storage locations can be obtained through a preset function, i.e., multiple access frequencies. Then, the sum of the multiple access frequencies can be obtained, i.e., the target access frequency can be obtained.

[0111] It should be noted that the preset function can be any function that can determine the third storage location. In this embodiment, the hash function is used as an example for explanation.

[0112] This invention mainly consists of two parts: data storage and processing algorithms.

[0113] like Figure 2 As shown, the storage structure includes: filter 21, record structure 22, and storage structure 23.

[0114] Wherein, the filter is divided into L1, L2, ..., Lw parts, and w is generally set to 4. Each part is a storage structure, specifically the storage structure is set to a smaller 2 bits, and the ratio of the number of structures between the bottom layer and the top layer is 4:5. When storing the frequency of IP addresses accessing the IoV cloud, it starts from the L1 part first. When the L1 part is insufficient, other parts such as L2 and L3 are dynamically used from low to high, which avoids the problem of memory waste caused by directly setting multiple levels in the prior art; the recording structure is divided into R1, R2, ..., Rw-1 parts, which is one part less than that of the filter, and indicates the number of filters used by a certain IP address frequency. The recording structure adopts multiple parts, and each part is set to 1 bit, which increases available recording structures, reduces hash function conflicts between different IP addresses, and improves the accuracy of the recording structure. Through the data structure proposed by the present invention, the accuracy of the frequency of IP addresses accessing the IoV cloud is greatly improved. The IoV cloud can obtain more accurate frequency statistics. When the frequency counted by the left structure exceeds the threshold, the data may become burst (brust) data, and then the data is stored in the storage structure. The storage structure is divided into a plurality of arrays, and the data structure stored in each array is <key, Cpre, Ccur, Win>, where key is an IP address, Cpre is the frequency value of the previous time period, Ccur is the frequency value of the current time period, and Win is the time period when the IP address changes. When Ccur / Cpre >= 2, update Win to the current time period; when Ccur / Cpre < 1 / 2, if cur-win < maxLength, it indicates a sudden decrease in data, the data is updated, and the data is excluded from the storage structure to provide space for other data.

[0115] The specific implementation of the processing algorithm is as follows:

[0116] Filter insertion algorithm:

[0117] When an IP address accesses the vehicle network cloud, the system first determines whether the data is already stored in the storage structure. If it exists, the value is directly incremented; otherwise, it is stored and statistically analyzed in the filter structure. First, a hash function is used to determine the storage location in the L1 part. If the value at that location is less than 3, the value is directly incremented. If the value is 3, the L2 storage location is used. First, a record is made in R1: the hash function determines the record location in R1, and the value is set to 1, indicating that the IP address has dynamically requested storage space. Then, the L1 operation is repeated, using a hash function to determine the storage location in L2. If the value at that location is less than 3, the value is directly incremented. If the value is 3, the L3 storage location is used, and this process continues until the last part of the storage structure is reached. After the data insertion operation is complete, the data value is queried. If the value exceeds the threshold, the data is stored in the storage structure. If there is an empty space in the storage structure, it is directly added; if there is no empty space, the data with the smallest current Ccur is replaced, and Cpre and Ccur are set to 0.

[0118] Filter query algorithm:

[0119] When querying the frequency of access to the vehicle network cloud by a certain IP address, the first step is to use a hash function to determine the position k of the IP address in the record structure. Then, the values ​​at the corresponding positions L1, L2, ..., Lk+1 are accessed. Finally, the values ​​are added together to obtain the final frequency value.

[0120] The filter data structure provided by this invention significantly reduces memory waste caused by varying IP address frequencies, improves memory utilization efficiency, increases the number of storage structures, reduces hash function collisions, and improves the accuracy of IP address access frequencies. This is highly useful for vehicle-to-everything (V2X) cloud computing, enabling the acquisition of accurate access frequencies. This data structure can store more accurate data and promptly detect data exhibiting bursting behavior. Proper analysis and processing of bursting data allows for the rational allocation of user resources and network bandwidth, providing a better user experience.

[0121] Example 2

[0122] According to another aspect of the present invention, a data processing apparatus is also provided, which can execute the data processing method provided in Embodiment 1 above. The specific implementation method and preferred application scenario are the same as those in Embodiment 1 above, and will not be repeated here.

[0123] Figure 3 This is a schematic diagram of the structure of a data processing apparatus according to an embodiment of the present invention, such as... Figure 3As shown, the device includes: a first determining module 30, configured to determine the access address of the device sending the access request in response to receiving an access request; a second determining module 32, configured to determine a first storage location in a second storage structure corresponding to the access address in response to the fact that the historical access frequency of the access address is not stored in the first storage structure, wherein the historical access frequency of the access address is stored in the second storage structure; a first updating module 34, configured to update the historical access frequency of the access address to obtain the current access frequency of the access address; and a first storage module 36, configured to store the current access frequency of the access address in the first storage location.

[0124] Optionally, the second storage structure includes: multiple layers of first storage space, the capacity of the multiple layers of first storage space constitutes a preset ratio, and the second determining module includes: a first processing unit, used to traverse the multiple layers of storage space and determine at least one storage space storing historical access frequencies; a first obtaining unit, used to obtain the first remaining capacity of the first storage space located at the highest level among the at least one storage space; a first determining unit, used to determine the position of the first storage space in the second storage structure as a first storage location in response to the first remaining capacity being greater than or equal to a first preset capacity; and a second determining unit, used to determine the position of the second storage space in the second storage structure as a first storage location in response to the first remaining capacity being less than the first preset capacity, wherein the level of the second storage space is higher than the level of the first storage space.

[0125] Optionally, the second determining unit includes: a first determining subunit, used to determine a second storage location in the third storage structure corresponding to the first storage location; and an updating subunit, used to update the target data stored in the storage space corresponding to the second storage location, wherein the target data is used to characterize whether data is stored in the storage space corresponding to the first storage location.

[0126] Optionally, the storage module includes: a second processing unit for comparing the current access frequency with a preset target threshold; and a first storage unit for storing the current access frequency in a first storage structure in response to the current access frequency being greater than or equal to the preset target threshold.

[0127] Optionally, in response to the fact that the historical access frequency of the access address is stored in the first storage structure, the device further includes: a second update module for updating the historical access frequency of the access address to obtain the current access frequency of the access address; and a second storage module for storing the current access frequency in the first storage structure.

[0128] Optionally, the second storage module includes: a judgment unit, used to judge whether the second remaining capacity of the first storage structure is greater than or equal to the second preset capacity; a second storage unit, used to store the current access frequency in the first storage structure in response to the second remaining capacity being greater than or equal to the second preset capacity; and a third processing unit, used to delete the minimum access frequency stored in the first storage structure and store the current access frequency in the first storage structure in response to the second remaining capacity being less than the second preset capacity.

[0129] Optionally, the third processing unit includes: an acquisition subunit, used to acquire the quotient of the current access frequency and the historical access frequency to obtain a frequency quotient; a storage subunit, used to store the access address, historical access frequency, current access frequency, and the time period corresponding to the current access frequency to a first storage result in response to the frequency quotient being greater than or equal to a first preset quotient; and a processing subunit, used to acquire the difference between the current time period and the historical time period corresponding to the historical access frequency in response to the frequency quotient being less than a second preset quotient to obtain a time period difference, and to delete the historical access frequency from the first storage space and store the current access frequency to a second storage structure in response to the time period difference being less than a preset length.

[0130] Optionally, the device further includes: a third determining module, configured to determine the query address contained in the query request in response to receiving the query request; a fourth determining module, configured to determine the third storage location corresponding to the query address in the third storage structure; a first obtaining module, configured to obtain the data stored in the storage space corresponding to the third storage location in the third storage structure to obtain the target storage location; a second obtaining module, configured to obtain the data stored in the storage space corresponding to the target storage location in the second storage structure to obtain the target access frequency corresponding to the query address; and an output module, configured to output the target access frequency.

[0131] Optionally, when the target storage location includes multiple storage locations, the second acquisition module includes: a second acquisition unit, used to acquire multiple data stored in the storage space corresponding to the multiple storage locations; and a third acquisition unit, used to acquire the sum of the multiple data to obtain the target access frequency.

[0132] Example 3

[0133] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform any of the data processing methods described in the above embodiments.

[0134] Example 4

[0135] According to another aspect of the present invention, a processor is also provided, which is used to run a program, wherein the program executes any of the data processing methods described in the above embodiments.

[0136] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0137] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0138] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0139] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0140] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0141] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0142] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that, include: In response to receiving an access request, determine the access address of the device that sent the access request; In response to the fact that the historical access frequency of the access address is not stored in the first storage structure, a first storage location corresponding to the access address in the second storage structure is determined, wherein the historical access frequency of the access address is stored in the second storage structure, the access frequency of the access address stored in the first storage structure is greater than or equal to a threshold, and the access frequency of the access address stored in the second storage structure is less than the threshold. The historical access frequency of the access address is updated to obtain the current access frequency of the access address; Store the current access frequency of the access address in the first storage location.

2. The method according to claim 1, characterized in that, The second storage structure includes: multiple layers of first storage space, the capacity of the multiple layers of first storage space forming a preset ratio, and determining the first storage location corresponding to the access address in the second storage structure includes: Traverse the multi-layered first storage space to determine at least one first storage space that stores the historical access frequency; Obtain the first remaining capacity of the lowest-level first storage space in the at least one first storage space; In response to the first remaining capacity being greater than or equal to the first preset capacity, the position of the first storage space of the lowest layer in the second storage structure is determined as the first storage location; In response to the first remaining capacity being less than the first preset capacity, the position of the first storage space of the second layer in the second storage structure is determined as the first storage location, wherein the first storage space of the second layer is at a higher level than the first storage space of the lowest layer.

3. The method according to claim 2, characterized in that, After determining the location of the first storage space of the second layer within the second storage structure as the first storage location, the method includes: Determine the second storage location in the third storage structure that corresponds to the first storage location; The target data stored in the storage space corresponding to the second storage location is updated, wherein the target data is used to characterize whether there is data stored in the storage space corresponding to the first storage location.

4. The method according to claim 1, characterized in that, In response to the fact that the historical access frequency of the access address is stored in the first storage structure, the method further includes: The historical access frequency of the access address is updated to obtain the current access frequency of the access address; The current access frequency is stored in the first storage structure.

5. The method according to claim 4, characterized in that, Storing the current access frequency to the first storage structure includes: Determine whether the second remaining capacity of the first storage structure is greater than or equal to the second preset capacity; In response to the second remaining capacity being greater than or equal to the second preset capacity, the current access frequency is stored in the first storage structure; In response to the second remaining capacity being less than the second preset capacity, the minimum access frequency stored in the first storage structure is deleted, and the current access frequency is stored in the first storage structure.

6. The method according to claim 5, characterized in that, Storing the current access frequency to the first storage structure includes: Obtain the quotient of the current access frequency and the historical access frequency to get the frequency quotient; In response to the frequency quotient being greater than or equal to a first preset quotient, the access address, the historical access frequency, the current access frequency, and the time period corresponding to the current access frequency are stored in the first storage structure; In response to the frequency quotient being less than a second preset quotient, the difference between the current time period and the historical time period corresponding to the historical access frequency is obtained to obtain the time period difference. In response to the time period difference being less than a preset length, the historical access frequency is deleted from the first storage space and the current access frequency is stored in the second storage structure.

7. The method according to claim 1, characterized in that, The method further includes: In response to receiving a query request, determine the query address contained in the query request; Determine the third storage location corresponding to the query address in the third storage structure; Obtain the data stored in the storage space corresponding to the third storage location in the third storage structure to obtain the target storage location; Obtain the data stored in the storage space corresponding to the target storage location in the second storage structure, and obtain the target access frequency corresponding to the query address; Output the target access frequency.

8. The method according to claim 7, characterized in that, When the target storage location includes multiple storage locations, the data stored in the storage space corresponding to the target storage location in the second storage structure is obtained, and the target access frequency corresponding to the query address is obtained by including: Obtain multiple data items stored in the storage space corresponding to the multiple storage locations; The sum of the multiple data points is obtained to determine the target access frequency.

9. A data processing apparatus, characterized in that, include: The first determining module is used to determine the access address of the device that sent the access request in response to receiving the access request; The second determining module is used to determine the first storage location corresponding to the access address in the second storage structure in response to the fact that the historical access frequency of the access address is not stored in the first storage structure, wherein the historical access frequency of the access address is stored in the second storage structure, the access frequency of the access address stored in the first storage structure is greater than or equal to a threshold, and the access frequency of the access address stored in the second storage structure is less than a threshold. The first update module is used to update the historical access frequency of the access address to obtain the current access frequency of the access address; The first storage module is used to store the current access frequency of the access address in the first storage location.

Images