An artificial intelligence-based customer data storage and migration system

Abstract

The present invention discloses a customer data storage migration system based on artificial intelligence, and relates to the field of data processing technology. The system discloses an initial storage module, a storage retrieval module, and a data migration module. The initial storage module and the storage retrieval module are set to comprehensively analyze customer stored and retrieved data, determine the best storage location of each numbered customer data, and rationalize the storage allocation of customer data. On the basis of facilitating customer data retrieval and storage, it avoids repeated transmission of customer data between storage nodes during data retrieval. The data migration module is set to periodically analyze the node usage value of each storage node, and optimize the storage rationality of each storage node on the premise of facilitating subsequent retrieval of customer data.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to an artificial intelligence-based customer data storage migration system. Background Art

[0002] At present, all types of enterprises need to store a large amount of customer data, especially Internet service enterprises. Currently, blockchain storage is the preferred storage method for all enterprises. The blockchain contains multiple storage nodes, each of which can store various customer data, and customers can also retrieve the stored data through the Internet. When the current customer terminal generates data, it will upload the data to the nearest storage node. When the customer needs to view the stored data, the storage node will send the data to the customer's terminal device through the blockchain. After the customer has finished viewing, the data will be uploaded to the nearest storage node. Although this data storage method can achieve fast storage, when the customer needs to retrieve the data, the data may need to be transmitted through multiple storage nodes in the blockchain, resulting in low data retrieval efficiency. Summary of the invention

[0003] In view of the deficiencies in the prior art, the object of the present invention is to provide a customer data storage migration system based on artificial intelligence.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] An artificial intelligence-based customer data storage and migration system, comprising an initial storage module, a storage retrieval module, and a data migration module;

[0006] The initial storage module is used to number and store customer data when the customer terminal device generates customer data;

[0007] The storage module retrieves the customer data and determines the storage location of the customer data according to the comparison result between the retrieved value of the customer data and the threshold value after the customer terminal device retrieves the customer data;

[0008] The data migration module periodically obtains the node usage value of each storage node, and determines whether to migrate the customer data based on the comparison result between the node usage value and its threshold value.

[0009] Furthermore, when a client terminal device generates client data, a number is assigned to the client data, the current location of the client terminal device is obtained, and the numbered client data is uploaded to the storage node closest to the location of the client terminal device. When the client data is first uploaded to the storage node after being generated, no data retrieval record is generated.

[0010] Furthermore, after the client terminal device retrieves the client data, the storage location of the client data is determined based on the comparison result between the data retrieval value of the client data and its threshold value, specifically: when the client terminal device needs to retrieve the client data of the number, the corresponding storage node sends the client data of the number to the client terminal device, and after the client finishes retrieving the client data of the number, the data retrieval value of the client data of the number is obtained, and the data retrieval threshold is set; when the data retrieval value of the client data of the number is greater than or equal to the data retrieval threshold, the client data of the number is marked as stable retrieval data, the first recommended storage node for the stable retrieval data is obtained, and the stable retrieval data is uploaded to the first recommended storage node;

[0011] When the data retrieval value of the numbered data is less than the data retrieval threshold, the current location of the client terminal device is obtained, and the numbered client data is uploaded to the storage node closest to the location of the client terminal device.

[0012] Furthermore, when customer data is uploaded to a storage node, a data retrieval record is synchronously generated including the data number, node number, retrieval start time, and retrieval end time.

[0013] Furthermore, the data retrieval value of the customer data of the number is obtained by the following method: obtain all data retrieval records of the customer data of the number before the current system time, calculate the time difference between the retrieval end time and the retrieval start time of the data retrieval record to obtain the data retrieval duration, sum up all the data retrieval durations and take the average to obtain the average retrieval duration, and mark it as FTW, mark the total number of data retrieval records as MKH, and use the formula The data retrieval value PBC of the numbered data is obtained, where a1 is the average retrieval duration coefficient and a2 is the total number coefficient of data retrieval records.

[0014] Furthermore, the preferred storage node for stable data retrieval is obtained in the following manner: obtain all data retrieval records of stable data retrieval before the current time of the system, mark the data retrieval records of nodes with the same number as retrieval records of nodes with the same number, mark the total number of retrieval records of nodes with the same number as GMB, sort all retrieval records of nodes with the same number according to the order of record generation, mark the retrieval end time of the retrieval records of nodes with the same number that are adjacent and preceding after sorting as the end time of the preceding record, mark the retrieval start time of the retrieval records of nodes with the same number that are adjacent and preceding after sorting as the start time of the following record, calculate the time difference between the start time of the following record and the end time of the preceding record to obtain the interval of retrieval records of nodes with the same number, sum and average all retrieval record intervals to obtain the average interval of retrieval records of nodes with the same number, and mark it as RSZ, and use the formula The recommended storage value DTZ of the node with the same number is obtained, where b1 is the coefficient of the number of records retrieved by the same-numbered node, and b2 is the coefficient of the average interval of records retrieved by the same-numbered node. The node with the largest recommended storage value is marked as the first recommended storage node.

[0015] Furthermore, the node usage value of each storage node is periodically obtained, and based on the comparison result between the node usage value and its threshold, it is determined whether to migrate the customer data. Specifically, based on a preset cycle, the node usage value of each numbered storage node is obtained at each cycle node, and the node usage threshold is set. When the node usage value of a storage node is greater than or equal to the node usage threshold, the storage node is marked as a migration node, and the migration node directly migrates the subsequent stored customer data to the storage node associated with it.

[0016] Furthermore, the node usage value of the storage node is obtained in the following manner: obtain all data retrieval records of the storage node in the current cycle, mark the storage node as the central node, mark the remaining storage nodes associated with the storage node as associated nodes, and mark the total number of associated nodes as FAK. When the total number of data retrieval records of the central node is greater than the total number of data retrieval records of the associated nodes, increase the core usage count by one. When the total number of data retrieval records of the central node is less than or equal to the total number of data retrieval records of the associated nodes, do not perform corresponding processing, sum the core usage counts to obtain the total core usage count, and mark it as JTE. Mark the remaining storage capacity of the storage node as YHB. Use the formula The node usage value PWZ of the storage node is obtained, where c1 is the coefficient of the total number of associated nodes, c2 is the coefficient of the total number of core usage times, and c3 is the coefficient of the remaining storage capacity.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1. Set up the initial storage module and the retrieval storage module, conduct a comprehensive analysis of the customer's stored and retrieved data, determine the best storage location for each customer's data, rationalize the storage allocation of customer data, and avoid repeated transmission of customer data between storage nodes during the data retrieval process on the basis of facilitating the retrieval and storage of customer data;

[0019] 2. Set up a data migration module to periodically analyze the node usage value of each storage node, and optimize the storage rationality of each storage node while facilitating subsequent retrieval of customer data. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart for determining the storage location of customer data;

[0021] Figure 2A flowchart for determining whether to migrate customer data. DETAILED DESCRIPTION

[0022] Example 1

[0023] Reference Figure 1 , an artificial intelligence-based customer data storage and migration system, including an initial storage module and a retrieval storage module.

[0024] Initial storage module: When a client terminal device generates client data, the client data is assigned a number (the number is i, i=1, 2, ..., N, where N is a positive integer). Each client data corresponds to a unique number. The current location of the client terminal device is obtained, and the client data with the number is uploaded to the storage node closest to the location of the client terminal device. When the client data is first uploaded to the storage node after being generated, no data retrieval record is generated.

[0025] Retrieval storage module: When a customer terminal device needs to retrieve the customer data of this number, the corresponding storage node sends the customer data of this number to the customer terminal device. When the customer retrieves the customer data of this number, he can view, modify, delete and other operations. The corresponding operations will not change the number of the customer data. After the customer completes the retrieval of the customer data of this number, the data retrieval value of the customer data of this number is obtained, and the data retrieval threshold is set. The data retrieval threshold is the system-set threshold and can be adjusted accordingly according to actual conditions. When the data retrieval value of the customer data of this number is greater than or equal to the data retrieval threshold, the customer data of this number is marked as stable retrieval data, the recommended storage node for stable retrieval data is obtained, and the stable retrieval data is uploaded to the recommended storage node.

[0026] When the data retrieval value of the numbered data is less than the data retrieval threshold, the current location of the client terminal device is obtained, and the numbered client data is uploaded to the storage node closest to the location of the client terminal device.

[0027] When customer data is uploaded to a storage node, a data retrieval record is synchronously generated including the data number, node number (i.e., the storage node number to which the customer data is finally uploaded, each storage node corresponds to a unique number, j, j = 1, 2, ..., N, N is a positive integer), retrieval start time, and retrieval end time.

[0028] For example, when a customer terminal device makes a purchase, shopping data is generated, a number is assigned to the customer data, and the customer terminal device uploads the shopping data to the nearest storage node without generating a data retrieval record.

[0029] The data retrieval value of the numbered data is obtained in the following way: obtain all data retrieval records of the numbered data before the current system time, calculate the time difference between the retrieval end time and the retrieval start time of the data retrieval record to obtain the data retrieval duration, sum up all the data retrieval durations and take the average to obtain the average retrieval duration, and mark it as FTW, mark the total number of data retrieval records as MKH, and use the formula The data retrieval value PBC of the numbered data is obtained, where a1 is the average retrieval duration coefficient, a2 is the total number coefficient of data retrieval records, the value of a1 is 0.31, and the value of a2 is 0.97.

[0030] For example, when customers shop on an enterprise's Internet platform, consumption data is generated, which needs to be stored. When customers evaluate their shopping experience, evaluation data is generated, which needs to be stored.

[0031] The preferred storage node for stable data retrieval is obtained in the following way: obtain all data retrieval records of stable data retrieval before the current time of the system, mark the data retrieval records of nodes with the same number as the retrieval records of the same node, mark the total number of retrieval records of the same node as GMB, sort all retrieval records of the same node according to the order of record generation, mark the retrieval end time of the adjacent and previous retrieval records of the same node after sorting as the end time of the previous record, mark the retrieval start time of the adjacent and previous retrieval records of the same node after sorting as the start time of the later record, calculate the time difference between the start time of the later record and the end time of the previous record to obtain the interval of retrieval records of the same node, sum up all the retrieval record intervals and take the average to obtain the average interval of retrieval records of the same node, and mark it as RSZ, and use the formula The recommended storage value DTZ of the node with the same number is obtained, where b1 is the coefficient of the number of records retrieved by the same-numbered node, and b2 is the coefficient of the average interval of records retrieved by the same-numbered node. The value of b1 is 0.82, and the value of b2 is 0.69. The node with the largest recommended storage value is marked as the first recommended storage node.

[0032] Set up the initial storage module and the retrieval storage module, conduct a comprehensive analysis of customer storage and retrieved data, determine the best storage location for each customer data number, rationalize the storage allocation of customer data, and avoid repeated transmission of customer data between storage nodes during data retrieval while facilitating customer data retrieval and storage.

[0033] Example 2

[0034] Reference Figure 2, based on Example 1, further comprising a data migration module, wherein the data migration module: based on a preset cycle, obtains the node usage value of each numbered storage node at each cycle node, and sets a node usage threshold, where the node usage threshold is a system setting threshold and can be modified according to actual needs;

[0035] When the node usage value of a storage node is greater than or equal to the node usage threshold, the storage node is marked as a migration node, and the migration node directly migrates the subsequent stored customer data to the storage node associated with it;

[0036] When the node usage value of a storage node is less than the node usage threshold, no corresponding processing is performed;

[0037] The node usage value of the storage node is obtained in the following way: obtain all data retrieval records of the storage node in the current cycle, mark the storage node as the central node, mark the remaining storage nodes associated with the storage node as associated nodes (storage nodes that are directly connected in the blockchain are associated nodes), and mark the total number of associated nodes as FAK. When the total number of data retrieval records of the central node is greater than the total number of data retrieval records of the associated nodes, increase the core usage count by one. When the total number of data retrieval records of the central node is less than or equal to the total number of data retrieval records of the associated nodes, no corresponding processing is performed. The core usage count is summed to obtain the total core usage count, which is marked as JTE. The remaining storage capacity of the storage node is marked as YHB. The formula is used The node usage value PWZ of the storage node is obtained, where c1 is the coefficient of the total number of associated nodes, c2 is the coefficient of the total number of core usage times, and c3 is the coefficient of the remaining storage capacity. The value of c1 is 0.78, the value of c2 is 0.62, and the value of c3 is 0.97.

[0038] By setting up a data migration module, you can periodically analyze the node usage value of each storage node, and optimize the storage rationality of each storage node while facilitating subsequent retrieval of customer data.

[0039] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0040] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0041] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0042] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0043] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0044] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0045] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0046] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A customer data storage migration system based on artificial intelligence, characterized in that: Including initial storage module, storage retrieval module, and data migration module; The initial storage module is used to number and store customer data when the customer terminal device generates customer data; The storage module retrieves the customer data and determines the storage location of the customer data according to the comparison result between the retrieved value of the customer data and the threshold value after the customer terminal device retrieves the customer data; The data migration module periodically obtains the node usage value of each storage node, and determines whether to migrate the customer data according to the comparison result between the node usage value and its threshold value; After the client terminal device retrieves the client data, the storage location of the client data is determined based on the comparison result between the data retrieval value of the client data and its threshold value, specifically: when the client terminal device needs to retrieve the client data of the number, the corresponding storage node sends the client data of the number to the client terminal device, and after the client has finished retrieving the client data of the number, the data retrieval value of the client data of the number is obtained, and a data retrieval threshold is set; when the data retrieval value of the client data of the number is greater than or equal to the data retrieval threshold, the client data of the number is marked as stable retrieval data, the first recommended storage node for stable retrieval data is obtained, and the stable retrieval data is uploaded to the first recommended storage node; When the data retrieval value of the numbered data is less than the data retrieval threshold, the current location of the client terminal device is obtained, and the numbered client data is uploaded to the storage node closest to the client terminal device location; The data retrieval value of the customer data of this number is obtained in the following way: obtain all data retrieval records of the customer data of this number before the current time of the system, calculate the time difference between the retrieval end time and the retrieval start time of the data retrieval record to obtain the data retrieval duration, sum up all the data retrieval durations and take the average to obtain the average retrieval duration, and mark it as FTW, mark the total number of data retrieval records as MKH, and use the formula Get the data retrieval value PBC of the numbered data, where a1 is the average retrieval duration coefficient, and a2 is the total number coefficient of data retrieval records; The preferred storage node for stable data retrieval is obtained in the following way: obtain all data retrieval records of stable data retrieval before the current time of the system, mark the data retrieval records of nodes with the same number as the retrieval records of the same node, mark the total number of retrieval records of the same node as GMB, sort all retrieval records of the same node according to the order of record generation, mark the retrieval end time of the adjacent and previous retrieval records of the same node after sorting as the end time of the previous record, mark the retrieval start time of the adjacent and previous retrieval records of the same node after sorting as the start time of the later record, calculate the time difference between the start time of the later record and the end time of the previous record to obtain the interval of retrieval records of the same node, sum up all the retrieval record intervals and take the average to obtain the average interval of retrieval records of the same node, and mark it as RSZ, and use the formula Get the recommended storage value DTZ of the numbered node, where b1 is the coefficient of the number of records retrieved by the same-numbered node, and b2 is the coefficient of the average interval of records retrieved by the same-numbered node. Mark the node with the largest recommended storage value as the first recommended storage node. Periodically obtain the node usage value of each storage node, and determine whether to migrate customer data based on the comparison result between the node usage value and its threshold value. Specifically, based on a preset cycle, obtain the node usage value of each numbered storage node at each cycle node, set the node usage threshold value, and when the node usage value of a storage node is greater than or equal to the node usage threshold value, mark the storage node as a migration node, and the migration node directly migrates the subsequent stored customer data to the storage node associated with it; The node usage value of the storage node is obtained in the following way: obtain all data retrieval records of the storage node in the current cycle, mark the storage node as the central node, mark the remaining storage nodes associated with the storage node as associated nodes, and mark the total number of associated nodes as FAK. When the total number of data retrieval records of the central node is greater than the total number of data retrieval records of the associated nodes, increase the core usage count by one. When the total number of data retrieval records of the central node is less than or equal to the total number of data retrieval records of the associated nodes, no corresponding processing is performed. The core usage counts are summed to obtain the total core usage count, which is marked as JTE. The remaining storage capacity of the storage node is marked as YHB. The formula is used The node usage value PWZ of the storage node is obtained, where c1 is the coefficient of the total number of associated nodes, c2 is the coefficient of the total number of core usage times, and c3 is the coefficient of the remaining storage capacity.

2. According to the artificial intelligence-based customer data storage migration system of claim 1, it is characterized in that: When a client terminal device generates client data, a number is assigned to the client data, the current location of the client terminal device is obtained, and the client data is uploaded to the storage node closest to the client terminal device location. When the client data is first uploaded to the storage node after being generated, no data retrieval record is generated.

3. According to the artificial intelligence-based customer data storage migration system of claim 1, it is characterized in that: When customer data is uploaded to a storage node, a data retrieval record is generated synchronously including the data number, node number, retrieval start time, and retrieval end time.

Images