Intelligent data management systems and methods applying artificial intelligence and big data technologies

Abstract

The present invention relates to the field of data management technology, and specifically to an intelligent data management system and method using artificial intelligence and big data technology. The system includes a data management platform, which is communicatively connected with a collection module, a screening module, a classification module and a storage module: the collection module is used to process and analyze the data uploaded by the sending end; the screening module is used to receive the data transmitted by the collection module and perform screening processing: by calculating the regression distance between the position of the data on the feature layer of different classification dimensions and the center of different classification dimensions, it is judged whether the regression distance exceeds the preset distance value, if so, the data meets the classification requirements, and the data is transmitted to the classification module, the first classification score of the data is calculated based on the regression distance, and the first classification score is multiplied by the classification confidence predicted by the classification branch to obtain a second classification score, and the data is classified according to the second classification score; the data can be quickly classified to improve the efficiency of data management.

Description

Technical Field

[0001] The present invention relates to the field of data management technology, and in particular to an intelligent data management system and method using artificial intelligence and big data technologies. Background Art

[0002] With the rapid development of the information age, data has become an important resource in modern society. Faced with massive amounts of data, how to effectively collect, process, analyze and utilize these data has become a major challenge facing enterprises. Artificial intelligence and big data technology, as two hot technologies in the current information technology field, provide strong technical support for intelligent data management.

[0003] The continuous development of artificial intelligence technology enables machines to simulate human intelligent behavior and have the ability to learn, reason, and make decisions. In data management, artificial intelligence technology can help the system realize automatic processing and analysis of data, reduce human intervention, and improve processing efficiency. For example, through machine learning algorithms, the system can automatically identify and classify data, and realize intelligent labeling and classification of data. In addition, deep learning technology can also be applied to data feature extraction and pattern recognition, providing strong support for in-depth analysis of data.

[0004] On the other hand, big data technology provides powerful storage and computing capabilities for data management. Big data technology can efficiently store and query massive amounts of data and support real-time processing and analysis of data. Through the big data platform, distributed processing and parallel computing of data can be achieved to improve the speed and efficiency of data processing. At the same time, big data technology can also conduct in-depth data mining, discover the potential value and rules in the data, and provide support for decision-making.

[0005] Although some existing data management systems have tried to apply artificial intelligence and big data technologies, there are still some challenges and problems. For example, how to design efficient algorithms to cope with complex data processing tasks is a problem we need to face.

[0006] In view of the shortcomings of existing technologies, there is an urgent need for an intelligent data management system and method that uses artificial intelligence and big data technologies to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to provide an intelligent data management system and method using artificial intelligence and big data technology to solve the technical problem that there is no effective data management system to improve data management efficiency in the existing solutions when faced with massive data.

[0008] The purpose of the present invention can be achieved through the following technical solutions:

[0009] An intelligent data management system that uses artificial intelligence and big data technologies. The system includes a data management platform, which is connected to a collection module, a screening module, a classification module, and a storage module:

[0010] The acquisition module is used to process and analyze the data uploaded by the sender: the management cycle is generated from the time when the sender generates the upload data request as the starting point, the risk coefficient and data transmission coefficient of the sender within the management cycle are obtained, and the data acquisition requirements are determined based on the risk coefficient and data transmission coefficient. If so, the data sent by the sender is received and transmitted to the screening module. If not, the data sent by the sender is rejected.

[0011] The screening module is used to receive the data transmitted by the acquisition module and perform screening processing: by calculating the regression distance between the position of the data on different classification dimension feature layers and the center of different classification dimensions, where the classification dimension feature layer is generated based on the similarity feature value of the data corresponding to different classifications, and judging whether the regression distance exceeds the preset distance value. If so, the data meets the classification requirements and the data is transmitted to the classification module. If not, the data is refused to be transmitted to the classification module, and the data management platform deletes the data, where the preset distance value is the sum of the minimum distance values ​​of the data on each classification dimension feature layer when the classification requirements are met;

[0012] The classification module is used to receive the data transmitted by the screening module and perform classification processing: calculate the first classification score of the data based on the regression distance, multiply the first classification score by the classification confidence predicted by the classification branch to obtain a second classification score, and classify the data according to the second classification score;

[0013] The storage module is used to store the classified data.

[0014] Furthermore, obtaining the risk factor of the sender within the management period specifically includes the following process:

[0015] Obtain and parse the risk indicator information of the sender within the management cycle to obtain the data encryption strength , Packet loss rate , Delay time And the connection stability rate ;

[0016] Encryption strength of data , Packet loss rate , Delay time And the connection stability rate Substitute the risk coefficient calculation formula to get the risk coefficient , the risk factor calculation formula is as follows:

[0017] ;

[0018] in, , , , is the weight, It is a constant correction coefficient, and its specific value can be adjusted by the user according to the size of the data.

[0019] Furthermore, obtaining the data transmission coefficient of the sender during the management period specifically includes the following process:

[0020] The upload speed of data transmission at the sending end within the management period is detected in real time, the upload speed value corresponding to the detection point is obtained, and an upload speed curve is drawn according to the upload speed value corresponding to the detection point. The integral value of the closed space is calculated based on the preset upload speed curve and the closed space enclosed by the upload speed curve, and the integral value is recorded as the data transmission coefficient.

[0021] Furthermore, judging whether the data collection requirements are met based on the risk factor and the data transmission factor specifically includes the following process:

[0022] The weight corresponding to the risk coefficient and the weight corresponding to the data transmission coefficient are obtained through the storage module; the product of the risk coefficient and the weight corresponding to the risk coefficient, and the product of the data transmission coefficient and the weight corresponding to the data transmission coefficient are calculated, and the sum of the two products is recorded as the comparison value; it is determined whether the comparison value is greater than the preset collection requirement value, if so, it is determined that the data meets the collection requirements, if not, it is determined that the data does not meet the collection requirements.

[0023] Furthermore, calculating the first classification score of the data based on the regression distance specifically includes the following process:

[0024] Get the regression distance , , , , where the number of layers of the classification dimension feature layer is 4, , , , They are the regression distances of the feature layer of the data distance classification dimension;

[0025] Calculate the first classification score based on the data proximity calculation formula , the calculation formula of the data closeness is as follows:

[0026] ;

[0027] in, Represents the regression distance and regression distance The minimum value of Take the regression distance and regression distance The minimum value of Represents the regression distance and regression distance The maximum value of Take the regression distance and regression distance The maximum value of .

[0028] Furthermore, the first classification score is multiplied by the classification confidence of the classification branch prediction to obtain a second classification score, and classifying the data according to the second classification score specifically includes the following process:

[0029] based on Get the classification confidence of the classification branch prediction;

[0030] Multiply the first classification score by the classification confidence predicted by the classification branch to obtain a second classification score;

[0031] Classify the data based on the second classification score:

[0032] Create a benchmark cluster And its corresponding benchmark score, set the range radius of the benchmark cluster , construct neighborhoods in the plane space, scatter the data in the plane space according to the size of the second score, and place the second category score in the benchmark cluster The data in the neighborhood is stored in the collection In the example, the second category scores that are not in the benchmark cluster The data in the neighborhood is stored in the collection middle.

[0033] Furthermore, we create benchmark clusters based on the k-means algorithm. .

[0034] An intelligent data management method using artificial intelligence and big data technology, the method comprising the following steps:

[0035] S1: Process and analyze the data uploaded by the sender: Generate a management cycle starting from the time when the sender generates a request to upload data, obtain the risk coefficient and data transmission coefficient of the sender within the management cycle, and determine whether the data collection requirements are met based on the risk coefficient and data transmission coefficient. If so, receive the data sent by the sender and transmit the data to the screening module. If not, refuse to receive the data sent by the sender;

[0036] S2: Receive the transmitted data and perform screening processing: by calculating the regression distance between the position of the data on different classification dimension feature layers and the center of different classification dimensions, where the classification dimension feature layer is generated based on the similarity feature value of the data corresponding to different classifications, determine whether the regression distance exceeds the preset distance value, if so, the data meets the classification requirements, and the data is transmitted to the classification module, if not, refuse to transmit the data to the classification module, and the data management platform deletes the data, where the preset distance value is the sum of the minimum distance values ​​of the data on each classification dimension feature layer when the classification requirements are met;

[0037] S3: receiving data transmitted by the screening module and performing classification processing: calculating a first classification score of the data based on the regression distance, and multiplying the first classification score by the classification confidence predicted by the classification branch to obtain a second classification score, and classifying the data according to the second classification score;

[0038] The classified data is stored.

[0039] Compared with the existing solutions, the present invention achieves the following beneficial effects:

[0040] On the one hand, the system of the present invention uses an acquisition module to process and analyze the data uploaded by the sender: a management cycle is generated starting from the moment when the sender generates a request to upload data, and the risk coefficient and data transmission coefficient of the sender within the management cycle are obtained. According to the risk coefficient and the data transmission coefficient, it is judged whether the data acquisition requirements are met. If so, the data sent by the sender is received and transmitted to the screening module. If not, the data sent by the sender is refused to be received. When faced with massive and complex data, some data that does not meet the requirements can be intercepted through the risk coefficient and the data transmission coefficient, thereby saving the computing power of the data management system and improving the management efficiency of the data management system.

[0041] On the other hand, the screening module is used to receive the data transmitted by the acquisition module and perform screening processing: by calculating the regression distance of each position of the data on different classification dimension feature layers from the center of different classification dimensions, wherein the classification dimension feature layer is generated based on the similarity feature value of the data corresponding to different classifications, and judging whether the regression distance exceeds the preset distance value, if so, the data meets the classification requirements and the data is transmitted to the classification module, if not, the data is refused to be transmitted to the classification module, and the data management platform deletes the data, wherein the preset distance value is the sum of the minimum distance values ​​of the data on each classification dimension feature layer when the classification requirements are met. Based on the fact that the screening module receives the data transmitted by the acquisition module and performs screening processing, it can quickly realize preliminary processing of the data, which helps to save data processing time.

[0042] The classification module is used to receive the data transmitted by the screening module and perform classification processing: calculate the first classification score of the data based on the regression distance, multiply the first classification score by the classification confidence predicted by the classification branch to obtain a second classification score, and classify the data according to the second classification score;

[0043] The storage module is used to store the classified data.

[0044] Finally, the classification and storage modules can quickly classify data and improve the efficiency of data management. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0046] Figure 1 This is a system block diagram of an intelligent data management system that applies artificial intelligence and big data technologies according to an embodiment of the present invention;

[0047] Figure 2 It is a workflow diagram of an intelligent data management system using artificial intelligence and big data technologies according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] In addition, the described features, structures or characteristics may be combined in one or more example embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the example embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, steps, etc. may be adopted. In other cases, well-known structures, methods, implementations or operations are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0050] This embodiment provides an intelligent data management system that uses artificial intelligence and big data technologies. Figure 1is a system block diagram of an intelligent data management system using artificial intelligence and big data technology according to an embodiment of the present invention. Figure 1 As shown, the system includes a data management platform, which is communicatively connected to a collection module, a screening module, a classification module and a storage module:

[0051] The acquisition module is used to process and analyze the data uploaded by the sender: the management cycle is generated from the time when the sender generates the upload data request as the starting point, the risk coefficient and data transmission coefficient of the sender within the management cycle are obtained, and the data acquisition requirements are determined based on the risk coefficient and data transmission coefficient. If so, the data sent by the sender is received and transmitted to the screening module. If not, the data sent by the sender is rejected.

[0052] The screening module is used to receive the data transmitted by the acquisition module and perform screening processing: by calculating the regression distance of each position of the data on different classification dimension feature layers from the center of different classification dimensions, where the classification dimension feature layer is generated based on the similarity feature value of the data corresponding to different classifications, and judging whether the regression distance exceeds the preset distance value. If so, the data meets the classification requirements and the data is transmitted to the classification module. If not, the data is refused to be transmitted to the classification module, and the data management platform deletes the data, where the preset distance value is the sum of the minimum distance values ​​of the data on each classification dimension feature layer when the classification requirements are met;

[0053] The classification module is used to receive the data transmitted by the screening module and perform classification processing: calculate the first classification score of the data based on the regression distance, multiply the first classification score by the classification confidence predicted by the classification branch to obtain a second classification score, and classify the data according to the second classification score;

[0054] The storage module is used to store the classified data.

[0055] In summary, the system of the present invention uses an acquisition module to process and analyze the data uploaded by the sender, and determines whether the data acquisition requirements are met based on the risk coefficient and the data transmission coefficient. If so, the data sent by the sender is received and transmitted to the screening module, and the data transmitted by the acquisition module is received and screened: by calculating the regression distance of each position of the data on the feature layer of different classification dimensions from the center of different classification dimensions, it is determined whether the regression distance exceeds the preset distance value. If so, the data meets the classification requirements, and the data is transmitted to the classification module, and the data transmitted by the screening module is received and classified: the first classification score of the data is calculated based on the regression distance, and the first classification score is multiplied by the classification confidence predicted by the classification branch to obtain a second classification score, and the data is classified according to the second classification score; the storage module is used to store the classified data, and can quickly classify the data, thereby improving the efficiency of data management.

[0056] In some embodiments, Figure 2 is a workflow diagram of an intelligent data management system using artificial intelligence and big data technology according to an embodiment of the present invention, such as Figure 2 As shown, obtaining the risk factor of the sender within the management period specifically includes the following steps:

[0057] Step S201: Obtain and parse the risk indicator information of the sender within the management period to obtain the data encryption strength , Packet loss rate , Delay time And the connection stability rate ; Among them, the connection stability rate The acquisition process may include:

[0058] Method 1: Calculation based on the number of connection interruptions

[0059] Record the number of connection interruptions:

[0060] During the management period, record the number of connection interruptions. This can be achieved by monitoring network logs, system logs, or dedicated connection monitoring tools.

[0061] To record the total number of connection attempts:

[0062] Also during the above time period, all attempts to establish a connection are recorded. This includes both successful connections and connections that failed to be established for various reasons.

[0063] Calculate the connection stability rate:

[0064] Connection stability rate = (total number of connection attempts - number of connection interruptions) / total number of connection attempts × 100%.

[0065] Method 2: Calculation based on connection duration

[0066] Record total connection time:

[0067] During the management period, the total duration (in milliseconds) of all successful connections is recorded.

[0068] Record the duration of connection interruption:

[0069] Also within the above management period, record the total duration of all connection interruptions.

[0070] Calculate the connection stability rate:

[0071] Connection stability rate = (total connection time - connection interruption time) / total connection time × 100%.

[0072] Step S202: Encryption strength of data , Packet loss rate , Delay time And the connection stability rate Substitute the risk coefficient calculation formula to get the risk coefficient , the risk factor calculation formula is as follows:

[0073] ;

[0074] in, , , , is the weight, It is a constant correction coefficient, and its specific value can be adjusted by the user according to the size of the data.

[0075] In some embodiments, obtaining the data transmission coefficient of the sender during the management period specifically includes the following process:

[0076] The upload speed of data transmission at the sending end within the management period is detected in real time, the upload speed value corresponding to the detection point is obtained, and an upload speed curve is drawn according to the upload speed value corresponding to the detection point. The integral value of the closed space is calculated based on the preset upload speed curve and the closed space enclosed by the upload speed curve, and the integral value is recorded as the data transmission coefficient.

[0077] Furthermore, judging whether the data collection requirements are met based on the risk factor and the data transmission factor specifically includes the following process:

[0078] The weight corresponding to the risk coefficient and the weight corresponding to the data transmission coefficient are obtained through the storage module; the product of the risk coefficient and the weight corresponding to the risk coefficient, and the product of the data transmission coefficient and the weight corresponding to the data transmission coefficient are calculated, and the sum of the two products is recorded as the comparison value; it is determined whether the comparison value is greater than the preset collection requirement value, if so, it is determined that the data meets the collection requirements, if not, it is determined that the data does not meet the collection requirements.

[0079] In some embodiments, calculating the first classification score of the data based on the regression distance specifically includes the following process:

[0080] Get the regression distance , , , , where the number of layers of the classification dimension feature layer is 4, , , , They are the regression distances of the feature layer of the data distance classification dimension;

[0081] Calculate the first classification score based on the data proximity calculation formula , the calculation formula of the data closeness is as follows:

[0082] ;

[0083] in, Represents the regression distance and regression distance The minimum value of Take the regression distance and regression distance The minimum value of Represents the regression distance and regression distance The maximum value of Take the regression distance and regression distance The maximum value of .

[0084] In some embodiments, the first classification score is multiplied by the classification confidence of the classification branch prediction to obtain a second classification score, and classifying the data according to the second classification score specifically includes the following process:

[0085] based on (Support Vector Machine: Support Vector Machine) Get the classification confidence of the classification branch prediction: It is a classification model based on statistical learning theory. It separates samples of different categories by finding an optimal hyperplane. In SVM, branch prediction is usually done by calculating the distance from the input sample to the hyperplane, and samples farther from the hyperplane usually have higher confidence.

[0086] Multiply the first classification score by the classification confidence predicted by the classification branch to obtain a second classification score;

[0087] Classify the data based on the second classification score:

[0088] Create a benchmark cluster And its corresponding benchmark score, set the range radius of the benchmark cluster , construct neighborhoods in the plane space, scatter the data in the plane space according to the size of the second score, and place the second category score in the benchmark cluster The data in the neighborhood is stored in the collection In the example, the second category scores that are not in the benchmark cluster The data in the neighborhood is stored in the collection middle.

[0089] Furthermore, we create benchmark clusters based on the k-means algorithm. :

[0090] The K-means algorithm is an unsupervised learning clustering algorithm used to divide data into K clusters. The following are the basic steps to create a benchmark cluster based on the K-means algorithm:

[0091] initialization:

[0092] Select K initial centroids. This can be done by randomly selecting K data points in the dataset as the initial centroids, or by some more sophisticated heuristic method (such as K-means++).

[0093] Allocate clusters:

[0094] For each data point in the dataset, calculate its distance to each centroid (e.g., using Euclidean distance).

[0095] Assign a data point to the cluster whose centroid is closest to it.

[0096] Update the centroid:

[0097] For each cluster, calculate the mean (or median) of all the data points within it and set that mean as the new centroid.

[0098] Iteration:

[0099] Repeat steps 2 and 3 until a certain stopping condition is met (for example, the change in the centroid is less than a certain threshold, or a preset maximum number of iterations is reached).

[0100] Output:

[0101] The final K centroids and their corresponding clusters are the benchmark clusters based on the K-means algorithm.

[0102] The present invention also provides an intelligent data management method using artificial intelligence and big data technology, the method comprising the following steps:

[0103] S1: Process and analyze the data uploaded by the sender: Generate a management cycle starting from the time when the sender generates a request to upload data, obtain the risk coefficient and data transmission coefficient of the sender within the management cycle, and determine whether the data collection requirements are met based on the risk coefficient and data transmission coefficient. If so, receive the data sent by the sender and transmit the data to the screening module. If not, refuse to receive the data sent by the sender;

[0104] S2: Receive the transmitted data and perform screening processing: by calculating the regression distance between the position of the data on different classification dimension feature layers and the center of different classification dimensions, where the classification dimension feature layer is generated based on the similarity feature value of the data corresponding to different classifications, determine whether the regression distance exceeds the preset distance value, if so, the data meets the classification requirements, and the data is transmitted to the classification module, if not, refuse to transmit the data to the classification module, and the data management platform deletes the data, where the preset distance value is the sum of the minimum distance values ​​of the data on each classification dimension feature layer when the classification requirements are met;

[0105] S3: receiving data transmitted by the screening module and performing classification processing: calculating a first classification score of the data based on the regression distance, and multiplying the first classification score by the classification confidence predicted by the classification branch to obtain a second classification score, and classifying the data according to the second classification score;

[0106] The classified data is stored.

[0107] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.

[0108] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0109] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0110] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only some logical function divisions. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0111] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0112] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0113] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. Intelligent data management system using artificial intelligence and big data technology, It is characterized in that The system includes a data management platform, which is communicatively connected to a collection module, a screening module, a classification module and a storage module; The acquisition module is used to process and analyze the data uploaded by the sender: the management cycle is generated from the time when the sender generates the upload data request as the starting point, the risk coefficient and data transmission coefficient of the sender within the management cycle are obtained, and the data collection requirements are determined based on the risk coefficient and data transmission coefficient. If so, the data sent by the sender is received and transmitted to the screening module. If not, the data sent by the sender is rejected. The specific process of obtaining the risk coefficient of the sender within the management cycle includes the following: Obtain and parse the risk indicator information of the sender within the management cycle to obtain the data encryption strength , Packet loss rate , Delay time And the connection stability rate ; Encryption strength of data , Packet loss rate , Delay time And the connection stability rate Substitute the risk coefficient calculation formula to get the risk coefficient , the risk factor calculation formula is as follows: ； in, , , is the weight, It is a constant correction coefficient, and its specific value can be adjusted and set by the user according to the size of the data; The specific process of obtaining the data transmission coefficient of the sender during the management period includes the following steps: Perform real-time detection on the upload speed of data transmission at the sending end within the management period, obtain the upload speed value corresponding to the detection point, draw an upload speed curve according to the upload speed value corresponding to the detection point, calculate the integral value of the closed space based on the preset upload speed curve and the closed space enclosed by the upload speed curve, and record the integral value as the data transmission coefficient; The screening module is used to receive the data transmitted by the acquisition module and perform screening processing: by calculating the regression distance between the position of the data on different classification dimension feature layers and the center of different classification dimensions, where the classification dimension feature layer is generated based on the similarity feature value of the data corresponding to different classifications, and judging whether the regression distance exceeds the preset distance value. If so, the data meets the classification requirements and the data is transmitted to the classification module. If not, the data is refused to be transmitted to the classification module, and the data management platform deletes the data, where the preset distance value is the sum of the minimum distance values ​​of the data on each classification dimension feature layer when the classification requirements are met; The classification module is used to receive the data transmitted by the screening module and perform classification processing: the first classification score of the data is calculated based on the regression distance, and the first classification score is multiplied by the classification confidence predicted by the classification branch to obtain the second classification score, and the data is classified according to the second classification score: based on Get the classification confidence of the classification branch prediction; Multiply the first classification score by the classification confidence predicted by the classification branch to obtain a second classification score; Classify the data based on the second classification score: Create a benchmark cluster And its corresponding benchmark score, set the range radius of the benchmark cluster , construct neighborhoods in the plane space, scatter the data in the plane space according to the size of the second score, and place the second category score in the benchmark cluster The data in the neighborhood is stored in the collection In the example, the second category scores that are not in the benchmark cluster The data in the neighborhood is stored in the collection middle; The storage module is used to store the classified data.

2. The intelligent data management system using artificial intelligence and big data technology according to claim 1, It is characterized in that Judging whether the data collection requirements are met based on the risk factor and data transmission factor specifically includes the following processes: The weight corresponding to the risk coefficient and the weight corresponding to the data transmission coefficient are obtained through the storage module; the product of the risk coefficient and the weight corresponding to the risk coefficient, and the product of the data transmission coefficient and the weight corresponding to the data transmission coefficient are calculated, and the sum of the two products is recorded as the comparison value; it is determined whether the comparison value is greater than the preset collection requirement value, if so, it is determined that the data meets the collection requirements, if not, it is determined that the data does not meet the collection requirements.

3. The intelligent data management system using artificial intelligence and big data technology according to claim 1, It is characterized in that The first classification score of the data calculated based on the regression distance specifically includes the following process: Get the regression distance , , , , where the number of layers of the classification dimension feature layer is 4, , , , They are the regression distances of the feature layer of the data distance classification dimension; Calculate the first classification score based on the data proximity calculation formula , the calculation formula of the data closeness is as follows: ； in, Represents the regression distance and regression distance The minimum value of Take the regression distance and regression distance The minimum value of Represents the regression distance and regression distance The maximum value of Take the regression distance and regression distance The maximum value of .

4. The intelligent data management system using artificial intelligence and big data technology according to claim 1, It is characterized in that Create benchmark clusters based on the k-means algorithm .

5. Intelligent data management methods using artificial intelligence and big data technologies, It is characterized in that The method comprises the following steps: S1: Process and analyze the data uploaded by the sender: Generate a management cycle starting from the time when the sender generates a request to upload data, obtain the risk coefficient and data transmission coefficient of the sender within the management cycle, and judge whether the data collection requirements are met according to the risk coefficient and data transmission coefficient. If so, receive the data sent by the sender and transmit the data to the screening module. If not, refuse to receive the data sent by the sender; wherein, obtaining the risk coefficient of the sender within the management cycle specifically includes the following process: Obtain and parse the risk indicator information of the sender within the management cycle to obtain the data encryption strength , Packet loss rate , Delay time And the connection stability rate ; Encryption strength of data , Packet loss rate , Delay time And the connection stability rate Substitute the risk coefficient calculation formula to get the risk coefficient , the risk factor calculation formula is as follows: ； in, , , is the weight, It is a constant correction coefficient, and its specific value can be adjusted and set by the user according to the size of the data; The specific process of obtaining the data transmission coefficient of the sender during the management period includes the following steps: Perform real-time detection on the upload speed of data transmission at the sending end within the management period, obtain the upload speed value corresponding to the detection point, draw an upload speed curve according to the upload speed value corresponding to the detection point, calculate the integral value of the closed space based on the preset upload speed curve and the closed space enclosed by the upload speed curve, and record the integral value as the data transmission coefficient; S2: Receive the transmitted data and perform screening processing: by calculating the regression distance between the position of the data on different classification dimension feature layers and the center of different classification dimensions, where the classification dimension feature layer is generated based on the similarity feature value of the data corresponding to different classifications, determine whether the regression distance exceeds the preset distance value, if so, the data meets the classification requirements, and the data is transmitted to the classification module, if not, refuse to transmit the data to the classification module, and the data management platform deletes the data, where the preset distance value is the sum of the minimum distance values ​​of the data on each classification dimension feature layer when the classification requirements are met; S3: Receive the data transmitted by the screening module and perform classification processing: calculate the first classification score of the data based on the regression distance, multiply the first classification score by the classification confidence predicted by the classification branch to obtain the second classification score, and classify the data according to the second classification score: based on Get the classification confidence of the classification branch prediction; Multiply the first classification score by the classification confidence predicted by the classification branch to obtain a second classification score; Classify the data based on the second classification score: Create a benchmark cluster And its corresponding benchmark score, set the range radius of the benchmark cluster , construct neighborhoods in the plane space, scatter the data in the plane space according to the size of the second score, and place the second category score in the benchmark cluster The data in the neighborhood is stored in the collection In the example, the second category scores that are not in the benchmark cluster The data in the neighborhood is stored in the collection middle; The classified data is stored.

Images