Data maintenance and transmission method based on artificial intelligence and big data

Abstract

The invention discloses a data maintenance and transmission method based on artificial intelligence and big data, and relates to the technical field of artificial intelligence and big data. According to the method, a to-be-extracted processing data pool is set, multi-source processing data is integrated and sent to the to-be-extracted processing data pool for data feature dimension retrieval, a processing data feature dimension set is obtained, and all corresponding demand feature data in the to-be-extracted processing data pool are extracted according to the processing data feature dimension set; performing data feature dimension verification on the demand feature data to obtain similar demand feature data; obtaining a reserved data feature dimension set, then obtaining a corresponding data feature similarity value, carrying out data tracing to obtain accurate demand feature data, carrying out processing to obtain presentation data, obtaining a presentation terminal network, obtaining a transmission data path corresponding to the presentation data according to the presentation terminal network, and then transmitting the presentation data according to the transmission data path; and the stability of data and the security of transmission are effectively maintained.

Description

Technical Field

[0001] The present invention relates to the technical fields of artificial intelligence and big data, and specifically to a data maintenance and transmission method based on artificial intelligence and big data. Background Art

[0002] Artificial intelligence refers to the ability to use computers and machines to simulate human intelligence, and big data refers to a large collection of data with huge scale, diverse types, and fast processing speed; big data provides a large amount of data support for artificial intelligence, enabling artificial intelligence systems to analyze and predict more accurately; In the current development of technology, a large amount of demand data needs to be maintained and transmitted during the processes of processing, analyzing, predicting, presenting, and sharing. For example, usually, a large amount of complex demand data needs to be transmitted and maintained during processing, analyzing, predicting, presenting, and sharing; however, in the prior art, when there are a large number of similar feature data in a large amount of complex demand data, the existing processing terminals have the problem of being unable to distinguish abnormal similar feature data, and cannot maintain the stability of the demand data, resulting in inaccurate processed data; furthermore, when transmitting the processed presentation data to one or more terminals for presentation and sharing, there are problems such as data loss and intrusion during the transmission process; therefore, to solve the above problems, the present invention provides a data maintenance and transmission method based on artificial intelligence and big data. Summary of the Invention

[0003] To solve the above technical problems, the present invention provides a data maintenance and transmission method based on artificial intelligence and big data; The object of the present invention can be achieved through the following technical solutions: A data maintenance and transmission method based on artificial intelligence and big data, the method comprising the following steps: Step S1: Set a processing data pool to be extracted in the processing terminal, integrate multi-source processing data and send it to the processing data pool to be extracted for data feature dimension retrieval, obtain a set of processing data feature dimensions, and then extract all corresponding demand feature data in the processing data pool to be extracted according to the set of processing data feature dimensions; Step S2: Set a feature verification mechanism to verify the data feature dimensions of the demand feature data to obtain similar demand feature data; obtain a set of reserved data feature dimensions, and obtain the data feature similarity values corresponding to the similar demand feature data according to the set of reserved data feature dimensions; Step S3: Trace the similar demand feature data according to the data feature similarity values to obtain accurate demand feature data; Step S4: Process the accurate demand feature data to obtain presentation data, and obtain a presentation terminal network, and obtain the transmission data path corresponding to the presentation data according to the presentation terminal network; Step S5: Transmit the presented data according to the transmission data path.

[0004] Further, the process of obtaining the set of processed data feature dimensions includes: The to-be-extracted processed data pool is used to receive and store multi-source processed data; an edge retrieval unit is set in the to-be-extracted processed data pool for retrieving the data feature dimensions of the multi-source processed data; a unique feature identifier corresponding to the data feature dimension is set in the edge retrieval unit; Based on user requirements and big data, multi-source processed data is obtained and sent to the to-be-extracted processed data pool; while receiving the multi-source processed data, the edge retrieval unit retrieves the data feature dimensions of all the processed data in the multi-source processed data according to the unique feature identifier, obtains all the data feature dimensions corresponding to each processed data, and combines them to generate the corresponding set of processed data feature dimensions.

[0005] Further, the process of obtaining the demand feature data includes: The set of demand data feature dimensions is obtained according to user requirements, and the set of demand data feature dimensions is matched with the set of processed data feature dimensions corresponding to each processed data in the multi-source processed data. If the set of processed data feature dimensions contains the set of demand data feature dimensions, the corresponding processed data is extracted from the to-be-extracted processed data pool and demand feature data is generated; otherwise, no processing is performed.

[0006] Further, the process of setting the feature verification mechanism includes: The difference sets between the sets of processed data feature dimensions corresponding to each demand feature data and the set of demand data feature dimensions are obtained, denoted as the set of demand data feature dimension differences, and the demand feature data corresponding to the set of demand data feature dimension differences is generated into similar demand feature data and mapped with the set of demand data feature dimension differences, thereby obtaining the feature verification mechanism.

[0007] Further, the process of obtaining the data feature similarity value includes: The reserved data feature dimension set is obtained according to user requirements, and the weights corresponding to each data feature dimension in the reserved data feature dimension set are set, denoted as the reserved weights; then the intersections and differences between each set of demand data feature dimension differences and the reserved data feature dimension set are obtained, and are respectively denoted as the set of normal demand data feature dimensions and the set of abnormal demand data feature dimensions; The weights corresponding to each data feature dimension in the set of abnormal demand data feature dimensions are set to -1, then the total abnormal weights corresponding to each set of abnormal demand data feature dimensions are obtained, and the total reserved weights corresponding to each set of normal demand data feature dimensions are obtained according to the reserved weights.

[0008] Furthermore, the abnormal total weight and reserved total weight corresponding to the similar demand feature data are obtained based on the normal feature dimension set of demand data and the abnormal feature dimension set of demand data corresponding to the similar demand feature data, and the sum of the abnormal total weight and the reserved total weight is obtained, and then marked as the data feature similarity value corresponding to the similar demand feature data.

[0009] Furthermore, the process of obtaining the precise demand feature data includes: Set a data feature similarity value threshold and compare it with the data feature similarity value, extract and integrate similar demand feature data corresponding to data feature similarity values ​​less than or equal to the data feature similarity value threshold to obtain a similar demand feature data set; otherwise, remove the corresponding similar demand feature data; Track the data sources corresponding to each similar demand feature data in the similar demand feature data set to obtain the data sources corresponding to the similar demand feature data; remove the similar demand feature data corresponding to the data sources that cannot be tracked; The data source set is obtained according to user needs, and the tracked data source is matched with the data source set. If the match is not successful, the corresponding similar demand feature data will be eliminated, otherwise, no processing will be done; then the multi-source processed data will be generated into accurate demand feature data.

[0010] Furthermore, the process of generating the transmission data path includes: Acquire the receiving terminal connected to the processing terminal and the location area corresponding to the receiving terminal, connect the receiving terminals corresponding to the same location area to generate a regional terminal data network, and then connect the regional terminal data networks to generate a presentation terminal network; According to the location area, the data transmission mode corresponding to the regional terminal data network is set, and different data encryption modes are set in the data transmission mode; then, the data transmission modes in the terminal grid are distributedly connected to generate a transmission data network; The precise demand feature data is sent to the processing terminal for processing to obtain the presentation data, and the presentation data is transmitted in a corresponding data transmission mode in the transmission data network according to different receiving terminals, and then the processing terminal reaches the corresponding receiving terminal of the presentation terminal network through the corresponding data transmission mode to generate a transmission data path.

[0011] The presentation data is encrypted according to the corresponding data encryption method to obtain the encrypted presentation data; and the encrypted presentation data is transmitted to the receiving terminal through the transmission data path.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: By setting a data pool to be extracted and processed in the processing terminal, integrating multi-source processing data and sending it to the data pool to be extracted and processed for data feature dimension retrieval, a set of processing data feature dimensions is obtained. Then, all corresponding required feature data in the data pool to be extracted and processed is extracted according to the set of processing data feature dimensions; A feature verification mechanism is set to verify the data feature dimensions of the required feature data to obtain similar required feature data; A reserved data feature dimension set is obtained, and a data feature similarity value corresponding to the similar required feature data is obtained according to the reserved data feature dimension set; Data traceability is performed on the similar required feature data according to the data feature similarity value to obtain accurate required feature data; The accurate required feature data is processed to obtain presentation data, and a presentation terminal network is obtained. According to the presentation terminal network, a transmission data path corresponding to the presentation data is obtained; The presentation data is transmitted according to the transmission data path; effectively maintaining the stability of the data and the security of the transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings.

[0014] Figure 1 It is a flowchart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings.

[0016] Embodiment 1

[0017] As Figure 1 shown, a data maintenance and transmission method based on artificial intelligence and big data, the method includes the following steps: Step S1: Set a data pool to be extracted and processed, integrate multi-source processing data and send it to the data pool to be extracted and processed for data feature dimension retrieval, obtain a set of processing data feature dimensions, and then extract all corresponding required feature data in the data pool to be extracted and processed according to the set of processing data feature dimensions; Step S2: Set up a feature verification mechanism to perform feature verification on the requirement feature data to obtain similar requirement feature data; obtain the reserved data feature dimension set, and obtain the data feature similarity values corresponding to the similar requirement feature data according to the reserved data feature dimension set; Step S3: Perform data traceability on the similar requirement feature data according to the data feature similarity values to obtain accurate requirement feature data; Step S4: Process the accurate requirement feature data to obtain presentation data, and obtain the transmission data network and the presentation terminal network. Obtain the transmission data path corresponding to the presentation data according to the presentation terminal network; Step S5: Transmit the presentation data according to the transmission data path.

[0018] Embodiment 2

[0019] This embodiment is a further limitation of Embodiment 1, and the step S1 is implemented through the following process: Set up an edge retrieval unit, which is wirelessly communicatively connected to the processing terminal for receiving and storing multi-source processing data; Set up an edge retrieval unit in the to-be-extracted processing data pool for performing data feature dimension retrieval on the multi-source processing data to obtain the processing data feature dimension set; Obtain multi-source processing data through user requirements and based on big data, and send it to the to-be-extracted processing data pool; while receiving the multi-source processing data, the edge retrieval unit performs data feature dimension retrieval on the multi-source processing data to obtain the processing data feature dimension set of the corresponding processing data in the multi-source processing data; It should be further noted that in the specific implementation process, the processing terminal includes but is not limited to servers, edge computing, cloud computing, etc.; the user requirements include but are not limited to enterprise user requirements, power plant user requirements, etc.; further, the multi-source processing data includes but is not limited to processing data from two sources, processing data from several sources, etc.; Further, the retrieval process of the edge retrieval unit includes: Set the unique feature identifier corresponding to the data feature dimension in the edge retrieval unit; perform data feature dimension retrieval on all the processing data in the multi-source processing data according to the unique feature identifier to obtain all the data feature dimensions corresponding to the processing data, generate the processing data feature dimension set, and perform data mapping on the processing data feature dimension set and the corresponding processing data, thereby obtaining the processing data feature dimension set corresponding to each processing data in the multi-source processing data; It should be further noted that in the specific embodiment, the data feature dimension includes data type dimension, industry dimension, domain dimension, sharing dimension, etc.; among them, the processing data feature dimension is the basis for classifying, managing, processing, and analyzing data; The process of obtaining the required feature data includes: Obtain the set of required data feature dimensions according to the user requirements, and match the set of required data feature dimensions with the set of processed data feature dimensions corresponding to the processed data in the multi-source processed data. If the set of processed data feature dimensions contains the set of required data feature dimensions, extract the corresponding processed data from the pool of processed data to be extracted, and generate the required feature data; otherwise, do nothing. In the above embodiment, it should be further noted that the set of required data feature dimensions is used to represent the data feature dimensions that must exist according to the user requirements. Embodiment 3

[0020] This embodiment is a further limitation of Embodiment 1, and the step S2 is implemented through the following process: The process of setting the feature verification mechanism includes: Obtain the difference set between the set of processed data feature dimensions corresponding to each required feature data and the set of required data feature dimensions, denoted as the required data feature dimension difference set, and generate the similar required feature data from the required feature data corresponding to the required data feature dimension difference set, and map it with the required data feature dimension difference set to obtain the feature verification mechanism. The process of obtaining the data feature similarity value includes: Obtain the set of reserved data feature dimensions according to the user requirements, and set the weights corresponding to each data feature dimension in the set of reserved data feature dimensions, denoted as the reserved weights; then obtain the intersection and difference set between each required data feature dimension difference set and the set of reserved data feature dimensions, and denote them as the required data normal feature dimension set and the required data abnormal feature dimension set respectively. Set the weights corresponding to each data feature dimension in the required data abnormal feature dimension set to -1, and then obtain the total abnormal weight corresponding to each required data abnormal feature dimension set. Obtain the total reserved weight corresponding to each required data normal feature dimension set according to the reserved weights. Obtain the total abnormal weight and the total reserved weight corresponding to the similar required feature data according to the required data normal feature dimension set and the required data abnormal feature dimension set corresponding to the similar required feature data, and add the total abnormal weight and the total reserved weight to obtain the data feature similarity value corresponding to the similar required feature data. In the above embodiments, it should be further noted that not all the multi-source processed data obtained based on big data may be available, and there may be data with similar demand characteristics that are dangerous; by re-dividing the data characteristic dimensions corresponding to the similar demand data, and using the intersection and difference sets of the sets, the set of normal characteristic dimensions of the demand data corresponding to the similar demand characteristic data and the set of abnormal characteristic dimensions of the demand data are obtained. Furthermore, according to the corresponding weights, the data characteristic similarity value corresponding to the similar demand characteristic data is obtained; Embodiment 4

[0021] This embodiment is a further limitation of Embodiment 1, and the step S3 is implemented through the following process: The process of obtaining the accurate demand characteristic data includes: Set a threshold for the data characteristic similarity value, and compare it with the data characteristic similarity value. Extract and integrate the similar demand characteristic data whose data characteristic similarity value is less than or equal to the data characteristic similarity value threshold to obtain a set of similar demand characteristic data; conversely, eliminate the corresponding similar demand characteristic data; Track the data sources corresponding to each similar demand characteristic data in the set of similar demand characteristic data to obtain the data sources corresponding to the similar demand characteristic data; eliminate the similar demand characteristic data for which no data source is traced; Obtain a data source set according to the user's demand, screen the data sources of the similar demand characteristic data whose data sources are traced, and match the data sources with the data source set. If the match is unsuccessful, eliminate the corresponding similar demand characteristic data; conversely, do not perform any processing; furthermore, generate accurate demand characteristic data from the multi-source processed data; In the above embodiments, it should be further noted that eliminate the abnormal similar demand characteristic data and the similar demand characteristic data with abnormal data sources in the multi-source processed data, and retain the remaining ones to generate accurate demand characteristic data to ensure the stability of the processed data and prevent the abnormal data processing results caused by misusing the abnormal similar demand characteristic data; Embodiment 5

[0022] This embodiment is a further limitation of Embodiment 1, and the step S4 is implemented through the following process: The process of generating the transmission data path includes: Obtain the receiving terminals connected to the processing terminal and the location areas corresponding to the receiving terminals, connect the receiving terminals corresponding to the same location area to generate a regional terminal data network, and then connect each regional terminal data network to generate a presentation terminal network; Set the corresponding data transmission mode for the regional terminal data network according to the location area, and set different data encryption methods for the data transmission mode; furthermore, distribute and connect the data transmission modes in the presentation terminal grid to generate a transmission data network; Send the precise demand feature data to the processing terminal for processing to obtain the presentation data, and select the corresponding data transmission mode in the transmission data network for the presentation data according to different receiving terminals. Furthermore, generate a transmission data path for the process of the processing terminal reaching the corresponding receiving terminal in the presentation terminal network through the corresponding data transmission mode; Encrypt the presentation data according to the corresponding data encryption method to obtain the encrypted presentation data; transmit the encrypted presentation data to the receiving terminal through the transmission data path; In the above embodiments, it should be further noted that the receiving terminal is used to represent the terminal corresponding to receiving the presentation data, including but not limited to one, multiple, etc.; the location area is used to represent the geographical location obtained by the receiving terminal according to the map, and then set the geographical location area division standard, and then divide the geographical location according to the geographical location area division standard to obtain the location area; set different data encryption methods according to different location areas to improve the security of data transmission; transmit the presentation data through the transmission data path to improve the data transmission speed and reduce the congestion of the presentation data transmitted through a single transmission channel.

[0023] The features and exemplary embodiments of various aspects of the present application will be described in detail above. In order to make the purpose, technical solution and advantages of the present application clearer, the above is combined with the accompanying drawings and specific embodiments to further describe the present application in detail; it should be understood that the specific embodiments described here are only intended to explain the present application, rather than limit the present application; for those skilled in the art, the present application can be implemented without some of these specific details; the above description of the embodiments is only to provide a better understanding of the present application by showing examples of the present application.

[0024] The above embodiments are only used to illustrate the technical method of the present invention and not to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical method of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A data maintenance and transmission method based on artificial intelligence and big data, characterized in that: The method comprises the following steps: Step S1: a processing data pool to be extracted is set up at the processing terminal, multi-source processing data is integrated and sent to the processing data pool to be extracted for data feature dimension retrieval, a processing data feature dimension set is obtained, and then all corresponding demand feature data in the processing data pool to be extracted are extracted according to the processing data feature dimension set; Step S2: Setting a feature verification mechanism, performing data feature dimension verification on the demand feature data to obtain similar demand feature data; obtaining a reserved data feature dimension set, and obtaining data feature similarity values ​​corresponding to the similar demand feature data according to the reserved data feature dimension set; Step S3: Tracing similar demand feature data according to data feature similarity values ​​to obtain accurate demand feature data; Step S4: Processing the precise demand feature data to obtain presentation data, and obtaining a presentation terminal network, and obtaining a transmission data path corresponding to the presentation data according to the presentation terminal network; Step S5: Transmitting the presented data according to the transmission data path.

2. According to claim 1, a data maintenance and transmission method based on artificial intelligence and big data is characterized in that: The process of obtaining the processing data feature dimension set includes: The to-be-extracted processed data pool is used to receive and store multi-source processed data; an edge retrieval unit is set in the to-be-extracted processed data pool to perform data feature dimension retrieval on the multi-source processed data; a unique feature identifier corresponding to the data feature dimension is set in the edge retrieval unit; Multi-source processing data is obtained based on user needs and big data, and sent to the processing data pool to be extracted; while receiving the multi-source processing data, the edge retrieval unit searches for data feature dimensions of all processing data in the multi-source processing data according to the unique feature identifier, obtains all data feature dimensions corresponding to each processing data, and merges them to generate a corresponding processing data feature dimension set.

3. The data maintenance and transmission method based on artificial intelligence and big data according to claim 2 is characterized in that: The process of obtaining the demand feature data includes: A set of demand data feature dimensions is obtained according to user needs, and the set of demand data feature dimensions is matched with the set of processing data feature dimensions corresponding to each processing data in the multi-source processing data. If the set of processing data feature dimensions contains the set of demand data feature dimensions, the corresponding processing data is extracted from the processing data pool to be extracted and the demand feature data is generated; otherwise, no processing is performed.

4. The data maintenance and transmission method based on artificial intelligence and big data according to claim 3 is characterized in that: The process of setting up the feature verification mechanism includes: Obtain the difference between the processing data feature dimension set corresponding to each demand feature data and the demand data feature dimension set, record it as the demand data feature dimension difference set, and generate the demand feature data corresponding to the demand data feature dimension difference set as similar demand feature data, map it with the demand data feature dimension difference set, and then obtain the feature verification mechanism.

5. The data maintenance and transmission method based on artificial intelligence and big data according to claim 4 is characterized in that: The process of obtaining the data feature similarity value includes: According to user needs, a reserved data feature dimension set is obtained, and a weight corresponding to each data feature dimension in the reserved data feature dimension set is set, which is recorded as a reserved weight; then, the intersection and difference set of each required data feature dimension difference set and the reserved data feature dimension set are obtained, and they are recorded as a required data normal feature dimension set and a required data abnormal feature dimension set respectively; The weight corresponding to each data feature dimension in the abnormal feature dimension set of demand data is set to -1, and then the total abnormal weight corresponding to each abnormal feature dimension set of demand data is obtained, and the reserved total weight corresponding to each normal feature dimension set of demand data is obtained according to the reserved weight.

6. The data maintenance and transmission method based on artificial intelligence and big data according to claim 5 is characterized in that: According to the normal feature dimension set of demand data and the abnormal feature dimension set of demand data corresponding to similar demand feature data, the abnormal total weight and the reserved total weight corresponding to the similar demand feature data are obtained, and the sum of the abnormal total weight and the reserved total weight is obtained, and then marked as the data feature similarity value corresponding to the similar demand feature data.

7. The data maintenance and transmission method based on artificial intelligence and big data according to claim 6 is characterized in that: The process of obtaining the precise demand feature data includes: Set a data feature similarity value threshold and compare it with the data feature similarity value, extract and integrate similar demand feature data corresponding to data feature similarity values ​​less than or equal to the data feature similarity value threshold to obtain a similar demand feature data set; otherwise, remove the corresponding similar demand feature data; Track the data sources corresponding to each similar demand feature data in the similar demand feature data set to obtain the data sources corresponding to the similar demand feature data; remove the similar demand feature data corresponding to the data sources that cannot be tracked; The data source set is obtained according to user needs, and the tracked data source is matched with the data source set. If the match is not successful, the corresponding similar demand feature data will be eliminated, otherwise, no processing will be done; then the multi-source processed data will be generated into accurate demand feature data.

8. The data maintenance and transmission method based on artificial intelligence and big data according to claim 7 is characterized in that: The generation process of the transmission data path includes: Acquire the receiving terminal connected to the processing terminal and the location area corresponding to the receiving terminal, connect the receiving terminals corresponding to the same location area to generate a regional terminal data network, and then connect the regional terminal data networks to generate a presentation terminal network; According to the location area, the data transmission mode corresponding to the regional terminal data network is set, and different data encryption modes are set in the data transmission mode; then, the data transmission modes in the terminal grid are distributedly connected to generate a transmission data network; The precise demand feature data is sent to the processing terminal for processing to obtain the presentation data, and the presentation data is transmitted in a corresponding data transmission mode in the transmission data network according to different receiving terminals, and then the processing terminal reaches the corresponding receiving terminal of the presentation terminal network through the corresponding data transmission mode to generate a transmission data path.

Images