Universe data quantitative management method and system based on AI data fusion

Abstract

The invention relates to a global data quantitative management method and system based on AI data fusion, and belongs to the technical field of data management. The method comprises the following steps: obtaining structured data, semi-structured data and unstructured data, and calculating to obtain converted structured data through a conversion model according to the semi-structured data and the unstructured data; storing the structured data and the converted structured data to obtain comprehensive structured data, and calculating through a preprocessing model according to the comprehensive structured data to obtain cleaning data; and presetting a data extraction rule, screening the cleaned data according to the data extraction rule to obtain a prediction data set and a decision data set, performing calculation through a BP neural network according to the prediction data set to obtain a prediction result, and performing calculation through a decision model according to the decision data set to obtain a decision result. And intelligent fusion and management of enterprise data are realized.

Description

Technical Field

[0001] The present invention belongs to the field of data management technology, and specifically relates to a global data quantitative management method and system based on AI data fusion. Background Art

[0002] In the information age, data is considered an important corporate asset that needs to be effectively managed and its quality gradually improved, and then its value can be mined and utilized to help business leaders make more accurate decisions and ensure the efficient operation of the enterprise.

[0003] The various fields of data management are both independent and interdependent. Data management areas such as data standards, metadata, master data, and data quality can all be self-contained, complex, and large; at the same time, each field is interdependent, and various management processes are closely related. To promote data management within an enterprise in a comprehensive and orderly manner is a long-term, arduous, and ineffective task. Under the limited human resources and data management foundation, in order to achieve the goal of promoting the implementation of data management and effectively improving the quality of enterprise data, it is necessary to determine the core objects of data management and implement corresponding work. Although these data contain rich value, due to their diversity and complexity, the effective use of data has become a major challenge. Existing data management systems can often only process a single type of data, and the application of data cleaning and predictive models is relatively limited. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a global data quantitative management method and system based on AI data fusion.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] Acquire structured data, semi-structured data, and unstructured data, and calculate converted structured data based on the semi-structured data and the unstructured data through a conversion model;

[0007] The structured data and the converted structured data are stored to obtain comprehensive structured data, and cleaned data is obtained by calculating the comprehensive structured data through a preprocessing model;

[0008] Preset data extraction rules, filter the cleaned data according to the data extraction rules to obtain a prediction data set and a decision data set, calculate the prediction result through a BP neural network based on the prediction data set, and calculate the decision result through a decision model based on the decision data set.

[0009] Specifically, the conversion model includes a semi-structured conversion model and an unstructured conversion model. The semi-structured conversion model includes:

[0010] Calculating outliers using a box plot method based on the unstructured data, and deleting the outliers to obtain cleaned data;

[0011] Filling missing values ​​using a KNN algorithm according to the cleaned data to obtain filled data;

[0012] According to the padded data, data conversion is performed by a map method to obtain normalized data, and according to the normalized data, reduced data is obtained by a filter method;

[0013] According to the reduced data, a word vector is calculated by word2vc, and a word library is obtained by using the GetScore algorithm according to the word vector;

[0014] According to the word library, IKAnlyzer is used to perform word segmentation to obtain correct word segmentation data, and according to the correct word segmentation data, a part-of-speech-based information extraction algorithm is used to calculate and obtain first structured data.

[0015] Specifically, the GetScord algorithm comprises the following steps:

[0016] S201: sequentially traverse the word vector to obtain a starting word, calculate an inter-word score according to the starting word by using the dot function, obtain a highest score according to the inter-word score by using the sortByKey function, and determine a highest-scoring word according to the highest score;

[0017] S202: setting the starting word and the highest score word as the key and value of a HashMap respectively, and storing the HashMap in an RDD;

[0018] S203: sequentially traverse the RDD to obtain a calibration HashMap, determine whether the position of the highest-participated word in the calibration HashMap is the position of the starting word plus one, if yes, concatenate the starting word and the highest-participated word to obtain a combined word, and execute step S206; otherwise, execute step S204;

[0019] S204: Calculate the right information entropy of the starting word and the left information entropy of the highest score word, preset a threshold, and determine whether the right information entropy and the left information entropy are both greater than the threshold. If yes, calculate the mutual information between the starting word and the secondary word; if no, execute step SS207;

[0020] S205: Determine whether the mutual information is greater than the threshold, if yes, concatenate the starting word and the secondary word to obtain the combined word; otherwise, execute step S207;

[0021] S206: Count the word frequency of the combination word, preset a word frequency threshold, and determine whether the word frequency is less than the word frequency threshold. If yes, execute step S207; if no, add the combination word to the word library;

[0022] S207: Sequentially traverse the next element in the RDD, set the next element to the calibration HashMap, and repeat steps S203-S206.

[0023] Specifically, the unstructured conversion model includes:

[0024] Obtain image preprocessing data by performing image preprocessing according to the image data, and obtain voice preprocessing data by performing voice preprocessing according to the voice data;

[0025] It should be noted that the speech preprocessing includes noise reduction and normalization, and the image preprocessing includes noise reduction, grayscale, binarization, and edge detection;

[0026] The image preprocessing data is calculated by an image recognition model to obtain image text data, and the speech preprocessing data is calculated by a speech recognition model to obtain speech text data;

[0027] The second structured data is obtained by calculating the image text data, the voice text data and the text data through a text structured model.

[0028] Specifically, the image recognition model includes:

[0029] A text recognition area is obtained by calculating the K-means algorithm according to the image preprocessing data, and distance data of image pixel points from a central area is calculated according to the text recognition area;

[0030] Preset a distance threshold, divide the layer into a layer set according to the distance threshold and the distance data, and perform layer screening according to the layer set by connected domain analysis and pixel noise removal to obtain a target layer;

[0031] The text block is calculated based on the target layer through the threshold segmentation method, and the image text data is obtained through OCR calculation based on the text block.

[0032] Specifically, the language recognition model includes:

[0033] Performing frame processing according to the speech preprocessing data to obtain speech frames, presetting a window function, and reducing the boundary effect of the speech frames by the window function to obtain processed speech frames;

[0034] Obtain a frequency domain signal by Fourier transform according to the processed speech frame, and extract Mel frequency spectrum coefficients and Mel spectrum energy of the frequency domain signal;

[0035] The acoustic features are calculated using a hidden Markov model based on the Mel frequency spectrum coefficients and the Mel frequency spectrum energy, and the speech text data is calculated using an n-gram model based on the acoustic features.

[0036] Specifically, the text structuring model includes:

[0037] Obtaining word segmentation data by Jieba word segmentation according to the image text data, the voice text data, and the text data;

[0038] Entity recognition data is obtained by calculating the entity recognition data through the CRFs model according to the word segmentation data, the entity recognition data has a part-of-speech tag, a correction rule is preset, and the entity recognition data is screened according to the part-of-speech tag and the correction rule to obtain entity data;

[0039] Entity relationship data is calculated based on the entity data by using a K-means clustering algorithm, a storage template is preset, and the entity data is stored in the storage template based on the entity relationship data to obtain the second structured data.

[0040] Specifically, the preprocessing model includes:

[0041] Calculating error data according to the comprehensive structured data by using an error detection algorithm based on integrity constraints, and eliminating the comprehensive structured data according to the error data to obtain eliminated data;

[0042] Filling missing values ​​by interpolation according to the removed data to obtain filled data;

[0043] The cleaned data is obtained by removing redundant data through a decision tree model according to the filled data.

[0044] Specifically, the decision model includes:

[0045] According to the decision data set, a decision support degree is obtained by triangular fuzzy number calculation, fuzzy semantic parameters are obtained, a weight value is obtained by OWA operator calculation according to the fuzzy semantic parameters, data credibility and importance are obtained, the decision support degree is converted according to the credibility, the importance and the weight value to obtain a conversion result, the conversion results are sorted in descending order to obtain a sorting result, and the decision result is obtained by weighted sum calculation according to the sorting result.

[0046] A global data quantitative management system based on AI data fusion, including: data conversion module, data preprocessing module, data decision module;

[0047] The data conversion module is used to obtain structured data, semi-structured data, and unstructured data, and obtain converted structured data through conversion model calculation according to the semi-structured data and the unstructured data;

[0048] The data preprocessing module is used to store the structured data and the converted structured data to obtain comprehensive structured data, and calculate the cleaned data according to the comprehensive structured data through a preprocessing model;

[0049] The data decision module is used to preset data extraction rules, screen the cleaned data according to the data extraction rules to obtain a prediction data set and a decision data set, calculate the prediction result through the BP neural network based on the prediction data set, and calculate the decision result through the decision model based on the decision data set.

[0050] The beneficial effects of the present invention are:

[0051] (1) Data collection and structured processing: Acquire structured data, semi-structured data, and unstructured data within the enterprise, use advanced algorithms and technologies to structure the semi-structured data and unstructured data to obtain converted structured data, store the structured data and converted structured data to form comprehensive structured data, and achieve comprehensive collection and processing of various types of data within the enterprise, thereby improving data utilization and accuracy.

[0052] (2) Data cleaning: A preprocessing model is established based on comprehensive structured data to process outliers, missing values, and redundant values. The preprocessing model uses intelligent algorithms to automatically identify and correct errors and incomplete parts in the data, thereby improving the accuracy and reliability of the data.

[0053] (3) Data classification and prediction: Preset extraction rules, classify the cleaned data according to the rules, obtain prediction data sets and decision data sets, and apply BP neural network to the prediction data sets for prediction, including sales forecast, inventory forecast, etc. BP neural network has powerful nonlinear mapping and self-learning capabilities, and can accurately predict future trends and changes. Apply decision models to the decision data sets for calculation to obtain decision results. The decision results are expressed in numerical form. The larger the numerical value, the higher the executable degree of the decision, which improves the accuracy of prediction and the scientific nature of decision, and provides strong support for the operation and management of the enterprise. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0055] Figure 1 It is a flow chart of a global data quantitative management method based on AI data fusion of the present invention. DETAILED DESCRIPTION

[0056] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0057] See also Figure 1 , a global data quantitative management method based on AI data fusion,

[0058] Acquire structured data, semi-structured data, and unstructured data, and calculate converted structured data based on the semi-structured data and the unstructured data through a conversion model;

[0059] The structured data and the converted structured data are stored to obtain comprehensive structured data, and cleaned data is obtained by calculating the comprehensive structured data through a preprocessing model;

[0060] Preset data extraction rules, filter the cleaned data according to the data extraction rules to obtain a prediction data set and a decision data set, calculate the prediction result through a BP neural network based on the prediction data set, and calculate the decision result through a decision model based on the decision data set.

[0061] In this embodiment, a data management system with three functions of sales forecasting, inventory forecasting, and decision-making calculation is constructed according to the global data quantitative management method based on AI data fusion, which specifically includes:

[0062] Step 1: Obtain structured data, semi-structured data, and unstructured data, calculate converted structured data based on the semi-structured data and the unstructured data through a conversion model, store the structured data and the converted structured data to obtain comprehensive structured data, and calculate cleaned data based on the comprehensive structured data through a preprocessing model;

[0063] Step 2: Preset data extraction rules, and filter the cleaned data according to the data extraction rules to obtain a sales forecast data set, an inventory forecast data set, and a decision data set;

[0064] Step 3: Calculate the sales forecast result through the BP neural network according to the sales forecast data set, calculate the inventory forecast result through the BP neural network according to the inventory forecast data set, and calculate the decision result through the decision model according to the decision data set;

[0065] Step 4: Visualize the sales forecast results, inventory forecast results, and decision results.

[0066] Specifically, the structured data includes product information, personnel information, and customer behavior data; the semi-structured data includes website visit data, customer survey reports, POI point of interest data, and XML file data; and the unstructured data includes image data, voice data, and text data.

[0067] Specifically, the converted structured data includes first structured data and second structured data.

[0068] Specifically, the conversion model includes a semi-structured conversion model and an unstructured conversion model. The semi-structured conversion model includes:

[0069] Calculating outliers using a box plot method based on the unstructured data, and deleting the outliers to obtain cleaned data;

[0070] Filling missing values ​​using a KNN algorithm according to the cleaned data to obtain filled data;

[0071] According to the padded data, data conversion is performed by a map method to obtain normalized data, and according to the normalized data, reduced data is obtained by a filter method;

[0072] According to the reduced data, a word vector is calculated by word2vc, and a word library is obtained by using the GetScore algorithm according to the word vector;

[0073] According to the word library, IKAnlyzer is used to perform word segmentation to obtain correct word segmentation data, and according to the correct word segmentation data, a part-of-speech-based information extraction algorithm is used to calculate and obtain first structured data.

[0074] Specifically, the GetScord algorithm comprises the following steps:

[0075] S201: sequentially traverse the word vector to obtain a starting word, calculate an inter-word score according to the starting word by using the dot function, obtain a highest score according to the inter-word score by using the sortByKey function, and determine a highest-scoring word according to the highest score;

[0076] S202: setting the starting word and the highest score word as the key and value of a HashMap respectively, and storing the HashMap in an RDD;

[0077] S203: sequentially traverse the RDD to obtain a calibration HashMap, determine whether the position of the highest-participated word in the calibration HashMap is the position of the starting word plus one, if yes, concatenate the starting word and the highest-participated word to obtain a combined word, and execute step S206; otherwise, execute step S204;

[0078] S204: Calculate the right information entropy of the starting word and the left information entropy of the highest score word, preset a threshold, and determine whether the right information entropy and the left information entropy are both greater than the threshold. If yes, calculate the mutual information between the starting word and the secondary word; if no, execute step SS207;

[0079] The information entropy calculation formula is:

[0080]

[0081] Among them, H(x) is information entropy, i is the variable parameter, n is the number of word vector elements, and x i represents the i-th word, P(x i ) represents the independence degree of the i-th word.

[0082] It should be noted that the left information entropy is calculated by taking the probability of occurrence of the left adjacent word of the highest score word as the degree of independence of the i-th word in the information entropy, and the right information entropy is calculated by taking the probability of occurrence of the right adjacent word of the starting word as the degree of independence of the i-th word in the information entropy; the secondary word is the next word of the starting word;

[0083] S205: Determine whether the mutual information is greater than the threshold, if yes, concatenate the starting word and the secondary word to obtain the combined word; otherwise, execute step S207;

[0084] The mutual information calculation formula is:

[0085]

[0086] Where x is the starting word, y is the secondary word, X and Y are word vectors, P(x, y) is the probability of the starting word and the secondary word appearing at the same time, P(x) is the probability of the starting word appearing alone, and P(y) is the probability of the secondary word appearing alone;

[0087] S206: Count the word frequency of the combination word, preset a word frequency threshold, and determine whether the word frequency is less than the word frequency threshold. If yes, execute step S207; if no, add the combination word to the word library;

[0088] S207: Sequentially traverse the next element in the RDD, set the next element to the calibration HashMap, and repeat steps S203-S206.

[0089] Specifically, the unstructured conversion model includes:

[0090] Obtain image preprocessing data by performing image preprocessing according to the image data, and obtain voice preprocessing data by performing voice preprocessing according to the voice data;

[0091] It should be noted that the speech preprocessing includes noise reduction and normalization, and the image preprocessing includes noise reduction, grayscale, binarization, and edge detection;

[0092] The image preprocessing data is calculated by an image recognition model to obtain image text data, and the speech preprocessing data is calculated by a speech recognition model to obtain speech text data;

[0093] The second structured data is obtained by calculating the image text data, the voice text data and the text data through a text structured model.

[0094] Specifically, the image recognition model includes:

[0095] A text recognition area is obtained by calculating the K-means algorithm according to the image preprocessing data, and distance data of image pixel points from a central area is calculated according to the text recognition area;

[0096] Preset a distance threshold, divide the layer into a layer set according to the distance threshold and the distance data, and perform layer screening according to the layer set by connected domain analysis and pixel noise removal to obtain a target layer;

[0097] The text block is calculated based on the target layer through the threshold segmentation method, and the image text data is obtained through OCR calculation based on the text block.

[0098] Specifically, the language recognition model includes:

[0099] Performing frame processing according to the speech preprocessing data to obtain speech frames, presetting a window function, and reducing the boundary effect of the speech frames by the window function to obtain processed speech frames;

[0100] Obtain a frequency domain signal by Fourier transform according to the processed speech frame, and extract Mel frequency spectrum coefficients and Mel spectrum energy of the frequency domain signal;

[0101] The acoustic features are calculated using a hidden Markov model based on the Mel frequency spectrum coefficients and the Mel frequency spectrum energy, and the speech text data is calculated using an n-gram model based on the acoustic features.

[0102] Specifically, the text structured model includes:

[0103] Obtaining word segmentation data by Jieba word segmentation according to the image text data, the voice text data, and the text data;

[0104] Entity recognition data is obtained by calculating the entity recognition data through the CRFs model according to the word segmentation data, the entity recognition data has a part-of-speech tag, a correction rule is preset, and the entity recognition data is screened according to the part-of-speech tag and the correction rule to obtain entity data;

[0105] It should be noted that the screening process is to determine whether the part-of-speech tag meets the correction rules. If yes, the entity recognition data is saved to obtain the entity data; otherwise, the entity recognition data is deleted;

[0106] Entity relationship data is calculated based on the entity data by using a K-means clustering algorithm, a storage template is preset, and the entity data is stored in the storage template based on the entity relationship data to obtain the second structured data.

[0107] Specifically, the preprocessing model includes:

[0108] Calculating error data according to the comprehensive structured data by using an error detection algorithm based on integrity constraints, and eliminating the comprehensive structured data according to the error data to obtain eliminated data;

[0109] Filling missing values ​​by interpolation according to the removed data to obtain filled data;

[0110] The cleaned data is obtained by removing redundant data through a decision tree model according to the filled data.

[0111] Specifically, the decision model includes:

[0112] According to the decision data set, a decision support degree is obtained by triangular fuzzy number calculation, a fuzzy semantic parameter is obtained, a weight value is obtained by OWA operator calculation according to the fuzzy semantic parameter, data credibility and importance are obtained, the decision support degree is converted according to the credibility, the importance, and the weight value to obtain a conversion result, the conversion result is sorted in descending order to obtain a sorting result, and the decision result is obtained by weighted sum calculation according to the sorting result;

[0113] The conversion result calculation formula is:

[0114] Y i =W i Q i V i X i ,

[0115] Among them, Y i is the transformation result, W i is the weight value, Q i is the credibility, V i is the importance, X iFor decision support.

[0116] It should be noted that the fuzzy semantic parameters are used to characterize the preference of enterprise decision makers for data sources.

[0117] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0118] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0119] The program code included on the computer readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or their combinations, and the programming language includes object-oriented programming languages-such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0120] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A global data quantitative management method based on AI data fusion, characterized in that: include: Acquire structured data, semi-structured data, and unstructured data, and calculate converted structured data based on the semi-structured data and the unstructured data through a conversion model; The structured data and the converted structured data are stored to obtain comprehensive structured data, and cleaned data is obtained by calculating the comprehensive structured data through a preprocessing model; Preset data extraction rules, filter the cleaned data according to the data extraction rules to obtain a prediction data set and a decision data set, calculate the prediction result through a BP neural network based on the prediction data set, and calculate the decision result through a decision model based on the decision data set.

2. The method according to claim 1, characterized in that The conversion model includes a semi-structured conversion model and an unstructured conversion model. The semi-structured conversion model includes: Calculating outliers using a box plot method based on the unstructured data, and deleting the outliers to obtain cleaned data; Filling missing values ​​using a KNN algorithm according to the cleaned data to obtain filled data; According to the padded data, data conversion is performed by a map method to obtain normalized data, and according to the normalized data, reduced data is obtained by a filter method; According to the reduced data, a word vector is calculated by word2vc, and a word library is obtained by using the GetScore algorithm according to the word vector; According to the word library, IKAnlyzer is used to perform word segmentation to obtain correct word segmentation data, and according to the correct word segmentation data, a part-of-speech-based information extraction algorithm is used to calculate and obtain first structured data.

3. The method according to claim 2, characterized in that The GetScord algorithm specifically includes the following steps: S201: sequentially traverse the word vector to obtain a starting word, calculate an inter-word score according to the starting word by using the dot function, obtain a highest score according to the inter-word score by using the sortByKey function, and determine a highest-scoring word according to the highest score; S202: setting the starting word and the highest score word as the key and value of a HashMap respectively, and storing the HashMap in an RDD; S203: sequentially traverse the RDD to obtain a calibration HashMap, determine whether the position of the highest-participated word in the calibration HashMap is the position of the starting word plus one, if yes, concatenate the starting word and the highest-participated word to obtain a combined word, and execute step S206; otherwise, execute step S204; S204: Calculate the right information entropy of the starting word and the left information entropy of the highest score word, preset a threshold, and determine whether the right information entropy and the left information entropy are both greater than the threshold. If yes, calculate the mutual information between the starting word and the secondary word; if no, execute step SS207; S205: Determine whether the mutual information is greater than the threshold, if yes, concatenate the starting word and the secondary word to obtain the combined word; otherwise, execute step S207; S206: Count the word frequency of the combination word, preset a word frequency threshold, and determine whether the word frequency is less than the word frequency threshold. If yes, execute step S207; if no, add the combination word to the word library; S207: Sequentially traverse the next element in the RDD, set the next element to the calibration HashMap, and repeat steps S203-S206.

4. The method according to claim 2, characterized in that: The unstructured conversion model includes: Obtain image preprocessing data by performing image preprocessing according to the image data, and obtain voice preprocessing data by performing voice preprocessing according to the voice data; The image preprocessing data is calculated by an image recognition model to obtain image text data, and the speech preprocessing data is calculated by a speech recognition model to obtain speech text data; The second structured data is obtained by calculating the image text data, the voice text data and the text data through a text structured model.

5. The method according to claim 4, characterized in that The image recognition model includes: A text recognition area is obtained by calculating the K-means algorithm according to the image preprocessing data, and distance data of image pixel points from a central area is calculated according to the text recognition area; Preset a distance threshold, divide the layer into a layer set according to the distance threshold and the distance data, and perform layer screening according to the layer set by connected domain analysis and pixel noise removal to obtain a target layer; The text block is calculated based on the target layer through the threshold segmentation method, and the image text data is obtained through OCR calculation based on the text block.

6. The method according to claim 4, characterized in that The language recognition model includes: Performing frame processing according to the speech preprocessing data to obtain speech frames, presetting a window function, and reducing the boundary effect of the speech frames by the window function to obtain processed speech frames; Obtain a frequency domain signal by Fourier transform according to the processed speech frame, and extract Mel frequency spectrum coefficients and Mel spectrum energy of the frequency domain signal; The acoustic features are calculated using a hidden Markov model based on the Mel frequency spectrum coefficients and the Mel frequency spectrum energy, and the speech text data is calculated using an n-gram model based on the acoustic features.

7. The method according to claim 4, characterized in that The text structured model includes: Obtaining word segmentation data by Jieba word segmentation according to the image text data, the voice text data, and the text data; Entity recognition data is obtained by calculating the entity recognition data through the CRFs model according to the word segmentation data, the entity recognition data has a part-of-speech tag, a correction rule is preset, and the entity recognition data is screened according to the part-of-speech tag and the correction rule to obtain entity data; Entity relationship data is calculated based on the entity data by using a K-means clustering algorithm, a storage template is preset, and the entity data is stored in the storage template based on the entity relationship data to obtain the second structured data.

8. The method according to claim 1, characterized in that The preprocessing model includes: Calculating error data according to the comprehensive structured data by using an error detection algorithm based on integrity constraints, and eliminating the comprehensive structured data according to the error data to obtain eliminated data; Filling missing values ​​by interpolation according to the removed data to obtain filled data; The cleaned data is obtained by removing redundant data through a decision tree model according to the filled data.

9. The method according to claim 1, characterized in that: The decision model includes: According to the decision data set, the decision support is obtained by triangular fuzzy number calculation, fuzzy semantic parameters are obtained, and weight values ​​are obtained by OWA operator calculation according to the fuzzy semantic parameters, and data credibility and importance are obtained. According to the credibility, the importance and the weight value, the decision support is converted to obtain a conversion result, the conversion results are sorted in descending order to obtain a sorting result, and the decision result is obtained by weighted sum calculation according to the sorting result.

10. A global data quantification management system based on AI data fusion, characterized in that: include: Data conversion module, data preprocessing module, data decision module; The data conversion module is used to obtain structured data, semi-structured data, and unstructured data, and obtain converted structured data through conversion model calculation according to the semi-structured data and the unstructured data; The data preprocessing module is used to store the structured data and the converted structured data to obtain comprehensive structured data, and calculate the cleaned data according to the comprehensive structured data through a preprocessing model; The data decision module is used to preset data extraction rules, screen the cleaned data according to the data extraction rules to obtain a prediction data set and a decision data set, calculate the prediction result through the BP neural network based on the prediction data set, and calculate the decision result through the decision model based on the decision data set.

Images