Automatic decision-making method and system based on data governance

Abstract

The invention discloses an automatic decision-making method and system based on data governance, and the method comprises the following steps: collecting data used for training an automatic decision-making model, preprocessing the data through a data governance tool, and forming a heterogeneous data source; constructing an automatic decision model, inputting a heterogeneous data source into a model front module, and extracting data features of multi-modal data by using a feature engineering method to form a feature set; inputting the feature set into a model input layer, carrying out feature fusion in a hidden layer, training a model decision by using fused features, and enabling a model output layer to output an automatic decision; calculating a comprehensive performance index of the model, and evaluating the quality of a model generation decision; adjusting feature weights and model parameters according to the comprehensive performance indexes, and performing model optimization; and storing records in the model training process according to the requirements of data governance. The automatic decision-making model is trained based on the principle of data governance, and the accuracy, reliability and compliance of model decision-making are ensured.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to an automated decision-making method and system based on data governance. Background Art

[0002] With the advent of the big data era, enterprises and organizations have accumulated massive amounts of data, and how to effectively manage and utilize this data has become a key challenge. Traditional decision-making methods are difficult to cope with complex, dynamic and changing environments, and more efficient and accurate decision-making support is needed. With the introduction of data privacy and security regulations, data governance has become a necessary condition for corporate compliance operations. Based on this, the present invention proposes an automated decision-making method and system based on data governance, which significantly improves the efficiency, accuracy and compliance of decision-making by integrating high-quality data and automated process decisions. Summary of the invention

[0003] The present invention provides an automated decision-making method based on data governance, comprising: S10. Collect data for training automated decision-making models, pre-process the data using data governance tools, and form heterogeneous data sources; S20, constructing an automated decision model, inputting heterogeneous data sources into the automated decision model front-end module, extracting data features of multimodal data using feature engineering methods, and forming a feature set; S30, inputting the feature set into the input layer of the automated decision model, performing feature fusion of the multimodal data features in the hidden layer, and using the fused features to train the model decision, so that the model output layer outputs the automated decision; S40, calculate the comprehensive performance index of the automated decision-making model to evaluate the quality of the decisions generated by the model; S50, adjusting feature weights and model parameters according to the comprehensive performance index to perform model optimization; S60. Keep records of the model training process according to data governance requirements to facilitate audit tracking.

[0004] As described above, an automated decision-making method based on data governance, in which the data direction, data type and corresponding decision used to train the automated decision-making model are identified and collected from multiple sources through data governance tools. The data governance tool identifies and repairs data quality problems on the collected data, including cleaning, deduplication, filling missing values, correcting erroneous data, and converting data into a unified format to ensure data consistency and integrity.

[0005] As described above, an automated decision-making method based on data governance is constructed, wherein an automated decision-making model is constructed, and the automated decision-making model includes a front module, an input layer, a hidden layer, and an output layer.

[0006] As described above, an automated decision-making method based on data governance, in which multimodal data features are fused in the hidden layer, includes the following sub-steps: initializing the automated decision model parameters; inputting the training set feature set into the automated decision model input layer for preliminary fusion; extracting high-level features of the audio, and fusing the high-level audio features with the preliminary fusion features in the hidden layer.

[0007] As described above, an automated decision-making method based on data governance, in which the decision of the model is trained using fusion features, is to place a training module in the hidden layer, map the fusion features with the decision features, and train the model for learning.

[0008] As described above, an automated decision-making method based on data governance, in which the comprehensive performance index is composed of the similarity and deviation between the decision result output by the model and the decision of the validation set. The larger the similarity value, the better the model performance, and the larger the deviation value, the worse the model performance.

[0009] An automated decision-making method based on data governance as described above, wherein model optimization is to dynamically adjust feature weights and model parameters if the comprehensive performance index score is lower than the preset standard, and to analyze the impact of the preset data quality standard on the deviation calculation if the comprehensive performance index score is far lower than the preset standard, and to improve the data quality standard if the impact is large.

[0010] The present invention also provides an automated decision-making system based on data governance, comprising: Acquisition module: collects data for training automated decision-making models, preprocesses data using data governance tools, and forms heterogeneous data sources; Feature extraction module: Build an automated decision model, input heterogeneous data sources into the automated decision model front-end module, and use feature engineering methods to extract data features of multimodal data to form a feature set; Fusion module: inputs the feature set into the input layer of the automated decision model, and multimodal data features are fused in the hidden layer; Training module: Use fusion features to train model decisions so that the model output layer outputs automated decisions; Evaluation module: Calculates the comprehensive performance index of the automated decision-making model and evaluates the quality of the decisions generated by the model; Optimization module: adjusts feature weights and model parameters according to the comprehensive performance index to optimize the model; Recording module: Save records of the model training process according to data governance requirements to facilitate audit tracking.

[0011] The beneficial effects achieved by the present invention are as follows: The present invention trains automated decision-making models based on the principles of data governance to ensure the accuracy, reliability and compliance of model decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0013] Figure 1 This is a flow chart of an automated decision-making method based on data governance provided in Example 1 of the present application.

[0014] Figure 2 This is a schematic diagram of an automated decision-making system based on data governance provided in Example 2 of the present application. DETAILED DESCRIPTION

[0015] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0016] Embodiment 1 like Figure 1 As shown, Embodiment 1 of the present application provides an automated decision-making method based on data governance, including: S10. Collect data for training automated decision-making models, use data governance tools to pre-process the data, and form heterogeneous data sources.

[0017] Determine the data direction, data type, and corresponding decisions used to train automated decision models, identify and collect data from multiple sources through data governance tools, including databases, APIs, sensors, log files, etc., and use data governance tools to identify and repair data quality issues on the collected data, including cleaning, deduplication, filling missing values, correcting erroneous data, and converting data into a unified format to ensure data consistency and integrity. Use statistical analysis methods and tools to measure whether the processed data meets the preset quality standards.

[0018] Data that meets quality standards constitute heterogeneous data sources, and data governance tools are used to manage heterogeneous data sources. Data search and discovery functions are provided based on the data catalog, and the source, transformation, and flow of data are tracked based on data lineage. According to the data governance strategy, data is classified and graded. The classification includes customer data, transaction data, device data, etc. The classification includes three levels: public, internal, and confidential. Access rights and security policies are assigned to data of different levels. The classified and graded heterogeneous data sources are stored in the data warehouse. At the same time, data backup and recovery strategies are established to ensure the security and reliability of data.

[0019] S20. Build an automated decision-making model, input heterogeneous data sources into the front-end module of the automated decision-making model, use feature engineering methods to extract data features of multimodal data, and form a feature set.

[0020] Construct an automated decision-making model, which includes a front-end module, an input layer, a hidden layer, and an output layer.

[0021] The heterogeneous data sources are input into the front-end module of the automated decision model, and the data features are extracted using feature engineering methods. The data in the heterogeneous data sources include numerical data, text data, image data, and audio data. The sub-steps for extracting data features are as follows: S21. Numerical data feature conversion.

[0022] For numerical data, normalization and standardization are performed so that it can be put into the model for training. For algorithms that use fixed-range data, all feature values ​​are converted to a specific range through normalization to ensure that no single feature has too much impact on the model results. The specific formula is: ,in, represents the normalized data value. Represents a single numeric data, Indicates the minimum value of numeric data. Indicates the maximum value of numerical data. For algorithms that are sensitive to the characteristic scale of data, standardization is used to eliminate the dimensional differences between numerical values. The specific formula is: ,in, represents the data value after standardization. represents the mean of numerical data, Represents the standard deviation of numerical data. The converted numerical set is .

[0023] S22. Text data feature extraction.

[0024] For text data, use the formula Extracting text features can accurately reflect the contribution of words to the content of the document, among which, Represents the characteristics of text data, Yes word In text The number of times it appears in is text The number of occurrences of all words in , Expressive words The frequency of occurrence in a single document, is the total number of texts in different categories, Contains words The number of texts. Use the formula to extract text features multiple times to form a text data feature set .

[0025] S23. Extracting image data features.

[0026] For image data, the accelerated robust feature algorithm is used to detect points of interest in the image to extract image features. The accelerated robust feature algorithm determines the feature points by detecting the determinant value of the matrix. At the point and scale s is defined as , where a represents the horizontal position of point H, and b represents the vertical position of point H. represents the second-order derivative of point H in the horizontal direction, represents the second-order derivative of point H in the vertical direction, It represents the mixed second-order derivative between the horizontal and vertical directions of point H, describing the changes of the image at the intersection of the horizontal and vertical directions. The determinant value of the definition matrix is The specific formula is , calculate the derivative The value of needs to define the box filter, in the filter range arrive Use the integral image to quickly obtain the sum of the pixel values ​​in the area covered by the filter.

[0027] The specific formula is , where the integral image The calculation formula is . The value of approximate derivative The value of is calculated using the same method , The value of , thus obtaining the determinant value , the position of the feature point is determined by the local maximum of the matrix determinant value.

[0028] Taking the feature point as the center, take a square neighborhood and divide it into The sub-region is further divided into smaller sub-regions. The entire neighborhood is divided into 16 small blocks. The sum of the horizontal and vertical Haar wavelet responses is calculated for each sub-region. , and the descriptor D of each interest point is obtained by combination. , normalize the descriptor D to obtain the unique features of the key point , the normalization operation is , Represents a constant to prevent the denominator from being zero. Use the Gaussian mixture model to model the distribution of key point features, aggregate all key point features of the image, and form an image feature set .

[0029] S24: extracting audio data features.

[0030] For audio data, audio features are extracted by calculating the cepstral coefficients. First, use the formula Pre-emphasize to enhance the high frequency part, where: represents the value of the pre-emphasized audio signal x at time point q, represents the sampling value of the original audio signal x at time point q, represents the pre-emphasis coefficient, Represents the sampling value of the original audio signal x at time point q-1. Secondly, the audio signal is divided into short time frames, and each frame signal is windowed to reduce spectrum leakage.

[0031] The specific formula is ,in, represents the value of the windowed audio signal at time point q, Represents the window function. Then use Fourier transform to calculate the spectrum of each frame , represents the frequency domain representation of the windowed signal, k is the frequency index, and FFT represents the fast Fourier transform function, which converts the time domain signal into the frequency domain signal. The spectrum is mapped to the Mel scale using the Mel filter bank. The specific formula is: , represents the output energy of the mth Mel filter, N represents the length of the frequency domain signal, Represents the value of the frequency domain signal at frequency index k, It represents the value of the mth Mel filter at frequency index k. The cepstral coefficient is calculated after taking the logarithm of the Mel spectrum. The specific formula is: ,in, represents the nth cepstral coefficient, M represents the number of Mel filters, represents the logarithmic energy of the mth Mel filter, Represents pi. Main cepstral coefficients are retained as audio features. Output is a two-dimensional matrix , of shape T×F, where T is the time step and F is the number of cepstral coefficients.

[0032] The feature set consists of a numerical feature set, a text feature set, an image feature set, an audio feature matrix, and a decision feature. The decision can be of various types, and the decision features are extracted using the corresponding method. The feature set is divided into a training set, a validation set, and a test set. All three include data features and corresponding decision features. A stratified sampling method is used to ensure that the data distribution of the training set, validation set, and test set is consistent with the overall data. In the classification task, the proportion of each category in the training set, validation set, and test set should be the same as the proportion in the overall data. The combination of different modal data in the training set and validation set should also be representative.

[0033] S30, inputting the feature set into the input layer of the automated decision-making model, performing feature fusion of the multimodal data features in the hidden layer, and using the fused features to train the model decision, so that the model output layer outputs the automated decision.

[0034] S31. Initialize the parameters of the automated decision model.

[0035] Initializing the parameters of the automated decision-making model can make the model converge faster in the early stages of training and reduce training time. According to the dimensions of input and output, the weight values ​​are initialized so that the output variance of each layer remains consistent, thereby promoting the stable propagation of the gradient. The specific formula is ,in, Representation weight Sampling from a uniform distribution u, Represents the input dimension. The number of neurons in the input dimension is the sum of the number of input modal features. represents the output dimension, and the number of neurons is the number of neurons in the current layer.

[0036] S32. Input the training set feature set into the input layer of the automated decision model for preliminary fusion.

[0037] The training set feature set is input into the input layer of the automated decision model, and the numerical features, text features, and image features in the feature set are concatenated at the input layer to form a preliminary fused feature vector set. The fusion formula is: ,in, represents the numerical feature weight, represents the j values ​​of the ith event in the numerical feature set, represents the text feature weight, represents the k text features of the i-th event in the text feature set, represents the image feature weight, Represents the o image features of the i-th event in the image feature set, i represents the number of events in the training set, and the attention mechanism is used to dynamically adjust the weights of different modalities.

[0038] S33, extracting high-level features of the audio, and fusing the high-level features of the audio with the preliminary fusion features in the hidden layer.

[0039] The audio feature matrix is ​​processed by an independent neural network branch to extract higher-level feature representations. Input audio feature matrix , local time-frequency features are extracted through the convolution layer. The specific formula is: , is the feature map after convolution, Represents the convolution operation function. The pooling layer is used to reduce the dimension and retain important features. The specific formula is: , represents the feature map after pooling, Represents the maximum pooling operation function. Mapping the feature map to a feature vector of fixed length, the specific formula is , Represents the audio feature vector, Dense represents the fully connected layer, and Flatten() is used to flatten the feature map into a one-dimensional vector.

[0040] At a certain position in the hidden layer, the audio feature vector is fused with the features of numerical, text and image fusion. This method is highly flexible and can be designed for targeted fusion according to the characteristics and importance of different modal data. The specific fusion formula is: , Represents the feature set after all features are fused. Indicates u features of an event, where u is the sum of numerical features, text features, and image features. represents the weight of the initial fusion feature set, Indicates m audio features of an event, Represents the weight of the audio feature.

[0041] S34, use the fusion feature training model decision to make the model output layer output automated decision. After all the multimodal features are fused, after several hidden layers, a more advanced feature representation can be extracted. The training module is placed in the next hidden layer to map the fusion feature with the decision feature, and the training model is learned. The specific formula is: , Represents a mapping function, which is used to map the fusion features and decision features of the same event. Indicates the impact of data quality on the accuracy of fusion features, represents the comprehensive representation of fused features, represents the transpose of the fused feature weights, represents the fused feature vector after several hidden layers, represents the bias term, represents the impact of decision quality on the accuracy of decision features, A comprehensive representation of decision features, represents the transpose of the decision feature weights, represents the decision feature vector, Represents the bias term. The model is trained by using multiple decision events in the training set, so that the model can match the features and decisions of the same event and improve the accuracy of the model's decision through training.

[0042] Set up a rule engine at the output layer, combine the output of the automated decision model with the rule engine to form the final decision. Define the rule logic. If the input numerical feature exceeds the threshold , then the decision is triggered. If the input text feature contains keywords, the decision is triggered. If the input image or audio feature contains some trigger information, the decision is triggered.

[0043] S40, calculate the comprehensive performance index of the model and evaluate the quality of the model-generated decisions.

[0044] Input the validation set data into the trained decision model, generate decision results, and calculate the comprehensive performance index of the model decision results.

[0045] The comprehensive performance index is composed of the similarity and deviation between the decision results output by the model and the decision of the validation set. The specific formula is: ,in, represents the weight of similarity, Indicates the similarity between the decision result output by the model and the decision of the validation set. The larger the value, the better the model performance. Represents the decision result output by the model, Denotes the decision in the verification set. represents the weight of the deviation, It indicates the deviation between the decision result output by the model and the decision of the validation set. The larger the value, the worse the model performance.

[0046] S50, adjusting feature weights and model parameters according to the comprehensive performance index to optimize the model.

[0047] If the comprehensive performance index score is lower than the preset standard, the feature weights and model parameters are dynamically adjusted to achieve continuous optimization of the model. Feature weight update formula , The feature weights at the tth iteration, is the learning rate, which controls the step size of weight update. is the CPI weight of the feature The partial derivative of represents the regularization parameter, which is used to balance the model complexity and fit.

[0048] If the comprehensive performance index score is far below the preset standard, analyze the impact of the preset data quality standard on the deviation calculation. If the impact is large, improve the data quality standard.

[0049] Let the weight of the output layer be , the bias is , the activation function is , then the output of the output layer is , represents the output of the hidden layer, represents the triggering rule of the output layer. The loss function formula is , Represents a real decision, Represents the model output decision, and the gradient of the loss to the output layer output is , loss to the output layer weight The gradient of , loss versus bias The gradient of .

[0050] Calculate the gradient of loss with respect to the hidden layer output as , loss versus hidden layer weights The gradient of , loss versus bias The gradient of . Use the gradient descent method to update the model parameters, and the output layer weight update formula is , the bias update formula is , the model weight update formula is , the model bias update formula is ,in, is the learning rate, is the loss function.

[0051] Adjust the model's feature weights and model parameters, and use the test set data to test the accuracy of the model's output decisions until the accuracy meets the preset standards.

[0052] S60. Keep records of the model training process according to data governance requirements to facilitate audit tracking.

[0053] In order to comply with the principles of security, compliance, standardization, and privacy protection of data governance, the entire process of data use, model calling, and result generation in the decision-making process is recorded to ensure the integrity and auditability of the information and support compliance inspection and audit requirements. The data source, processing method, model version used, parameter settings, and final results of each decision are recorded in the system. During the recording process, the data security and privacy protection principles in data governance are followed to desensitize sensitive data. Regularly audit the recorded data to check the accuracy and compliance of the data, and improve the data governance process and decision-making methods based on the audit results.

[0054] Ensure that the entire decision-making process complies with relevant laws and regulations, provide detailed logging and reporting functions, and facilitate internal and external auditors to review the legality and rationality of the decision-making process. At the same time, manage the permissions of model calls to ensure that only authorized personnel can use the model, in line with the data access control requirements in data governance.

[0055] Embodiment 2 like Figure 2 As shown, Embodiment 2 of the present application provides an automated decision-making system based on data governance, including: Collection module: collects data for training automated decision-making models, preprocesses data using data governance tools, and forms heterogeneous data sources.

[0056] Determine the data direction, data type, and corresponding decisions used to train automated decision models, identify and collect data from multiple sources through data governance tools, including databases, APIs, sensors, log files, etc., and use data governance tools to identify and repair data quality issues on the collected data, including cleaning, deduplication, filling missing values, correcting erroneous data, and converting data into a unified format to ensure data consistency and integrity. Use statistical analysis methods and tools to measure whether the processed data meets the preset quality standards.

[0057] Data that meets quality standards constitute heterogeneous data sources, and data governance tools are used to manage heterogeneous data sources. Data search and discovery functions are provided based on the data catalog, and the source, transformation, and flow of data are tracked based on data lineage. According to the data governance strategy, data is classified and graded. The classification includes customer data, transaction data, device data, etc. The classification includes three levels: public, internal, and confidential. Access rights and security policies are assigned to data of different levels. The classified and graded heterogeneous data sources are stored in the data warehouse. At the same time, data backup and recovery strategies are established to ensure the security and reliability of data.

[0058] Feature extraction module: includes construction submodule and front submodule.

[0059] Construction submodule: used to build automated decision-making models.

[0060] The automated decision-making model includes a front module, an input layer, a hidden layer, and an output layer.

[0061] Pre-module: includes numerical sub-module, text sub-module, image sub-module and audio sub-module.

[0062] Numerical submodule: used for numerical data feature conversion.

[0063] For numerical data, normalization and standardization are performed so that it can be put into the model for training. For algorithms that use fixed-range data, all feature values ​​are converted to a specific range through normalization to ensure that no single feature has too much impact on the model results. The specific formula is: ,in, represents the normalized data value. Represents a single numeric data, Indicates the minimum value of numeric data. Indicates the maximum value of numerical data. For algorithms that are sensitive to the characteristic scale of data, standardization is used to eliminate the dimensional differences between numerical values. The specific formula is: ,in, represents the data value after standardization. represents the mean of numerical data, Represents the standard deviation of numerical data. The converted numerical set is .

[0064] Text submodule: used for text data feature extraction.

[0065] For text data, use the formula Extracting text features can accurately reflect the contribution of words to the content of the document, among which, Represents the characteristics of text data, Yes word In text The number of times it appears in is text The number of occurrences of all words in , Expressive words The frequency of occurrence in a single document, is the total number of texts in different categories, Contains words The number of texts. Use the formula to extract text features multiple times to form a text data feature set .

[0066] Image submodule: used for image data feature extraction.

[0067] For image data, the accelerated robust feature algorithm is used to detect points of interest in the image to extract image features. The accelerated robust feature algorithm determines the feature points by detecting the determinant value of the matrix. At the point and scale s is defined as , where a represents the horizontal position of point H, and b represents the vertical position of point H. represents the second-order derivative of point H in the horizontal direction, represents the second-order derivative of point H in the vertical direction, It represents the mixed second-order derivative between the horizontal and vertical directions of point H, describing the changes of the image at the intersection of the horizontal and vertical directions. The determinant value of the definition matrix is The specific formula is , calculate the derivative The value of needs to define the box filter, in the filter range arrive Use the integral image to quickly obtain the sum of the pixel values ​​in the area covered by the filter. The specific formula is: , where the integral image The calculation formula is . The value of approximate derivative The value of is calculated using the same method , The value of , thus the determinant value , the position of the feature point is determined by the local maximum of the matrix determinant value.

[0068] Taking the feature point as the center, take a square neighborhood and divide it into The sub-region is further divided into smaller sub-regions. The entire neighborhood is divided into 16 small blocks. The sum of the horizontal and vertical Haar wavelet responses is calculated for each sub-region. , and the descriptor D of each interest point is obtained by combination. , normalize the descriptor D to obtain the unique features of the key point , the normalization operation is , Represents a constant to prevent the denominator from being zero. Use the Gaussian mixture model to model the distribution of key point features, aggregate all key point features of the image, and form an image feature set .

[0069] Audio submodule: used for audio data feature extraction.

[0070] For audio data, audio features are extracted by calculating the cepstral coefficients. First, use the formula Pre-emphasize to enhance the high frequency part, where: represents the value of the pre-emphasized audio signal x at time point q, represents the sampling value of the original audio signal x at time point q, represents the pre-emphasis coefficient, Represents the sampling value of the original audio signal x at time point q-1. Secondly, the audio signal is divided into short time frames, and each frame signal is windowed to reduce spectrum leakage. The specific formula is: ,in, represents the value of the windowed audio signal at time point q, Represents the window function. Then use Fourier transform to calculate the spectrum of each frame , represents the frequency domain representation of the windowed signal, k is the frequency index, and FFT represents the fast Fourier transform function, which converts the time domain signal into the frequency domain signal. The spectrum is mapped to the Mel scale using the Mel filter bank. The specific formula is: , represents the output energy of the mth Mel filter, N represents the length of the frequency domain signal, Represents the value of the frequency domain signal at frequency index k, It represents the value of the mth Mel filter at frequency index k. The cepstral coefficient is calculated after taking the logarithm of the Mel spectrum. The specific formula is: ,in, represents the nth cepstral coefficient, M represents the number of Mel filters, represents the logarithmic energy of the mth Mel filter, Represents pi. Main cepstral coefficients are retained as audio features. Output is a two-dimensional matrix , of shape T×F, where T is the time step and F is the number of cepstral coefficients.

[0071] The feature set consists of a numerical feature set, a text feature set, an image feature set, an audio feature matrix, and a decision feature. The decision can be of various types, and the decision features are extracted using the corresponding method. The feature set is divided into a training set, a validation set, and a test set. All three include data features and corresponding decision features. A stratified sampling method is used to ensure that the data distribution of the training set, validation set, and test set is consistent with the overall data. In the classification task, the proportion of each category in the training set, validation set, and test set should be the same as the proportion in the overall data. The combination of different modal data in the training set and validation set should also be representative.

[0072] Fusion module: includes initialization submodule, preliminary fusion submodule, audio advanced feature submodule and fusion submodule.

[0073] Initialization submodule: Initialize the parameters of the automated decision model.

[0074] Initializing the parameters of the automated decision-making model can make the model converge faster in the early stages of training and reduce training time. According to the dimensions of input and output, the weight values ​​are initialized so that the output variance of each layer remains consistent, thereby promoting the stable propagation of the gradient. The specific formula is ,in, Representation weight Sampling from a uniform distribution u, Represents the input dimension. The number of neurons in the input dimension is the sum of the number of input modal features. represents the output dimension, and the number of neurons is the number of neurons in the current layer.

[0075] Preliminary fusion submodule: Input the training set feature set into the input layer of the automated decision model for preliminary fusion.

[0076] The training set feature set is input into the input layer of the automated decision model, and the numerical features, text features, and image features in the feature set are concatenated at the input layer to form a preliminary fused feature vector set. The fusion formula is: ,in, represents the numerical feature weight, represents the j values ​​of the ith event in the numerical feature set, represents the text feature weight, represents the k text features of the i-th event in the text feature set, represents the image feature weight, Represents the o image features of the i-th event in the image feature set, i represents the number of events in the training set, and the attention mechanism is used to dynamically adjust the weights of different modalities.

[0077] Audio high-level feature submodule: used to extract high-level features of audio.

[0078] The audio feature matrix is ​​processed by an independent neural network branch to extract higher-level feature representations. Input audio feature matrix , local time-frequency features are extracted through the convolution layer. The specific formula is: , is the feature map after convolution, Represents the convolution operation function. The pooling layer is used to reduce the dimension and retain important features. The specific formula is: , represents the feature map after pooling, Represents the maximum pooling operation function. Mapping the feature map to a feature vector of fixed length, the specific formula is , Represents the audio feature vector, Dense represents the fully connected layer, and Flatten() is used to flatten the feature map into a one-dimensional vector.

[0079] Fusion submodule: The audio high-level features and preliminary fusion features are fused in the hidden layer.

[0080] At a certain position in the hidden layer, the audio feature vector is fused with the features of numerical, text and image fusion. This method is highly flexible and can be designed for targeted fusion according to the characteristics and importance of different modal data. The specific fusion formula is: , Represents the feature set after all features are fused. Indicates u features of an event, where u is the sum of numerical features, text features, and image features. represents the weight of the initial fusion feature set, Indicates m audio features of an event, Represents the weight of the audio feature.

[0081] Training module: Use fusion features to train model decisions so that the model output layer can output automated decisions.

[0082] After all the multimodal features are fused, after several hidden layers, a more advanced feature representation can be extracted. A training module is placed in the next hidden layer to map the fused features with the decision features, and the training model is trained for learning. The specific formula is: , Represents a mapping function, which is used to map the fusion features and decision features of the same event. Indicates the impact of data quality on the accuracy of fusion features, represents the comprehensive representation of fused features, represents the transpose of the fused feature weights, represents the fused feature vector after several hidden layers, represents the bias term, represents the impact of decision quality on the accuracy of decision features, A comprehensive representation of decision features, represents the transpose of the decision feature weights, represents the decision feature vector, Represents the bias term. The model is trained by using multiple decision events in the training set, so that the model can match the features and decisions of the same event and improve the accuracy of the model's decision through training.

[0083] Set up a rule engine at the output layer, combine the output of the automated decision model with the rule engine to form the final decision. Define the rule logic. If the input numerical feature exceeds the threshold , then the decision is triggered. If the input text feature contains keywords, the decision is triggered. If the input image or audio feature contains some trigger information, the decision is triggered.

[0084] Evaluation module: Calculates the comprehensive performance index of the automated decision-making model and evaluates the quality of the decisions generated by the model.

[0085] Input the validation set data into the trained decision model, generate decision results, and calculate the comprehensive performance index of the model decision results.

[0086] The comprehensive performance index is composed of the similarity and deviation between the decision results output by the model and the decision of the validation set. The specific formula is: ,in, represents the weight of similarity, Indicates the similarity between the decision result output by the model and the decision of the validation set. The larger the value, the better the model performance. Represents the decision result output by the model, Denotes the decision in the verification set. represents the weight of the deviation, It indicates the deviation between the decision result output by the model and the decision of the validation set. The larger the value, the worse the model performance.

[0087] Optimization module: adjusts feature weights and model parameters according to the comprehensive performance index to optimize the model.

[0088] If the comprehensive performance index score is lower than the preset standard, the feature weights and model parameters are dynamically adjusted to achieve continuous optimization of the model. Feature weight update formula , The feature weights at the tth iteration, is the learning rate, which controls the step size of weight update. is the CPI weight of the feature The partial derivative of represents the regularization parameter, which is used to balance the model complexity and fit.

[0089] If the comprehensive performance index score is far below the preset standard, analyze the impact of the preset data quality standard on the deviation calculation. If the impact is large, improve the data quality standard.

[0090] Let the weight of the output layer be , the bias is , the activation function is , then the output of the output layer is , represents the output of the hidden layer, represents the triggering rule of the output layer. The loss function formula is , Represents a real decision, Represents the model output decision, and the gradient of the loss to the output layer output is , loss to the output layer weight The gradient of , loss versus bias The gradient of .

[0091] Calculate the gradient of loss with respect to the hidden layer output as , loss versus hidden layer weights The gradient of , loss versus bias The gradient of . Use the gradient descent method to update the model parameters, and the output layer weight update formula is , the bias update formula is , the model weight update formula is , the model bias update formula is ,in, is the learning rate, is the loss function.

[0092] Adjust the model's feature weights and model parameters, and use the test set data to test the accuracy of the model's output decisions until the accuracy meets the preset standards.

[0093] Recording module: Save records of the model training process according to data governance requirements to facilitate audit tracking.

[0094] In order to comply with the principles of security, compliance, standardization, and privacy protection of data governance, the entire process of data use, model calling, and result generation in the decision-making process is recorded to ensure the integrity and auditability of the information and support compliance inspection and audit requirements. The data source, processing method, model version used, parameter settings, and final results of each decision are recorded in the system. During the recording process, the data security and privacy protection principles in data governance are followed to desensitize sensitive data. Regularly audit the recorded data to check the accuracy and compliance of the data, and improve the data governance process and decision-making methods based on the audit results.

[0095] Ensure that the entire decision-making process complies with relevant laws and regulations, provide detailed logging and reporting functions, and facilitate internal and external auditors to review the legality and rationality of the decision-making process. At the same time, manage the permissions of model calls to ensure that only authorized personnel can use the model, in line with the data access control requirements in data governance.

[0096] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. An automated decision-making method based on data governance, characterized in that: include: S10. Collect data for training automated decision-making models, pre-process data using data governance tools, and form heterogeneous data sources; S20, constructing an automated decision model, inputting heterogeneous data sources into the automated decision model front-end module, extracting data features of multimodal data using feature engineering methods, and forming a feature set; S30, inputting the feature set into the input layer of the automated decision model, performing feature fusion of the multimodal data features in the hidden layer, and using the fused features to train the model decision, so that the model output layer outputs the automated decision; S40, calculate the comprehensive performance index of the automated decision-making model to evaluate the quality of the decisions generated by the model; S50, adjusting feature weights and model parameters according to the comprehensive performance index to optimize the model; S60. Keep records of the model training process according to data governance requirements to facilitate audit tracking.

2. The automated decision-making method based on data governance according to claim 1, characterized in that: The data direction, data type and corresponding decisions used to train automated decision models are identified and collected from multiple sources through data governance tools. Data governance tools identify and repair data quality issues on the collected data, including cleaning, deduplication, filling missing values, correcting erroneous data, and converting data into a unified format to ensure data consistency and integrity.

3. The automated decision-making method based on data governance according to claim 1, characterized in that: Construct an automated decision-making model, which includes a front-end module, an input layer, a hidden layer, and an output layer.

4. The automated decision-making method based on data governance according to claim 1, characterized in that: The feature fusion of multimodal data features in the hidden layer includes the following sub-steps: initializing the parameters of the automated decision model; inputting the training set feature set into the automated decision model input layer for preliminary fusion; extracting high-level features of the audio, and fusing the high-level audio features with the preliminary fusion features in the hidden layer.

5. The automated decision-making method based on data governance according to claim 1, characterized in that: The decision of using fusion feature training model is to place a training module in the hidden layer, map the fusion feature with the decision feature, and train the model for learning.

6. The automated decision-making method based on data governance according to claim 1, characterized in that: The comprehensive performance index is composed of the similarity and deviation between the decision results output by the model and the decision of the validation set. The larger the similarity value, the better the model performance, and the larger the deviation value, the worse the model performance.

7. An automated decision-making system based on data governance, characterized in that: include: Acquisition module: collects data for training automated decision-making models, preprocesses data using data governance tools, and forms heterogeneous data sources; Feature extraction module: Build an automated decision model, input heterogeneous data sources into the automated decision model front-end module, and use feature engineering methods to extract data features of multimodal data to form a feature set; Fusion module: inputs the feature set into the input layer of the automated decision model, and multimodal data features are fused in the hidden layer; Training module: Use fusion features to train model decisions so that the model output layer outputs automated decisions; Evaluation module: Calculates the comprehensive performance index of the automated decision-making model and evaluates the quality of the decisions generated by the model; Optimization module: adjusts feature weights and model parameters according to the comprehensive performance index to optimize the model; Recording module: Save records of the model training process according to data governance requirements to facilitate audit tracking.

Images