Data management full-link monitoring system and method based on artificial intelligence

Abstract

The invention discloses a data management full-link monitoring system and method based on artificial intelligence, and relates to the technical field of full-link monitoring, and the system comprises an entity feature extraction module, an entity prediction module, a change degree analysis module, an entity state judgment module, an entity association analysis module and a visual display module. The method comprises the following steps: analyzing a task issued by a user, extracting features in multi-modal data of entities, defining an entity set, predicting states of the entities at a next time point, setting feature dimensions, calculating to obtain change degrees of the entities, considering the change degrees of the entities, dynamically judging whether the entities are normal or not, and checking association degrees among the entities. According to the method, the dynamic threshold value is calculated through the preset reference time point, the problem of false report or missing report of a traditional static threshold value is avoided, and the monitoring accuracy and timeliness are improved.

Description

Technical Field

[0001] The present invention relates to the field of full-link monitoring technology, and specifically to an artificial intelligence-based data governance full-link monitoring system and method. Background Art

[0002] Under the traditional data governance framework, enterprises need to go through lengthy steps such as assessing the status quo, setting goals, executing plans, and continuous monitoring, involving multiple modules such as data quality, security, and lifecycle management. The governance link is cumbersome and lacks core grasps. Problems such as high data cost, low efficiency, and poor stability are common. Data warehouse development lacks specifications, resulting in resource waste and task duplication. In addition, traditional monitoring methods rely on static thresholds, which are difficult to adapt to dynamically changing business scenarios, easily cause false positives or omissions, and cannot accurately identify anomalies. This lack of intelligent and standardized governance method causes enterprises to fall into internal friction in data governance and is difficult to form sustainable value. As business systems evolve towards cloud, microservice, and middle-end, business requests frequently interact across platforms and systems, and data transmission links become increasingly complex. Traditional monitoring methods are often limited to single-point or local monitoring, making it difficult to fully capture the operating status of nodes in the entire link, resulting in system anomalies that are difficult to detect in a timely manner. For example, State Grid Information and Communication Company faces the need to monitor core business links in its digital transformation, and needs to integrate multi-source data to achieve full-link visualization, but existing tools cannot meet the real-time perception and autonomous discovery capabilities in complex scenarios. This limitation makes enterprises inefficient in troubleshooting cross-system failures, seriously affecting business stability. Some systems rely only on a single data source or simple algorithm, lacking fusion analysis of multimodal data, resulting in incomplete feature extraction. At the same time, traditional methods lack deep modeling of the correlation between entities, making it difficult to trace the root cause of anomalies and insufficient global governance capabilities. Summary of the invention

[0003] The purpose of the present invention is to provide an artificial intelligence-based data governance full-link monitoring system and method to solve the problems raised in the above-mentioned background technology.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: a data governance full-link monitoring method based on artificial intelligence, comprising the following steps: S1. Analyze the tasks assigned by the user and extract the features in the multimodal data of the entity; S2, define the entity set and predict the state of the entity at the next time point; S3. Set the number of feature dimensions and calculate the degree of change of the entity; S4, considering the degree of change of the entity, dynamically determine whether the entity is normal; S5. Check the correlation between entities and find the entity set with abnormal impact; S6. Perform visual display and mark the time points of abnormal entities.

[0005] Furthermore, in step S1, feature extraction is performed on the multimodal data of any entity, where the entity is a task assigned by any user, and the multimodal data includes: log text, time series indicators, and interface response records, and any log text feature X is obtained, where X={X1,X2,…,X u ,…,X U}, where U represents the number of keywords in the log text, X u represents the ratio of the number of times the u-th keyword appears in the log text to the number of times all keywords appear; obtain any time series indicator feature Y, Y={Y1,Y2,…,Y v ,…,Y V}, where V represents the number of tasks in the timing indicator, Y v represents the ratio of the duration required for the vth task to the duration required for all tasks in the timing indicator; obtain any interface response record feature Z, Z={Z1,Z2,…,Z w ,…,Z W}, where W represents the number of interface responses in the interface response record, Z w Represents the ratio of the time required for the wth response to the total time of W responses in the interface response record. Step S1 significantly improves the comprehensiveness and accuracy of full-link monitoring of data governance by extracting features from multimodal data such as log text, timing indicators and interface response records. The frequency ratio of keywords extracted from log text features can effectively capture key information and abnormal patterns in the task execution process; the timing indicator features intuitively reflect the distribution of task time consumption and efficiency changes through task duration ratio analysis; the interface response record features use the response duration ratio to characterize the fluctuation of interface performance. This multi-dimensional and standardized feature extraction method not only comprehensively covers all aspects of entity operation, but also eliminates data dimension differences through ratio calculation, making data of different modes comparable, and providing richer and more accurate basic data for subsequent state prediction, change analysis and abnormal judgment, thereby enhancing the system's adaptability to complex business scenarios and improving the efficiency and reliability of abnormality identification.

[0006] Furthermore, in step S2, define the entity set E={e1,e2,…,e i ,…,e j ,…,e I}, where I represents the number of entities, e i Represents the i-th entity, and defines the relationship set R={r ij (t)}, where r ij (t) represents the entity e at the tth time point iand entity e j The correlation between i The state at the tth time point is S i (t), the S i (t) is a set of log text features, time series index features and interface response record features for any entity. The method of obtaining the correlation degree is, for example, to obtain the entity e with time point t as the deadline. j The number of abnormalities A j , get the entity e i Abnormal Entity j The number of abnormalities A ij , then r ij (t) = A ij / A j ; According to the time interval Δt between adjacent time points, the entity e at the (t+1)th time point is obtained i The status is S i (t+1): ; Step S2 builds a systematic entity management and association analysis framework by defining entity sets and relationship sets. The clear division of entity sets enables the system to clearly identify and track each task entity, providing an accurate object basis for full-link monitoring; the quantitative calculation of the correlation in the relationship set (such as proportional analysis based on the number of abnormalities) effectively reveals the potential impact relationship between entities and helps to quickly locate the abnormal propagation path. This structured modeling not only enhances the system's ability to understand complex business links, but also achieves an upgrade from static monitoring to dynamic prediction by dynamically predicting the state of the entity at the next moment. Through time interval analysis and state deduction, the system can perceive the trend of entity state changes in advance, thereby discovering abnormal signs more promptly, improving the foresight and initiative of monitoring, and laying a solid foundation for early intervention and precise governance of full-link abnormalities.

[0007] Further, in step S3, the feature dimension number M=U+V+W is set, the mth dimension is set as an important dimension or a non-important dimension, and a weight ω is set for the mth dimension feature. m , based on the tth time point S i The m-th dimension feature of (t) , the (t+1)th time point S i The m-th dimension feature of (t+1) , and thus calculate the entity e i The degree of change ΔS i : ; Step S3 significantly improves the scientificity and flexibility of the calculation of the degree of entity change by setting the number of feature dimensions and assigning weights to different dimensions. The log text, timing indicators, interface response records and other dimensions of multimodal data are integrated into a unified feature space, which comprehensively covers all aspects of the entity state; the differentiated weight setting for important dimensions and unimportant dimensions enables the system to highlight the impact of key indicators according to business needs and avoid interference from secondary information, thereby more accurately reflecting the actual change trend of the entity. The dynamic calculation of the degree of change based on the features of adjacent time points not only realizes the real-time tracking of the evolution of the entity state, but also captures subtle, progressive changes that may indicate anomalies, enhancing the sensitivity and foresight of monitoring. This quantitative analysis method provides a more convincing basis for subsequent anomaly judgments, effectively improves the accuracy and reliability of full-link monitoring, and can more efficiently identify potential risks, especially in complex business scenarios.

[0008] Further, in step S4, a number of reference time points C is preset, and ΔS at C time points before time point t is calculated. i The mean μ i and standard deviation σ i , thus setting e i The change threshold K i (t): ; When the degree of change is greater than the change threshold, the entity e is judged i Abnormal; when the degree of change is less than or equal to the change degree threshold, the entity e is judged i normal; Step S4 significantly improves the scientificity and adaptability of anomaly judgment by presetting reference time points and dynamically calculating thresholds. By setting the degree of change threshold based on the mean and standard deviation of historical data, the system can automatically adjust the judgment criteria according to the real-time data distribution characteristics of the entity, avoiding the drawbacks of traditional static thresholds that are divorced from actual business scenarios. This dynamic mechanism can keenly capture the laws of data fluctuations. It will neither miss potential anomalies due to excessively high thresholds nor generate a large number of false alarms due to excessively low thresholds, thereby more accurately identifying true abnormal states. At the same time, combined with the statistical analysis of time series, the system can adapt to changes in data characteristics in different time periods, enhance the robustness to complex business environments, provide a more reliable judgment basis for full-link monitoring, and effectively improve the accuracy and stability of anomaly detection; Further, in step S5, check the jth entity e j , if entity e j There is ΔS C time points before time point t j ≥K j (t), and r ij When (t)>r0, judge ei The exception is caused by e j Influence, substitute j=1,2,3,…,I one by one, and find the influence e i , where ΔSj is the degree of change of entity ej from the tth time point to the (t+1)th time point; Step S5 significantly improves the accuracy and efficiency of tracing the impact of anomalies by combining the entity's own abnormal state with correlation analysis. When judging the root cause of entity anomalies, this mechanism not only requires that the inspected entity has significant changes within the time window, but also requires that its correlation with the target entity exceeds the threshold, thereby avoiding misjudgment under a single condition. This double verification method can accurately lock the set of entities that actually produce abnormal impacts, rather than relying solely on local anomalies or simple associations. By traversing all entities and verifying them one by one, the system can comprehensively sort out the abnormal propagation path, helping operation and maintenance personnel to quickly locate the source of the problem and shorten troubleshooting time. At the same time, this global correlation analysis capability effectively enhances the prediction and control of the spread of anomalies in complex business links, fundamentally reduces the impact of anomalies on the entire link, and improves the stability and reliability of the data governance system, especially in large-scale distributed systems, where its efficiency and practicality can be better reflected; Furthermore, in step S6, the abnormal entity e i To display the entity i Set up a set of subordinate influencing entities that will affect e i Fill in the name of the abnormal entity and mark the time point t.

[0009] An artificial intelligence-based data governance full-link monitoring system, the system comprising: an entity feature extraction module, an entity prediction module, a change degree analysis module, an entity state judgment module, an entity association analysis module and a visualization display module; The entity feature extraction module is used to analyze the tasks assigned by the user and extract features from the multimodal data of the entity; The entity prediction module is used to define an entity set and predict the state of the entity at the next time point; The change degree analysis module is used to set the number of feature dimensions and calculate the change degree of the entity; The entity status judgment module is used to consider the degree of change of the entity and dynamically judge whether the entity is normal; The entity association analysis module is used to check the association between entities and find the entity set with abnormal impact; The visualization module is used to perform visualization and mark time points for abnormal entities.

[0010] Furthermore, the entity feature extraction module, entity prediction module, change degree analysis module, entity state judgment module, entity association analysis module and visualization display module are connected through a wireless network. When the system updates data, it is uploaded to the cloud platform for backup through the wireless network.

[0011] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: On the one hand, the method can comprehensively capture the multi-dimensional changes of entity states by extracting and fusion analyzing the features of multimodal data such as log text, timing indicators and interface response records. Combined with the dynamic prediction of entity states and the quantitative calculation of the degree of change, anomalies can be accurately identified in real time. By calculating dynamic thresholds at preset reference time points, the system can adapt to changes in different business scenarios, avoiding the problem of false positives or false negatives of traditional static thresholds, and significantly improving the accuracy and timeliness of monitoring; On the one hand, the method can quickly locate the propagation path of abnormal impact by building a correlation model between entities. When an entity is abnormal, the system can calculate the correlation between entities based on historical data and trace the root entity set that affects its abnormality. This global correlation analysis capability not only shortens the troubleshooting time, but also helps operation and maintenance personnel to fundamentally solve problems and avoid the spread of abnormalities, thereby improving the stability and reliability of the entire data governance chain.

[0012] On the other hand, through the visualization module, the system presents complex monitoring data in an intuitive form, allowing operation and maintenance personnel to quickly grasp the status of the entire link. The time point marking function of abnormal entities further simplifies the problem location process, allowing operators to take timely countermeasures. In addition, the wireless network connection and cloud backup between system modules ensure the security and scalability of data, providing users with an efficient and convenient one-stop monitoring solution, significantly improving the efficiency and management level of data governance. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 It is a structural diagram of a data governance full-link monitoring system based on artificial intelligence of the present invention; Figure 2 It is a flow chart of a full-link monitoring method for data governance based on artificial intelligence of the present invention. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] See also Figure 1 and Figure 2 , the present invention provides a technical solution: a data governance full-link monitoring method based on artificial intelligence, comprising the following steps: S1. Analyze the tasks assigned by the user and extract the features in the multimodal data of the entity; S2, define the entity set and predict the state of the entity at the next time point; S3. Set the number of feature dimensions and calculate the degree of change of the entity; S4, considering the degree of change of the entity, dynamically determine whether the entity is normal; S5. Check the correlation between entities and find the entity set with abnormal impact; S6. Perform visual display and mark the time points of abnormal entities.

[0016] In step S1, feature extraction is performed on the multimodal data of any entity, where the entity is a task assigned by any user, and the multimodal data includes log text, time series indicators, and interface response records, and any log text feature X is obtained, where X={X1,X2,…,X u ,…,X U}, where U represents the number of keywords in the log text, X u represents the ratio of the number of times the u-th keyword appears in the log text to the number of times all keywords appear; obtain any time series indicator feature Y, Y={Y1,Y2,…,Y v ,…,Y V}, where V represents the number of tasks in the timing indicator, Y v represents the ratio of the duration required for the vth task to the duration required for all tasks in the timing indicator; obtain any interface response record feature Z, Z={Z1,Z2,…,Z w ,…,Z W}, where W represents the number of interface responses in the interface response record, Z wRepresents the ratio of the time required for the wth response to the total time of W responses in the interface response record. Step S1 significantly improves the comprehensiveness and accuracy of full-link monitoring of data governance by extracting features from multimodal data such as log text, timing indicators and interface response records. The frequency ratio of keywords extracted from log text features can effectively capture key information and abnormal patterns in the task execution process; the timing indicator features intuitively reflect the distribution of task time consumption and efficiency changes through task duration ratio analysis; the interface response record features use the response duration ratio to characterize the fluctuation of interface performance. This multi-dimensional and standardized feature extraction method not only comprehensively covers all aspects of entity operation, but also eliminates data dimension differences through ratio calculation, making data of different modes comparable, and providing richer and more accurate basic data for subsequent state prediction, change analysis and abnormal judgment, thereby enhancing the system's adaptability to complex business scenarios and improving the efficiency and reliability of abnormality identification.

[0017] In step S2, define the entity set E={e1,e2,…,e i ,…,e j ,…,e I}, where I represents the number of entities, e i Represents the i-th entity, and defines the relationship set R={r ij (t)}, where r ij (t) represents the entity e at the tth time point i and entity e j The correlation between i The state at the tth time point is S i (t), the S i (t) is a set of log text features, time series index features and interface response record features for any entity. The method of obtaining the correlation degree is, for example, to obtain the entity e with time point t as the deadline. j The number of abnormalities A j , get the entity e i Abnormal Entity j The number of abnormalities A ij , then r ij (t) = A ij / A j ; According to the time interval Δt between adjacent time points, the entity e at the (t+1)th time point is obtained i The status is S i (t+1): ; Step S2 builds a systematic entity management and association analysis framework by defining entity sets and relationship sets. The clear division of entity sets enables the system to clearly identify and track each task entity, providing an accurate object basis for full-link monitoring; the quantitative calculation of the correlation in the relationship set (such as proportional analysis based on the number of abnormalities) effectively reveals the potential impact relationship between entities and helps to quickly locate the abnormal propagation path. This structured modeling not only enhances the system's ability to understand complex business links, but also achieves an upgrade from static monitoring to dynamic prediction by dynamically predicting the state of the entity at the next moment. Through time interval analysis and state deduction, the system can perceive the trend of entity state changes in advance, thereby discovering abnormal signs more promptly, improving the foresight and initiative of monitoring, and laying a solid foundation for early intervention and precise governance of full-link abnormalities.

[0018] In step S3, the feature dimension number M=U+V+W is set, the mth dimension is set as an important dimension or a non-important dimension, and the weight ω is set for the mth dimension feature m , based on the tth time point S i The m-th dimension feature of (t) , the (t+1)th time point S i The m-th dimension feature of (t+1) , and thus calculate the entity e i The degree of change ΔS i : ; Step S3 significantly improves the scientificity and flexibility of the calculation of the degree of entity change by setting the number of feature dimensions and assigning weights to different dimensions. The log text, timing indicators, interface response records and other dimensions of multimodal data are integrated into a unified feature space, which comprehensively covers all aspects of the entity state; the differentiated weight setting for important dimensions and unimportant dimensions enables the system to highlight the impact of key indicators according to business needs and avoid interference from secondary information, thereby more accurately reflecting the actual change trend of the entity. The dynamic calculation of the degree of change based on the features of adjacent time points not only realizes the real-time tracking of the evolution of the entity state, but also captures subtle, progressive changes that may indicate anomalies, enhancing the sensitivity and foresight of monitoring. This quantitative analysis method provides a more convincing basis for subsequent anomaly judgments, effectively improves the accuracy and reliability of full-link monitoring, and can more efficiently identify potential risks, especially in complex business scenarios.

[0019] In step S4, a number of reference time points C is preset, and ΔS at C time points before time point t is calculated. i The mean μ i and standard deviation σ i , thus setting e i The change threshold Ki (t): ; When the degree of change is greater than the change threshold, the entity e is judged i Abnormal; when the degree of change is less than or equal to the change degree threshold, the entity e is judged i normal; Step S4 significantly improves the scientificity and adaptability of anomaly judgment by presetting reference time points and dynamically calculating thresholds. By setting the change degree threshold based on the mean and standard deviation of historical data, the system can automatically adjust the judgment criteria according to the real-time data distribution characteristics of the entity, avoiding the disadvantages of traditional static thresholds being out of touch with actual business scenarios. This dynamic mechanism can keenly capture the laws of data fluctuations, and will neither miss potential anomalies due to excessively high thresholds nor generate a large number of false alarms due to excessively low thresholds, thereby more accurately identifying true abnormal states. At the same time, combined with the statistical analysis of time series, the system can adapt to changes in data characteristics in different time periods, enhance the robustness to complex business environments, provide a more reliable basis for judgment for full-link monitoring, and effectively improve the accuracy and stability of anomaly detection.

[0020] In step S5, check the jth entity e j , if entity e j There is ΔS C time points before time point t j ≥K j (t), and r ij When (t)>r0, judge e i The exception is caused by e j Influence, substitute j=1,2,3,…,I one by one, and find the influence e i , where ΔSj is the degree of change of entity ej from the tth time point to the (t+1)th time point; Step S5 significantly improves the accuracy and efficiency of tracing the impact of anomalies by combining the entity's own abnormal state with correlation analysis. When judging the root cause of entity anomalies, this mechanism not only requires that the inspected entity has significant changes within the time window, but also requires that its correlation with the target entity exceeds the threshold, thereby avoiding misjudgment under a single condition. This double verification method can accurately lock the set of entities that actually produce abnormal impacts, rather than relying solely on local anomalies or simple associations. By traversing all entities and verifying them one by one, the system can comprehensively sort out the abnormal propagation path, helping operation and maintenance personnel to quickly locate the source of the problem and shorten troubleshooting time. At the same time, this global correlation analysis capability effectively enhances the prediction and control of the spread of anomalies in complex business links, fundamentally reduces the impact of anomalies on the entire link, and improves the stability and reliability of the data governance system, especially in large-scale distributed systems, where its efficiency and practicality can be better reflected; In step S6, the abnormal entity e i To display the entity i Set up a set of subordinate influencing entities that will affect e i Fill in the name of the abnormal entity and mark the time point t.

[0021] An artificial intelligence-based data governance full-link monitoring system, the system comprising: an entity feature extraction module, an entity prediction module, a change degree analysis module, an entity state judgment module, an entity association analysis module and a visualization display module; The entity feature extraction module is used to analyze the tasks assigned by the user and extract features from the multimodal data of the entity; The entity prediction module is used to define an entity set and predict the state of the entity at the next time point; The change degree analysis module is used to set the number of feature dimensions and calculate the change degree of the entity; The entity status judgment module is used to consider the degree of change of the entity and dynamically judge whether the entity is normal; The entity association analysis module is used to check the association between entities and find the entity set with abnormal impact; The visualization module is used to perform visualization and mark time points for abnormal entities.

[0022] Example 1: An embodiment of a data governance full-link monitoring method based on artificial intelligence can be applied to the cargo dispatching system of a logistics company. The specific implementation steps are as follows: S1 Feature extraction: For cargo dispatching tasks, the system extracts features from multimodal data. In terms of log text, the key words in the records, such as "route planning failure" and "vehicle delay", are analyzed, and the proportion of the number of occurrences of each keyword is calculated to form text features that reflect the operating status of the system; in terms of timing indicators, the proportion of the time consumed by tasks in each link (such as order allocation, vehicle dispatching, and path optimization) to the total task time is counted to measure the efficiency changes of each link; in terms of interface response records, the proportion of the response time of each interface (such as order interface and vehicle status interface) to the total response time is counted to evaluate the stability of interface performance.

[0023] S2 Entity Prediction: Define entity sets, including order allocation system, vehicle dispatch system, route optimization system, etc. Calculate the association between entities through historical data. For example, when the order allocation system is abnormal, the probability of abnormality in the vehicle dispatch system. Based on time interval analysis, combined with the current status of each entity (such as order allocation delay, vehicle dispatch timeout), predict the status of each entity at the next moment. For example, predict that the route optimization system may have calculation deviations due to vehicle dispatch delays.

[0024] S3 change analysis: Integrate the characteristic dimensions of logs, time series, and interfaces, and set weights for each dimension (such as setting vehicle dispatch time as a high weight). By comparing the changes in data of each dimension at adjacent time points, calculate the comprehensive degree of change, and quantify the dynamic fluctuations of each entity. For example, if the proportion of vehicle dispatch time suddenly increases and the weight is high, the overall degree of change will increase significantly.

[0025] S4 Abnormal Judgment: Preset the reference time range (such as the last 10 time points), calculate the average value and fluctuation range of the historical change degree, and dynamically set the abnormal threshold. If the current change degree of an entity exceeds the threshold, it is judged as abnormal. For example, if the change degree of the path optimization system exceeds the threshold for three consecutive times, the system immediately triggers an abnormal alarm.

[0026] S5 Association tracing: Traverse all entities and check entities that have anomalies within the time window and are highly correlated with the target anomaly entity. For example, when an order distribution system anomaly is found, the vehicle dispatch system is detected to have frequent anomalies in the same time period, and the historical correlation between the two exceeds the preset value, thus determining that the anomaly of the vehicle dispatch system is the main cause of the order distribution delay.

[0027] S6 Visual Display: Displays abnormal entities (such as the order distribution system) and their affected entities (vehicle dispatch system) in an intuitive graphical interface, marking the specific time when the abnormality occurred. Operation and maintenance personnel can quickly locate the problem through the interface. For example, in the dispatch link diagram, the order distribution system node is highlighted in red and marked with "14:20 abnormality". The subordinate vehicle dispatch system node simultaneously displays the related abnormalities, which makes it easy for the team to quickly develop solutions and restore the normal operation of the system.

[0028] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A full-link monitoring method for data governance based on artificial intelligence, characterized by: The method comprises the following steps: S1. Analyze the tasks assigned by the user and extract the features in the multimodal data of the entity; S2, define the entity set and predict the state of the entity at the next time point; S3. Set the number of feature dimensions and calculate the degree of change of the entity; S4, considering the degree of change of the entity, dynamically determine whether the entity is normal; S5. Check the correlation between entities and find the entity set with abnormal impact; S6. Perform visual display and mark the time points of abnormal entities.

2. According to the artificial intelligence-based data governance full-link monitoring method of claim 1, it is characterized by: In step S1, feature extraction is performed on the multimodal data of any entity, where the entity is a task assigned by any user, and the multimodal data includes log text, time series indicators, and interface response records, and any log text feature X is obtained, where X={X1,X2,…,X u ,…,X U }, where U represents the number of keywords in the log text, X u represents the ratio of the number of times the u-th keyword appears in the log text to the number of times all keywords appear; obtain any time series indicator feature Y, Y={Y1,Y2,…,Y v ,…,Y V }, where V represents the number of tasks in the timing indicator, Y v represents the ratio of the duration required for the vth task to the duration required for all tasks in the timing indicator; obtain any interface response record feature Z, Z={Z1,Z2,…,Z w ,…,Z W }, where W represents the number of interface responses in the interface response record, Z w It represents the ratio of the time required for the wth response to the total time required for W responses in the interface response record.

3. According to claim 2, a data governance full-link monitoring method based on artificial intelligence is characterized in that: In step S2, define the entity set E={e1,e2,…,e i ,…,e j ,…,e I }, where I represents the number of entities, e i Represents the i-th entity, and defines the relationship set R={r ij (t)}, where r ij (t) represents the entity e at the tth time point i and entity e j The correlation between i The state at the tth time point is S i (t), the S i (t) is a set of log text features, time series index features and interface response record features for any entity. The method for obtaining the association degree is: taking time point t as the deadline, obtain entity e j The number of abnormalities A j , get the entity e i Abnormal Entity j The number of abnormalities A ij , then r ij (t) = A ij / A j ; According to the time interval Δt between adjacent time points, the entity e at the (t+1)th time point is obtained i The status is S i (t+1).

4. According to claim 3, a data governance full-link monitoring method based on artificial intelligence is characterized in that: In step S3, the feature dimension number M=U+V+W is set, the mth dimension is set as an important dimension or a non-important dimension, and the weight ω is set for the mth dimension feature m , based on the tth time point S i The m-th dimension feature of (t) , the (t+1)th time point S i The m-th dimension feature of (t+1) , and thus calculate the entity e i The degree of change ΔS i .

5. According to claim 4, a data governance full-link monitoring method based on artificial intelligence is characterized in that: In step S4, a number of reference time points C is preset, and ΔS at C time points before time point t is calculated. i The mean μ i and standard deviation σ i , thus setting e i The change threshold K i (t): When the degree of change is greater than the change threshold, the entity e is judged i abnormal; When the degree of change is less than or equal to the change degree threshold, the entity e is judged i normal.

6. According to claim 5, a data governance full-link monitoring method based on artificial intelligence is characterized in that: In step S5, check the jth entity e j , if entity e j There is ΔS C time points before time point t j ≥K j (t), and r ij When (t)>r0, judge e i The exception is caused by e j Influence, substitute j=1,2,3,…,I one by one, and find the influence e i , where ΔSj is the degree of change of entity ej from the tth time point to the (t+1)th time point.

7. According to claim 5, a data governance full-link monitoring method based on artificial intelligence is characterized in that: In step S6, the abnormal entity e i To display the entity i Set up a set of subordinate influencing entities that will affect e i Fill in the name of the abnormal entity and mark the time point t.

8. A data governance full-link monitoring system based on artificial intelligence, the system is applied to a data governance full-link monitoring method based on artificial intelligence according to any one of claims 1-7, characterized in that: The system includes: an entity feature extraction module, an entity prediction module, a change degree analysis module, an entity state judgment module, an entity association analysis module and a visualization display module; The entity feature extraction module is used to analyze the tasks assigned by the user and extract features from the multimodal data of the entity; The entity prediction module is used to define an entity set and predict the state of the entity at the next time point; The change degree analysis module is used to set the number of feature dimensions and calculate the change degree of the entity; The entity status judgment module is used to consider the degree of change of the entity and dynamically judge whether the entity is normal; The entity association analysis module is used to check the association between entities and find the entity set with abnormal impact; The visualization module is used to perform visualization and mark time points for abnormal entities.

9. The artificial intelligence-based data governance full-link monitoring system according to claim 8 is characterized in that: The entity feature extraction module, entity prediction module, change degree analysis module, entity state judgment module, entity association analysis module and visualization display module are connected through a wireless network. When the system updates data, it is uploaded to the cloud platform for backup through the wireless network.

Images