Data processing analysis method, apparatus and device, and computer program product

Abstract

The invention discloses a data processing analysis method, device and equipment and a computer program product, and the data processing analysis method comprises the steps: determining a cloud native observable object, and carrying out the collection and processing of observable data of the cloud native observable object, and obtaining initial observable data; performing feature information extraction based on object categories on the initial observable data to obtain observable feature basic data; and obtaining real-time index observation data of the cloud native observable object, and performing traceability positioning analysis according to the real-time index observation data and the observable feature basic data to obtain traceability positioning result data. By reasonably associating different data types, more efficient and reasonable data correlation analysis processing is realized, the root of a fault problem can be more accurately determined, the efficiency of data operation is ensured, the observability of data is realized, and the accuracy of fault diagnosis and the efficiency of data processing are improved.

Description

Technical Field

[0001] The present application relates to the technical field of data processing and analysis, and in particular to a data processing and analysis method, apparatus and equipment, and a computer program product. Background Art

[0002] Cloud-native observability is the ability to measure, prevent, discover, locate, and resolve business problems through correlation analysis of machine data such as logs, indicators, and links across the entire business operation process, including development and testing, IT operations and maintenance, business operations, security and compliance, thereby improving business efficiency.

[0003] Due to the large data types and volumes observed by cloud native, how to reasonably and efficiently organize different types of data to achieve interrelated analysis and processing and ensure the normal operation of data has become a direct problem faced by cloud native.

[0004] Currently, most cloud-native approaches focus on independently evaluating different types of data and then developing comprehensive analysis and evaluation plans to efficiently process and analyze the interrelated data. However, this approach does not substantially change the way data is analyzed and processed; it merely extends data monitoring to a certain extent, while still processing and analyzing different types of data independently. These approaches are inefficient and costly. Summary of the Invention

[0005] The embodiments of the present application provide a data processing and analysis method, apparatus and equipment, and a computer program product to improve the efficiency of data processing and analysis and to improve the accuracy of fault tracing and location.

[0006] The embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a data processing and analysis method, the data processing and analysis method comprising:

[0008] Determine a cloud native observable object, and collect and process observable data for the cloud native observable object to obtain initial observable data;

[0009] Extracting feature information based on object categories from the initial observable data to obtain observable feature basic data;

[0010] Obtain real-time indicator observation data of the cloud native observable object, and perform traceability and positioning analysis based on the real-time indicator observation data and the observable feature basic data to obtain traceability and positioning result data.

[0011] Optionally, determining a cloud native observable object and collecting and processing observable data for the cloud native observable object to obtain initial observable data includes:

[0012] According to the determined cloud native observable object, the indicator-type observable data is collected and processed to obtain the indicator-type observable raw data;

[0013] According to the determined cloud native observable object, log observable data is collected and processed to obtain log observable raw data;

[0014] According to the determined cloud native observable object, link observable data is collected and processed to obtain link observable raw data;

[0015] Performing log content analysis and data summarization on the log-type observable data to obtain log observable summarized content data;

[0016] According to the log observable summarized content data, correlation analysis is performed on the indicator-type observable original data and the link-type observable original data to obtain observable correlation data;

[0017] The observable summarized content data and observable associated data of the log are collected to obtain the initial observable data.

[0018] Optionally, the analyzing and summarizing the log content of the log-type observable data to obtain the log observable summarized content data includes:

[0019] Performing type-based log data clustering on the log observable data to obtain typed log observable data;

[0020] Performing classification statistics on different data items in the same type of log observable data in a time series, and calibrating the source and destination of the data items to obtain type log content data;

[0021] Gather the type log content data of the type log observable data of the same type to obtain type log observable content data;

[0022] Different types of log observable content data are collected to obtain the log observable summarized content data.

[0023] Optionally, performing correlation analysis on the indicator-type observable raw data and the link-type observable raw data based on the observable summarized content data of the log to obtain observable correlation data includes:

[0024] Extracting each complete observable link based on the link observable raw data;

[0025] According to the observable raw data of the indicator class, the indicator observable information is associated with the corresponding observable link to obtain the indicator observable link;

[0026] According to the log observable summarized content data, data items in different types of log observable content data are associated with corresponding indicator observable links to obtain indicator log associated link data;

[0027] Different indicator log association link data are collected to obtain the observable association data.

[0028] Optionally, the initial observable data includes observable associated data, and the observable associated data includes different indicator log associated link data. The extracting feature information based on object categories from the initial observable data to obtain observable feature basic data includes:

[0029] According to the observable correlation data, different indicator log correlation link data are subjected to observation feature extraction based on big data for different indicators to obtain link correlation indicator feature data;

[0030] According to the observable correlation data, different indicator log correlation link data are subjected to observation feature extraction based on different logs of big data to obtain link correlation log feature data;

[0031] For different indicator log-associated link data, the corresponding link-associated indicator feature data and the link-associated log feature data are collected to obtain the observable feature basic data.

[0032] Optionally, the step of extracting observation features of different indicators based on big data from different indicator log-associated link data according to the observable associated data to obtain link-associated indicator feature data includes:

[0033] For different indicators in the indicator log associated link data, obtaining historical big data corresponding to the indicator log associated link data;

[0034] The following observation features are extracted based on the historical big data of the indicator log-associated link data and the type of indicator to obtain the link-associated indicator feature data:

[0035] If the indicator type is a numerical range type, the value range is defined based on historical big data to obtain the observation characteristics;

[0036] If the indicator type is a numerical periodic type, the periodic function is defined based on historical big data to obtain the observation characteristics;

[0037] If the indicator type is a fixed value type, the fixed value determined based on historical big data is calibrated as the observation feature;

[0038] If the indicator type is random value type, the random function is defined based on historical big data to obtain the observation characteristics;

[0039] The observation features of all indicators corresponding to the indicator log-associated link data are collected to obtain the link-associated indicator feature data.

[0040] Optionally, the step of extracting observation features of different logs based on big data from different indicator log-associated link data according to the observable associated data to obtain link-associated log feature data includes:

[0041] For different link sequence positions of logs generated in the indicator log-associated link data, extract all types of log observable contents corresponding to the big data to obtain link position log observation feature data;

[0042] The link position log observation feature data corresponding to the link sequence positions of all generated logs in the indicator log association link are collected to obtain the corresponding link association log feature data.

[0043] Optionally, performing source tracing and positioning analysis based on the real-time indicator observation data and the observable feature basic data to obtain source tracing and positioning result data includes:

[0044] According to the real-time indicator observation data, different links involved in the real-time indicator are determined and marked as real-time observation links;

[0045] Extracting observation features of corresponding indicators from link-related indicator feature data matched by different real-time observation links;

[0046] If the real-time indicator observation data exists and belongs to the observation characteristics of the corresponding indicator in the matched link-related indicator characteristic data, then the indicator normal information is obtained;

[0047] If there is no real-time indicator observation data belonging to the observation characteristics of the corresponding indicator in any of the matched link-related indicator feature data, the link-related log feature data matched with each real-time observation link is extracted for traceability analysis to obtain traceability result data.

[0048] Optionally, extracting link-associated log feature data matching each of the real-time observation links to perform source tracing and positioning analysis to obtain source tracing and positioning result data includes:

[0049] If the real-time indicator on the real-time observation link is the initial input indicator of the link, an initial input error positioning result is obtained;

[0050] If the real-time indicator on the real-time observation link is not the initial input indicator of the link, obtain the previous real-time direct association log corresponding to the real-time indicator on the real-time observation link, and obtain the link position log observation feature data corresponding to the real-time direct association log in the link association indicator feature data matched with the real-time observation link, perform traceability positioning based on sameness judgment, and obtain the sameness traceability positioning result.

[0051] Optionally, the obtaining of the link location log observation feature data corresponding to the real-time direct association log in the link association indicator feature data matched with the real-time observation link, performing source tracing and positioning based on sameness judgment, and obtaining a sameness source tracing and positioning result includes:

[0052] Performing classification statistics on different data items in the log content of the real-time directly associated log in a time series, and calibrating the direction of the source and destination of the data items to obtain real-time direct log content data;

[0053] If a data item in the real-time direct log content data is different from a corresponding data item in any of the type log observable content, and the real-time source log content corresponding to the different data items on the real-time observation link is the same as the type log observable content at the corresponding position in the matching link location log observation feature data, then a direct log location processing error location result is obtained;

[0054] If there is a data item in the real-time direct log content data that is different from the corresponding data item in any of the type log observable content, and the real-time source log content corresponding to the different data item on the real-time observation link is also different from the type log observable content at the corresponding position in the matching link position log observation feature data, then continue to trace the source positioning according to the direct source log of the data item that deviates from the real-time direct log content determined on the real-time observation link until it is determined that the data item of the previous real-time log content on the real-time observation link is the same as the data item of the type log observable content at the corresponding position in the link position log observation feature data, then the real-time log content that is the first data item in the sequence along the real-time observation link that is different from the data item of the type log observable content at the corresponding position in the link position log observation feature data is determined as the traceability object log, and the data item with problems in the traceability object log is marked as a log error processing item to obtain the traceability log position processing error positioning result.

[0055] In a second aspect, an embodiment of the present application further provides a data processing and analysis device, the data processing and analysis device comprising:

[0056] a collection and processing unit, configured to determine a cloud native observable object and collect and process observable data of the cloud native observable object to obtain initial observable data;

[0057] A feature extraction unit, configured to extract feature information based on object categories from the initial observable data to obtain observable feature basic data;

[0058] The traceability and positioning unit is used to obtain real-time indicator observation data of the cloud native observable object, and perform traceability and positioning analysis based on the real-time indicator observation data and the observable feature basic data to obtain traceability and positioning result data.

[0059] In a third aspect, an embodiment of the present application further provides a device, including:

[0060] A processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform any of the aforementioned data processing and analysis methods.

[0061] In a fourth aspect, an embodiment of the present application further provides a computer program product, comprising a computer program / instruction, which implements any of the aforementioned data processing and analysis methods when executed by a processor.

[0062] At least one of the above-mentioned technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the data processing and analysis method of the present application obtains different types of raw data by determining specific observable objects and collecting observable data in a targeted manner. This not only lays the foundation for subsequent data analysis and processing, but also provides the possibility for correlation analysis between different types of data. By extracting feature information from the collected initial data, the fault problem can be accurately located, and traceability and location analysis can be performed in combination with real-time data. This method not only improves the efficiency of data monitoring management and processing analysis, but also can more accurately determine the root cause of the fault problem, provide accurate guidance information for solving the problem, ensure the efficiency of data operation, and achieve data observability. The present application improves the accuracy of fault diagnosis and the efficiency of data processing through precise data collection and analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0064] Figure 1 A flow chart of a data processing and analysis method according to an embodiment of the present application;

[0065] Figure 2 This is a structural diagram of a data processing and analysis device in an embodiment of the present application;

[0066] Figure 3 This is a structural diagram of a device in an embodiment of the present application. DETAILED DESCRIPTION

[0067] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0068] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0069] The present application embodiment provides a data processing and analysis method, such as Figure 1 , a flow chart of a data processing and analysis method according to an embodiment of the present application is provided, wherein the data processing and analysis method comprises at least the following steps S110 to S130:

[0070] Step S110: determining a cloud-native observable object, and collecting and processing observable data for the cloud-native observable object to obtain initial observable data.

[0071] The data processing and analysis method of the embodiment of the present application is mainly based on cloud native observability. When performing data processing and analysis, it is necessary to first determine the specific cloud native observable objects in combination with the actual application scenario, and then collect and process the observable data of these cloud native observable objects under normal operation to form initial observable data.

[0072] These cloud-native observable objects can include objects within the system's operational processes, including operational data processing, system operation, and information flow. Examples include microservices developed for distributed software on cloud platforms, components of the container platform itself, such as CoreDNS, container resource objects such as Services, and different processing terminals completing specific parts of development. Observable objects include operational logs and operational metrics for each process terminal, as well as data flow information for the entire development system.

[0073] By identifying specific observable objects to collect observable data in a targeted manner, and then obtaining different types of observable raw data, on the one hand, it establishes a data foundation for subsequent reasonable analysis and processing of observable data, and on the other hand, it also provides the possibility for subsequent analysis and processing of the correlation between different types of data.

[0074] The observable data of this application mainly includes the following types of data: logs, indicators, links, and events; the processing of data will form log records, and log data is obtained in the form of feature information extraction, mainly obtaining non-formatted feature data in the log records to form observable data; indicator data mainly includes performance indicators, operating environment indicators and other indicator data that need to be controlled during the data processing process, which is used to reflect the data processing operation status under the corresponding operating scenario; link data mainly obtains the direction information of the data flow to clarify the correspondence between log data and indicator data, and has the function of associating log data and indicator data to achieve comprehensive data observability. Events are mainly event objects in K8s, which obtain metadata and event details, etc., for system monitoring and troubleshooting.

[0075] Step S120 , extracting feature information based on object categories from the initial observable data to obtain observable feature basic data.

[0076] After big data collection of initial observable data such as logs, indicators and link data collected under normal operation, feature extraction is performed to obtain big data features, which serve as the judgment standard for subsequent accurate fault location.

[0077] For example, metrics from the network protocol layer and transport layer are obtained, and correlation analysis is performed using network quintuples and container cloud metadata. By acquiring kernel metrics, every layer from the kernel to the user program is covered. This allows the full stack path of a request to be traced from the application, through system calls, network transmission, gateway services, security services, and finally to the database service or peer microservice, providing sufficient neutral observation data for rapid fault demarcation. Furthermore, after extracting unformatted data from the captured log records, big data analysis is performed to determine which reasonable record values ​​or record languages ​​may exist at the corresponding locations in the extracted unformatted data. This serves as the basis for subsequent comparative analysis to determine whether new record values ​​or record languages ​​appear at the corresponding locations in the real-time log data. For indicator data, the possible range or variation pattern of the indicator values ​​observed at the corresponding link locations is determined based on big data, which serves as the basis for subsequent judgment of whether the real-time indicator data is abnormal. For link data, the possible links formed by the data flow direction based on big data are obtained, which serve as the basis for subsequent judgment of whether the data flow is abnormal.

[0078] Step S130: Acquire real-time indicator observation data of the cloud native observable object, and perform traceability and positioning analysis based on the real-time indicator observation data and the observable feature basic data to obtain traceability and positioning result data.

[0079] Combined with the data information of these observable objects collected in real time and the above-mentioned observable feature basic data, reasonable and real-time traceability and positioning analysis is carried out, and the specific in-depth analysis and positioning of the fault problem is completed on the basis of fully realizing the analysis and processing of the correlation between different data. On the one hand, it more efficiently realizes the monitoring and management of data and in-depth data processing and analysis, further improving the efficiency of data processing and analysis. On the other hand, it can also more accurately and specifically determine the root cause of the fault problem, providing more accurate guidance reference information for the solution of the fault problem, effectively ensuring the efficiency of the normal operation of the data, and fully realizing the observability of the data.

[0080] In some embodiments of the present application, the determining of cloud-native observable objects and collecting and processing observable data of the cloud-native observable objects to obtain initial observable data includes: collecting and processing indicator-type observable data according to the determined cloud-native observable objects to obtain indicator-type observable original data; collecting and processing log-type observable data according to the determined cloud-native observable objects to obtain log-type observable original data; collecting and processing link-type observable data according to the determined cloud-native observable objects to obtain link-type observable original data; analyzing and summarizing log content of the log-type observable data to obtain log observable summarized content data; performing correlation analysis on the indicator-type observable original data and the link observable original data according to the log observable summarized content data to obtain observable correlated data; and combining the log observable summarized content data and observable correlated data to obtain the initial observable data.

[0081] Data collection for cloud-native observable objects primarily provides a data foundation for subsequent observability data analysis and processing. Data collection here considers two aspects. First, it involves the collection of different data types. As you can understand, cloud-native observability primarily relies on in-depth processing and analysis of different types of data, so the necessary different types of data must be collected before analysis and processing. The selection of cloud-native observable objects can be determined based on actual circumstances. The collected data types include metrics, logs, and data links. Metrics are the basic data type for data monitoring and analysis. Logs are essential for in-depth fault analysis and location within observability. Data links accurately guide the direction of data flow and provide the necessary correlation data for combining different types of data. Therefore, data collection primarily focuses on these three types of data.

[0082] Another aspect is that after data collection is complete, it's necessary to establish relationships between different data types to provide a foundation for subsequent data analysis based on the relationships between the different data types. Here, the relationship is established based on data links, introducing indicator data and log data into the links to connect all different types of data.

[0083] In some embodiments of the present application, the log content analysis and data summary of the log type observable data to obtain log observable summary content data includes: performing type-based log data clustering on the log observable data to obtain type log observable data; performing classification statistics on different data items in the same type of type log observable data under time series, and calibrating the source and destination of the data items to obtain type log content data; aggregating the type log content data of the same type of type log observable data to obtain type log observable content data; aggregating different types of log observable content data to obtain the log observable summary content data.

[0084] Data collection and processing is not necessarily a simple data collection, especially for log data. As important data information for cloud native objectivity, simple data collection will bring two problems. The first is that the amount of log data is huge. Simple data collection will cause a rapid increase in the space required for log data storage, which will bring a huge burden on the storage space for data analysis and processing. On the other hand, directly collecting log data cannot improve the efficiency of subsequent in-depth analysis of fault problems based on log data. After all, the content and form of log data determine that direct processing and analysis of log content requires screening and processing among a large amount of non-parametric data, which in turn reduces the efficiency of log data processing and analysis.

[0085] Based on this, when collecting log-type data, the embodiment of the present application first reasonably structures the log-type data, and realizes the reasonable processing and analysis of the Japanese-type data by structuring the collected log-type data based on the data items in the time dimension. Of course, structuring is not a single statistical processing of data items. Since the collected data can form a big data foundation, it is necessary to reasonably cluster the log-type data of the same type when structuring. Here, the same type can be defined according to the actual situation. It can be just a cluster of data items of the same type, or a cluster of data items of the same type and the same order of magnitude corresponding to some necessary data items, etc. The clustering process only needs to provide a reasonable data division reference for the subsequent correlation analysis. Through structured clustering, the space occupied by log-type data can be greatly reduced, and while saving resources, it can also achieve efficient and reasonable processing of data.

[0086] The type of indicator is the form in which the indicator is expressed. Some indicators are range indicators, such as transport layer TCP indicator collection - TCP indicator trend analysis, transport layer TCP indicator collection - TCP abnormal message identification, the number of IP datagrams provided for transmission by the local high-level protocol: spacketsIP, the number of received IP datagrams forwarded to other interfaces: fpacketsIP, the number of IP datagrams passed to the local high-level protocol: dpacketslIP, the number of received ICMP datagrams: icmpi, the number of sent ICMP datagrams: icmpo, the number of received TCP segments (including receive errors), the number of sent TCP segments, excluding those containing only retransmitted bytes, etc.

[0087] Log types refer to logs with different recorded content or data recording formats, including device logs, cloud platform logs, container component logs, operating system logs, middleware logs, application logs, etc.

[0088] In some embodiments of the present application, the log observable summary content data is used to perform correlation analysis on the indicator-type observable original data and the link observable original data to obtain observable correlation data, including: extracting each complete observable link based on the link observable original data; associating the indicator observable information with the corresponding observable link based on the indicator-type observable original data to obtain the indicator observable link; associating data items in different types of log observable content data with the corresponding indicator observable link based on the log observable summary content data to obtain the indicator log correlation link data; and aggregating different indicator log correlation link data to obtain the observable correlation data.

[0089] The purpose of associating different types of data is to establish the relationship between them. This provides fault analysis and judgment on associating different types of data for subsequent cloud-native observable data analysis and processing, thereby achieving simultaneous analysis and judgment of different types of data under the same fault problem. This is more efficient and reasonable than conducting independent analysis of the same fault problem on different data.

[0090] The embodiment of the present application establishes the correlation based on the observable link. On the observable link, the indicators and logs have a definite sequential position, which reflects their logical order on the observable link. When locating and analyzing the fault problem, this logical order can be used to simultaneously analyze and process different data including indicators and logs. In addition, the indicators and logs associated with different observable links are different. Under this association, the specific application of different indicators and log data can be accurately grasped, making the data operation process clearer and making the observability of the data more vivid and specific.

[0091] The observable link essentially refers to the direction of information flow in data processing, that is, the direction in which data is processed sequentially. This allows us to track a request from the application, transmit information using ThreadLocal, and thus correlate the incoming and outgoing calls of a service. This full-stack path, through system calls, network transmission, gateway services, and security services, reaches the database service or peer microservice, providing sufficient neutral observation data for rapid fault localization. By linking with existing metadata systems such as CMDBs, business semantics at both call and service granularity are injected. Specific tracking, full links, complete topology relationships, trace queries, and key insights are implemented using eBPF collection technology, with continued in-depth development and expansion.

[0092] In some embodiments of the present application, the initial observable data includes observable association data, and the observable association data includes different indicator log association link data. The feature information extraction based on object category of the initial observable data to obtain observable feature basic data includes: according to the observable association data, the observation feature extraction of different indicators based on big data for different indicator log association link data to obtain link association indicator feature data; according to the observable association data, the observation feature extraction of different logs based on big data for different indicator log association link data to obtain link association log feature data; for different indicator log association link data, the corresponding link association indicator feature data and the link association log feature data are collected to obtain the observable feature basic data.

[0093] After completing the extraction and correlation of different types of data, in order to facilitate the subsequent reasonable fault problem analysis and location, it is necessary to extract reasonable feature information based on big data for the observable data under normal operation, so that the analysis and judgment in the fault location analysis can be faster and more efficient. Of course, the feature information extraction for different types of data is also different, and reasonable extraction processing is required respectively. The embodiment of the present application extracts observation features of different indicators and different logs based on big data for different indicator log-associated link data, thereby obtaining indicator feature data and log feature data related to the link as observable feature basic data.

[0094] In some embodiments of the present application, according to the observable associated data, observation feature extraction of different indicators based on big data is performed on different indicator log-associated link data to obtain link-associated indicator feature data, including: obtaining historical big data corresponding to different indicators in the indicator log-associated link data; extracting the following observation features according to the historical big data of the indicator log-associated link data and the type of indicator to obtain the link-associated indicator feature data: if the type of the indicator is a numerical range type, the value range is defined according to the historical big data to obtain the observation feature; if the type of the indicator is a numerical periodic type, the periodic function is defined according to the historical big data to obtain the observation feature; if the type of the indicator is a fixed value type, the determined fixed value is calibrated as the observation feature according to the historical big data; if the type of the indicator is a random value type, the random function is defined according to the historical big data to obtain the observation feature; and the observation features of all indicators corresponding to the indicator log-associated link data are collected to obtain the link-associated indicator feature data.

[0095] Extracting characteristic information from indicator data primarily relies on understanding the type and form of the indicator data within the associated data. Indicator data can exhibit ranged, periodic, fixed, or random values. By determining this form and applying big data analysis and processing, corresponding observational features can be established efficiently and quickly.

[0096] In some embodiments of the present application, according to the observable association data, observation features of different logs based on big data are extracted for different indicator log association link data to obtain link association log feature data, including: for different link sequence positions of generated logs in the indicator log association link data, all types of corresponding observable content of logs under big data are extracted to obtain link position log observation feature data; and the link position log observation feature data corresponding to the link sequence positions of all generated logs in the indicator log association link are collected to obtain the corresponding link association log feature data.

[0097] The extraction of feature information from log data is also based on correlation. Therefore, for log data, this can only be achieved based on data application. Therefore, by considering the specific content of the log content on the observable link, feature information can be extracted in a more targeted manner. It is understandable that logs at the same location on the same link may not necessarily contain a specific type of log content; different types of log content may be formed at this location based on actual conditions. Therefore, it is sufficient and reasonable to use different types of log content as feature information for the corresponding log location.

[0098] A data link has multiple data processing points, and each point likely generates multiple types of log records. For example, logs may record processing time or processing volume. Extracting data from a single type of log as the log observable content corresponding to that processing point would be inaccurate and incomplete. Therefore, we extract data from all types of logs at each processing point based on their type to generate the log observable content for that processing point. This is known as "big data representing all types of log observable content." It should be noted that the big data for each type of log data is generated in the time dimension.

[0099] In some embodiments of the present application, the traceability and positioning analysis is performed based on the real-time indicator observation data and the observable feature basic data to obtain traceability and positioning result data, including: determining different links involved in the real-time indicator based on the real-time indicator observation data, and marking them as real-time observation links; extracting the observation characteristics of the corresponding indicators in the link-associated indicator feature data matched by different real-time observation links; if there is an observation characteristic of the corresponding indicator in the link-associated indicator feature data that the real-time indicator observation data belongs to, then the normal indicator information is obtained; if there is no observation characteristic of the corresponding indicator in any of the matched link-associated indicator feature data that the real-time indicator observation data belongs to, then extracting the link-associated log feature data matched with each of the real-time observation links for traceability and positioning analysis to obtain traceability and positioning result data.

[0100] Cloud-native observability primarily relies on analyzing and processing diverse data to achieve more accurate and in-depth fault location. Therefore, after extracting observable feature data, efficient fault location can be performed based on this data. Of course, for fault analysis and troubleshooting, the most direct observation of upper-layer data is indicator data, so faults often begin with observed anomalies in this indicator data. Of course, indicator data should also be analyzed on the link. If the indicator data matches the feature information of the matching link, the system can be considered normal. If not, a fault has occurred, and further troubleshooting and locating the source of the fault is necessary.

[0101] In some embodiments of the present application, the link-associated log feature data matched with each of the real-time observation links is extracted for source tracing and positioning analysis to obtain source tracing and positioning result data, including: if the real-time indicator on the real-time observation link is the initial input indicator of the link, then the initial input error positioning result is obtained; if the real-time indicator on the real-time observation link is not the initial input indicator of the link, then the previous real-time direct association log corresponding to the real-time indicator on the real-time observation link is obtained, and the link position log observation feature data corresponding to the real-time direct association log in the link-associated indicator feature data matched with the real-time observation link is obtained, and source tracing and positioning based on sameness judgment is performed to obtain sameness source tracing and positioning results.

[0102] Tracing the source of a fault primarily considers two scenarios for indicator data. The first scenario involves the deviating indicator data being the starting data directly input on the link. In this case, it can be directly determined that the input indicator data has a problem. The other scenario involves the deviating indicator data being in the middle or end of the link. In this case, it's not straightforward to determine that the anomaly occurred in this link, as it's possible that the fault occurred before without any indicator manifestation. Therefore, further analysis is needed based on the preceding real-time, directly correlated logs corresponding to the real-time indicators on the real-time observation link to conduct reasonable tracing and location analysis.

[0103] For example, when an exception is detected at a corresponding node on a link, the traceID in the application call chain is correlated with the traceID in the log content, automatically jumping to the log content of that node and correlating it with upstream and downstream logs, as well as the operating system log of that node. If this log is not generated by the most recent processing point on the link, the exception may be caused by a record deviation at the previous data processing point on the link. Therefore, anomaly analysis is required on the log that precedes the abnormal log in the link sequence. This most recent log is the real-time directly correlated log.

[0104] In some embodiments of the present application, the link location log observation feature data corresponding to the real-time direct association log in the link association indicator feature data matched with the real-time observation link is obtained, and the source tracing positioning based on the sameness judgment is performed to obtain the sameness tracing positioning result, including: classifying and statistically analyzing the different data items in the log content of the real-time direct association log under the time series, and calibrating the source and destination directions of the data items to obtain real-time direct log content data; if the data items in the real-time direct log content data are different from the corresponding data items in any of the observable contents of the type log, and the real-time source log content corresponding to the different data items on the real-time observation link is the same as the observable content of the type log at the corresponding position in the matched link location log observation feature data, then a direct log location processing error positioning result is obtained; if the data items in the real-time direct log content data are different from the corresponding data items in any of the observable contents of the type log, If the corresponding data items in the observable content of the log are different, and the real-time source log content corresponding to the different data items on the real-time observation link is also different from the type log observable content at the corresponding position in the matching link position log observation feature data, then continue to trace the source positioning according to the direct source log of the data item that deviates from the real-time direct log content determined on the real-time observation link until it is determined that the data item of the previous real-time log content on the real-time observation link is the same as the data item of the type log observable content at the corresponding position in the link position log observation feature data, then the real-time log content that is the first data item in the sequence along the real-time observation link that is different from the data item of the type log observable content at the corresponding position in the link position log observation feature data is determined as the traceability object log, and the data item with problems in the traceability object log is marked as a log error processing item to obtain the traceability log position processing error positioning result.

[0105] When tracing the source and locating the analysis, logs are important and in-depth data analysis objects. Therefore, we can first extract the previous logs that are directly related to the deviated indicators, and analyze and judge the log content at the corresponding position based on the characteristic information. If there is a deviation in the data item in the direct log content, and there is no deviation in the previous log from which the deviated data item originated, it can be determined that an error occurred in the processing of generating the direct log content. If the log determined before the direct log also has a problem with the data item, it is necessary to continue to extract the directly related logs forward for judgment until it is determined that there is no deviation in the data item in the previous log content. It is then determined that the data processing process corresponding to the log directly related to this log is the root cause of the problem. Through such tracing, different data are combined, and the most essential problem situation can be located more accurately and efficiently, thereby improving the efficiency and accuracy of data processing and analysis.

[0106] The present application also provides a data processing and analysis device 200, such as Figure 2 , a schematic diagram of the structure of a data processing and analysis device in an embodiment of the present application is provided. The data processing and analysis device 200 includes: an acquisition and processing unit 210, a feature extraction unit 220, and a traceability and positioning unit 230, wherein:

[0107] The collection and processing unit 210 is used to determine a cloud native observable object and collect and process observable data of the cloud native observable object to obtain initial observable data;

[0108] A feature extraction unit 220 is configured to extract feature information based on object categories from the initial observable data to obtain observable feature basic data;

[0109] The traceability and positioning unit 230 is used to obtain real-time indicator observation data of the cloud native observable object, and perform traceability and positioning analysis based on the real-time indicator observation data and the observable feature basic data to obtain traceability and positioning result data.

[0110] In some embodiments of the present application, the collection and processing unit 210 is specifically used to: collect and process indicator-type observable data according to the determined cloud-native observable object to obtain indicator-type observable original data; collect and process log-type observable data according to the determined cloud-native observable object to obtain log-type observable original data; collect and process link-type observable data according to the determined cloud-native observable object to obtain link-type observable original data; perform log content analysis and data summary on the log-type observable data to obtain log observable summarized content data; perform correlation analysis on the indicator-type observable original data and the link observable original data according to the log observable summarized content data to obtain observable correlated data; and collect the log observable summarized content data and observable correlated data to obtain the initial observable data.

[0111] In some embodiments of the present application, the acquisition and processing unit 210 is specifically used to: perform type-based log data clustering on the log observable data to obtain type log observable data; perform classification statistics on different data items in the type log observable data of the same type under time series, and calibrate the source and destination directions of the data items to obtain type log content data; aggregate the type log content data of the type log observable data of the same type to obtain type log observable content data; aggregate different types of log observable content data to obtain the log observable summary content data.

[0112] In some embodiments of the present application, the acquisition and processing unit 210 is specifically used to: extract each complete observable link based on the link observable original data; associate the indicator observable information with the corresponding observable link based on the indicator class observable original data to obtain the indicator observable link; associate the data items in different types of log observable content data with the corresponding indicator observable link based on the log observable summary content data to obtain the indicator log associated link data; and collect different indicator log associated link data to obtain the observable associated data.

[0113] In some embodiments of the present application, the initial observable data includes observable association data, and the observable association data includes different indicator log association link data. The feature extraction unit 220 is specifically used to: according to the observable association data, perform observation feature extraction of different indicators based on big data on different indicator log association link data to obtain link association indicator feature data; according to the observable association data, perform observation feature extraction of different logs based on big data on different indicator log association link data to obtain link association log feature data; for different indicator log association link data, collect the corresponding link association indicator feature data and the link association log feature data to obtain the observable feature basic data.

[0114] In some embodiments of the present application, the feature extraction unit 220 is specifically used to: obtain historical big data corresponding to different indicators in the indicator log-associated link data; extract the following observation features according to the historical big data of the indicator log-associated link data and the type of indicator to obtain the link-associated indicator feature data: if the type of the indicator is a numerical range type, the value range is defined according to the historical big data to obtain the observation feature; if the type of the indicator is a numerical periodic type, the periodic function is defined according to the historical big data to obtain the observation feature; if the type of the indicator is a fixed value type, the determined fixed value is calibrated as the observation feature according to the historical big data; if the type of the indicator is a random value type, the random function is defined according to the historical big data to obtain the observation feature; the observation features of all indicators corresponding to the indicator log-associated link data are collected to obtain the link-associated indicator feature data.

[0115] In some embodiments of the present application, the feature extraction unit 220 is specifically used to: extract all types of observable log content corresponding to the big data for different link sequence positions of the generated logs in the indicator log associated link data, and obtain link position log observation feature data; and collect the link position log observation feature data corresponding to the link sequence positions of all generated logs in the indicator log associated link to obtain the corresponding link associated log feature data.

[0116] In some embodiments of the present application, the traceability and positioning unit 230 is specifically used to: determine the different links involved in the real-time indicator based on the real-time indicator observation data, and mark them as real-time observation links; extract the observation characteristics of the corresponding indicators in the link-related indicator feature data matched by different real-time observation links; if there is an observation characteristic of the indicator corresponding to the real-time indicator observation data belonging to the matched link-related indicator feature data, then obtain the normal indicator information; if there is no observation characteristic of the indicator corresponding to any of the matched link-related indicator feature data belonging to the real-time indicator observation data, then extract the link-related log feature data matched with each real-time observation link for traceability and positioning analysis to obtain traceability and positioning result data.

[0117] In some embodiments of the present application, the traceability and positioning unit 230 is specifically used to: if the real-time indicator on the real-time observation link is the initial input indicator of the link, then obtain the initial input error positioning result; if the real-time indicator on the real-time observation link is not the initial input indicator of the link, then obtain the previous real-time direct association log corresponding to the real-time indicator on the real-time observation link, and obtain the link position log observation feature data corresponding to the real-time direct association log in the link association indicator feature data matched with the real-time observation link, perform traceability and positioning based on sameness judgment, and obtain the same traceability and positioning result.

[0118] In some embodiments of the present application, the source tracing and positioning unit 230 is specifically used to: perform classification statistics on different data items in the log content of the real-time directly associated log in a time series, and calibrate the source and destination directions of the data items to obtain real-time direct log content data; if the data items in the real-time direct log content data are different from the corresponding data items in any of the observable contents of the type log, and the real-time source log content corresponding to the different data items on the real-time observation link is the same as the type log observable content at the corresponding position in the matching link position log observation feature data, then a direct log position processing error positioning result is obtained; if the data items in the real-time direct log content data are different from the corresponding data items in any of the observable contents of the type log, and the different data items correspond to the real-time observation link. If the real-time source log content is also different from the type log observable content at the corresponding position in the matching link location log observation feature data, then continue to trace the source positioning based on the direct source log of the data item that deviates from the real-time direct log content determined on the real-time observation link until it is determined that the data item of the previous real-time log content on the real-time observation link is the same as the data item of the type log observable content at the corresponding position in the link location log observation feature data, then the real-time log content that is the first data item in the sequence along the real-time observation link that is different from the data item of the type log observable content at the corresponding position in the link location log observation feature data is determined as the traceability object log, and the data item with problems in the traceability object log is marked as a log error processing item to obtain the traceability log position processing error positioning result.

[0119] It can be understood that the above-mentioned data processing and analysis device can implement each step of the data processing and analysis method provided in the aforementioned embodiment. The relevant explanations about the data processing and analysis method are applicable to the data processing and analysis device and will not be repeated here.

[0120] Figure 3 This is a schematic diagram of the structure of a device in the embodiment of the present application. Figure 3 As shown, the device includes one or more processors (or processing units), may further include one or more memories coupled to the processors, and may further include a communication module coupled to the processors.

[0121] The communication module can be used to communicate with other devices or apparatuses, such as sending or receiving data and / or signals. The communication module can include at least one communication module for communication. The communication module can include any interface necessary for communicating with other devices. Exemplarily, the communication module can be a transceiver, circuit, bus, module, or other type of communication module.

[0122] The processor may include, but is not limited to, at least one of the following: a general-purpose computer, a special-purpose computer, a microcontroller, a digital signal processor (DSP), or one or more of a controller-based multi-core controller architecture. A device may have multiple processors, such as application-specific integrated circuit chips, which are time-slave to a clock synchronized with a main processor.

[0123] The memory may include one or more non-volatile memories and one or more volatile memories. Examples of non-volatile memories include, but are not limited to, at least one of the following: read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, hard disk, compact disc (CD), digital video disc (DVD), or other magnetic storage and / or optical storage. Examples of volatile memories include, but are not limited to, at least one of the following: random access memory (RAM), or other volatile memories that do not persist during a power outage.

[0124] A computer program includes computer-executable instructions that are executed by an associated processor. The program may be stored in ROM. The processor may perform any suitable actions and processes by loading the program into RAM.

[0125] The possible implementation of the present application can be realized by means of a program, so that the communication device can perform any process discussed in the above embodiments. The possible implementation of the present application can also be realized by hardware or by a combination of software and hardware.

[0126] In some embodiments, the program may be tangibly contained in a computer-readable storage medium that may be included in the device (such as in a memory) or other storage device accessible by the device. The program may be loaded from the computer-readable storage medium into RAM for execution. The computer-readable storage medium may include any type of tangible non-volatile memory, such as ROM, EPROM, flash memory, hard disk, CD, DVD, etc.

[0127] The present application also provides a computer-readable storage medium having computer instructions or program codes stored thereon, which, when executed by a processor, causes the processor to perform the methods and functions described in any of the above embodiments. A computer-readable medium may be any tangible medium containing or storing a program for or related to an instruction execution system, apparatus, or device. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. Computer-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination thereof. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrated therein. More detailed examples of computer-readable storage media include electrical connections with one or more wires, magnetic media (e.g., magnetic disks, floppy disks, hard disks, tapes, magnetic storage devices), optical media (e.g., optical storage devices, DVDs), semiconductor media (e.g., solid-state drives), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), or any suitable combination thereof.

[0128] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The embodiments of the present application also provide at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes one or more computer-executable instructions, such as instructions included in a program module, which are executed in a device on a real or virtual processor of the target to perform the processes, methods and functions involved in any of the above embodiments. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method.

[0129] The present application also provides a computer program product, including a computer program or instructions, which, when run on a computer, causes the computer to perform the processes, methods, and functions in the above-described embodiments. Typically, a program module includes routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functions of the program modules can be combined or divided between program modules as needed. The machine executable instructions for the program modules can be executed in local or distributed devices. In distributed devices, the program modules can be located in local and remote storage media.

[0130] In general, various embodiments of the present application can be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects can be implemented in hardware, while other aspects can be implemented in firmware or software, which can be executed by a controller, microprocessor, or other computing device. Although various aspects of the embodiments of the present disclosure are shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that the blocks, devices, systems, techniques, or methods described herein can be implemented as, by way of non-limiting example, hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0131] It should be noted that although the embodiments of the present application are described above in conjunction with the accompanying drawings, the above embodiments are not independent of each other, and they can also be combined to obtain other embodiments. The methods, situations, categories, and divisions of the embodiments in the embodiments of the present application are only for the convenience of description and should not constitute special limitations. The features of the various methods, categories, situations, and embodiments can be combined with each other when they are logical. The various embodiments of the present application can be combined arbitrarily to achieve different technical effects. The embodiments of the present application no longer list various combinations.

[0132] In addition, although the operations of the method of the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that these operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart can change the order of execution. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps. It should also be noted that the features and functions of two or more devices according to the present disclosure can be embodied in one device. Conversely, the features and functions of a device described above can be further divided into being embodied by multiple devices.

[0133] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0134] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A data processing and analysis method, characterized in that: The data processing and analysis method comprises: Determine a cloud native observable object, and collect and process observable data for the cloud native observable object to obtain initial observable data; Extracting feature information based on object categories from the initial observable data to obtain observable feature basic data; Obtain real-time indicator observation data of the cloud native observable object, and perform traceability and positioning analysis based on the real-time indicator observation data and the observable feature basic data to obtain traceability and positioning result data.

2. The data processing and analysis method according to claim 1, characterized in that: The determining of cloud native observable objects and collecting and processing observable data of the cloud native observable objects to obtain initial observable data includes: According to the determined cloud native observable object, the indicator-type observable data is collected and processed to obtain the indicator-type observable raw data; According to the determined cloud native observable object, log observable data is collected and processed to obtain log observable raw data; According to the determined cloud native observable object, link observable data is collected and processed to obtain link observable raw data; Performing log content analysis and data summarization on the log-type observable data to obtain log observable summarized content data; According to the log observable summarized content data, correlation analysis is performed on the indicator-type observable original data and the link-type observable original data to obtain observable correlation data; The observable summarized content data and observable associated data of the log are collected to obtain the initial observable data.

3. The data processing and analysis method according to claim 2, characterized in that: The analyzing and summarizing the log content of the log-type observable data to obtain the log observable summarized content data includes: Performing type-based log data clustering on the log observable data to obtain typed log observable data; Performing classification statistics on different data items in the same type of log observable data in a time series, and calibrating the source and destination of the data items to obtain type log content data; Gather the type log content data of the type log observable data of the same type to obtain type log observable content data; Different types of log observable content data are collected to obtain the log observable summarized content data.

4. The data processing and analysis method according to claim 3, characterized in that: The step of performing correlation analysis on the observable raw data of the indicator class and the observable raw data of the link based on the observable summarized content data of the log to obtain observable correlation data includes: Extracting each complete observable link based on the link observable raw data; According to the observable raw data of the indicator class, the indicator observable information is associated with the corresponding observable link to obtain the indicator observable link; According to the log observable summarized content data, data items in different types of log observable content data are associated with corresponding indicator observable links to obtain indicator log associated link data; Different indicator log association link data are collected to obtain the observable association data.

5. The data processing and analysis method according to claim 1, characterized in that: The initial observable data includes observable associated data, and the observable associated data includes different indicator log associated link data. The object category-based feature information extraction of the initial observable data to obtain observable feature basic data includes: According to the observable correlation data, different indicator log correlation link data are subjected to observation feature extraction based on big data for different indicators to obtain link correlation indicator feature data; According to the observable correlation data, different indicator log correlation link data are subjected to observation feature extraction based on different logs of big data to obtain link correlation log feature data; For different indicator log-associated link data, the corresponding link-associated indicator feature data and the link-associated log feature data are collected to obtain the observable feature basic data.

6. The data processing and analysis method according to claim 5, characterized in that: According to the observable correlation data, different indicator log correlation link data are subjected to observation feature extraction based on big data for different indicators to obtain link correlation indicator feature data, including: For different indicators in the indicator log associated link data, obtaining historical big data corresponding to the indicator log associated link data; The following observation features are extracted based on the historical big data of the indicator log-associated link data and the type of indicator to obtain the link-associated indicator feature data: If the indicator type is a numerical range type, the value range is defined based on historical big data to obtain the observation characteristics; If the indicator type is a numerical periodic type, the periodic function is defined based on historical big data to obtain the observation characteristics; If the indicator type is a fixed value type, the fixed value determined based on historical big data is calibrated as the observation feature; If the indicator type is random value type, the random function is defined based on historical big data to obtain the observation characteristics; The observation features of all indicators corresponding to the indicator log-associated link data are collected to obtain the link-associated indicator feature data.

7. The data processing and analysis method according to claim 1, characterized in that: The step of extracting observation features of different logs based on big data from different indicator log-associated link data according to the observable associated data to obtain link-associated log feature data includes: For different link sequence positions of logs generated in the indicator log-associated link data, extract all types of log observable contents corresponding to the big data to obtain link position log observation feature data; The link position log observation feature data corresponding to the link sequence positions of all generated logs in the indicator log association link are collected to obtain the corresponding link association log feature data.

8. The data processing and analysis method according to claim 1, characterized in that: The tracing and positioning analysis is performed based on the real-time indicator observation data and the observable feature basic data to obtain tracing and positioning result data, including: According to the real-time indicator observation data, different links involved in the real-time indicator are determined and marked as real-time observation links; Extracting observation features of corresponding indicators from link-related indicator feature data matched by different real-time observation links; If the real-time indicator observation data exists and belongs to the observation characteristics of the corresponding indicator in the matched link-related indicator characteristic data, then the indicator normal information is obtained; If there is no real-time indicator observation data belonging to the observation characteristics of the corresponding indicator in any of the matched link-related indicator feature data, the link-related log feature data matched with each real-time observation link is extracted for traceability analysis to obtain traceability result data.

9. The data processing and analysis method according to claim 8, characterized in that: The extracting link-associated log feature data matched with each of the real-time observation links to perform source tracing and positioning analysis to obtain source tracing and positioning result data includes: If the real-time indicator on the real-time observation link is the initial input indicator of the link, an initial input error positioning result is obtained; If the real-time indicator on the real-time observation link is not the initial input indicator of the link, obtain the previous real-time direct association log corresponding to the real-time indicator on the real-time observation link, and obtain the link position log observation feature data corresponding to the real-time direct association log in the link association indicator feature data matched with the real-time observation link, perform traceability positioning based on sameness judgment, and obtain the sameness traceability positioning result.

10. The data processing and analysis method according to claim 9, characterized in that: The obtaining of the link location log observation feature data corresponding to the real-time direct association log in the link association indicator feature data matched with the real-time observation link, performing source tracing and positioning based on sameness judgment, and obtaining a sameness source tracing and positioning result includes: Performing classification statistics on different data items in the log content of the real-time directly associated log in a time series, and calibrating the direction of the source and destination of the data items to obtain real-time direct log content data; If a data item in the real-time direct log content data is different from a corresponding data item in any of the type log observable content, and the real-time source log content corresponding to the different data items on the real-time observation link is the same as the type log observable content at the corresponding position in the matching link location log observation feature data, then a direct log location processing error location result is obtained; If there is a data item in the real-time direct log content data that is different from the corresponding data item in any of the type log observable content, and the real-time source log content corresponding to the different data item on the real-time observation link is also different from the type log observable content at the corresponding position in the matching link position log observation feature data, then continue to trace the source positioning according to the direct source log of the data item that deviates from the real-time direct log content determined on the real-time observation link until it is determined that the data item of the previous real-time log content on the real-time observation link is the same as the data item of the type log observable content at the corresponding position in the link position log observation feature data, then the real-time log content that is the first data item in the sequence along the real-time observation link that is different from the data item of the type log observable content at the corresponding position in the link position log observation feature data is determined as the traceability object log, and the data item with problems in the traceability object log is marked as a log error processing item to obtain the traceability log position processing error positioning result.

11. A data processing and analysis device, characterized in that: The data processing and analysis device comprises: a collection and processing unit, configured to determine a cloud native observable object and collect and process observable data of the cloud native observable object to obtain initial observable data; A feature extraction unit, configured to extract feature information based on object categories from the initial observable data to obtain observable feature basic data; The traceability and positioning unit is used to obtain real-time indicator observation data of the cloud native observable object, and perform traceability and positioning analysis based on the real-time indicator observation data and the observable feature basic data to obtain traceability and positioning result data.

12. A device comprising: processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the data processing and analysis method according to any one of claims 1 to 10.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the data processing and analysis method according to any one of claims 1 to 10 is implemented.

Images