Equipment connection platform intelligent observation method, medium and electronic device

Abstract

The invention discloses an equipment connection platform intelligent observation method, a medium and an electronic device, and relates to the technical field of computers, and the method comprises the steps: obtaining a full-link burying point log; determining a corresponding evaluation index based on a source channel of a full-link burying point log, associating an external data source by using a device identifier, an internet protocol IP address and a gateway node, expanding a multi-dimensional attribute for the corresponding evaluation index, and forming a corresponding device evaluation index with a service context; index insight agents are asynchronously invoked based on the device evaluation metrics to identify abnormal metrics that deviate from normal periodic or trending fluctuations according to the device evaluation metrics. According to the method and the device, the defect that quick and accurate fault positioning cannot be realized due to relatively dispersed buried point logs and relatively large data volume is overcome, the abnormal indexes deviating from normal periodic or trend fluctuation can be identified, false alarms of business peak periods are avoided, alarms of normal growth trends are avoided, and the abnormal indexes are quickly and accurately positioned.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an intelligent observation method, medium, and electronic device for a device connection platform. Background Technology

[0002] With the rapid development of IoT and Industrial Internet technologies, device-connected business systems have become core infrastructure in fields such as smart manufacturing, smart cities, and smart energy. These systems, through the widespread access of massive numbers of device terminals, achieve real-time data collection, remote control, and business collaboration. Their stability and reliability directly affect production efficiency and operational safety. To ensure the healthy operation of these systems, the industry has generally established corresponding monitoring systems to perceive the system's operating status in real time and issue anomaly alerts, thereby ensuring business continuity.

[0003] Currently, monitoring the operation of business systems connected to devices mainly relies on log-based data collection technology. The basic process is as follows: data collection points (i.e., data collection points) are pre-set at key nodes such as devices, gateways, and business service links to record key events such as device heartbeats, online / offline status, data reporting, and command responses. These scattered log data are collected in real time and aggregated into a centralized data platform (such as a data lake, message queue, or log center). Subsequently, the data is cleaned, aggregated, and analyzed by a data processing engine. Finally, various core indicators (such as device online rate, message throughput, API response time, etc.) are displayed through a visual dashboard, and static threshold alarms are set to achieve macro-level monitoring of the system's operating status.

[0004] However, because the logging points are generated from different components and levels of the system, the data sources are naturally scattered and expand exponentially with the growth of the equipment scale. This results in the monitoring perspective often being isolated and localized, making it difficult to form a systematic and holistic understanding of the entire business system's operation. The comprehensiveness of monitoring is seriously insufficient. Moreover, when the system has problems, the existing monitoring methods can only provide alarms at the phenomenon level. They are inefficient at investigating the root causes. Operation and maintenance personnel need to spend a lot of energy manually performing correlation analysis on massive and scattered historical data and making comprehensive judgments based on data trends and fluctuations. This process is not only time-consuming and labor-intensive, but also highly dependent on the professional experience of the personnel, making it impossible to achieve fast and accurate fault location. Summary of the Invention

[0005] This application provides an intelligent observation method, medium, and electronic device for a device connection platform to address the shortcomings of existing technologies where the embedded logs are relatively scattered and the data volume is large, making it impossible to achieve fast and accurate fault location. It can identify abnormal indicators that deviate from normal periodic or trend fluctuations, avoid false alarms during peak business periods, and avoid alarms for normal growth trends, thereby greatly filtering out invalid alarms and achieving fast and accurate location of abnormal indicators.

[0006] This application provides an intelligent observation method for a device connection platform, including: acquiring end-to-end tracking logs; determining corresponding evaluation indicators based on the source channels of the end-to-end tracking logs, and using device identifiers, Internet Protocol IP addresses, and gateway nodes to associate with external data sources to extend multi-dimensional attributes for the corresponding evaluation indicators, forming corresponding device evaluation indicators with business context; and asynchronously calling an indicator insight agent based on the device evaluation indicators to identify abnormal indicators that deviate from normal periodic or trend fluctuations.

[0007] According to the intelligent observation method for device connection platform provided in this application, based on the source channels of the full-link event logs, corresponding evaluation indicators are determined, including: based on the fact that the source channel of the event logs is at least one of attribute reporting, version reporting, and whole machine version, a time consumption indicator is determined as the corresponding evaluation indicator based on the device gateway sending time and the message platform receiving time; based on the fact that the source channel of the event logs is the device online / offline channel, an online device number indicator is determined as the corresponding evaluation indicator based on the device online / offline logs; based on the fact that the source channel of the event logs is the connection request channel, a request number indicator and a connection status number indicator are determined as the corresponding evaluation indicators; wherein, the request number indicator is determined by first requesting the device main gateway using the device gateway, and the connection status number indicator is determined by first responding to the device gateway using the device main gateway.

[0008] According to the intelligent observation method for a device connection platform provided in this application, after determining the number of online devices as the corresponding evaluation indicator, the method further includes: comparing the predicted number of online devices in the current time window with the number of online devices indicator, and generating a dynamic fluctuation alarm indicator when the actual number of online devices deviates from the predicted number of online devices; wherein, the predicted number of online devices in the current time window is predicted based on historical full-link data entry logs. After determining the request count metric and the connection state count metric as the corresponding evaluation metrics, the method further includes: determining the growth rate of the number of requests in the request count metric and the growth rate of the number of closed states in the connection state metric based on the request count metric and the connection state count metric; and generating a predictive metric for connection quality anomalies when the growth rate of the request count and the growth rate of the number of closed states do not match.

[0009] According to the intelligent observation method for a device connection platform provided in this application, the external data sources include the distributed open-source database HBase, the IP address location database ip2region, and the gateway business database MySQL. The method uses device identifiers, Internet Protocol (IP) addresses, and gateway nodes to associate with the external data sources, expanding multi-dimensional attributes for corresponding evaluation indicators. This includes: asynchronously initiating parallel query requests based on the evaluation indicators, combined with the device identifier, IP address, and gateway node, and expanding the attributes of the corresponding evaluation indicators in multiple dimensions based on the asynchronous query results. Specifically, the asynchronous parallel query requests are used to: query HBase using the device identifier to obtain device attribute information as the corresponding query result; query ip2region using the IP address to obtain regional information as the corresponding query result; and query MySQL using the gateway node to obtain gateway group information as the corresponding query result.

[0010] According to the intelligent observation method for a device connection platform provided in this application, before asynchronously initiating parallel query requests, the method further includes: determining a priority strategy for corresponding dimension expansion based on the type and / or importance of the evaluation indicators; wherein, the dimension expansion priority strategy is used to: determine the priority based on the type and / or importance of the evaluation indicators, and determine the parallel initiation of query requests to HBase, ip2region, and / or MySQL based on the priority; and asynchronously initiate corresponding parallel query requests based on the selected dimension expansion priority strategy.

[0011] According to the intelligent observation method for a device connection platform provided in this application, after forming device evaluation indicators with corresponding business context, the method further includes: storing the device evaluation indicators in a time series database; Based on equipment evaluation metrics, an asynchronous metric insight agent is invoked to identify anomalous metrics that deviate from normal periodic or trend fluctuations. This includes: asynchronously invoking the metric insight agent based on equipment evaluation metrics to pull equipment evaluation metrics for a target time period from a time-series database, and using a long short-term memory network or isolated forest algorithm to analyze historical time-series data of equipment evaluation metrics to identify anomalous metrics that deviate from normal periodic or trend fluctuations.

[0012] According to the intelligent observation method for a device connectivity platform provided in this application, after asynchronously calling an indicator insight agent based on device evaluation indicators to identify abnormal indicators that deviate from normal periodic or trend fluctuations, the method further includes: triggering correlation analysis of multi-dimensional attributes based on the detected abnormal indicators to determine the correlation between each dimension attribute and the abnormal indicators, and locating the key dimensions causing the abnormality; based on the detected abnormal indicators, using a prior knowledge graph built based on devices, gateways, and business services to perform path reasoning to determine the downstream services affected by the abnormal indicators; and generating an insight report containing potential root causes and the scope of impact based on the key dimensions causing the abnormality and the downstream services affected by the abnormal indicators.

[0013] According to the intelligent observation method for a device connectivity platform provided in this application, after forming corresponding device evaluation indicators with business context, the method further includes: storing the device evaluation indicators in a time-series database; the method further includes: using a query statement predefined by the visualization platform to search for the corresponding device evaluation indicators from the time-series database; wherein, the visualization platform pre-uses the time-series database as the target data source and defines corresponding query statements for different types of visualization; based on the found device evaluation indicators, analyze multi-dimensional data and combine it with business objectives or detected abnormal indicators to generate or recommend visualization dashboard templates; based on the found device evaluation indicators and the visualization dashboard templates, construct a visualization dashboard.

[0014] According to the intelligent observation method for device connection platform provided in this application, the full-link event tracking logs are obtained from the event tracking logs stored in the corresponding message queue Kafka based on the subscribed log topics; the event tracking logs stored in Kafka are obtained by collecting and encapsulating the context information of device connection behavior based on captured user-triggered device connection events; the context information includes behavior information, environment information, identity information, business information, and tracking identifiers.

[0015] This application also provides an intelligent observation device for a device connection platform, comprising: a log acquisition module for acquiring end-to-end embedded logs; an evaluation expansion module for determining corresponding evaluation indicators based on the source channels of the end-to-end embedded logs, and extending multi-dimensional attributes for the corresponding evaluation indicators by using device identifiers, Internet Protocol IP addresses, and gateway nodes to associate with external data sources, thereby forming corresponding device evaluation indicators with business context; and an anomaly identification module for asynchronously calling an indicator insight agent based on the device evaluation indicators to identify abnormal indicators that deviate from normal periodic or trend fluctuations.

[0016] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the intelligent observation method for the device connection platform as described above through the computer program.

[0017] This application also provides a computer-readable storage medium comprising a stored program, wherein the program, when executed, implements the intelligent observation method for the device connection platform as described above.

[0018] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the intelligent observation method for the device connection platform as described above.

[0019] The intelligent observation method, medium, and electronic device for device connection platforms provided in this application acquire end-to-end event logs to cover the entire call path from user operation, device end, gateway to backend microservices, providing a global perspective for subsequent analysis. Based on the source channels of the acquired end-to-end event logs, corresponding evaluation indicators are determined to cross-validate from different dimensions, providing a comprehensive and in-depth understanding of device status. Furthermore, by using device identifiers, Internet Protocol IP addresses, and gateway nodes to associate with external data sources, multi-dimensional attributes are extended to the corresponding evaluation indicators, giving them rich business context. This facilitates the identification of abnormal indicators that deviate from normal periodic or trend fluctuations through asynchronously called indicator insight agents, avoiding false alarms during peak business periods and alarms for normal growth trends, thereby greatly filtering out invalid alarms and achieving rapid and accurate location of abnormal indicators. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the hardware environment of a device connection platform intelligent observation method according to an embodiment of this application; Figure 2 This is one of the flowcharts illustrating the intelligent observation method for the device connection platform provided in this application; Figure 3 This is the second flowchart illustrating the intelligent observation method for the device connection platform provided in this application; Figure 4 This is a schematic diagram of the intelligent observation device for the equipment connection platform provided in this application; Figure 5This is a schematic diagram of the electronic device provided in this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] According to one aspect of the embodiments of this application, a device connection platform intelligent observation method is provided. This device connection platform intelligent observation method is widely used in whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, smart home device ecosystems, and intelligence house ecosystems. Optionally, in this embodiment, the above-mentioned device connection platform intelligent observation method can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0026] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0027] Figure 2 This is one of the flowcharts illustrating the intelligent observation method for the device connection platform provided by the present invention, such as... Figure 2 As shown, the method includes the following: S21, obtain full-link event logs; S22. Based on the source channels of the full-link event logs, determine the corresponding evaluation indicators, and use device identifiers, Internet Protocol IP addresses and gateway nodes to associate with external data sources to extend multi-dimensional attributes for the corresponding evaluation indicators, forming corresponding device evaluation indicators with business context. S23, based on device evaluation metrics, asynchronously invokes the metric insight agent to identify abnormal metrics that deviate from normal periodic or trend fluctuations.

[0028] It should be noted that the step number "S2N" in this manual does not represent the order in which the equipment is connected to the platform's intelligent observation method. The following details will explain this in more detail. Figure 3 The present invention describes an intelligent observation method for a device connection platform.

[0029] Step S21: Obtain the full-link event logs.

[0030] In this embodiment, the end-to-end event tracking logs are obtained from the event tracking logs stored in the corresponding message queue Kafka based on the subscribed log topics; the event tracking logs stored in Kafka are obtained by collecting and encapsulating the context information of device connection behavior based on captured user-triggered device connection events; the context information includes behavior information, environment information, identity information, business information, and tracking identifiers.

[0031] It should be noted that there is no direct dependency between the data producer (device or service) and the data consumer. The data producer sends the generated event logs to Kafka. When a massive amount of logs are generated at certain times, Kafka's high throughput can be used as a buffer to smoothly distribute the instantaneous high traffic to the backend processing system, preventing the backend from crashing due to excessive pressure. The data consumer subscribes on demand. Changes, upgrades, or temporary failures on either side will not directly affect the other side, making the system architecture more flexible and robust.

[0032] It should be added that behavioral information is used to characterize operational behavior, environmental information is used to characterize operational environment information, including hardware model, operating system version, application (App) version, and network type, identity information includes user login ID and device ID, business information includes the target connected device category, device model, and connection method, and tracking identifier is generated based on connected device operations and is used to identify all relevant data entry logs from the start to the end of the current connected device operation.

[0033] In addition, based on the subscribed log topic, the instrumentation logs are obtained from the corresponding message queue Kafka, including: based on the subscribed log topic, using the target time window, and according to the trace identifier traceID, the full-link instrumentation logs are obtained; among them, the full-link instrumentation logs include log data from multiple channels, including log data from attribute reporting, version reporting, whole machine version, device online / offline and connection request channels. The log data from attribute reporting, version reporting, whole machine version and device online / offline channels are generated based on the data sent from the device gateway to the message platform, and the log data from the connection request channel are generated based on the interaction between the device gateway and the device main gateway.

[0034] It is worth noting that the above method can be implemented using a pre-built Flink application. The specific application can be built according to the actual design requirements, and no further restrictions are made here.

[0035] In an optional embodiment, before the subscription-based log topic retrieves the event tracking logs from the corresponding message queue Kafka, the method further includes: a data producer generating event tracking logs and sending them to Kafka. Specifically, generating event tracking logs and sending them to Kafka includes: capturing user-triggered device connection events, waking up a pre-built data acquisition software development kit (SDK) to collect context information about the device connection behavior and encapsulate it into a structured data packet to obtain the corresponding event tracking logs; and sending the event tracking logs to the message queue Kafka.

[0036] Accordingly, the process of obtaining full-link event tracking logs includes: obtaining event tracking logs from the corresponding message queue Kafka based on the subscribed log topic, and performing extraction, transformation, and ETL loading to obtain full-link event tracking logs; grouping and aggregating the full-link event tracking logs according to the tracking identifier, and sorting them according to the timestamps corresponding to each event tracking log.

[0037] Step S22: Based on the source channels of the full-link event logs, determine the corresponding evaluation indicators, and use device identifiers, Internet Protocol IP addresses, and gateway nodes to associate with external data sources to extend multi-dimensional attributes for the corresponding evaluation indicators, forming corresponding device evaluation indicators with business context.

[0038] In this embodiment, based on the source channels of the end-to-end event logs, corresponding evaluation metrics are determined, including: If the source channel of the event logs is at least one of attribute reporting, version reporting, and overall device version, a time consumption metric is determined as the corresponding evaluation metric based on the device gateway sending time and the message platform receiving time; if the source channel of the event logs is the device online / offline channel, an online device count metric is determined as the corresponding evaluation metric based on the device's online and offline logs; if the source channel of the event logs is the connection request channel, a request count metric and a connection status count metric are determined as the corresponding evaluation metric; wherein, the request count metric is determined by first requesting the device's main gateway using the device gateway, and the connection status count metric is determined by first responding to the device gateway using the device's main gateway.

[0039] Furthermore, after determining the number of online devices as the corresponding evaluation indicator, the method also includes: comparing the predicted number of online devices in the current time window with the number of online devices indicator, and generating a dynamic fluctuation alarm indicator when the actual number of online devices deviates from the predicted number of online devices; wherein, the predicted number of online devices in the current time window is predicted based on historical full-link data entry logs.

[0040] In addition, after determining the request count metric and the connection state count metric as the corresponding evaluation metrics, the method also includes: determining the growth rate of the number of requests in the request count metric and the growth rate of the number of closed states in the connection state metric based on the request count metric and the connection state count metric; and generating a predictive metric for connection quality anomalies when the growth rate of the request count and the growth rate of the number of closed states do not match.

[0041] It's worth noting that multi-indicator joint analysis can uncover deeper issues that a single indicator cannot reveal, such as "a large number of requests but few successful connections." Furthermore, by determining corresponding evaluation indicators based on different log channels, cross-validation from different dimensions can be performed to gain a comprehensive and in-depth understanding of the device's status. This allows for drilling down to micro-level indicators to pinpoint the root cause when macro-level indicators are abnormal. This hierarchical and progressive indicator design perfectly supports a standardized troubleshooting process.

[0042] In addition, external data sources include the distributed open-source database HBase, the IP address location database ip2region, and the gateway business database MySQL. By associating external data sources with device identifiers, Internet Protocol (IP) addresses, and gateway nodes, multi-dimensional attributes are extended for the corresponding evaluation metrics. This includes: asynchronously initiating parallel query requests based on the evaluation metrics, combined with device identifiers, IP addresses, and gateway nodes, and expanding the attributes of the corresponding evaluation metrics in multiple dimensions based on the asynchronous query results. Specifically, the asynchronous parallel query requests are used for: querying HBase using the device identifier to obtain device attribute information as the corresponding query result; querying ip2region using the IP address to obtain regional information as the corresponding query result; and querying MySQL using the gateway node to obtain gateway group information as the corresponding query result.

[0043] It should be added that the equipment attribute information includes at least one of the following: equipment category, module type, and module version; and the geographical information includes at least one of the following: province, city, and district / county.

[0044] Furthermore, before asynchronously initiating parallel query requests, the method also includes: determining the priority strategy for corresponding dimension expansion based on the type and / or importance of the evaluation metrics; wherein, the dimension expansion priority strategy is used to: determine the priority based on the type and / or importance of the evaluation metrics, and determine the parallel initiation of query requests to HBase, ip2region, and / or MySQL based on the priority; and asynchronously initiating the corresponding parallel query requests based on the selected dimension expansion priority strategy.

[0045] It's worth noting that when the priority is high, all query requests to HBase, ip2region, and MySQL are initiated in parallel; when the priority is medium, query requests to HBase and ip2region are initiated in parallel; and when the priority is low, only query requests to ip2region are initiated or dimension expansion is skipped. Furthermore, this priority strategy avoids excessive queries on unimportant metrics, thus concentrating limited computing resources on ensuring the dimension expansion of core business metrics, significantly reducing the overall system load and operating costs.

[0046] In an optional embodiment, after forming the device evaluation metrics corresponding to the business context, the method further includes storing the device evaluation metrics in a time-series database. It should be noted that the time-series database can be selected according to actual design requirements, such as Prometheus; no further limitation is made here.

[0047] Step S23: Based on the device evaluation metrics, asynchronously call the metric insight agent to identify abnormal metrics that deviate from normal periodic or trend fluctuations according to the device evaluation metrics.

[0048] In this embodiment, based on the device evaluation metrics, an indicator insight agent is asynchronously invoked to identify abnormal metrics that deviate from normal periodic or trend fluctuations. This includes: based on the device evaluation metrics, the indicator insight agent is asynchronously invoked to retrieve device evaluation metrics for a target time period from a time-series database, and using a long short-term memory network or an isolated forest algorithm to analyze the historical time-series data of the device evaluation metrics to identify abnormal metrics that deviate from normal periodic or trend fluctuations.

[0049] It's important to note that by asynchronously invoking the indicator insight agent, the indicator calculation and agent invocation are decoupled. This allows indicator calculation to run continuously at high speed without being slowed down by complex AI analysis tasks. Even if the indicator insight agent malfunctions or responds slowly, the indicator calculation process remains unaffected, ensuring the high availability of the entire monitoring system. When the number of indicators to be analyzed surges, the number of agent instances can be increased independently to enhance analytical capabilities without modifying the upstream indicator calculation process. This allows intelligent analysis capabilities to scale on demand, improving cost-effectiveness. Furthermore, by leveraging machine learning, it accurately captures subtle trend deviations, improving the accuracy of anomaly indicator identification, enabling intervention and remediation before problems escalate into major failures.

[0050] In an optional embodiment, after asynchronously invoking the indicator insight agent based on device evaluation indicators to identify abnormal indicators that deviate from normal periodic or trend fluctuations according to the device evaluation indicators, the method further includes: notifying the operations and maintenance developers to pay attention to and investigate indicator problems based on the abnormal indicators.

[0051] In addition, after asynchronously invoking the indicator insight agent based on device evaluation metrics to identify abnormal indicators that deviate from normal periodic or trend fluctuations, the method further includes: triggering correlation analysis of multi-dimensional attributes based on the detected abnormal indicators to determine the correlation between each dimension attribute and the abnormal indicators, and locating the key dimensions causing the abnormality; using a previously constructed knowledge graph based on devices, gateways, and business services to perform path reasoning based on the detected abnormal indicators, and determining the downstream services affected by the abnormal indicators; and generating an insight report containing potential root causes and the scope of impact based on the key dimensions causing the abnormality and the downstream services affected by the abnormal indicators.

[0052] It should be noted that by identifying the key dimensions causing the anomalies when abnormal indicators are detected, complex connections that are difficult for human intuition to perceive can be discovered, avoiding pitfalls in troubleshooting, achieving automatic root cause localization, improving operational efficiency, and using knowledge graphs for path reasoning to achieve accurate assessment of the scope of impact. Based on the key dimensions and scope of impact, a structured and easy-to-understand insight report is automatically integrated, allowing for precise remedial actions to be executed directly based on the root causes and scope of impact in the report, achieving precise fault handling and avoiding distortion during information transmission. At the same time, it can be archived as a historical case, forming organizational knowledge accumulation for future review and training.

[0053] In an optional embodiment, the method further includes: using a query statement predefined by the visualization platform to search for corresponding device evaluation indicators from a time-series database; wherein the visualization platform predefined the time-series database as the target data source and defined corresponding query statements for different types of visualization; analyzing multi-dimensional data based on the found device evaluation indicators and combining them with business objectives or detected abnormal indicators to generate or recommend visualization dashboard templates; and constructing a visualization dashboard based on the found device evaluation indicators and the visualization dashboard templates.

[0054] It should be added that the visualization platform can be determined according to actual usage needs, such as Grafana, etc. There are no further restrictions here. The visualization platform Grafana is used to connect to the time series database Prometheus to build visualization dashboards. The visualization dashboards include indicator cards, trend charts, etc. The specifics can be determined according to actual display design requirements, and there are no further restrictions here.

[0055] In addition, before using the pre-defined query statements of the visualization platform to find the corresponding device evaluation metrics from the time series database, the method also includes: adding a target data source in Grafana; wherein, the target data source is used to provide the Prometheus service address (URL) and access credentials; and defining the query statements according to the visualization type.

[0056] Specifically, Grafana offers a variety of visualization options, including charts, heatmaps, and tables. Depending on the type of query results and the desired effect, select the appropriate visualization type. Correspondingly, define the query statement based on the visualization type, including: for indicator cards, write queries that return a single value; for trend charts, write queries that return a series of data changing over time. It's worth noting that queries can be written using the Prometheus Query Language (PromQL) built into the time-series database Prometheus.

[0057] In addition, the visualization platform uses pre-defined query statements to search for corresponding device evaluation metrics from the time series database, including: connecting to the corresponding Prometheus based on the target data source, and using query statements to search for corresponding device evaluation metrics from the time series database.

[0058] Furthermore, business objectives are determined based on business calendars, system change records (such as version releases), or user-specified monitoring topics (such as 'focus on network quality in XX region'), thereby grounding business objectives in specific and actionable data sources, such as calendars, continuous integration / continuous delivery (CI / CD) records, and user input, making the implementation path of the entire solution clearer.

[0059] In addition, based on the identified equipment evaluation metrics, multi-dimensional data is analyzed, and combined with business objectives or detected abnormal indicators, a visual dashboard template is generated or recommended. This includes: based on the identified equipment evaluation metrics, multi-dimensional data is analyzed, and combined with business objectives or detected abnormal indicators, a corresponding visual dashboard template is generated or a business scenario is determined, so as to match the corresponding dashboard template according to the business scenario and recommend the corresponding visual dashboard template; or, the equipment evaluation metrics and abnormal indicators are input into a pre-trained graph neural network model to obtain an optimized dashboard layout configuration.

[0060] In addition, after constructing the visual dashboard, the method also includes: recording users' adoption, modification, or abandonment of recommended dashboard templates, and using these behaviors as feedback signals to optimize template scene matching rules or update the graph neural network model, so as to achieve the self-evolution of the dashboard template recommendation capability.

[0061] It should be noted that by introducing a feedback loop and a self-learning mechanism, the system is no longer static, but can be continuously optimized based on the user's actual usage, thereby improving the visual dashboard.

[0062] Furthermore, the method also includes: monitoring the execution performance of query statements; and optimizing query statements, for example by increasing aggregation granularity or shortening the query time range, when the query time exceeds the corresponding threshold, to ensure the response speed of the dashboard.

[0063] In summary, this invention, by acquiring end-to-end event logs, covers the entire call path from user operation, device, gateway to backend microservice, providing a global perspective for subsequent analysis. Based on the source channels of the acquired end-to-end event logs, corresponding evaluation indicators are determined to cross-validate from different dimensions, providing a comprehensive and in-depth understanding of device status. Furthermore, by using device identifiers, Internet Protocol IP addresses, and gateway nodes to associate with external data sources, multi-dimensional attributes are extended to the corresponding evaluation indicators, giving them rich business context. This facilitates the identification of abnormal indicators that deviate from normal periodic or trend fluctuations through asynchronously called indicator insight agents, avoiding false alarms during peak business periods and alarms for normal growth trends. This significantly filters out invalid alarms, enabling rapid and accurate location of abnormal indicators.

[0064] The intelligent observation device for the equipment connection platform provided in this application is described below. The intelligent observation device for the equipment connection platform described below can be referred to in correspondence with the intelligent observation method for the equipment connection platform described above.

[0065] Figure 4 A schematic diagram of a smart observation device for connecting equipment to a platform is shown. The device includes: Log acquisition module 41 acquires full-link event logs; The evaluation expansion module 42 determines the corresponding evaluation indicators based on the source channels of the full-link embedded logs, and uses device identifiers, Internet Protocol IP addresses and gateway nodes to associate external data sources to expand the multi-dimensional attributes of the corresponding evaluation indicators, forming corresponding device evaluation indicators with business context. The anomaly identification module 43 asynchronously calls the indicator insight agent based on the device evaluation indicators to identify abnormal indicators that deviate from normal periodic or trend fluctuations.

[0066] In this embodiment, the end-to-end event tracking logs are obtained from the event tracking logs stored in the corresponding message queue Kafka based on the subscribed log topics; the event tracking logs stored in Kafka are obtained by collecting and encapsulating the context information of device connection behavior based on captured user-triggered device connection events; the context information includes behavior information, environment information, identity information, business information, and tracking identifiers.

[0067] In addition, the log acquisition module 41 is used to: acquire full-link event logs based on the subscribed log topics, using the target time window, and according to the trace identifier (traceID); wherein, the full-link event logs include log data from multiple channels, including log data from attribute reporting, version reporting, whole machine version, device online / offline, and connection request channels. The log data from attribute reporting, version reporting, whole machine version, and device online / offline channels are generated based on data sent from the device gateway to the message platform, and the log data from the connection request channel are generated based on data generated based on data from the interaction between the device gateway and the device main gateway.

[0068] In one optional embodiment, the data producer is configured to: generate and send event logs to Kafka before retrieving event logs from the corresponding message queue Kafka based on the subscription-based log topic. Specifically, generating and sending event logs to Kafka includes: capturing user-triggered device connection events, waking up a pre-built data acquisition software development kit (SDK) to collect context information about the device connection behavior and encapsulate it into a structured data packet to obtain the corresponding event logs; and sending the event logs to the message queue Kafka.

[0069] Correspondingly, the log acquisition module 41 includes: a log acquisition unit, which acquires the instrumentation logs in the corresponding message queue Kafka based on the subscribed log topic, and performs extraction, transformation, and ETL loading processing to obtain the full-link instrumentation logs; and an aggregation processing unit, which groups and aggregates the full-link instrumentation logs according to the tracking identifier, and sorts them according to the timestamps corresponding to each instrumentation log.

[0070] The evaluation expansion module 42 includes an indicator evaluation unit, used for: determining the time consumption indicator as the corresponding evaluation indicator based on the source channel of the event logs being at least one of attribute reporting, version reporting, and whole machine version, according to the device gateway sending time and message platform receiving time; determining the number of online devices as the corresponding evaluation indicator based on the device online and offline logs, according to the device online and offline logs, based on the source channel of the event logs being the connection request channel; and determining the number of requests and the number of connection statuses as the corresponding evaluation indicators based on the source channel of the event logs being the connection request channel. The number of requests is determined by first requesting the device main gateway using the device gateway, and the number of connection statuses is determined by first responding to the device gateway using the device main gateway.

[0071] Furthermore, the indicator evaluation unit is also used to: after determining the number of online devices as the corresponding evaluation indicator, compare the predicted number of online devices in the current time window with the number of online devices based on the previously predicted number of online devices in the current time window, and generate a dynamic fluctuation alarm indicator when the actual number of online devices deviates from the predicted number of online devices; wherein, the predicted number of online devices in the current time window is predicted based on historical full-link data entry logs.

[0072] In addition, the indicator evaluation unit is also used to: after determining the request count indicator and the connection status count indicator as the corresponding evaluation indicators, determine the growth rate of the number of requests in the request count indicator and the growth rate of the number of closed states in the connection status indicator based on the request count indicator and the connection status count indicator; and generate a prediction indicator for connection quality anomalies when the growth rate of the request count and the growth rate of the number of closed states do not match.

[0073] In addition, external data sources include the distributed open-source database HBase, the IP address location library ip2region, and the gateway business library MySQL. The evaluation expansion module 42 also includes a dimension expansion unit, which is used to: asynchronously initiate parallel query requests based on the evaluation indicators, combined with device identifiers, IP addresses, and gateway nodes, and expand the attributes of the corresponding evaluation indicators in multiple dimensions based on the asynchronous query results. Among them, the asynchronous initiation of parallel query requests is used to: use the device identifier to query HBase and obtain device attribute information as the corresponding query result; use the IP address to query ip2region and obtain regional information as the corresponding query result; and use the gateway node to query MySQL and obtain gateway group information as the corresponding query result.

[0074] Furthermore, the evaluation dimension expansion module 42 also includes an asynchronous request initiation unit, used to: determine the priority strategy for the corresponding dimension expansion based on the type and / or importance of the evaluation metrics before asynchronously initiating parallel query requests; wherein, the dimension expansion priority strategy is used to: determine the priority based on the type and / or importance of the evaluation metrics, and determine the parallel initiation of query requests to HBase, ip2region, and / or MySQL based on the priority; and asynchronously initiate the corresponding parallel query requests based on the selected dimension expansion priority strategy.

[0075] In an optional embodiment, the evaluation expansion module 42 further includes a data storage unit, which stores the device evaluation indicators into a time-series database after forming device evaluation indicators corresponding to business contexts.

[0076] The anomaly identification module 43 is used to: asynchronously call the indicator insight agent based on the equipment evaluation indicators to pull the equipment evaluation indicators within the target time period from the time series database, and use the long short-term memory network or the isolated forest algorithm to analyze the historical time series data of the equipment evaluation indicators and identify abnormal indicators that deviate from normal periodic or trend fluctuations.

[0077] In an optional embodiment, the device further includes: an alarm module that, after asynchronously calling an indicator insight agent based on device evaluation indicators to identify abnormal indicators that deviate from normal periodic or trend fluctuations according to the device evaluation indicators, notifies the operation and maintenance developers to pay attention to and investigate indicator problems based on the abnormal indicators.

[0078] In addition, the device also includes: a root cause localization module, which asynchronously calls an indicator insight agent based on device evaluation indicators to identify abnormal indicators that deviate from normal periodic or trend fluctuations based on the device evaluation indicators, and triggers correlation analysis of multi-dimensional attributes based on the detected abnormal indicators to determine the correlation between each dimension attribute and the abnormal indicators, and locate the key dimensions causing the abnormality; an impact scope determination module, which uses a knowledge graph previously built based on devices, gateways, and business services to perform path reasoning based on the detected abnormal indicators, and determines the downstream services affected by the abnormal indicators; and a report generation module, which generates an insight report containing potential root causes and impact scope based on the key dimensions causing the abnormality and the downstream services affected by the abnormal indicators.

[0079] In an optional embodiment, the device further includes a visualization module for: searching for corresponding device evaluation indicators from a time-series database using query statements predefined by the visualization platform; wherein the visualization platform predefined the time-series database as the target data source and defined corresponding query statements for different types of visualization; analyzing multi-dimensional data based on the found device evaluation indicators and combining them with business objectives or detected abnormal indicators to generate or recommend visualization dashboard templates; and constructing a visualization dashboard based on the found device evaluation indicators and the visualization dashboard templates.

[0080] In addition, the device also includes a configuration module for: adding a target data source in Grafana before using a query statement predefined by the visualization platform to retrieve the corresponding device evaluation metrics from the time-series database; wherein the target data source is used to provide the Prometheus service address (URL) and access credentials; and defining the query statement according to the visualization type.

[0081] Specifically, Grafana offers a variety of visualization options, including charts, heatmaps, and tables. Depending on the type of query results and the desired presentation, you can select the appropriate visualization type. Correspondingly, the configuration module allows you to: write queries that return a single value for indicator cards; and write queries that return a series of data that changes over time for trend charts. It's worth noting that queries can be written using the Prometheus Query Language (PromQL) built into the time-series database Prometheus.

[0082] In addition, the visualization module is also used to: connect to the corresponding Prometheus based on the target data source, and use query statements to find the corresponding device evaluation indicators from the time series database.

[0083] Furthermore, the visualization module is also used to: analyze multi-dimensional data based on the found equipment evaluation indicators, and combine them with business objectives or detected abnormal indicators to generate corresponding visualization dashboard templates or determine business scenarios, so as to match the corresponding dashboard templates according to the business scenarios and recommend corresponding visualization dashboard templates; or, input the equipment evaluation indicators and abnormal indicators into the pre-trained graph neural network model to obtain optimized dashboard layout configurations.

[0084] In addition, the device also includes an update module, which is used to: after building the visual dashboard, record the user's adoption, modification or abandonment of the recommended dashboard template, and use these behaviors as feedback signals to optimize the template scene matching rules or update the graph neural network model, so as to realize the self-evolution of the dashboard template recommendation capability.

[0085] Furthermore, the device also includes a statement optimization module, which is used to: monitor the execution performance of query statements; and optimize the query statements when the query time exceeds the corresponding threshold, for example by increasing the aggregation granularity or shortening the query time range, so as to ensure the response speed of the dashboard.

[0086] In summary, this embodiment of the invention acquires end-to-end event logs through a log acquisition module, covering the entire call path from user operation, device, gateway to backend microservices. This provides a global perspective for subsequent analysis. Furthermore, the evaluation and expansion module determines corresponding evaluation metrics based on the source channels of the acquired end-to-end event logs, enabling cross-validation from different dimensions to comprehensively and deeply understand device status. By using device identifiers, Internet Protocol IP addresses, and gateway nodes to associate with external data sources, multi-dimensional attributes are extended to the corresponding evaluation metrics, giving them rich business context. This facilitates the identification of abnormal metrics that deviate from normal periodic or trend fluctuations through the asynchronous call to the metric insight agent via the anomaly identification module. This avoids false alarms during peak business periods and alarms for normal growth trends, significantly filtering out invalid alarms and enabling rapid and accurate location of abnormal metrics.

[0087] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540. The processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a device connection platform intelligent observation method. This method includes: acquiring end-to-end tracking logs; determining corresponding evaluation indicators based on the source channels of the end-to-end tracking logs, and using device identifiers, Internet Protocol IP addresses, and gateway nodes to associate with external data sources to extend multi-dimensional attributes for the corresponding evaluation indicators, forming corresponding device evaluation indicators with business context; and asynchronously calling an indicator insight agent based on the device evaluation indicators to identify abnormal indicators that deviate from normal periodic or trend fluctuations.

[0088] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0089] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the intelligent observation method for the device connection platform provided by the above methods. The method includes: acquiring end-to-end tracking logs; determining corresponding evaluation indicators based on the source channels of the end-to-end tracking logs, and using device identifiers, Internet Protocol IP addresses, and gateway nodes to associate with external data sources to extend multi-dimensional attributes for the corresponding evaluation indicators, forming corresponding device evaluation indicators with business context; and asynchronously calling an indicator insight agent based on the device evaluation indicators to identify abnormal indicators that deviate from normal periodic or trend fluctuations according to the device evaluation indicators.

[0090] On another front, this application also provides a computer-readable storage medium, which includes a stored program, wherein the program executes the intelligent observation method for the device connection platform provided by the above methods when it runs. The method includes: acquiring end-to-end tracking logs; determining corresponding evaluation indicators based on the source channels of the end-to-end tracking logs, and using device identifiers, Internet Protocol IP addresses, and gateway nodes to associate with external data sources to extend multi-dimensional attributes for the corresponding evaluation indicators, forming corresponding device evaluation indicators with business context; and asynchronously calling an indicator insight agent based on the device evaluation indicators to identify abnormal indicators that deviate from normal periodic or trend fluctuations according to the device evaluation indicators.

[0091] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0092] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for intelligent observation of a device connection platform, characterized in that, include: Obtain full-link event logs; Based on the source channels of the full-link event logs, the corresponding evaluation indicators are determined, and the device identifier, Internet Protocol IP address and gateway node are used to associate external data sources to extend the multi-dimensional attributes of the corresponding evaluation indicators, forming corresponding device evaluation indicators with business context. Based on the device evaluation metrics, the metrics insight agent is asynchronously invoked to identify abnormal metrics that deviate from normal periodic or trend fluctuations, according to the device evaluation metrics.

2. The intelligent observation method for the equipment connection platform according to claim 1, characterized in that, Based on the source channels of the aforementioned end-to-end event logs, corresponding evaluation metrics are determined, including: Based on the fact that the source channels of the event logs are at least one of attribute reporting, version reporting, and whole machine version, the time consumption index is determined as the corresponding evaluation index according to the device gateway sending time and message platform receiving time. Based on the fact that the source channel of the aforementioned data collection logs is the device online / offline channel, the number of online devices is determined as the corresponding evaluation indicator based on the device online / offline logs. Based on the fact that the source channel of the aforementioned event logs is the connection request channel, the request count metric and the connection status count metric are determined as corresponding evaluation metrics; wherein, the request count metric is determined by first requesting the device main gateway using the device gateway, and the connection status count metric is determined by first responding to the device gateway using the device main gateway.

3. The intelligent observation method for the equipment connection platform according to claim 2, characterized in that, After determining the number of online devices as the corresponding evaluation indicator, the method further includes: Based on the predicted number of online devices in the current time window, a dynamic fluctuation alarm indicator is generated when the actual number of online devices deviates from the predicted number of online devices, compared with the online device count indicator. The predicted number of online devices in the current time window is based on historical full-link data entry logs. After determining the request count metric and the connection state count metric as the corresponding evaluation metrics, the method further includes: Based on the request count metric and the connection status count metric, determine the growth rate of the number of requests in the request count metric and the growth rate of the number of closed states in the connection status metric; When the growth rate of the number of requests does not match the growth rate of the number of closed states, a predictive indicator of abnormal connection quality is generated.

4. The intelligent observation method for the equipment connection platform according to claim 1, characterized in that, The external data sources include the distributed open-source database HBase, the IP address location library ip2region, and the gateway business database MySQL. By using device identifiers, Internet Protocol (IP) addresses, and gateway nodes to associate with the external data sources, multi-dimensional attributes are extended to the corresponding evaluation metrics, including: Based on the evaluation metrics, combined with device identifier, IP address, and gateway node, asynchronous parallel query requests are initiated, and the attributes of the corresponding evaluation metrics are expanded in multiple dimensions based on the asynchronous query results; wherein, the asynchronous initiation of parallel query requests is used for: Using the device identifier, query the hbase to obtain the device attribute information as the corresponding query result; Based on the IP address, query the ip2region to obtain the region information as the corresponding query result; Based on the gateway node, query the MySQL database to obtain the gateway group information as the corresponding query result.

5. The intelligent observation method for the equipment connection platform according to claim 4, characterized in that, Before asynchronously initiating parallel query requests, the method further includes: Based on the type and / or importance of the evaluation metrics, a priority strategy for the corresponding dimension expansion is determined; wherein, the dimension expansion priority strategy is used to: determine the priority based on the type and / or importance of the evaluation metrics, and determine, based on the priority, to initiate query requests to the hbase, the ip2region, and / or the mysql in parallel; Based on the selected dimension, the priority strategy is extended, and corresponding parallel query requests are initiated asynchronously.

6. The intelligent observation method for the equipment connection platform according to claim 1, characterized in that, After generating device evaluation metrics with corresponding business contexts, the method further includes: The equipment evaluation metrics are stored in a time-series database; Based on the device evaluation metrics, an asynchronous metric insight agent is invoked to identify anomalous metrics that deviate from normal periodic or trend fluctuations, including: Based on the device evaluation metrics, the metric insight agent is asynchronously invoked to retrieve device evaluation metrics for a target time period from the time-series database. Then, using a long short-term memory network or an isolated forest algorithm, the historical time-series data of the device evaluation metrics is analyzed to identify abnormal metrics that deviate from normal periodic or trend fluctuations.

7. The intelligent observation method for the equipment connection platform according to claim 1, characterized in that, After asynchronously invoking the indicator insight agent based on the device evaluation indicators to identify abnormal indicators that deviate from normal periodic or trend fluctuations according to the device evaluation indicators, the method further includes: Based on the detection of abnormal indicators, a correlation analysis of multi-dimensional attributes is triggered to determine the degree of correlation between each dimension attribute and the abnormal indicators, and to locate the key dimension that caused the abnormality. Based on the detected abnormal indicators, path reasoning is performed using a knowledge graph previously built based on devices, gateways, and business services to determine the downstream services affected by the abnormal indicators. Based on the key dimensions that caused the anomaly and the downstream services affected by the anomaly indicators, an insight report containing potential root causes and the scope of impact is generated.

8. The intelligent observation method for the equipment connection platform according to claim 1, characterized in that, After generating device evaluation metrics with corresponding business contexts, the method further includes: The equipment evaluation metrics are stored in a time-series database; The method further includes: The visualization platform uses pre-defined query statements to search for corresponding device evaluation indicators from the time series database; wherein, the visualization platform uses the time series database as the target data source and defines corresponding query statements for different types of visualization. Based on the identified equipment evaluation metrics, analyze multi-dimensional data and combine it with business objectives or detected abnormal metrics to generate or recommend visual dashboard templates. Based on the identified equipment evaluation metrics and the aforementioned visualization dashboard template, a visualization dashboard is constructed.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 8.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 8 through the computer program.

Images