A method and system for cross-device multi-application traffic performance evaluation

Abstract

The application discloses a cross-device multi-application service performance evaluation method and system, and is used for industrial internet. The method comprises the following steps: in response to cross-device multi-application service, a communication path and a communication timing between multiple applications of the same service are obtained through a device monitoring interface, a service graph is generated, a default set of application layer deterministic parameters and monitoring results are obtained, and a deterministic evaluation value of a sub-chain of the multiple applications in the service graph is counted. The system of the application comprises an application layer deterministic monitoring module connected with a monitoring agent of the device, and is used for implementing the method. The application solves the problem that the prior art cannot meet the monitoring and evaluation requirements of complex services of the industrial internet.

Description

Technical Field

[0001] This application relates to the field of industrial internet technology, and in particular to a method and system for evaluating business performance across multiple devices and applications. Background Technology

[0002] The Industrial Internet connects various elements within an enterprise's intranet to form a digital and intelligent production model. It is the result of the deep integration of industrial systems, network technology, computing technology, and sensing technology. Enterprise intranets contain a large number of devices with diverse functions. In the Industrial Internet, these devices primarily refer to deterministic networks, edge computing devices, industrial terminal devices, and cloud platforms. Network determinism refers to ensuring that data streams meet requirements in terms of latency, jitter, packet loss rate, reliability, and bandwidth during network transmission, achieving deterministic boundaries for these indicators through hardware and software resource scheduling. Application-layer determinism refers to achieving determinism in business processes completed collaboratively by multiple applications. From the user's perspective, they are only concerned with the performance level under which the expected function is achieved after the business has passed through multiple devices and applications. Therefore, monitoring application-layer deterministic business requires comprehensive monitoring methods targeting multiple applications across devices.

[0003] Currently, various devices in the Industrial Internet are relatively independent. When performing deterministic configuration, it is necessary to collect real-time parameters from the edge computing network cloud platform at the terminal, and then perform global control and resource scheduling based on these parameters. In this process, existing solutions mainly monitor and measure individual network devices or applications that have already been deployed, without comprehensively recording the various interactions between network applications and then evaluating application-layer deterministic parameters based on the overall control variables.

[0004] For example, existing monitoring methods include network monitoring, edge computing monitoring, and application monitoring. Network monitoring methods mainly collect data and statistical information through protocols such as sFlow, netflow, netconfig, and SDN. SDN-based monitoring methods can only provide network quality and deterministic assessments of links. Current network monitoring methods are mostly for Layer 2-3 testing, providing traffic distribution oriented towards data flows, focusing on analyzing network resource load. Edge computing monitoring methods mainly collect resource usage information of applications, virtual machines, or containers through the operating system. Application monitoring methods mainly include the following categories: first, monitoring client and server trigger commands; second, extracting application characteristic information and file modification operations; third, collecting application remote communication data and extracting application layer information; and fourth, monitoring time information generated by applications periodically or abnormally. These methods mainly monitor applications on individual devices. The above-mentioned solutions can only monitor applications on a certain type of device in the Industrial Internet, and cannot monitor and evaluate the overall business of multiple applications cooperating with each other. Therefore, they cannot meet the monitoring and evaluation needs of complex business in the Industrial Internet. Summary of the Invention

[0005] This application proposes a cross-device, multi-application business performance evaluation method and system to address the problem that applications on a certain type of device in the Industrial Internet cannot monitor and evaluate the overall business of multiple applications working together, thus failing to meet the monitoring and evaluation needs of complex business in the Industrial Internet.

[0006] In a first aspect, embodiments of this application propose a cross-device, multi-application service performance evaluation method for the Industrial Internet, comprising the following steps:

[0007] In response to services that span multiple applications, which are processes involving applications on multiple devices and have calling relationships between them, the communication paths and timing sequences between multiple applications of the same service are obtained through the device monitoring interface, and a service graph is generated with applications as nodes and connected according to the communication paths and timing sequences.

[0008] Obtain the default set of application-layer deterministic parameters and monitoring results, and statistically analyze the deterministic evaluation values ​​of multiple applications in the sub-chain of the business graph. The deterministic evaluation value of each application is a comparison value of the default set of application-layer deterministic parameters and monitoring results for each application.

[0009] Preferably, the identifier of the service is a network-unique identifier used to determine that the applications belong to the same service.

[0010] Preferably, the cross-device multi-application service performance evaluation method further includes a service graph self-learning process based on service identifiers: the service initiator sends a service traversal message, and records the association between the application and the service for each application it passes through, records the mapping relationship between the application and the service as a tag and adds it to the payload of the message, and uploads the message at the receiving end.

[0011] Preferably, the deterministic evaluation values ​​of multiple applications in the sub-chains of the business graph are statistically analyzed, including one or more of the following parameters:

[0012] Subchain latency, jitter, packet loss rate, reliability, bandwidth, number of computing units, and priority;

[0013] Chain length, node degree, relaxation factor;

[0014] The node's application type, business data payload, and business parallelization coefficient.

[0015] Preferably, the business graph is recorded by real-time collected information, and a classifier is used to map the business graph onto a known business graph, thereby providing a classification result.

[0016] More preferably, for each application-layer deterministic metric, a Sketch test structure is established, each Sketch matrix contains the same type of parameters from multiple businesses, and each Sketch uses an independent statistical algorithm.

[0017] Secondly, this application also proposes a cross-device, multi-application business performance evaluation system for implementing the method described in the first aspect of this application, including an application-layer deterministic monitoring module connected to the monitoring agent of the device.

[0018] In response to the device running the service, the application layer deterministic monitoring module obtains the communication path and communication timing between multiple applications of the same service through the device monitoring interface, and generates a service graph with applications as nodes connected according to the communication path and timing.

[0019] The application layer deterministic monitoring module is also used to: obtain the default set of application layer deterministic parameters and monitoring results, and to statistically analyze the deterministic evaluation values ​​of multiple applications in the sub-chain of the business graph.

[0020] Preferably, the cross-device, multi-application service performance evaluation system further includes the monitoring agents. Multiple monitoring agents are respectively installed on terminal devices, edge devices, and / or network controllers.

[0021] In any device, the monitoring agent is used for at least one of the following functions: during the data receiving phase, collecting the timestamp of the received data and / or the data volume monitoring results; during the application execution phase, collecting the deterministic parameter monitoring results during business execution; and during the data sending phase, collecting the timestamp of the sent data and / or the data volume monitoring results.

[0022] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0023] This invention addresses the complex business processes comprised of multiple collaborative applications, enabling a holistic assessment of the closed-loop business control chain. Building upon existing single-type device monitoring solutions and addressing the deterministic needs of the application layer, it innovatively proposes a monitoring method and assessment tool to measure the overall performance of cross-device business processes and construct a closed-loop business control chain. It directly provides quantitative assessment results of complex business processes from the user's perspective.

[0024] This solution is mainly based on this to form a monitoring method for cross-device and multi-application applications. It then realizes the detection of various managed devices by combining active and passive control. Optimally, the corresponding data is then classified and summarized in the form of sketches. Finally, based on the combination calculation method and weight of each sketch, a deterministic evaluation result is finally summarized. At the same time, each parameter can be queried, which improves the efficiency of system monitoring. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0026] Figure 1 This is a flowchart illustrating an embodiment of the method of this application;

[0027] Figure 2 Here is an example of a business diagram;

[0028] Figure 3 A cross-device application monitoring architecture;

[0029] Figure 4 A schematic diagram illustrating the application information monitoring structure;

[0030] Figure 5 This is a schematic diagram of a deterministic evaluation based on the Sketch group;

[0031] Figure 6 This application describes the process of implementing application-layer deterministic monitoring using the methods and systems described herein. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] Application-layer deterministic systems need to monitor the relationships between multiple applications. In the Industrial Internet, various devices are relatively independent. When performing deterministic configuration, it is necessary to collect real-time parameters from the edge computing network cloud platform on the terminal, and then perform global control and resource scheduling based on these parameters. In this process, existing solutions mainly monitor and measure based on the deployed individual network devices or applications, without comprehensively recording the various interactions between network applications.

[0034] This solution is a comprehensive monitoring solution for applications across multiple devices, including networks, edge computing, and terminals. It targets services encompassing multiple applications, combines remote call relationships between applications to measure overall cross-device metrics, and can provide metrics for each node of the service, service topology metrics, and deterministic assessments.

[0035] This solution defines categories of application-layer deterministic parameters. Therefore, in the preferred embodiment of this application, a Sketch group is used to store the monitored values ​​of the application-layer deterministic parameters for each service. Reference: Cormode, G., & Muthukrishnan, S. (2004 An Improved Data Stream Summary: The Count-Min Sketch and Its Applications. Paper presented at the Journal of Algorithms).

[0036] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0037] Figure 1 This is a flowchart illustrating an embodiment of the method of this application.

[0038] This application proposes a cross-device, multi-application service performance evaluation method for the Industrial Internet, comprising the following steps:

[0039] Step 10: Run the service. The service is a process in which multiple devices' applications participate in the operation, and there are calling relationships between the multiple applications.

[0040] The types of equipment include, but are not limited to, terminals, network equipment, edge computing devices, and cloud platforms;

[0041] It should be noted that the business proposed in this application is a specific instance of a complex business, in which applications on multiple devices participate in sub-businesses such as computing and storage, and there are remote call relationships between the applications;

[0042] Preferably, the identifier of the service is a unique network identifier used to determine that the applications belong to the same service. The managed device uploads application information and opens the monitoring interface through registration. An application can run multiple services simultaneously. The application layer deterministic monitoring module records the unique network identifier of the application layer deterministic service and records the communication sequence between applications based on the representations reported by subsequent applications. Application layer determinism is monitored on a service graph basis for a certain service.

[0043] Step 20: In response to the business, obtain the communication path and communication sequence between multiple applications of the same business through the device monitoring interface, and generate a business graph with applications as nodes connected according to the communication path and sequence.

[0044] The communication path can be either a one-way path or a closed-loop path formed by feedback after the remote service is completed.

[0045] The business involved in this application involves multiple devices, related services and communication paths, ultimately forming a directed acyclic graph consisting of multiple processing stages, which is referred to as the business graph in this solution.

[0046] Taking 5G+factory internal logistics as an example, an AGV (Automated Guided Vehicle) is responsible for transporting materials to a conveyor belt. The conveyor belt then transfers the materials to processing equipment, which in turn calls complex algorithms on the platform. All devices are interconnected via a 5G network. In this scenario, the applications of the AGV, the conveyor belt controller, the processing equipment, and the platform have fixed calling relationships. The nodes in the business graph represent the applications running on each device that perform operations related to that business, and the communication path is the end-to-end communication between any two applications. This method installs monitoring agents on these devices. Once a business is created, the agents on each device report the execution status of the business on each application using a business identifier, thereby enabling monitoring and analysis based on the business graph.

[0047] Examples of business diagrams include Figure 2 As shown, the nodes are circles, representing each application involved in the business, which generally performs specific operations in the business diagram; the communication paths are arrows, representing the process of information transmission between applications, with the transmitted content being the input data of the next application.

[0048] Similarly, taking a visual inspection scenario in industry as an example, a camera inspects the condition of industrial materials, transmits the data to an edge computing server, runs a machine learning algorithm to determine how to process them, and sends the operation and parameters to a robotic arm. After the robotic arm performs the processing steps, the condition of the materials is re-identified by the camera. In this case, the applications on the camera, edge computing server, and robotic arm have a fixed calling relationship. Similarly, within a 5G network, different applications run on multiple devices. After each application is executed in a specific order, it can complete functions such as terminal identification, access, and billing. Each service completed by multiple applications across devices is the monitoring and evaluation object of this invention.

[0049] Preferably, the performance indicators of the service graph are monitored, and the topological complexity of the service graph is analyzed. The cross-device multi-application service performance evaluation method also includes a service graph self-learning process based on service identifiers: the initiating end sends a service traversal message, and for each application traversed, the association between the application and the service is recorded, and the mapping relationship between the application and the service is recorded as a tag and added to the message payload. The receiving end uploads the message to the reporting and analysis module.

[0050] Preferably, the business graph is recorded by collecting business information in real time, and a classifier is used to map the business graph to a known business graph, thereby providing classification results and similarity analysis. The computational load and time required to calculate the network edge configuration for each business graph are relatively large. By analyzing the similarity between the new task graph and the existing task graph, the near-optimal parameters can be found in a short time, thereby improving the efficiency of subsequent resource scheduling.

[0051] Step 30: Obtain the default set of application layer deterministic parameters and monitoring results, and statistically analyze the deterministic evaluation values ​​of multiple applications in the sub-chain of the business graph. The deterministic evaluation value of each application is the comparison value of the default set of application layer deterministic parameters and monitoring results of each application.

[0052] Deterministic networking refers to the ability to provide deterministic service guarantees for services carried within a network domain. Deterministic service refers to a service whose performance indicators have defined boundaries after being processed by multiple applications and transmitted over multiple networks (for example, deterministic latency, if expected to be 10ms and jitter 1ms, then the total service latency should be controlled between 9.5ms and 10.5ms). A service can have multiple deterministic indicators. The deterministic evaluation value takes all indicators as input and outputs a unique quantified value, used to assess the degree of conformity between the actual service quality and application-layer determinism.

[0053] In one embodiment of this application, a monitoring module is used to monitor deterministic applications from multiple levels simultaneously. Specifically, this includes: enabling start and end time monitoring functions for application-layer deterministic applications to monitor the operation process of application-layer deterministic applications; enabling a monitoring agent to monitor the usage of relevant device resources, including monitoring of deterministic applications and ordinary applications; when the monitoring agent identifies an anomaly in the application, it sends the operation information corresponding to the anomaly to the server; and the application should periodically report application-layer deterministic related data.

[0054] Regarding the method for application-layer deterministic evaluation, in terms of overall metrics, we obtain application-layer deterministic parameters that are self-describing to the application, including but not limited to latency, jitter, packet loss rate, reliability, bandwidth, number of computational units, priority, chain length of the service graph, degree of each node, relaxation coefficient, etc.

[0055] The chain length of a business graph refers to the number of all nodes in the business graph. When the business graph is a non-chained structure, the chain length is the longest sub-chain of the business graph.

[0056] The degree of each node is the number of predecessors and successors in remote communications associated with that node, and is divided into node in-degree and out-degree.

[0057] The relaxation coefficient refers to the space within which various indicators of a system can be flexibly varied.

[0058] In step 30, for each application node, parameters such as application type, business data payload, and business parallelization coefficient are evaluated. The application type is labeled according to typical classifications, and the business parallelization coefficient reflects the number of input data differentials that can be supported when parallel computing methods are used.

[0059] The deterministic evaluation values ​​of multiple applications in the sub-chains of the business graph are statistically analyzed. For example, this solution will obtain the default parameter set of application-layer determinism during application registration, where the i-th parameter is defined by P. default (i) Expression. The monitoring results of the i-th parameter in application a are expressed as... Express. The representative parameter represents the influence coefficient of the determination row evaluation. Therefore, the deterministic evaluation value of the business is:

[0060]

[0061] In other words, the closer the various parameters of the business are to the default parameters submitted by the application, the greater the certainty.

[0062] More preferably, for each application-layer deterministic indicator, a Sketch test structure is established. Each Sketch matrix contains the same type of parameters from multiple business processes, and each Sketch employs an independent statistical algorithm. The deterministic satisfaction is evaluated using an independent Sketch at each node, and then the results are summarized and analyzed according to the business graph structure.

[0063] Figure 3 This application describes a cross-device application monitoring architecture, including embodiments of the system described in this application. The application data acquisition architecture consists of an application-layer deterministic detection module and a monitoring agent.

[0064] This application proposes a cross-device, multi-application business performance evaluation system for implementing the method described in the first aspect of this application, including an application-layer deterministic monitoring module connected to the monitoring agent of the device.

[0065] The application-layer deterministic detection module collects and analyzes data reported by various applications, and establishes associations between the corresponding applications to form a real-time business graph. This graph reflects the dynamic relationships between specific applications under the same business.

[0066] In response to the device running the service, the application layer deterministic monitoring module obtains the communication path and communication timing between multiple applications of the same service through the device monitoring interface, and generates a service graph with applications as nodes connected according to the communication path and timing.

[0067] The application layer deterministic monitoring module is also used to: obtain the default set of application layer deterministic parameters and monitoring results, and to statistically analyze the deterministic evaluation values ​​of multiple applications in the sub-chain of the business graph.

[0068] Preferably, the cross-device, multi-application service performance evaluation system further includes the monitoring agents. Multiple monitoring agents are respectively installed on terminal devices, edge devices, and / or network controllers.

[0069] In any device, the monitoring agent is used for at least one of the following functions: during the data receiving phase, collecting the timestamp of the received data and / or the data volume monitoring results; during the application execution phase, collecting the deterministic parameter monitoring results during business execution; and during the data sending phase, collecting the timestamp of the sent data and / or the data volume monitoring results.

[0070] For example, the monitoring agent is installed on terminal devices, edge computing devices, and network controllers. Since the monitoring application does not actively report metrics, the application-layer deterministic detection module obtains various monitoring data by accessing the monitoring agent. The monitoring agent primarily performs five types of monitoring functions: 1. Business monitoring: obtaining the real-time execution status of business processes through open interfaces and monitoring business execution patterns. 2. Application monitoring: monitoring the normal and abnormal behavior of applications within the operating system, including file reading and writing, resource usage, etc. 3. Operating system business queue monitoring: obtaining parameters such as the scheduling priority and expected completion time of business processes on the current device through the operating system interface. 4. Operating system resource monitoring: obtaining real-time values ​​of the computing, storage, and network resources currently used by the business processes. 5. Data packet monitoring for the device's network interface card: analyzing the address, port number, and service number of data packets in the business processes and statistically analyzing the data transmission rate.

[0071] The application-layer deterministic detection module combines data obtained from the monitoring agent to update the business graph of each service in real time, and performs deterministic assessments by combining timestamps, operations, and other information proactively reported by the applications. The monitoring agent mainly monitors basic application information and status, thereby comprehensively acquiring relevant data from various dimensions. Since the services involve many applications and the amount of uploaded data is large, deterministic assessments can be performed by combining historical data and machine learning algorithms to provide the service's classification and level of determinism.

[0072] Figure 4 This is a schematic diagram illustrating the application information monitoring structure. The primary data source for this solution is passive application information monitoring. A monitoring agent is installed on each device. After the previous application transmits data to the currently executing application, the monitoring agent performs three stages of data collection, as detailed below. Figure 3 As shown.

[0073] First, during the data receiving phase, the monitoring results such as the timestamp, data volume, and throughput of the received data are sent to the real-time business graph. Based on these indicators, the certainty and service quality of the corresponding links are described.

[0074] Secondly, during the formal execution phase of the application, information such as key operations, operation objects, and operation logs of deterministic business at the application layer will be monitored, and relevant information will be updated to the real-time business graph.

[0075] Third, in the result sending stage after the application processing is completed, different result data is generated than that received, and key parameters such as the timestamp, data volume, and throughput of the sent result data are reported.

[0076] The above process acquires full-process monitoring information for each application, supporting the application-layer deterministic monitoring module in analyzing the monitoring results for each application.

[0077] Figure 5 This is a schematic diagram of the deterministic evaluation based on the Sketch group.

[0078] Sketch is a highly efficient storage structure that primarily uses a series of counter matrices to store statistical results of various churn data, as detailed in the reference (Cormode & Muthukrishnan, 2004). This solution establishes a Sketch test structure for each application-layer deterministic metric of the business. When n businesses are running, each business can map its deterministic parameters to the corresponding Sketch matrix, thus saving and querying relevant parameters with non-linear storage overhead. Storing each metric separately maximizes parallel access to statistical results. Each monitoring message includes an application ID, business ID, deterministic parameter, and timestamp. After obtaining the monitoring value of a business's deterministic parameter, it is hashed using a hash function and statistically analyzed in the sketch. The Sketch structure can reduce data collisions between different businesses by adjusting the hash function and the number of counters. Each Sketch employs an independent statistical algorithm, such as maximum, mean, cumulative, minimum, etc., allowing for appropriate aggregation methods for different parameters. Finally, during statistical analysis, queries can be performed based on the application ID and business ID. The statistical results can be used for business deterministic assessment, business type classification, and model training.

[0079] Each business involves multiple applications, and monitoring the various deterministic metrics of all business nodes incurs significant storage overhead. Therefore, a data statistics structure with lower storage requirements, called Sketch, is adopted. This invention does not specify the size of the detection data and registers; various solutions have been proposed in related research. The number of Sketches is equivalent to the number of deterministic parameters of the business. Figure 6This application describes the process of implementing application-layer deterministic monitoring using its method and system. It involves evaluating application-layer deterministic parameters based on the entire control variable. The main purpose of this solution is to develop a multi-application-oriented monitoring method, enabling the combined active and passive monitoring of various managed devices. The relevant data is then categorized and summarized in sketch format. Finally, based on the combination calculations and weights of each sketch, a deterministic evaluation result is generated, and each parameter can be queried. The specific monitoring process is as follows: Figure 5 As shown, the foundation for application monitoring is registering deterministic applications at the application layer with the monitoring module. Subsequently, the monitoring module issues monitoring requests to terminal devices based on application and device type, then to the network controller, and finally to the edge computing devices. Based on feedback from various devices, an application relationship graph is constructed, and storage is designed using a Sketch structure and a database, based on existing business requirements. Throughout the system's operation, real-time collected data is updated to counters in the Sketch structure, while real-time status information and resource information of monitored devices are stored in the database. Finally, the status and data of cloud applications are actively queried at fixed intervals.

[0080] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0081] Therefore, this application also proposes a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the methods described in any embodiment of this application.

[0082] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0083] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0084] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0085] Furthermore, this application also proposes an electronic device (or computing device) including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in any embodiment of this application.

[0086] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-persistent storage in computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media. Computer-readable media includes both permanent and non-persistent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information that can be accessed by the computing device. As defined in this article, computer-readable media do not include transient media, such as modulated data signals and carrier waves.

[0087] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0088] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A cross-device, multi-application business performance evaluation method for the Industrial Internet, characterized in that, Includes the following steps: In response to services that span multiple applications, where the service involves applications on multiple devices running and there are calling relationships between the multiple applications, this refers to a service that is responsive to multiple applications. Through the device monitoring interface, the communication paths and timing between multiple applications of the same service are obtained, and a service graph is generated with applications as nodes and connected according to the communication paths and timing. Data collection is performed in three phases through monitoring agents installed on various devices: In the data receiving phase, the timestamps of the received data and the data volume monitoring results are collected; During the application execution phase, collect the monitoring results of deterministic parameters during business execution; During the data transmission phase, the timestamps of the transmitted data and the data volume monitoring results are collected. Obtain the default set of application layer deterministic parameters and monitoring results, and statistically analyze the deterministic evaluation values ​​of multiple applications in the sub-chain of the business graph, wherein the deterministic evaluation value of each application is a comparison value of the default set of application layer deterministic parameters and monitoring results of each application. Where the i-th parameter is P default The expression, i-th parameter, in the monitoring results of application a, is... Express; If the representative parameter represents the influence coefficient in the deterministic assessment, then the deterministic assessment value of the business is: H Establish a Sketch test structure. Each Sketch matrix contains the same type of parameters for multiple businesses. Each business involves multiple applications. Determinism is evaluated using an independent Sketch at each node, and then the results are summarized according to the business graph structure.

2. The cross-device, multi-application service performance evaluation method as described in claim 1, characterized in that, The identifier of the service is a unique network identifier used to determine that the applications belong to the same service.

3. The cross-device multi-application service performance evaluation method as described in claim 1, characterized in that, It also includes a business graph self-learning process based on business identifiers: The business initiator sends a business traversal message. For each application it passes through, it records the association between the application and the business, records the mapping relationship between the application and the business as a tag and adds it to the message payload, and then uploads the message at the receiving end.

4. The cross-device multi-application service performance evaluation method as described in claim 1, characterized in that, The deterministic evaluation values ​​of multiple applications in the sub-chains of the business graph are statistically analyzed, including one or more of the following parameters: Subchain latency, jitter, packet loss rate, reliability, bandwidth, number of computing units, and priority; Chain length, node degree, relaxation factor; The node's application type, business data payload, and business parallelization coefficient.

5. The cross-device multi-application service performance evaluation method as described in claim 1, characterized in that, By collecting business information in real time to record a business graph, a classifier is used to map the business graph onto a known business graph, thereby providing a classification result.

6. The cross-device multi-application service performance evaluation method as described in claim 1, characterized in that, Each Sketch uses an independent statistical algorithm.

7. A cross-device, multi-application service performance evaluation system, used to implement the method described in any one of claims 1 to 6, characterized in that, Includes an application-layer deterministic monitoring module that connects to the device's monitoring agent; In response to the device running the service, the application layer deterministic monitoring module obtains the communication path and communication timing between multiple applications of the same service through the device monitoring interface, and generates a service graph with applications as nodes connected according to the communication path and timing. The application layer deterministic monitoring module is also used to: obtain the default set of application layer deterministic parameters and monitoring results, and to statistically analyze the deterministic evaluation values ​​of multiple applications in the sub-chain of the business graph.

8. The cross-device, multi-application service performance evaluation system as described in claim 7, characterized in that, It also includes the monitoring agent; Multiple monitoring agents are respectively installed on terminal devices, edge devices, and / or network controllers; In any device, the monitoring agent is used for at least one of the following functions: during the data receiving phase, collecting the timestamp of the received data and / or the data volume monitoring results; During the application execution phase, collect the monitoring results of deterministic parameters during business execution; During the data transmission phase, collect the timestamps of the transmitted data and / or the data volume monitoring results.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-8.

10. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-8.

Images