Application Processing Method, Apparatus, Device, and Medium
By extracting target data of target features from real-time data streams on the target application server, and analyzing the performance of target applications in real-time, the problems of link extension and resource consumption in existing streaming calculation methods are solved, and efficient real-time performance analysis is achieved.
Patent Information
- Application Number
- CN201910709067.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-08-01
- Publication Date
- 2025-05-27
- Estimated Expiration
- 2039-08-01
AI Technical Summary
In the existing streaming calculation method, the connection of the third-party server side causes the data processing link to be lengthened, and the process of extracting target metrics from the log consumes additional storage resources and requires a third-party system to be built.
During the execution of the target application, a real-time data stream is determined and the target data of the target characteristics is extracted from it to analyze the performance of the target application in real time. This method does not need to rely on a third-party server and directly performs data processing on the application server.
It reduces the length of data processing links and the consumption of computing resources on third-party servers, reduces the cost of third-party system construction, and achieves real-time performance analysis of target applications.
Smart Images

Figure CN110427293B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to the technical field of data processing, and in particular, to an application processing method, apparatus, device, and medium. Background Art
[0002] With the development of big data technology, large-scale data processing technology has become a standard for enterprises. The long time span of data calculation results is no longer appropriate, and the streaming computing technology with lower latency has become increasingly important. Streaming computing is a system that uses limited computing resources to achieve endless data calculation.
[0003] In the prior art, the streaming computing method is as follows: during the process that the application server responds to the client request and executes the target application, the third-party server obtains the execution log of the target application from the application server, extracts the target metrics to be analyzed from the log, performs data processing of the set service on the execution of the target application according to the target metrics, and feeds back the processing result to the application server.
[0004] However, the above method has the following defects: the connection of the third-party server leads to an increase in the data processing link length; the process of the third-party server extracting the target metrics to be analyzed from the log consumes additional storage resources and requires building a data processing system for the third-party server. Summary of the Invention
[0005] Embodiments of the present invention provide an application processing method, apparatus, device, and medium to reduce the data processing link length, the consumption of third-party server computing resources, and the third-party system construction cost.
[0006] In a first aspect, an embodiment of the present invention provides an application processing method, which includes:
[0007] During the execution of the target application, determine the real-time data stream generated by the target application;
[0008] Extract the target data belonging to the target feature from the real-time data stream;
[0009] Determine the performance of the target application according to the extracted target data.
[0010] In a second aspect, an embodiment of the present invention further provides an application processing apparatus, which includes:
[0011] A data stream determination module, configured to determine the real-time data stream generated by the target application during the execution of the target application;
[0012] A target data extraction module, configured to extract the target data belonging to the target feature from the real-time data stream;
[0013] A performance determination module, configured to determine the performance of the target application according to the extracted target data.
[0014] In a third aspect, an embodiment of the present invention further provides a device, including:
[0015] One or more processors;
[0016] A storage device, configured to store one or more programs,
[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the application processing method according to any one of the embodiments of the present invention.
[0018] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the application processing method according to any one of the embodiments of the present invention is implemented.
[0019] In the embodiment of the present invention, target data belonging to target features is extracted from the real-time data stream; the performance of the target application is determined according to the extracted target data, so as to realize real-time analysis of the performance of the target application. Since the above data processing process does not require an application on a third-party server, the consumption of third-party computing resources and the cost of building a third-party system are reduced.
[0020] In addition, since there is no need to transmit data to a third-party server, the length of the data processing link is also reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 It is a flowchart of an application processing method provided in Embodiment 1 of the present invention;
[0022] Figure 2 It is a flowchart of an application processing method provided in Embodiment 2 of the present invention;
[0023] Figure 3 It is an execution schematic diagram of an application processing method provided in Embodiment 3 of the present invention;
[0024] Figure 4 It is a schematic diagram of the display effect of an analysis result provided in Embodiment 3 of the present invention;
[0025] Figure 5 It is a schematic structural diagram of an application processing device provided in Embodiment 4 of the present invention;
[0026] Figure 6 It is a schematic structural diagram of a device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0027] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. In addition, it should be noted that for the sake of description, only parts related to the present invention rather than all structures are shown in the drawings.
[0028] Embodiment 1
[0029] Figure 1 The flowchart of an application processing method provided for Embodiment 1 of the present invention is shown. This embodiment is applicable to the situation of performing real-time performance analysis on a target application based on the real-time data stream generated by the target application. This method can be executed by an application processing device. The device can be implemented in software and / or hardware. Optionally, the device can be configured on the application server or the client. Refer to Figure 1 , the application processing method provided in this embodiment includes:
[0030] S110. During the execution of the target application, determine the real-time data stream generated by the target application.
[0031] Among them, the target application is the application to be determined for performance, and this application can be an application with any logic, and this embodiment does not impose any restrictions on this.
[0032] The real-time data stream generated by the target application is the data stream that is generated in real time by each application logic in the target application during the execution of the target application.
[0033] Specifically, during the execution of the target application, determining the real-time data stream generated by the target application includes:
[0034] During the execution of the target application, determine the real-time data streams generated by each computing node associated with the target application, and use the real-time data streams generated by each computing node associated with the target application as the real-time data stream generated by the target application.
[0035] Among them, the target application includes at least two logical units (Processors), and the at least two logical units form a directed acyclic graph (DAG, topological graph) describing streaming computing. Each logical unit includes at least one concurrent instance (instance), and each instance is executed by an entity computing node. Therefore, the target application is associated with at least one computing node.
[0036] Optionally, during the execution of the target application, determining the real-time data stream generated by the target application includes:
[0037] During the execution of the target application, obtain the real-time output data of each application logic of the target application, and use the obtained real-time output data of each application logic as the real-time data stream generated by the target application.
[0038] S120. Extract target data belonging to the target feature from the real-time data stream.
[0039] Among them, the target feature refers to the feature required for determining the target performance.
[0040] The target performance refers to the performance to be detected of the target application.
[0041] The target data is the data belonging to the target feature in the real-time data stream.
[0042] Specifically, before extracting the target data belonging to the target feature from the real-time data stream, the method further includes:
[0043] Obtain the target performance to be detected of the target application;
[0044] According to the mapping relationship between the candidate performance and the candidate feature, determine the target feature associated with the target performance.
[0045] According to the mapping relationship between the candidate performance and the candidate feature, determining the target feature associated with the target performance includes:
[0046] Match the target performance with the candidate performance, and use the candidate feature associated with the candidate performance that matches as the target feature.
[0047] Specifically, extracting the target data belonging to the target feature from the real-time data stream includes:
[0048] Determine the data feature of the real-time data in the real-time data stream, and match the determined data feature with the target feature;
[0049] Use the real-time data that matches as the target data.
[0050] S130. Determine the performance of the target application according to the extracted target data.
[0051] Optionally, based on any performance detection method, use the extracted target data to determine the performance of the target application.
[0052] The technical solution of the embodiment of the present invention realizes real-time analysis of the performance of the target application by extracting target data belonging to the target feature from the real-time data stream; and determining the performance of the target application according to the extracted target data. Since the above data processing process does not require an application to a third-party server, it reduces the consumption of third-party computing resources and the cost of building a third-party system.
[0053] In addition, since there is no need to transmit data to a third-party server, the length of the data processing link is also reduced.
[0054] Embodiment 2
[0055] Figure 2 is a flowchart of an application processing method provided by Embodiment 2 of the present invention. This embodiment is an alternative solution proposed on the basis of the above embodiment. Refer to Figure 2 , the application processing method provided by this embodiment includes:
[0056] S210. During the execution of the target application, determine the real-time data stream generated by the target application.
[0057] S220. Extract the target data belonging to the target feature from the real-time data stream.
[0058] S230. If it is determined that a computing node is congested according to the target data of any computing node extracted, determine that the logic unit to which the computing node belongs is congested.
[0059] Wherein, any of the above computing nodes is associated with the target application.
[0060] Specifically, determining whether the computing node is congested according to the target data of any computing node extracted includes:
[0061] Determine the output data of the computing node according to the target data of any computing node extracted, and when it is detected that a downstream computing node reads any output data of the computing node, delete the output data;
[0062] Determine whether the computing node is congested according to the remaining output data quantity of the computing node.
[0063] Wherein, the remaining output data of the computing node refers to the data output by the computing node but not flowing to the downstream node.
[0064] Specifically, determining whether the computing node is congested according to the remaining output data quantity of the computing node includes:
[0065] If the remaining output data quantity of the computing node is greater than the set quantity threshold, determine that the computing node is congested; or
[0066] Determine the ratio of the data volume of the output data stored in the result output queue associated with the computing node to the total storage data volume of the result output queue; wherein, the result output queue associated with the computing node is used to cache the output data of the computing node for the downstream computing node to read the output data of the computing node from the result output queue and delete the read output data from the result output queue;
[0067] If the ratio is greater than the set ratio threshold, determine that the computing node is congested.
[0068] S240. Take the topological structure between congestion logic units with a connection relationship as the congestion topological structure.
[0069] Among them, the congestion logic unit refers to a logic unit with congestion. This logic unit belongs to the set of logic units included in the target application, and the set of logic units constitutes a directed acyclic graph (DAG, topological graph) describing stream computing.
[0070] S250. Take the downstream node connected to the congestion node in the most downstream congestion logic unit in the congestion topological structure as the abnormal node.
[0071] Among them, the abnormal node refers to a computing node with abnormalities.
[0072] Because the abnormal node will reduce the computing speed or stop the calculation, it will cause congestion of the upstream computing nodes connected to the abnormal node. Therefore, by detecting the downstream node connected to the congestion node in the most downstream congestion logic unit in the congestion topological structure, the detection of the abnormal node can be realized.
[0073] The technical solution implemented in the present invention determines the congested computing node and the congestion logic unit according to the target data of any computing node extracted; determines the congestion topological structure according to the congestion logic unit; determines the abnormal node according to the congested computing node and the congestion topological structure, so as to realize the real-time detection of the abnormal computing node in the target application, and further solve the problem that in case of an urgent online problem, it is impossible to quickly and conveniently obtain the reason for abnormal exit and the abnormal node causing the full-link backpressure of stream computing.
[0074] For the convenience of viewing the abnormal node information, after taking the downstream node connected to the congestion node in the most downstream congestion logic unit in the congestion topological structure as the abnormal node, the method further includes:
[0075] Display at least one of the abnormal node, the logic unit to which the abnormal node belongs, the congestion topological structure, and the target data of the abnormal node for the user to view.
[0076] Embodiment III
[0077] Figure 3 It is a schematic execution diagram of an application processing method provided by Embodiment III of the present invention. This embodiment is an alternative solution proposed on the basis of the above embodiment. Refer to Figure 3 , the application processing method provided in this embodiment includes:
[0078] During the execution of the target application, the computing nodes associated with the target application will send the output data with target characteristics to the application management unit, and the application management unit will converge and manage the received data;
[0079] After detecting an event for determining the target performance of a target application, the performance analysis unit obtains the target data that has been converged and managed from the application management unit, where the target data is the output data with target characteristics;
[0080] The performance analysis unit analyzes the target performance of the target application by using the obtained target data, and displays the analysis result on the user interface for the user to view.
[0081] Both the application management unit and the performance analysis unit are set at the execution end of the target application.
[0082] Therefore, by converging and managing the received data by the application management unit, the convergence and management of the output data of the streaming computing node without relying on external components are realized.
[0083] Specifically, after detecting an event for determining the target performance of a target application, the performance analysis unit obtains the target data that has been converged and managed from the application management unit, including:
[0084] After the performance analysis unit obtains the data stream congestion threshold input by the user, it requests the target metric data (i.e., the target data) from the application management unit, and obtains the target metric data returned by the application management unit in response to the request.
[0085] Correspondingly, using the obtained target metric data to locate abnormal nodes in the target application, including:
[0086] According to the target metric data of any computing node obtained, determine the data volume of the output data stored in the result output queue associated with the computing node;
[0087] The result output queue associated with the computing node is used to cache the output data of the computing node for the downstream computing node to read the output data from the result output queue and delete the read output data from the result output queue;
[0088] If the data volume of the stored output data is greater than the data stream congestion threshold input by the user, determine that the computing node is congested;
[0089] Take the topological structure between the congested logic units with a connection relationship as the congestion topological structure;
[0090] Take the downstream node connected to the congested node in the most downstream congested logic unit in the congestion topological structure as the abnormal node;
[0091] Display at least one of the abnormal node, the logic unit to which the abnormal node belongs, the congestion topological structure, and the target metric data of the abnormal node.
[0092] See Figure 4 The specific display method is described as follows:
[0093] In the topology graph composed of the logic units of the target application, whether there are abnormal computing nodes is characterized by the color of the computing nodes (red indicates abnormal), and whether the data stream is congested is represented by the color of the directed edges between the logic units.
[0094] Through the data subscription relationship of the topology graph, the position of the logic unit causing the backpressure (i.e., congestion) phenomenon in the topology of the target application can be clearly seen. At the same time, the abnormal instances (instances calculated by abnormal nodes) in the backpressure logic unit will also be listed in detail.
[0095] At the same time, the target metric data of all logic units included in the target application is displayed to understand the target metric data of each logic unit in the current streaming computing job. And the abnormal data information is marked in red, providing intuitive, powerful and fast problem positioning information for locating the abnormal cause of the logic unit causing backpressure in the topology.
[0096] Such as Figure 4 As shown, the logic unit causing the backpressure phenomenon (i.e., logic unit 5) is marked in red in the topology graph. Due to the existence of abnormal instances in logic unit 5, all its upstream logic units in the topological relationship show a backpressure state (upstream data stream congestion), so the directed edges between the logic units upstream of logic unit 5 are marked red.
[0097] Since there is no upstream and downstream dependency between logic unit 4 and logic unit 5, logic unit 4 is not affected by the abnormality of logic unit 5, and the directed edge from logic unit 1 to logic unit 4 is not marked red.
[0098] At the same time, the abnormal metric values x and y existing in logic unit 5 are marked in the table of metric aggregation and marked in red, and the serial numbers (0, 1, 3) of the instances with abnormalities in logic unit 5 are displayed in the table of abnormal instances.
[0099] Through the above display method, the performance analysis results are graphically displayed to the user.
[0100] Users can also select a reasonable data congestion threshold according to business characteristics to implement customized positioning services.
[0101] In the technical solution of the embodiment of the present invention, by setting both the application management unit and the performance analysis unit at the execution end of the target application, an integrated design of metric reporting, aggregation, and access is realized, without relying on an additional system.
[0102] By using the obtained target metric data to locate abnormal nodes in the target application, real-time problem location is achieved.
[0103] By displaying the abnormal nodes, the logical units to which the abnormal nodes belong, the congestion topology structure and the target indicator data of the abnormal nodes, an interactive design close to user usage habits is achieved, which facilitates users to directly locate abnormal nodes and display abnormal exit problems.
[0104] This embodiment simply and conveniently opens up the streaming computing indicator reporting, aggregation, and access links, is transparent to the business, and does not consume additional resources.
[0105] It should be noted that, after learning from the technical teachings of this embodiment, those skilled in the art are motivated to combine any of the implementation methods described in the above embodiments to reduce the link length of data processing, the consumption of computing resources on the third-party server side, and the cost of setting up the third-party system.
[0106] Embodiment 4
[0107] Figure 5 Schematic diagram of the structure of an application processing device provided by Embodiment 4 of the present invention. Figure 5 The application processing device provided in this embodiment includes: a data flow determination module 10, a target data extraction module 20 and a performance determination module 30.
[0108] The data flow determination module 10 is used to determine the real-time data flow generated by the target application during the execution of the target application;
[0109] A target data extraction module 20, for extracting target data belonging to target features from the real-time data stream;
[0110] The performance determination module 30 is used to determine the performance of the target application according to the extracted target data.
[0111] The technical solution of the embodiment of the present invention extracts target data belonging to the target feature from the real-time data stream; determines the performance of the target application based on the extracted target data, thereby realizing real-time performance analysis of the target application. Because the above data processing process does not require the application of the third-party server, the consumption of third-party computing resources and the cost of building a third-party system are reduced.
[0112] In addition, since there is no need to transmit data to a third-party server, the length of the data processing link is also reduced.
[0113] Furthermore, the device also includes: a target performance acquisition module and a target feature determination module.
[0114] Wherein, the target performance acquisition module is used to acquire the target performance to be detected of the target application before extracting the target data belonging to the target feature from the real-time data stream;
[0115] A target feature determination module, configured to determine a target feature associated with the target performance according to a mapping relationship between candidate performances and candidate features.
[0116] Further, the performance determination module includes: a congestion node determination unit, a congestion topology determination unit, and an abnormal node determination unit.
[0117] Among them, the congestion node determination unit is configured to determine that the logic unit to which the computing node belongs is congested if it is determined that the computing node is congested according to the target data of any computing node extracted.
[0118] The congestion topology determination unit is configured to use the topological structure between congested logic units with a connection relationship as the congestion topology structure.
[0119] The abnormal node determination unit is configured to use the downstream node connected to the congested node in the most downstream congested logic unit in the congestion topology structure as the abnormal node.
[0120] Further, the congestion node determination unit is specifically configured to:
[0121] Determine the output data of any computing node according to the target data of any computing node extracted, so as to delete the output data when it is detected that a downstream computing node reads any output data of the computing node;
[0122] Determine whether the computing node is congested according to the remaining number of output data of the computing node.
[0123] Further, the apparatus further includes: an abnormal display module.
[0124] Among them, the abnormal display module is configured to display at least one of the abnormal node, the logic unit to which the abnormal node belongs, the congestion topology structure, and the target data of the abnormal node after using the downstream node connected to the congested node in the most downstream congested logic unit in the congestion topology structure as the abnormal node.
[0125] Further, the performance determination module includes: a target data summarization unit and a performance determination unit.
[0126] Among them, the target data summarization unit is configured to summarize the extracted target data;
[0127] The performance determination unit is configured to determine the performance of the target application according to the summarized target data if a performance determination event of the target application is detected.
[0128] The application processing apparatus provided by the embodiments of the present invention can execute the application processing method provided by any embodiment of the present invention, and has corresponding function modules and beneficial effects for executing the method.
[0129] Example 5
[0130] Figure 6 FIG. is a schematic structural diagram of a device provided in Example 5 of the present invention. Figure 6 FIG. shows a block diagram of an exemplary device 12 suitable for use in implementing embodiments of the present invention. Figure 6 The shown device 12 is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present invention.
[0131] As Figure 6 shown, the device 12 is presented in the form of a general-purpose computing device. The components of the device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).
[0132] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the multiple bus structures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.
[0133] The device 12 typically includes a variety of computer system-readable media. These media can be any available media accessible by the device 12, including volatile and non-volatile media, removable and non-removable media.
[0134] The system memory 28 may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 34 may be used for reading and writing non-removable, non-volatile magnetic media ( Figure 6 not shown, typically referred to as a "hard disk drive"). Although Figure 6 not shown in, a disk drive for reading and writing removable non-volatile disks (such as "floppy disks") and an optical disk drive for reading and writing removable non-volatile optical disks (such as CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 through one or more data media interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.
[0135] A program / utilities 40 having a set (at least one) of program modules 42 can be stored, for example, in a memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods in the embodiments described in the present invention.
[0136] The device 12 can also communicate with one or more external devices 14 (such as a keyboard, a pointing device, a display 24, etc.), and can also communicate with one or more devices that enable a user to interact with the device 12, and / or communicate with any device that enables the device 12 to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication can be carried out through an input / output (I / O) interface 22. Moreover, the device 12 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the device 12 through a bus 18. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.
[0137] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the application processing method provided by the embodiments of the present invention.
[0138] Embodiment Six
[0139] Embodiment Six of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the application processing method described in any one of the embodiments of the present invention. The method includes:
[0140] During the execution of a target application, determine the real-time data stream generated by the target application;
[0141] Extract target data belonging to a target feature from the real-time data stream;
[0142] Determine the performance of the target application according to the extracted target data.
[0143] The computer storage medium of the embodiments of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.
[0144] The computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.
[0145] The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0146] The computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).
[0147] Note that the above is only the preferred embodiment of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, it may also include more other equivalent embodiments, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. An application processing method, characterized in that, the method includes: During the execution of the target application, determining the real-time data stream generated by the target application; Extracting target data belonging to the target feature from the real-time data stream; Determining the performance of the target application according to the extracted target data; Wherein, the determining the performance of the target application according to the extracted target data includes: If it is determined that a computing node is congested according to the target data of any computing node extracted, it is determined that the logic unit to which the computing node belongs is congested; Taking the topological structure between congested logic units with a connection relationship as the congested topological structure; Taking the downstream node connected to the congested node in the most downstream congested logic unit in the congested topological structure as the abnormal node; the performance of the target application includes at least one of the abnormal node of the target application, the logic unit to which the abnormal node belongs, the congested topological structure, and the target data of the abnormal node.
2. The method according to claim 1, characterized in that, Before extracting the target data belonging to the target feature from the real-time data stream, the method further includes: Obtaining the target performance to be detected of the target application; Determining the target feature associated with the target performance according to the mapping relationship between the candidate performance and the candidate feature.
3. The method according to claim 1, characterized in that, The determining whether a computing node is congested according to the target data of any computing node extracted includes: Determining the output data of any computing node extracted according to the target data of any computing node extracted, and deleting the output data when it is detected that a downstream computing node reads any output data of the computing node; Determining whether the computing node is congested according to the remaining output data quantity of the computing node.
4. The method according to claim 1, characterized in that, After taking the downstream node connected to the congested node in the most downstream congested logic unit in the congested topological structure as the abnormal node, the method further includes: Displaying at least one of the abnormal node, the logic unit to which the abnormal node belongs, the congested topological structure, and the target data of the abnormal node.
5. The method according to claim 1, characterized in that, The determining the performance of the target application according to the extracted target data includes: Summarizing the extracted target data; If a performance determination event of the target application is detected, determining the performance of the target application according to the summarized target data.
6. An application processing device, characterized in that, the device includes: A data stream determination module, configured to determine the real-time data stream generated by the target application during the execution of the target application; A target data extraction module, configured to extract target data belonging to the target feature from the real-time data stream; A performance determination module, configured to determine the performance of the target application according to the extracted target data; Wherein, the performance determination module includes: A congested node determination unit, configured to determine that the logic unit to which a computing node belongs is congested if it is determined that the computing node is congested according to the target data of any computing node extracted; A congestion topology determination unit, configured to use the topology structure between congestion logic units with a connection relationship as the congestion topology structure; An abnormal node determination unit, configured to use the downstream node connected to the congestion node in the most downstream congestion logic unit in the congestion topology structure as the abnormal node; the performance of the target application includes at least one of the abnormal node of the target application, the logic unit to which the abnormal node belongs, the congestion topology structure, and the target data of the abnormal node.
7. The apparatus according to claim 6, wherein, the apparatus further includes: A target performance acquisition module, configured to acquire the target performance to be detected of the target application before extracting the target data belonging to the target feature from the real-time data stream; A target feature determination module, configured to determine the target feature associated with the target performance according to the mapping relationship between the candidate performance and the candidate feature.
8. The apparatus according to claim 6, wherein, the congestion node determination unit is specifically configured to: Determine the output data of any computing node according to the extracted target data of the computing node, and delete the output data when it is detected that a downstream computing node reads any output data of the computing node; Determine whether the computing node is congested according to the remaining number of output data of the computing node.
9. The apparatus according to claim 6, wherein, the apparatus further includes: An abnormal display module, configured to display at least one of the abnormal node, the logic unit to which the abnormal node belongs, the congestion topology structure, and the target data of the abnormal node after using the downstream node connected to the congestion node in the most downstream congestion logic unit in the congestion topology structure as the abnormal node.
10. The apparatus according to claim 6, wherein, the performance determination module includes: A target data summarization unit, configured to summarize the extracted target data; A performance determination unit, configured to determine the performance of the target application according to the summarized target data if a performance determination event of the target application is detected.
11. A device, wherein, the device includes: One or more processors; A storage device, configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the application processing method according to any one of claims 1-5.
12. A computer-readable storage medium, on which a computer program is stored, wherein, When the program is executed by a processor, it implements the application processing method according to any one of claims 1-5.
Citation Information
Patent Citations
Data acquisition method and device, computer equipment and storage medium
CN109783533A