Image batch processing performance monitoring and bottleneck diagnosis method, program product and device

Abstract

The invention provides a Java Agent-based image batch processing performance monitoring and bottleneck diagnosis method, a program product and a Java Agent-based image batch processing performance monitoring and bottleneck diagnosis device. The method comprises the following steps: when an image processing service to be monitored is started, loading and initializing a Java Agent; byte code enhancement is carried out on a predetermined target class through the Java Agent, and a monitoring code is implanted in a method of the target class; when the image processing service runs, the monitoring code is executed to collect performance data executed by the method, a unique tracking identifier is generated for the same image batch processing request based on the tracking context, and cross-method and cross-thread performance data are associated through the tracking identifier; sending the collected performance data to a performance analysis server; and aggregating a complete calling link based on the tracking identifier by the performance analysis server, and carrying out association analysis on performance data of the calling link and corresponding environment data during operation of the Java virtual machine so as to identify a performance bottleneck of image batch processing and generate a diagnosis result.

Description

Technical Field

[0001] This invention relates to the field of image batch processing, and more specifically, to methods, programs, and devices for monitoring and diagnosing performance bottlenecks in image batch processing. Background Technology

[0002] With the development of digital imaging technology, batch image processing services are widely used in fields such as medical image analysis, security monitoring, industrial inspection, and internet content processing. These services typically require performing various processing operations on large amounts of image data, including decoding, format conversion, feature extraction, compression, storage, and network uploading. The processing flow is complex, computationally resource-intensive, and places high demands on system performance and stability. Therefore, effectively monitoring and analyzing the performance of batch image processing services is crucial for ensuring efficient system operation. Summary of the Invention

[0003] This invention provides a technical solution for performance monitoring and bottleneck diagnosis of batch image processing based on Java Agent.

[0004] According to one embodiment of the present invention, a method for monitoring and diagnosing performance bottlenecks in batch image processing based on a Java Agent is provided, comprising the following steps: When the image processing service to be monitored starts, the Java Agent is loaded and initialized; The Java Agent is used to enhance the bytecode of a predetermined target class, and monitoring code is injected into the methods of the target class. When the image processing service is running, the monitoring code is executed to collect performance data of the method execution, and generates a unique tracking identifier for the same batch processing request of image based on the tracking context. The tracking identifier is used to associate performance data across methods and threads. The collected performance data is sent to the performance analysis server; The performance analysis server aggregates the complete call chain based on the tracing identifier, and performs correlation analysis between the performance data of the call chain and the corresponding Java Virtual Machine (JVM) runtime environment data to identify performance bottlenecks in the batch image processing process and generate diagnostic results.

[0005] In some embodiments of the present invention, the method further includes: visualizing the diagnostic results and outputting alarm information when preset alarm conditions are met.

[0006] In some embodiments of the present invention, the Java Agent is encapsulated as a Jar file, which is deployed to the server running the image processing service; wherein, when the image processing service starts, the Java Agent is loaded during the startup or operation of the Java Virtual Machine corresponding to the image processing service.

[0007] In some embodiments of the present invention, the target class includes at least image processing related classes, file input / output related classes, and network communication related classes.

[0008] In some embodiments of the present invention, the bytecode enhancement of the predetermined target class includes: inserting first monitoring code at the entry point of the method of the target class to record the method identifier, start time, and tracking context; and inserting second monitoring code at all normal return points and exception throwing points of the method of the target class to record the method identifier, end time, and exception information.

[0009] In some embodiments of the present invention, the tracking context is bound to the currently executing thread through thread-local storage, and when the image processing flow involves cross-thread operations, the tracking identifier is passed to the child thread by encapsulating a task object.

[0010] In some embodiments of the present invention, the performance data includes at least: method identifier, start time, end time, time elapsed, exception information, and the tracking identifier.

[0011] In some embodiments of the present invention, the step of performing correlation analysis between the performance data of the call chain and the corresponding Java Virtual Machine runtime environment data includes: Statistical indicators of execution time based on time window calculation method; Identify slow methods that take longer than a threshold, and query the call chain of these slow methods to form hotspot paths; The hotspot paths are correlated with Java Virtual Machine runtime environment data to determine the causes of performance degradation.

[0012] According to another embodiment of the present invention, a computer program product is provided, which includes a computer program that, when executed by a processor, implements the steps of the method described in any embodiment of the present invention.

[0013] Furthermore, embodiments of the present invention also relate to a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any embodiment of the present invention.

[0014] Furthermore, embodiments of the present invention also relate to a computer device comprising: A memory on which computer programs are stored; A processor that, when executing the computer program, implements the steps of the method described in any embodiment of the present invention.

[0015] According to embodiments of the present invention, the following beneficial effects can be achieved by implementing the present invention.

[0016] First, it enables non-intrusive performance monitoring, significantly reducing deployment and maintenance costs.

[0017] This implementation method enhances the target application's bytecode through a Java Agent during Java Virtual Machine startup or runtime. No modifications to the business source code of the image batch processing service are required; monitoring capabilities can be deployed simply by configuring JVM startup parameters. This approach avoids the intrusion of traditional manual instrumentation into the business code structure, reduces the maintenance costs associated with frequent adjustments to monitoring code due to changes in business logic, and minimizes interference from monitoring logic on the business execution process, thereby improving the overall maintainability and stability of the system.

[0018] Secondly, it provides method-level fine-grained monitoring capabilities, enhancing the observability of operational performance.

[0019] This implementation method enhances the bytecode of the target class's methods, allowing it to delve into the Java Virtual Machine and monitor file input / output operations (I / O), image decoding and processing library calls, and the execution of business logic methods involved in batch image processing. It obtains method-level execution time, call counts, and exception information. Compared to monitoring methods that rely solely on system resource metrics, this method directly reflects the performance status of specific business operations, enabling performance bottlenecks to be precisely located at specific methods or processing stages, thereby improving the accuracy of performance analysis.

[0020] Furthermore, it enables end-to-end tracing and has strong correlation analysis capabilities.

[0021] The implementation method automatically embeds tracking context during operation, associating execution data of the same batch image processing request across different processing stages, components, and threads to construct a complete end-to-end call chain. Based on this call chain, the time consumption percentage of each processing stage can be calculated and critical paths can be identified, making cross-component and cross-thread performance issues readily apparent. This is particularly suitable for multi-step pipeline-style batch image processing scenarios, thereby significantly improving the efficiency of locating complex performance problems.

[0022] In addition, it supports automated bottleneck diagnosis and early warning, improving the efficiency of operation and maintenance and optimization decision-making.

[0023] By performing aggregation analysis, hotspot path analysis, and correlation analysis on the collected performance data on the server side, the system can automatically identify performance anomalies and bottlenecks during batch image processing. For example, it can pinpoint methods with abnormally high processing times or performance degradation factors related to virtual machine runtime. Combined with preset diagnostic rules and alarm conditions, the system can proactively output diagnostic results and alarm information when performance problems occur or worsen, thereby reducing reliance on manual comparison and experience-based judgment, and improving system operation and maintenance efficiency and the targeted nature of performance optimization.

[0024] Various aspects, features, advantages, etc., of the embodiments of the present invention will be specifically described below in conjunction with the accompanying drawings. These aspects, features, advantages, etc., will become clearer from the following detailed description in conjunction with the accompanying drawings. Attached Figure Description

[0025] Figure 1 This is a schematic flowchart illustrating the image batch processing performance monitoring and bottleneck diagnosis method based on Java Agent according to an embodiment of the present invention.

[0026] Figure 2 This is a schematic block diagram illustrating a computing device according to an embodiment of the present invention. Detailed Implementation

[0027] The exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the present invention can be implemented in various forms and is not limited to the specific embodiments described herein or shown in the accompanying drawings.

[0028] The terms “comprising,” “including,” and “having” are used herein to indicate the presence of certain features, steps, operations, elements, and / or components, but do not exclude the presence or addition of other features, steps, operations, elements, components, or combinations thereof. Unless explicitly defined by context, the terms “first,” “second,” and similar terms do not indicate any priority or order, but are used only to distinguish different elements in the description.

[0029] Reference Figure 1This is a schematic flowchart illustrating the performance monitoring and bottleneck diagnosis method for batch image processing based on a Java Agent according to an embodiment of the present invention. According to one embodiment of the present invention, the method includes the following steps: S100, when the image processing service to be monitored starts, a Java Agent (also known as a Java probe or Java agent) is loaded and initialized; S200, the Java Agent performs bytecode enhancement on a predetermined target class, injecting monitoring code into the methods of the target class; S300, when the image processing service is running, the monitoring code is executed to collect performance data of the method execution, and a unique tracing identifier is generated for the same batch image processing request based on the tracing context, and the performance data across methods and threads is associated through the tracing identifier; S400, the collected performance data is sent to a performance analysis server; S500, the performance analysis server aggregates a complete call chain based on the tracing identifier, and performs correlation analysis between the performance data of the call chain and the corresponding Java Virtual Machine (JVM) runtime environment data to identify performance bottlenecks in the batch image processing process and generate diagnostic results.

[0030] In some implementations, the method further includes: visualizing the diagnostic results and outputting alarm information when preset alarm conditions are met. For example, if the error rate of a certain method is >1%, an alarm notification is sent to the operations and development team through preset channels (email, WeChat Work, etc.).

[0031] In some implementations, the Java Agent is encapsulated as a Jar file, which is deployed to the server running the image processing service; simultaneously, the file path of the Jar file is added to the JVM startup parameters of the image processing service that needs to be monitored. Thus, in step S100, when the image processing service starts, the Java Agent is loaded during the startup or runtime of the Java Virtual Machine corresponding to the image processing service.

[0032] In some implementations, the target class includes at least image processing-related classes, file input / output-related classes, and network communication-related classes. In step S200, bytecode enhancement of the predetermined target class includes: inserting first monitoring code at the entry point of a method in the target class to record the method identifier (method ID), start time, and tracking context; and inserting second monitoring code at all normal return points and exception throwing points of the method in the target class to record the method identifier, end time, and exception information.

[0033] In some implementations, the tracking context is bound to the currently executing thread via thread-local storage, and when the image processing flow involves cross-thread operations, the tracking identifier is passed to the child thread by encapsulating a task object.

[0034] In some implementations, the performance data includes at least: method identifier, start time, end time, execution time, exception information, and the trace identifier (trace ID). In step S500, the correlation analysis between the performance data of the call chain and the corresponding Java Virtual Machine runtime environment data includes: calculating statistical indicators of method execution time based on time windows; identifying slow methods whose execution time exceeds a threshold, and querying the call chain of the slow methods to form hotspot paths; and performing correlation analysis between the hotspot paths and the Java Virtual Machine runtime environment data to determine the cause of performance degradation. The Java Virtual Machine runtime environment data includes garbage collection activity, disk I / O activity, and CPU wait activity.

[0035] To facilitate a better understanding of the present invention by those skilled in the art, specific examples are provided below to illustrate the present invention.

[0036] The image batch processing performance monitoring and bottleneck diagnosis method based on Java Agent of the present invention includes the following steps: (1) Agent Deployment and Initialization: Deploy the JAR package containing the probe functionality to the server. Add the file path to the JVM startup parameters of the image processing service to be monitored, for example: -javaagent: / path / to / agent.jar, and pass the configuration file via something like -Dagent.config=config.properties. Load and initialize the Agent when the JVM corresponding to the image processing service starts.

[0037] (2) Class Loading Interception and Bytecode Enhancement: The Agent listens for JVM class loading events. When a target class specified in the configuration file (e.g., all classes containing keywords such as "ImageProcess", "Decoder", "Uploader") is loaded, the Agent's ClassFileTransformer intervenes, using the bytecode framework to modify the target class. The modification logic is as follows: insert code at the beginning of the target method, calling MethodMonitor.start(methodId, TraceContext.get()); insert code at all exit points (return or throw) of the method, calling MethodMonitor.end(methodId, TraceContext.get(), throwable). Simultaneously, similar instrumentation can be performed on classes involving file operations (such as FileInputStream) and network operations (such as HttpURLConnection) to monitor I / O latency.

[0038] (3) Runtime Data Acquisition and Context Passing: During the operation of the image processing service, whenever a monitored method is called, the instrumentation code is automatically executed, recording the start timestamp. The MethodMonitor component temporarily stores the call information (method ID, start time, trace ID) in a thread-local variable. When the method ends, the execution time is calculated, and the complete method execution record (Span) is placed in the asynchronous queue. For new business requests (such as those identified through Servlet Filter or Spring Interceptor), the TraceContext generates a new TraceId and binds it to the current thread via ThreadLocal. If a new thread is created to execute a task during processing, the TraceId is passed to the child thread by encapsulating it in Runnable / Callable.

[0039] (4) Data reporting and aggregation: The reporter in the Agent periodically (e.g., every 10 seconds) or when the queue backlog reaches a threshold, sends the Span data in the queue in batches to the performance analysis server. After receiving the data, the performance analysis server stores the original Span data in the link database for tracking and querying, and performs real-time aggregation of the time consumption according to the method ID and time window, updating the metrics in the time series database.

[0040] (5) Bottleneck Analysis and Diagnosis: The diagnostic engine in the performance analysis server performs diagnostic tasks periodically (e.g., every minute), including: Query the time series database to find the top N methods with the largest year-on-year / month-on-month growth in average time or P99 time within the current period; By sampling and querying the call chains of these slow methods from the link database, analyzing their upstream and downstream calls, hotspot paths are formed; Correlate with JVM monitoring data to check for abnormal GC activity, CPU wait, or disk I / O during the time period in which the slow method occurred; Based on the above analysis, a diagnostic report is generated, for example: "During the period from 14:05 to 14:10, the average execution time of the PNGImageDecoder.decode() method increased from 50ms to 450ms, a year-on-year increase of 800%. Frequent Young GC was observed during this period, which may be related to the generally larger size of PNG images processed recently. It is recommended to check the image preprocessing stage or adjust the JVM young generation size." (6) Results visualization and alerts: Diagnostic reports and raw metrics are displayed in real time through the Web console. Users can view the overall health status of the application, method performance rankings, and call chain flame graphs. When a serious bottleneck is diagnosed or an alert rule is triggered (such as an error rate of >1% for a certain method), the system automatically sends an alert notification to the operations and development teams through preset channels (email, WeChat Work).

[0041] According to the above embodiments, through a Java Agent, bytecode instrumentation is dynamically performed on image processing frameworks (such as OpenCV calls and ImageIO operations), I / O operations (file read / write, network communication), and key business methods during target JVM startup or runtime, automatically collecting method-level performance metrics. Simultaneously, a complete processing chain is constructed by embedding a distributed tracing context. The collected data is aggregated locally and then sent to a performance analysis server for storage, analysis, and visualization, ultimately achieving automatic bottleneck diagnosis and early warning.

[0042] It should be understood that the steps, processes, and operations described in the above embodiments can be implemented by computer software programs. Therefore, the embodiments of the present invention also relate to a computer program product, which includes a computer program that, when executed by a processor, implements the steps, processes, or operations of the image batch processing performance monitoring and bottleneck diagnosis method based on Java Agent described in any embodiment of the present invention. In some embodiments, program code for executing the computer program of the embodiments of the present invention can be written in any combination of one or more programming languages. For example, the computer program can be written using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on a personal computing device, partially on a personal computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of remote computing devices, the remote computing device can be connected to the personal computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0043] It should also be understood that the computer program product may be stored on a computer-readable storage medium, such as hard disks, floppy disks, magnetic tapes, optical disks, solid-state drives, flash memory, etc.

[0044] Furthermore, it can be understood that the method of any embodiment of the present invention can be executed by a computer device. Accordingly, embodiments of the present invention also relate to a computer device comprising: a memory storing a computer program thereon; and a processor that, when executing the computer program, implements the steps (processing, operations, etc.) of the method described in any embodiment of the present disclosure.

[0045] Figure 2 An exemplary structure of a computer device is shown. Figure 2 As shown, the computer device 100 may include at least a processor 101 and a memory 102, wherein the memory 102 may be or include a storage medium storing a computer program (or computer-readable instructions). The processor 101 executes the computer program to perform some or all of the operations described in any embodiment of this disclosure, including but not limited to: S100, when the image processing service to be monitored starts, loading and initializing a Java Agent; S200, performing bytecode enhancement on a predetermined target class through the Java Agent, and injecting monitoring code into the methods of the target class; S300, when the image processing service is running, the monitoring code is executed to collect performance data of method execution, and generating a unique tracking identifier for the same batch processing request of images based on the tracking context, and associating cross-method and cross-thread performance data through the tracking identifier; S400, sending the collected performance data to a performance analysis server; S500, the performance analysis server aggregates a complete call chain based on the tracking identifier, and performs correlation analysis between the performance data of the call chain and the corresponding Java Virtual Machine (JVM) runtime environment data to identify performance bottlenecks in the batch processing of images and generate diagnostic results.

[0046] In some embodiments, the computer device 100 may further include a data storage device 103, a display 104, a speaker 105, and a communication module 106. The processor 101, memory 102, and data storage device 103 communicate with each other via a bus and interact with peripheral devices, such as the display 104 and communication module 106, via the bus and I / O module 107.

[0047] Specifically, the data storage device 103 can store application programs and their various configuration files and data. The memory 102 stores computer programs. The processor 101 executes the computer programs to perform various processes, operations, or steps according to any embodiment of this disclosure. The display 104 can be used to display various data, images, etc.

[0048] In optional embodiments, the communication module 106 may be omitted where communication with the other systems, devices, or equipment is not required. In some embodiments, the computer device 100 may further include a speaker 105 for voice output.

[0049] In some implementations, processor 101 may include any suitable semiconductor-based electronic processing unit, chip, microchip, or integrated circuit (IC). Memory 102 is a programmable memory, which may include any suitable electronic storage device configured to store instructions and be reprogrammable. For example, programmable memory may include an erasable programmable read-only memory (EPROM) device. Data storage device 103 may be a persistent storage device, which may include any suitable electronic storage device configured to retain stored information as power cycles. For example, data storage device 103 may include a hard disk drive, solid-state drive (SSD), flash memory drive, hybrid drive, etc., or any combination thereof.

[0050] Therefore, processor 101 can control the output of information on I / O module 107 by storing information in memory and / or executing programs / instructions stored in memory. For example, several aspects of the methods described herein can be executed by processor 101 according to programs / instructions stored in memory (e.g., memory 102 and / or data storage device 103).

[0051] Furthermore, the processor 101 can electronically communicate with the I / O module 107 and / or the communication module 106 to receive or send relevant instructions and information. The I / O module 107 may include any suitable mechanical or virtual user interface configured to enable a user to interact with the computer device 100 or to allow the user to perform one or more functions of the computer device 100 itself, such as a graphical user interface (GUI) on a screen or other display. In some instances, the user interface may include a voice interface capable of speech recognition, through which an operator can provide voice commands to the processor.

[0052] The communication module 106 may include any suitable devices and / or structures configured to facilitate information exchange between the computer device 100 and external electronic devices. The communication module 106 may include devices configured to send and / or receive wireless or wired information with other devices. For example, the communication module 106 may include an antenna, a transceiver, a connector for wired reception and / or transmission of data, a data exchange device, etc., or any combination thereof. The communication module 106 may also include ancillary components, such as filtering circuitry, encryption / decryption circuitry, and / or integrated circuit (IC) chips (e.g., Bluetooth® chips) for processing signals. In some embodiments, the communication module 106 may include a WiFi device configured to connect to a local wireless network.

[0053] In some implementations, the computer device 100 may include a smartphone, wearable computer, portable / mobile electronic device, tablet computer, smartwatch, personal digital assistant (PDA), personal computer (PC), desktop computer, laptop computer, server, etc. The computer device 100 may include or have installed one or more application programs (APPs), one of which is configured to execute the Java Agent-based image batch processing performance monitoring and bottleneck diagnosis method described herein.

[0054] Although not shown, it should be understood that the computer device 100 also includes a power supply component, which may include any suitable device and / or structure configured to provide an electrical interface between the computer device 100 and a power source. The power source may include any suitable source of electrical energy, such as a battery, socket, capacitor, fuel cell, etc., or any combination thereof. Additionally or optionally, the power source may be included within the power supply component. For example, a battery or battery pack may be included within the computer device 100. In some embodiments, the battery may be rechargeable, for example, charged via a cable or an interface provided by the power supply component. In some embodiments, the power supply component may share features with or repeat features of the communication module 106. For example, a USB or micro-USB cable connector may be included in the computer device 100, so that power supply or data communication is performed by the same component.

[0055] Those skilled in the art should understand that the above disclosure is merely illustrative of embodiments of this disclosure, and the scope of patent protection claimed in this application is not limited thereto. Various modifications, alterations, substitutions, and other changes can be made to the embodiments disclosed herein without departing from the spirit and essence of this disclosure, and such changes are within the scope covered by the claims of this application.

Claims

1. A method for performance monitoring and bottleneck diagnosis of batch image processing based on Java Agent, characterized in that, Includes the following steps: When the image processing service to be monitored starts, the Java Agent is loaded and initialized; The Java Agent is used to enhance the bytecode of a predetermined target class, and monitoring code is injected into the methods of the target class. When the image processing service is running, the monitoring code is executed to collect performance data of the method execution, and generates a unique tracking identifier for the same batch processing request of image based on the tracking context. The tracking identifier is used to associate performance data across methods and threads. The collected performance data is sent to the performance analysis server; The performance analysis server aggregates the complete call chain based on the tracing identifier, and performs correlation analysis between the performance data of the call chain and the corresponding Java Virtual Machine runtime environment data to identify performance bottlenecks in the batch image processing process and generate diagnostic results.

2. The method according to claim 1, characterized in that, Also includes: The diagnostic results are visualized, and alarm information is output when preset alarm conditions are met.

3. The method according to claim 2, characterized in that, The Java Agent is encapsulated as a Jar file, which is then deployed to the server running the image processing service. When the image processing service is started, the Java Agent is loaded during the startup or operation of the Java Virtual Machine corresponding to the image processing service.

4. The method according to claim 1, characterized in that, The target class includes at least image processing related classes, file input / output related classes, and network communication related classes.

5. The method according to claim 1, characterized in that, The bytecode enhancement of the predetermined target class includes: Insert first monitoring code at the entry point of the target class method to record the method identifier, start time, and tracking context. Insert second monitoring code at all normal return points and exception throwing points of the target class method to record the method identifier, end time, and exception information.

6. The method according to claim 5, characterized in that, The tracking context is bound to the currently executing thread through thread-local storage, and when the image processing flow involves cross-thread operations, the tracking identifier is passed to the child thread by encapsulating a task object.

7. The method according to claim 5, characterized in that, The performance data includes at least: method identifier, start time, end time, duration, exception information, and the tracking identifier.

8. The method according to claim 7, characterized in that, The correlation analysis between the performance data of the call chain and the corresponding Java Virtual Machine runtime environment data includes: Statistical indicators of execution time based on time window calculation method; Identify slow methods that take longer than a threshold, and query the call chain of these slow methods to form hotspot paths; The hotspot paths are correlated with Java Virtual Machine runtime environment data to determine the causes of performance degradation.

9. A computer program product comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 8.

11. A computer device, characterized in that, include: A memory on which computer programs are stored; A processor that, when executing the computer program, implements the steps of the method according to any one of claims 1 to 8.

Images