System performance data processing method and system, electronic equipment and readable storage medium

Abstract

The invention relates to a system performance data processing method and system, electronic equipment and a readable storage medium. The method comprises the following steps: receiving delay diagnosis information sent by a detection end; acquiring corresponding sampling data from a preset data buffer area according to the delay diagnosis information as target performance data, and writing the target performance data into a target file; wherein the preset data buffer area is used for storing sampling data of a plurality of operation tasks in the system. According to the scheme provided by the invention, the I / O load pressure of the system can be effectively reduced, the real-time response performance of the automatic driving system is ensured, the storage and transmission quantity of redundant data is reduced, the effectiveness of the performance analysis process of the system is improved, and the low-overhead operation of the system is maintained.

Description

Technical Field

[0001] This application relates to the technical field of system data analysis, and in particular, to a system performance data processing method, system, electronic device, and readable storage medium. Background Art

[0002] In the field of autonomous driving technology, an autonomous driving system generally realizes intelligent decision-making by integrating multiple types of algorithms such as perception, planning, and control. In order to identify the bottlenecks of the autonomous driving system, optimize the allocation of computing resources, and improve the overall efficiency, performance analysis tools are often used for system-level monitoring to achieve the performance profiling function of the autonomous driving system.

[0003] In the related art, when using traditional performance analysis tools to monitor the performance of an autonomous driving system, high-frequency data collection is required, and the generated performance events are written to the disk in real time, which easily leads to an increase in the I / O (input / output) load in the system, and further has an adverse impact on the real-time performance of the autonomous driving system. Summary of the Invention

[0004] To solve or partially solve the problems existing in the related art, this application provides a system performance data processing method, system, electronic device, and readable storage medium, which can effectively reduce the system I / O load pressure, ensure the real-time response performance of the autonomous driving system, reduce the storage and transmission volume of redundant data, improve the effectiveness of the system performance analysis process, and maintain the system running with low overhead.

[0005] The first aspect of this application provides a system performance data processing method, which is applied to a processing end for processing system performance data, and includes: Receiving delay diagnosis information sent by a detection end; Obtaining corresponding sampled data from a preset data buffer as target performance data according to the delay diagnosis information and writing it into a target file; wherein, the preset data buffer is used to store the sampled data of multiple running tasks in the system.

[0006] In some embodiments, the delay diagnosis information is obtained by the detection end through the following steps: Detecting the running temporal characteristics of multiple running tasks in the system; When it is detected that the running temporal characteristics of any one of the running tasks satisfy a preset trigger delay condition, generating delay diagnosis information.

[0007] In some embodiments, the running temporal characteristics of the running task include: the processing time for processing preset unit data; The step of generating delay diagnosis information when it is detected that the running temporal characteristics of any one of the running tasks satisfy a preset trigger delay condition includes: After detecting that the processing time of any of the running tasks for processing the preset unit data exceeds the preset time threshold, determine the running task whose processing time exceeds the preset time threshold as the target task; Obtain the task identifier information of the target task, generate associated timestamp information according to the target task, and generate delay diagnosis information according to the task identifier information and the timestamp information.

[0008] In some embodiments, the obtaining corresponding sampling data as target performance data from the preset data buffer according to the delay diagnosis information and writing the data into the target file includes: Identify the task identifier information and / or the timestamp information in the delay diagnosis information, and determine the target index range according to the task identifier information and / or the timestamp information; Obtain the corresponding sampling data in the preset data buffer as target performance data according to the target index range and write the data into the target file.

[0009] In some embodiments, the method further includes: Add unique identification information to the target file according to the timestamp information.

[0010] In some embodiments, the method further includes: Perform real-time data sampling on multiple running tasks in the system, and store the sampling data obtained by the real-time data sampling in the preset data buffer according to the preset cyclic overwrite rule.

[0011] A second aspect of the present application provides a system performance data processing system, including: A delay detection module, configured to detect multiple running tasks in the system, and send delay diagnosis information to the performance analysis module according to the detection result; A performance analysis module, configured to receive the delay diagnosis information sent by the delay detection module; obtain corresponding sampling data as target performance data from the preset data buffer according to the delay diagnosis information and write the data into the target file; wherein, the preset data buffer is used to store the sampling data of multiple running tasks in the system.

[0012] In some embodiments, a communication pipeline is constructed between the delay detection module and the performance analysis module; the delay detection module and the performance analysis module are communicatively connected through the communication pipeline.

[0013] A third aspect of the present application provides an electronic device, including: A processor; and A memory, on which executable code is stored, and when the executable code is executed by the processor, the processor is caused to execute the method as described above.

[0014] The fourth aspect of the present application provides a computer-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0015] The technical solution provided by the present application may include the following beneficial effects: In the technical solution of the present application, the sampling data of multiple running tasks in the system is stored through a preset data buffer, and after receiving the delay diagnosis information, the target performance data is extracted from the preset data buffer according to the delay diagnosis information. This effectively avoids the operation mode of continuously high-frequency data acquisition and real-time writing to the disk in the traditional technology, thereby effectively reducing the system I / O load pressure, avoiding the real-time task delay problem caused by frequent read and write operations of the storage device, and effectively ensuring the real-time response performance of the autonomous driving system; The mechanism of triggering the extraction of target performance data by using delay diagnosis information can effectively reduce the storage and transmission volume of redundant data, and effectively ensure the integrity of key performance event data, thereby improving the effectiveness of the performance analysis process of the system and maintaining the low-overhead operation of the system.

[0016] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By describing the exemplary embodiments of the present application in more detail in conjunction with the drawings, the above and other objects, features, and advantages of the present application will become more obvious. Among them, in the exemplary embodiments of the present application, the same reference numerals generally represent the same components.

[0018] Figure 1 is a schematic flowchart of a system performance data processing method shown in an embodiment of the present application; Figure 2 is another schematic flowchart of a system performance data processing method shown in an embodiment of the present application; Figure 3 is a schematic structural diagram of a system performance data processing system shown in an embodiment of the present application; Figure 4 is an application schematic diagram of a system performance data processing system shown in an embodiment of the present application; Figure 5 is a schematic structural diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present application will be more thorough and complete, and can fully convey the scope of the present application to those skilled in the art.

[0020] The terms used in the present application are for the purpose of describing specific embodiments only and are not intended to limit the present application. The singular forms "a", "the", and "said" used in the present application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0021] It should be understood that although the terms "first", "second", "third", etc. may be used in the present application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present application, "a plurality" means two or more unless otherwise specifically defined.

[0022] In the related art, when using traditional performance analysis tools to monitor the performance of an autonomous driving system, high-frequency data collection is required, and the generated performance events are written to the disk, which easily leads to an increase in the I / O (input / output) load in the system, thereby having an adverse impact on the real-time performance of the autonomous driving system.

[0023] Among them, in an autonomous driving scenario, the autonomous driving system is a system with high real-time performance and used for multi-data complex calculations. When directly applying traditional performance analysis tools to the autonomous driving system, the following defects and deficiencies may exist: 1. High runtime overhead: Generally, when sampling, performance analysis tools need to write performance events to the disk. In the above process, the performance analysis tools need to perform high-frequency sampling, thereby increasing the overall I / O load of the autonomous driving system, which easily affects the normal operation of related real-time tasks in the autonomous driving system. For example, the perception module in the autonomous driving system may experience delays due to the increase in the overall I / O load of the autonomous driving system, thereby leading to a decrease in the stability of the entire system; 2. Large data volume: Multiple complex modules are usually integrated in an autonomous driving system. Due to the large volume of performance data corresponding to the complex modules, the performance analysis files output by the performance analysis tool also have a large amount of data, which is easily up to the GB level or even the TB level. When the autonomous driving system reads and analyzes the above performance analysis files, it requires additional storage and processing time, resulting in the inability to meet the requirements of real-time monitoring and affecting the response efficiency of dynamically adjusting system parameters or optimizing bottlenecks.

[0024] In view of the above problems, the embodiment of the present application provides a method for processing system performance data, which can effectively reduce the system I / O load pressure, ensure the real-time response performance of the autonomous driving system, reduce the storage and transmission volume of redundant data, and improve the effectiveness of the process of performing performance analysis on the system and maintain the system running with low overhead.

[0025] The method for processing system performance data of the present application is mainly applied to the processing end for processing system performance data. Among them, the processing end can be a data analysis module located inside or outside the system, and the processing end can also be a tool resident in memory within the system. There is no limitation on the application type of the processing end here. Among them, the system described in the present application can be an autonomous driving system. Among them, the autonomous driving system can be deployed at the vehicle end, the cloud end or the road end, and there is no limitation here.

[0026] The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0027] Figure 1 is a schematic flowchart of the method for processing system performance data shown in the embodiment of the present application.

[0028] See Figure 1 , the method for processing system performance data of the present application includes: S110, receiving the delay diagnosis information sent by the detection end.

[0029] In this step, the delay diagnosis information sent by the detection end is received.

[0030] Among them, the detection end is used to detect the specified part or all running tasks currently running in the system. The detection end can be a data detection module for detecting system running tasks or a tool resident in memory within the system. There is no limitation on the application type of the detection end here either.

[0031] It should be understood that the running tasks in the present application can refer to the smallest logical execution units that can be independently scheduled, executed and occupy computing resources in an autonomous driving system or a computer system, such as at least one of operating system-level tasks, application service-level tasks, hardware resource operation tasks, and distributed collaborative tasks.

[0032] For the convenience of understanding the solution of this application, the following gives examples of the foregoing four types of operating system-level tasks, application service-level tasks, hardware resource operation tasks, and distributed collaboration tasks: Operating system-level tasks can be processes, threads, and interrupt service routines (ISRs) directly scheduled by the kernel, such as file read / write operations, inter-process communication (IPC), and device driver responses; Application service-level tasks can be execution units oriented to business logic, including user requests (such as HTTP transactions, database queries), batch processing jobs (such as log compression, data backup), and scheduled tasks (such as Cron Jobs); Hardware resource operation tasks can be atomic operations that interact with physical devices, covering disk I / O requests, network packet sending and receiving, GPU computing instructions, and memory page swapping; Distributed collaboration tasks can be collaboration units across nodes or services, such as microservice call chains (ServiceChain), message queue consumption (Message Consumption), and distributed transaction coordination (such as the two-phase commit protocol).

[0033] S120, obtain corresponding sampled data from a preset data buffer as target performance data according to the delay diagnosis information, and write it into a target file; wherein, the preset data buffer is used to store the sampled data of multiple running tasks of the system.

[0034] In this step, in response to the received delay diagnosis information, determine the target performance data from the preset data buffer storing the sampled data of multiple running tasks of the system, and write the determined target performance data into the target file for performance analysis.

[0035] Among them, the sampled data in the preset data buffer is obtained by sampling the running tasks of the system during the performance data analysis process of the system.

[0036] In this embodiment, the system performance data processing method of the present application stores the sampled data of multiple running tasks in the system through a preset data buffer, and after receiving the delay diagnosis information, extracts the target performance data from the preset data buffer according to the delay diagnosis information, that is, writes the sampled data into the buffer of the memory, and when a preset condition is triggered, writes it into the disk in a centralized manner, effectively avoiding the operation mode of continuous high-frequency data acquisition and real-time writing to the disk in the traditional technology, thereby effectively reducing the system I / O load pressure, avoiding the real-time task delay problem caused by frequent reading and writing of the storage device, and effectively ensuring the real-time response performance of the autonomous driving system; adopting the mechanism of triggering the extraction of target performance data by delay diagnosis information can effectively reduce the storage and transmission volume of redundant data, and effectively ensure the integrity of key performance event data, thereby improving the effectiveness of the performance analysis process of the system and maintaining the low-overhead operation of the system.

[0037] Figure 2 It is another schematic flowchart of the system performance data processing method shown in the embodiment of the present application.

[0038] See Figure 2 , the system performance data processing method of the present application includes: S210, perform real-time data sampling on multiple running tasks in the system, and store the sampled data obtained by real-time data sampling in a preset data buffer according to a preset cyclic overwrite rule.

[0039] In this step, real-time data sampling is performed on multiple currently running tasks in the system, and the sampled data corresponding to each running task is stored in a preset data buffer with a fixed capacity size according to a preset cyclic overwrite rule. It can be understood that the sampled data stored in the preset data buffer is cyclically overwritten and stored according to the first-in-first-out storage principle, so as to ensure that the latest sampled data is stored in the preset data buffer in real time.

[0040] Among them, the preset data buffer can be a circular buffer with a preset storage space size.

[0041] Among them, the sampled data in the preset data buffer can be classified and stored according to the task identifier information and timestamp information of the running task corresponding to the sampled data.

[0042] S220, receive the delay diagnosis information sent by the detection end.

[0043] In this step, receive the delay diagnosis information sent by the detection end.

[0044] Among them, the delay diagnosis information can be transmitted between the detection end and the processing end through a pre-established communication pipeline. That is, in the present application, the delay diagnosis information sent by the detection end can be received through the communication pipeline.

[0045] It should be understood that the pipe communication mechanism is mainly used for one-way communication, such as data transmission between related processes (such as parent and child processes) on the same system. Among them, receiving the delay diagnosis information sent by the detection end through the communication pipe can avoid additional overheads such as serialization and disk operations, reduce the impact of the detection end sending delay diagnosis information on system performance, and moreover, sending delay diagnosis information through the communication pipe has high real-time performance, making the finally output target performance data highly correlated with the delay diagnosis information, and can avoid the delay caused by polling.

[0046] In some embodiments, the delay diagnosis information is obtained by the detection end through the following steps: S2201, detecting the running temporal characteristics of multiple running tasks in the system.

[0047] The detection end runs in real time and detects the running temporal characteristics of multiple running tasks in the system. Among them, the running temporal characteristics are used to determine whether a running task has a delay.

[0048] Among them, the running temporal characteristics may refer to the process characteristics associated with a running task during its running process (i.e., a single run). It is not difficult to understand that for different types of running tasks, the corresponding running temporal characteristics may be different. For example, the running temporal characteristic of the perception module may be the frame processing time, that is, the total time required for the perception module (such as a camera, lidar, vision algorithm, etc.) to obtain a frame of raw data and complete the processing of this frame of data (object detection, segmentation, tracking, etc.).

[0049] S2202, when it is detected that the running temporal characteristic of any running task satisfies the preset trigger delay condition, generating delay diagnosis information.

[0050] Among them, when it is detected that the running temporal characteristic of any running task satisfies the corresponding preset trigger delay condition, delay diagnosis information corresponding to the running task is generated and sent to the processing end.

[0051] Among them, it should be understood that the detection processes of the running temporal characteristics between different running tasks do not interfere with each other, that is, the detection processes of whether the running temporal characteristics of different running tasks satisfy the preset trigger delay condition can be carried out simultaneously and do not interfere with each other.

[0052] For example, the running temporal characteristic of running task A corresponding to the perception module is set as the frame processing time, and the corresponding preset frame processing time threshold for running task A. When it is detected during any running temporal characteristic detection process of running task A that the frame processing time of running task A exceeds the preset frame processing time threshold, it is determined that running task A satisfies the preset trigger delay condition during this running temporal characteristic detection process, and corresponding delay diagnosis information is generated.

[0053] Among them, the delay diagnosis information of the detection end is automatically sent to the processing end through the communication pipeline immediately after it is generated.

[0054] Among them, the preset trigger delay condition can be a multi-level delay determination mechanism. For example, the running state feature of the running task A corresponding to the sensing module is the frame processing time, and the preset frame processing time threshold corresponding to the running task A. When in the detection process of any running state feature of the running task A, the frame processing time exceeds the preset frame processing time threshold and exceeds the standard deviation of the frame processing time of any continuous N running state feature detection processes, it is determined that the running task A meets the preset trigger delay condition in this running state feature detection process, and the corresponding delay diagnosis information is generated. Of course, the preset trigger delay condition can also be set according to actual application requirements, and there is no limitation here.

[0055] S230, identify the task identifier information and / or timestamp information in the delay diagnosis information, and determine the target index range according to the task identifier information and / or timestamp information.

[0056] In this step, after obtaining the delay diagnosis information, the task identifier information and / or timestamp information in the delay diagnosis information are identified, and the target index range is determined according to the pre-set index determination method.

[0057] Among them, the pre-set index determination method can at least include the following three methods: First, determine according to the task identifier information in the delay diagnosis information; Second, determine according to the timestamp information in the delay diagnosis information; Third, determine according to the task identifier information and timestamp information in the delay diagnosis information.

[0058] It can be understood that different target index ranges can be determined according to the task identifier information, timestamp information or task identifier information + timestamp information in the delay diagnosis information.

[0059] In the first method of determining the target index range, since the task identifier uniquely identifies a task, determining the target index range according to the task identifier information can obtain all relevant sampling data of the corresponding running task in the preset data buffer, which can be used for in-depth analysis of the performance bottleneck or abnormal behavior of a single running task, and is applicable to the application scenario of comprehensively analyzing a certain running task.

[0060] For example, after it is determined that the running task has a delay, the target index range corresponding to the running task is determined through the task identifier information, so as to obtain all sampling data related to the running task from the preset buffer subsequently, and perform a more comprehensive performance analysis on the running task.

[0061] In the second method for determining the target index range, since the timestamp information corresponds to a specific time period, by determining the target index range based on the timestamp information, all relevant sampled data within the preset data buffer for the corresponding time period can be obtained, which can be used for analyzing all running tasks during the specific time period, thereby achieving performance analysis of the running tasks with delays. At the same time, it can also check whether other running tasks within the same time period are affected, thus discovering potential performance problems, and it is applicable to the application scenario of horizontally comparing different running tasks within the same time period.

[0062] It should be understood that the preset data buffer is generally of a fixed size and stores data in a circular overwrite manner, and the amount of data corresponding to the target index range determined by the first or second method will not be too large either.

[0063] In the third method for determining the target index range, since it combines the task identifier information and the timestamp information, by determining the target index range through the task identifier information and the timestamp information, double filtering of the sampled data can be achieved, and it is possible to accurately determine the relevant sampled data of the running task with a delay during a specific time period from the preset data buffer data, thus reducing the amount of irrelevant data, and it is applicable to the application scenarios with high precision and high real-time performance.

[0064] Among them, the delay diagnosis information can be generated in the following fixed format: {PID, timestamp}, so that it is possible to quickly obtain the task identifier information and the timestamp information according to the delay diagnosis information.

[0065] S240. Obtain the corresponding sampled data in the preset data buffer as the target performance data according to the target index range and write it into the target file.

[0066] In this step, according to the determined target index range, quickly index and locate the sampled data in the corresponding range in the preset data buffer, and write the sampled data obtained by the above positioning as the target performance data into the target file for performance analysis.

[0067] Among them, when determining the target index range through the task identifier information and the timestamp information in the delay diagnosis information, the target performance data can be determined by using a two-level index method. For example, first determine all relevant sampled data of the corresponding running task in the preset data buffer through the task identifier information, and then determine the sampled data in the corresponding time period through the timestamp information, so as to achieve rapid positioning and obtain the target performance data.

[0068] Among them, the sampled data corresponding to the corresponding time period determined by the timestamp information can be the sampled data determined according to the preset time range before and after the time point corresponding to the timestamp information. For example, if the preset time range is ±100 milliseconds, the sampled data corresponding to ±100 milliseconds before and after the time point corresponding to the timestamp information is determined as the target performance data.

[0069] Among them, the target file can be a system file used to view the sampled data of the CPU usage in the system when a performance problem occurs. For example, it can be used to view the number of times each function is collected when running on the CPU or the duration of the function running in the CPU. The target file can have a fixed output path, such as being output to the path on the disk used to store the logs generated during the process operation, such as the following path: / home / mogo / data / log directory. It should be understood that each time the delay diagnosis information is received, a corresponding target file is generated, that is, each time a performance problem occurs in the system, a new target file is generated to record the sampled data of the CPU usage in the system.

[0070] S250, add unique identification information to the target file according to the timestamp information.

[0071] In this step, after obtaining the target performance data and writing it into the target file, add unique identification information to the target file according to the timestamp information. It should be understood that after adding unique identification information to the target file, the target file has uniqueness and traceability, which is convenient for subsequent analysis and application of the target file.

[0072] Among them, unique identification information can be added to the target file by adding timestamp information to the file name of the target file. Further, the time accuracy of the timestamp information added to the file name of the target file can be millisecond level, so as to further improve the stability of the identification information.

[0073] In this embodiment, the system performance data processing method of the present application can accurately track the performance data associated with the target task with delay through the joint data extraction mechanism based on the task identifier information and the timestamp information, ensuring the data relevance and integrity written into the target file; it can also automatically generate a target file with identification information related to the timestamp and used for performance data analysis, effectively ensuring the uniqueness of the generated performance data file, which is convenient for subsequent application of the performance data file, such as historical data backtracking, trend statistics or multi-dimensional data analysis through an offline analysis tool.

[0074] Corresponding to the foregoing application function implementation method embodiment, the present application also provides a system performance data processing system, an electronic device and corresponding embodiments.

[0075] Figure 3 It is a schematic structural diagram of the system performance data processing system shown in the embodiments of the present application.

[0076] See Figure 3 , the system performance data processing system 300 of the present application includes a delay detection module 310 and a performance analysis module 320. Among them, the delay detection module 310 can be used as a detection end for detecting a specified part or all running tasks currently running in the system. The detection end can be a data detection module for detecting system running tasks or a resident memory tool in the system, and the application type of the detection end is not limited here. Among them, the performance analysis module 320 can be used as a processing end for processing system performance data. The performance analysis module 320 can be a data analysis module located inside or outside the system, and the performance analysis module 320 can also be a resident memory tool in the system. The application type of the processing end is not limited here.

[0077] The delay detection module 310 is used to detect multiple running tasks in the system and send delay diagnosis information to the performance analysis module according to the detection results.

[0078] In some embodiments, the delay diagnosis information is obtained by the delay detection module 310 through the following steps: detecting the running temporal characteristics of multiple running tasks in the system; when it is detected that the running temporal characteristics of any running task meet the preset trigger delay condition, generating delay diagnosis information.

[0079] In some embodiments, the running temporal characteristics of the running task include: the processing time for processing preset unit data; the delay detection module 310 can also be used to determine the running task with the processing time exceeding the preset time threshold as the target task after detecting that the processing time of any running task for processing preset unit data exceeds the preset time threshold; obtaining the task identifier information of the target task, generating associated timestamp information according to the target task, and generating delay diagnosis information according to the task identifier information and the timestamp information.

[0080] The performance analysis module 320 is used to receive the delay diagnosis information sent by the delay detection module 310; obtaining corresponding sampled data from the preset data buffer as target performance data according to the delay diagnosis information and writing it into the target file; where the preset data buffer is used to store the sampled data of multiple running tasks in the system.

[0081] In some embodiments, a communication pipeline is built between the delay detection module 310 and the performance analysis module 320; the delay detection module 310 and the performance analysis module 320 are communicatively connected through the communication pipeline.

[0082] In some embodiments, the performance analysis module 320 may also be used to identify task identifier information and / or timestamp information in the latency diagnosis information, determine a target index range based on the task identifier information and / or timestamp information; and obtain corresponding sampled data in the preset data buffer as target performance data according to the target index range and write it into the target file.

[0083] In some embodiments, the performance analysis module 320 may also be used to add unique identification information to the target file according to the timestamp information.

[0084] In some embodiments, the performance analysis module 320 may also be used to perform real-time data sampling on multiple running tasks in the system, and store the sampled data obtained by the real-time data sampling in the preset data buffer according to a preset cyclic overwrite rule.

[0085] For the convenience of understanding the technical solutions of the system performance data processing system of the present application, the following provides the application process steps of the system performance data processing system of the present application.

[0086] Figure 4 is an application schematic diagram of the system performance data processing system shown in the embodiments of the present application.

[0087] See Figure 4 , the working process of the system performance data processing system 300 of the present application may include the following steps: S410, detecting whether multiple running tasks have latency through the latency detection module 310; wherein, when the latency detection module 310 detects that any running task has latency, it generates latency diagnosis information and sends it to the communication pipeline.

[0088] S420, collecting performance data of multiple running tasks through the performance analysis module 320 and storing it in the circular buffer, and reading the communication pipeline in real time.

[0089] S430, determining whether the latency detection module 310 receives latency diagnosis information through the communication pipeline. If so, enter step S440; otherwise, re-enter step S420.

[0090] S440, writing the performance data corresponding to the latency diagnosis information into the target file located on the disk through the performance analysis module 320, and feeding back a write success message to the latency detection module 310 through the communication pipeline.

[0091] It should be understood that after the feedback of the write success message is completed in step S440, it can also return to step S420 again to realize the continuous performance data analysis process of the system.

[0092] In this embodiment, the system performance data processing system of the present application stores the sampled data of multiple running tasks in the system through a preset data buffer, and after receiving the delay diagnosis information, extracts the target performance data from the preset data buffer according to the delay diagnosis information, that is, writes the sampled data into the buffer of the memory, and when a preset condition is triggered, writes it into the disk in a centralized manner, effectively avoiding the operation mode of continuous high-frequency data acquisition and real-time writing to the disk in the traditional technology, thereby effectively reducing the system I / O load pressure and avoiding the real-time task delay problem caused by frequent reading and writing of the storage device, and effectively ensuring the real-time response performance of the autonomous driving system; adopting the mechanism of triggering the extraction of target performance data by delay diagnosis information can effectively reduce the storage and transmission volume of redundant data, and effectively ensure the integrity of key performance event data, thereby improving the effectiveness of the performance analysis process of the system and maintaining the low-overhead operation of the system.

[0093] Regarding the device in the above embodiment, the specific manners in which each module performs operations have been described in detail in the embodiment related to the method, and will not be elaborated herein.

[0094] Figure 5 It is a schematic structural diagram of an electronic device shown in an embodiment of the present application.

[0095] See Figure 5 , the electronic device 1000 includes a memory 1010 and a processor 1020.

[0096] The processor 1020 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0097] The memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, the ROM may store static data or instructions required by the processor 1020 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device employs a mass storage device (such as a magnetic or optical disk, flash memory) as the permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory. The system memory may store some or all of the instructions and data required by the processor during operation. In addition, the memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a removable storage device that is readable and / or writable, such as a compact disc (CD), read-only digital versatile disc (such as DVD-ROM, dual-layer DVD-ROM), read-only Blu-ray disc, ultra density disc, flash memory card (such as SD card, min SD card, Micro-SD card, etc.), magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and instantaneous electronic signals transmitted wirelessly or wired.

[0098] Executable code is stored on the memory 1010, and when the executable code is processed by the processor 1020, it can cause the processor 1020 to execute some or all of the methods described above.

[0099] In addition, the method according to the present application can also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps of the above method of the present application.

[0100] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium), on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or a server, etc.), it causes the processor to execute some or all of the steps of the above method according to the present application.

[0101] The embodiments of the present application have been described above. The above description is exemplary and not exhaustive, and is also not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of technologies in the market, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for processing system performance data, characterized in that, Applied to the processing end for processing system performance data, including: Receiving the delay diagnosis information sent by the detection end; Obtaining corresponding sampling data from a preset data buffer as target performance data according to the delay diagnosis information and writing it into a target file; wherein, the preset data buffer is used to store the sampling data of multiple running tasks in the system.

2. The method according to claim 1, wherein The delay diagnosis information is obtained by the detection end through the following steps: Detecting the running state characteristics of multiple running tasks in the system; When it is detected that the running state characteristics of any one of the running tasks meet the preset trigger delay condition, generating delay diagnosis information.

3. The method according to claim 2, wherein The running state characteristics of the running task include: the processing time for processing preset unit data; The step of generating delay diagnosis information when it is detected that the running state characteristics of any one of the running tasks meet the preset trigger delay condition includes: After detecting that the processing time of any one of the running tasks for processing preset unit data exceeds the preset time threshold, determining the running task whose processing time exceeds the preset time threshold as the target task; Obtaining the task identifier information of the target task, generating associated timestamp information according to the target task, and generating delay diagnosis information according to the task identifier information and the timestamp information.

4. The method according to claim 1, characterized in that The step of obtaining corresponding sampling data from a preset data buffer as target performance data according to the delay diagnosis information and writing it into a target file includes: Identifying the task identifier information and / or timestamp information in the delay diagnosis information, and determining the target index range according to the task identifier information and / or timestamp information; Obtaining the corresponding sampling data in the preset data buffer as target performance data according to the target index range and writing it into the target file.

5. The method according to claim 4, characterized in that, The method further includes: Adding unique identification information to the target file according to the timestamp information.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Performing real-time data sampling on multiple running tasks in the system, and storing the sampling data obtained by the real-time data sampling in the preset data buffer according to a preset cyclic overwrite rule.

7. A system performance data processing system, characterized in that, Including: A delay detection module, configured to detect multiple running tasks in the system and send delay diagnosis information to the performance analysis module according to the detection result; A performance analysis module, configured to receive the delay diagnosis information sent by the delay detection module; obtaining corresponding sampling data from a preset data buffer as target performance data according to the delay diagnosis information and writing it into a target file; wherein, the preset data buffer is used to store the sampling data of multiple running tasks in the system.

8. The system according to claim 7, wherein A communication pipeline is constructed between the delay detection module and the performance analysis module; the delay detection module and the performance analysis module are communicatively connected through the communication pipeline.

9. An electronic device, characterized in that, Including: A processor; And A memory, on which executable code is stored, and when the executable code is executed by the processor, the processor executes the method according to any one of claims 1-6.

10. A computer-readable storage medium, on which executable code is stored, characterized in that: When the executable code is executed by the processor of an electronic device, the processor executes the method according to any one of claims 1-6.

Images