Model diagnosis method and device, electronic equipment and storage medium
By collecting and preprocessing performance data on electronic devices, dividing and analyzing sub-data sequences, the problem of low diagnostic efficiency of artificial intelligence models is solved, and higher diagnostic accuracy and efficiency are achieved.
Patent Information
- Application Number
- CN202411334987.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2025-01-17
AI Technical Summary
The existing artificial intelligence model defect diagnosis methods are inefficient and cannot accurately locate defects to be optimized, resulting in low diagnostic accuracy and efficiency.
By collecting performance data on electronic devices, dividing the data sequence into sub-data sequences, and pre-processing them, the diagnostic results are determined to improve the accuracy and efficiency of defect diagnosis.
Defect diagnosis is performed after the electronic device starts to run the model to be detected, which improves the accuracy and efficiency of defect diagnosis, and can determine local characteristics in the time dimension, improves data quality, and ensures the accuracy of diagnostic results.
Smart Images
Figure CN119322932A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model diagnosis method, device, electronic device and storage medium. Background Art
[0002] At present, with the development of artificial intelligence technology, more and more organizations or individuals tend to use artificial intelligence models to assist production in order to improve productivity. Since improving the training efficiency of artificial intelligence models is of great significance for enterprises to control the R&D costs of artificial intelligence models, more and more companies usually focus on diagnosing defects in artificial intelligence models during the training of artificial intelligence models, and improving the computing efficiency of the graphics computing units used to run artificial intelligence models, in order to complete more computing tasks under the same hardware conditions, or use less hardware resources to achieve the same training effect, which directly helps enterprises or individuals reduce the economic cost of artificial intelligence models.
[0003] Due to the low efficiency of the open source software currently used for defect diagnosis of artificial intelligence models, it is unable to accurately locate the defects to be optimized, and the output defect-related information is not comprehensive enough, resulting in low accuracy and efficiency of defect diagnosis of artificial intelligence models. Summary of the invention
[0004] In view of the above, it is necessary to propose a model diagnosis method, device, electronic device and storage medium to solve the technical problem of low efficiency in diagnosing artificial intelligence models.
[0005] The present application provides a model diagnosis method, which is applied to an electronic device. The method includes: when the first performance data of the electronic device meets a preset first condition, collecting second performance data of the electronic device within a preset time length when the electronic device runs a model to be detected to obtain a data sequence; determining a preset time length for dividing the data sequence based on the second performance data; dividing the data sequence based on the preset time length to obtain multiple sub-data sequences; preprocessing the multiple sub-data sequences to obtain a sequence to be diagnosed; and determining a diagnosis result of the model to be detected based on the data sequence and / or the sequence to be diagnosed.
[0006] An embodiment of the present application also provides a model diagnosis device, which includes: an acquisition module, which is used to acquire second performance data of the electronic device within a preset time length when the first performance data of the electronic device runs the model to be detected, so as to obtain a data sequence; a determination module, which is used to determine the preset time length for dividing the data sequence based on the second performance data; a division module, which is used to divide the data sequence based on the preset time length to obtain multiple sub-data sequences; a preprocessing module, which is used to preprocess the multiple sub-data sequences to obtain a sequence to be diagnosed; and a diagnosis module, which is used to determine the diagnosis result of the model to be detected based on the data sequence and / or the sequence to be diagnosed.
[0007] An embodiment of the present application also provides an electronic device, which includes: a memory storing at least one instruction; and a processor executing the instructions stored in the memory to implement the model diagnosis method.
[0008] An embodiment of the present application also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the model diagnosis method.
[0009] It can be seen from the above technical solutions that, in the case where the first performance data of the electronic device meets the preset first condition, the embodiment of the present application collects the data sequence corresponding to the second performance data of the electronic device within the preset time length of the running model to be detected, so as to avoid the problem of reduced training efficiency of the model to be detected due to frequent diagnosis of the model to be detected. And it is possible to start diagnosing the defects of the model to be detected after the electronic device starts running the model to be detected, so as to improve the accuracy of defect diagnosis of the model to be detected. Based on the second performance data, the preset time length of the data sequence is determined, and then the data sequence is divided into multiple sub-data sequences according to the preset time length, so that the data sequence with a longer time length can be divided into fine-grained divisions, so as to determine the local characteristics of the second performance data in the time dimension, and then the accuracy of subsequent defect diagnosis of the model to be detected can be improved. In addition, multiple sub-data sequences are pre-processed to obtain the sequence to be diagnosed, which can further improve the quality of the data involved in defect diagnosis, and finally determine the diagnosis result of the model to be detected based on the data sequence and / or the sequence to be diagnosed, so as to obtain the diagnosis result by performing different diagnostic processing on different data, and the defect diagnosis efficiency of the model to be detected can be improved while ensuring the accuracy of diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 This is an application scenario diagram of a model diagnosis method provided in one embodiment of the present application.
[0011] Figure 2This is a flow chart of a model diagnosis method provided in one embodiment of the present application.
[0012] Figure 3 This is a flowchart of a method for determining a preset duration provided by an embodiment of the present application.
[0013] Figure 4 It is a flowchart of a method for determining a sequence to be diagnosed provided by an embodiment of the present application.
[0014] Figure 5 It is a flowchart of a method for determining a diagnosis result according to a sequence to be diagnosed provided in one embodiment of the present application.
[0015] Figure 6 It is a functional module diagram of a model diagnosis device provided in one embodiment of the present application.
[0016] Figure 7 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0017] In order to more clearly understand the purpose, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. In the following description, many specific details are set forth to facilitate a full understanding of the present application, and the embodiments described are only a part of the embodiments of the present application, rather than all of the embodiments.
[0018] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art to which the present application belongs. The terms used herein in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.
[0020] The embodiment of the present application provides a model diagnosis method, which can be applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a programmable gate array (Field-Programmable Gate Array, FPGA), a digital processor (Digital Signal Processor, DSP), an embedded device, etc.
[0021] An electronic device can be any electronic product that can interact with a customer, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.
[0022] The electronic device may also include a network device and / or a client device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.
[0023] The network where the electronic device is located includes but is not limited to the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.
[0024] like Figure 1 As shown, a model diagnosis method provided by the present application can be applied to an electronic device 100, and a model to be detected 200 is running in the electronic device 100. The model to be detected 200 can be any model used for data processing, for example, the model to be detected 200 can be a multi-layer perceptron model, a convolutional neural network model, a recurrent neural network model, or a support vector machine model. The present application does not limit the specific type of the model to be detected 200.
[0025] The electronic device 100 also includes a resource monitoring module 300, a trigger module 400, a performance acquisition module 500 and a diagnostic module 600. Among them, the resource monitoring module 300 is used to periodically collect the first performance data of the electronic device 100. When the trigger module 400 determines that the resource utilization data meets the preset first condition, it indicates that the model to be detected 200 has started to run. At this time, the performance acquisition module 500 periodically collects the second performance data of the electronic device 100, and the diagnostic module 600 determines the defects existing in the operation process of the model to be detected 200 according to the second performance data and outputs the corresponding defect type. Among them, the defects existing in the operation process of the model to be detected 200 can be factors that reduce the training efficiency of the model to be detected 200. For example, the defects of the model to be detected 200 can be that the model to be detected 200 is stuck for a long time, or the efficiency of the pre-processing process or the post-processing process of the CPU during the training process of the model to be detected 200 is low, or the hardware resources are not fully utilized, etc. This application does not limit the specific category of the defects of the model to be detected 200.
[0026] like Figure 2 , is a flow chart of a model diagnosis method provided by an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. A model diagnosis method provided by an embodiment of the present application includes the following steps.
[0027] S20, when the first performance data of the electronic device meets a preset first condition, collecting second performance data of the electronic device within a preset time period when the electronic device runs the model to be detected to obtain a data sequence.
[0028] In one embodiment of the present application, since frequent diagnosis of the model to be detected may have a negative impact on the training efficiency of the model to be detected, and since the data used for training needs to be initialized before training the model to be detected, and the model to be detected is not running during the data initialization stage, it is possible to determine in real time whether the first performance data of the electronic device meets the preset first condition. When the first performance data of the electronic device meets the preset first condition, the second performance data of the electronic device within the preset time length of the running model to be detected is collected, so that the defects of the model to be detected can be diagnosed after the electronic device starts running the model to be detected, thereby improving the accuracy of defect diagnosis of the model to be detected. Wherein, the preset first condition includes: when the first performance data is obtained based on the first period, the first performance data within the preset time length is greater than the first value; and when the first performance data is collected based on the second period, the first performance data is less than or equal to the second value.
[0029] In one embodiment of the present application, since the graphics processing unit (GPU) has a large number of floating-point data operation units (GPU cores), it can perform multiple computing tasks simultaneously, thereby accelerating large-scale matrix and vector operations, so the first performance data can be the utilization rate of the graphics processing unit (GPU). In addition, the central processing unit (CPU) can also be used for general computing tasks, which can further improve the training efficiency of the artificial intelligence model, so the first performance data can also be the utilization rate of the central processing unit (CPU), and the present application does not limit the specific type of the first performance data.
[0030] Exemplarily, the first cycle can be 1 minute, the preset duration can be 3 minutes, and the first value can be 0; at a time interval of 1 minute, the GPU usage rate is collected once, and if the GPU usage rate obtained for 3 consecutive minutes is equal to 0, it indicates that the model to be detected is still in the data initialization stage, and there is no need to start the defect diagnosis process at this time; if the GPU usage rate obtained for 3 consecutive minutes is greater than 0, it indicates that the model to be detected is still in the data initialization stage, and the first performance data can be collected based on the second cycle at this time, wherein the second cycle can be 10 minutes, and the second value can be 0.8; at a time interval of 10 minutes, if the GPU usage rate is greater than 0.8, it indicates that the model to be detected currently has no defects; at a time interval of 10 minutes, if the GPU usage rate is less than or equal to 0.8, it indicates that the model to be detected has defects, and the defect diagnosis process of the model to be detected can be started.
[0031] S21: Determine a preset time length for dividing the data sequence based on the second performance data.
[0032] In one embodiment of the present application, since the second performance data is the performance data of the electronic device continuously collected after the GPU starts running, the data volume of the second performance data is large, and the time length corresponding to the data sequence is long. When analyzing the second performance data corresponding to the longer time length, the data sequence can be divided into multiple sub-data sequences according to a certain time length, thereby reducing the data dimension and improving the efficiency of subsequent data analysis.
[0033] Specifically, when analyzing a long time series, dividing it into multiple sub-data series of shorter time and equal length includes the following advantages: improving analysis efficiency and reducing the amount of calculation. After dividing a long time series into short sub-data series, the amount of data in each sub-data series is significantly reduced, thereby reducing the complexity and time cost of data analysis, processing and calculation. The independence of sub-data series allows them to be processed in parallel, further improving the analysis efficiency. And because the time span of the sub-data series is short and the correlation between data is stronger, the sub-data series is more likely to reveal local trends and patterns, and it is easy to identify the law of change in the short term. In addition, the division of sub-data series helps to quickly locate outliers or mutations in the second performance data. Each sub-data series may have different statistical characteristics and trends, so a suitable model can be selected for each sub-data series for fitting and prediction, thereby improving the accuracy of the prediction. After dividing a long time series into short sub-data series, the changing trends and characteristics of the data can be more intuitively displayed in the form of charts and the like.
[0034] In an embodiment of the present application, in order to ensure that the multiple sub-data sequences divided according to the preset time length can conform to the change trend of the second performance data, the preset time length for dividing the data sequence can be determined based on the second performance data. For specific methods for determining the preset time length, please refer to Figure 3 The corresponding detailed description.
[0035] S22, dividing the data sequence based on the preset time length to obtain multiple sub-data sequences.
[0036] In one embodiment of the present application, in order to reduce the data dimension involved in the defect diagnosis of the model to be detected, thereby improving the efficiency of defect diagnosis of the model to be detected, the data sequence can be divided into multiple sub-data according to a preset time length, so that the amount of calculation for analyzing the second performance data can be reduced, thereby improving the defect diagnosis efficiency of the model to be detected. Wherein, the data sequence includes multiple data points, each data point corresponds to a timestamp, and the time difference between the timestamps corresponding to any two adjacent data points is the same. Specifically, dividing the data sequence based on the preset time length to obtain multiple sub-data sequences includes: determining the data points corresponding to multiple timestamps in order from early to late as a sub-data sequence; wherein, the time difference between the timestamp corresponding to the first data point in the sub-data sequence and the timestamp corresponding to the last data point in the sub-data sequence is equal to the preset time length.
[0037] For example, when the time difference between the timestamps corresponding to any two adjacent data points in a data sequence is 0.01 seconds, and the preset duration is 10 seconds, it can be determined that the first data point in the data sequence and all the data points from the earliest to the latest 101st data point are a sub-data sequence, so that the duration of the sub-data sequence can be ensured to be 10 seconds. Similarly, the 102nd data point in the data sequence is determined as the first data point in the next sub-data sequence, and multiple sub-data sequences are determined in sequence.
[0038] S23, preprocessing the multiple sub-data sequences to obtain a sequence to be diagnosed.
[0039] In one embodiment of the present application, multiple sub-data sequences can be used to characterize the local characteristics of the data sequence obtained by collecting the second performance data, wherein each sub-data sequence can be used to characterize the fluctuation trend (e.g., rising trend, falling trend, stable trend) of the second performance data within a certain period of time, and analyzing multiple sub-data sequences helps to determine the local characteristics within the time period. In addition, in the case where the data sequence is periodic data, each sub-data sequence may contain a complete periodic fluctuation or a part of the periodic fluctuation, then analyzing multiple sub-data sequences helps to identify the periodic pattern of the data sequence. In the case where the data sequence is a non-stationary sequence, the sub-data sequences corresponding to different time periods may have different local characteristics. When analyzing multiple sub-data sequences, each sub-data sequence can be modeled separately to achieve dynamic modeling, thereby improving the accuracy of model defect diagnosis. Decomposing a long sequence into short sequences can simplify the complexity of data analysis and make the processing process more efficient. By comparing the statistical characteristics of the sub-data sequences (such as mean, standard deviation, etc.), it is easier to identify outliers or abnormal behaviors. Modeling and forecasting based on shorter sub-data series of equal length can reduce the noise and data redundancy problems caused by too long time series, thereby improving forecasting accuracy.
[0040] In one embodiment of the present application, since the data sequence is divided into multiple sub-data sequences according to the order of time from early to late, the duration of the last sub-data sequence may be different from the duration of the remaining sub-data sequences. For example, when the duration of the data sequence is 60 seconds and the preset duration is 9 seconds, the duration of the first six sub-data sequences divided is 9 seconds, and the duration of the last sub-data sequence is 6 seconds. In order to ensure that the duration of the sub-data sequences involved in defect diagnosis is consistent, the sequence to be diagnosed can be determined based on the remaining sub-data sequences except the last sub-data sequence.
[0041] In one embodiment of the present application, each sub-data sequence can be used to characterize the fluctuation trend of the second performance data within a certain period of time. In order to determine the global features corresponding to the data sequence, the sequence to be diagnosed can be obtained by determining the mean of multiple sub-data sequences except the last sub-data sequence. The duration of the sequence to be diagnosed is the same as the preset duration. For details, please refer to Figure 4 The corresponding detailed description.
[0042] S24, determining the diagnosis result of the model to be detected based on the data sequence and / or the sequence to be diagnosed.
[0043] In one embodiment of the present application, in order to improve the accuracy of the diagnosis result of the model to be detected, a pre-trained prediction model can be used to process the sequence to be diagnosed to obtain the first diagnosis result of the model to be detected. Among them, the pre-trained prediction model can be a multi-layer perceptron model, a convolutional neural network model, a recurrent neural network model, or a support vector machine model. The present application does not limit the specific type of the prediction model. For a specific method of determining the diagnosis result based on the sequence to be diagnosed, please refer to Figure 5 The corresponding detailed description.
[0044] In one embodiment of the present application, in order to diagnose some specific defects in the running process of the model to be detected, it is also possible to determine whether the data sequence meets the preset second condition. If the data sequence meets the preset second condition, it is determined that the model to be detected has a defect category corresponding to the second condition. Wherein, the second preset condition includes one or more of the following: the ratio of the number of calls to the video memory allocation interface in the data sequence to the number of calls to all graphics card interfaces is greater than the preset third threshold; the ratio of the number of calls to the video memory release interface in the data sequence to the number of calls to all graphics card interfaces is greater than the preset third threshold; the ratio of the graphics card calculation time in the data sequence to the preset acquisition time is less than the preset fourth threshold.
[0045] Specifically, when the ratio of the number of calls of the video memory allocation interface in the data sequence to the number of calls of all graphics card interfaces is greater than the preset third threshold, it indicates that the electronic device allocates too many video memory times in the process of training the model to be detected, which may cause the training process of the model to be detected to consume more time waiting for the electronic device to allocate video memory for the model to be detected. The defect category corresponding to the second condition is "unreasonable video memory allocation", and it can be determined that the model to be detected has the defect of unreasonable video memory allocation. When the ratio of the number of calls of the video memory release interface in the data sequence to the number of calls of all graphics card interfaces is greater than the third threshold, it indicates that the electronic device releases too many video memory times in the process of training the model to be detected. The defect category corresponding to the second condition is "unreasonable video memory release", and it can be determined that the model to be detected has the defect of unreasonable video memory release. When the ratio of the graphics card calculation time in the data sequence to the preset acquisition time is less than the fourth threshold, it indicates that the graphics card needs to wait for a long time after running a code before starting to execute the next code, which leads to the problem of low efficiency of the underlying code running of the model to be detected. The defect category corresponding to the second condition is "low code running efficiency", and it can be determined that the model to be detected has the defect of low code running efficiency. Among them, according to experiments, it can be determined that the third threshold can be 0.02, and the fourth threshold can be 0.85. This application does not limit the values of the third threshold and the fourth threshold.
[0046] It can be seen from the above technical solutions that, in the case where the first performance data of the electronic device meets the preset first condition, the embodiment of the present application collects the data sequence corresponding to the second performance data of the electronic device within the preset time length of the running model to be detected, so as to avoid the problem of reduced training efficiency of the model to be detected due to frequent diagnosis of the model to be detected. And it is possible to start diagnosing the defects of the model to be detected after the electronic device starts running the model to be detected, so as to improve the accuracy of defect diagnosis of the model to be detected. Based on the second performance data, the preset time length of the data sequence is determined, and then the data sequence is divided into multiple sub-data sequences according to the preset time length, so that the data sequence with a longer time length can be divided into fine-grained divisions, so as to determine the local characteristics of the second performance data in the time dimension, and then the accuracy of subsequent defect diagnosis of the model to be detected can be improved. In addition, multiple sub-data sequences are pre-processed to obtain the sequence to be diagnosed, which can further improve the quality of the data involved in defect diagnosis, and finally determine the diagnosis result of the model to be detected based on the data sequence and / or the sequence to be diagnosed, so as to obtain the diagnosis result by performing different diagnostic processing on different data, and the defect diagnosis efficiency of the model to be detected can be improved while ensuring the accuracy of diagnosis.
[0047] like Figure 3, is a flow chart of a method for determining a preset duration provided by an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for determining a preset duration provided by an embodiment of the present application includes the following steps.
[0048] S30: Determine a first data sequence in the second performance data, wherein the first data sequence includes a plurality of first data points and corresponding timestamps.
[0049] In one embodiment of the present application, the first data sequence may be a GPU utilization sequence, wherein each first data point is used to represent a GPU utilization value collected by the electronic device at a certain timestamp.
[0050] S31, determining the autocorrelation coefficient of the first data point corresponding to each timestamp.
[0051] In an embodiment of the present application, the method for determining the autocorrelation coefficient of the first data point corresponding to each timestamp satisfies the following relationship:
[0052]
[0053] Among them, τ and t both represent the index of the timestamp, and the values of τ and t are both integers between 0 and N-1, R(τ) represents the autocorrelation coefficient of the first data point corresponding to the τth timestamp; f(t) represents the value of the first data point corresponding to the tth timestamp.
[0054] S32, determining a ratio of each of the autocorrelation coefficients to the autocorrelation coefficient corresponding to the first timestamp.
[0055] In one embodiment of the present application, the larger the ratio of the autocorrelation coefficient corresponding to any timestamp to the autocorrelation coefficient corresponding to the first timestamp, the higher the correlation between all data points from the first timestamp to the timestamp and the first data sequence, and the sub-data sequence composed of all data points from the first timestamp to the timestamp can be used to characterize the changing trend of the first data sequence.
[0056] S33: When the ratio is greater than or equal to a preset first threshold, determine the time difference between the timestamp corresponding to the ratio and the first timestamp as the preset duration.
[0057] In one embodiment of the present application, when the ratio is greater than or equal to a preset first threshold, it indicates that the higher the correlation between all data points from the first timestamp to the timestamp and the first data sequence, the sub-data sequence composed of all data points from the first timestamp to the timestamp can be used to characterize the change trend of the first data sequence. Therefore, the time difference between the timestamp corresponding to the ratio and the first timestamp can be determined as the preset time length for dividing the data sequence.
[0058] Exemplarily, if the first threshold is 0.8, when the 10th timestamp is 10 minutes, 10 seconds, and 0 milliseconds, and the ratio corresponding to the timestamp is 0.9, it can be determined that the preset duration is 10 minutes and 10 seconds.
[0059] like Figure 4 As shown, it is a flow chart of a method for determining a sequence to be diagnosed provided by an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for determining a sequence to be diagnosed provided by an embodiment of the present application includes the following steps.
[0060] S40, determining the last sub-data sequence according to the order of the timestamps from earliest to latest, and determining a plurality of sub-data sequences other than the last sub-data sequence as a plurality of sequences to be processed.
[0061] In one embodiment of the present application, since the data sequence is divided into multiple sub-data sequences according to the order of time from early to late, the duration of the last sub-data sequence may be different from the duration of the remaining sub-data sequences. For example, when the duration of the data sequence is 60 seconds and the preset duration is 9 seconds, the duration of the first six sub-data sequences divided is 9 seconds, and the duration of the last sub-data sequence is 6 seconds. In order to ensure that the duration of the sub-data sequences involved in defect diagnosis is consistent, the remaining sub-data sequences other than the last sub-data sequence can be determined as the sequence to be processed, and the sequence to be diagnosed can be determined based on the sequence to be processed.
[0062] S41, calculating the mean values corresponding to the data points having the same order in the plurality of sequences to be processed.
[0063] In one embodiment of the present application, obtaining a mean sequence for multiple sub-data sequences in a longer data sequence can reduce random fluctuations in the subsequent data analysis process. Time series data often contains random noise, which may cause data to fluctuate violently in a short period of time. By obtaining the mean of multiple sub-data sequences, these random fluctuations can be effectively smoothed, making the change trend of the data clearer and more stable. It can also improve data quality. The generation process of the mean sequence is actually a kind of data filtering, which can eliminate or reduce outliers and extreme values in the data, thereby improving the quality of the overall data. And it can characterize the long-term trend in the data sequence based on the mean sequence with a shorter duration. In time series analysis, the long-term trend is one of the important information that analysts pay attention to. By obtaining the mean of multiple sub-data sequences, the long-term trend of data changes over time can be more prominently displayed, and the overall trend of data can be analyzed more accurately in the subsequent data analysis process. After dividing the data sequence into multiple sub-data sequences and obtaining the mean, the amount of data will be greatly reduced, which can not only reduce the complexity of subsequent data analysis, but also improve analysis efficiency and simplify the data analysis process. In addition, since the mean sequence has smoothed the random fluctuations and outliers in the original data, making the prediction results more stable and reliable, using the mean sequence as data for defect diagnosis can reduce diagnostic errors.
[0064] In one embodiment of the present application, the order of each data point in the sequence to be processed in the sequence to which it belongs can be determined, and the mean values corresponding to the data points with the same order in multiple sequences to be processed are calculated. For example, when the sub-data sequences are [0, 1, 2, 3, 4, 5], [1, 2, 3, 4, 5, 6], and [2, 3, 4, 5, 6, 7], the means corresponding to the data points with the same order are 1, 2, 3, 4, 5, 6.
[0065] S42, combining the average values in order from first to last to obtain the sequence to be diagnosed.
[0066] For example, if the order of the means is 1, 2, 3, 4, 5 and 6, then the sequence to be diagnosed is [1, 2, 3, 4, 5, 6].
[0067] like Figure 5 As shown, it is a flow chart of a method for determining a diagnosis result based on a sequence to be diagnosed provided by an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for determining a diagnosis result based on a sequence to be diagnosed provided by an embodiment of the present application includes the following steps.
[0068] S50, inputting the sequence to be diagnosed into a pre-trained prediction model to obtain a corresponding prediction result, wherein the prediction result includes a plurality of predicted defect categories and a probability corresponding to each predicted defect category.
[0069] In one embodiment of the present application, the second performance data includes at least one dimension. When the second performance data includes multiple dimensions, the corresponding sequence to be diagnosed also includes multiple dimensions. The data to be diagnosed can be input into a pre-trained prediction model to obtain a corresponding prediction result. The prediction result is used to characterize the defect category corresponding to the defect that may exist in the model to be detected, and the probability corresponding to each defect category.
[0070] The prediction model may be a BERT model, a convolutional neural network model or a recurrent neural network model, which is not limited in this application. Exemplary defect categories include: long-term freezing, low efficiency of CPU pre-processing or CPU post-processing, and insufficient utilization of hardware resources.
[0071] S51, determining the diagnosis result according to the prediction result.
[0072] In one embodiment of the present application, when the probability corresponding to the predicted defect category is greater than a preset second threshold, it is determined that the model to be detected has the corresponding defect category.
[0073] For example, the prediction result may be [long-term lag, 0.8], or [CPU pre-processing or CPU post-processing efficiency is too low, 0.7], or [insufficient hardware resource utilization, 0.3]. When the second threshold is 0.7, it can be determined that the model to be tested has the defects of "long-term lag" and "CPU pre-processing or CPU post-processing efficiency is too low".
[0074] See also Figure 6 , Figure 6 6 is a functional module diagram of a model diagnosis device provided in an embodiment of the present application. A model diagnosis device 61 includes an acquisition module 610, a determination module 611, a division module 612, a preprocessing module 613 and a diagnosis module 614. The module / unit referred to in the present application refers to a series of computer-readable instruction segments that can be executed by the processor 13 and can complete fixed functions, which are stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.
[0075] The acquisition module 610 is used to acquire second performance data of the electronic device within a preset time period when the first performance data of the electronic device runs the model to be detected, so as to obtain a data sequence.
[0076] The determination module 611 is used to determine a preset time length for dividing the data sequence based on the second performance data.
[0077] The division module 612 is used to divide the data sequence based on the preset time length to obtain multiple sub-data sequences.
[0078] The preprocessing module 613 is used to preprocess the multiple sub-data sequences to obtain a sequence to be diagnosed.
[0079] The diagnosis module 614 is used to determine the diagnosis result of the model to be detected based on the data sequence and / or the sequence to be diagnosed.
[0080] The determination module 611 determines the preset time length for dividing the data sequence based on the second performance data, including: determining the first data sequence in the second performance data, wherein the first data sequence includes multiple first data points and corresponding timestamps; determining the autocorrelation coefficient of the first data point corresponding to each timestamp; determining the ratio of each of the autocorrelation coefficients to the autocorrelation coefficient corresponding to the first timestamp; when the ratio is greater than or equal to a preset first threshold, determining the time difference between the timestamp corresponding to the ratio and the first timestamp as the preset time length.
[0081] The preprocessing module 613 preprocesses the multiple sub-data sequences to obtain a sequence to be diagnosed, including: determining the last sub-data sequence according to the order of timestamps from early to late, and determining multiple sub-data sequences other than the last sub-data sequence as multiple sequences to be processed; calculating the mean values corresponding to the data points with the same order in the multiple sequences to be processed; and combining the mean values in order from earliest to latest to obtain the sequence to be diagnosed.
[0082] The diagnostic module 614 determines the diagnostic result of the model to be tested based on the sequence to be diagnosed, including: inputting the sequence to be diagnosed into a pre-trained prediction model to obtain a corresponding prediction result, wherein the prediction result includes multiple predicted defect categories and the probability corresponding to each predicted defect category; and determining the diagnostic result according to the prediction result.
[0083] The diagnosis module 614 determines the diagnosis result of the model to be detected based on the data sequence, including: when the data sequence satisfies a preset second condition, determining that the model to be detected has a defect category corresponding to the second condition.
[0084] See also Figure 7 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 100 includes a memory 12 and a processor 13. The memory 12 is used to store computer-readable instructions, and the processor 13 is used to execute the computer-readable instructions stored in the memory to implement a model diagnosis method described in any of the above embodiments.
[0085] In an embodiment of the present application, the electronic device 100 further includes a bus, and a computer program stored in the memory 12 and executable on the processor 13, such as a model diagnosis program.
[0086] Figure 7 Only the electronic device 100 having the memory 12 and the processor 13 is shown, and those skilled in the art can understand that Figure 7 The structure shown does not constitute a limitation on the electronic device 100 , and the electronic device 100 may include fewer or more components than shown in the figure, or combine some components, or arrange the components differently.
[0087] Combination Figure 2 The memory 12 in the electronic device 100 stores a plurality of computer-readable instructions to implement a model diagnosis method, and the processor 13 can execute the plurality of instructions to implement: when the first performance data of the electronic device meets a preset first condition, collect the second performance data of the electronic device within a preset time length when the electronic device is running the model to be detected to obtain a data sequence; determine the preset time length for dividing the data sequence based on the second performance data; divide the data sequence based on the preset time length to obtain a plurality of sub-data sequences; pre-process the plurality of sub-data sequences to obtain a sequence to be diagnosed; and determine the diagnosis result of the model to be detected based on the data sequence and / or the sequence to be diagnosed.
[0088] Specifically, the specific implementation method of the processor 13 for the above instructions can refer to Figure 2 The description of the relevant steps in the corresponding embodiments will not be repeated here.
[0089] Those skilled in the art will appreciate that the schematic diagram is merely an example of the electronic device 100 and does not constitute a limitation on the electronic device 100. The electronic device 100 may be a bus-type structure or a star-type structure. The electronic device 100 may also include more or less other hardware or software than shown in the diagram, or a different arrangement of components. For example, the electronic device 100 may also include input and output devices, network access devices, etc.
[0090] It should be noted that the electronic device 100 is only an example, and other existing or future electronic products that are suitable for the present application should also be included in the protection scope of the present application and included here by reference.
[0091] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium can be non-volatile or volatile. The readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 100, such as a mobile hard disk of the electronic device 100. In other embodiments, the memory 12 can also be an external storage device of the electronic device 100, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 100. The memory 12 can not only be used to store application software and various types of data installed in the electronic device 100, such as a code of a model diagnostic program, etc., but also can be used to temporarily store data that has been output or is to be output.
[0092] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 13 is the control core (Control Unit) of the electronic device 100, and uses various interfaces and lines to connect various components of the entire electronic device 100, and executes or executes programs or modules stored in the memory 12 (for example, executes a model diagnosis program, etc.), and calls data stored in the memory 12 to execute various functions of the electronic device 100 and process data.
[0093] The processor 13 executes the operating system of the electronic device 100 and various installed applications. The processor 13 executes the applications to implement the steps in each of the above-mentioned model diagnosis method embodiments, for example Figure 2 Steps shown.
[0094] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the electronic device 100. For example, the computer program may be divided into an acquisition module 610, a determination module 611, a division module 612, a preprocessing module 613, and a diagnosis module 614.
[0095] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute a part of a model diagnosis method described in each embodiment of the present application.
[0096] If the module / unit integrated in the electronic device 100 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also instruct the relevant hardware devices to complete through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented.
[0097] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory and other memory, etc.
[0098] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.
[0099] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. Figure 7 Only one arrow is used in the figure, but it does not mean that there is only one bus or one type of bus. The bus is configured to realize the connection and communication between the memory 12 and at least one processor 13, etc.
[0100] An embodiment of the present application also provides a computer-readable storage medium (not shown), in which computer-readable storage medium is stored computer-readable instructions, and the computer-readable instructions are executed by a processor in an electronic device to implement the model diagnosis method described in any of the above embodiments.
[0101] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.
[0102] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0103] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.
[0104] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the specification can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.
[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.
Claims
1. A model diagnosis method, applied to electronic equipment, characterized in that: The method comprises: When the first performance data of the electronic device meets a preset first condition, collecting second performance data of the electronic device within a preset time period when the model to be detected is run, to obtain a data sequence; Determining a preset time length for dividing the data sequence based on the second performance data; Divide the data sequence based on the preset time length to obtain multiple sub-data sequences; Preprocessing the multiple sub-data sequences to obtain a sequence to be diagnosed; The diagnosis result of the model to be detected is determined based on the data sequence and / or the sequence to be diagnosed.
2. The model diagnosis method according to claim 1, characterized in that: The determining, based on the second performance data, a preset duration for dividing the data sequence includes: Determine a first data sequence in the second performance data, wherein the first data sequence includes a plurality of first data points and corresponding timestamps; Determine the autocorrelation coefficient of the first data point corresponding to each timestamp; Determine the ratio of each of the autocorrelation coefficients to the autocorrelation coefficient corresponding to the first timestamp; When the ratio is greater than or equal to a preset first threshold, the time difference between the timestamp corresponding to the ratio and the first timestamp is determined to be the preset duration.
3. The model diagnosis method according to claim 1, characterized in that: The preprocessing of the plurality of sub-data sequences to obtain a sequence to be diagnosed comprises: Determine the last sub-data sequence according to the order of timestamps from earliest to latest, and determine multiple sub-data sequences other than the last sub-data sequence as multiple sequences to be processed; Calculating the mean values corresponding to the data points having the same order in the plurality of sequences to be processed; The sequence to be diagnosed is obtained by combining the average values in order from first to last.
4. The model diagnosis method according to claim 1, characterized in that: Determining the diagnosis result of the model to be detected based on the sequence to be diagnosed includes: Inputting the sequence to be diagnosed into a pre-trained prediction model to obtain a corresponding prediction result, wherein the prediction result includes a plurality of predicted defect categories and a probability corresponding to each predicted defect category; The diagnosis result is determined according to the prediction result.
5. The model diagnosis method according to claim 4, characterized in that: Determining the diagnosis result according to the prediction result includes: when the probability corresponding to the predicted defect category is greater than a preset second threshold, determining that the model to be detected has the corresponding defect category.
6. The model diagnosis method according to claim 1, characterized in that: Determining the diagnosis result of the model to be detected based on the data sequence includes: When the data sequence satisfies a preset second condition, it is determined that the model to be inspected has a defect category corresponding to the second condition.
7. The model diagnosis method according to claim 6, characterized in that: The preset second condition includes one or more of the following: The ratio of the number of calls of the video memory allocation interface in the data sequence to the number of calls of all graphics card interfaces is greater than a preset third threshold; The ratio of the number of calls to the video memory release interface in the data sequence to the number of calls to all graphics card interfaces is greater than the preset third threshold; The ratio of the graphics card calculation time in the data sequence to the preset collection time is less than a preset fourth threshold.
8. A model diagnosis device, characterized in that: The device comprises: A collection module, configured to collect second performance data of the electronic device within a preset time period when the first performance data of the electronic device runs the model to be detected, to obtain a data sequence, when the first performance data of the electronic device meets a preset first condition; A determination module, configured to determine a preset time length for dividing the data sequence based on the second performance data; A division module, used for dividing the data sequence based on the preset time length to obtain multiple sub-data sequences; A preprocessing module, used for preprocessing the plurality of sub-data sequences to obtain a sequence to be diagnosed; A diagnosis module is used to determine the diagnosis result of the model to be detected based on the data sequence and / or the sequence to be diagnosed.
9. An electronic device, characterized in that: The electronic device comprises a processor and a memory, and the processor is used to implement the model diagnosis method according to any one of claims 1 to 7 when executing a computer program stored in the memory.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the model diagnosis method according to any one of claims 1 to 7 is implemented.