Model training method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The application provides a model training method and device, electronic equipment, computer program product and computer readable storage medium; the method comprises: acquiring a plurality of abnormal events for heterogeneous computing resources, and determining abnormal information and event categories corresponding to each abnormal event; for each abnormal event meeting the matching processing condition, the abnormal information and event categories of the abnormal event are matched to obtain a matching result; based on the matching result of each abnormal event, at least one target event is determined from the plurality of abnormal events; based on the abnormal information corresponding to each target event, a first training data set is constructed; and the trained abnormal detection model is obtained by training the abnormal detection model to be trained using the first training data set. Through the application, the training efficiency and accuracy of the model can be improved, and through the trained abnormal detection model, automatic detection of the abnormality of the heterogeneous computing resources can be realized, thereby improving the detection efficiency and accuracy.

Description

Technical Field

[0001] This application relates to computer technology, and more particularly to a model training method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Due to the unique architecture and operating mode of heterogeneous computing devices, the frequency of failures during operation is significantly higher than that of central processing unit (CPU) computing devices, and the types of failures in heterogeneous computing devices are also numerous. Related technologies typically employ detection and alarm methods to analyze and handle faults in heterogeneous computing devices; that is, after a fault occurs and an alarm is triggered, manual diagnosis and analysis are relied upon. However, relying solely on manual diagnosis and analysis of each fault is insufficient for quickly identifying the root cause of the anomaly, resulting in very low processing efficiency and severely limiting fault handling efficiency. Summary of the Invention

[0003] This application provides a model training method, apparatus, computer-readable storage medium, and computer program product, which can improve the training efficiency and accuracy of the model. Furthermore, the trained anomaly detection model can automatically detect anomalies in heterogeneous computing resources, thereby improving detection efficiency and accuracy.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a model training method, the method comprising:

[0006] Acquire multiple abnormal events for heterogeneous computing resources, and determine the abnormal information and event category corresponding to each abnormal event;

[0007] For each abnormal event that meets the matching processing conditions, the abnormal information and event category of the abnormal event are matched to obtain the matching result;

[0008] Based on the matching result of each of the aforementioned anomalous events, at least one target event is determined from the plurality of anomalous events;

[0009] Based on the anomaly information corresponding to each target event, a first training dataset is constructed.

[0010] The anomaly detection model to be trained is trained using the first training dataset to obtain the trained anomaly detection model.

[0011] This application provides a model training apparatus, including:

[0012] The acquisition module is used to acquire multiple abnormal events for heterogeneous computing resources and determine the abnormal information and event category corresponding to each abnormal event.

[0013] The matching module is used to perform matching processing on the abnormal information and event category of each abnormal event that meets the matching processing conditions, and obtain the matching result.

[0014] The determination module is used to determine at least one target event from a plurality of abnormal events based on the matching result of each of the abnormal events;

[0015] A construction module is used to construct a first training dataset based on the anomaly information corresponding to each of the target events;

[0016] The training module is used to train the anomaly detection model to be trained using the first training dataset, so as to obtain the trained anomaly detection model.

[0017] This application provides an electronic device, the electronic device comprising:

[0018] Memory is used to store executable instructions or computer programs.

[0019] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the model training method provided in the embodiments of this application.

[0020] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the model training method provided in this application when executed by a processor.

[0021] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the model training method provided in this application.

[0022] The embodiments of this application have the following beneficial effects:

[0023] In this embodiment, multiple abnormal events related to heterogeneous computing resources are acquired, and the abnormal information and event category corresponding to each abnormal event are determined. For each abnormal event that meets the matching processing conditions, the abnormal information and event category of the abnormal event are matched to obtain a matching result. Based on the matching result of each abnormal event, at least one target event is determined from the multiple abnormal events. This allows for accurate determination of the required target event from multiple abnormal events of heterogeneous computing resources based on the matching processing conditions and matching results, thereby improving the efficiency and accuracy of target event determination. Subsequently, a first training dataset is constructed based on the abnormal information corresponding to each target event, which improves the accuracy and quality of the training data in the first training dataset and ensures that the training data comprehensively covers various abnormal situations. Finally, the anomaly detection model to be trained is trained using the first training dataset to obtain the trained anomaly detection model, which improves the training efficiency and model accuracy. Furthermore, the trained anomaly detection model enables automatic detection of anomalies in heterogeneous computing resources, thereby reducing the cost of detecting anomalies in heterogeneous computing resources and improving the efficiency and accuracy of detection. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the architecture of the model training system provided in the embodiments of this application;

[0025] Figure 2 This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application;

[0026] Figure 3A This is a flowchart illustrating the model training method provided in the embodiments of this application;

[0027] Figure 3B This is a flowchart illustrating the process of determining abnormal information and event categories provided in an embodiment of this application;

[0028] Figure 3C This is a schematic diagram of the process for determining whether the matching conditions are met, provided in an embodiment of this application.

[0029] Figure 3D This is a schematic diagram of the process for constructing the first training dataset provided in an embodiment of this application;

[0030] Figure 3E This is a flowchart illustrating the trained anomaly detection model provided in an embodiment of this application.

[0031] Figure 3F This is a schematic diagram of the process for obtaining the initial anomaly detection model provided in an embodiment of this application;

[0032] Figure 3GThis is another flowchart illustrating the trained anomaly detection model provided in an embodiment of this application;

[0033] Figure 4 This is a schematic diagram showing the distribution of different types of faults and anomalies in heterogeneous computing power provided in the embodiments of this application;

[0034] Figure 5 This is a schematic diagram of the system architecture of the artificial intelligence computing engineering provided in the embodiments of this application;

[0035] Figure 6 This is a schematic diagram of the intelligent assistant training process provided in an embodiment of this application;

[0036] Figure 7 This is another schematic diagram of the intelligent assistant training process provided in the embodiments of this application;

[0037] Figure 8 This is a schematic diagram of the composition structure of the heterogeneous computing platform provided in the embodiments of this application;

[0038] Figure 9 This is another flowchart illustrating the training process for an intelligent assistant provided in this application embodiment;

[0039] Figure 10 This is a flowchart illustrating an abnormal event analysis provided in an embodiment of this application;

[0040] Figure 11 This is a schematic diagram of an abnormal event screening process provided in an embodiment of this application;

[0041] Figure 12 This is another schematic diagram of the process for constructing the first training dataset provided in the embodiments of this application;

[0042] Figure 13 This is a schematic diagram of the iterative training process of the intelligent assistant provided in the embodiments of this application;

[0043] Figure 14 This is another flowchart illustrating the intelligent assistant training method provided in the embodiments of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0046] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0047] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0048] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0049] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0050] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0051] 1) Heterogeneous computing power: Heterogeneous computing power refers to the ability to use processors of different types or architectures in the same system to work together to complete computing tasks. It usually combines central processing units, graphics processing units, tensor processing units and other dedicated processors.

[0052] 2) Intelligent Assistant: A software application that utilizes artificial intelligence technology to interact with users, provide information, perform tasks, or assist in decision-making. In this embodiment, the intelligent assistant represents an anomaly detection model used to diagnose and analyze anomalies or faults in heterogeneous computing power;

[0053] 3) Abnormal events: These refer to unexpected or unusual events that occur during normal operation or behavior, deviating from expected standards or patterns, and may indicate errors, security issues, or other significant problems in the system. In the embodiments of this application, abnormal events are used to represent faults or anomalies occurring in heterogeneous computing power.

[0054] Due to the unique architecture and operating mode of heterogeneous computing power devices, malfunctions frequently occur during operation, and these malfunctions are diverse. Related technologies typically employ detection and alarm methods to analyze and handle these malfunctions, relying on manual diagnosis and analysis after a malfunction occurs and an alarm is triggered. However, because heterogeneous computing power devices have a high failure rate and a wide variety of anomalies, manual anomaly localization and analysis are inefficient, severely hindering the maintenance efficiency of faulty devices.

[0055] This application provides a model training method, apparatus, device, computer-readable storage medium, and computer program product, which can improve the training efficiency and accuracy of the model. Furthermore, the trained anomaly detection model can automatically detect anomalies in heterogeneous computing resources, thereby improving detection efficiency and accuracy. The following describes exemplary applications of the electronic devices provided in this application. These devices can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals, or as servers. The following describes exemplary applications when the device is implemented as a server.

[0056] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the model training system provided in this application embodiment. Terminal 400 connects to server 200 via network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both. Database 100 can be independent of server 200 or deployed on server 200. Figure 1 The database 100 is shown as an example, independent of the server 200.

[0057] Server 200 is equipped with heterogeneous computing resources for operations such as big data analysis, graphics rendering, and training machine learning models. During the processing of task requests using these heterogeneous computing resources, abnormal events frequently occur. Therefore, rapid diagnosis of these abnormal events is necessary to improve fault repair efficiency. In this embodiment, server 200 acquires multiple abnormal events related to the heterogeneous computing resources and determines the abnormal information and event category corresponding to each abnormal event. For each abnormal event that meets the matching processing conditions, the abnormal information and event category of the abnormal event are matched to obtain a matching result. Based on the matching result of each abnormal event, at least one target event is identified from the multiple abnormal events. Based on the abnormal information corresponding to each target event, a first training dataset is constructed. The first training dataset is used to train the anomaly detection model to be trained, resulting in a trained anomaly detection model. The trained anomaly detection model is stored in database 100. When a user needs to diagnose and process newly emerging abnormal events related to heterogeneous computing resources, they can initiate a prediction request on terminal 400. The prediction request carries the data to be processed, and terminal 400 sends the prediction request to server 200 via network 300. After receiving a prediction request, server 200 calls the trained anomaly detection model from database 100 to perform prediction processing on the data to be processed carried in the prediction request, and obtains the first prediction result, thereby completing the diagnosis and analysis of the abnormal event.

[0058] Taking the server mentioned above as an example, which is the electronic device used for model training, see [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application. Figure 2 The server 200 shown includes at least one processor 210, memory 230, and at least one network interface 220. The various components of server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 240.

[0059] The processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0060] The memory 230 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 230 may optionally include one or more storage devices physically located away from the processor 210.

[0061] The memory 230 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 230 described in this application embodiment is intended to include any suitable type of memory.

[0062] In some embodiments, memory 230 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0063] Operating system 231 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0064] The network communication module 232 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including Bluetooth, WiFi, and Universal Serial Bus (USB).

[0065] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A model training device 233 stored in memory 230 is shown. This device can be software in the form of programs and plugins, and includes the following software modules: an acquisition module 2331, a matching module 2332, a determination module 2333, a construction module 2334, and a training module 2335. These modules are logically connected and can therefore be arbitrarily combined or further split according to their implemented functions. The functions of each module will be described below.

[0066] In other embodiments, the apparatus provided in this application can be implemented in hardware. For example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the model training method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0067] The model training method provided in this application will be described in conjunction with exemplary applications and implementations of the terminals provided in the embodiments of this application.

[0068] The following describes the model training method provided in the embodiments of this application. As mentioned above, the electronic device that implements the model training method in the embodiments of this application can be a server. Therefore, the execution subject of each step will not be described again below.

[0069] It should be noted that the model training examples below are based on an anomaly detection model under heterogeneous computing resources. Those skilled in the art can apply the model training method provided in the embodiments of this application to the training of other types of models based on their understanding of the following text.

[0070] See Figure 3A , Figure 3A This is a flowchart illustrating the model training method provided in the embodiments of this application, which will be combined with... Figure 3A The steps shown are explained.

[0071] In step 101, multiple abnormal events for heterogeneous computing resources are obtained, and the abnormal information and event category corresponding to each abnormal event are determined.

[0072] Here, heterogeneous computing resources are resources composed of different types of devices, typically including computing devices, network devices, and storage devices, which work together to provide efficient and flexible computing services. The computing units can include different types of computing units, such as central processing units (CPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), and application-specific integrated circuits (ASICs). Heterogeneous computing resources can support training services for mixed-mode models, visual models, speech models, game models, medical models, and language models. However, during model training, due to the special architecture and operating modes of heterogeneous computing devices, failures frequently occur; these failures are called anomalies. Anomalies can be collected by configuring logs on each device, or by detecting system performance indicators, triggering alarms and reporting anomalies when they are detected, and extracting the anomaly information from the reported information. Anomaly information describes the anomaly event and can include the anomaly phenomenon, the cause of the anomaly, the anomaly diagnosis result, the anomaly handling strategy, and the anomaly feedback result. Among them, the anomaly phenomenon describes the phenomenon of an abnormal event, such as a decrease in model training performance; the anomaly cause is the possible reason for the occurrence of the anomaly after analyzing the anomaly phenomenon, such as abnormal communication performance; the anomaly diagnosis result is a further analysis of the anomaly cause to determine the specific cause of the anomaly, such as packet loss in the network; the anomaly handling strategy is the method and strategy for handling the anomaly based on the anomaly cause, with the aim of resolving the anomaly event, such as expanding network bandwidth; the anomaly feedback result is the processing result fed back by the system after the anomaly handling strategy is executed. The anomaly feedback result can be processing result information, such as "anomaly restored to normal" or "anomaly not restored to normal"; or, the anomaly feedback result can also be the phenomenon information after processing, such as "training performance restored to normal". The event category is used to characterize the category to which the anomaly event belongs. Since the devices corresponding to heterogeneous computing power resources include computing devices, network devices, and storage devices, the location where the anomaly event occurs also includes computing devices, network devices, and storage devices. Therefore, anomaly events can be classified according to computing devices, network devices, and storage devices. That is to say, the event category can include computing anomaly category, network anomaly category, and storage anomaly category.

[0073] In some embodiments, see Figure 3B Step 101 can be implemented through steps 1011 to 1013, including:

[0074] In step 1011, the event category corresponding to the abnormal event is determined based on the resource category corresponding to the abnormal event.

[0075] Here, resource categories include computing resources, network resources, and storage resources. Computing resources correspond to computing devices, indicating that the abnormal event occurred on a computing device, thus classifying the event as a computing anomaly. Network resources correspond to network devices, indicating that the abnormal event occurred on a network device, thus classifying the event as a network anomaly. Storage resources correspond to storage devices, indicating that the abnormal event occurred on a storage device, thus classifying the event as a storage anomaly. Resource categories can be determined through manual inspection by operations and maintenance personnel or based on the anomaly reporting information corresponding to the abnormal event.

[0076] In step 1012, the abnormal events are analyzed and processed based on the event category to obtain analysis results, which include at least one feature information.

[0077] Here, since the anomaly analysis approach and handling scheme differ for each event category, targeted analysis and handling can be performed based on the event category when analyzing and handling abnormal events. For example, if the event category is "computational anomaly," it means the abnormal event occurred on a computing device, so the anomaly event can be analyzed and handled based on the status of each computing device. Analysis and handling include analyzing the cause of the anomaly based on the anomaly phenomenon, further determining the anomaly diagnosis result, then determining the anomaly handling strategy based on the anomaly diagnosis result, and obtaining the anomaly feedback result after the anomaly handling strategy is executed. Therefore, the analysis result typically includes the anomaly phenomenon, the cause of the anomaly, the anomaly diagnosis result, the anomaly handling strategy, the anomaly feedback result, and other relevant information. Among these, the anomaly phenomenon, the cause of the anomaly, the anomaly diagnosis result, the anomaly handling strategy, and the anomaly feedback result are characteristic information. However, in special cases, the analysis result may have missing information, such as missing one or more of the "anomaly phenomenon, the cause of the anomaly, the anomaly diagnosis result, the anomaly handling strategy, and the anomaly feedback result." This is because, in real-world scenarios, initial analysis of abnormal events may reveal errors, such as a system malfunction being detected but an abnormal event being reported when no actual malfunction has occurred. Alternatively, the abnormal event might be a warning message indicating a potential risk, but no real problem has materialized. In such cases, analysis and processing of the abnormal event will cease, and no handling strategy will be developed or implemented. Consequently, the analysis results will lack key information. Furthermore, data storage anomalies or staff errors could lead to the loss of one or more pieces of information related to the abnormal phenomenon, its cause, diagnostic results, handling strategies, or feedback, again resulting in missing key information in the analysis results.

[0078] In step 1013, a preset template is obtained.

[0079] Here, the preset template is used to integrate and process abnormal information. The preset template includes multiple positions for filling feature information.

[0080] In step 1014, each feature information in the abnormal information is filled into the corresponding position in the preset template to obtain the abnormal information.

[0081] Here, each feature has a corresponding position in the preset template, so each feature can be filled into the corresponding position in the preset template. For example, the feature corresponding to "abnormal phenomenon" is "model training failure". Filling the feature into the corresponding position in the preset template will give the abnormal information "abnormal phenomenon: model training failure".

[0082] In this embodiment, the event category corresponding to the abnormal event is determined based on the resource category corresponding to the abnormal event; the abnormal event is analyzed and processed based on the event category to obtain analysis results, which include at least one feature information; a preset template is obtained, which is used to integrate and process the abnormal information; each feature information in the abnormal information is filled into the corresponding position in the preset template to obtain the abnormal information. Thus, determining the event category based on the resource category makes it easy to intuitively see the location of the abnormal event through the event category, and integrating the abnormal information through the preset template facilitates the analysis and retrieval of the abnormal information. Therefore, the management efficiency of abnormal events is improved, and the convenience and efficiency of matching abnormal information and event categories are enhanced.

[0083] In step 102, for each abnormal event that meets the matching processing conditions, the abnormal information and event category of the abnormal event are matched to obtain the matching result.

[0084] Here, the abnormal events are first screened to remove those with errors or missing information, leaving only those that meet the matching criteria. Then, for each matching event, the abnormal information and event category are matched. The purpose of this matching process is to analyze whether the abnormal phenomena, causes, diagnostic results, handling strategies, and feedback results within the abnormal information are logically consistent. For example, if the cause is insufficient storage space, and the handling strategy is to expand network bandwidth, then the cause and strategy are illogical, and the match fails. The event category serves as the basis for matching. For instance, if the event category is "computational anomaly," it indicates the anomaly occurred on a computing device, so the cause should also be related to the computing device; that is, the cause and event category should match. Similarly, the abnormal phenomena, diagnostic results, handling strategies, and feedback results should also match the event category. When every feature in the abnormal information matches the event category, each feature in the abnormal information can also match with each other.

[0085] In some embodiments, the anomaly information includes at least one feature information describing the anomaly event. See also Figure 3C Steps 201 to 203 can be used to determine if an abnormal event meets the matching processing conditions, including:

[0086] In step 201, for each abnormal event, the number of feature information included in the abnormal information is determined.

[0087] Here, complete anomaly information includes the anomaly phenomenon, the cause of the anomaly, the diagnostic result of the anomaly, the handling strategy for the anomaly, and the feedback result of the anomaly—that is, five characteristic pieces of information. However, as mentioned above, in real-world scenarios, characteristic information may be missing, in which case the number of characteristic pieces of information will be less than five. Therefore, the number of characteristic pieces of information is an integer greater than 0 and less than or equal to 5. By determining the number of characteristic pieces of information included in the anomaly information, we can judge whether the anomaly information is complete. Understandably, the number of characteristic pieces of information included in the anomaly information can vary depending on different application scenarios. For example, if the anomaly information is set to include seven characteristic pieces of information when complete, then the number of characteristic pieces of information will be an integer greater than 0 and less than or equal to 7.

[0088] In step 202, when the number of feature information is a preset number, the status change information of the heterogeneous computing resources corresponding to the abnormal event is determined.

[0089] Here, the preset quantity represents the number of feature information included in a complete anomaly report. This preset quantity can be flexibly set according to different application scenarios. As mentioned above, a complete anomaly report includes 5 feature information items, so the preset quantity is 5. When the number of feature information items is not the preset quantity, it indicates that the anomaly report is missing, and the anomaly event is discarded. When the number of feature information items is the preset quantity, it indicates that the anomaly report is complete, and the next step is to determine the state change information of the heterogeneous computing resources corresponding to the anomaly event. State change information can characterize the state changes of heterogeneous computing resources over a period of time and can include historical state information and current state information. Historical state information is used to characterize the state of the heterogeneous computing resources when the anomaly event is reported. It can be used to determine whether the anomaly event was reported correctly. If the historical state information indicates that the heterogeneous computing resources are abnormal, the anomaly event was reported accurately; if it indicates that the heterogeneous computing resources are normal, the anomaly event was reported incorrectly. Current state information characterizes the latest state of the heterogeneous computing resources after the anomaly handling strategy is implemented, and this can be determined through the anomaly feedback results. If the current status information indicates that the heterogeneous computing resources are normal, it means that the anomaly has been recovered and the anomaly handling strategy is effective. If the current status information indicates that the heterogeneous computing resources are still abnormal, it means that the anomaly has not been recovered and the anomaly handling strategy is ineffective.

[0090] In step 203, when the state change information indicates that the state of the heterogeneous resource has changed from abnormal to normal, it is determined that the abnormal event meets the matching processing conditions.

[0091] Here, when the state change information indicates that the heterogeneous resource's state has changed from abnormal to normal, it means that the abnormal event was reported correctly and has been effectively resolved. At this point, it can be determined that the abnormal event meets the matching processing conditions. If the state change information does not indicate that the heterogeneous resource's state has changed from abnormal to normal, it means that the abnormal event was reported incorrectly or that the abnormal event has not yet been resolved. In this case, the abnormal event is discarded. This allows for the filtering of abnormal events, retaining only high-quality and valid ones.

[0092] In this embodiment, the anomaly information includes at least one feature information describing the anomaly event. For each anomaly event, the number of feature information included in the anomaly information is determined. When the number of feature information is a preset number, the state change information of the heterogeneous computing resources corresponding to the anomaly event is determined. When the state change information indicates that the state of the heterogeneous resource has changed from abnormal to normal, the anomaly event is determined to meet the matching processing conditions. In this way, reported errors or anomaly events that have not been resolved temporarily can be filtered out, and only high-quality and effective anomaly events can be retained. This achieves the cleaning and filtering of anomaly events, improves the accuracy and efficiency of the identified anomaly events that meet the matching processing conditions, and thus improves the efficiency of constructing the first training dataset to train the anomaly detection model, and improves the prediction accuracy of the trained anomaly detection model.

[0093] In some embodiments, step 102 can be implemented by the following process: for each feature in the abnormal information, perform matching processing on the feature information and the event category; when each feature matches the event category, determine the matching result as a successful match; or, when any feature does not match the event category, determine the matching result as a failed match.

[0094] Here, the event category can be used as the basis for matching. For example, if the event category corresponding to an abnormal event is the computational anomaly category, it means the abnormal event occurred on a computing device. Therefore, every feature in the anomaly information should be related to the computing device, i.e., match the computational anomaly category. For instance, a computing device malfunction might cause model training failure. The abnormal phenomenon could be model training failure, the cause could be computing device malfunction, the diagnostic result could be a power outage on the computing device, the handling strategy could be powering on the computing device, and the feedback result could be the computing device being powered on and the model training successfully completed. In this case, every feature in the anomaly information matches the event category, and each feature also matches the others, so the matching result is considered successful. If any feature does not match the event category—for example, if the event category corresponding to the abnormal event is the computational anomaly category, and the handling strategy is to expand the storage device—then the handling strategy is incorrect, and the matching result is considered a failure.

[0095] In this embodiment, for each feature in the abnormal information, a matching process is performed between the feature and the event category. If every feature matches the event category, the matching result is considered successful; otherwise, if any feature does not match the event category, the matching result is considered unsuccessful. Thus, a successful match is only determined when the abnormal phenomenon, cause, diagnosis, handling strategy, and feedback result in the abnormal information all conform to the matching logic with the event category. This improves the accuracy of the matching process and ensures the efficiency and accuracy of screening abnormal events.

[0096] In step 103, based on the matching results of each anomalous event, at least one target event is determined from the multiple anomalous events.

[0097] Here, a target event refers to an anomaly event that is accurately reported, has complete anomaly information, and whose anomaly information matches the event category according to the matching logic. Based on the matching results of each anomaly event, at least one high-quality and accurate target event can be identified from multiple anomalies.

[0098] In some embodiments, for each abnormal event, when the matching result is successful, the abnormal event is identified as the target event. This ensures that the abnormal event is identified as the target event only when the abnormal phenomenon, abnormal cause, abnormal diagnosis result, abnormal handling strategy, and abnormal feedback result in the abnormal information all match the event category, thereby improving the efficiency and accuracy of identifying the target event.

[0099] In step 104, a first training dataset is constructed based on the anomaly information corresponding to each target event.

[0100] Here, the abnormal phenomenon, abnormal cause, abnormal handling strategy, and abnormal feedback result in the abnormal information of each target event can all be used as sample data, and the abnormal diagnosis result in the abnormal information can be used as label information to construct the first training dataset. Alternatively, one or more of the abnormal phenomenon, abnormal cause, abnormal handling strategy, and abnormal feedback result can be used as sample data, and the abnormal diagnosis result can be used as label information to construct the first training dataset.

[0101] In some embodiments, see Figure 3D Step 104 can be implemented through steps 1041 to 1043, including:

[0102] In step 1041, for each target event's abnormal information, the abnormal description information and abnormal detection results of the target event are determined.

[0103] Here, anomaly description information is used to describe anomalous events, and anomaly detection results are used to characterize the anomaly diagnosis results of anomalous events. Alternatively, the anomaly detection results may also include methods for handling anomalous events. Specifically, the anomalous phenomenon, anomalous cause, anomalous handling strategy, and anomalous feedback result from the anomaly information can be directly used as the anomaly description information; or, one or more of the anomalous phenomenon, anomalous cause, anomalous handling strategy, and anomalous feedback result can be used as the anomaly description information; or, one or more of the anomalous phenomenon, anomalous cause, anomalous handling strategy, and annomalous feedback result can be given a general description to generate the anomaly description information. For anomaly detection results, the anomaly diagnosis result from the anomaly information can be used as the anomaly detection result; the anomaly diagnosis result and the anomaly handling strategy from the anomaly information can also be used as the anomaly detection result; or, the anomaly diagnosis result and the anomaly handling strategy from the anomaly information can be given a general description to generate the anomaly detection result.

[0104] In step 1042, the anomaly description information is used as sample data and the anomaly detection results are used as label information to construct training data corresponding to the target event.

[0105] Here, sample data refers to each individual data point in the dataset used to train the machine learning model. Sample data contains several features that describe the attributes or characteristics of the anomalous event. Label information corresponds to the features in the sample data; each sample data point has a corresponding label, which is the result the model needs to predict. By using the anomalous description information of each target event as sample data and the anomaly detection results as label information, the training data corresponding to that target event is constructed.

[0106] In step 1043, a first training dataset is constructed based on the training data corresponding to each target event.

[0107] Here, the training data corresponding to each target event can be directly used to construct the first training dataset. Alternatively, differentiated annotation processing can be performed for certain specific target events, that is, determining the training weights corresponding to each training data and labeling the training data to obtain accurate and differentiated labeled training data, and then constructing the first training dataset based on the labeled training data. Alternatively, training data corresponding to multiple identical target events can be deleted or merged, and then the processed training data can be used to construct the first training dataset, thereby improving the quality of the first training dataset and reducing storage costs and training cycle.

[0108] In this embodiment, for each target event's anomaly information, anomaly description information and anomaly detection results are determined; the anomaly description information is used as sample data, and the anomaly detection results are used as label information to construct training data corresponding to the target event; based on the training data corresponding to each target event, a first training dataset is constructed. This improves the accuracy of the first training dataset and ensures that it covers more types of anomaly events, thus providing comprehensiveness and richness.

[0109] In some embodiments, the first training dataset can be constructed through the following process: determining the training weights of each training data point, labeling the training data based on the training weights to obtain labeled training data; and constructing the first training dataset from each labeled training data point.

[0110] Here, the training weights are positive numbers greater than 0. The upper limit typically depends on the importance or difficulty of the training data and is not fixed. Training weights represent the importance or difficulty of the training data; a larger value indicates higher importance or difficulty. A default value can be preset as the initial training weight, in which case all training data have the default weight, such as 1. Then, training data with importance or difficulty exceeding the preset condition can be identified through analysis. Examples include data with difficult-to-analyze fault causes, less frequent occurrences than a preset frequency, or specific types of training data. Then, for each piece of training data, a training weight is determined based on its importance or difficulty. Higher importance or difficulty results in a higher training weight, and the weight must be greater than the default value. For example, training weights can be set to 1.5, 3, 5, etc. Alternatively, the training weights can be normalized so that the sum of the training weights of all training data is 1, resulting in a training weight between 0 and 1. Labeling the training data using these weights creates differentiated labeled training data, which is then used as the first training dataset.

[0111] In this embodiment, the training weights of each training data point are determined, and the training data are labeled based on these weights to obtain labeled training data. Each labeled training data point is then used to construct a first training dataset. This allows for differentiated labeling of the first training dataset, resulting in an accurate and differentiated first training dataset, thus improving the accuracy of the first training dataset and the model training effect.

[0112] In step 105, the anomaly detection model to be trained is trained using the first training dataset to obtain the trained anomaly detection model.

[0113] Here, the anomaly detection model to be trained refers to a machine learning model that has not yet been trained. The anomaly detection model is used to diagnose and predict anomalies or faults occurring in heterogeneous computing power. By inputting information related to the anomaly, such as the anomaly phenomenon and its cause, it can predict the anomaly diagnosis result and the methods and strategies for handling the anomaly. For each training data point in the first training dataset, the anomaly detection model to be trained performs prediction processing on the sample data in the training dataset to obtain a prediction result. Then, based on the difference between the label information in the training data and the prediction result, a loss value is determined. Based on the loss value, the parameters of the anomaly detection model to be trained are adjusted using a gradient descent algorithm. Then, for the next training data point in the first training dataset, the above steps are repeated sequentially until the training termination condition is met, resulting in the trained anomaly detection model. The training termination condition can be that the loss value is lower than a loss threshold or that the difference between the current loss value and the previous adjacent loss value is less than a preset difference.

[0114] In this embodiment, multiple abnormal events related to heterogeneous computing resources are acquired, and the abnormal information and event category corresponding to each abnormal event are determined. For each abnormal event that meets the matching processing conditions, the abnormal information and event category of the abnormal event are matched to obtain a matching result. Based on the matching result of each abnormal event, at least one target event is determined from the multiple abnormal events. This allows for accurate determination of the required target event from multiple abnormal events of heterogeneous computing resources based on the matching processing conditions and matching results, thereby improving the efficiency and accuracy of target event determination. Subsequently, a first training dataset is constructed based on the abnormal information corresponding to each target event, which improves the accuracy and quality of the training data in the first training dataset and ensures that the training data comprehensively covers various abnormal situations. Finally, the anomaly detection model to be trained is trained using the first training dataset to obtain the trained anomaly detection model, which improves the training efficiency and model accuracy. Furthermore, the trained anomaly detection model enables automatic detection of anomalies in heterogeneous computing resources, thereby reducing the cost of detecting anomalies in heterogeneous computing resources and improving the efficiency and accuracy of detection.

[0115] In some embodiments, see Figure 3E Step 105 can be implemented through steps 1051 to 1054, including:

[0116] In step 1051, the anomaly detection model to be trained is trained using the first training dataset to obtain the initial anomaly detection model.

[0117] Here, the initial anomaly detection model refers to the anomaly detection model obtained after training with the first training dataset. At this time, the initial anomaly detection model can predict and process most anomaly events, for example, it can predict and process more than 90% of anomaly events.

[0118] In some embodiments, see Figure 3F Step 1051 can be implemented through steps 511 to 515, including:

[0119] In step 511, for each labeled training data in the first training dataset, the training weights corresponding to the labeled training data are determined.

[0120] Here, when the training data in the first training dataset is labeled training data, the training weights can be determined based on the labeling scores of the labeled training data, and the anomaly detection model can be trained in a targeted manner using the training weights.

[0121] In step 512, the sample data in the labeled training data is predicted based on the anomaly detection model to be trained, and a second prediction result is obtained.

[0122] Here, the sample data from the labeled training data is used as input to the anomaly detection model to be trained. The model then performs prediction processing to obtain a second prediction result. This second prediction result can be an anomaly diagnosis result for the anomaly event corresponding to the sample data, or it can include anomaly handling strategies formulated for the anomaly event corresponding to the sample data.

[0123] In step 513, the initial loss value is determined based on the label information in the labeled training data and the second prediction result.

[0124] Here, the initial loss value is used to measure the difference between the label information and the second prediction result. The initial loss value can be obtained by using a specified loss function to calculate the second prediction result and the label information.

[0125] In step 514, the product of the initial loss value and the training weights is determined as the first loss value of the anomaly detection model to be trained.

[0126] Here, the first loss value of the anomaly detection model to be trained is obtained by multiplying the initial loss value by the training weights. For example, if the initial loss value is 0.35 and the training weights are 0.2, then the first loss value is 0.07.

[0127] In step 515, the anomaly detection model to be trained is trained using each first loss value to obtain the initial anomaly detection model.

[0128] Here, based on each first loss value, the parameters of the anomaly detection model to be trained are adjusted using the gradient descent algorithm until all first loss values ​​have been traversed, thus obtaining the initial anomaly detection model.

[0129] In this embodiment, for each labeled training data point in the first training dataset, training weights corresponding to the labeled training data are determined; the sample data in the labeled training data are predicted based on the anomaly detection model to be trained, resulting in a second prediction result; an initial loss value is determined based on the label information in the labeled training data and the second prediction result; the product of the initial loss value and the training weights is determined as the first loss value of the anomaly detection model to be trained; the anomaly detection model to be trained is trained using each first loss value to obtain an initial anomaly detection model. In this way, the anomaly detection model to be trained can be specifically trained using the training weights corresponding to the labeled training data, thereby improving the efficiency of model training and the model's generalization ability.

[0130] In step 1052, a second training dataset is obtained, and the initial anomaly detection model is trained using the second training dataset to obtain a candidate anomaly detection model.

[0131] Here, because the first training dataset may not contain training data for some rare anomalous events, the initial anomaly detection model may not accurately predict a few anomalous events. Therefore, the initial anomaly detection model can be retrained using a second training dataset to fine-tune and optimize the model, resulting in a candidate anomaly detection model with a wider prediction range. The second training dataset can be pre-set by technical personnel and contains training data for anomalous events in a specified scenario, supplementing the training data missing from the first training dataset.

[0132] In step 1053, the data to be processed is obtained, and the data to be processed is predicted based on the candidate anomaly detection model to obtain the first prediction result.

[0133] Here, the data to be processed refers to descriptive information corresponding to anomalies or faults occurring in heterogeneous computing power. This information can be related to anomalies or faults input into the candidate anomaly detection model when it is applied online to assist operations personnel in analyzing and diagnosing these anomalies or faults. The candidate anomaly detection model performs predictive processing on the data to be processed and outputs a first prediction result. This first prediction result can be an anomaly diagnosis result corresponding to the anomaly or fault in the data to be processed, or it can include an anomaly handling strategy formulated for the anomaly or fault corresponding to the data to be processed.

[0134] In step 1054, the trained anomaly detection model is obtained based on the first prediction result and the candidate anomaly detection model.

[0135] Here, if the difference between the first predicted result and the expected result is less than or equal to a preset difference, the candidate anomaly detection model can be directly used as the trained anomaly detection model. If the difference between the first predicted result and the expected result is greater than the preset difference, it indicates that the prediction accuracy of the candidate anomaly detection model is low. In this case, the candidate anomaly detection model needs to be strengthened and the prediction results corrected to obtain the trained anomaly detection model. When analyzing the difference between the first predicted result and the expected result, key text can be extracted from both results, converted into vector representations, and then the cosine similarity between the two vectors can be calculated to measure their directional similarity, thus obtaining the difference between the first predicted result and the expected result. Alternatively, the first predicted result and the expected result can be displayed, and the difference can be determined manually based on the input judgment result.

[0136] In this embodiment, an initial anomaly detection model is trained using a first training dataset; a second training dataset is then acquired, and the initial anomaly detection model is trained using the second training dataset to obtain a candidate anomaly detection model; data to be processed is acquired, and prediction processing is performed on the data to be processed based on the candidate anomaly detection model to obtain a first prediction result; based on the first prediction result and the candidate anomaly detection model, the trained anomaly detection model is obtained. Thus, by iteratively training the anomaly detection model multiple times, the accuracy of the trained anomaly detection model can be maximized.

[0137] In some embodiments, see Figure 3G Step 1054 can be implemented through steps 541 to 543, including:

[0138] In step 541, when the first prediction result matches the expected result, the candidate anomaly detection model is determined as the trained anomaly detection model.

[0139] Here, the expected result refers to the accurate prediction result that the anomaly detection model can output under ideal conditions. The expected result can be determined by technicians through manual analysis. When the first prediction result is the same as the expected result, or the difference between the first prediction result and the expected result is less than or equal to the preset difference, it indicates that the prediction result of the candidate anomaly detection model is accurate, and the first prediction result is determined to meet the expected result. At this time, the candidate anomaly detection model can be directly determined as the trained anomaly detection model.

[0140] In step 542, when the first prediction result does not meet the expected result, a second loss value for the candidate anomaly detection model is determined based on the first prediction result and the expected result.

[0141] Here, when the first prediction result differs from the expected result, or when the difference between the first prediction result and the expected result is greater than a preset difference, it indicates that the prediction result of the candidate anomaly detection model is inaccurate. Therefore, the first prediction result is determined to be inconsistent with the expected result, and the candidate anomaly detection model needs to be trained again. The second loss value is used to measure the difference between the first prediction result and the expected result, and can be calculated using a loss function.

[0142] In step 543, the candidate anomaly detection model is trained using the second loss value to obtain the trained anomaly detection model.

[0143] Here, based on the second loss value, the parameters of the anomaly detection model to be trained are adjusted using the gradient descent algorithm, thereby training the candidate anomaly detection model and obtaining the trained anomaly detection model.

[0144] In this embodiment, when the first prediction result matches the expected result, the candidate anomaly detection model is determined as the trained anomaly detection model; when the first prediction result does not match the expected result, a second loss value for the candidate anomaly detection model is determined based on the first prediction result and the expected result; the candidate anomaly detection model is then trained using the second loss value to obtain the trained anomaly detection model. In this way, during the actual application of the anomaly detection model, the model can be continuously reinforced based on the first prediction result, further improving the prediction accuracy of the trained anomaly detection model.

[0145] In some embodiments, in response to receiving abnormal data for heterogeneous computing resources, a trained anomaly detection model is obtained, and the abnormal data is predicted using the trained anomaly detection model to obtain a prediction result.

[0146] In this way, when new anomalies or faults occur in a heterogeneous computing network, the anomaly data related to the anomaly or fault can be directly input into the trained anomaly detection model. For example, the anomalous phenomena and anomalous indicator data can be input into the trained anomaly detection model. Then, the trained anomaly detection model diagnoses and analyzes the anomaly data to obtain prediction results. These prediction results can include anomaly diagnosis results corresponding to the anomaly data, as well as anomaly handling strategies developed for the anomaly data.

[0147] In this embodiment, in response to receiving abnormal data concerning heterogeneous computing resources, a trained anomaly detection model is acquired, and the abnormal data is predicted using the trained anomaly detection model to obtain a prediction result. Thus, by using the trained anomaly detection model to assist heterogeneous computing resource maintenance personnel in locating and diagnosing and analyzing anomalies, the efficiency of maintaining heterogeneous computing resources can be improved, and the maintenance cost of heterogeneous computing resources can be reduced.

[0148] The following will describe exemplary applications of the embodiments of this application in heterogeneous computing scenarios.

[0149] Due to the unique architecture and operating mode of heterogeneous computing devices, malfunctions frequently occur during operation, and these malfunctions can be of many types. (See also...) Figure 4 , Figure 4 The distribution of different types of faults in heterogeneous computing power is shown, where each region represents a type of fault, such as GPU fault, self-reboot failure, power supply failure, network card failure, memory failure, motherboard failure, network cable failure, memory failure, fan failure, etc.

[0150] Because model training computation based on heterogeneous computing power is a systems engineering project that integrates computing power, network, and storage, such as Figure 5 As shown, Figure 5 The system architecture for an Artificial Intelligence (AI) computing project is illustrated, including a computing device 51, a network device 52, and a storage device 53, corresponding to computing power resources, network resources, and storage resources, respectively. Therefore, in this embodiment, to facilitate the classification of various types of faults in heterogeneous computing power, faults are categorized and integrated into three types: computing power, network, and storage.

[0151] This application proposes a method for using an automated intelligent assistant (corresponding to the anomaly detection model in other embodiments) to prioritize screening and analysis, thereby assisting heterogeneous computing power operators in quickly locating faults and anomalies and improving resource flow efficiency. The intelligent assistant in this application embodiment can continuously iterate and train itself based on a set of standardized fault cases (corresponding to abnormal events in other embodiments) and handling strategies. Furthermore, the execution results of the intelligent assistant can be corrected by operators, thereby continuously optimizing the analysis effect of the intelligent assistant. See also... Figure 6First, the computing power anomaly events 61, network anomaly events 62, and storage anomaly events 63 in the heterogeneous computing power are standardized to construct the first training dataset 64. Then, the intelligent assistant 65 is iteratively trained using the first training dataset 64. Simultaneously, the intelligent assistant 65 is fine-tuned and optimized through manual correction 66 and feedback optimization through the peripheral system 67, resulting in the trained intelligent assistant 65. In this way, other operating systems connected to the heterogeneous devices can trigger the linkage of each operating system through the intelligent assistant, thereby iteratively improving the intelligence of the intelligent assistant. This is called an intelligent diagnostic heterogeneous computing power assistant.

[0152] See Figure 7 This application embodiment, from the perspective of building an intelligent assistant, categorizes faults into computing power anomaly events 711, network anomaly events 712, and storage anomaly events 713. Each anomaly event is analyzed step-by-step manually to obtain anomaly phenomenon 721, anomaly cause 722, anomaly diagnosis result 723, and anomaly handling strategy 724. This information is integrated into training data and a first training dataset is constructed. Based on the first training dataset, a model that can assist heterogeneous computing power operation and maintenance personnel in locating problems can be trained, namely, the intelligent assistant 73. When an anomaly event occurs, the intelligent assistant 73 first diagnoses the anomaly event to obtain a diagnosis result 741 (corresponding to the first prediction result in other embodiments). Then, the diagnosis result 741 is corrected by human feedback 742 to obtain the expected result 743. The expected result 743 is fed back into the intelligent assistant 73 to enhance the intelligent assistant 73.

[0153] The heterogeneous computing platform can support training services for various business models, including mixed-model large-scale computing, advertising business models, video account recommendation, and game artificial intelligence (AI) models. Figure 8As shown, in the heterogeneous container 82 section, different heterogeneous computing power containers can be constructed according to the computing power resource requirements of different tasks, including: computing power container 821, computing power container 822, computing power container 823, and computing power container 824. In the product layer 81, these computing power containers serve the training tasks of various models, including: mixed model 811, visual model 812, speech model 813, game model 814, medical model 815, and language model 816. In the construction of the intelligent assistant 83, the constructed intelligent assistant 836 can perform anomaly diagnosis on various abnormal events in the heterogeneous container 82. The process of building the intelligent assistant 836 includes: organizing computing power anomaly events 831, network anomaly events 832, and storage anomaly events 833, and manually cleaning these anomaly events 834. The purpose of cleaning is to check whether the information related to the anomaly events is complete and accurate. After cleaning, the data is summarized to obtain the first training dataset 835, which is used to train the intelligent assistant 836. The trained intelligent assistant 836 will be applied to the live network to assist operation and maintenance personnel in diagnosing anomalies in computing power containers.

[0154] See Figure 9 Heterogeneous computing resources can be used for training various models, including: speech models 911, vision models 912, medical models 913, language models 914, and game models 915. During model training, abnormal events will occur, categorized as computing power abnormal events 921, network abnormal events 922, and storage abnormal events 923. Analyzing these abnormal events yields abnormal information, including: abnormal phenomena 924, abnormal causes 925, abnormal diagnosis results 936, abnormal handling strategies 927, and abnormal feedback results 928. This information is then manually cleaned 93, and through manual screening of abnormal events and unified management 94, the first training dataset is constructed and stored in database 941. This ensures that only valuable training data is uniformly managed and stored in database 941. As the amount of material data from abnormal events increases, the database capacity needs to be expanded. Therefore, unified operation and maintenance 95 can be used to maintain the database, expanding it to database 951 when storage space in database 941 becomes insufficient. Simultaneously, unified operation and maintenance 95 also needs to ensure the security of the database. In the intelligent assistant training section 96, the abnormal information 961 stored in the database, including: abnormal phenomena 924, abnormal causes 925, abnormal diagnosis results 936, abnormal handling strategies 927, and abnormal feedback results 928, is used to determine the abnormal description information 962 and abnormal diagnosis results 963, and the intelligent assistant 964 is obtained through training. Then, in the intelligent assistant feedback correction section 97, the intelligent assistant is reinforced through feedback evaluation 971 and manual correction 972.

[0155] See Figure 10 , Figure 10This is a flowchart illustrating an anomaly analysis process provided in an embodiment of this application. When an anomaly occurs, an anomaly phenomenon 1 is first generated: training performance degrades. Then, technicians analyze anomaly phenomenon 1 based on their experience, obtaining an anomaly cause 2: abnormal communication performance, and an anomaly diagnosis result 3: packet loss in the network. Next, an anomaly handling strategy 4 is formulated: expanding network bandwidth or adjusting service distribution. After handling the anomaly according to anomaly handling strategy 4, an anomaly feedback result 5 is obtained: training performance returns to normal.

[0156] See Figure 11 , Figure 11 This is a flowchart illustrating an abnormal event filtering process provided in an embodiment of this application. The filtering process can be implemented by a server, such as... Figure 11 As shown, the process includes:

[0157] In step 111, obtain the abnormal feedback result.

[0158] Here, the abnormal feedback result is obtained based on the abnormal information of the abnormal event. The abnormal feedback result is used to characterize whether the abnormality has been resolved.

[0159] In step 112, it is determined whether the system has returned to normal.

[0160] Here, based on the abnormality feedback results, it is determined whether the abnormality in the heterogeneous computing resources has returned to normal, and if so, step 113 is executed.

[0161] In step 113, the abnormal events are matched to obtain the matching results.

[0162] Here, the matching process determines whether the cause of an abnormal event can be matched with an exception handling strategy. If the cause and the strategy match, the matching result is considered successful; otherwise, it is considered a failure. By performing matching processing on abnormal events, we can analyze whether the abnormal phenomenon, cause, diagnosis result, handling strategy, and feedback result are logically consistent.

[0163] In step 114, it is determined whether the match was successful.

[0164] Here, the matching result determines whether the match was successful, and if the matching result is successful, step 115 is executed.

[0165] In some implementations, this exception event is discarded when the matching result is a match failure.

[0166] In step 115, the abnormal description information and abnormal diagnosis results are determined.

[0167] Here, since the abnormal phenomenon includes: abnormal phenomenon, abnormal cause, abnormal diagnosis result, abnormal handling strategy and abnormal feedback result, the abnormal diagnosis result can be obtained directly. Then, the abnormal phenomenon, abnormal cause, abnormal handling strategy and abnormal feedback result in the abnormal information are integrated to obtain the abnormal description information.

[0168] In step 116, the abnormal description information and abnormal diagnosis results are output.

[0169] Here, the anomaly description information and anomaly diagnosis results are output and stored for subsequent construction of the first training dataset.

[0170] See Figure 12 When constructing the first training dataset, the training data can be labeled using steps 121 to 123. The construction of this first training dataset can be implemented by a server, such as... Figure 12 As shown, the process includes:

[0171] In step 121, abnormal events are filtered to obtain target events.

[0172] Here, manual annotation can be performed to filter out misleading abnormal events, thereby obtaining a dataset of accurate target events.

[0173] In step 122, training data is constructed based on the anomaly description information and anomaly detection results of the target event.

[0174] Here, the anomaly description information is used as sample data, and the anomaly detection results are used as label information to construct the training data corresponding to the target event.

[0175] In step 123, the training weights of each training data point are determined, and the training data is labeled to obtain labeled training data.

[0176] Here, for certain specific target events, differentiated labeling scores can be implemented. That is, the training weights corresponding to each training data point are determined, and the training data is labeled to obtain accurate and differentiated labeled training data.

[0177] In step 124, each labeled training data is used to construct the first training dataset.

[0178] Here, each labeled training data is used to construct the first training dataset, which is then used to train the intelligent assistant.

[0179] See Figure 13 , Figure 13This is a schematic diagram of the iterative training process for an intelligent assistant provided in an embodiment of this application. This iterative training of the intelligent assistant can be implemented by a server. Figure 13 As shown, the training process includes multiple iterations to continuously improve the types and richness of problems that the intelligent assistant can diagnose. First, the intelligent assistant model is pre-trained using the first training dataset 131 to obtain the intelligent assistant base model 132 (corresponding to the initial anomaly detection model in other embodiments). The intelligent assistant base model 132 supports question and answer for more than 90% of anomaly events. For the remaining less than 10% of anomaly events, the intelligent assistant base model 132 needs to be fine-tuned and optimized using the second training dataset 133 to obtain the intelligent assistant fine-tuned model 134 (corresponding to the candidate anomaly detection model in other embodiments). At this time, the intelligent assistant fine-tuned model 134 can cover low-frequency anomaly events. After the intelligent assistant fine-tuning model 134 is deployed as an inference service, the abnormal events 135 encountered in heterogeneous operation and maintenance work will be input into the intelligent assistant fine-tuning model 134 as data to be processed. The intelligent assistant fine-tuning model 134 will execute the inference process and output the first prediction result 136. The first prediction result 136 needs to be confirmed by humans. The purpose is to strengthen the training of the intelligent assistant fine-tuning model 134 through manual correction, so as to improve the accuracy of the intelligent assistant by fine-tuning the diagnostic results.

[0180] See Figure 14 The intelligent assistant can be trained through steps 141 to 1412. This training process can be implemented by a server, such as... Figure 14 As shown, the process includes:

[0181] In step 141, multiple abnormal events are obtained.

[0182] Here, we collect and analyze the abnormal cases in various scenarios of heterogeneous computing examples.

[0183] In step 142, the event category corresponding to each abnormal event is determined.

[0184] Here, since abnormal events can be divided into computing power abnormal events, network abnormal events, and storage abnormal events, the event category corresponding to the abnormal event can be determined according to the computing power category, network category, and storage category.

[0185] In step 143, the abnormal information corresponding to each abnormal event is determined.

[0186] Here, by analyzing abnormal events, we determine the corresponding abnormal phenomena, causes, diagnostic results, handling strategies, and feedback results, which are then used as abnormal information.

[0187] In step 144, it is determined whether the abnormal information is complete and accurate.

[0188] Here, through manual cleaning, it is determined whether any information recorded in the abnormal information is missing or contains errors. If the abnormal information is complete and error-free, it indicates that the abnormal information is complete and accurate, and the judgment result is yes, proceeding to step S145. If the abnormal information is missing or contains errors, it indicates that the abnormal information is incomplete or inaccurate, and the judgment result is no, proceeding to step S147.

[0189] In step 145, it is determined whether the abnormality has been restored to normal.

[0190] Here, the abnormal feedback result in the abnormal information is used to determine whether the abnormality has been resolved. If the abnormality has been resolved, the result is yes, and step 146 is executed; if the abnormality has not been resolved, the result is no, and step 147 is executed.

[0191] In step 146, it is determined whether there is a match between the exception information and the event category.

[0192] Here, it can be determined whether each piece of information in the exception message matches the event category, or whether the exception cause matches the exception handling strategy. If they match, the result is "yes," and step 148 is executed. If they do not match, the result is "no," and step 147 is executed.

[0193] In step 147, the abnormal event is discarded.

[0194] Here, exceptions that do not meet the requirements are discarded.

[0195] In step 148, the abnormal description information and abnormal diagnosis results are determined.

[0196] Here, the filtered abnormal events are used as target events, and the abnormal description information and abnormal diagnosis results are determined based on the abnormal information of each target event.

[0197] In step 149, the first training dataset is constructed, and the intelligent assistant is trained.

[0198] Here, a first training dataset is constructed based on each anomaly description and anomaly diagnosis result, and the intelligent assistant is trained using the first training dataset.

[0199] In step 1410, prediction processing is performed based on the intelligent assistant to obtain the first prediction result.

[0200] Here, the trained intelligent assistant performs predictive processing on abnormal events to obtain the first prediction result.

[0201] In step 1411, it is determined whether the first prediction result matches the expected result.

[0202] Here, the operations and maintenance personnel perform analysis and diagnosis to determine whether the first prediction result meets the expected result. If the first prediction result meets the expected result, the judgment result is yes, and step 1410 is executed again to wait for the next abnormal event and perform prediction processing again. If the first prediction result does not meet the expected result, the judgment result is no, and step 1412 is executed.

[0203] In step 1412, the smart assistant is retrained based on the expected results.

[0204] Here, the intelligent assistant is retrained based on the expected results, thereby feeding back and strengthening the intelligent assistant.

[0205] This application embodiment analyzes abnormal events in various scenarios, categorizing them according to computing power, network, and storage categories. Data is then standardized according to the formats of abnormal phenomena, causes, diagnostic results, handling strategies, and feedback results. The standardized data undergoes manual cleaning to remove records with missing or incorrect information. For the filtered data, the feedback results are evaluated, retaining only records where the anomaly has resolved. Next, the anomaly handling strategy is assessed to ensure it matches the abnormal phenomena and causes, based on the event category. The first training dataset is constructed using the integrated anomaly descriptions and diagnostic results and input into the training of the intelligent assistant. The training process iteratively completes the intelligent assistant's training. When the intelligent assistant is used online to assist maintenance personnel in performing analysis and diagnosis, the initial prediction results are manually corrected, thus reinforcing the intelligent assistant.

[0206] The embodiments of this application have the following technical effects: 1) They provide an intelligent assistant to assist heterogeneous computing power operation and maintenance personnel in performing fault and anomaly analysis, thereby improving the efficiency of anomaly location and analysis, and can serve large-scale heterogeneous computing power, thus improving operation and maintenance efficiency; 2) They perform problem diagnosis through the intelligent assistant, and then provide feedback and correction based on the diagnosis results. By continuously strengthening and iterating the assistant, the diagnostic accuracy of the assistant is improved; 3) The intelligent diagnostic assistant constructed in the embodiments of this application improves the efficiency of fault and anomaly diagnosis, reduces the impact time of fault and anomalies, thereby increasing the availability time of heterogeneous computing power resources and reducing the overall business expenditure cost; 4) The operation mode based on the intelligent assistant in the embodiments of this application reduces the operation and maintenance manpower input of operating massive heterogeneous computing power, improving efficiency while reducing the input of operation and maintenance manpower.

[0207] In this embodiment, from the perspective of building an intelligent assistant, after summarizing and organizing abnormal events, an intelligent assistant is trained based on a first training dataset to assist heterogeneous computing power operation and maintenance personnel in locating problems, thereby improving the efficiency of heterogeneous device fault location. Since the results of the intelligent assistant's diagnosis are subject to manual inspection and feedback, if the results need correction, they will be fed back to the intelligent assistant, thus strengthening it. This embodiment improves the efficiency of abnormal problem location and analysis while reducing the manpower required for operating massive amounts of heterogeneous computing power, and can serve large-scale heterogeneous computing power. Furthermore, the intelligent diagnostic assistant built in this embodiment improves the efficiency of fault and anomaly diagnosis, reduces the impact time of faults and anomalies, and thus increases the usable time of heterogeneous computing power resources, reducing overall business expenditure costs. Moreover, the cost savings become more significant as the scale of heterogeneous computing power increases.

[0208] The following description continues to illustrate the exemplary structure of the model training device 233 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 2 As shown, the software modules stored in the model training device 233 of the memory 230 may include:

[0209] The acquisition module 2331 is used to acquire multiple abnormal events for heterogeneous computing power resources and determine the abnormal information and event category corresponding to each abnormal event.

[0210] The matching module 2332 is used to perform matching processing on the abnormal information and event category of each abnormal event that meets the matching processing conditions, and obtain the matching result.

[0211] The determining module 2333 is used to determine at least one target event from a plurality of abnormal events based on the matching result of each of the abnormal events;

[0212] Module 2334 is used to construct a first training dataset based on the anomaly information corresponding to each of the target events;

[0213] The training module 2335 is used to train the anomaly detection model to be trained using the first training dataset to obtain the trained anomaly detection model.

[0214] In some embodiments, the determining module 2333 is further configured to determine the number of feature information included in the abnormal information for each abnormal event; when the number of feature information is a preset number, determine the state change information of the heterogeneous computing resources corresponding to the abnormal event; when the state change information indicates that the state of the heterogeneous resources has changed from abnormal to normal, determine that the abnormal event meets the matching processing conditions.

[0215] In some embodiments, the determining module 2333 is further configured to determine the event category corresponding to the abnormal event based on the resource category corresponding to the abnormal event; analyze and process the abnormal event based on the event category to obtain an analysis result, the analysis result including at least one feature information; obtain a preset template, the preset template being used to integrate and process the abnormal information; fill each of the feature information in the abnormal information into the corresponding position in the preset template to obtain the abnormal information.

[0216] In some embodiments, the matching module 2332 is further configured to perform matching processing on each feature information in the abnormal information and the event category; when each feature information matches the event category, the matching result is determined to be a successful match; or, when any feature information does not match the event category, the matching result is determined to be a failed match.

[0217] In some embodiments, the determining module 2333 is further configured to determine the target event based on the matching result of each of the abnormal events, when the matching result is a successful match.

[0218] In some embodiments, the construction module 2334 is further configured to, for each target event, determine the abnormal description information and the abnormal detection result of the target event; use the abnormal description information as sample data and the abnormal detection result as label information to construct training data corresponding to the target event; and construct the first training dataset based on the training data corresponding to each target event.

[0219] In some embodiments, the construction module 2334 is further configured to determine the training weight of each training data, label the training data based on the training weight to obtain labeled training data, and construct the first training dataset from each labeled training data.

[0220] In some embodiments, the training module 2335 is further configured to: train the anomaly detection model to be trained using the first training dataset to obtain an initial anomaly detection model; obtain a second training dataset and train the initial anomaly detection model using the second training dataset to obtain a candidate anomaly detection model; obtain data to be processed and perform prediction processing on the data to be processed based on the candidate anomaly detection model to obtain a first prediction result; and obtain the trained anomaly detection model based on the first prediction result and the candidate anomaly detection model.

[0221] In some embodiments, the training module 2335 is further configured to: determine the training weights corresponding to each labeled training data in the first training dataset; perform prediction processing on the sample data in the labeled training data based on the anomaly detection model to be trained to obtain a second prediction result; determine an initial loss value based on the label information in the labeled training data and the second prediction result; determine the first loss value of the anomaly detection model to be trained by multiplying the initial loss value and the training weights; and train the anomaly detection model to be trained using each of the first loss values ​​to obtain the initial anomaly detection model.

[0222] In some embodiments, the training module 2335 is further configured to: determine the candidate anomaly detection model as the trained anomaly detection model when the first prediction result meets the expected result; determine a second loss value of the candidate anomaly detection model based on the first prediction result and the expected result when the first prediction result does not meet the expected result; and train the candidate anomaly detection model using the second loss value to obtain the trained anomaly detection model.

[0223] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the model training method described above in this application.

[0224] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the model training method provided in this application. For example, ... Figure 3A The model training method is shown.

[0225] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0226] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0227] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored in a portion of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0228] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0229] In summary, through the embodiments of this application, multiple abnormal events for heterogeneous computing resources are obtained, and the abnormal information and event category corresponding to each abnormal event are determined. For each abnormal event that meets the matching processing conditions, the abnormal information and event category of the abnormal event are matched to obtain a matching result. Based on the matching result of each abnormal event, at least one target event is determined from multiple abnormal events. In this way, the required target event can be accurately determined from multiple abnormal events of heterogeneous computing resources according to the matching processing conditions and matching results, thereby improving the efficiency and accuracy of determining the target event. Subsequently, a first training dataset is constructed based on the abnormal information corresponding to each target event, which can improve the accuracy and quality of the training data in the first training dataset and ensure that the training data can comprehensively cover multiple abnormal situations. Finally, the anomaly detection model to be trained is trained using the first training dataset to obtain the trained anomaly detection model, which can improve the training efficiency and model accuracy of the model. Furthermore, through the trained anomaly detection model, automatic detection of anomalies in heterogeneous computing resources can be achieved, thereby reducing the cost of detecting anomalies in heterogeneous computing resources and improving the efficiency and accuracy of detection.

[0230] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A model training method, characterized in that, The method includes: Acquire multiple abnormal events for heterogeneous computing resources, and determine the abnormal information and event category corresponding to each abnormal event; For each abnormal event that meets the matching processing conditions, the abnormal information and event category of the abnormal event are matched to obtain the matching result; Based on the matching result of each of the aforementioned anomalous events, at least one target event is determined from the plurality of anomalous events; Based on the anomaly information corresponding to each target event, a first training dataset is constructed. The anomaly detection model to be trained is trained using the first training dataset to obtain the trained anomaly detection model.

2. The method according to claim 1, characterized in that, The anomaly information includes at least one feature information for describing the anomaly event, and the method further includes: For each abnormal event, determine the number of feature information included in the abnormal information. When the number of feature information is a preset number, determine the status change information of the heterogeneous computing resources corresponding to the abnormal event; When the status change information indicates that the status of the heterogeneous resource has changed from abnormal to normal, it is determined that the abnormal event meets the matching processing condition.

3. The method according to claim 1, characterized in that, Determining the anomaly information and event category corresponding to each of the aforementioned anomalies includes: Based on the resource category corresponding to the abnormal event, determine the event category corresponding to the abnormal event; The abnormal event is analyzed and processed based on the event category to obtain the analysis result, which includes at least one feature information. Obtain a preset template, which is used to integrate and process the abnormal information; Each of the aforementioned feature information in the abnormal information is filled into the corresponding position in the preset template to obtain the abnormal information.

4. The method according to claim 1, characterized in that, The matching process of the abnormal information and event category of the abnormal event to obtain the matching result includes: For each feature in the anomaly information, a matching process is performed between the feature information and the event category; When each of the aforementioned feature pieces of information matches the event category, the matching result is determined to be a successful match; Alternatively, if any of the aforementioned feature information does not match the event category, the matching result is determined to be a matching failure.

5. The method according to claim 1, characterized in that, The step of determining at least one target event from a plurality of anomalous events based on the matching result of each of the anomalous events includes: For each of the aforementioned abnormal events, if the matching result is successful, the abnormal event is identified as the target event.

6. The method according to claim 1, characterized in that, The construction of the first training dataset based on the anomaly information corresponding to each target event includes: For each of the target events, determine the anomaly description information and anomaly detection result of the target event; Using the anomaly description information as sample data and the anomaly detection results as label information, training data corresponding to the target event is constructed. The first training dataset is constructed based on the training data corresponding to each target event.

7. The method according to claim 6, characterized in that, The construction of the first training dataset based on the training data corresponding to each target event includes: Determine the training weight for each piece of training data, and label the training data based on the training weight to obtain labeled training data; Each of the labeled training data is used to construct the first training dataset.

8. The method according to any one of claims 1 to 7, characterized in that, The step of training the anomaly detection model to be trained using the first training dataset to obtain the trained anomaly detection model includes: The anomaly detection model to be trained is trained using the first training dataset to obtain the initial anomaly detection model; Obtain a second training dataset, and use the second training dataset to train the initial anomaly detection model to obtain a candidate anomaly detection model; Acquire the data to be processed, and perform prediction processing on the data to be processed based on the candidate anomaly detection model to obtain a first prediction result; Based on the first prediction result and the candidate anomaly detection model, the trained anomaly detection model is obtained.

9. The method according to claim 8, characterized in that, The step of training the anomaly detection model to be trained using the first training dataset to obtain the initial anomaly detection model includes: For each labeled training data in the first training dataset, determine the training weight corresponding to the labeled training data; Based on the anomaly detection model to be trained, the sample data in the labeled training data are predicted to obtain a second prediction result. Based on the label information in the labeled training data and the second prediction result, the initial loss value is determined; The product of the initial loss value and the training weights is determined as the first loss value of the anomaly detection model to be trained. The anomaly detection model to be trained is trained using each of the first loss values ​​to obtain the initial anomaly detection model.

10. The method according to claim 8, characterized in that, The step of obtaining the trained anomaly detection model based on the first prediction result and the candidate anomaly detection model includes: When the first prediction result matches the expected result, the candidate anomaly detection model is determined as the trained anomaly detection model; When the first prediction result does not meet the expected result, a second loss value of the candidate anomaly detection model is determined based on the first prediction result and the expected result. The candidate anomaly detection model is trained using the second loss value to obtain the trained anomaly detection model.

11. A model training device, characterized in that, The device includes: The acquisition module is used to acquire multiple abnormal events for heterogeneous computing resources and determine the abnormal information and event category corresponding to each abnormal event. The matching module is used to perform matching processing on the abnormal information and event category of each abnormal event that meets the matching processing conditions, and obtain the matching result. The determination module is used to determine at least one target event from a plurality of abnormal events based on the matching result of each of the abnormal events; A construction module is used to construct a first training dataset based on the anomaly information corresponding to each of the target events; The training module is used to train the anomaly detection model to be trained using the first training dataset, so as to obtain the trained anomaly detection model.

12. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method according to any one of claims 1 to 10.

13. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the method described in any one of claims 1 to 10.

14. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the method according to any one of claims 1 to 10.

Images