Classification task processing method, device, electronic device and storage medium

By converting the model file format of the random forest model into an acceleration processor-readable format and performing classification task processing on the acceleration processor, the problem of CPU delay and low efficiency in large-scale data processing is solved, achieving more efficient classification task processing.

CN113900820BActive Publication Date: 2024-11-05BEIJING DAJIA INTERNET INFORMATION TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202111255496.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-27
Publication Date
2024-11-05
Estimated Expiration
2041-10-27

AI Technical Summary

Technical Problem

In classification task processing based on random forest model, the CPU cannot efficiently perform parallel computing of large-scale data, resulting in low computational delay and task processing efficiency.

Method used

By converting the model file of the target classification model in format, the node information that can be readable by the acceleration processor is obtained, and classification task processing is performed on the acceleration processor, and data storage and loading are used to improve processing efficiency.

Benefits of technology

It effectively reduces the delay in classifying task processing, improves the speed and efficiency of classifying task processing, and improves the performance of the server.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113900820B_ABST
    Figure CN113900820B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a classification task processing method, device, electronic device and storage medium, the method comprising receiving a processing request for a target classification task based on an acceleration processor, the processing request comprising information to be classified corresponding to the target classification task; obtaining target model structure information of a target classification model corresponding to the target classification task based on the acceleration processor, the target model structure information being node information stored in a target format after format conversion based on a model file of the target classification model in advance; the target format being a readable format of the acceleration processor, the node information being node information in a decision tree corresponding to the target classification model; controlling the acceleration processor to perform operations for classification task processing based on the information to be classified and the target model structure information, and obtaining classification result information of the target classification task. The disclosed embodiments can effectively reduce the delay of classification task processing and improve the speed and efficiency of classification task processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a classification task processing method, device, electronic device and storage medium. Background Art

[0002] With the development of artificial intelligence technology, artificial intelligence technology has been widely used in many fields. The random forest model in artificial intelligence technology is a machine learning method that uses multiple decision trees to discriminate and classify data. It is often used in classification tasks such as traffic identification and interest identification.

[0003] In the related art, in the task processing of classification based on the random forest model, the random forest model is often trained based on the Python xgboost library (a python method library for training and reasoning of the random forest model). After the training is completed, the model is saved in pkl format (a file storage format in python, used to save the serialization of the random forest model) based on the Python pickle library (a python method library for saving and loading the model file of the trained random forest model). The CPU (central processing unit) combines the model file in pkl format for classification processing. However, the CPU cannot efficiently perform parallel calculations on large-scale data, resulting in the classification task processing of the related technology being too slow, and there are problems such as calculation delay and low task processing efficiency. Summary of the invention

[0004] The present disclosure provides a classification task processing method, device, electronic device and storage medium to at least solve the problems of computing delay, low task processing efficiency and the like in the related art. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a classification task processing method is provided, comprising:

[0006] Receiving a processing request for a target classification task based on an acceleration processor, the processing request including information to be classified corresponding to the target classification task;

[0007] Based on the acceleration processor, the target model structure information of the target classification model corresponding to the target classification task is obtained, wherein the target model structure information is node information stored in a target format after format conversion is performed in advance based on the model file of the target classification model; the target format is a readable format of the acceleration processor, and the node information is node information in a decision tree corresponding to the target classification model;

[0008] The acceleration processor performs an operation of processing a classification task based on the information to be classified and the target model structure information to obtain classification result information of the target classification task.

[0009] Optionally, the acceleration processor includes a data storage block, and before acquiring the target model structure information of the target classification model corresponding to the target classification task based on the acceleration processor, the method further includes:

[0010] Based on the acceleration processor, reading the node information in the target format;

[0011] The node information is stored in the data storage block based on the acceleration processor.

[0012] Optionally, obtaining the target model structure information of the target classification model corresponding to the target classification task based on the acceleration processor includes:

[0013] The operation of loading the node information from the data storage block is performed by the acceleration processor.

[0014] Optionally, the method further includes:

[0015] Based on a central processing unit, obtaining the model file;

[0016] By means of the central processor, an operation of exporting initial model structure information in a preset text format from the model file is performed based on a preset export method, wherein the initial model structure information includes the node information;

[0017] Based on the central processor, the node information is stored according to the target format.

[0018] Optionally, the decision tree corresponding to the target classification model is a plurality of decision trees, and the method further includes:

[0019] Based on the central processor, determining node position information of each node information;

[0020] By means of the central processor, an operation of writing the node information corresponding to the same decision tree into the corresponding preset array is executed based on the node position information;

[0021] Based on the central processor, storing the node information according to the target format includes:

[0022] Based on the central processor, the node information in the preset array is stored according to the target format.

[0023] Optionally, the node information stored in the target format is a plurality of data blocks of preset sizes; based on a central processor, storing the node information in the preset array in the target format includes:

[0024] Based on the central processor, the node information corresponding to the same node position information in the preset array corresponding to any two decision trees is stored as the same data block.

[0025] Optionally, the method further includes:

[0026] Based on the central processor, the node information corresponding to the leaf node and the node information of the parent node of the leaf node are merged to obtain merged node information;

[0027] Based on the central processor, storing the node information corresponding to the same node position information in the preset array corresponding to any two decision trees as the same data block includes:

[0028] Through the central processor, the merged node information corresponding to the same node position information in the preset array corresponding to any two decision trees is stored as the same data block.

[0029] Optionally, the acceleration processor includes a plurality of serial data storage blocks under the same parallel thread, each data storage block includes node information of at least one decision tree in the decision tree corresponding to the target classification model, and the method further includes:

[0030] Based on the central processor, determining a target data storage block at the end of the parallel thread from the plurality of serial data storage blocks;

[0031] Based on the central processor, determining a target decision tree in at least one decision tree corresponding to the target data storage block;

[0032] Based on the central processor, the node information of the target decision tree, the position information in the data blocks of the plurality of preset sizes and the data block size corresponding to the target decision tree are recorded.

[0033] Optionally, the method further includes:

[0034] Based on the central processor, obtaining a model type of the target classification model;

[0035] The operation of exporting the initial model structure information in a preset text format from the model file based on a preset export method by the central processor includes:

[0036] In the case that the model type is a preset compatible model type, the central processor exports initial model structure information in a preset text format from the model file based on a preset export method.

[0037] According to a second aspect of an embodiment of the present disclosure, there is provided a classification task processing device, comprising:

[0038] A processing request receiving module, running on the acceleration processor, is configured to execute a processing request for receiving a target classification task, wherein the processing request includes information to be classified corresponding to the target classification task;

[0039] A target model structure information acquisition module, running on the acceleration processor, is configured to execute acquisition of target model structure information of the target classification model corresponding to the target classification task, wherein the target model structure information is node information stored in a target format after format conversion is performed in advance based on a model file of the target classification model; the target format is a readable format of the acceleration processor, and the node information is node information in a decision tree corresponding to the target classification model;

[0040] The classification task processing module runs on the acceleration processor and is configured to perform classification task processing based on the information to be classified and the target model structure information to obtain classification result information of the target classification task.

[0041] Optionally, the acceleration processor includes a data storage block, and the device further includes:

[0042] A node information reading module, running on the acceleration processor, is configured to read the node information in the target format before acquiring the target model structure information of the target classification model corresponding to the target classification task;

[0043] The node information storage module runs on the acceleration processor and is configured to execute the storage of the node information into the data storage block.

[0044] Optionally, the target model structure information acquisition module is specifically configured to execute loading of the node information from the data storage block.

[0045] Optionally, the device further comprises:

[0046] A model file acquisition module, running on the central processing unit, is configured to execute acquisition of the model file;

[0047] An initial model structure information export module, running on the central processing unit, is configured to execute, based on a preset export method, exporting the initial model structure information in a preset text format from the model file, the initial model structure information including the node information;

[0048] The node information storage module runs on the central processing unit and is configured to store the node information according to the target format.

[0049] Optionally, the decision tree corresponding to the target classification model is a plurality of decision trees, and the device further includes:

[0050] An information determination module, running on the central processor, is configured to determine the node position information of each node information;

[0051] A node information writing module, running on the central processing unit, is configured to write the node information corresponding to the same decision tree into a corresponding preset array based on the node position information;

[0052] The node information storage module is further configured to store the node information in the preset array according to the target format.

[0053] Optionally, the node information stored in the target format is a plurality of data blocks of preset sizes; the node information storage module is further configured to execute the storage of the node information corresponding to the same node position information in the preset array corresponding to any two decision trees as the same data block.

[0054] Optionally, the device further comprises:

[0055] A merging processing module, running on the central processing unit, is configured to perform merging processing on the node information corresponding to the leaf node and the node information of the parent node of the leaf node to obtain merged node information;

[0056] The node information storage module is further configured to execute the storage of the merged node information corresponding to the same node position information in the preset array corresponding to any two decision trees as the same data block.

[0057] Optionally, the acceleration processor includes a plurality of serial data storage blocks under the same parallel thread, each data storage block includes node information of at least one decision tree in the decision tree corresponding to the target classification model, and the device further includes:

[0058] A target data storage block determination module, running on the central processing unit, is configured to determine, from the plurality of serial data storage blocks, a target data storage block at the end of the parallel thread;

[0059] A target decision tree determination module, running on the central processing unit, is configured to determine a target decision tree in at least one decision tree corresponding to the target data storage block;

[0060] The information recording module runs on the central processing unit and is configured to record the node information of the target decision tree, the position information in the data blocks of the plurality of preset sizes and the data block size corresponding to the target decision tree.

[0061] Optionally, the device further comprises:

[0062] A model type acquisition module, running on the central processing unit, is configured to execute acquisition of the model type of the target classification model;

[0063] The initial model structure information export module is further configured to execute:

[0064] In the case that the model type is a preset compatible model type, initial model structure information in a preset text format is exported from the model file based on a preset export method.

[0065] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a method as described in any one of the above-mentioned first aspects.

[0066] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any method described in the first aspect of the embodiment of the present disclosure.

[0067] According to a fifth aspect of an embodiment of the present disclosure, there is provided a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect of the embodiment of the present disclosure.

[0068] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0069] By converting the format of the model file of the target classification model in advance, the node information obtained is stored in a format readable by the acceleration processor. During the task processing, the classification task processing operations can be performed based on the acceleration processor, thereby effectively reducing the delay of the classification task processing and improving the speed and efficiency of the classification task processing.

[0070] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0072] Figure 1 is a schematic diagram of an application environment according to an exemplary embodiment;

[0073] Figure 2 is a flow chart of a classification task processing method according to an exemplary embodiment;

[0074] Figure 3 It is a flow chart showing a method of pre-storing node information in a decision tree corresponding to a target classification model in a target format according to an exemplary embodiment;

[0075] Figure 4 It is another flowchart showing how to store node information in a decision tree corresponding to a target classification model in advance according to a target format according to an exemplary embodiment;

[0076] Figure 5 is a flowchart of another classification task processing method according to an exemplary embodiment;

[0077] Figure 6 is a schematic diagram showing model structure information corresponding to a decision tree according to an exemplary embodiment;

[0078] Figure 7 is a block diagram of a classification task processing device according to an exemplary embodiment;

[0079] Figure 8 The invention is a block diagram of an electronic device for processing classification tasks according to an exemplary embodiment. DETAILED DESCRIPTION

[0080] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0081] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0082] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0083] See also Figure 1 , Figure 1 is a schematic diagram of an application environment according to an exemplary embodiment. Figure 1 As shown, the application environment may include a terminal 100 and a server 200 .

[0084] The terminal 100 can be used to provide classified services to any user. Specifically, the terminal 100 may include but is not limited to electronic devices such as smart phones, desktop computers, tablet computers, laptops, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart wearable devices, etc., and may also be software running on the above electronic devices, such as applications. Optionally, the operating system running on the electronic device may include but is not limited to Android, IOS, Linux, Windows, etc.

[0085] In an optional embodiment, the server 200 stores the target model structure information and can provide background services for the terminal 100. Specifically, the server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0086] In an optional embodiment, the server 200 is provided with an acceleration processor. In actual applications, the terminal 100 can send a processing request for a target classification task to the server 200; the acceleration processor on the server 200 receives the request, reads the target model structure information after model conversion, executes the classification task processing, and returns the classification result information to the terminal 100.

[0087] Optionally, the CPU on the server 200 can pre-convert the format of the random forest model (target classification model) used to process the target classification task, obtain the target model structure information, and store the target model structure information. Optionally, the target model structure information can be stored in a DDR (Double Data Rate) synchronous dynamic random access memory so that when performing task processing, the acceleration processor can be called to obtain the target model structure information.

[0088] In an optional embodiment, the acceleration processor may include but is not limited to FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), GPU (Graphics Processing Unit), etc.

[0089] In addition, it should be noted that Figure 1 What is shown is only one application environment provided by the present disclosure. In actual application, other application environments may also be included, for example, more terminals may be included.

[0090] In the embodiments of this specification, the terminal 100 and the server 200 may be directly or indirectly connected via wired or wireless communication, which is not limited in this disclosure.

[0091] Figure 2 is a flowchart of a classification task processing method according to an exemplary embodiment. Figure 2 As shown, the classification task processing method is used in devices such as servers, and includes the following steps.

[0092] In step S201, the acceleration processor receives a processing request for a target classification task, where the processing request includes information to be classified corresponding to the target classification task;

[0093] In the embodiments of this specification, the target classification task may be different according to the actual application scenario requirements. Optionally, the target classification task may be a traffic identification task (e.g., identifying whether the traffic in a certain application is abnormal traffic, etc.). Optionally, the target classification task may be an interest identification task (e.g., identifying whether a user is interested in a certain short video, etc.).

[0094] In actual applications, the terminal may trigger a processing request for sending a target classification task to the server. Specifically, the information to be classified may vary depending on the actual application scenario. In a specific embodiment, taking the above-mentioned traffic identification scenario as an example, the information to be classified may be traffic feature information that needs to be identified. In another specific embodiment, taking the above-mentioned interest identification scenario as an example, the information to be classified may be user feature information and short video feature information.

[0095] In step S203, based on the acceleration processor, the target model structure information of the target classification model corresponding to the target classification task is obtained.

[0096] In practical applications, the model file of the target classification model is often in pkl format, but the acceleration processor cannot read the model file in pkl format. Accordingly, the model file can be converted into information in a readable format of the acceleration processor in advance. Specifically, the target classification model can be pre-trained based on the training data corresponding to the target classification task.

[0097] In an optional embodiment, the target model structure information may be node information stored in a target format after format conversion is performed in advance on a model file of the target classification model. Specifically, the target format may be a readable format of an acceleration processor.

[0098] In the embodiment of this specification, the node information stored in the target format can be a plurality of data blocks of preset sizes. In a specific embodiment, the readable format of the acceleration processor can be CacheLine (cache line); optionally, the preset size can be the unit size of the cache line: 512 bits.

[0099] In a specific embodiment, the target classification model may be a random forest model for processing target classification tasks. The node information may be node information in a decision tree corresponding to the target classification model.

[0100] In practical applications, the nodes in a decision tree may include leaf nodes and non-leaf nodes, and the node information in a decision tree (i.e., the node information) may include whether the node is a leaf node. Specifically, the node information of a leaf node may also include the weight information of the tree where the node is located (the importance of the decision tree where the node is located in the decision tree corresponding to the target classification model) and the node result (the node result represents the category probability corresponding to the leaf node). Specifically, the node information of a non-leaf node may also include the identification information of the classification feature corresponding to the non-leaf node (i.e., used to determine which feature information in the information to be classified needs to be combined for classification), the classification symbol identification corresponding to the classification feature (e.g., greater than sign, less than sign, etc.), the classification threshold corresponding to the classification feature, the identification information of the left child node of the non-leaf node, and the identification information of the right child node of the non-leaf node.

[0101] In an optional embodiment, the method may further include: pre-storing the node information in the decision tree corresponding to the target classification model in accordance with the target format. Specifically, the step of pre-storing the node information in the decision tree corresponding to the target classification model in accordance with the target format may be performed by a central processing unit, such as Figure 3 As shown, the following steps may be included:

[0102] In step S301, based on the central processing unit, the above-mentioned model file is obtained;

[0103] In step S303, the central processing unit performs an operation of exporting the initial model structure information in a preset text format from the model file based on a preset export method, wherein the initial model structure information includes node information in a decision tree corresponding to the target classification model;

[0104] In step S305, based on the central processing unit, the node information is stored in the target format.

[0105] In a specific embodiment, the preset text format may be any text format, specifically, for example, TXT format. Specifically, the preset export method may be a pre-set method for exporting node information in a decision tree from a model file.

[0106] In the above embodiment, the central processing unit can effectively ensure that the classification task processing is performed in conjunction with the acceleration processor during the subsequent classification task processing by pre-storing the model file in a target format readable by the acceleration processor, thereby effectively improving the classification task processing efficiency.

[0107] In an optional embodiment, if Figure 4 As shown, the above method also includes:

[0108] In step S307, based on the central processor, node position information of each node information is determined;

[0109] In step S309, the central processor executes an operation of writing the node information corresponding to the same decision tree into the corresponding preset array based on the node position information;

[0110] Accordingly, in the above step S305, based on the central processor, storing the node information in the target format may include:

[0111] Based on the central processor, the node information in the preset array is stored according to the target format.

[0112] In a specific embodiment, the node position information may be the position information of the preset node in the corresponding preset array. Optionally, the decision tree corresponding to the target classification model may be multiple decision trees, each decision tree corresponds to a preset array, and accordingly, the node information may be written into the corresponding preset array in combination with the node position information.

[0113] In a specific embodiment, the node information may be stored in sequence and in a target format in combination with the hierarchical information of the nodes in the decision tree corresponding to the preset array.

[0114] In the above embodiment, the central processing unit can facilitate the distinction between different decision trees and improve the orderliness of the node information by writing the node information corresponding to the same decision tree into the corresponding preset array, thereby facilitating the central processing unit to perform format conversion more quickly and improving the accuracy and conversion efficiency of information during the format conversion process.

[0115] In an optional embodiment, the above method may further include:

[0116] Based on the central processing unit, obtaining a model type of a target classification model;

[0117] Accordingly, the operation of exporting the initial model structure information in a preset text format from the model file by the central processor based on the preset export method includes:

[0118] When the model type is a preset compatible model type, the central processing unit exports the initial model structure information in a preset text format from the model file based on a preset export method.

[0119] In practical applications, there are many types of random forest models (target classification models). In a specific embodiment, the preset compatible model type can be a model type of initial model structure information that can be preset in a preset text format in combination with a preset export method.

[0120] In a specific embodiment, in order to ensure the success rate of exporting the initial model structure information in a preset text format, the model type of the target classification model can be obtained. Accordingly, when the model type is a preset compatible model type, it can be determined that the preset export method supports exporting the initial model structure information in a preset text format from the model file of the target classification model.

[0121] In an optional embodiment, when the model type is not a preset compatible model type, the model format conversion process is exited, and optionally, an incompatible prompt message may be fed back to the terminal.

[0122] In the above embodiment, the central processing unit first identifies whether the model type of the target classification model is a compatible model type, which can effectively ensure the success rate of exporting the initial model structure information.

[0123] In an optional embodiment, the acceleration processor may include a data storage block. Accordingly, before the acceleration processor obtains the target model structure information of the target classification model corresponding to the target classification task, as Figure 5 As shown, the above method may further include the following steps:

[0124] In step S207, based on the acceleration processor, the node information in the target format is read;

[0125] In step S209, the node information is stored in the data storage block based on the acceleration processor.

[0126] In a specific embodiment, the acceleration processor resources can be divided into multiple slots, each slot is divided into multiple threads, each thread is divided into multiple blocks, and each thread is run in parallel in the slot, and each block in the thread is run in series. Optionally, the node information corresponding to each decision tree can be randomly written into a specified block of a specified thread.

[0127] In an optional embodiment, the method of obtaining the target model structure information of the target classification model corresponding to the target classification task based on the acceleration processor may include:

[0128] The operation of loading node information from the data storage block is performed by the acceleration processor.

[0129] In the above embodiment, before the classification task is processed, by pre-storing the node information in the target format in the data storage block of the acceleration processor, the target model structure information can be quickly obtained during task processing, thereby reducing the device processing delay and improving the task processing speed and efficiency.

[0130] In an optional embodiment, the node information stored in the target format is a plurality of data blocks of preset sizes; correspondingly, based on the central processor, storing the node information in the preset array in the target format may include: based on the central processor, storing the node information corresponding to the same node position information in the preset array corresponding to any two decision trees as the same data block.

[0131] In a specific embodiment, the node information corresponding to the same node position information is stored as the same data block, which can facilitate the parallel acquisition of the node information corresponding to different decision trees, thereby improving the subsequent task processing speed.

[0132] In practical applications, in the process of storing node information in a data block of a preset size, the address information of the data storage block corresponding to the node information can also be stored in the data block of the preset size. In a specific embodiment, the address information corresponding to the same node position information in different blocks (data storage blocks) is the same, and the identification information of the data storage block is often much smaller than the address information. Accordingly, in order to reduce the space occupied by the address information, the identification information of the data storage block where the two node information are located and the same address information corresponding to the two node information can be stored in the data block.

[0133] In a specific embodiment, taking a data block of a preset size of 512-bit CacheLine as an example, the 512-bit CacheLine can be divided into a 32-bit header data block and a 480-bit data block; in a specific embodiment, assuming that the node information of a node occupies 240 bits, correspondingly, two node information can be stored in a 480-bit data block; the 32-bit header data block stores the identification information of the data storage blocks where the two node information are located and the same address information corresponding to the two node information.

[0134] In the above embodiment, by storing the node information corresponding to the same node position information in the preset array corresponding to any two decision trees as the same data block, it is convenient to obtain the node information corresponding to different decision trees in parallel, improve the subsequent task processing speed, and thus improve the performance of the server.

[0135] In an optional embodiment, the above method may further include:

[0136] Based on the central processing unit, the node information corresponding to the leaf node and the node information of the parent node of the leaf node are merged to obtain the merged node information;

[0137] Based on the central processing unit, the node information corresponding to the same node position information in the preset array corresponding to any two decision trees is stored as the same data block including:

[0138] Through the central processing unit, the merged node information corresponding to the same node position information in the preset array corresponding to any two decision trees is stored as the same data block.

[0139] In practical applications, the amount of node information data of a leaf node is relatively small. Optionally, the node information corresponding to the leaf node and the node information of the parent node of the leaf node may be merged to obtain merged node information.

[0140] In addition, it should be noted that, when a non-leaf node has no leaf node, after the merging process, the merged node information of the non-leaf node is the node information of the non-leaf node.

[0141] In the above embodiment, by storing the node information of the leaf node in the node information of the parent node, the space occupied by the node information of the leaf node can be saved. Since the node information of the parent node of a leaf node is often obtained before the node information of the leaf node (i.e., the child node), storing the node information of the leaf node in the node information of the parent node can ensure that in the subsequent task processing process, the node information of the leaf node that can reflect the classification result can be obtained in advance, which can effectively improve the task processing speed and thus improve the performance of the server.

[0142] In an optional embodiment, the acceleration processor may include multiple serial data storage blocks under the same parallel thread, each data storage block includes node information of at least one decision tree in the decision tree corresponding to the target classification model. Optionally, the method may further include:

[0143] Based on the central processing unit, determining a target data storage block at the end of the parallel thread from a plurality of serial data storage blocks;

[0144] Based on the central processor, determining a target decision tree in at least one decision tree corresponding to the target data storage block;

[0145] Based on the central processing unit, the node information of the target decision tree, the position information in the data blocks of a plurality of preset sizes and the data block sizes corresponding to the target decision tree are recorded.

[0146] In a specific embodiment, the node information of at least one decision tree stored in the target data storage block may be arranged in a preset order. Optionally, the target decision tree may be the decision tree corresponding to the last node information stored in the at least one decision tree.

[0147] In a specific embodiment, the data block size corresponding to the target decision tree can be the size of all data blocks of the node information corresponding to the target decision tree. By recording the node information of the target decision tree, the position information and the data block size corresponding to the target decision tree in multiple data blocks of preset sizes, it is convenient for the subsequent acceleration processor to efficiently load the node information.

[0148] In a specific embodiment, after reading the node information from the last block of the current thread, combined with the above-mentioned location information and data block size, the CacheLine information of the leaf node of the last decision tree (target decision tree) of the block can be marked as the calculation end point; otherwise, the CacheLine information of the leaf node of the last decision tree is pointed to the first address of the next block of the current thread, so that the accelerated processor can more efficiently complete the information access of single slot, multiple threads and multiple blocks.

[0149] In the above embodiment, by recording the location information of the node information of the target decision tree in multiple data blocks of preset sizes, and the data block size corresponding to the target decision tree, it is convenient for the subsequent acceleration processor to efficiently load the node information, improve the task processing speed, and thus improve the performance of the server.

[0150] In step S205, the acceleration processor executes the operation of processing the classification task based on the information to be classified and the target model structure information to obtain the classification result information of the target classification task.

[0151] In a specific embodiment, in the process of processing the classification task based on the information to be classified and the target model structure information, the classification result information can be determined by comparing the node information of each decision tree in the information to be classified and the target model structure information.

[0152] In a specific embodiment, the target model structure information includes node information in multiple decision trees corresponding to the target classification model. The following takes the node information (model structure information) corresponding to a decision tree as an example to introduce the classification task processing process, specifically, Figure 6 As shown, Figure 6 It is a schematic diagram of model structure information corresponding to a decision tree according to an exemplary embodiment. Specifically, 603, 606, 607, 608 and 609 are leaf nodes, and Y is the node result of the leaf node; 601, 602, 604, 605 are non-leaf nodes. Assume that the information to be classified is a vector [X1, X2], and X1=1, X2=2; first, the classification feature of the root node 601 of the decision tree is X2, the classification symbol corresponding to the classification feature X2 is <=, and the classification threshold corresponding to the classification feature X2 is 2.5; because X2=2<=2.5 in the information to be classified; accordingly, the result is yes, and further classification processing can be performed in combination with the left child node 602, the classification feature of the left child node 602 is X2, the classification symbol corresponding to the classification feature X2 is <=, and the classification threshold corresponding to the classification feature X2 is 1.5; by In the information to be classified, X2=2>1.5, the result is no, and further classification can be performed in combination with the right child node 605. The classification feature of the right child node 605 is X1, and the classification symbol corresponding to the classification feature X1 is <=, and the classification threshold corresponding to the classification feature X1 is 1.5; since X1=1<=1.5 in the information to be classified, the result is yes, and further classification can be performed in combination with the left child node 608, and since the left child node 608 is a leaf node, the category probability can be determined to be the node result Y=0.2 corresponding to the leaf node 608. Further, assuming Figure 6The weight information corresponding to the decision tree is 0.3; and the classification results corresponding to the other decision trees corresponding to the target classification model (assuming that the number of other decision trees is 3) are 0.25, 0.35, and 0.4 respectively; and the corresponding weight information is 0.25, 0.25, and 0.2 respectively; accordingly, the final category probability can be:

[0153] 0.2*0.3+0.25*0.25+0.35*0.25+0.4*0.2=0.06+0.0625+0.0875+0.08=0.29

[0154] In a specific embodiment, assuming that the target classification task is a traffic identification task (identifying abnormal traffic), accordingly, 0.29 may be the probability that the information to be classified belongs to abnormal traffic; further, 0.29 may be compared with a threshold for identifying abnormal traffic, assuming that the threshold is 0.8; since 0.29<0.8; accordingly, the target classification result may be non-abnormal traffic.

[0155] It can be seen from the technical solutions provided in the above embodiments of this specification that, in this specification, after the model file of the target classification model is formatted in advance, the node information obtained by storing it in a readable format of the acceleration processor can be used to perform classification task processing operations based on the acceleration processor during the task processing process, thereby effectively reducing the delay of classification task processing, improving the speed and efficiency of classification task processing, and improving the performance of the server.

[0156] Figure 7 is a block diagram of a classification task processing device according to an exemplary embodiment. Figure 7 , the device comprises:

[0157] The processing request receiving module 710 runs on the acceleration processor and is configured to execute and receive a processing request for a target classification task, wherein the processing request includes information to be classified corresponding to the target classification task;

[0158] The target model structure information acquisition module 720 runs on the acceleration processor and is configured to execute the acquisition of the target model structure information of the target classification model corresponding to the target classification task, where the target model structure information is node information stored in a target format after the format conversion is performed in advance based on the model file of the target classification model; the target format is a readable format of the acceleration processor, and the node information is node information in the decision tree corresponding to the target classification model;

[0159] The classification task processing module 730 runs on the acceleration processor and is configured to perform classification task processing based on the information to be classified and the target model structure information to obtain classification result information of the target classification task.

[0160] Optionally, the acceleration processor includes a data storage block, and the above device further includes:

[0161] A node information reading module, running on the acceleration processor, is configured to read the node information in the target format before acquiring the target model structure information of the target classification model corresponding to the target classification task;

[0162] The node information storage module runs on the acceleration processor and is configured to execute the storage of the node information into the data storage block.

[0163] Optionally, the target model structure information acquisition module 720 is specifically configured to execute loading of node information from a data storage block.

[0164] Optionally, the above device further comprises:

[0165] A model file acquisition module, running on the central processing unit, is configured to execute acquisition of the model file;

[0166] An initial model structure information export module, running on a central processing unit, is configured to execute, based on a preset export method, exporting initial model structure information in a preset text format from a model file, the initial model structure information including node information;

[0167] The node information storage module runs on the central processing unit and is configured to execute the storage of the node information according to the target format.

[0168] Optionally, the decision tree corresponding to the target classification model is a plurality of decision trees, and the above device further includes:

[0169] An information determination module, running on the central processing unit, is configured to determine the node position information of each node information;

[0170] The node information writing module runs on the central processing unit and is configured to write the node information corresponding to the same decision tree into the corresponding preset array based on the node position information;

[0171] The node information storage module is also configured to store the node information in the preset array according to the target format.

[0172] Optionally, the node information stored in the target format is a plurality of data blocks of preset sizes; the node information storage module is further configured to execute the storage of the node information corresponding to the same node position information in the preset array corresponding to any two decision trees as the same data block.

[0173] Optionally, the above device further comprises:

[0174] A merging processing module, running on the central processing unit, is configured to execute merging processing of the node information corresponding to the leaf node and the node information of the parent node of the leaf node to obtain merged node information;

[0175] The node information storage module is also configured to execute the storage of the merged node information corresponding to the same node position information in the preset array corresponding to any two decision trees as the same data block.

[0176] Optionally, the acceleration processor includes a plurality of serial data storage blocks under the same parallel thread, each data storage block includes node information of at least one decision tree in the decision tree corresponding to the target classification model, and the above device also includes:

[0177] A target data storage block determination module, running on the central processing unit, is configured to determine a target data storage block at the end of the parallel thread from a plurality of serial data storage blocks;

[0178] A target decision tree determination module, running on the central processing unit, is configured to execute and determine a target decision tree in at least one decision tree corresponding to the target data storage block;

[0179] The information recording module runs on the central processing unit and is configured to execute and record the node information of the target decision tree, the position information in a plurality of data blocks of preset sizes and the data block size corresponding to the target decision tree.

[0180] Optionally, the above device further includes:

[0181] A model type acquisition module, running on the central processing unit, is configured to execute acquisition of the model type of the target classification model;

[0182] The initial model structure information export module is also configured to perform:

[0183] When the model type is a preset compatible model type, initial model structure information in a preset text format is exported from the model file based on a preset export method.

[0184] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0185] Figure 8 is a block diagram of an electronic device for processing classification tasks according to an exemplary embodiment. The electronic device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8As shown. The electronic device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a classification task processing method is implemented.

[0186] Those skilled in the art will understand that Figure 8 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the electronic device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0187] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the classification task processing method in the embodiment of the present disclosure.

[0188] In an exemplary embodiment, a computer-readable storage medium is also provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can perform the classification task processing method in the embodiment of the present disclosure.

[0189] In an exemplary embodiment, a computer program product including instructions is also provided. When the computer program product is run on a computer, the computer is enabled to execute the classification task processing method in the embodiment of the present disclosure.

[0190] In addition, it should be noted that in the embodiments of this specification, the processor may include a central processing unit and an accelerated processing unit.

[0191] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0192] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0193] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A classification task processing method, characterized in that: include: Receiving a processing request for a target classification task based on an acceleration processor, the processing request including information to be classified corresponding to the target classification task; Based on the acceleration processor, the target model structure information of the target classification model corresponding to the target classification task is obtained, wherein the target model structure information is node information stored in a target format after format conversion is performed in advance based on the model file of the target classification model; the target format is a readable format of the acceleration processor, and the node information is node information in a decision tree corresponding to the target classification model; The acceleration processor performs an operation of processing a classification task based on the information to be classified and the target model structure information to obtain classification result information of the target classification task.

2. The classification task processing method according to claim 1, characterized in that: The acceleration processor includes a data storage block. Before acquiring the target model structure information of the target classification model corresponding to the target classification task based on the acceleration processor, the method further includes: Based on the acceleration processor, reading the node information in the target format; The node information is stored in the data storage block based on the acceleration processor.

3. The classification task processing method according to claim 2, characterized in that: The acquiring, based on the accelerated processor, target model structure information of the target classification model corresponding to the target classification task comprises: The operation of loading the node information from the data storage block is performed by the acceleration processor.

4. The classification task processing method according to any one of claims 1 to 3, characterized in that: The method further comprises: Based on a central processing unit, obtaining the model file; By means of the central processor, an operation of exporting initial model structure information in a preset text format from the model file is performed based on a preset export method, wherein the initial model structure information includes the node information; Based on the central processor, the node information is stored according to the target format.

5. The classification task processing method according to claim 4, characterized in that: The decision tree corresponding to the target classification model is a plurality of decision trees, and the method further includes: Based on the central processor, determining node position information of each node information; By means of the central processor, an operation of writing the node information corresponding to the same decision tree into the corresponding preset array is executed based on the node position information; Based on the central processor, storing the node information according to the target format includes: Based on the central processor, the node information in the preset array is stored according to the target format.

6. The classification task processing method according to claim 5, characterized in that: The node information stored in the target format is a plurality of data blocks of preset sizes; Based on the central processor, storing the node information in the preset array according to the target format includes: Based on the central processor, the node information corresponding to the same node position information in the preset array corresponding to any two decision trees is stored as the same data block.

7. The classification task processing method according to claim 6, characterized in that: The method further comprises: Based on the central processor, the node information corresponding to the leaf node and the node information of the parent node of the leaf node are merged to obtain merged node information; Based on the central processor, storing the node information corresponding to the same node position information in the preset array corresponding to any two decision trees as the same data block includes: Through the central processor, the merged node information corresponding to the same node position information in the preset array corresponding to any two decision trees is stored as the same data block.

8. The classification task processing method according to claim 6, characterized in that: The acceleration processor includes a plurality of serial data storage blocks under the same parallel thread, each data storage block includes node information of at least one decision tree in the decision tree corresponding to the target classification model, and the method further includes: Based on the central processor, determining a target data storage block at the end of the parallel thread from the plurality of serial data storage blocks; Based on the central processor, determining a target decision tree in at least one decision tree corresponding to the target data storage block; Based on the central processor, the node information of the target decision tree, the position information in the data blocks of the plurality of preset sizes and the data block size corresponding to the target decision tree are recorded.

9. The classification task processing method according to claim 4, characterized in that: The method further comprises: Based on the central processor, obtaining a model type of the target classification model; The operation of exporting the initial model structure information in a preset text format from the model file based on a preset export method by a central processing unit includes: In the case that the model type is a preset compatible model type, the central processor exports initial model structure information in a preset text format from the model file based on a preset export method.

10. A classification task processing device, characterized in that: include: A processing request receiving module, running on the acceleration processor, is configured to execute a processing request for receiving a target classification task, wherein the processing request includes information to be classified corresponding to the target classification task; A target model structure information acquisition module, running on the acceleration processor, is configured to execute acquisition of target model structure information of the target classification model corresponding to the target classification task, wherein the target model structure information is node information stored in a target format after format conversion is performed in advance based on a model file of the target classification model; the target format is a readable format of the acceleration processor, and the node information is node information in a decision tree corresponding to the target classification model; The classification task processing module runs on the acceleration processor and is configured to perform classification task processing based on the information to be classified and the target model structure information to obtain classification result information of the target classification task.

11. The classification task processing device according to claim 10, characterized in that: The acceleration processor includes a data storage block, and the device further includes: A node information reading module, running on the acceleration processor, is configured to read the node information in the target format before acquiring the target model structure information of the target classification model corresponding to the target classification task; The node information storage module runs on the acceleration processor and is configured to execute the storage of the node information into the data storage block.

12. The classification task processing device according to claim 11, characterized in that: The target model structure information acquisition module is specifically configured to execute loading of the node information from the data storage block.

13. The classification task processing device according to any one of claims 10 to 12, characterized in that: The device also includes: A model file acquisition module, running on the central processing unit, is configured to execute acquisition of the model file; An initial model structure information export module, running on the central processing unit, is configured to execute, based on a preset export method, exporting the initial model structure information in a preset text format from the model file, the initial model structure information including the node information; The node information storage module runs on the central processing unit and is configured to store the node information according to the target format.

14. The classification task processing device according to claim 13, characterized in that: The decision tree corresponding to the target classification model is a plurality of decision trees, and the device further includes: An information determination module, running on the central processor, is configured to determine the node position information of each node information; A node information writing module, running on the central processing unit, is configured to write the node information corresponding to the same decision tree into a corresponding preset array based on the node position information; The node information storage module is further configured to store the node information in the preset array according to the target format.

15. The classification task processing device according to claim 14, characterized in that: The node information stored in the target format is a plurality of data blocks of preset sizes; the node information storage module is further configured to execute the storage of the node information corresponding to the same node position information in the preset array corresponding to any two decision trees as the same data block.

16. The classification task processing device according to claim 15, characterized in that: The device also includes: A merging processing module, running on the central processing unit, is configured to perform merging processing on the node information corresponding to the leaf node and the node information of the parent node of the leaf node to obtain merged node information; The node information storage module is further configured to execute the storage of the merged node information corresponding to the same node position information in the preset array corresponding to any two decision trees as the same data block.

17. The classification task processing device according to claim 15, characterized in that: The acceleration processor includes a plurality of serial data storage blocks under the same parallel thread, each data storage block includes node information of at least one decision tree in the decision tree corresponding to the target classification model, and the device also includes: A target data storage block determination module, running on the central processing unit, is configured to determine, from the plurality of serial data storage blocks, a target data storage block at the end of the parallel thread; A target decision tree determination module, running on the central processing unit, is configured to determine a target decision tree in at least one decision tree corresponding to the target data storage block; The information recording module runs on the central processing unit and is configured to record the node information of the target decision tree, the position information in the data blocks of the plurality of preset sizes and the data block size corresponding to the target decision tree.

18. The classification task processing device according to claim 13, characterized in that: The device also includes: A model type acquisition module, running on the central processing unit, is configured to execute acquisition of the model type of the target classification model; The initial model structure information export module is further configured to execute: In the case that the model type is a preset compatible model type, initial model structure information in a preset text format is exported from the model file based on a preset export method.

19. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the classification task processing method as described in any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the classification task processing method as described in any one of claims 1 to 9.

21. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the classification task processing method according to any one of claims 1 to 9 is implemented.

Citation Information

Patent Citations

  • Target data classification method and device, storage medium and electronic device

    CN108334895A

  • File format conversion method, system and related components

    CN110399596A

  • Medical image classification method and device, mobile terminal and medium

    CN111310793A

  • Training neural network accelerators using mixed precision data formats

    US20200193274A1