Applet classification method and apparatus, device, and computer-readable storage medium

Abstract

Embodiments of the present application provide a kind of applet classification method, device, equipment and computer readable storage medium, wherein the method comprises: obtaining the applet code of applet to be classified;Run the applet code, to obtain the dynamic characteristics of the applet to be classified in the running process;Based on the dynamic characteristics and the trained classifier model, determine the classification information of the applet to be classified;The classification information of the applet to be classified is stored.Through the present application, the dynamic characteristics of applet can be used to classify applet, and the recall rate of classification result is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of applet classification, and relate to but are not limited to an applet classification method, device, equipment and computer readable storage medium. BACKGROUND

[0002] An applet is an application form between a traditional H5 web page and a traditional native Android / IOS application. The applet can be used without downloading and installing, which saves the installation process compared with a dedicated client and realizes the dream of "reaching out to touch" the application, and therefore has a very broad user and developer.

[0003] At present, applets can be divided into service-free applets and service-providing applets. The service-free applet refers to an applet that can only display some basic information such as enterprise introduction or resume display, and does not provide other actual services. In order to present the applet that can provide actual services to the user when the user searches for the applet, the service-free applet can be identified by classifying when the applet is uploaded. SUMMARY

[0004] Embodiments of the present application provide an applet classification method, device, equipment and computer readable storage medium, which classifies the applet through the dynamic characteristics of the applet, and can improve the recall rate of the classification result.

[0005] The technical solution of the embodiments of the present application is as follows:

[0006] The embodiments of the present application provide an applet classification method, which comprises:

[0007] Obtaining the applet code of the applet to be classified;

[0008] Running the applet code to obtain the dynamic characteristics of the applet to be classified in the running process;

[0009] Determining the classification information of the applet to be classified based on the dynamic characteristics and the trained classifier model;

[0010] Storing the classification information of the applet to be classified.

[0011] The embodiments of the present application provide an applet classification device, which comprises:

[0012] A first obtaining module for obtaining the applet code of the applet to be classified;

[0013] A running module for running the applet code to obtain the dynamic characteristics of the applet to be classified in the running process;

[0014] The first determining module is configured to determine the classification information of the small program to be classified based on the dynamic feature and the trained classifier model.

[0015] The storage module is configured to store the classification information of the small program to be classified.

[0016] In some embodiments, the running module is further configured to:

[0017] run the small program code to obtain a first small program interface image;

[0018] trigger each preset event in sequence, obtain a target event that is triggered successfully, and obtain application program interface information called when the target event is triggered and control information corresponding to the target event;

[0019] obtain a second small program interface image after triggering the preset events;

[0020] determine a dynamic feature of the small program to be classified in a running process based on the number of target events, the application program interface information, the control information, the first small program interface image, and the second small program interface image.

[0021] In some embodiments, the running module is further configured to:

[0022] determine a total number of calls of each application program interface based on the application program interface information;

[0023] determine a number of interactive controls based on the control information;

[0024] determine image difference information between the first small program interface image and the second small program interface image;

[0025] determine the dynamic feature based on the number of target events, the total number of calls of each application program interface, the number of interactive controls, and the image difference information.

[0026] In some embodiments, the trained classifier model includes at least K first classifier models, and the first determining module is further configured to:

[0027] input the dynamic feature into the K first classifier models to obtain K initial prediction values, where K is an integer greater than 1;

[0028] determine a target prediction value based on the K initial prediction values;

[0029] determine the classification information of the small program to be classified based on the target prediction value and a preset classification threshold.

[0030] In some embodiments, the trained classifier model further comprises a trained second classifier model, and the first determining module is further configured to:

[0031] input the K initial prediction values into the second classifier model to perform ensemble processing on the K initial prediction values to obtain a target prediction value.

[0032] In some embodiments, the first determining module is further configured to:

[0033] when the target prediction value is greater than the classification threshold, determine that the classification information of the small program to be classified is a first type of small program;

[0034] when the target prediction value is less than or equal to the classification threshold, determine that the classification information of the small program to be classified is a second type of small program.

[0035] In some embodiments, the apparatus further comprises:

[0036] a second obtaining module configured to obtain a first training data set and a preset M first candidate classifier models, wherein the first training data set comprises dynamic features of training small programs and label information of the training small programs, and M is an integer greater than 2;

[0037] a second determining module configured to determine performance parameters corresponding to the M first candidate classifier models based on the first training data set;

[0038] a third determining module configured to determine K first classifier models based on the performance parameters corresponding to the M first candidate classifier models;

[0039] a first training module configured to train the K first classifier models using the first training data set to obtain K trained first classifier models.

[0040] In some embodiments, the apparatus further comprises:

[0041] a data construction module configured to construct a second training data set using the first training data set and the K first classifier models, wherein the second training data set comprises prediction information of the K first classifier models on training small programs and label information of the training small programs;

[0042] a third obtaining module configured to obtain a preset N second candidate classifier models and determine performance parameters corresponding to the N second candidate classifier models based on the second training data set, wherein N is an integer greater than 1;

[0043] a fourth determining module, configured to determine a second classifier model from the N second candidate classifier models based on the performance parameters corresponding to the N second candidate classifier models and the performance parameters corresponding to the K first classifier models;

[0044] a second training module, configured to train the second classifier model by using a second training data set to obtain a trained second classifier model.

[0045] In some embodiments, the data construction module is further configured to:

[0046] divide the first training data set into P first training data subsets, where P is an integer greater than 1;

[0047] determine the i-th first training data subset as the i-th test data set; i = 1, 2, …, P;

[0048] train the K first classifier models by using other first training data subsets to obtain K trained first classifier models; where the other first training data subsets are P-1 first training data subsets except the i-th first training data subset;

[0049] perform prediction processing on the i-th test data set by using the K trained first classifier models to obtain prediction information of the K first classifier models on training sub-programs in the i-th test data set;

[0050] determine the prediction information of the K first classifier models on training sub-programs in the first to P-th test data sets and the label information of the training sub-programs as a second training data set.

[0051] In some embodiments, the fourth determining module is further configured to:

[0052] compare the performance parameters corresponding to the N second candidate classifier models and the performance parameters corresponding to the K first classifier models in sequence;

[0053] when it is determined that the performance parameters corresponding to the j-th second candidate classifier model are all better than the performance parameters corresponding to the K first classifier models, determine a performance difference value between the j-th second candidate classifier model and the K first classifier models;

[0054] when the performance difference value is greater than a preset difference threshold, determine the j-th second candidate classifier model as the second classifier model, where j is an integer between 1 and N.

[0055] In some embodiments, the fourth determining module is further configured to:

[0056] Based on the performance parameters corresponding to the N second candidate classifier models and the performance parameters corresponding to the K first classifier models, Q second target classifier models are determined, and the performance parameters corresponding to the second target classifier models are all better than the performance parameters corresponding to the K first classifier models;

[0057] Q performance difference values between the Q second target classifier models and the K first classifier models are respectively determined.

[0058] Based on the Q performance difference values, a second classifier model is determined from the Q second target classifier models.

[0059] Embodiments of the present application provide a small program classification device, comprising:

[0060] The memory is configured to store executable instructions, and the processor is configured to execute the executable instructions stored in the memory to implement the method described above.

[0061] Embodiments of the present application provide a computer readable storage medium storing executable instructions for causing a processor to execute when the executable instructions are executed, to implement the method described above.

[0062] Embodiments of the present application have the following beneficial effects:

[0063] After obtaining the small program code of the small program to be classified, the small program code is run to obtain the dynamic characteristics of the small program to be classified in the running process, and then based on the dynamic characteristics and the trained classifier model, the classification information of the small program to be classified is determined and stored. Since the dynamic characteristics are extracted during the running of the small program, the events, API calls, etc. that can actually be triggered during the use of the small program can be reflected, and then the small program is classified by using the dynamic characteristics of the small program, which can improve the recall rate of the classification result. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 A network architecture schematic diagram of a small program classification system provided by embodiments of the present application is provided.

[0065] Figure 2 A structure schematic diagram of a server 300 provided by embodiments of the present application is provided.

[0066] Figure 3 An implementation flow schematic diagram of a small program classification method provided by embodiments of the present application is provided.

[0067] Figure 4A An implementation flow schematic diagram of obtaining a trained classifier model provided by embodiments of the present application is provided.

[0068] Figure 4BAnother implementation process schematic diagram for acquiring the trained classifier model provided by the embodiment of the present application is provided.

[0069] Figure 5 Another implementation process schematic diagram for the applet classification method provided by the embodiment of the present application is provided. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be described in further detail below with reference to the drawings, and the described embodiments should not be regarded as limiting the present application. All other embodiments obtained by those skilled in the art without making creative efforts belong to the scope of protection of the present application.

[0071] In the following description, "some embodiments" are related to a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as understood by those skilled in the art to which the embodiments of the present application belong. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0072] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0073] 1) Applet, also known as web application, is software downloaded by a client (such as a browser or any client with an embedded browser core) via a network (such as the Internet) and interpreted and executed in the browser environment of the client. It is an application form between traditional H5 web pages and traditional native Android / IOS applications; for example, in a social network client, a web application for implementing ticket purchase, ride code and various services can be downloaded and run.

[0074] 2) Accuracy, an index for evaluating the performance of a classification model or a machine learning model, expressed by the percentage of correct predictions in total samples.

[0075] The expression of accuracy is: accuracy = (TP + TN) / (TP + TN + FP + FN); where TP is predicted as 1, actually as 1, and predicted correctly; FP is predicted as 1, actually as 0, and predicted incorrectly; FN is predicted as 0, actually as 1, and predicted incorrectly; TN is predicted as 0, actually as 0, and predicted correctly.

[0076] Although the accuracy can determine the total accuracy, it cannot be a good indicator to measure the result in the case of uneven samples.

[0077] 3) Precision, also known as accuracy, precision (Precision) is for the predicted results, which means the probability of actual positive samples in all predicted positive samples.

[0078] The expression of accuracy is: precision = TP / (TP+FP).

[0079] 4) Recall, which is for the original sample, which means the probability of being predicted as positive in the actual positive sample.

[0080] The expression of recall is: recall = TP / (TP+FN).

[0081] 5) F1 score (F1-Score), F1 score considers both precision and recall, and balances them to the highest.

[0082] The expression of F1 score is: F1 score = 2*precision*recall / (precision+recall).

[0083] 6) Receiver Operating Characteristic (ROC) curve, which is used to evaluate the pros and cons of a binary classifier. Compared with evaluation indicators such as accuracy, recall and F-score, ROC curve has a good feature: when the distribution of positive and negative samples in the test set changes, ROC curve can remain unchanged.

[0084] 7) Area Under Curve (AUC), which represents the area under the ROC curve, and is used to judge the pros and cons of the model. As shown in the ROC curve, the area of the connecting diagonal is exactly 0.5, and the meaning of the diagonal line is to randomly judge the prediction result, and the positive and negative samples should be 50%. In addition, the steeper the ROC curve, the better, so the ideal value is 1, that is, a square. Therefore, the value of AUC is generally between 0.5 and 1.

[0085] In order to better understand the small program classification method provided in the embodiments of the present application, first, the small program classification method in the related art is described:

[0086] In the related art, when classifying small programs, the method adopted is a method based on static statistical characteristics and rules, that is, the number of keys in the static code of the small program is counted, and the small program with the number of keys less than the specified value is taken as a small program without service.

[0087] Due to the difference between the static code of the applet and the online page display, and the limitation of the single feature division according to rules, the classification method can cause more misjudgments and omissions.

[0088] Based on this, in the embodiments of the present application, an applet classification method based on dynamic features is proposed. After obtaining the source code of the applet, the source code is executed to run the applet, so as to obtain the original dynamic features of the applet, extract and construct effective dynamic features, and classify the applet based on the dynamic features of the applet.

[0089] The following describes an exemplary application of the applet classification device provided by the embodiments of the present application. The applet classification device provided by the embodiments of the present application can be implemented as a notebook computer, a tablet computer, a desktop computer, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated message device, a portable game device), a smart robot, or any terminal with a screen display function, and can also be implemented as a server. In the following, an exemplary application when the applet classification device is implemented as a server will be described.

[0090] Referring to Figure 1 , Figure 1 A network architecture schematic diagram of the applet classification system provided by the embodiments of the present application is shown. As Figure 1 indicated, the applet classification system includes a user terminal 100, a developer terminal 200, and a server 300. The developer of the applet deploys the development framework of the applet (for example, can be Weex) on the developer terminal 200 (for example, a computer or other user terminal, a server) to complete the code development for the applet. The applet can be used to implement various services provided by service providers, for example, a ride code service, a delivery service, and online shopping, etc. The development framework provides a construction tool of the applet to encapsulate the code in the project of the applet into one or more JavaScript files capable of running in the browser environment of the client, and upload to the server 300 to request review and upload after the review of the server 300. The server 300 can be a server carrying the business logic of the business party, for example, a background server carrying the ride service of the ride service provider. In Figure 1 , the applet is stored in the server 300 corresponding to the first client. In some embodiments, the server 300 can also be a dedicated storage server, for example, a node with the shortest link to the terminal of the user in the content distribution network (CDN, Content Delivery Network).

[0091] The server 300 performs review on the applet after receiving the applet file, and runs the applet and triggers each event in turn after the review is passed, to obtain dynamic characteristics in the running process of the applet, and then classifies the applet according to the dynamic characteristics. Further, the applet is classified into a service-free applet and a service-enabled applet. When the server 300 receives the applet search request sent by the user terminal 100, the server 300 queries the applets matched with the search keyword carried in the search request based on the search keyword, and obtains the classification information of each applet matched. When the classification information of the matched applet is a service-free applet, the server 300 can filter out these service-free applets as shown in Figure 1 , and returns the search result including only the service-enabled applets to the user terminal 100. Alternatively, in some embodiments, the server 300 can arrange these service-free applets after the service-enabled applets, and returns the sorted applets to the user terminal 100.

[0092] Referring to Figure 2 , Figure 2 , a structural schematic diagram of the server 300 provided by the embodiments of the present application is shown in Figure 2 , the server 300 includes at least one processor 310, a memory 350, at least one network interface 320, and a user interface 330. The various components in the server 300 are coupled together through a bus system 340. It can be understood that the bus system 340 is used to realize the connection and communication between the components. The bus system 340 includes not only a data bus, but also a power bus, a control bus, and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 340 in Figure 2 .

[0093] The processor 310 can be an integrated circuit chip with signal processing capability, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0094] The user interface 330 includes one or more output devices 331 enabling presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 330 also includes one or more input devices 332, including user interface components that facilitate user input, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, other input buttons and controls.

[0095] The memory 350 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 350 optionally includes one or more storage devices remotely located from the processor(s) 310. The memory 350 comprises a volatile memory or non-volatile memory, and can also include both volatile and non-volatile memory. Non-volatile memory can be read only memory (ROM), volatile memory can be random access memory (RAM). The memory 350 described in embodiments of the present application is intended to include any suitable type of memory. In some embodiments, the memory 350 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or a subset or superset thereof, which are exemplarily explained below.

[0096] The operating system 351 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks.

[0097] The network communication module 352 is used to communicate with other computing devices via one or more (wired or wireless) network interfaces 320, exemplary network interfaces 320 include Bluetooth, wireless fidelity (WiFi), universal serial bus (USB), etc.

[0098] The input processing module 353 is used to detect and translate one or more user inputs or interactions from one or more input devices 332.

[0099] In some embodiments, the apparatus provided by the embodiments of the present application can be realized in software, Figure 2 A small program classification apparatus 354 stored in the memory 350 is shown, which can be a small program classification apparatus in the server 300, which can be software in the form of programs and plug-ins, etc., including the following software modules: a first acquisition module 3541, a running module 3542, a first determination module 3543, and a storage module 3544. These modules are logical, and can be combined or further split according to the functions implemented. The functions of each module will be explained below.

[0100] In some embodiments, the apparatus provided by the embodiments of the present application can be implemented in a hardware manner. For example, the apparatus provided by the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to perform the small program classification method provided by the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can be implemented by using one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic elements.

[0101] The small program classification method provided by the embodiments of the present application will be described below in combination with an exemplary application and implementation of the server 300 provided by the embodiments of the present application. The small program classification method provided by the embodiments of the present application is a small program classification method applied to a server, as shown in FIG. 1. Figure 3 , Figure 3 An implementation flowchart of the small program classification method provided by the embodiments of the present application will be described below in combination with the steps shown in FIG. 2. Figure 3

[0102] In step S101, the small program code of a small program to be classified is obtained.

[0103] Here, the small program to be classified can be submitted to the server by a small program developer after the small program code is developed. In implementation, step S101 can be that the server obtains the small program to be classified once every interval of time, for example, the server can obtain the small program to be classified received within a time period from a historical time point to a current time point every 12 hours. In some embodiments, in implementation, step S101 can also be that the server obtains the small program to be classified every time the server receives the small program submitted by the developer terminal, that is, determines the small program to be classified, and then performs the subsequent steps, that is, the small program to be classified can be obtained in real time.

[0104] In step S102, the small program code is run to obtain the dynamic feature of the small program to be classified in the running process.

[0105] ​Here, step S102, when implemented, can be to render and dynamically run the applet code, trigger each preset event in the applet running process, record the JS API called when triggering each event, count the number of controls bound to the events triggered successfully, respectively take screenshots of the applet pages that have not started triggering events and have triggered all events and perform base64 encoding, merge the above obtained data, and save as a json file, and then extract the dynamic features of the applet to be classified from the json file.

[0106] In the embodiments of the present application, the dynamic features of the applet can include statistical features, API features, and image features, wherein the statistical features can include the number of events triggered successfully, the number of APIs, and the number of interactive controls; the API features include the total number of times each API is called; and the image features can include difference information between the screenshots of the applet pages that have not started triggering events and have triggered all events.

[0107] Step S103, based on the dynamic features and the trained classifier model, determines the classification information of the applet to be classified.

[0108] Here, the trained classifier model can include, but is not limited to, logistic regression, support vector machine, decision tree, naive Bayes, K nearest neighbor, bagging K nearest neighbor, bagging decision tree, random forest, adaboost, gradient boosting decision tree. In the embodiments of the present application, the trained classification model can only include one or more trained base classifiers, at which time step S103, when implemented, can input the dynamic features into the one or more trained base classifiers to respectively correspond to obtain one or more initial prediction values, and then determine a target prediction value according to the one or more initial prediction values. In actual application, when multiple initial prediction values are obtained, the multiple initial prediction values can be averaged to obtain a target prediction value. The target prediction value is the probability that the applet to be classified is a non-service applet, and finally the classification information of the applet to be classified is determined according to the target prediction value. The classification information can be a service applet or a non-service applet.

[0109] In some embodiments, the trained classifier model can not only include multiple trained base classifiers, but also include a trained ensemble classifier, at which time step S103, when implemented, can input the dynamic features into the multiple trained base classifiers to respectively correspond to obtain multiple initial prediction values, then input the multiple initial prediction values into the ensemble classifier for data integration to obtain a target prediction value, and finally determine the classification information of the applet to be classified according to the target prediction value.

[0110] Step S104, stores the classification information of the applet to be classified.

[0111] Here, step S104, when implemented, can be to store the correspondence between the identification of the small program to be classified and the classification information, or to store the classification information as an attribute information of the small program to be classified.

[0112] In the small program classification method provided in the embodiments of the present application, after obtaining the small program code of the small program to be classified, the small program code is run to obtain the dynamic feature of the small program to be classified in the running process, and then based on the dynamic feature and the trained classifier model, the classification information of the small program to be classified is determined and the classification information is stored. Since the dynamic feature is extracted in the running process of the small program, it can reflect the events that can be actually triggered, the API called, etc. in the use process of the small program, and then the small program is classified by using the dynamic feature of the small program, which can improve the recall rate of the classification result.

[0113] In some embodiments, Figure 3 The step S102 shown can be implemented by the following steps:

[0114] Step S1021, running the small program code to obtain a first small program interface image.

[0115] Here, step S1021, when implemented, can render and run the small program code, at this time, the small program interface can be rendered in the display interface of the server, and the current first small program interface image is obtained, which is the interface image before any event is triggered.

[0116] Step S1022, triggering each preset event in turn, obtaining a target event triggered successfully, and obtaining application program interface information called when the target event is triggered and control information corresponding to the target event.

[0117] Here, the preset event can be a single-click event, a double-click event, or a long-press event on each control in the small program to be classified. Step S1022, when implemented, first identifies each control in the small program interface image based on the small program interface image, then triggers the preset event based on each control, obtains the target event triggered successfully, and obtains the application program interface information called when the target event is triggered, wherein the application program interface information at least includes an application program interface identification and the number of times the application program interface is called. In the embodiments of the present application, the control information corresponding to the target event is also obtained, wherein the control information at least includes a control identification.

[0118] In the embodiments of the present application, the mini-program interface can change after a successful event triggering, and therefore, after a successful event triggering, the mini-program interface image can be acquired again, the controls in the current mini-program interface image can be identified, and then the preset events can be triggered again for the controls in the current mini-program interface image, and the target event triggered successfully and the application program interface information called when triggering the target event and the control information corresponding to the target event can be acquired.

[0119] In step S1023, after triggering the preset events, the second mini-program interface image is acquired.

[0120] Here, after triggering all the preset events, the screen capture is performed to capture the second mini-program interface image.

[0121] In step S1024, the dynamic features of the to-be-classified mini-program in the running process are determined based on the target event number, the application program interface information, the control information, the first mini-program interface image and the second mini-program interface image.

[0122] Here, in the implementation of step S1024, the total number of calls of each application program interface can be determined based on the application program interface information, the number of interactive controls can be determined based on the control information, and the image difference information between the first mini-program interface image and the second mini-program interface image can be determined. The image difference information can be obtained by calculating the Hamming distance between the first mini-program interface image and the second mini-program interface image.

[0123] After obtaining the total number of calls of each application program interface, the total number of application program interfaces called by the to-be-classified mini-program can also be determined. In the embodiments of the present application, the target event number, the total number of calls of each application program interface, the total number of application program interfaces, the number of interactive controls, and the image difference information are determined as the dynamic features of the to-be-classified mini-program.

[0124] In the embodiments of steps S1021 to S1024, the dynamic features such as the total number of application program interfaces, the number of interactive controls, the total number of calls of each application program interface and the number of target events that can be triggered of the mini-program are obtained by running the mini-program, so that the obtained features can truly reflect the actual situation of the mini-program, and therefore, the recall rate of the classification result can be improved when the dynamic features are used to determine the classification information of the mini-program.

[0125] In some embodiments, the trained classifier model at least includes K trained first classifier models, and correspondingly, step S103 can be implemented by the following steps:

[0126] Step S1031: Input the dynamic feature into K first classifier models respectively to obtain K initial prediction values.

[0127] Where K is a positive integer, that is, K is an integer greater than or equal to 1. The first classifier model can be, but is not limited to, logistic regression, support vector machine, decision tree, Naive Bayes, K-nearest neighbors, bagging K-nearest neighbors, bagging decision tree, random forest, AdaBoost, and gradient boosting decision tree. The dynamic features of the mini-program to be classified are input into the K pre-trained first classifier models, which perform prediction processing on the mini-program to be classified, and obtain K initial prediction values. The initial prediction value is the initial probability value of the mini-program to be classified as a no-service mini-program, which is a real number between 0 and 1.

[0128] Step S1032: Determine the target predicted value based on the K initial predicted values.

[0129] Here, in step S1032, when K is 1, the initial predicted value is directly determined as the target predicted value. When K is an integer greater than 1, the target predicted value can be obtained by averaging the K initial predicted values. This averaging process can be either an arithmetic average or a weighted average.

[0130] In some embodiments, when K is an integer greater than 1, the first classifier model can be a base classifier model. The trained classifier model also includes a trained second classifier model, which is an ensemble classifier model. Correspondingly, in the implementation of step S1032, the K initial predicted values ​​can be input into the second classifier model to perform ensemble processing on the K initial predicted values ​​to obtain the target predicted value.

[0131] Step S1033: Based on the target predicted value and the preset classification threshold, determine the classification information of the mini-program to be classified.

[0132] Here, when implementing step S1033, the classification information of the mini-program to be classified can be determined by judging the relationship between the target predicted value and the classification threshold. Specifically, when the target predicted value is greater than the classification threshold, the classification information of the mini-program to be classified is determined to be a first type mini-program; when the target predicted value is less than or equal to the classification threshold, the classification information of the mini-program to be classified is determined to be a second type mini-program. The first type mini-program is a service-free mini-program, and the second type mini-program is a service-enabled mini-program.

[0133] In some embodiments, a trained classifier model needs to be obtained before step S103. In actual implementation, this can be achieved through... Figure 4A The following steps are shown to obtain the trained classifier model:

[0134] Step S001, obtaining a first training data set and a preset M first candidate classifier models.

[0135] The first training data set includes dynamic features of a plurality of training applets and label information of the plurality of training applets, and the label information is used to represent whether the training applet is a service-free applet or a service applet. For example, when the training applet is a service-free applet, the label information is 1, and when the training applet is a service applet, the label information is 0.

[0136] M is an integer greater than 1, and M is greater than or equal to K. The first candidate classifier model includes, but is not limited to, logistic regression, support vector machine, decision tree, naive Bayes, K nearest neighbor, bagging K nearest neighbor, bagging decision tree, random forest, adaboost, and gradient boosting decision tree. In implementation, M first candidate classifier models can be obtained by first determining the types of the M first candidate classifier models, and then searching for optimal hyperparameters of each first candidate classifier by using a grid search method.

[0137] Step S002, determining performance parameters corresponding to the M first candidate classifier models based on the first training data set.

[0138] Here, step S002 can be implemented by using an S-fold cross-validation method (for example, a ten-fold cross-validation method) to determine the performance parameters of the M first candidate classifier models under the first training data set. For each first candidate classifier model, one or more performance parameters can be determined. The performance parameters include, but are not limited to, accuracy, precision, recall, F1-score, ROC, and AU C. For example, for each first candidate classifier model, two performance parameters, precision and recall, can be determined.

[0139] Step S003, determining K first classifier models based on the performance parameters corresponding to the M first candidate classifier models.

[0140] Here, if in step S002, for each first candidate classifier model, one performance parameter is determined, then based on the performance parameter, the K first classifier models with the best performance are determined. For example, for each first candidate classifier model, only one performance parameter, precision, is determined. In implementation, step S003 can be implemented by sorting the precisions of the M first candidate classifier models, and then selecting the K first classifier models with the highest precisions from the M first candidate classifier models.

[0141] If it is determined that there are at least two performance parameters for each of the first candidate classifier models in step S002, step S003, when implemented, can be to select one most concerned performance parameter from the at least two performance parameters, and determine the K first classifier models from the M first candidate classifier models based on the most concerned performance parameter; or can be to select multiple most concerned performance parameters from the at least two performance parameters, and then perform an arithmetic average or a weighted average on the multiple most concerned performance parameters, or perform a summation operation, so as to determine the K first classifier models from the M first candidate classifier models.

[0142] Step S004, training the K first classifier models by using the first training data set, to obtain K trained first classifier models.

[0143] Here, step S004, when implemented, can be to input the dynamic features of multiple training sub-programs in the first training data set into the K first classifier models respectively, to correspondingly obtain training prediction values, and then adjust parameters of the K first classifier models according to differences between the training prediction values and label information of each training sub-program, until a preset training completion condition is reached, so as to obtain the K trained first classifier models. The training completion condition can be a preset training number of times, or can be that the difference between the training prediction value and the actual label information is less than a preset threshold.

[0144] Through the above steps S001 to S004, after obtaining the first training data set and the M first candidate classifier models, the K first classifier models with the best performance can be selected from the M first candidate classifier models based on the performance parameters of the M first candidate classifier models, and then the K first classifier models are trained based on the first training data set, so as to obtain the K trained first classifier models, and then the trained K first classifier models are used to classify the sub-programs to be classified, so as to determine classification information of the sub-programs to be classified.

[0145] In some embodiments, as shown in FIG. 5, after step S003, the trained second classifier model can also be obtained through the following steps: Figure 4B

[0146] Step S005, constructing a second training data set by using the first training data set and the K first classifier models.

[0147] Here, the second training data set includes prediction information of the K first classifier models on the training sub-programs and label information of the training sub-programs, and the prediction information at least includes a prediction probability value of the training sub-programs being non-service sub-programs.

[0148] ​Step S006, obtain N second candidate classifier models, and determine performance parameters of the N second candidate classifier models based on the second training data set.

[0149] Here, N is an integer greater than 1. In implementation, the N second candidate classifier models can be obtained by first determining the types of the N second candidate classifier models, and then searching for optimal hyperparameters of each second candidate classifier by grid search method, thereby obtaining the N second candidate classifier models.

[0150] In implementation, step S006 can be to determine the performance parameters of the N second candidate classifier models under the second training data set by S-fold cross-validation method (for example, ten-fold cross-validation method). For each second candidate classifier model, one or more performance parameters can be determined. The performance parameters include, but are not limited to, accuracy, precision, recall, F1-score, ROC, and AUC.

[0151] It should be noted that the types of the performance parameters determined in step S006 are the same as the types of the performance parameters determined in step S002. For example, if the precision and recall of the first classifier model are determined in step S002, then the precision and recall of the second classifier model are also determined in step S006.

[0152] Step S007, based on the performance parameters of the N second candidate classifier models and the performance parameters of the K first classifier models, determine a second classifier model from the N second candidate classifier models.

[0153] Here, step S007 has at least the following two implementation manners:

[0154] The first implementation manner is to compare the performance parameters of the N second candidate classifier models with the performance parameters of the K first classifier models in sequence. Once it is determined that the performance parameters of a second candidate classifier model are all better than those of the K first classifier models, when the performance difference value between this second candidate classifier model and the K first classifier models is greater than a preset threshold, the second candidate classifier model is determined as the second classifier model.

[0155] The second implementation manner is to determine the second candidate classifier model with the best performance parameters from the N second candidate classifier models whose performance parameters are all better than those of the K first classifier models as the second classifier model.

[0156] In some embodiments, the performance parameters of the N second candidate classifier models can also be compared with the performance parameters of the K first classifier models one by one, and once it is determined that the performance parameters of a second candidate classifier model are all better than those of the K first classifier models, the second candidate classifier model is determined as the second classifier model.

[0157] In step S008, the second classifier model is trained by using the second training data set to obtain a trained second classifier model.

[0158] Here, in the implementation of step S008, the prediction information of the K first classifier models on the training subprograms in the second training data set can be input into the second classifier model to obtain the training prediction value of the second classifier model on the training subprograms, and then the difference between the training prediction value and the label information of each training subprogram is used to adjust the parameters of the second classifier model until a preset training completion condition is reached, thereby obtaining the trained second classifier model. The training completion condition can be a preset training number of times, or the difference between the training prediction value and the actual label information can be less than a preset threshold.

[0159] In some embodiments, Figure 4B The step S005 shown can be implemented by the following steps:

[0160] In step S051, the first training data set is divided into P first training data subsets.

[0161] Here, P is an integer greater than 1. The value of P is determined by the S-fold cross-validation method used in step S002, where P=S, that is, the ten-fold cross-validation method is used in step S002, and then the first training data set is divided into 10 first training data subsets in this step S051.

[0162] In step S052, the i-th first training data subset is determined as the i-th test data set.

[0163] Here, i=1, 2, …, P.

[0164] In step S053, the K first classifier models are trained by using other first training data subsets to obtain K trained first classifier models.

[0165] Here, the other first training data subsets are (P-1) first training data subsets excluding the i-th first training data subset.

[0166] Step S054, predicting the ith test data set by using the K trained first classifier models to obtain the prediction information of the K first classifier models on the training applet in the ith test data set.

[0167] Here, steps S052 to S054 are performed cyclically P times, thereby obtaining the prediction information of the K first classifier models on the training applets in the first to Pth test data sets.

[0168] Step S055, determining the prediction information of the K first classifier models on the training applets in the first to Pth test data sets and the label information of the training applets as the second training data set.

[0169] Here, each training applet corresponds to K training prediction values, and the K training prediction values represent the prediction probability values of the K first classifier models on the training applet being a non-service applet.

[0170] In some embodiments, Figure 4B The illustrated step S007 can have the following two implementation manners:

[0171] The first implementation manner can be implemented by the following steps:

[0172] Step S071A, comparing the performance parameters corresponding to the N second candidate classifier models and the performance parameters corresponding to the K first classifier models in sequence.

[0173] Step S072A, when it is determined that the performance parameters corresponding to the jth second candidate classifier model are all better than the performance parameters corresponding to the K first classifier models, determining the performance difference value between the jth second candidate classifier model and the K first classifier models.

[0174] Step S073A, when the performance difference value is greater than a preset difference threshold, determining the jth second candidate classifier model as the second classifier model.

[0175] Wherein, j is an integer between 1 and N. When the performance difference value is less than or equal to the difference threshold, the performance parameters between the j+1th second candidate classifier model and the K first classifier models are continuously compared.

[0176] In the first implementation manner, the second classifier model can be determined without comparing all the N second classifier models, but in the first implementation manner, it cannot be guaranteed that the determined second classifier model is the one with the best performance among the N second candidate classifier models.

[0177] The second implementation manner can be implemented by the following steps:

[0178] Step S071B, determining Q second target classifier models based on the performance parameters corresponding to the N second candidate classifier models and the performance parameters corresponding to the K first classifier models.

[0179] Here, the performance parameters corresponding to the second target classifier models are all better than the performance parameters corresponding to the K first classifier models.

[0180] Step S072B, determining Q performance difference values between the Q second target classifier models and the K first classifier models respectively.

[0181] Step S073B, determining a second classifier model from the Q second target classifier models based on the Q performance difference values.

[0182] Here, step S073B can be implemented by determining the second classifier model with the best performance from the Q second target classifier models based on the Q performance difference values.

[0183] In the second implementation, Q second target classifier models with all performances better than the K first classifier models are first determined from the N second candidate classifier models, and then one with the best performance is determined from the Q second target classifier models as the second classifier model. The second implementation has a larger calculation amount than the first implementation, but can determine the second classifier model with the best performance. In actual application, whether to use the first implementation or the second implementation can be determined according to actual needs.

[0184] Based on the foregoing embodiments, the embodiments of the present application further provide a small program classification method, applied to Figure 1 a network architecture as shown in Figure 5 Another implementation process of the small program classification method provided by the embodiments of the present application is shown in Figure 5 The method comprises the following steps:

[0185] Step S501, a small program developer develops a small program code by using a developer terminal.

[0186] Step S502, the developer terminal sends the developed small program code to a server based on a received uploading operation.

[0187] Here, step S502 can be implemented by encapsulating the developed small program code into one or more JavaScript files capable of running in a browser environment of a client terminal and uploading the JavaScript files to the server after the small program code is developed.

[0188] Step S503, the server reviews the small program, and when the review is passed, the small program is put on the shelf.

[0189] Here, the server reviewing the applet can be to review whether the content of the applet conforms to the rules, such as whether it involves rumors, fraud, gambling and other illegal content, and can also review whether the applet code has defects (bugs), whether the function is complete, and the like. After the applet is reviewed and passed, the applet is put on the shelf, that is, the user can search and use the applet.

[0190] Step S504, the server runs the applet code to obtain the dynamic characteristics of the applet in the running process.

[0191] Step S505, the server inputs the dynamic characteristics into K first classifier models respectively, and correspondingly obtains K initial prediction values.

[0192] Wherein, K is an integer greater than 1. The first classifier model corresponds to the base classifier in other embodiments, and the initial prediction value is the probability value of the applet being a serviceless applet, which is a real number between 0 and 1.

[0193] Step S506, the server inputs the K initial prediction values into the second classifier model to integrate the K initial prediction values, and obtains a target prediction value.

[0194] Here, the second classifier model corresponds to the integrated classifier in other embodiments, and is used to integrate the K initial prediction values to obtain the final target prediction value.

[0195] Step S507, the server determines the classification information of the applet based on the target prediction value and a preset classification threshold.

[0196] Here, step S507 can determine the classification information of the applet to be classified by judging the size relationship between the target prediction value and the classification threshold, wherein when the target prediction value is greater than the classification threshold, the classification information of the applet to be classified is determined as the first type applet; when the target prediction value is less than or equal to the classification threshold, the classification information of the applet to be classified is determined as the second type applet, the first type applet is a serviceless applet, and the second type applet is a service applet.

[0197] Step S508, the server stores the classification information of the applet.

[0198] Here, it can be to store the correspondence between the identification of the applet and the classification information, and can also be to store the classification information as an attribute information of the applet.

[0199] Step S509, the user terminal acquires a search keyword in response to an applet search operation.

[0200] Step S510, the user terminal sends a search request to the server.

[0201] Here, the search request carries a search keyword.

[0202] Step S511, the server searches based on the search keyword in the search request to obtain a first search result.

[0203] Here, the server determines the applet identifier matching the search keyword from the applet identifiers stored by itself based on the search keyword, and determines the applet identifier as the first search result.

[0204] Step S512, the server obtains the classification information corresponding to each applet identifier in the first search result.

[0205] Step S513, the server deletes the applet identifier with no service applet from the first search result according to the classification information corresponding to each applet identifier to obtain a second search result.

[0206] Step S514, the server returns the second search result to the user terminal.

[0207] In the applet classification method provided in the embodiments of the present application, after the applet developer completes the applet code development, the applet code is sent to the server for review, and after the review is passed, the applet is put on the shelf. In addition, the server also runs the applet and triggers each preset event in turn to obtain the dynamic characteristics in the running process of the applet, and then determines the classification information of the applet according to the dynamic characteristics of the applet and the trained first classification model and second classification model. In this way, when the user terminal searches for the applet and sends a search request to the server, the server determines the first applet matching the search keyword, further obtains the classification information of each first applet, deletes the applet with no service according to the classification information, and returns the search result of the deleted applet with no service to the user terminal, so as to ensure that the user finally obtains the search result of the applet that can provide service.

[0208] Next, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0209] In the process of content portrait analysis of the applet, the applet needs to be classified to determine whether the applet is an applet with no service or an applet with service. "No service" means that the service provided is to display some basic information, such as enterprise introduction or resume display, etc., without other actual services.

[0210] The applet classification method provided in the embodiments of the present application includes the following steps:

[0211] Step S601, obtain original dynamic features of the applet.

[0212] The dynamic features of the applet are relative to the static features. The static features of the applet refer to the features that can be extracted from the static code of the applet, such as the number of controls, DOM, custom components, etc. However, the elements written in the static code are not necessarily presented or called, so the actual page presented to the user and the static code of the applet cannot correspond to each other. The dynamic features of the applet refer to the features that can be obtained after the applet code is rendered and dynamically run, such as which events can be triggered, the JS API called when the event is triggered, the control bound to the event, etc. This actually simulates the interaction between the user and the page. Therefore, the dynamic features can better reflect the real user interaction experience than the static features.

[0213] In addition to the dynamic features directly obtained from the dynamic code, the service also needs to pay attention to whether there is a change in the page seen by the user after the event is triggered, so the pages before and after the event is triggered need to be captured for comparison.

[0214] In the embodiments of the present application, when obtaining the original dynamic features of the applet, the applet code is rendered and dynamically run, each event is triggered in turn, the JS API called when each event is triggered is recorded, the number of controls bound to the triggerable events is counted, the applet pages before and after the event is triggered are captured and base64 encoded, all the above features are combined and stored as a json file, and the json file is saved to a database, such as a mysql database.

[0215] Step S602, extract and construct effective dynamic features.

[0216] Here, the extracted dynamic features can include statistical features, API features and picture features, wherein the statistical features can include the number of triggerable events, the number of APIs, and the number of interactive controls; the API features can include the total number of times each API is called, such as the total number of times the system information API is called, the total number of times the two-dimensional code scanning API is called, the total number of times the message prompt box display API is called, etc.; and the picture features can include the page capture before the event is triggered and the page capture after all events are triggered.

[0217] Step S603, classify the applet based on the dynamic features of the applet.

[0218] Here, step S603, when implemented, can be inputting the obtained small program dynamic features into the trained plurality of base classification models to obtain each prediction value of each base classification model for the small program, where the prediction value is a probability value that the small program is a serviceless small program; and then inputting each prediction value into the trained ensemble classifier model to integrate each prediction value through the ensemble classifier model to obtain a final prediction value, which is also a probability value that the small program is a serviceless small program. After obtaining the final prediction value, the final prediction value is compared with a preset classification threshold to obtain the classification information of the small program, for example, if the final prediction value is greater than the classification threshold, it is determined as a serviceless small program, and if the final prediction value is less than the classification threshold, it is determined as not a serviceless small program.

[0219] The dynamic features extracted in the embodiments of the present application are further described below. Table 1 is the small program dynamic features extracted from the json file in the embodiments of the present application:

[0220] Table 1, features extracted from the json file in the embodiments of the present application

[0221]

[0222]

[0223] In the above table 1, the feature category, feature name and variable name of the dynamic features needed to be extracted from the json file are shown.

[0224] In the embodiments of the present application, based on the picture features, the following features shown in table 2 are additionally constructed:

[0225] Table 2, dynamic features constructed based on picture features in the embodiments of the present application

[0226]

[0227] In the embodiments of the present application, step S602, when implemented, retains the statistical features and JS API features in table 1 and Hash_diff in table 2, removes Pic_0 and Pic_1 in table 1 and Hash_0 and Hash_1 in table 2, and obtains 22-dimensional dynamic features. The 22-dimensional dynamic features are input into the classifier model as input information in step S603 to determine the classification information of the small program.

[0228] In the embodiments of the present application, before step S603, the trained base classifier model and ensemble classifier model need to be obtained through the following steps:

[0229] Step S701, collect data with label information to construct a training set (X, y).

[0230] Here, the label information is used to represent whether the applet is a service-free applet or a service applet, and the label information can be 0 or 1. When the label information is 1, it indicates that the applet is a service-free applet, and when the label information is 0, it indicates that the applet is a service applet.

[0231] The data with the label information can be applet code. When step S501 is implemented, after the applet code with the label information is collected, the applet code is run and rendered, and the dynamic features of the applet are obtained, and then a training set (X, y) is constructed according to the label information and the dynamic features of the applet. Wherein, X ∈ R n×22 , y ∈ {0, 1} n×1 , n represents n samples, and 22 is the 22-dimensional features extracted in step S602. If y i is the i-th element of y, y i = 1 represents the i-th positive sample, that is, a service-free applet; y i = 0 represents the i-th negative sample, that is, not a service-free applet.

[0232] Step S702, constructing m classifiers.

[0233] Here, the m classifiers include but are not limited to logistic regression, support vector machine, decision tree, naive Bayes, K nearest neighbor, bagging K nearest neighbor, bagging decision tree, random forest, adaboost, gradient boosting decision tree. In the embodiments of the present application, the grid search method can be used to search for the optimal hyperparameters of each classifier, and the performance of each classifier on the training set (X, y) is evaluated under the optimal parameters by ten-fold cross-validation. The performance indicators include but are not limited to accuracy, precision, recall, F1-score, ROC AUC.

[0234] Step S703, selecting the best k classifiers in the most concerned indicators from the m classifiers as base classifiers, and adding a new classifier to stack the base classifiers.

[0235] Here, if the most concerned indicator is recall, then the k classifiers with the highest recall are selected from the m classifiers as base classifiers, and a new classifier is added as an integrated classifier to stack the base classifiers. The type of the added classifier includes but is not limited to logistic regression, support vector machine, decision tree, naive Bayes, K nearest neighbor, bagging K nearest neighbor, bagging decision tree, random forest, adaboost, gradient boosting decision tree.

[0236] In some embodiments, step S703 can be implemented by the following steps:

[0237] Step S7031, the training set (X, y) is divided into 10 parts, under each base classifier, each time 9 parts are taken as a training set to train the base classifier, and the remaining 1 part is input into the base classifier for prediction processing to obtain the probability value of no service. The above process is repeated 10 times.

[0238] Thus, the original training set (X, y) is converted into (X1, y) through k classifiers, where X1 ∈ R n×k .

[0239] Step S7032, a new classifier is added to integrate the prediction results of the base classifiers.

[0240] Here, step S7032 can be pre-set with the type of the candidate new classifier when implemented, for example, the type of the candidate new classifier can be three, which are logistic regression, support vector machine and decision tree. Then, the grid search method is used to search for the optimal hyperparameters of each candidate new classifier, and the performance of each candidate new classifier under the training set (X1, y) is evaluated by ten-fold cross-validation. If the performance of a certain candidate new classifier is worse than that of at least one base classifier, another candidate new classifier is used to re-try; if the performance of a certain candidate new classifier is better than that of all base classifiers, the candidate new classifier is determined as the integrated classifier, and (X, y) is used to train each base classifier and (X1, y) is used to train the integrated classifier. The trained base classifiers and integrated classifier are serialized and saved.

[0241] In some embodiments, after the performance parameters of each candidate new classifier are determined, the performance parameters of each candidate classifier are compared with the performance parameters of each base classifier, and the candidate new classifier that is superior to each base classifier is determined as the integrated classifier from each candidate new classifier.

[0242] Using the single-feature plus rule scheme, the accuracy of the no-service applet identification is 95%, and the recall rate is 28.1%. Although the accuracy is high, there is a large proportion of missed, which is not suitable for low-quality filtering scenarios. Using the classification method provided in the embodiments of the present application, the accuracy of the no-service applet identification is 77%, and the recall rate is 84%. The accuracy is exchanged for 18% and the recall rate is exchanged for 56%. Both the accuracy and the recall rate are high, and the accuracy and the recall rate can be balanced by selecting the decision threshold. The method can be applied to search low-quality filtering scenarios.

[0243] In the applet classification method provided in the embodiments of the present application, dynamic characteristics of the applet in the running process are obtained, and the applet is classified based on the dynamic characteristics, so that misjudgment and missed classification caused by differences between static code of the applet and online page display and limitations of single-feature classification according to rules can be avoided, and the overall classification performance is improved.

[0244] The following continues to describe an exemplary structure of the applet classification apparatus 354 provided in the embodiments of the present application, which is implemented as a software module. In some embodiments, as shown in FIG. 3, the software module stored in the applet classification apparatus 354 in the memory 350 can be an applet classification apparatus in the server 300, which includes: Figure 2

[0245] The first obtaining module 3541 is configured to obtain applet code of an applet to be classified.

[0246] The running module 3542 is configured to run the applet code to obtain dynamic characteristics of the applet to be classified in the running process.

[0247] The first determining module 3543 is configured to determine classification information of the applet to be classified based on the dynamic characteristics and a trained classifier model.

[0248] The storage module 3544 is configured to store the classification information of the applet to be classified.

[0249] In some embodiments, the running module 3542 is further configured to:

[0250] run the applet code to obtain a first applet interface image;

[0251] trigger each preset event in sequence, obtain a target event that is successfully triggered, and obtain application program interface information called when the target event is triggered and control information corresponding to the target event;

[0252] after triggering the preset events, obtain a second applet interface image;

[0253] determine the dynamic characteristics of the applet to be classified in the running process based on the number of target events, the application program interface information, the control information, the first applet interface image, and the second applet interface image.

[0254] In some embodiments, the running module 3542 is further configured to:

[0255] determine a total number of calls of each application program interface based on the application program interface information;

[0256] determine a number of interactive controls based on the control information;

[0257] ​determine image difference information between the first applet interface image and the second applet interface image;

[0258] determine the dynamic feature based on the target event quantity, the total number of invocations of each application program interface, the number of interactive controls, and the image difference information.

[0259] In some embodiments, the trained classifier model includes at least K first trained classifier models, and the first determination module 3543 is further configured to:

[0260] input the dynamic feature into the K first trained classifier models respectively, and correspondingly obtain K initial prediction values, where K is an integer greater than 1;

[0261] determine a target prediction value based on the K initial prediction values;

[0262] determine the classification information of the applet to be classified based on the target prediction value and a preset classification threshold.

[0263] In some embodiments, the trained classifier model further includes a second trained classifier model, and the first determination module 3543 is further configured to:

[0264] input the K initial prediction values into the second trained classifier model to perform ensemble processing on the K initial prediction values, and obtain a target prediction value.

[0265] In some embodiments, the first determination module 3543 is further configured to:

[0266] when the target prediction value is greater than the classification threshold, determine that the classification information of the applet to be classified is a first type of applet;

[0267] when the target prediction value is less than or equal to the classification threshold, determine that the classification information of the applet to be classified is a second type of applet.

[0268] In some embodiments, the apparatus further includes:

[0269] a second acquisition module configured to acquire a first training data set and a preset M number of first candidate classifier models, where the first training data set includes dynamic features of training applets and label information of the training applets, and M is an integer greater than 2;

[0270] a second determination module configured to determine performance parameters corresponding to the M first candidate classifier models based on the first training data set;

[0271] a third determination module configured to determine K first classifier models based on the performance parameters corresponding to the M first candidate classifier models.

[0272] The first training module is configured to train the K first classifier models by using the first training data set, to obtain K trained first classifier models.

[0273] In some embodiments, the apparatus further includes:

[0274] The data construction module is configured to construct a second training data set by using the first training data set and the K first classifier models, wherein the second training data set includes predicted information of the training small programs by the K first classifier models and label information of the training small programs.

[0275] The third obtaining module is configured to obtain N second candidate classifier models, and determine performance parameters corresponding to the N second candidate classifier models based on the second training data set, wherein N is an integer greater than 1.

[0276] The fourth determining module is configured to determine a second classifier model from the N second candidate classifier models based on the performance parameters corresponding to the N second candidate classifier models and the performance parameters corresponding to the K first classifier models.

[0277] The second training module is configured to train the second classifier model by using the second training data set, to obtain a trained second classifier model.

[0278] In some embodiments, the data construction module is further configured to:

[0279] divide the first training data set into P first training data subsets, wherein P is an integer greater than 1;

[0280] determine the i-th first training data subset as the i-th test data set; i = 1, 2, …, P;

[0281] train the K first classifier models by using other first training data subsets, to obtain K trained first classifier models; wherein the other first training data subsets are P-1 first training data subsets except the i-th first training data subset;

[0282] perform prediction processing on the i-th test data set by using the K trained first classifier models, to obtain predicted information of training small programs in the i-th test data set by the K first classifier models;

[0283] determine the predicted information of the training small programs in the first to P-th test data sets by the K first classifier models and the label information of the training small programs as the second training data set.

[0284] In some embodiments, the fourth determining module is further configured to:

[0285] compare the performance parameters corresponding to the N second candidate classifier models and the performance parameters corresponding to the K first classifier models in sequence;

[0286] when it is determined that the performance parameters corresponding to the jth second candidate classifier model are all better than the performance parameters corresponding to the K first classifier models, determine a performance difference value between the jth second candidate classifier model and the K first classifier models;

[0287] when the performance difference value is greater than a preset difference threshold, determine the jth second candidate classifier model as a second classifier model, where j is an integer between 1 and N.

[0288] In some embodiments, the fourth determination module is further configured to:

[0289] based on the performance parameters corresponding to the N second candidate classifier models and the performance parameters corresponding to the K first classifier models, determine Q second target classifier models, the performance parameters corresponding to the second target classifier models are all better than the performance parameters corresponding to the K first classifier models;

[0290] respectively determine Q performance difference values between the Q second target classifier models and the K first classifier models;

[0291] based on the Q performance difference values, determine a second classifier model from the Q second target classifier models.

[0292] It should be noted that the description of the device embodiments of the present application is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments, and therefore will not be described here. For technical details not disclosed in the present device embodiments, please refer to the description of the method embodiments of the present application for understanding.

[0293] The present application provides a storage medium storing executable instructions, wherein the storage medium stores executable instructions, when the executable instructions are executed by a processor, the processor will execute the method provided by the embodiments of the present application, for example, the method shown in Figure 3 、 Figure 4A 、 Figure 4B and Figure 5 .

[0294] In some embodiments, the storage medium can be a computer-readable storage medium, such as a ferroelectric memory (FRAM), a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a flash memory, a magnetic surface storage, an optical disc, or a compact disc read only memory (CD-ROM), and the like. It can also be various devices including one or any combination of the above memories.

[0295] In some embodiments, the executable instructions can take the form of a program, software, software modules, scripts, or code, written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and they can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0296] As an example, the executable instructions can or can not correspond to a file in a file system, can be stored in a part of a file that holds other programs or data, e.g., one or more scripts stored in a hypertext markup language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code. The executable instructions may, for example, be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0297] The above description is only some embodiments of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, and improvement within the spirit and scope of the present application shall be included in the protection scope of the present application.

Claims

1. A method for classifying mini-programs, characterized in that, include: Obtain the mini-program code of the mini-program to be categorized; The mini-program code is rendered and dynamically executed to obtain the first mini-program interface image; The first mini-program interface image is the interface image before the event is triggered; Each preset event is triggered sequentially, the target event that is successfully triggered is obtained, and the application interface information called when the target event is triggered and the control information corresponding to the target event are obtained. After all the preset events are triggered, the second mini-program interface image is obtained; The total number of calls to each application interface is determined based on the application interface information. The number of interactive controls is determined based on the control information; Determine the image difference information between the first mini-program interface image and the second mini-program interface image; Dynamic features are determined based on the number of target events, the total number of calls to each application interface, the number of interactive controls, and the image difference information. Based on the dynamic features and the trained classifier model, the classification information of the mini-program to be classified is determined; wherein, the classification information is used to characterize whether the mini-program to be classified is a mini-program with service or a mini-program without service; Store the classification information of the mini-programs to be classified.

2. The method according to claim 1, characterized in that, The trained classifier model includes at least K trained first classifier models. Correspondingly, determining the classification information of the mini-program to be classified based on the dynamic features and the trained classifier models includes: The dynamic features are input into K first classifier models respectively, resulting in K initial predicted values, where K is a positive integer; The target predicted value is determined based on the K initial predicted values; Based on the target predicted value and the preset classification threshold, the classification information of the mini-program to be classified is determined.

3. The method according to claim 2, characterized in that, When K is an integer greater than 1, the trained classifier model also includes a trained second classifier model. Correspondingly, determining the target predicted value based on the K initial predicted values ​​includes: The K initial predicted values ​​are input into the second classifier model to integrate the K initial predicted values ​​and obtain the target predicted value.

4. The method according to claim 2, characterized in that, The process of determining the classification information of the mini-program to be classified based on the target predicted value and the preset classification threshold includes: When the target predicted value is greater than the classification threshold, the classification information of the mini-program to be classified is determined to be a no-service mini-program; When the target predicted value is less than or equal to the classification threshold, the classification information of the mini-program to be classified is determined to be a mini-program with service.

5. The method according to claim 2, characterized in that, The method further includes: Obtain a first training dataset and M preset first candidate classifier models, wherein the first training dataset includes the dynamic features of the training mini-program and the label information of the training mini-program, and M is an integer greater than 1; Based on the first training dataset, determine the performance parameters corresponding to the M first candidate classifier models; Based on the performance parameters corresponding to the M first candidate classifier models, K first classifier models are determined. The first training dataset is used to train the K first classifier models to obtain K trained first classifier models.

6. The method according to claim 5, characterized in that, The method further includes: Using the first training dataset and the K first classifier models, a second training dataset is constructed, wherein the second training dataset includes: the prediction information of the K first classifier models for the training mini-program and the label information of the training mini-program; Obtain N preset second candidate classifier models, and determine the performance parameters corresponding to the N second candidate classifier models based on the second training dataset, where N is an integer greater than 1; Based on the performance parameters corresponding to the N second candidate classifier models and the performance parameters corresponding to the K first classifier models, a second classifier model is determined from the N second candidate classifier models. The second classifier model is trained using the second training dataset to obtain the trained second classifier model.

7. The method according to claim 6, characterized in that, The step of constructing a second training dataset using the first training dataset and the K first classifier models includes: The first training dataset is divided into P first training data subsets, where P is an integer greater than 1; The i-th first training data subset is determined as the i-th test dataset; i = 1, 2, ..., P; The K first classifier models are trained using other first training data subsets to obtain K trained first classifier models; wherein, the other first training data subsets are (P-1) first training data subsets other than the i-th first training data subset; The K trained first classifier models are used to perform prediction processing on the i-th test dataset to obtain the prediction information of the K first classifier models for the training mini-program in the i-th test dataset; The prediction information of the K first classifier models for the training mini-programs in the test datasets from the 1st to the Pth, and the label information of the training mini-programs are determined as the second training dataset.

8. The method according to claim 6, characterized in that, The step of determining the second classifier model from the N candidate second classifier models based on the performance parameters corresponding to the N candidate second classifier models and the performance parameters corresponding to the K first classifier models includes: The performance parameters corresponding to the N second candidate classifier models are compared with the performance parameters corresponding to the K first classifier models in turn. When it is determined that the performance parameters corresponding to the j-th second candidate classifier model are all better than the performance parameters corresponding to the K first classifier models, the performance difference value between the j-th second candidate classifier model and the K first classifier models is determined. When the performance difference value is greater than the preset difference threshold, the j-th second candidate classifier model is determined as the second classifier model, where j is an integer between 1 and N.

9. The method according to claim 6, characterized in that, The step of determining the second classifier model from the N candidate second classifier models based on the performance parameters corresponding to the N candidate second classifier models and the performance parameters corresponding to the K first classifier models includes: Based on the performance parameters corresponding to the N second candidate classifier models and the performance parameters corresponding to the K first classifier models, Q second target classifier models are determined, and the performance parameters corresponding to the second target classifier models are all better than the performance parameters corresponding to the K first classifier models. Determine the Q performance difference values ​​between the Q second target classifier models and the K first classifier models respectively; Based on the Q performance difference values, a second classifier model is determined from the Q second target classifier models.

10. A mini-program classification device, characterized in that, include: The first acquisition module is used to acquire the mini-program code of the mini-program to be categorized. The running module is used to render and dynamically run the mini-program code to obtain a first mini-program interface image; the first mini-program interface image is an interface image before the event is triggered. Each preset event is triggered sequentially, the target event that is successfully triggered is obtained, and the application interface information called when the target event is triggered and the control information corresponding to the target event are obtained. After all the preset events are triggered, the second mini-program interface image is obtained; The total number of calls to each application interface is determined based on the application interface information; the number of interactive controls is determined based on the control information; and the image difference information between the first mini-program interface image and the second mini-program interface image is determined. Dynamic features are determined based on the number of target events, the total number of calls to each application interface, the number of interactive controls, and the image difference information. The first determining module is used to determine the classification information of the mini-program to be classified based on the dynamic features and the trained classifier model; wherein, the classification information is used to characterize whether the mini-program to be classified is a mini-program with service or a mini-program without service; The storage module is used to store the classification information of the mini-programs to be classified.

11. A mini-program-based sorting device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, It stores executable instructions for causing a processor to execute, thereby implementing the method of any one of claims 1 to 9.

Images