Application user classification method and evaluation method, apparatus and device

Abstract

The application discloses an application program user classification method and an evaluation method, a device and equipment, and relates to the technical field of application program evaluation. The application program user classification method comprises the following steps: acquiring interface calling information of each function module in a target application program in a preset period, wherein the interface calling information is generated based on the operation of a target user on the target application program; converting the interface calling information into an operation statement sequence to obtain a real-time corpus; and obtaining a user type of the target user according to the real-time corpus and a trained user classification model. The application solves the problem that the existing application program evaluation method has poor evaluation accuracy, and achieves the effect of improving the application program evaluation accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of application program evaluation, in particular to an application program user classification method and evaluation method, device and equipment. BACKGROUND

[0002] With the gradual increase of smart phones and application programs, more and more businesses are transferred to application programs for processing, such as financial investment, payment and recharge, ticket purchase and the like. An application program often has multiple business function modules. When a developer repairs or upgrades the application program, it is necessary to know whether the function module is practical for users and whether it really solves the user's demand, so it is necessary to evaluate the application program. However, the existing application program evaluation method has the problem of poor evaluation accuracy, and cannot obtain more practical evaluation results for the developer, which may lead to the fact that the application program developed by the developer cannot meet the real needs of the users. SUMMARY

[0003] The main purpose of the present application is to provide an application program user classification method and evaluation method, device and equipment, which aims to solve the technical problem of poor evaluation accuracy of the application program evaluation method in the prior art.

[0004] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0005] In a first aspect, the present application provides an application program user classification method, which comprises the following steps:

[0006] Obtain interface call information of each function module in a target application program in a preset period, wherein the interface call information is generated based on the operation of a target user on the target application program;

[0007] Convert the interface call information into an operation statement sequence to obtain a real-time corpus;

[0008] Obtain the user type of the target user according to the real-time corpus and a trained user classification model, wherein the user classification model is constructed based on a natural language processing algorithm.

[0009] Optionally, in the application program user classification method, the step of converting the interface call information into an operation statement sequence to obtain a real-time corpus specifically comprises:

[0010] According to a preset word library, the interface call information is converted into an operation statement sequence to obtain a real-time corpus, wherein the preset word library includes word groups defined according to the interface call sequence.

[0011] Optionally, in the application program user classification method, before the step of converting the interface call information into operation statement sequences according to the preset word library to obtain a real-time corpus, the method further comprises:

[0012] obtaining an interface call relationship graph of the target application program;

[0013] obtaining preset interface call sequences corresponding to each functional module in the target application program according to the interface call relationship graph;

[0014] defining a plurality of functional word groups and pause word groups according to the preset interface call sequences to obtain a functional word library and a pause word library;

[0015] obtaining a preset word library according to the functional word library and the pause word library.

[0016] Optionally, in the application program user classification method, the step of converting the interface call information into operation statement sequences according to the preset word library to obtain a real-time corpus specifically comprises:

[0017] converting the interface call information into a plurality of interface call sequences according to a preset time interval;

[0018] converting each of the interface call sequences into operation statement sequences composed of a plurality of word groups according to a preset word library to obtain a plurality of operation statement sequences;

[0019] constructing a real-time corpus according to the plurality of operation statement sequences.

[0020] Optionally, in the application program user classification method, before the step of obtaining a user type of the target user according to the real-time corpus and a trained user classification model, the method further comprises:

[0021] automatically simulating the use of each functional module of the target application program by a user by using a script to obtain a corresponding preset interface call sequence;

[0022] converting the preset interface call sequence into a preset operation statement sequence according to a preset word library to obtain a training corpus;

[0023] annotating the preset operation statement sequence in the training corpus with a user type to obtain a plurality of annotation samples to construct a training data set;

[0024] establishing an initial model according to a Bayesian text classification algorithm;

[0025] training the initial model according to the training data set to obtain a trained user classification model.

[0026] Optionally, in the application user classification method, the step of obtaining the user type of the target user according to the real-time corpus and the trained user classification model specifically comprises:

[0027] inputting the operation statement sequence in the real-time corpus into the trained user classification model;

[0028] classifying the operation statement sequence based on a Bayesian text classification algorithm to obtain a classification result;

[0029] obtaining the user type of the target user according to the classification result.

[0030] In a second aspect, the application provides an application evaluation method, which comprises the following steps:

[0031] obtaining operation information of a plurality of users using a target application in a preset period;

[0032] obtaining user types of the plurality of users by using the application user classification method according to the operation information;

[0033] evaluating the target application according to a proportion of each user type in the plurality of user types and a preset user type proportion threshold to obtain an evaluation result.

[0034] In a third aspect, the application provides an application user classification device, which comprises:

[0035] an information collection module configured to obtain interface call information of each function module in a target application in a preset period, wherein the interface call information is generated based on operation of a target user on the target application;

[0036] a real-time corpus acquisition module configured to convert the interface call information into an operation statement sequence to obtain a real-time corpus;

[0037] a user type acquisition module configured to obtain a user type of the target user according to the real-time corpus and a trained user classification model, wherein the user classification model is constructed based on a natural language processing algorithm.

[0038] In a fourth aspect, the application provides a computer device, which comprises a processor and a memory, and the memory stores a computer program, and the computer program is executed by the processor to implement the method described above.

[0039] In a fifth aspect, the application provides a storage medium, which stores a computer program, and the computer program can be executed by one or more processors to implement the method described above.

[0040] The one or more technical solutions provided by the application can have the following advantages or at least achieve the following technical effects:

[0041] The application program user classification method and evaluation method, device and equipment provided by the application obtain interface call information of each function module in a target application program in a preset period, establish a corresponding relationship between actual users and data to be analyzed by the user classification method according to operation data of a target user on the target application program, convert the interface call information into an operation statement sequence to obtain a real-time corpus, define the interface call information in combination with natural language to preliminarily perform character-to-sentence classification processing and reduce subsequent calculation amount, and then obtain a user type of the target user according to the real-time corpus and a trained user classification model, so that the user classification result can be obtained by directly inputting the real-time corpus by using the user classification model constructed and trained based on a natural language processing algorithm, which is convenient and fast and can further improve the accuracy of user classification as the model is continuously optimized. In addition, the application program evaluation is performed by using the application program user classification method, so that when the obtained user type is closer to the actual user type, the support effectiveness evaluation of the application program will also be more accurate, thereby achieving the effect of improving the accuracy of application program evaluation. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only constitute some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0043] Figure 1 The flowchart of the first embodiment of the application program user classification method of the application;

[0044] Figure 2 The hardware structure schematic diagram of the computer device related to the application;

[0045] Figure 3 The flowchart of the second embodiment of the application program user classification method of the application;

[0046] Figure 4 The interface call relationship graph of the second embodiment of the application program user classification method of the application;

[0047] Figure 5 The demonstration schematic diagram of the second embodiment of the application program user classification method of the application;

[0048] Figure 6 The flowchart of the first embodiment of the application program evaluation method of the application;

[0049] Figure 7 The functional module schematic diagram of the application program user classification device first embodiment of the present application;

[0050] Figure 8 The functional module schematic diagram of the application program evaluation device first embodiment of the present application.

[0051] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0052] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0053] It should be noted that in the present application, the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive containing, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or system. Without more limitations, the elements defined by the statement "comprising" do not exclude the presence of other identical elements in the process, method, article or system including the element. In the present application, if there is a description of "first", "second", etc., the description of "first", "second", etc. is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include at least one of the features. In addition, in the present application, the suffixes such as "module", "component" or "unit" used to represent elements are only for the convenience of description of the present application, and have no specific meaning. Therefore, "module", "component" or "unit" can be mixedly used.

[0054] For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the fact that the technical solutions can be realized by those of ordinary skill in the art. When the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, and is not within the protection scope required by the present application.

[0055] With the gradual increase of smart phones and applications, more and more businesses are transferred to the application for processing, such as financial investment, payment, ticket purchase and the like. An application often has multiple business function modules. When the developer repairs or upgrades the application, it is necessary to know whether the function module is practical for the user and whether it really solves the user's demand, so it is necessary to evaluate the application.

[0056] The existing application evaluation method generally sets the user role classification in advance, matches the corresponding function module classification based on the user role classification, generates user operation logs through function burying, collects operation logs in the background, analyzes the user role based on the function modules used by the user, portraits the user operation based on the use frequency and use time, obtains the individual portrait of the user use behavior, and then based on the set user role classification, all user use behavior individual portraits are summarized to form a group portrait, and finally the individual portrait and the group portrait are used to evaluate the effectiveness of each function module of the application. The existing method directly obtains the corresponding operation log for evaluation according to the user type, which can only start from the scattered operation behavior of the user, analyze the calling behavior of various function modules from the operation log file, and obtain the operation log and the actual user without a clear corresponding relationship. Due to the fine granularity of calling, too much analysis noise is easily introduced, which leads to the difference between the analyzed user role and the actual user type, so that the application evaluation does not reflect the actual use state of the user, and the evaluation accuracy is poor, so that the developer cannot obtain more actual evaluation results.

[0057] In view of the technical problems in the prior art that the application evaluation method according to the existing user role and the existing function module design obtains the use condition to perform user role portrait, and too much analysis noise is easily introduced, which leads to a large difference between the obtained user classification and the actual user type, and the application evaluation has the technical problem of poor evaluation accuracy, the application provides an application user classification method, and the overall idea is as follows:

[0058] Obtain interface calling information of each function module in a target application in a preset period, wherein the interface calling information is generated based on operation of a target user on the target application; convert the interface calling information into an operation statement sequence to obtain a real-time corpus; obtain a user type of the target user according to the real-time corpus and a trained user classification model, wherein the user classification model is constructed based on a natural language processing algorithm.

[0059] By the technical solution, the interface calling information of each function module in the target application program in a preset period is acquired, the corresponding relationship between the actual user and the data analyzed by the user classification method is established according to the operation data of the target user on the target application program, the interface calling information is converted into an operation statement sequence to obtain a real-time corpus, the interface calling information is defined in combination with a natural language to preliminarily perform character-to-sentence classification processing and reduce subsequent calculation amount, the user type of the target user is obtained according to the real-time corpus and the trained user classification model, the user classification model is constructed and trained based on a natural language processing algorithm, and the user classification result can be obtained by directly inputting the real-time corpus, which is convenient and fast, and the accuracy of user classification can be further improved as the model is continuously optimized. The application program is evaluated by using the application program user classification method, when the obtained user type is closer to the actual user type, the support effectiveness evaluation of the application program is also more accurate, and the effect of improving the accuracy of application program evaluation is realized.

[0060] Embodiment one

[0061] Referring to Figure 1 a flowchart, the first embodiment of the application program user classification method is proposed, and the application program user classification method is applied to a computer device.

[0062] The computer device refers to a terminal device or a network connection device capable of realizing network connection, and the computer device can be a terminal device such as a mobile phone, a computer, a tablet computer, and an embedded industrial computer, or a network device such as a server.

[0063] As shown in Figure 2 , it is a schematic diagram of the hardware structure of the computer device. The device can include a processor 1001, such as a CPU (Central Processing Unit, central processor), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005.

[0064] Those skilled in the art can understand that Figure 2 the hardware structure shown in the embodiment does not constitute a limitation on the computer device of the application, and can include more or fewer components than the diagram, or combine certain components, or different component arrangements.

[0065] Specifically, the communication bus 1002 is used to realize the connection and communication between the components;

[0066] The user interface 1003 is used to connect the client and communicate data with the client. The user interface 1003 can include an output unit such as a display screen and an input unit such as a keyboard. Optionally, the user interface 1003 can also include other input / output interfaces, such as a standard wired interface and a wireless interface.

[0067] The network interface 1004 is used to connect the background server and communicate data with the background server, and the network interface 1004 can include an input / output interface, such as a standard wired interface, a wireless interface, such as a Wi-Fi interface;

[0068] The memory 1005 is used to store various types of data, which can include, for example, instructions of any application program or method in the device, and application-related data. The memory 1005 can be a high-speed RAM memory, or a stable memory, such as a disk memory. Optionally, the memory 1005 can also be a storage device independent of the processor 1001;

[0069] Specifically, continuing to refer to Figure 2 The memory 1005 can include an operating system, a network communication module, a user interface module, and a computer program. The network communication module is mainly used to connect the server and communicate data with the server;

[0070] The processor 1001 is used to call the computer program stored in the memory 1005 and perform the following operations:

[0071] Obtain interface call information of each function module in the target application program within a preset period, wherein the interface call information is generated based on the operation of the target user on the target application program;

[0072] Convert the interface call information into an operation statement sequence to obtain a real-time corpus;

[0073] Obtain the user type of the target user according to the real-time corpus and a trained user classification model, wherein the user classification model is constructed based on a natural language processing algorithm.

[0074] Based on the above computer device, the application program user classification method of the present embodiment will be described in detail in combination with the flowchart shown in Figure 1 The method can include the following steps:

[0075] Step S20: Obtain interface call information of each function module in the target application program within a preset period, wherein the interface call information is generated based on the operation of the target user on the target application program.

[0076] Specifically, a target application program, APP, is installed on a plurality of user computer devices, and when a user performs an operation of logging in to the APP, querying data, processing business, and the like, interface call information of all interfaces called in the APP is recorded and stored to the cloud for use. When a developer needs to determine the user type of each user using the target application program, the interface call information in a certain period of time is obtained from the cloud to obtain corresponding interface call information generated by the plurality of target users in operating the target application program.

[0077] Randomly obtaining operation information of unknown types of users on each functional module of the target application program, rather than obtaining operation information of known types of users on specific functional modules, makes the data more comprehensive. Obtaining data in a preset period of time can determine the user types of a plurality of users, rather than separately collecting specific operations of each user multiple times, thereby preventing excessive analysis noise and affecting the accuracy of classification analysis.

[0078] Step S40: converting the interface call information into an operation statement sequence to obtain a real-time corpus.

[0079] Specifically, each interface call of the interface call information of a certain target user is positioned as a character, thereby obtaining a plurality of characters, a plurality of sub-services that can be implemented by a functional module, and a sub-service that can involve one or more interface calls. Therefore, the plurality of interface calls involved in a sub-service are converted into an interface call sequence composed of a plurality of characters, i.e., a phrase, thereby obtaining a complete use process of a functional module, converting the interface call sequences of a plurality of sub-services into an operation statement sequence, and further converting the use of the APP by the user in this period of time into a plurality of operation statement sequences. In this way, the operation statement sequences of other users are continuously obtained, thereby constructing a real-time corpus.

[0080] The interface calls of the application program are defined using natural language, so that the user classification can be continuously performed using natural language classification in the subsequent process, the classification of actual users is based on actual business operation behaviors, and the correspondence between actual users and user types analyzed according to operation behaviors is not affected by analysis and processing.

[0081] Step S60: obtaining the user type of the target user according to the real-time corpus and a trained user classification model, wherein the user classification model is constructed based on a natural language processing algorithm.

[0082] Specifically, the user classification model can be a binary classification or multi-classification deep learning model, which can include a corresponding relationship between operation statement sequences and user types. After a user classification model is constructed and trained based on a natural language processing algorithm, the operation statement sequences of the real-time corpus obtained in step S40 are input into the user classification model, and the corresponding user types are output, so that the user types of the target users can be known, and the effect of associating the user types obtained after analysis with the actual target users is achieved.

[0083] The user classification model is analyzed by using the trained deep learning model, which is convenient and fast, and the user classification model can be continuously optimized as the amount of data increases, further improving the accuracy of application user classification.

[0084] The application user classification method provided in the embodiment obtains the interface call information of each function module in the target application program in a preset period, establishes a corresponding relationship between the actual users and the data to be analyzed by the user classification method according to the operation data of the target users on the target application program, converts the interface call information into operation statement sequences to obtain a real-time corpus, defines the interface call information in combination with natural language to preliminarily classify characters into sentences and reduce subsequent calculation amount, and then obtains the user types of the target users according to the real-time corpus and the trained user classification model. The user classification model constructed and trained based on the natural language processing algorithm is directly input into the real-time corpus to obtain the user classification result, which is convenient and fast, and the accuracy of user classification can be further improved as the model is continuously optimized.

[0085] Embodiment Two

[0086] Based on the same inventive concept, referring to Figures 3 to 5 , a second embodiment of the application application user classification method is provided, which is applied to a computer device.

[0087] The application user classification method of the embodiment will be described in detail below with reference to the flowchart shown in Figure 3 . The method can include the following steps:

[0088] Step S20: obtaining interface call information of each function module in a target application program in a preset period, wherein the interface call information is generated based on the operation of a target user on the target application program.

[0089] Specifically, the target application program, i.e., APP, is installed on computer devices of multiple users, and when the users perform operations such as login account, data query, business processing, etc. on the APP, the interface call information of all interfaces called in the APP is recorded and stored to the cloud for use.

[0090] The embodiment takes a financial investment APP as an example for illustration. The APP has a customer fund claim function. Correspondingly, there is a function module corresponding to the function. The interface access situation of all users to the customer fund claim function module of the APP can be recorded and stored through an interface log system. When a developer needs to make user type judgment on each user using the APP, the interface call information of a certain specified function module in a certain time period is first obtained from the cloud, and the corresponding interface call information generated by the operation of the APP by a plurality of target users is obtained.

[0091] The user type is automatically classified by collecting the interface call information of the target user using the specified function module, instead of obtaining the corresponding operation log for the user type. Compared with the prior art, the correspondence between the actual target user and the user type obtained by subsequent automatic classification can be established, and more accurate user type classification can be obtained.

[0092] Step S30: constructing a preset word library.

[0093] Specifically, the step S30 can include:

[0094] Step S31: obtaining an interface call relationship graph of a target application program;

[0095] In the specific implementation process, a plurality of interfaces need to be continuously called for a specific function to perform an operation. For example, a function can call m interfaces, and the calling behaviors of the m interfaces are represented by m characters, such as A1, A2, …, and Am, where m≥1.

[0096] In the embodiment, as shown in Figure 4 The interface call relationship graph of the customer fund claim function module of the financial investment APP of the embodiment is shown. There are 5 interfaces, and each interface call corresponds to an operation. The interface call is represented by characters A1-A5. A1 represents a login operation, A2 represents a fund list obtaining operation, A3 represents a fund detail opening operation, A4 represents a fund claim operation, and A5 represents a conditional search operation.

[0097] Step S32: obtaining a preset interface call sequence corresponding to each function module in the target application program according to the interface call relationship graph.

[0098] Specifically, a sequence is an object or event arranged in a column. When a user performs a sub-business on the APP, there will be one or more operations, and accordingly one or more interface calls will be generated. Different interfaces can be arranged and combined to obtain different sub-businesses. After the interface calls are represented by characters, the interface call sequence is set according to the specific interface call relationship of the preset sub-business.

[0099] In this embodiment, according to the interface call relationship graph of Figure 4 obtain the preset interface call sequence corresponding to each sub-business in the customer fund claim function module, such as A2A5, A5A2 or A2A3 for checking the to-be-processed claimed fund; A2A3A4 for executing fund claim, etc. The specific setting can be made according to actual conditions.

[0100] Step S33: According to the preset interface call sequence, define a plurality of function word groups and pause word groups to obtain a function word library and a pause word library.

[0101] Specifically, one sub-business corresponds to one preset interface call sequence, and the preset interface call sequence is represented by a word group W. One function module has a plurality of sub-businesses, and correspondingly has a plurality of word groups. For example, one function module of an APP has n sub-businesses, which are represented by n word groups, respectively W1, W2, …, Wn. The function word group W and the pause word group T can also be set respectively according to the different business and some operations irrelevant to the business. The sub-businesses of the plurality of function modules in the application program are all set with corresponding function word groups and pause word groups, so as to obtain a plurality of function word groups and pause word groups. Then, a function word library is constructed according to the plurality of function word groups, and a pause word library is constructed according to the plurality of pause word groups.

[0102] In this embodiment, according to the above-mentioned preset interface call sequence, the function word group is defined, for example, the function word group W1=A2A5, A5A2 or A2A3, and the function word group W2=A2A3A4. The interface irrelevant to the sub-business can also be defined as a pause word group, for example, the interface A1 for verifying login information is defined as a pause word group T1=A1. The function word library W=[W1, W2] is constructed according to the above-mentioned function word groups W1 and W2, and the pause word library T=[T1] is constructed according to the above-mentioned pause word group T1.

[0103] Step S34: Obtain a preset word library according to the function word library and the pause word library.

[0104] In this embodiment, the preset word library is constructed according to the above-mentioned function word library W and pause word library T, so as to be called for actual use.

[0105] Step S40: Convert the interface call information into an operation statement sequence to obtain a real-time corpus.

[0106] In one embodiment, the step S40 can include:

[0107] Step S40a: Convert the interface call information into an operation statement sequence to obtain a real-time corpus according to a preset word library, wherein the preset word library includes word groups defined according to the interface call sequence.

[0108] Specifically, using predefined word groups from a pre-defined lexicon, the interface call information is converted into a sequence of operation statements, and then these statement sequences are used to build a real-time corpus for subsequent classification.

[0109] In another embodiment, step S40a may include:

[0110] Step S41: Convert the interface call information into multiple interface call sequences according to a preset time interval.

[0111] In this embodiment, in a practical application scenario, if a user calls s interfaces A1, A2, ..., As in a single use, the obtained interface call information can be converted into an interface call sequence similar to A1A2A4...AsA2A3As. A user may use the interface multiple times within a certain period, resulting in multiple interface call sequences.

[0112] Step S42: Based on a preset dictionary, each interface call sequence is converted into an operation statement sequence consisting of multiple word groups to obtain multiple operation statement sequences.

[0113] like Figure 5 The illustrated diagram shows how this embodiment converts an interface call sequence into a sequence of operation statements composed of multiple phrases based on a preset dictionary. For example, if the preset dictionary defines W1 = A1A2, W2 = As, ..., Wn = A2A3As, T1 = A4, etc., then the interface call sequence can be converted into an operation statement sequence L1 = W1T1...W2Wn. Multiple interface call sequences are then followed by multiple operation statement sequences L1, L2, ..., etc. This completes the conversion from basic interface log call information to a single statement. Undefined interface calls can be directly ignored in the above analysis process.

[0114] Step S43: Construct a real-time corpus based on the multiple operation statement sequences.

[0115] A real-time corpus L is constructed based on the above sequence of multiple operation statements L1, L2, ...

[0116] In this embodiment, it is assumed that there are two usage scenarios, for example:

[0117] 1) Directly access the latest fund list and claim related operations. The obtained interface call sequence is: A1A2A3A4. Through the above steps, it can be converted into the operation statement sequence L1=T1W2.

[0118] 2) the relevant operation of obtaining the fund list through the query and claiming, and the obtained interface call sequence is: A1A2A5A2A3A4, which can be converted into the operation statement sequence L2=T1W1W2 through the above steps.

[0119] According to the above two cases, for the relevant operation of a certain user within a preset time, a real-time corpus L can be formed, such as L=[T1L1L2L1L2, T1L1L1].

[0120] After the step of converting the interface call information into the operation statement sequence and obtaining the real-time corpus, the following steps S50-S70 can be sequentially executed to realize the training, use and optimization of the user classification model; or the step S50 can be an independent step branch to obtain the trained user classification model.

[0121] Step S50: establishing an initial model and training to obtain a trained user classification model.

[0122] Specifically, the step S50 can include:

[0123] Step S51: automatically simulating the use of each functional module of the target application program by a user by using a script to obtain a corresponding preset interface call sequence;

[0124] Step S52: converting the preset interface call sequence into a preset operation statement sequence according to a preset vocabulary to obtain a training corpus.

[0125] Specifically, the preset vocabulary constructed according to the step S30 can be used to continuously use various functional scenarios, and all generated statements can be added to a training corpus. In order to take into account the training effect, various scenarios of simulating the use of each functional module of the target application program by a user can be used to quickly generate a training corpus containing a sufficient number of statements.

[0126] At this time, some typical use scenarios of different types of users can be assumed, such as assuming that there are two types of users with different "personalities":

[0127] "Active claiming" type: representing a user who actively queries and claims the return;

[0128] "Negative claiming" type: representing a user who does not actively claim the return.

[0129] According to the above setting, the "active claiming" type is simulated, the query is opened in the morning and in the afternoon respectively every day, and the claiming operation is performed as much as possible. After obtaining the simulated preset interface call sequence, the preset operation statement sequence T1L1L2L1L2, T1L1L1, T1L1L1L1L2 is converted according to the preset vocabulary. These preset operation statement sequences are added to the training corpus.

[0130] Step S53: User type labeling is performed on the preset operation sentence sequence in the training corpus to obtain a plurality of labeled samples to construct a training data set.

[0131] Specifically, user type labeling is performed on the preset operation sentence sequence in the simulated training corpus, for example, T1L1L2L1L2, T1L1L1, T1L1L1L1L2 are labeled as "positive recognition" type users to obtain labeled samples labeled with user types. Multiple simulations and labeling are performed in the above manner to obtain a plurality of labeled samples to construct a training data set.

[0132] Step S54: An initial model is established according to a Bayesian text classification algorithm.

[0133] Specifically, an initial model of binary or multi-classification is constructed based on a natural language processing algorithm, for example, a deep learning model can be established according to a Bayesian text classification algorithm, or a deep learning model can be established according to a convolutional neural network, and the like.

[0134] Step S55: The initial model is trained according to the training data set to obtain a trained user classification model.

[0135] Specifically, the initial model of step S54 is trained according to the training data set obtained in step S53 to obtain a trained user classification model for use.

[0136] In this embodiment, the training corpus and classification labeling obtained according to the above example obtain a user classification model that can classify users into "positive recognition type" and "negative recognition type" according to operation sequences.

[0137] Step S60: The user type of the target user is obtained according to the real-time corpus and the trained user classification model, wherein the user classification model is constructed based on a natural language processing algorithm.

[0138] Specifically, step S60 can include:

[0139] Step S61: The operation sentence sequence in the real-time corpus is input into the trained user classification model.

[0140] Step S62: The user classification model classifies the operation sentence sequence based on a Bayesian text classification algorithm to obtain a classification result.

[0141] Step S63: The user type of the target user is obtained according to the classification result.

[0142] Specifically, according to the real-time corpus L formed according to step S40, a series of operations of user A in a week are obtained, and the real-time corpus L = [T1L1L2L1L2, T1L1L1L1L2] is obtained. According to the user classification model obtained by training according to step S50, the sentences in the real-time corpus L are matched with the user type, and it is obtained that the user A is a "positive claim" type user. By analogy, all actual users who operate in a preset period can be classified into the two types, and the user types of the final target users are obtained.

[0143] Step S70: optimizing the user classification model according to the user type of the target user.

[0144] Specifically, the step S70 can include:

[0145] Step S71: obtaining a training sample according to the operation sentence sequence in the real-time corpus and the user type of the target user.

[0146] Step S72: reconstructing the training data set according to the training sample and the labeled sample, so as to optimize the user classification model according to the training data set.

[0147] Specifically, since the user classification model is a deep learning model, the model can be optimized according to the obtained classification result to update the user classification model, so that better classification effect can be achieved in subsequent classification, and the classification accuracy can be further improved.

[0148] The application program user classification method provided in the embodiment defines the interface call sequence of all functions of the App as a word group to construct a word library, then generates sentences from the interface call sequence in use through the word library to construct a training corpus according to the simulation of the user using the App, manually labels the corresponding user classification, uses a natural language classification algorithm to train a model to obtain a user classification model, and finally classifies the sentences generated by the actual use scene of the user through the model to obtain the classification of the actual user. The natural language classification algorithm is used, the behavior of the user using the App is regarded as a special language expression, and the natural language classification algorithm is used to classify the users using the App according to the operation data of various existing users on the App, and the users are classified based on the actual business operation behavior. The classification result can be compared with the existing design in the future to provide an effective method for evaluating the design of the App and the actual effect.

[0149] Embodiment three

[0150] Based on the same inventive concept, refer to Figure 6The application provides an application program evaluation method.

[0151] The application program evaluation method provided by the application will be described in detail below with reference to the flowchart shown in the accompanying drawings. Figure 6 The application program evaluation method provided by the application can include the following steps.

[0152] Step S1: Obtain operation information of a plurality of users using a target application program in a preset period.

[0153] Specifically, the operation information includes interface call information of a specified function module, and the preset period is a specified period that a developer wants to collect, such as one day, one week, or one month, etc. The plurality of users use the target application program on a computer device, and the corresponding interface call information is collected through an interface log system and stored. In this embodiment, it is assumed that there are 10 users A to J, and the operation information of each user is obtained.

[0154] Step S2: According to the operation information, the user types of the plurality of users are obtained by using the application program user classification method described above.

[0155] Specifically, according to the operation information of step S1, the user types of the plurality of users are obtained by using all or part of the steps of the first embodiment or the second embodiment of the application program user classification method. In this embodiment, users A, C, D, E, F, H, I, and J are obtained as “active recognition” types by using the method described above, and users B and G are obtained as “passive recognition” types.

[0156] Step S3: According to the proportion of each type of user type in the plurality of user types and a preset user type proportion threshold, the target application program is evaluated to obtain an evaluation result.

[0157] Specifically, the proportion of each type of user type in the plurality of user types is obtained. In this embodiment, according to the 10 users described above, the proportion of the “active recognition” type is 80%, and the proportion of the “passive recognition” type is 20%. If the expected proportion threshold of the user type corresponding to the specified function module of the target application program is 70% or more, then 80% in this embodiment exceeds the preset user type proportion threshold, which indicates that the design of the function module meets the expectation. Otherwise, if it is lower than the expectation, it indicates that the design and use of the module are not ideal, and the developer can consider whether to delete the function module or whether to improve the function module when updating the application program in the future.

[0158] It should be noted that more implementation details of the method step S2 can be found in the description of the first embodiment or the second embodiment. For the sake of brevity of the description, they will not be repeated here.

[0159] The application program evaluation method provided in the embodiment evaluates whether the actual use of the user conforms to the expectation of the application program design based on the user classification obtained by the application program user classification method, and evaluates the design effect of the App by comparing the difference between the actual user category and the expected user category. When the obtained user type is closer to the actual user type, the support effectiveness evaluation of the application program will also be more accurate, thereby achieving the effect of improving the accuracy of the application program evaluation.

[0160] Embodiment four

[0161] Based on the same inventive concept, referring to Figure 7 , the first embodiment of the application program user classification device is provided, which can be a virtual device and applied to a computer device.

[0162] The application program user classification device provided in the embodiment will be described in detail below in combination with the function module schematic diagram shown in Figure 7 , which can include:

[0163] An information collection module is configured to obtain interface call information of each function module in a target application program in a preset period, wherein the interface call information is generated based on the operation of a target user on the target application program.

[0164] A real-time corpus acquisition module is configured to convert the interface call information into an operation statement sequence and obtain a real-time corpus.

[0165] A user type acquisition module is configured to obtain a user type of the target user according to the real-time corpus and a trained user classification model, wherein the user classification model is constructed based on a natural language processing algorithm.

[0166] Further, the real-time corpus acquisition module can also be configured to:

[0167] convert the interface call information into an operation statement sequence according to a preset vocabulary library to obtain a real-time corpus, wherein the preset vocabulary library includes word groups defined according to the interface call sequence.

[0168] Further, the device can also include:

[0169] A preset vocabulary library construction module is configured to construct a preset vocabulary library.

[0170] Further, the preset vocabulary library construction module can include:

[0171] An interface call relationship acquisition unit is configured to obtain an interface call relationship graph of the target application program.

[0172] The preset interface call sequence acquisition unit is configured to acquire preset interface call sequences corresponding to the functional modules in the target application according to the interface call graph.

[0173] The phrase definition unit is configured to define a plurality of functional phrases and pause phrases according to the preset interface call sequences, so as to obtain a functional phrase library and a pause phrase library.

[0174] The preset phrase library acquisition unit is configured to acquire a preset phrase library according to the functional phrase library and the pause phrase library.

[0175] Further, the real-time corpus acquisition module can include:

[0176] The first conversion unit is configured to convert the interface call information into a plurality of interface call sequences according to a preset time interval.

[0177] The second conversion unit is configured to convert each of the interface call sequences into a plurality of operation statement sequences composed of phrases according to a preset phrase library, so as to obtain a plurality of operation statement sequences.

[0178] The real-time corpus construction unit is configured to construct a real-time corpus according to the plurality of operation statement sequences.

[0179] Further, the device can further include:

[0180] The model training module is configured to establish an initial model and perform training to obtain a trained user classification model.

[0181] Further, the model training module can include:

[0182] The simulation unit is configured to automatically simulate the use of the functional modules of the target application by a user by using a script, so as to obtain corresponding preset interface call sequences.

[0183] The training corpus construction unit is configured to convert the preset interface call sequences into preset operation statement sequences according to a preset phrase library, so as to obtain a training corpus.

[0184] The training data set acquisition unit is configured to perform user type labeling on the preset operation statement sequences in the training corpus, so as to obtain a plurality of labeled samples and construct a training data set.

[0185] The initial model establishment unit is configured to establish an initial model according to a Bayesian text classification algorithm.

[0186] The user classification model acquisition unit is configured to train the initial model according to the training data set, so as to obtain a trained user classification model.

[0187] Further, the user type obtaining module can include:

[0188] an input unit configured to input the operation statement sequence in the real-time corpus into the trained user classification model;

[0189] a classification and recognition unit configured to classify the operation statement sequence based on a Bayesian text classification algorithm to obtain a classification result;

[0190] an output unit configured to obtain the user type of the target user according to the classification result.

[0191] It should be noted that the functions and technical effects of the modules in the application program user classification device provided in the embodiments can be referred to the descriptions of the application program user classification method in the embodiments, and will not be repeated here for the sake of brevity.

[0192] Embodiment Five

[0193] Based on the same inventive concept, referring to Figure 8 , the first embodiment of the application program evaluation device is provided, which can be a virtual device and applied to a computer device.

[0194] The application program evaluation device provided in the embodiment will be described in detail below with reference to the function module schematic diagram shown in Figure 8 The device can include:

[0195] an operation information obtaining module configured to obtain operation information of a plurality of users using a target application program in a preset period;

[0196] a user classification module configured to obtain user types of the plurality of users according to the operation information by using an application program user classification method;

[0197] an application program evaluation module configured to evaluate the target application program according to a proportion of each user type in the plurality of user types and a preset user type proportion threshold value to obtain an evaluation result.

[0198] It should be noted that the functions and technical effects of the modules in the application program evaluation device provided in the embodiments can be referred to the descriptions of the first embodiment of the application program evaluation method, and will not be repeated here for the sake of brevity.

[0199] Embodiment Six

[0200] Based on the same inventive concept, referring to Figure 2Fig. 1 is a schematic diagram of a hardware structure of a computer device related to various embodiments of the present application. The embodiment provides a computer device, which can include a processor and a memory, and the memory stores a computer program, and the computer program is executed by the processor to implement all or part of the steps of various embodiments of the method of the present application.

[0201] Specifically, the computer device refers to a terminal device or a network connection device that can realize network connection, which can be a terminal device such as a mobile phone, a computer, a tablet computer, a portable computer, etc., or a network device such as a server, a cloud platform, etc.

[0202] It can be understood that the device can also include a communication bus, a user interface, and a network interface.

[0203] The communication bus is used to realize the connection and communication between these components.

[0204] The user interface is used to connect the client and communicate data with the client, and the user interface can include an output unit such as a display screen, an input unit such as a keyboard, and optionally, the user interface can also include other input / output interfaces such as standard wired interfaces and wireless interfaces.

[0205] The network interface is used to connect the background server and communicate data with the background server, and the network interface can include input / output interfaces such as standard wired interfaces and wireless interfaces such as Wi-Fi interfaces.

[0206] The memory is used to store various types of data, which can include, for example, instructions of any application program or method in the computer device, and application-related data. The memory can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk, and optionally, the memory can also be a storage device independent of the processor.

[0207] The processor is configured to invoke a computer program stored in the memory and execute all or part of the steps of the embodiments of the method described above. The processor can be an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic devices, configured to execute all or part of the steps of the embodiments of the method described above.

[0208] Embodiment Seven

[0209] Based on the same inventive concept, the embodiments provide a computer readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable read only memory (EEPROM), a programmable read only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application store, and the like, in which a computer program is stored. The computer program can be executed by one or more processors, and when the computer program is executed by the processor, all or part of the steps of the embodiments of the method of the present application can be implemented.

[0210] It should be noted that the above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0211] The above description is only an optional embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made under the inventive concept of the present application, or directly or indirectly applied to other related technical fields, is included in the patent protection scope of the present application.

Claims

1. An application user classification method, characterized by, The method comprises the following steps: Obtaining interface call information of each function module in a target application program in a preset period, wherein the interface call information is generated based on the operation of a target user on the target application program; Converting the interface call information into an operation statement sequence according to a preset vocabulary library to obtain a real-time corpus, wherein the preset vocabulary library comprises word groups defined according to the interface call sequence; Obtaining the user type of the target user according to the real-time corpus and a trained user classification model, wherein the user classification model is constructed based on a natural language processing algorithm; The step of converting the interface call information into an operation statement sequence according to a preset vocabulary library to obtain a real-time corpus specifically comprises: Converting the interface call information into a plurality of interface call sequences according to a preset time interval; Converting each interface call sequence into an operation statement sequence composed of a plurality of word groups according to a preset vocabulary library to obtain a plurality of operation statement sequences; Constructing a real-time corpus according to a plurality of operation statement sequences.

2. The application user classification method of claim 1, wherein, Before the step of converting the interface call information into an operation statement sequence according to a preset vocabulary library to obtain a real-time corpus, the method further comprises: Obtaining an interface call relationship graph of the target application program; Obtaining a preset interface call sequence corresponding to each function module in the target application program according to the interface call relationship graph; Defining a plurality of function word groups and pause word groups according to the preset interface call sequence to obtain a function word library and a pause word library; Obtaining a preset vocabulary library according to the function word library and the pause word library.

3. The application user classification method of claim 1 or 2, wherein Before the step of obtaining the user type of the target user according to the real-time corpus and a trained user classification model, the method further comprises: Simulating the use of each function module of the target application program by a user automatically through a script to obtain a corresponding preset interface call sequence; Converting the preset interface call sequence into a preset operation statement sequence according to a preset vocabulary library to obtain a training corpus; Labeling the preset operation statement sequence in the training corpus with a user type to obtain a plurality of labeled samples to construct a training data set; Establishing an initial model according to a Bayesian text classification algorithm; Training the initial model according to the training data set to obtain a trained user classification model.

4. The application user classification method of claim 3, wherein, The step of obtaining the user type of the target user according to the real-time corpus and a trained user classification model specifically comprises: Inputting the operation statement sequence in the real-time corpus into the trained user classification model; Classifying the operation statement sequence based on the Bayesian text classification algorithm of the user classification model to obtain a classification result; Obtaining the user type of the target user according to the classification result.

5. An application evaluation method characterized by, The method comprises the following steps: Obtaining operation information of a plurality of users using a target application program in a preset period; Obtaining the user types of the plurality of users by using the application program user classification method in any one of claims 1 to 4 according to the operation information. According to the proportion of each type of user in the plurality of types of users and a preset user type proportion threshold, the target application program is evaluated to obtain an evaluation result.

6. An application user classifying apparatus characterized by comprising: The method comprises the following steps: An information collection module is configured to collect interface call information of each functional module in a target application program within a preset period, wherein the interface call information is generated based on operations of a target user on the target application program; A real-time corpus acquisition module is configured to convert the interface call information into an operation statement sequence according to a preset vocabulary to obtain a real-time corpus, wherein the preset vocabulary comprises word groups defined according to the interface call sequence; A user type acquisition module is configured to obtain a user type of the target user according to the real-time corpus and a trained user classification model, wherein the user classification model is constructed based on a natural language processing algorithm; The real-time corpus acquisition module comprises: A first conversion unit is configured to convert the interface call information into a plurality of interface call sequences according to a preset time interval; A second conversion unit is configured to convert each of the interface call sequences into an operation statement sequence comprising a plurality of word groups according to a preset vocabulary to obtain a plurality of operation statement sequences; A real-time corpus construction unit is configured to construct a real-time corpus according to the plurality of operation statement sequences.

7. An application program evaluation device characterized by comprising: The method comprises the following steps: An operation information acquisition module is configured to collect operation information of a plurality of users using a target application program within a preset period; A user classification module is configured to obtain user types of the plurality of users according to the operation information by using the application program user classification method of any one of claims 1 to 4; An application program evaluation module is configured to evaluate the target application program according to the proportion of each type of user in the plurality of types of users and a preset user type proportion threshold to obtain an evaluation result.

8. A computer device, comprising: The device comprises a memory and a processor, and the memory stores a computer program which, when executed by the processor, implements the method of any one of claims 1 to 5.

9. A storage medium having stored thereon a computer program, characterized in that The computer program can be executed by one or more processors to implement the method of any one of claims 1 to 5.

Images