Model training method, device, system and program product
By constructing calculation diagrams and using word embedding output and correlation weight to predict travel characteristics, the problem that deep network is difficult to extract complex travel characteristics is solved, effective training of the recommended driver order model is realized, and the successful order acceptance of online ride-hailing drivers is improved Rate.
Patent Information
- Application Number
- CN202411026509.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-12-13
AI Technical Summary
Existing deep networks are difficult to extract complex travel features from massive travel data and express the correlation between travel features, resulting in poor training of recommended drivers for single models.
By constructing a calculation chart, the travel words are determined using word embedding output and correlation weights, the travel characteristics are predicted, and the model parameters are updated through backpropagation, the original travel data is processed in combination with business logic and single-hot encoding, and the attention mechanism and classifier calculation are used. Loss value, optimize the model training process.
Effectively train and recommend drivers to make orders, improve the success rate of online ride-hailing drivers to take orders, and optimize travel route recommendations.
Smart Images
Figure CN119128621A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model training method, device, system and program product. Background Art
[0002] With the development of artificial intelligence technology, people's social lifestyle is undergoing a profound change, and this cutting-edge technology is showing great potential in various industries. Taking the large language model as an example, it has a unique advantage in processing natural language text tasks and generative tasks. Combining it with the task of online car-hailing can prompt drivers to go to areas with more orders for passenger transport at specific times. On the one hand, it can reduce the pressure of passenger flow in local areas; on the other hand, it can improve the success rate of drivers in accepting orders. For modeling the driving behavior of drivers, deep networks are usually used to process the massive data involved, but the current mainstream deep networks are difficult to extract complex travel features from the data; it is difficult to effectively express the correlation between various travel features. Therefore, how to effectively train a model to recommend drivers to do orders is an urgent problem to be solved. Summary of the invention
[0003] In order to solve the problem of training a model for recommending drivers, this application provides the following technical solutions:
[0004] In a first aspect, a model training method is provided, comprising:
[0005] In response to a loss value of the trained model being greater than a preset loss value, performing model training;
[0006] Obtaining a corresponding word embedding output according to the original travel data, wherein the word embedding output includes at least one word embedding output element;
[0007] Taking at least one word embedding output element as a node, constructing a computation graph, and determining a trip word based on the computation graph;
[0008] According to the trip word, determining the trip word prediction value;
[0009] The loss value of the trained model is calculated based on the predicted value of the trip word, and the model parameters are updated through back propagation.
[0010] Further, determining the trip words includes:
[0011] according to:
[0012]
[0013] Determine the trip word, where t is the trip word, i and j are integers representing the node numbers in the computation graph, n is the number of nodes in the computation graph, and e iis the word embedding output element corresponding to the i-th node in the computation graph, a ij To calculate the correlation weight between the i-th node and the j-th node in the graph, v(i) is the set of nodes adjacent to the i-th node.
[0014] Further, determining the word embedding output element corresponding to the i-th node in the computation graph includes:
[0015] Initialize the computation graph;
[0016] Select at least one node from the initialized computation graph, obtain neighbor nodes of each node in the at least one selected node, and form a neighbor node set, wherein the neighbor node is a node having a correlation weight with any node in the selected node;
[0017] Aggregate the nodes in the neighbor node set to obtain the node characteristics of any node in the computation graph;
[0018] According to the node characteristics of any node in the computation graph, the word embedding output of any node in the computation graph is determined by a first activation function.
[0019] Further, determining the correlation weight between the i-th node and the j-th node in the computation graph includes:
[0020] Use the weight matrix to transform any node in the computation graph and the neighboring nodes corresponding to any node in the computation graph, and concatenate the corresponding transformation results;
[0021] The second activation function is used to perform nonlinear transformation on the concatenated transformation results to obtain the attention mechanism weight;
[0022] Normalize the attention mechanism weights and use the normalized attention mechanism weights as the correlation weights between the i-th node and the j-th node in the computation graph.
[0023] Further, according to the trip word, determining the trip word prediction value includes:
[0024] Obtain the position code of the trip word, and add it to the trip word to obtain the corresponding trip data, wherein the position code is used to indicate the density of the corresponding trip word;
[0025] Sampling the trip word history data to generate a trip word time series, wherein the trip word history data corresponds to the trip word data at the time of historical data collection;
[0026] Predict the next trip word based on the travel data and the travel word time series.
[0027] Furthermore, the loss value of the trained model is calculated according to the predicted value of the trip word, and the model parameters are updated through back propagation, including:
[0028] Processing the travel data using at least one classifier to obtain at least one loss value, wherein the at least one loss value corresponds to at least one travel feature of the travel data;
[0029] Determining a travel feature weight of at least one travel feature according to the degree of easy distinction between positive and negative samples in at least one travel feature;
[0030] according to:
[0031]
[0032] Determine the loss value, where L is the loss value, L a is the loss value corresponding to the a-th travel feature, is the weight of the a-th loss value, a is the travel feature number, and N is the total number of travel features.
[0033] Furthermore, according to the original travel data, the corresponding word embedding output is obtained, including:
[0034] S210: Processing the original travel data based on the business logic to obtain the original travel data identifier, wherein the business logic is used to convert the original travel data into a processing logic that is recognizable and processable by a computer;
[0035] S220: Perform one-hot encoding on the original travel data identifier to obtain travel encoding data;
[0036] S230: Process the travel coding data based on the word embedding layer to obtain a word embedding output corresponding to the travel coding data.
[0037] In a second aspect, a model training device is provided, comprising:
[0038] A training judgment module, configured to perform model training in response to a loss value of the trained model being greater than a preset loss value;
[0039] A word embedding acquisition module, used to obtain corresponding word embedding output according to the original travel data, wherein the word embedding output includes at least one word embedding output element;
[0040] A computational graph construction module, configured to use at least one word embedding output element as a node, construct a computational graph, and determine a row word based on the computational graph;
[0041] A trip word prediction module, used for determining a trip word prediction value according to the trip word;
[0042] The loss value of the trained model is calculated based on the predicted value of the trip word, and the model parameters are updated through back propagation.
[0043] In a third aspect, a model training system is provided, comprising a memory, a processor, and a model training program stored in the memory and executable on the processor, wherein when the processor executes the model training program, the model training method described in the first aspect is implemented.
[0044] In a fourth aspect, a computer-readable storage medium is provided, on which a model training program is stored. When the model training program is executed by a processor, the model training method recorded in the first aspect is implemented.
[0045] In a fifth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the model training method described in the first aspect.
[0046] The beneficial effects of the technical solution provided by the embodiment of the present application are:
[0047] By implementing a model training method, device, system and program product disclosed in the embodiments of the present application, a model for recommending drivers to place orders can be trained. The model can recommend driving routes with high success rates of accepting orders to online car-hailing drivers through learning, thereby improving the success rate of online car-hailing drivers in accepting orders. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0049] Figure 1 It is a schematic diagram of a model training method provided in an embodiment of the present application;
[0050] Figure 2 This is a schematic diagram of neighbor nodes provided in an embodiment of the present application;
[0051] Figure 3 is a schematic diagram of a model training device provided in an embodiment of the present application;
[0052] Figure 4 It is a schematic diagram of a model training system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the implementation mode of this application will be clearly and completely described below in conjunction with the drawings in the implementation mode of this application. Obviously, the described implementation mode is only a part of the embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0054] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure shall have the usual meanings understood by persons with ordinary skills in the field to which the present disclosure belongs. The words "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one", "one" or "the" do not indicate a quantity limitation, but indicate that there is at least one. The numbers in the drawings of the specification only indicate the distinction between various functional components or modules, and do not indicate the logical relationship between components or modules. Words such as "include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0055] Hereinafter, various embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that in the accompanying drawings, the same reference numerals are given to components having substantially the same or similar structures and functions, and repeated descriptions thereof will be omitted.
[0056] To solve the problem of training a single model for recommending drivers, this application provides the following technical solutions:
[0057] In some embodiments, Figure 1 As shown, a model training method includes:
[0058] S100: In response to a loss value of the trained model being greater than a preset loss value, performing model training;
[0059] S200: Obtain corresponding word embedding output according to the original travel data, wherein the word embedding output includes at least one word embedding output element;
[0060] S300: Using at least one word embedding output element as a node, constructing a computation graph, and determining a row word based on the computation graph;
[0061] S400: Determine a trip word prediction value according to the trip word;
[0062] S500: Calculate the loss value of the trained model according to the trip word prediction value, and update the model parameters through back propagation.
[0063] The original travel data includes at least one of the following: time, spatial coordinates, city name, current order status, and number of successful space-time PK. Among them, time is the moment when the corresponding original travel data is collected. The spatial coordinates are the longitude and latitude of the location of the mobile terminal. The current order status indicates whether the mobile terminal user has received the order at that time. The number of successful space-time PK is the number of successful orders between the user and other users using the same application. The original travel data can be represented in the form of a vector or an array, for example: original travel data = [time, longitude and latitude, city name, order status, number of successful space-time PK, ...].
[0064] Usually, the original travel data is obtained by processing the real-time data collected by the mobile terminal at the data collection time through Spark or Flink. The original data can be online data or offline data. Adjacent data collection times can be set at equal time intervals. For example: 1 second, 10 milliseconds, etc. The initial point of the data collection time can be the time when the corresponding application in the mobile terminal is activated, or it can be determined by pre-setting.
[0065] According to the original travel data, the corresponding word embedding output can be obtained through the following process, including:
[0066] S210: Process the original travel data based on the business logic to obtain an original travel data identifier.
[0067] Business logic is the processing logic that converts the original travel data into computer-recognizable, computationally readable identification. For example, business logic processing is performed on time to obtain a time identification; business logic processing is performed on spatial coordinates to obtain an area identification, etc. The area identification can be the grid code corresponding to any grid after a city or region is divided into grids. The original travel data identification can be represented as a vector or array, for example: the original travel data identification = [time identification, area identification, city name identification, order status identification, space-time PK success number identification, ...].
[0068] S220: Perform one-hot encoding on the original travel data identifier to obtain travel encoding data.
[0069] Similarly, travel coding data can be expressed as: travel coding data = [time code, area code, city code, order acceptance status code, spatiotemporal PK success number code, ...].
[0070] S230: Process the travel coding data based on the word embedding layer to obtain a word embedding output corresponding to the travel coding data.
[0071] Usually, a set of travel coding data includes at least one travel coding data element. Accordingly, the word embedding output includes at least one word embedding output element. The word embedding output element corresponds one-to-one with the travel coding data element.
[0072] In some embodiments, determining the trip word includes:
[0073] according to:
[0074]
[0075] Determine the trip word, where t is the trip word, i and j are integers representing the node numbers in the computation graph, n is the number of nodes in the computation graph, and e i is the word embedding output element corresponding to the i-th node in the computation graph, a ij To calculate the correlation weight between the i-th node and the j-th node in the graph, v(i) is the set of nodes adjacent to the i-th node.
[0076] In some embodiments, the word embedding output element e corresponding to the i-th node in the computation graph is determined i include:
[0077] S311: Initialize the calculation graph;
[0078] Initializing the computation graph is to initialize the nodes contained in the computation graph. Node initialization is to initialize the value of the word embedding output corresponding to the node.
[0079] Preferably, the computation graph is initialized by random initialization. That is, the value of the word embedding output corresponding to the node in the computation graph is initialized with a random number. After random initialization, the value of the word embedding output corresponding to the node in the computation graph is a random number.
[0080] S312: Select at least one node from the initialized computation graph, obtain neighbor nodes of each node in the at least one selected node, and form a neighbor node set S, wherein the neighbor node is a node having a correlation weight with any node in the selected node;
[0081] Preferably, the nodes are selected from the computation graph by random selection.
[0082] Indicatively, in Figure 2 In the computation graph shown, M1, M2, and M3 are three nodes randomly selected after the computation graph is initialized. Correspondingly, the neighbor node set S includes: S1, S2, S3, and S4.
[0083] S313: Aggregate nodes in the neighbor node set to obtain node characteristics of any node in the computation graph;
[0084] Preferably, a long short-term memory network is used to aggregate nodes in a neighbor node set to obtain node characteristics of any node in the computational graph, wherein the node characteristics include at least one of the following: time-space correlation characteristics, time-order status correlation characteristics.
[0085] Specifically, according to:
[0086]
[0087] is the node characteristic of any node in the computational graph, LSTM is the aggregation operation performed using a long short-term memory network, e u is a node in the neighbor node set.
[0088] S314: According to the node characteristics of any node in the computation graph, determine the word embedding output of any node in the computation graph through a first activation function.
[0089] Specifically, according to:
[0090]
[0091] Calculate e i The word embedding output of the node at the k+1 layer, where k represents The corresponding number of network layers, σ is the first activation function, and k+1 represents The corresponding number of network layers, Indicates that and Splicing, W k is the first learnable parameter of the k-th layer network; the first learnable parameter indicates that the parameter can be optimized through learning.
[0092] Preferably, the first activation function is a hyperbolic tangent function.
[0093] Preferably, in calculating e i The word embedding output of the node at layer k+1 is increased by the first regularization coefficient b k , which can effectively prevent the model from overfitting. After increasing the first regularization coefficient, e i The word embedding output of the node at the k+1 layer is as follows:
[0094]
[0095] In some embodiments, determining the correlation weight between the i-th node and the j-th node in the computation graph includes:
[0096] S321: Use the weight matrix to transform any node in the calculation graph and the neighboring nodes corresponding to any node in the calculation graph, and concatenate the corresponding transformation results.
[0097] The weight matrix V is the i , e u The matrix for linear transformation of e i , e u Matrix to perform linear transformation.
[0098] S322: Use the second activation function to perform nonlinear transformation on the concatenated transformation results to obtain the attention mechanism weight.
[0099] Preferably, the second activation function is a LeakyReLU() function.
[0100] Specifically, the attention mechanism weights are based on:
[0101] p iu =LeakyReLU(a T [Ve i ||Ve u ])
[0102] Determine, where p iu is the attention mechanism weight, a T is the transpose of the second learnable parameter, a is the second learnable parameter, and the second learnable parameter can optimize the specific value of the parameter through learning.
[0103] S323: Normalize the attention mechanism weights, and use the normalized attention mechanism weights as the correlation weights between the i-th node and the j-th node in the calculation graph.
[0104] Specifically, according to:
[0105]
[0106] Among them, k is a node in the computation graph whose number does not overlap with i.
[0107] In some embodiments, determining a trip word prediction value according to the trip word includes:
[0108] S410: Obtaining a position code of a trip word, and adding it to the trip word to obtain corresponding trip data, wherein the position code is used to indicate the density of the corresponding trip word;
[0109] S420: Sampling trip word history data to generate a trip word time series, wherein the trip word history data corresponds to the trip word data at the time of historical data collection;
[0110] S430: Predicting the travel word at the next moment based on the travel data and the travel word time series.
[0111] The next moment's travel word can be expressed as:
[0112] y m =Cx m +Du m
[0113] Here y m Indicates the predicted next-time word. m It is the sum of the current word and position code. m is an intermediate variable. It is used to superimpose the features of the addition result of the current word and the position code. C and D are matrices with analytical solutions.
[0114] Preferably, the calculation of the above matrices C and D can be performed in parallel using a GPU to improve the efficiency of model training.
[0115] In some embodiments, calculating the loss value of the trained model according to the trip word prediction value and updating the model parameters by back propagation include:
[0116] S510: Processing the travel data using at least one classifier to obtain at least one loss value, wherein the at least one loss value corresponds to at least one travel feature of the travel data;
[0117] S520: Determine a travel feature weight of at least one travel feature according to the degree of easy distinction between positive and negative samples in the at least one travel feature;
[0118] S530: According to:
[0119]
[0120] Determine the loss value, where L is the loss value, L a is the loss value corresponding to the a-th travel feature, is the weight of the a-th loss value, a is the travel feature number, and N is the total number of travel features.
[0121] Preferably, the loss function is a focal loss function. The advantage of using the focal loss function is that the weight can be focused on the travel data that is difficult to distinguish between positive and negative samples. For each travel feature class, a multi-objective loss function based on uncertainty-based Gaussian likelihood maximization is used to weight the travel features by measuring the relative confidence between different travel features.
[0122] For example: For the loss function L of the a-th type of travel data a . Its uncertainty parameter is σa , the parameters are updated as the network is trained. And adopt σ a The corresponding loss value L is the reciprocal of the stretch a , in order to increase the weight of the determined travel data in the model and reduce the weight of the uncertain travel data in the model.
[0123] Preferably, a greedy algorithm is used to adjust the above weights to achieve the optimum.
[0124] Preferably, in order to prevent overfitting and excessive uncertainty parameter values, a second regularization coefficient logσ is added. Then the optimal loss function can be expressed as:
[0125]
[0126] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0127] In other embodiments, Figure 3 As shown, a model training device comprises:
[0128] A training judgment module, configured to perform model training in response to a loss value of the trained model being greater than a preset loss value;
[0129] A word embedding acquisition module, used to obtain corresponding word embedding output according to the original travel data, wherein the word embedding output includes at least one word embedding output element;
[0130] A computational graph construction module, configured to use at least one word embedding output element as a node, construct a computational graph, and determine a row word based on the computational graph;
[0131] A trip word prediction module, used for determining a trip word prediction value according to the trip word;
[0132] The loss value of the trained model is calculated based on the predicted value of the trip word, and the model parameters are updated through back propagation.
[0133] For the specific limitations of the model training device recorded above, please refer to the limitations of the commodity queue adjustment method above, which will not be repeated here. Each module in the above-mentioned model training device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0134] In other embodiments, Figure 4 As shown, a model training system includes a memory, a processor, and a model training program stored in the memory and executable on the processor. When the processor executes the model training program, the model training method described above is implemented, which will not be repeated here.
[0135] In other embodiments, a computer-readable storage medium stores a model training program, and when the model training program is executed by a processor, the model training method described above is implemented, which will not be repeated here.
[0136] In other embodiments, a computer program product includes a computer program, which, when executed by a processor, implements the model training method described above, which will not be described in detail here.
[0137] Usually, the computer program product is set in a computer device with strong computing power such as a server or workstation to perform model training. The computer program product can also be set in a vehicle computer to perform model training when the vehicle computer is idle.
[0138] By implementing a model training method, device, system and program product disclosed in the embodiments of the present application, a model for recommending drivers to place orders can be trained. The model can recommend driving routes with high success rates of accepting orders to online car-hailing drivers through learning, thereby improving the success rate of online car-hailing drivers in accepting orders.
[0139] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.
[0140] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program loaded on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a memory, or installed from a ROM. When the computer program is executed by an external processor, the above-mentioned functions defined in the method of the embodiment of the present application are executed.
[0141] It should be noted that the computer-readable medium of the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In an embodiment of the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.
[0142] The computer-readable medium may be included in the server; or it may exist independently without being installed in the server. The computer-readable medium carries one or more programs. When the one or more programs are executed by the server, the server: in response to detecting that the peripheral mode of the terminal is not activated, obtains the frame rate of the application on the terminal; when the frame rate meets the screen-off condition, determines whether the user is obtaining the screen information of the terminal; in response to the judgment result that the user is not obtaining the screen information of the terminal, controls the screen to enter the immediate dimming mode.
[0143] Computer program code for performing the operation of the embodiments of the present application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).
[0144] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.
[0145] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples. The description of the above embodiments is only used to help understand the method and core idea of the present application. At the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present application.
[0146] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A model training method, characterized in that: include: In response to a loss value of the trained model being greater than a preset loss value, performing model training; Obtaining a corresponding word embedding output according to the original travel data, wherein the word embedding output includes at least one word embedding output element; Using the at least one word embedding output element as a node, constructing a computation graph, and determining a row word based on the computation graph; Determining a trip word prediction value according to the trip word; The loss value of the trained model is calculated according to the row word prediction value, and the model parameters are updated through back propagation.
2. The model training method according to claim 1, characterized in that: The determined words include: according to: Determine the trip word, where t is the trip word, i and j are integers representing the node numbers in the computation graph, n is the number of nodes in the computation graph, and e i is the word embedding output element corresponding to the i-th node in the computation graph, a ij is the correlation weight between the i-th node and the j-th node in the calculation graph, and v(i) is the set of nodes adjacent to the i-th node.
3. The model training method according to claim 2, characterized in that: Determining the word embedding output element corresponding to the i-th node in the computation graph includes: Initializing the computation graph; Select at least one node from the initialized computation graph, obtain neighbor nodes of each node in the at least one selected node, and form a neighbor node set, wherein the neighbor node is a node having a correlation weight with any node in the selected node; Aggregate the nodes in the neighbor node set to obtain node characteristics of any node in the computation graph; According to the node characteristics of any node in the computation graph, a word embedding output of any node in the computation graph is determined by a first activation function.
4. The model training method according to claim 2, characterized in that: Determining the correlation weight between the i-th node and the j-th node in the computation graph includes: Use the weight matrix to transform each node in the computation graph and the neighboring nodes corresponding to each node in the computation graph, and concatenate the corresponding transformation results; Using a second activation function to perform a nonlinear transformation on the concatenated transformation results to obtain an attention mechanism weight; The attention mechanism weight is normalized, and the normalized attention mechanism weight is used as the correlation weight between the i-th node and the j-th node in the computation graph.
5. The model training method according to claim 1, characterized in that: The step of determining a trip word prediction value according to the trip word comprises: Obtaining a position code of the trip word, and adding the position code to the trip word to obtain corresponding trip data, wherein the position code is used to indicate the density of the corresponding trip word; Sampling the row word history data to generate a row word time series, wherein the row word history data corresponds to the row word data at the time of historical data collection; The travel word at the next moment is predicted based on the travel data and the travel word time series.
6. The model training method according to claim 1, characterized in that: The step of calculating the loss value of the trained model according to the row word prediction value and updating the model parameters by back propagation includes: Processing the travel data using at least one classifier to obtain at least one loss value, wherein the at least one loss value corresponds to at least one travel feature of the travel data; Determining a travel feature weight of the at least one travel feature according to the degree of easy distinction between positive and negative samples in the at least one travel feature; according to: Determine the loss value, where L is the loss value, L a is the loss value corresponding to the a-th travel feature, is the weight of the a-th loss value, a is the travel feature number, and N is the total number of travel features.
7. The model training method according to claim 1, characterized in that: The step of obtaining corresponding word embedding output according to the original travel data includes: S210: Processing the original travel data based on the business logic to obtain an original travel data identifier, wherein the business logic is used to convert the original travel data into a processing logic that is recognizable and processable by a computer; S220: Perform one-hot encoding on the original travel data identifier to obtain travel encoding data; S230: Process the travel coding data based on a word embedding layer to obtain a word embedding output corresponding to the travel coding data.
8. A model training device, characterized in that: include: A training judgment module, configured to perform model training in response to a loss value of the trained model being greater than a preset loss value; A word embedding acquisition module, used to obtain corresponding word embedding output according to the original travel data, wherein the word embedding output includes at least one word embedding output element; A computational graph construction module, configured to construct a computational graph using the at least one word embedding output element as a node, and determine a row word based on the computational graph; A trip word prediction module, used for determining a trip word prediction value according to the trip word; The loss value of the trained model is calculated according to the row word prediction value, and the model parameters are updated through back propagation.
9. A model training system, characterized in that: It includes a memory, a processor, and a model training program stored in the memory and executable on the processor. When the processor executes the model training program, the model training method described in any one of claims 1 to 7 is implemented.
10. A computer program product, characterized in that It includes a computer program, which, when executed by a processor, implements the model training method described in any one of claims 1 to 7.