Vehicle recommendation method and device and vehicle recommendation model training method and device
By introducing a pre-fine-tuned large language model and attention mechanism into the dual-tower model, the problems of insufficient feature representation and insufficient information interaction in vehicle recommendation are solved, achieving more accurate vehicle recommendation results.
Patent Information
- Application Number
- CN202510554601.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-28
- Publication Date
- 2025-09-09
AI Technical Summary
The dual-tower model has problems in the field of vehicle recommendation, such as insufficient feature representation ability and insufficient information interaction, which affects the accuracy of the recommendation results.
A pre-fine-tuned large language model is introduced to process text features, and an attention mechanism is introduced into the dual-tower model to enhance the information interaction between sub-networks by learning the correlation between different features.
The comprehensiveness and richness of feature representation are improved, and the accuracy of recommendation results is improved.
Smart Images
Figure CN120611089A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine learning technology, and in particular to a vehicle recommendation and model training method and device. Background Art
[0002] In the field of vehicle scheduling and recommendation, vehicle-cargo matching is a key issue. It involves effectively matching suitable vehicles with cargo to improve logistics efficiency and reduce transportation costs. With the rapid development of the logistics industry, the accuracy and efficiency of vehicle recommendation systems are becoming increasingly demanding.
[0003] The Twin Towers algorithm is a widely used recommendation model in industry. It processes user and item features separately through two independent sub-networks, then matches the outputs of the two sub-networks to generate recommendations. This model structure demonstrates good scalability and efficiency when processing large amounts of data.
[0004] However, the application of the twin-tower model in the vehicle recommendation field faces several challenges. For example, when processing the spatiotemporal characteristics of vehicles and cargo, it often relies on artificially constructed features, which leads to insufficient feature representation capabilities. Furthermore, the twin-tower model has shortcomings in feature interaction, that is, insufficient information exchange between the two sub-networks, which affects the accuracy of recommendation results. Summary of the Invention
[0005] The present embodiments provide a vehicle recommendation and model training method and apparatus. To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is provided below. This summary is not intended to be a comprehensive review, identify key or important elements, or delineate the scope of these embodiments. Its sole purpose is to present some concepts in a simplified form, serving as a prelude to the detailed description that follows.
[0006] In a first aspect, an embodiment of the present application provides a vehicle recommendation model training method, the method comprising:
[0007] The vehicle characteristics and cargo characteristics of multiple historical freight orders are constructed using historical transaction data within a preset period. Vehicle characteristics include numerical vehicle characteristics and text vehicle characteristics, and cargo characteristics include numerical cargo characteristics and text cargo characteristics.
[0008] Build a vehicle recommendation model; the vehicle recommendation model is the target dual-tower model, which is obtained by introducing an attention mechanism and a pre-fine-tuned large language model into the dual-tower model algorithm;
[0009] A vehicle recommendation model is trained based on numerical vehicle features and textual vehicle features, as well as numerical cargo features and textual cargo features.
[0010] Optionally, the vehicle characteristics and cargo characteristics of multiple historical freight orders can be constructed using historical transaction data within a preset period, including:
[0011] From the historical transaction data, determine the basic information of the vehicle, the historical behavior of the vehicle, the statistical characteristics of the vehicle, and the real-time characteristics of the vehicle for each historical freight order as the vehicle characteristics of each historical freight order;
[0012] From the historical transaction data, the basic information of the goods and the cargo demand information of each historical freight order are determined as the cargo characteristics of each historical freight order.
[0013] Optionally, the target dual-tower model includes a vehicle tower branch network and a cargo tower branch network;
[0014] Based on the numerical and textual vehicle features, as well as the numerical and textual cargo features, a vehicle recommendation model is trained, including:
[0015] Construct a feature set of cargo carried by each vehicle;
[0016] Input numerical vehicle features and text vehicle features into the vehicle tower branch network, and output DNN features and final vehicle features for each vehicle;
[0017] The DNN features are weightedly fused with the cargo feature set of each vehicle to obtain the vehicle cargo attention features of each vehicle;
[0018] Input the numerical cargo features, textual cargo features, and vehicle-cargo attention features of each vehicle into the cargo tower branch network, and output the final cargo features of each cargo;
[0019] A pre-trained vehicle recommendation model is generated based on the final vehicle features of each vehicle and the final cargo features of each cargo.
[0020] Optionally, construct a cargo feature set for each vehicle, including:
[0021] Determine the historical shipping orders for each vehicle based on historical transaction data;
[0022] The attention mechanism is used to process the historical shipping orders of each vehicle to obtain the feature set of the cargo carried by each vehicle.
[0023] Optionally, the vehicle tower branch network includes a pre-fine-tuned large language model, SENet network, convolutional neural network, and fully connected layers;
[0024] The numerical vehicle features and text vehicle features are input into the vehicle tower branch network, and the DNN features and final vehicle features of each vehicle are output, including:
[0025] Perform feature mapping on the numerical vehicle features to obtain the Embedding vehicle features;
[0026] Input the text vehicle features into the pre-fine-tuned large language model to obtain LLM vehicle features;
[0027] Splice the Embedding vehicle features with the LLM vehicle features to obtain the vehicle splicing features;
[0028] The vehicle splicing features are processed sequentially through the SENet network and the convolutional neural network, and the DNN features of each vehicle are output;
[0029] The DNN features of each vehicle are input into the fully connected layer for processing to obtain the final vehicle features of each vehicle.
[0030] Optionally, the cargo tower branch network includes a pre-fine-tuned large language model, SENet network, convolutional neural network, and fully connected layers;
[0031] The numerical cargo features, textual cargo features, and vehicle-cargo attention features of each vehicle are input into the cargo tower branch network, and the final cargo features of each cargo are output, including:
[0032] Perform feature mapping on the numerical cargo features to obtain the Embedding cargo features;
[0033] Input the text product features and the preset text format into the pre-tuned large language model to obtain the LLM product features;
[0034] Combine the Embedding cargo features with the LLM cargo features to obtain cargo concatenation features;
[0035] Output the concatenated features of the goods to the SENet network, and output the SENet features of each goods;
[0036] Concatenate the vehicle cargo attention features of each vehicle with the SENet features of each cargo, and output the secondary concatenated features of each cargo;
[0037] The secondary splicing features of each cargo are sequentially input into the convolutional neural network and the fully connected layer for processing, and the final cargo features of each cargo are output.
[0038] Optionally, a pre-trained vehicle recommendation model is generated based on the final vehicle features of each vehicle and the final cargo features of each cargo, including:
[0039] Perform a dot product operation on the final vehicle feature of each vehicle and the final cargo feature of each cargo to obtain the vehicle-cargo matching similarity between each vehicle and each cargo;
[0040] Determine the actual matching vehicle and cargo orders based on the vehicle and cargo characteristics of each historical freight order;
[0041] From the vehicle-cargo matching similarities between each vehicle and each cargo, multiple target vehicle-cargo matching similarities corresponding to the actual matching vehicle-cargo orders are obtained;
[0042] Based on the similarity of multiple target vehicle-cargo matching, calculate the average precision of all cargo sources;
[0043] When the mean is within a preset mean range, a pre-trained vehicle recommendation model is obtained.
[0044] In a second aspect, an embodiment of the present application provides a vehicle recommendation method, the method comprising:
[0045] Receive a vehicle recommendation request, which carries cargo data;
[0046] Inputting cargo data into a pre-trained vehicle recommendation model and outputting a first cargo vector corresponding to the cargo data; the pre-trained vehicle recommendation model is trained using a vehicle recommendation model training method;
[0047] Performing a similarity search between the first cargo vector and vehicle data pre-stored in a database to obtain a plurality of matched candidate vehicle data and a historical vehicle vector for each candidate vehicle;
[0048] Obtain real-time characteristics of each candidate vehicle;
[0049] Perform secondary sorting based on the real-time characteristics of each candidate vehicle, the historical vehicle vector of each candidate vehicle, and the first cargo vector to obtain the final vehicle sorting result;
[0050] Display the final vehicle sorting results.
[0051] Optionally, the method further includes:
[0052] Obtain historical vehicle data according to a preset period;
[0053] Input historical vehicle data into the pre-trained vehicle recommendation model and output the historical vehicle vector corresponding to the historical vehicle data;
[0054] The historical vehicle vectors corresponding to the historical vehicle data and the historical vehicle data are stored in a database.
[0055] In a third aspect, an embodiment of the present application provides a vehicle recommendation model training device, the device comprising:
[0056] A feature construction module is used to construct vehicle features and cargo features of multiple historical freight orders based on historical transaction data within a preset period; vehicle features include numerical vehicle features and text vehicle features, and cargo features include numerical cargo features and text cargo features;
[0057] The model building module is used to build a vehicle recommendation model. The vehicle recommendation model is a target dual-tower model, which is obtained by introducing an attention mechanism and a pre-fine-tuned large language model into the dual-tower model algorithm.
[0058] The model training module is used to train a vehicle recommendation model based on numerical vehicle features and text vehicle features, numerical cargo features and text cargo features.
[0059] The technical solutions provided by the embodiments of the present application may have the following beneficial effects:
[0060] In the embodiments of the present application, on the one hand, by introducing a pre-fine-tuned large language model to process text features, the automatic processing method of the large language model is highly efficient and avoids manually constructed features, thereby making the feature representation more comprehensive and rich, thereby improving the ability of feature representation; on the other hand, by introducing the attention mechanism in the dual-tower model, the model can pay more attention to important information when processing vehicle and cargo features. The attention mechanism enhances the information interaction between the two sub-networks by learning the correlation between different features, thereby improving the accuracy of the recommendation results.
[0061] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0063] Figure 1 This is a flow chart of a vehicle recommendation model training method provided in an embodiment of the present application;
[0064] Figure 2 This is a schematic diagram of a model architecture of a vehicle recommendation model provided in an embodiment of the present application;
[0065] Figure 3 This is a schematic diagram of the processing process of an attention mechanism provided in an embodiment of the present application;
[0066] Figure 4 This is a flow chart of a vehicle recommendation method provided in an embodiment of the present application;
[0067] Figure 5This is a schematic block diagram of a vehicle recommendation process provided by an embodiment of the present application;
[0068] Figure 6 This is a schematic diagram of a user interface provided in an embodiment of the present application;
[0069] Figure 7 This is a structural diagram of a vehicle recommendation model training device provided in an embodiment of the present application;
[0070] Figure 8 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0071] The following description and the drawings sufficiently illustrate specific embodiments of the application to enable those skilled in the art to practice them.
[0072] It should be clear that the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0073] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with certain aspects of the present application, as detailed in the appended claims.
[0074] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances. In addition, in the description of this application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.
[0075] The present application provides a vehicle recommendation and model training method and device to solve the problems existing in the above-mentioned related technical problems. In the embodiment of the present application, on the one hand, by introducing a pre-fine-tuned large language model to process text features, the large language model's automated processing method is highly efficient, avoiding manually constructed features, thereby making the feature representation more comprehensive and rich, thereby improving the ability of feature representation; on the other hand, by introducing an attention mechanism in the dual-tower model, the model can pay more attention to important information when processing vehicle and cargo features. The attention mechanism enhances the information interaction between the two sub-networks by learning the correlation between different features, thereby improving the accuracy of the recommendation results. The following uses an exemplary embodiment for detailed description.
[0076] This application provides a vehicle recommendation and model training method and device to solve the problems existing in the above-mentioned related technical issues. Figure 1 -Attached Figure 6 This article details the vehicle recommendation model training method provided in the embodiments of this application. This method can be implemented using a computer program and run on a vehicle recommendation model training device based on the von Neumann architecture. This computer program can be integrated into an application or run as a standalone tool application.
[0077] See Figure 1 , is a flow chart of a vehicle recommendation model training method provided in an embodiment of the present application. Figure 1 As shown, the method of the embodiment of the present application may include the following steps:
[0078] S101, constructing vehicle characteristics and cargo characteristics of multiple historical freight orders based on historical transaction data within a preset period; the vehicle characteristics include numerical vehicle characteristics and textual vehicle characteristics, and the cargo characteristics include numerical cargo characteristics and textual cargo characteristics;
[0079] Historical transaction data refers to completed freight orders within a predefined time period. This data includes information such as vehicle and cargo matching, transportation time, transportation route, and cargo loading and unloading locations. Vehicle characteristics refer to data describing various vehicle attributes and behaviors. Numerical vehicle characteristics refer to numerically quantifiable vehicle attributes, such as vehicle load capacity, dimensions, fuel type, registration year, maintenance history, and mileage. Textual vehicle characteristics refer to vehicle-related information expressed in text format, such as vehicle make, model, registration location, usage description, and historical driving routes. These features are extracted and converted into a format that can be processed by the model using natural language processing techniques. Cargo characteristics refer to data describing various cargo attributes and requirements. Numerical cargo characteristics refer to numerically quantifiable cargo attributes, such as cargo weight, volume, type, value, and loading and unloading time windows. Textual cargo characteristics refer to textual cargo-related information, such as cargo description, special handling requirements, and the detailed address of the loading and unloading location.
[0080] In some embodiments of the present application, the specific process of constructing vehicle characteristics and cargo characteristics of multiple historical freight orders through historical transaction data within a preset period includes: determining the basic vehicle information, vehicle historical behavior, vehicle statistical characteristics and vehicle real-time characteristics of each historical freight order from the historical transaction data as the vehicle characteristics of each historical freight order; determining the basic cargo information and cargo demand information of each historical freight order from the historical transaction data as the cargo characteristics of each historical freight order.
[0081] Specifically, the basic information of the vehicle includes the model, length, registration province and city, fuel type, etc.; the vehicle's historical behavior includes a list of cities it has passed through, a list of cities it has stopped in, and a list of related POI names based on the reported trajectory; the vehicle's statistical features include mileage, proportion of highways, proportion of national highways, etc.; the vehicle's real-time features include vehicle location, speed, length of stop, etc.
[0082] Specifically, the basic information of the cargo includes the starting and ending cities, loading and unloading addresses, cargo type, cargo name, weight, and transportation distance; the cargo demand information includes loading time, vehicle type and length, time limit, quantity, etc.
[0083] S102, constructing a vehicle recommendation model; the vehicle recommendation model is a target dual-tower model, which is obtained by introducing an attention mechanism and a pre-fine-tuned large language model into the dual-tower model algorithm;
[0084] Among them, the Two-Tower Model algorithm is an architecture commonly used in recommendation systems, consisting of two parallel sub-networks (or "towers"). The Attention Mechanism is a mechanism widely used in deep learning models that allows the model to focus more on the most important parts of the input data when processing information. Large Language Models (LLMs) refer to natural language processing models with a large number of parameters. These models are typically pre-trained on large amounts of text data to learn the complex structure and semantics of language.
[0085] S103 , training a vehicle recommendation model based on the numerical vehicle features and the textual vehicle features, and the numerical cargo features and the textual cargo features.
[0086] The target dual-tower model includes a vehicle tower branch network and a cargo tower branch network.
[0087] In some embodiments of the present application, the specific process of training a vehicle recommendation model based on numerical vehicle features and textual vehicle features, numerical cargo features and textual cargo features includes: constructing a set of cargo carrying features for each vehicle; inputting the numerical vehicle features and textual vehicle features into the vehicle tower branch network, and outputting the DNN features and final vehicle features of each vehicle; weightedly fusing the DNN features with the cargo carrying feature set of each vehicle to obtain the vehicle-cargo attention features of each vehicle; inputting the numerical cargo features and textual cargo features, and the vehicle-cargo attention features of each vehicle into the cargo tower branch network, and outputting the final cargo features of each cargo; generating a pre-trained vehicle recommendation model based on the final vehicle features of each vehicle and the final cargo features of each cargo.
[0088] The cargo feature set refers to the set of cargo features associated with each vehicle. These features can include information such as cargo type, weight, volume, and destination. The Vehicle Tower Network is a subnetwork in the dual-tower model that specifically processes vehicle-related features. The Goods Tower Network is another subnetwork in the dual-tower model that specifically processes cargo-related features. Weighted fusion is the process of combining DNN features with the vehicle's cargo feature set.
[0089] In some embodiments of the present application, the specific process of constructing a feature set of cargo carried by each vehicle includes: determining the historical cargo carrying orders of each vehicle based on historical transaction data; and using an attention mechanism to process the historical cargo carrying orders of each vehicle to obtain a feature set of cargo carried by each vehicle.
[0090] Among them, for example Figure 2 As shown, the vehicle tower branch network includes a pre-fine-tuned large language model, SENet network, convolutional neural network and fully connected layers.
[0091] In some embodiments of the present application, the specific process of inputting numerical vehicle features and textual vehicle features into the vehicle tower branch network and outputting the DNN features and final vehicle features of each vehicle includes: feature mapping the numerical vehicle features to obtain Embedding vehicle features; inputting the textual vehicle features into a pre-fine-tuned large language model to obtain LLM vehicle features; splicing the Embedding vehicle features with the LLM vehicle features to obtain vehicle splicing features; processing the vehicle splicing features through the SENet network and the convolutional neural network in sequence to output the DNN features of each vehicle; inputting the DNN features of each vehicle into the fully connected layer for processing to obtain the final vehicle features of each vehicle.
[0092] Among them, for example Figure 2 As shown in Figure 1, the cargo tower branch network includes a pre-fine-tuned large language model, a SENet network, a convolutional neural network, and a fully connected layer.
[0093] In some embodiments of the present application, the numerical cargo features, textual cargo features, and vehicle-cargo attention features of each vehicle are input into the cargo tower branch network, and the specific process of outputting the final cargo features of each cargo includes: feature mapping the numerical cargo features to obtain Embedding cargo features; inputting the textual cargo features and the preset text format into the pre-fine-tuned large language model to obtain LLM cargo features; splicing the Embedding cargo features with the LLM cargo features to obtain cargo splicing features; outputting the cargo splicing features into the SENet network to output the SENet features of each cargo; splicing the vehicle-cargo attention features of each vehicle with the SENet features of each cargo to output the secondary splicing features of each cargo; inputting the secondary splicing features of each cargo into the convolutional neural network and the fully connected layer in sequence for processing to output the final cargo features of each cargo.
[0094] The default text formats are: LLM (Vehicle Historical Loading and Unloading Cities: xx City xx times, xx City xx times; Historical Passage Cities: xx City xx times; Historical Loading and Unloading Locations: xx Company xx times), LLM (Vehicle Search Order Origin City: xx City, Destination City: xx City, Loading Address: xx Company, Unloading Address: xx Company). LLM can be fine-tuned to a certain extent by selecting bge-m3.
[0095] Among them, the cargo feature set of each vehicle is as follows: Figure 3 As shown in the dotted box, Figure 2The vehicle feature Q output by the last layer of the convolutional neural network (DNN) in the vehicle tower branch network is fused with the cargo feature set shown in the dotted box to obtain the vehicle cargo attention feature V of each vehicle.
[0096] In some embodiments of the present application, the specific process of generating a pre-trained vehicle recommendation model based on the final vehicle characteristics of each vehicle and the final cargo characteristics of each cargo includes: performing a dot product operation on the final vehicle characteristics of each vehicle and the final cargo characteristics of each cargo to obtain the vehicle-cargo matching similarity between each vehicle and each cargo; determining the actual matching vehicle-cargo order based on the vehicle characteristics and cargo characteristics of each historical freight order; obtaining multiple target vehicle-cargo matching similarities corresponding to the actual matching vehicle-cargo order from the vehicle-cargo matching similarities between each vehicle and each cargo; calculating the mean of the average precision of all cargo sources based on the multiple target vehicle-cargo matching similarities; and obtaining the pre-trained vehicle recommendation model when the mean is within a preset mean range.
[0097] Specifically, the formula for calculating the mean of the average precision of all sources is:
[0098]
[0099] Among them, MAP is the mean of the average precision of all sources, AP i is the similarity of the i-th target vehicle-cargo matching.
[0100] In the embodiments of the present application, on the one hand, by introducing a pre-fine-tuned large language model to process text features, the automatic processing method of the large language model is highly efficient and avoids manually constructed features, thereby making the feature representation more comprehensive and rich, thereby improving the ability of feature representation; on the other hand, by introducing the attention mechanism in the dual-tower model, the model can pay more attention to important information when processing vehicle and cargo features. The attention mechanism enhances the information interaction between the two sub-networks by learning the correlation between different features, thereby improving the accuracy of the recommendation results.
[0101] See Figure 4 , provides a flow chart of a vehicle recommendation method according to an embodiment of the present application. Figure 4 As shown, the method of the embodiment of the present application may include the following steps:
[0102] S201, receiving a vehicle recommendation request, the vehicle recommendation request carrying cargo data;
[0103] S202, inputting cargo data into a pre-trained vehicle recommendation model, and outputting a first cargo vector corresponding to the cargo data; the pre-trained vehicle recommendation model is trained using a vehicle recommendation model training method;
[0104] S203, performing a similarity search between the first cargo vector and vehicle data pre-stored in the database to obtain a plurality of matched candidate vehicle data and a historical vehicle vector for each candidate vehicle;
[0105] S204, obtaining real-time characteristics of each candidate vehicle;
[0106] S205 , performing secondary sorting based on the real-time characteristics of each candidate vehicle, the historical vehicle vector of each candidate vehicle, and the first cargo vector to obtain a final vehicle sorting result;
[0107] In some embodiments of the present application, the real-time features of each candidate vehicle and the historical vehicle vector of each candidate vehicle are weightedly fused to obtain the final fused features of each candidate vehicle. The final similarity between the final fused features of each candidate vehicle and the first cargo vector is calculated, and the multiple calculated final similarities are sorted to obtain the final vehicle sorting result.
[0108] For example Figure 5 As shown in the figure, because vehicle location information is updated dynamically in real time, the cost of re-indexing by computing features in real time is high. Therefore, a hybrid search approach (coarse sorting + fine sorting) is adopted in actual online deployments. Real-time features such as location and stop duration are introduced into the dual-tower model algorithm for refined secondary sorting. For example, vehicle data is processed and input into the model periodically (e.g., weekly). The output result vector is stored synchronously with the vehicle data in the ES database. Cargo information is input into the model in real time. The output result vector is queried in ES through a hybrid search (filtering + TOPK). The features are then recombined for refined secondary sorting, and the final ranking result is output.
[0109] S206, displaying the final vehicle sorting result.
[0110] In some embodiments of the present application, the method also includes: obtaining historical vehicle data according to a preset period; inputting the historical vehicle data into a pre-trained vehicle recommendation model, and outputting a historical vehicle vector corresponding to the historical vehicle data; and storing the historical vehicle vector corresponding to the historical vehicle data and the historical vehicle data in a database.
[0111] In one possible implementation, the source of goods in the vehicle recommendation request needs to find a vehicle from Zhengzhou to Foshan. After being processed by the solution provided by this application, the displayed result is as follows: Figure 6 shown.
[0112] In the embodiments of the present application, on the one hand, by introducing a pre-fine-tuned large language model to process text features, the automatic processing method of the large language model is highly efficient and avoids manually constructed features, thereby making the feature representation more comprehensive and rich, thereby improving the ability of feature representation; on the other hand, by introducing the attention mechanism in the dual-tower model, the model can pay more attention to important information when processing vehicle and cargo features. The attention mechanism enhances the information interaction between the two sub-networks by learning the correlation between different features, thereby improving the accuracy of the recommendation results.
[0113] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.
[0114] See Figure 7 , which shows a schematic diagram of the structure of a vehicle recommendation model training device provided by an exemplary embodiment of the present application. The vehicle recommendation model training device can be implemented as all or part of an electronic device through software, hardware, or a combination of both. The device includes a private dataset, a general model deployed for federated learning, and a local model. The general model is a model shared by all clients, and the local model is trained for local services. The device 1 includes a feature construction module 10, a model construction module 20, and a model training module 30.
[0115] A feature construction module 10 is configured to construct vehicle features and cargo features of multiple historical freight orders based on historical transaction data within a preset period; vehicle features include numerical vehicle features and textual vehicle features, and cargo features include numerical cargo features and textual cargo features;
[0116] The model building module 20 is used to build a vehicle recommendation model; the vehicle recommendation model is a target twin-tower model, which is obtained by introducing an attention mechanism and a pre-fine-tuned large language model into the twin-tower model algorithm;
[0117] The model training module 30 is used to train a vehicle recommendation model based on the numerical vehicle features and textual vehicle features, and the numerical cargo features and textual cargo features.
[0118] It should be noted that the vehicle recommendation model training device provided in the above embodiment, when executing the vehicle recommendation model training method, only uses the division of the above-mentioned functional modules as an example. In actual application, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the vehicle recommendation model training device provided in the above embodiment and the vehicle recommendation model training method embodiment are based on the same concept. The implementation process is detailed in the method embodiment and will not be repeated here.
[0119] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0120] In the embodiments of the present application, on the one hand, by introducing a pre-fine-tuned large language model to process text features, the automatic processing method of the large language model is highly efficient and avoids manually constructed features, thereby making the feature representation more comprehensive and rich, thereby improving the ability of feature representation; on the other hand, by introducing the attention mechanism in the dual-tower model, the model can pay more attention to important information when processing vehicle and cargo features. The attention mechanism enhances the information interaction between the two sub-networks by learning the correlation between different features, thereby improving the accuracy of the recommendation results.
[0121] The present application also provides a computer-readable medium having program instructions stored thereon, which, when executed by a processor, implement the vehicle recommendation model training method provided by the above-mentioned various method embodiments.
[0122] The present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the vehicle recommendation model training methods of the above-mentioned various method embodiments.
[0123] See Figure 8 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device 1000 may include: at least one processor 1001 , at least one network interface 1004 , a user interface 1003 , a memory 1005 , and at least one communication bus 1002 .
[0124] The communication bus 1002 is used to implement the connection and communication between these components.
[0125] The user interface 1003 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.
[0126] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).
[0127] The processor 1001 may include one or more processing cores. The processor 1001 utilizes various interfaces and circuits to connect the various components within the entire electronic device 1000. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and calling data stored in the memory 1005, the processor 1001 performs various functions of the electronic device 1000 and processes data. Optionally, the processor 1001 may be implemented in at least one hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 1001 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display; and the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 1001 and may be implemented separately on a single chip.
[0128] Among them, the memory 1005 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 1005 may also be optionally at least one storage system located away from the aforementioned processor 1001. As Figure 8 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a vehicle recommendation model training application.
[0129] exist Figure 8In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and obtain user input data; and the processor 1001 can be used to call the vehicle recommendation model training application stored in the memory 1005 and specifically perform the following operations:
[0130] The vehicle characteristics and cargo characteristics of multiple historical freight orders are constructed using historical transaction data within a preset period. Vehicle characteristics include numerical vehicle characteristics and text vehicle characteristics, and cargo characteristics include numerical cargo characteristics and text cargo characteristics.
[0131] Build a vehicle recommendation model; the vehicle recommendation model is the target dual-tower model, which is obtained by introducing an attention mechanism and a pre-fine-tuned large language model into the dual-tower model algorithm;
[0132] A vehicle recommendation model is trained based on numerical vehicle features and textual vehicle features, as well as numerical cargo features and textual cargo features.
[0133] In one embodiment, when the processor 1001 constructs vehicle characteristics and cargo characteristics of multiple historical freight orders using historical transaction data within a preset period, the processor 1001 specifically performs the following operations:
[0134] From the historical transaction data, determine the basic information of the vehicle, the historical behavior of the vehicle, the statistical characteristics of the vehicle, and the real-time characteristics of the vehicle for each historical freight order as the vehicle characteristics of each historical freight order;
[0135] From the historical transaction data, the basic information of the goods and the cargo demand information of each historical freight order are determined as the cargo characteristics of each historical freight order.
[0136] In one embodiment, when the processor 1001 trains a vehicle recommendation model based on numerical vehicle features and textual vehicle features, and numerical cargo features and textual cargo features, the processor 1001 specifically performs the following operations:
[0137] Construct a feature set of cargo carried by each vehicle;
[0138] Input numerical vehicle features and text vehicle features into the vehicle tower branch network, and output DNN features and final vehicle features for each vehicle;
[0139] The DNN features are weightedly fused with the cargo feature set of each vehicle to obtain the vehicle cargo attention features of each vehicle;
[0140] Input the numerical cargo features, textual cargo features, and vehicle-cargo attention features of each vehicle into the cargo tower branch network, and output the final cargo features of each cargo;
[0141] A pre-trained vehicle recommendation model is generated based on the final vehicle features of each vehicle and the final cargo features of each cargo.
[0142] In one embodiment, when constructing a cargo feature set for each vehicle, the processor 1001 specifically performs the following operations:
[0143] Determine the historical shipping orders for each vehicle based on historical transaction data;
[0144] The attention mechanism is used to process the historical shipping orders of each vehicle to obtain the feature set of the cargo carried by each vehicle.
[0145] In one embodiment, when the processor 1001 inputs the numerical vehicle features and the textual vehicle features into the vehicle tower branch network and outputs the DNN features and the final vehicle features of each vehicle, the processor 1001 specifically performs the following operations:
[0146] Perform feature mapping on the numerical vehicle features to obtain the Embedding vehicle features;
[0147] Input the text vehicle features into the pre-fine-tuned large language model to obtain LLM vehicle features;
[0148] Splice the Embedding vehicle features with the LLM vehicle features to obtain the vehicle splicing features;
[0149] The vehicle splicing features are processed sequentially through the SENet network and the convolutional neural network, and the DNN features of each vehicle are output;
[0150] The DNN features of each vehicle are input into the fully connected layer for processing to obtain the final vehicle features of each vehicle.
[0151] In one embodiment, when the processor 1001 inputs the numerical cargo features, the textual cargo features, and the vehicle-cargo attention features of each vehicle into the cargo tower branch network and outputs the final cargo features of each cargo, the processor 1001 specifically performs the following operations:
[0152] Perform feature mapping on the numerical cargo features to obtain the Embedding cargo features;
[0153] Input the text product features and the preset text format into the pre-tuned large language model to obtain the LLM product features;
[0154] Combine the Embedding cargo features with the LLM cargo features to obtain cargo concatenation features;
[0155] Output the concatenated features of the goods to the SENet network, and output the SENet features of each goods;
[0156] Concatenate the vehicle cargo attention features of each vehicle with the SENet features of each cargo, and output the secondary concatenated features of each cargo;
[0157] The secondary splicing features of each cargo are sequentially input into the convolutional neural network and the fully connected layer for processing, and the final cargo features of each cargo are output.
[0158] In one embodiment, when the processor 1001 generates a pre-trained vehicle recommendation model based on the final vehicle features of each vehicle and the final cargo features of each cargo, the processor 1001 specifically performs the following operations:
[0159] Perform a dot product operation on the final vehicle feature of each vehicle and the final cargo feature of each cargo to obtain the vehicle-cargo matching similarity between each vehicle and each cargo;
[0160] Determine the actual matching vehicle and cargo orders based on the vehicle and cargo characteristics of each historical freight order;
[0161] From the vehicle-cargo matching similarities between each vehicle and each cargo, multiple target vehicle-cargo matching similarities corresponding to the actual matching vehicle-cargo orders are obtained;
[0162] Based on the similarity of multiple target vehicle-cargo matching, calculate the average precision of all cargo sources;
[0163] When the mean is within a preset mean range, a pre-trained vehicle recommendation model is obtained.
[0164] In the embodiments of the present application, on the one hand, by introducing a pre-fine-tuned large language model to process text features, the automatic processing method of the large language model is highly efficient and avoids manually constructed features, thereby making the feature representation more comprehensive and rich, thereby improving the ability of feature representation; on the other hand, by introducing the attention mechanism in the dual-tower model, the model can pay more attention to important information when processing vehicle and cargo features. The attention mechanism enhances the information interaction between the two sub-networks by learning the correlation between different features, thereby improving the accuracy of the recommendation results.
[0165] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The vehicle recommendation model training program can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory, or a random access memory.
[0166] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.
Claims
1. A vehicle recommendation model training method, characterized in that: The method comprises: Constructing vehicle features and cargo features of multiple historical freight orders using historical transaction data within a preset period; the vehicle features include numerical vehicle features and text vehicle features, and the cargo features include numerical cargo features and text cargo features; Constructing a vehicle recommendation model; the vehicle recommendation model is a target twin-tower model, which is obtained by introducing an attention mechanism and a pre-fine-tuned large language model into the twin-tower model algorithm; The vehicle recommendation model is trained based on the numerical vehicle features and textual vehicle features, and the numerical cargo features and textual cargo features.
2. The method according to claim 1, characterized in that The vehicle characteristics and cargo characteristics of multiple historical freight orders are constructed by using historical transaction data within a preset period, including: Determining basic vehicle information, historical vehicle behavior, statistical characteristics, and real-time characteristics of each historical freight order from the historical transaction data as vehicle characteristics of each historical freight order; From the historical transaction data, basic cargo information and cargo demand information of each historical freight order are determined as cargo features of each historical freight order.
3. The method according to claim 1, characterized in that The target dual-tower model includes a vehicle tower branch network and a cargo tower branch network; The step of training the vehicle recommendation model according to the numerical vehicle features and the textual vehicle features, and the numerical cargo features and the textual cargo features, comprises: Construct a feature set of cargo carried by each vehicle; Input the numerical vehicle features and text vehicle features into the vehicle tower branch network, and output the DNN features and final vehicle features of each vehicle; Performing weighted fusion of the DNN features with the cargo feature set of each vehicle to obtain a vehicle cargo attention feature of each vehicle; Inputting the numerical cargo features, the textual cargo features, and the vehicle-cargo attention features of each vehicle into the cargo tower branch network, and outputting the final cargo features of each cargo; A pre-trained vehicle recommendation model is generated according to the final vehicle features of each vehicle and the final cargo features of each cargo.
4. The method according to claim 3, characterized in that The step of constructing a cargo feature set for each vehicle includes: Determining historical shipping orders for each vehicle based on the historical transaction data; An attention mechanism is used to process the historical shipping orders of each vehicle to obtain a feature set of the cargo shipped by each vehicle.
5. The method according to claim 3, characterized in that The vehicle tower branch network includes a pre-fine-tuned large language model, a SENet network, a convolutional neural network, and a fully connected layer; The step of inputting the numerical vehicle features and the text vehicle features into the vehicle tower branch network and outputting the DNN features and the final vehicle features of each vehicle includes: Perform feature mapping on the numerical vehicle features to obtain Embedding vehicle features; Inputting the text vehicle features into a pre-fine-tuned large language model to obtain LLM vehicle features; Splicing the Embedding vehicle feature with the LLM vehicle feature to obtain a vehicle splicing feature; Processing the vehicle splicing features sequentially through the SENet network and the convolutional neural network, and outputting the DNN features of each vehicle; The DNN features of each vehicle are input into the fully connected layer for processing to obtain the final vehicle features of each vehicle.
6. The method according to claim 3, characterized in that The cargo tower branch network includes a pre-fine-tuned large language model, a SENet network, a convolutional neural network, and a fully connected layer; The step of inputting the numerical cargo features, the textual cargo features, and the vehicle-cargo attention features of each vehicle into the cargo tower branch network and outputting the final cargo features of each cargo comprises: Perform feature mapping on the numerical cargo features to obtain Embedding cargo features; Inputting the text product features and the preset text format into a pre-tuned large language model to obtain LLM product features; Splicing the Embedding cargo feature with the LLM cargo feature to obtain a cargo splicing feature; Output the goods splicing features to the SENet network, and output the SENet features of each goods; Concatenate the vehicle cargo attention features of each vehicle with the SENet features of each cargo, and output a secondary concatenated feature of each cargo; The secondary splicing features of each cargo are sequentially input into the convolutional neural network and the fully connected layer for processing, and the final cargo features of each cargo are output.
7. The method according to claim 3, characterized in that The generating of a pre-trained vehicle recommendation model according to the final vehicle characteristics of each vehicle and the final cargo characteristics of each cargo includes: Performing a dot product operation on the final vehicle feature of each vehicle and the final cargo feature of each cargo to obtain a vehicle-cargo matching similarity between each vehicle and each cargo; Determining an actual matching vehicle and cargo order based on the vehicle characteristics and cargo characteristics of each historical freight order; Obtaining a plurality of target vehicle-cargo matching similarities corresponding to the actually matched vehicle-cargo orders from the vehicle-cargo matching similarities between each vehicle and each cargo; Calculate the average precision of all cargo sources based on the multiple target vehicle-cargo matching similarities; When the mean is within a preset mean interval, a pre-trained vehicle recommendation model is obtained.
8. A vehicle recommendation method, characterized in that: The method comprises: Receive a vehicle recommendation request, which carries cargo data; Inputting the cargo data into a pre-trained vehicle recommendation model and outputting a first cargo vector corresponding to the cargo data; the pre-trained vehicle recommendation model is trained by the vehicle recommendation model training method according to any one of claims 1 to 7; Performing a similarity search between the first cargo vector and vehicle data pre-stored in a database to obtain a plurality of matched candidate vehicle data and a historical vehicle vector for each candidate vehicle; Obtain real-time characteristics of each candidate vehicle; Perform secondary sorting based on the real-time characteristics of each candidate vehicle, the historical vehicle vector of each candidate vehicle, and the first cargo vector to obtain a final vehicle sorting result; The final vehicle sorting result is displayed.
9. The method according to claim 8, characterized in that The method further comprises: Obtain historical vehicle data according to a preset period; Inputting the historical vehicle data into a pre-trained vehicle recommendation model, and outputting a historical vehicle vector corresponding to the historical vehicle data; The historical vehicle vector corresponding to the historical vehicle data and the historical vehicle data are stored in a database.
10. A vehicle recommendation model training device, characterized in that: The device comprises: A feature construction module is used to construct vehicle features and cargo features of multiple historical freight orders based on historical transaction data within a preset period; the vehicle features include numerical vehicle features and text vehicle features, and the cargo features include numerical cargo features and text cargo features; A model building module is used to build a vehicle recommendation model; the vehicle recommendation model is a target twin-tower model, which is obtained by introducing an attention mechanism and a pre-fine-tuned large language model into the twin-tower model algorithm; The model training module is used to train the vehicle recommendation model based on the numerical vehicle features and text vehicle features, and the numerical cargo features and text cargo features.