Personnel operation style quantitative identification method based on preference learning
The method uses preference learning and a multi-modal Transformer model to quantify driving styles, addressing the limitations of traditional classification methods by providing continuous metrics and accurate alignment with individual driver preferences.
Patent Information
- Application Number
- CN202411205204.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-13
AI Technical Summary
Existing methods for identifying human operating styles in driving tasks are inadequate, as they either oversimplify human behavior into a limited number of categories or lack sufficient training data when attempting to classify into many categories, leading to inaccurate alignment with individual driver preferences.
A method using preference learning to quantify driving styles by recording driver behavior data, labeling with a GPT-4 large model, and constructing a multi-modal multi-head encoder and Transformer-based model to iteratively train a quantitative identification model, utilizing a Bradley-Terry preference model for alignment.
The method achieves precise alignment of human driving styles by providing continuous numerical metrics and reducing labeling complexity and cost, ensuring accurate style identification regardless of the driver's current operation status.
Smart Images
Figure CN119129635A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of human-computer interaction technology, and in particular to a method for quantitatively identifying a person's operating style based on preference learning. Background Art
[0002] At present, many robot systems still require operators to operate, such as space teleoperation, human skill teaching, driving tasks, etc. Space teleoperation can be divided into manipulator operation on orbit and mobile robot operation on the surface of other celestial bodies. For mobile robot operation on the surface of celestial bodies, due to its complex environment and many unseen scenes, human teleoperation is often required, which is similar to the situation of ground driving tasks. For such tasks, the characteristics of human operation usually affect the design of such robot systems. For example, in driving tasks, the autonomous driving system will be able to generate appropriate vehicle control according to the current vehicle scene like a human driver. For safety reasons, the behavior generated by an autonomous driving system is usually deterministic, such as cruising at a specific speed. However, human operating styles are usually personalized and diverse. Different drivers have different habits in the same scene. If the autonomous driving behavior is different from human habits, it will increase the driver's distrust of the autonomous driving system, which brings great difficulties to how to develop a human-friendly autonomous driving system. The vehicle must not only ensure the safety of driving behavior, but also align with the current driver's operating style to increase the driver's trust in the autonomous driving system. Therefore, from the perspective of human-computer interaction, studying human operating characteristics has gradually become a concern.
[0003] Learning-based methods provide a general approach to this type of problem. Technicians can train a human operation style classification network through some human-computer interaction data annotated with style labels. For example, for some selected driving trajectories, humans can easily judge whether the driving trajectory is aggressive, neutral, or conservative, and then train a classifier network based on this data set. However, this classification method is rough. It can only divide human behavior into a limited number of categories, and more detailed comparisons within the class are insufficient due to lack of information. If too many classes are set, it will lead to a lack of training data of various types. Therefore, the classification-based solution cannot achieve accurate alignment of human operation styles. Summary of the invention
[0004] In view of this, an embodiment of the present application proposes a method for quantitatively identifying human operating style based on preference learning, which uses preference comparison to distinguish driving trajectories in pairs to determine which driving trajectory is more aggressive, thereby forming a continuous value network to achieve precise alignment of human style.
[0005] To achieve the above-mentioned purpose, an embodiment of the present application proposes a method for quantitatively identifying a person's operating style based on preference learning, including: recording the behavioral data generated by different drivers when driving to generate a training data set; constructing prompt words containing the context of the driving task, using the GPT-4 large model to label the training data in the training data set in a preference sorting manner, and determining the preference label of each training data; inputting the training data in the training data set into a pre-constructed quantitative operating style identification model to obtain the output of the quantitative operating style identification model; wherein the quantitative operating style identification model is composed of a multi-modal multi-head encoder, a Transformer encoder, and a Transformer decoder; based on the output of the quantitative operating style identification model for each training data in a batch, and the preference label of each training data in the batch, constructing a loss function, iteratively training the quantitative operating style identification model based on the constructed loss function until convergence, and obtaining a trained quantitative operating style identification model; inputting the behavior data to be identified into the trained quantitative operating style identification model to obtain the style aggressiveness corresponding to the behavior data to be identified output by the trained quantitative operating style identification model.
[0006] To achieve the above objectives, the embodiment of the present application also proposes a personnel operation style quantitative identification system based on preference learning, including: a data acquisition module, used to record the behavior data generated by different drivers when driving, and generate a training data set; a data labeling module, used to construct prompt words containing driving task context, and use the GPT-4 large model to label the training data in the training data set in a preference sorting manner to determine the preference label of each training data; a model construction module, used to construct an operation style quantitative identification model, and the operation style quantitative identification model consists of a multi-modal multi-head encoder, a Transformer encoder and a Transformer decoder A model training module is used to input the training data in the training data set into the operation style quantitative identification model to obtain the output of the operation style quantitative identification model, and construct a loss function based on the output of the operation style quantitative identification model for each training data in a batch and the preference label of each training data in the batch, and iteratively train the operation style quantitative identification model based on the constructed loss function until convergence to obtain a trained operation style quantitative identification model; a model use module is used to input the behavior data to be identified into the trained operation style quantitative identification model to obtain the style aggressiveness corresponding to the behavior data to be identified output by the trained operation style quantitative identification model.
[0007] To achieve the above-mentioned purpose, an embodiment of the present application also proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned method for quantitatively identifying a person's operating style based on preference learning.
[0008] To achieve the above objectives, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the above-mentioned method for quantitatively identifying a person's operating style based on preference learning.
[0009] The embodiment of the present application proposes a method for quantitative identification of personnel operation style based on preference learning, selects driving tasks as representatives of human operation tasks in human-computer interaction, and studies the problem of quantitative identification of style in driving tasks. The behavioral data generated by different drivers while driving are recorded to generate a training data set, and then the GPT-4 large model is used to label the training data in the training data set with preference labels in a preference sorting manner. This labeling method greatly reduces the difficulty and cost of labeling, and improves the labeling efficiency and labeling accuracy. Next, an operation style quantitative identification model composed of a multi-modal multi-head encoder, a Transformer encoder, and a Transformer decoder is constructed and trained. The operation style quantitative identification model has good feature extraction capabilities, can fully identify fine-grained operation styles, and obtain continuous numerical style metrics. When training the operation style quantitative identification model, the idea of the Bradley-Terry preference model is used for style alignment. In this way, the trained style quantitative identification model can always identify a corresponding style value that is aligned with the operator's operation behavior regardless of the operator's current operation status.
[0010] Optionally, when recording the behavioral data generated by different drivers while driving, a metadrive simulator is used as a recording platform, and each piece of recorded behavioral data is composed of road map data, the current posture data of the own vehicle, the historical posture data of the own vehicle, the global navigation data of the own vehicle, the radar sensor data of the own vehicle, and the historical posture data of other vehicles; wherein the road map data is a bird's-eye view generated by the metadrive simulator containing road information, the position of the own vehicle, and the position of other vehicles, and is saved in an image format, and the current posture data of the own vehicle, the historical posture data of the own vehicle, the global navigation data of the own vehicle, the radar sensor data of the own vehicle, and the historical posture data of other vehicles are all saved in a vector format.
[0011] Optionally, construct prompt words containing driving task context, use the GPT-4 large model to label the training data in the training data set by preference sorting, and determine the preference label of each training data, including: construct prompt words containing driving task context and provide them to the GPT-4 large model, and set analysis reference basis for the GPT-4 large model; wherein the set analysis reference basis includes the distance between the vehicle and the vehicle in front of it in the lane, the speed of the vehicle, the acceleration of the vehicle, the behavior of the vehicle when interacting with other vehicles, and overtaking situations; randomly select two training data and input them into the GPT-4 large model, the GPT-4 large model uses preference sorting to label the two input training data, the GPT-4 large model labels the training data that it believes has a more radical operating style with an aggressive label, and labels the training data that it believes has a more conservative operating style with a conservative label.
[0012] Optionally, the multi-modal multi-head encoder of the operating style quantization identification model is composed of a map encoder, a posture encoder, a navigation encoder, a radar encoder, a self-vehicle history encoder and an other vehicle encoder; among which, the map encoder is specifically a pre-trained ResNet-18 network, and the posture encoder, navigation encoder, radar encoder, self-vehicle history encoder and other vehicle encoder are all small fully connected networks.
[0013] Optionally, the map encoder, attitude encoder, navigation encoder, radar encoder, own-vehicle history encoder and other-vehicle encoder are respectively used to encode information of road map data, current attitude data of own-vehicle, global navigation data of own-vehicle, radar sensor data of own-vehicle, historical attitude data of own-vehicle and historical attitude data of other-vehicle in training data, and input the encoded embedding vector into the Transformer encoder; the Transformer encoder is used to extract features based on each input embedding vector to generate a global feature vector; the Transformer decoder is used to query and decode the global feature vector to obtain the output of the operation style quantification identification model, and the output of the operation style quantification identification model is the style aggressiveness corresponding to the training data.
[0014] Optionally, a loss function is constructed based on the output of the operation style quantification identification model for each training data in a batch and the preference label of each training data in the batch, including: constructing a preference learning loss term based on the output of the operation style quantification identification model for training data with preference labels as radical labels and training data with preference labels as conservative labels in a batch; constructing a regularization loss term based on the maximum absolute value of the output of the operation style quantification identification model for each training data in the batch and a preset regularization coefficient; and constructing a loss function based on the preference learning loss term and the regularization loss term.
[0015] Optionally, the constructed loss function is expressed by the following formula:
[0016] L total =L PR (V)+γmax|V(x i ,y i )|
[0017]
[0018] Among them, L PR (V) represents the preference learning loss term, γmax|V(x i ,y i )| represents the regularization loss term, L total represents the constructed loss function, represents the expected function, y + represents radical label, x + represents the training data with radical labels, y - represents a conservative label, x - represents the training data with conservative labels, V(x + ,y + ) represents the output of the operation style quantitative identification model for the training data with the preference label as the aggressive label, V(x - ,y - ) represents the output of the operation style quantitative identification model for the training data with the preference label as conservative label, σ(·) represents the sigmoid function, Represents a batch, V(x i ,y i ) represents the output of the operation style quantization identification model for the i-th training data in the batch, and γ represents the regularization coefficient. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0020] Figure 1 is a flowchart of a method for quantitatively identifying a person's operating style based on preference learning provided in one embodiment of the present application;
[0021] Figure 2 is a schematic diagram of road map data provided in one embodiment of the present application;
[0022] Figure 3 It is a schematic diagram provided in an embodiment of the present application, which uses the GPT-4 large model to label the training data in the training data set in a preference sorting manner;
[0023] Figure 4 is a model structure diagram of an operation style quantitative identification model provided in one embodiment of the present application;
[0024] Figure 5 It is a change diagram of the loss curve corresponding to the training validation set and the validation validation set provided by an embodiment of the present application;
[0025] Figure 6 is a schematic diagram of online identification of an operation style quantitative identification model provided by an embodiment of the present application;
[0026] Figure 7 is a structural schematic diagram of a personnel operation style quantitative identification system based on preference learning provided in another embodiment of the present application;
[0027] Figure 8 It is a structural schematic diagram of an electronic device provided in another embodiment of the present application. DETAILED DESCRIPTION
[0028] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.
[0029] An embodiment of the present application proposes a method for quantitatively identifying a person's operating style based on preference learning, which is applied to an electronic device, wherein the electronic device may be a terminal or a server. This embodiment and the following embodiments are all described using the server as an example. The implementation details of the method for quantitatively identifying a person's operating style based on preference learning proposed in this embodiment are described in detail below. The following content is only the implementation details provided for ease of understanding and is not necessary for the implementation of this solution.
[0030] The specific process of the method for quantitatively identifying personnel operation style based on preference learning proposed in this embodiment can be as follows: Figure 1 As shown, including:
[0031] Step 101 , recording the behavior data generated by different drivers while driving to generate a training data set.
[0032] In the specific implementation, the driving task is selected as a representative of human operation tasks in human-computer interaction. To study the problem of style quantitative identification in driving tasks, it is first necessary to record the behavioral data generated by different drivers while driving and generate a training data set.
[0033] In one example, the recording of behavior data is multimodal. When recording the behavior data generated by different drivers while driving, the MetaDrive simulator is used as the recording platform. The MetaDrive simulator has the advantages of high simulation efficiency and support for multiple sensor types, and is easy to record multiple data interfaces. Each piece of recorded behavior data is composed of road map data, the current posture data of the vehicle, the historical posture data of the vehicle, the global navigation data of the vehicle, the radar sensor data of the vehicle, and the historical posture data of other vehicles.
[0034] The road map data is a bird's-eye view of the road information, the position of the vehicle, and the position of other vehicles generated by the MetaDrive simulator (such as Figure 2 As shown), the road map data is saved in image format and subsequently imported, and the current posture data of the vehicle, the historical posture data of the vehicle, the global navigation data of the vehicle, the radar sensor data of the vehicle and the historical posture data of other vehicles are all saved in vector format and subsequently imported.
[0035] In one example, while generating training data, it is also necessary to construct a verification data set based on part of the recorded behavior data for subsequent verification of the performance of the trained operation style quantitative recognition model.
[0036] Step 102, construct prompt words containing the driving task context, use the GPT-4 large model to label the training data in the training data set in a preference sorting manner, and determine the preference label of each training data.
[0037] In the specific implementation, considering that the traditional labeling method often relies on manual labeling, which increases the manual workload during labeling, in this embodiment, a prompt word containing the driving task context is constructed, and the GPT-4 large model is used to label the training data in the training data set in a preference sorting manner to determine the preference label of each training data. The GPT-4 large model is a discriminator with strong performance and certain intelligence, which can replace the task of manual labeling.
[0038] In one example, a prompt word containing the driving task context is constructed and provided to the GPT-4 large model, and an analysis reference is set for the GPT-4 large model, wherein the analysis reference includes the distance between the vehicle and the vehicle in front of it in the lane, the speed of the vehicle, the acceleration of the vehicle, the behavior of the vehicle when interacting with other vehicles, and overtaking situations. After completing the configuration of the GPT-4 large model, two training data are randomly selected and input into the GPT-4 large model. The GPT-4 large model labels the two input training data in a preference sorting manner. The GPT-4 large model labels the training data that it believes to have a more radical operating style with radical labels, and labels the training data that it believes to have a more conservative operating style with conservative labels. Although the GPT-4 large model has not yet reached the level of human intelligence, this labeling method avoids artificial scoring and only requires binary judgment, so the labeling results are credible. Figure 3 Shows an example of labeling, from Figure 3 It can be seen that the GPT-4 large model can combine multiple information to make credible judgments.
[0039] Step 103, input the training data in the training data set into a pre-built operation style quantization identification model to obtain the output of the operation style quantization identification model, wherein the operation style quantization identification model is composed of a multi-modal multi-head encoder, a Transformer encoder and a Transformer decoder.
[0040] In the specific implementation, a quantitative operation style recognition model is pre-built, which consists of a multi-modal multi-head encoder, a Transformer encoder, and a Transformer decoder. When training the model, the training data in the training data set is input into the model to obtain the output of the model.
[0041] In one example, the model structure of the operation style quantitative identification model is as follows: Figure 4 As shown in the figure, the DETR model is used as the model skeleton for improvement. Based on the DETR model, the model adds a multi-modal multi-head encoder, which is composed of a map encoder, a posture encoder, a navigation encoder, a radar encoder, a self-vehicle history encoder, and a other-vehicle encoder. Among them, the map encoder is specifically a pre-trained ResNet-18 network, and the posture encoder, navigation encoder, radar encoder, self-vehicle history encoder, and other-vehicle encoder are all small fully connected networks.
[0042] The map encoder, attitude encoder, navigation encoder, radar encoder, ego vehicle history encoder and other vehicle encoder are used to encode the road map data, the current attitude data of the ego vehicle, the global navigation data of the ego vehicle, the radar sensor data of the ego vehicle, the historical attitude data of the ego vehicle and the historical attitude data of other vehicles in the training data respectively, and input the encoded embedding vector into the Transformer encoder.
[0043] The Transformer encoder is mainly composed of an autocorrelation attention layer and a fully connected layer, which is used to extract features based on each input embedding vector and generate a global feature vector.
[0044] The Transformer decoder is mainly composed of a cross-correlation attention layer and a fully connected layer. As a generator, the Transformer decoder can produce corresponding results based on the query embedding. The global feature vector generated by the Transformer encoder is input into the Transformer decoder. The Transformer decoder combines the self-learned query embedding vector to generate a value (i.e., the output of the operation style quantization recognition model), which is the style aggressiveness corresponding to the training data.
[0045] Step 104, based on the output of the operation style quantitative identification model for each training data in a batch and the preference label of each training data in the batch, a loss function is constructed, and the operation style quantitative identification model is iteratively trained until convergence based on the constructed loss function to obtain a trained operation style quantitative identification model.
[0046] In a specific implementation, a loss function can be constructed based on the output of the operation style quantification identification model for each training data in a batch, and the preference label of each training data in the batch. The operation style quantification identification model is iteratively trained until convergence based on the constructed loss function. The converged operation style quantification identification model is performance verified using a verification data set. If the performance verification passes, a trained operation style quantification identification model is obtained.
[0047] In one example, inspired by the reward function learned through preference in the RLHF field, this embodiment refers to the Bradley-Terry preference model idea as a style alignment training scheme. For a training batch, a preference learning loss term can be constructed based on the output of the operation style quantification recognition model for the training data with a preference label of an aggressive label and the output of the training data with a preference label of a conservative label in a batch. A regularization loss term is constructed based on the maximum absolute value of the output of the operation style quantification recognition model for each training data in the batch and a preset regularization coefficient. Finally, a loss function is constructed based on the preference learning loss term and the regularization loss term.
[0048] In an example, the constructed loss function can be expressed by the following formula:
[0049] L total =L PR (V)+γmax|V(x i ,y i )|
[0050]
[0051] Among them, L PR (V) represents the preference learning loss term, γmax|V(x i ,y i )| represents the regularization loss term, L total represents the constructed loss function, represents the expected function, y + represents radical label, x + represents the training data with radical labels, y - represents a conservative label, x - represents the training data with conservative labels, V(x + ,y + ) represents the output of the operation style quantitative identification model for the training data with the preference label as the aggressive label, V(x - ,y - ) represents the output of the operation style quantitative identification model for the training data with the preference label as conservative label, σ(·) represents the sigmoid function, Represents a batch, V(x i ,y i ) represents the output of the operation style quantization identification model for the i-th training data in the batch, and γ represents the regularization coefficient.
[0052] The regularization loss term encourages V(x i ,y i) will not be too large until a balance is reached with the preference alignment task. The trained model can obtain online and good style value alignment. Regardless of the operator's current operation, the model can always identify a corresponding style value and behavior alignment. After training, the loss curves corresponding to the training validation set and the validation validation set are as follows Figure 5 The verification results are shown as Figure 6 shown.
[0053] Step 105 , input the behavior data to be identified into the trained operation style quantitative identification model, and obtain the style aggressiveness corresponding to the behavior data to be identified output by the trained operation style quantitative identification model.
[0054] In the specific implementation, after obtaining the trained operation style quantitative identification model, it can be deployed in the application scenario where it is needed. When using the model, the behavior data to be identified is input into the model to obtain the style aggressiveness corresponding to the behavior data to be identified output by the model, thereby achieving style alignment.
[0055] In this embodiment, driving tasks are selected as representatives of human operation tasks in human-computer interaction to study the problem of style quantification and identification in driving tasks. The behavioral data generated by different drivers while driving are recorded to generate a training data set, and then the GPT-4 large model is used to label the training data in the training data set with preference labels in a preference sorting manner. This labeling method greatly reduces the difficulty and cost of labeling, and improves the labeling efficiency and labeling accuracy. Next, an operation style quantification and identification model consisting of a multi-modal multi-head encoder, a Transformer encoder, and a Transformer decoder is constructed and trained. The operation style quantification and identification model has good feature extraction capabilities, can fully identify fine-grained operation styles, and obtain continuous numerical style metrics. When training the operation style quantification and identification model, the idea of the Bradley-Terry preference model is used for style alignment. In this way, the trained style quantification and identification model can always identify a corresponding style value that is aligned with the operator's operation behavior regardless of the operator's current operation status.
[0056] The step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this application.
[0057] Another embodiment of the present application proposes a personnel operation style quantitative identification system based on preference learning. The following is a specific description of the details of the personnel operation style quantitative identification system based on preference learning proposed in this embodiment. The following content is only the implementation details provided for the convenience of understanding and is not necessary for the implementation of this example. Figure 6 It is a schematic diagram of a system for quantitatively identifying operating styles of personnel based on preference learning proposed in this embodiment, including: a data collection module 201, a data annotation module 202, a model construction module 203, a model training module 204 and a model use module 205.
[0058] The data collection module 201 is used to record the behavior data generated by different drivers while driving and generate a training data set.
[0059] The data labeling module 202 is used to construct prompt words containing the driving task context, and use the GPT-4 large model to label the training data in the training data set in a preference sorting manner to determine the preference label of each training data.
[0060] The model building module 203 is used to build an operation style quantitative identification model, which is composed of a multi-modal multi-head encoder, a Transformer encoder and a Transformer decoder.
[0061] The model training module 204 is used to input the training data in the training data set into the operation style quantitative identification model to obtain the output of the operation style quantitative identification model, construct a loss function based on the output of the operation style quantitative identification model for each training data in a batch, and the preference label of each training data in the batch, and iteratively train the operation style quantitative identification model based on the constructed loss function until convergence, so as to obtain a trained operation style quantitative identification model.
[0062] The model using module 205 is used to input the behavior data to be identified into the trained operation style quantitative identification model to obtain the style aggressiveness corresponding to the behavior data to be identified output by the trained operation style quantitative identification model.
[0063] It is not difficult to find that this embodiment is a system embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details and technical effects mentioned in the above embodiments are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiments.
[0064] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.
[0065] Another embodiment of the present application provides an electronic device, whose specific structure can be as follows: Figure 7 As shown, it includes: at least one processor 301; and a memory 302 that is communicatively connected to the at least one processor 301; wherein the memory 302 stores instructions that can be executed by the at least one processor 301, and the instructions are executed by the at least one processor 301 so that the at least one processor 301 can execute the method for quantitatively identifying personnel operating style based on preference learning described in the above-mentioned method embodiments.
[0066] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus is used to connect various circuits of one or more processors and memories together. The bus can also connect other circuits such as peripherals, voltage regulators, and power management circuits, which are all well known in the art, so this article will not describe them further. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium through an antenna, and further, the antenna also receives data and transmits the data to the processor.
[0067] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.
[0068] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the method for quantitatively identifying a person's operating style based on preference learning described in the above method embodiment.
[0069] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM (Read-Only Memory), RAM (Random Access Memory), disk or optical disk and other media that can store program codes.
[0070] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.
Claims
1. A method for quantitatively identifying personnel operation styles based on preference learning, characterized in that: include: Record the behavior data generated by different drivers while driving to generate a training data set; Construct prompt words containing driving task context, use the GPT-4 large model to label the training data in the training data set by preference sorting, and determine the preference label of each training data; Input the training data in the training data set into the pre-built operation style quantization identification model to obtain the output of the operation style quantization identification model; wherein the operation style quantization identification model is composed of a multi-modal multi-head encoder, a Transformer encoder and a Transformer decoder; Based on the output of the operation style quantitative identification model for each training data in a batch and the preference label of each training data in the batch, a loss function is constructed, and the operation style quantitative identification model is iteratively trained until convergence based on the constructed loss function to obtain a trained operation style quantitative identification model; The behavior data to be identified is input into the trained operation style quantitative identification model to obtain the style aggressiveness corresponding to the behavior data to be identified output by the trained operation style quantitative identification model.
2. The method for quantitatively identifying personnel operation styles based on preference learning according to claim 1, characterized in that: When recording the behavior data generated by different drivers while driving, the MetaDrive simulator is used as the recording platform. Each piece of recorded behavior data is composed of road map data, the current posture data of the own vehicle, the historical posture data of the own vehicle, the global navigation data of the own vehicle, the radar sensor data of the own vehicle, and the historical posture data of other vehicles. Among them, the road map data is a bird's-eye view generated by the metadrive simulator that contains road information, the position of the own vehicle, and the position of other vehicles. It is saved in the image format. The current posture data of the own vehicle, the historical posture data of the own vehicle, the global navigation data of the own vehicle, the radar sensor data of the own vehicle, and the historical posture data of other vehicles are all saved in the vector format.
3. The method for quantitatively identifying personnel operation styles based on preference learning according to claim 2 is characterized in that: Construct prompt words containing driving task context, use the GPT-4 large model to label the training data in the training data set in a preference sorting manner, and determine the preference label of each training data, including: Construct prompt words containing driving task context and provide them to the GPT-4 large model, and set analysis references for the GPT-4 large model; the analysis references set include the distance between the vehicle and the vehicle in front of it in the lane, the speed of the vehicle, the acceleration of the vehicle, the behavior of the vehicle when interacting with other vehicles, and the overtaking situation; Two training data are randomly selected and input into the GPT-4 large model. The GPT-4 large model labels the two input training data using a preference sorting method. The GPT-4 large model labels the training data that it believes has a more radical operating style with radical labels, and labels the training data that it believes has a more conservative operating style with conservative labels.
4. The method for quantitatively identifying personnel operation styles based on preference learning according to claim 3 is characterized in that: The multi-modal multi-head encoder of the operation style quantization identification model consists of a map encoder, a posture encoder, a navigation encoder, a radar encoder, a self-vehicle history encoder, and an other-vehicle encoder. Among them, the map encoder is specifically a pre-trained ResNet-18 network, and the attitude encoder, navigation encoder, radar encoder, self-vehicle history encoder and other vehicle encoder are all small fully connected networks.
5. The method for quantitatively identifying personnel operation style based on preference learning according to claim 4 is characterized in that: The map encoder, attitude encoder, navigation encoder, radar encoder, ego vehicle history encoder and other vehicle encoder are used to encode the road map data, the current attitude data of the ego vehicle, the global navigation data of the ego vehicle, the radar sensor data of the ego vehicle, the historical attitude data of the ego vehicle and the historical attitude data of other vehicles in the training data, respectively, and input the encoded embedding vector into the Transformer encoder; The Transformer encoder is used to extract features based on each input embedding vector and generate a global feature vector; The Transformer decoder is used to query and decode the global feature vector to obtain the output of the operation style quantitative identification model. The output of the operation style quantitative identification model is the style radicalness corresponding to the training data.
6. The method for quantitatively identifying personnel operation style based on preference learning according to claim 5, characterized in that: Based on the output of the operation style quantization recognition model for each training data in a batch and the preference label of each training data in the batch, a loss function is constructed, including: Based on the output of the operation style quantitative identification model for the training data with the preference label of the radical label and the training data with the preference label of the conservative label in a batch, a preference learning loss term is constructed; Constructing a regularization loss term based on the maximum absolute value of the output of the operation style quantization recognition model for each training data in the batch and a preset regularization coefficient; Based on the preference learning loss term and the regularization loss term, a loss function is constructed.
7. The method for quantitatively identifying personnel operation style based on preference learning according to claim 6, characterized in that: The constructed loss function is expressed by the following formula: L total =L PR (V)+γmax|V(x i ,y i )| Among them, L PR (V) represents the preference learning loss term, γmax|V(x i ,y i )| represents the regularization loss term, L total represents the constructed loss function, represents the expected function, y + represents radical label, x + represents the training data with radical labels, y - represents a conservative label, x - represents the training data with conservative labels, V(x + ,y + ) represents the output of the operation style quantitative identification model for the training data with the preference label as the aggressive label, V(x - ,y - ) represents the output of the operation style quantitative identification model for the training data with the preference label as conservative label, σ(·) represents the sigmoid function, D represents a batch, V(x i ,y i ) represents the output of the operation style quantization identification model for the i-th training data in the batch, and γ represents the regularization coefficient.
8. A quantitative identification system for personnel operation style based on preference learning, characterized in that: include: The data collection module is used to record the behavior data generated by different drivers while driving and generate a training data set; The data labeling module is used to construct prompt words containing the driving task context. The GPT-4 large model is used to label the training data in the training data set in a preference sorting manner to determine the preference label of each training data. The model building module is used to build a quantitative operation style identification model. The quantitative operation style identification model consists of a multi-modal multi-head encoder, a Transformer encoder, and a Transformer decoder. A model training module is used to input the training data in the training data set into the operation style quantitative identification model to obtain the output of the operation style quantitative identification model, construct a loss function based on the output of the operation style quantitative identification model for each training data in a batch and the preference label of each training data in the batch, and iteratively train the operation style quantitative identification model based on the constructed loss function until convergence, so as to obtain a trained operation style quantitative identification model; The model using module is used to input the behavior data to be identified into the trained operation style quantitative identification model to obtain the style aggressiveness corresponding to the behavior data to be identified output by the trained operation style quantitative identification model.
9. An electronic device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for quantitatively identifying a person's operating style based on preference learning as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for quantitatively identifying a person's operating style based on preference learning as claimed in any one of claims 1 to 7 can be implemented.