User behavior prediction method and device, electronic equipment and storage medium

Abstract

The invention provides a user behavior prediction method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining use data of a user in a vehicle, environmental data of an environment where the vehicle is located and attribute data of the vehicle and / or the user; inputting the use data into a pre-trained long short-term memory network for user behavior feature extraction to obtain user behavior features; inputting the attribute data into a pre-trained full-connection neural network for auxiliary feature extraction to obtain auxiliary features; inputting the environmental data into a pre-trained convolutional neural network for context feature extraction to obtain environmental factor features; determining target features according to the user behavior features, the auxiliary features and the environmental factor features; and inputting the target features into a pre-trained behavior prediction model for behavior prediction to obtain a behavior prediction result of the user on the vehicle. According to the invention, the accuracy of user behavior prediction can be improved.

Description

Technical Field

[0001] This application relates to the technical field of automobiles, and more specifically, to a method, apparatus, electronic device, and storage medium for predicting user behavior. Background Art

[0002] With the rapid development of technology, intelligent vehicles are used more and more frequently in people's daily lives. In order to ensure the user experience in intelligent vehicles, the vehicle can be controlled by predicting the user's behavior, that is, without the user's manual control operation. However, the current methods for predicting user behavior have limitations in processing time series data and long-term dependency relationships, resulting in the inability to ensure the accuracy of model prediction. Therefore, how to improve the accuracy of predicting user behavior has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, embodiments of the present application propose a method, apparatus, electronic device, and storage medium for predicting user behavior to improve the above problems.

[0004] According to the first aspect of the embodiments of the present application, a method for predicting user behavior is provided. The method includes: obtaining usage data of a user in a vehicle, environmental data of the environment where the vehicle is located, and attribute data of the vehicle and / or the user; inputting the usage data into a pre-trained long short-term memory network for user behavior feature extraction to obtain user behavior features; inputting the attribute data into a pre-trained fully connected neural network for auxiliary feature extraction to obtain auxiliary features; inputting the environmental data into a pre-trained convolutional neural network for context feature extraction to obtain environmental factor features; determining target features according to the user behavior features, the auxiliary features, and the environmental factor features; and inputting the target features into a pre-trained behavior prediction model for behavior prediction to obtain a behavior prediction result of the user on the vehicle.

[0005] According to a second aspect of the embodiments of the present application, there is provided a device for predicting user behavior, the device comprising: a data acquisition module, configured to acquire usage data of a user in a vehicle, environmental data of the environment where the vehicle is located, and attribute data of the vehicle and / or the user; a user behavior feature determination module, configured to input the usage data into a pre-trained long short-term memory network for user behavior feature extraction to obtain user behavior features; an auxiliary feature determination module, configured to input the attribute data into a pre-trained fully connected neural network for auxiliary feature extraction to obtain auxiliary features; an environmental factor feature determination module, configured to input the environmental data into a pre-trained convolutional neural network for context feature extraction to obtain environmental factor features; a target feature determination module, configured to determine target features according to the user behavior features, the auxiliary features, and the environmental factor features; and a behavior prediction result determination module, configured to input the target features into a pre-trained behavior prediction model for behavior prediction to obtain a behavior prediction result of the user on the vehicle.

[0006] According to a third aspect of the embodiments of the present application, there is provided an electronic device, comprising: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the above-mentioned method for predicting user behavior is implemented.

[0007] According to a fourth aspect of the embodiments of the present application, there is provided a computer-readable storage medium, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the above-mentioned method for predicting user behavior is implemented.

[0008] In the solution of the present application, first, a pre-trained long short-term memory network is used to extract user behavior features from the acquired usage data of the user in the vehicle to obtain user behavior features; a pre-trained fully connected neural network is used to extract auxiliary features from the attribute data of the vehicle and / or the user to obtain auxiliary features, and a pre-trained convolutional neural network is used to extract context features from the environmental data to obtain environmental factor features. Then, target features are determined according to the user behavior features, the auxiliary features, and the environmental factor features. Finally, a pre-trained behavior prediction model is used to perform behavior prediction on the target features to obtain a behavior prediction result of the user on the vehicle. By separately processing different types of data in the embodiments of the present application, the data information of each can be more deeply mined, while avoiding interference between data. And by determining target features based on user behavior features, auxiliary features, and environmental factor features, user behavior, vehicle state, and environmental factors can be comprehensively captured, thereby generating a richer feature representation, improving the prediction accuracy, and enabling the model to provide a clearer explanation to understand the driving factors behind user behavior, which is convenient for subsequent optimization and improvement.

[0009] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0011] Figure 1 is a schematic flowchart of a method for predicting user behavior shown according to an embodiment of the present application.

[0012] Figure 2 is a schematic flowchart of a method for predicting user behavior shown according to another embodiment of the present application.

[0013] Figure 3 is a schematic flowchart of the specific steps of step 250 shown according to an embodiment of the present application.

[0014] Figure 4 is a schematic flowchart of a method for predicting user behavior shown according to still another embodiment of the present application.

[0015] Figure 5 is a schematic flowchart of a method for predicting user behavior shown according to yet another embodiment of the present application.

[0016] Figure 6 is a schematic flowchart of a method for predicting user behavior shown according to yet another embodiment of the present application.

[0017] Figure 7 is a schematic flowchart of a method for predicting user behavior shown according to an embodiment of the present application.

[0018] Figure 8 is a block diagram of a device for predicting user behavior shown according to an embodiment of the present application.

[0019] Figure 9 is a hardware structure diagram of an electronic device shown according to an embodiment of the present application.

[0020] Through the above drawings, specific embodiments of the present invention have been shown, and there will be more detailed descriptions hereinafter. These drawings and written descriptions are not intended to limit the scope of the inventive concept in any way, but to illustrate the concept of the present invention to those skilled in the art through specific embodiments. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0023] In addition, the described features, structures or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to give a full understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid obscuring aspects of the present application.

[0024] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices. The flowcharts shown in the drawings are only exemplary illustrations and do not necessarily include all the content and operations / steps, nor do they have to be executed in the order described. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0025] Please refer to Figure 1 , Figure 1 which shows a method for predicting user behavior provided by an embodiment of the present application. In a specific embodiment, the method for predicting user behavior can be applied to a user behavior prediction device 500 as shown in Figure 8 and an electronic device 600 configured with the user behavior prediction device 500 ( Figure 9)。The following will describe the specific process of this embodiment. Of course, it can be understood that this method can be executed by an in-vehicle terminal with computing and processing capabilities. The following will elaborate in detail on the Figure 1 process shown below. The user behavior prediction method may specifically include the following steps:

[0026] Step 110, obtain the usage data of the user in the vehicle, the environmental data of the environment where the vehicle is located, and the attribute data of the vehicle and / or the user.

[0027] As a way, when the vehicle is powered on, the usage data of the vehicle can be obtained in the vehicle's electronic processor (CPU). Among them, the usage data is the data generated by the user's immediate operation or other operations in the vehicle. For example, when the user operates the music player in the vehicle's in-vehicle entertainment system to play audio, corresponding usage data of the in-vehicle entertainment system will be generated. This usage data can intuitively represent the user's specific operations in the in-vehicle entertainment system, so as to facilitate identifying the usage data through a long short-term memory network to determine the user behavior characteristics of the user.

[0028] Optionally, the environmental data of the environment where the vehicle is located may include weather data, light intensity, location information corresponding to the current environment where the vehicle is located, current time, and other data of the current environment where the vehicle is located. Optionally, the weather data of the current environment where the vehicle is located can be obtained through the vehicle's in-vehicle server or a cloud server or electronic device communicatively connected to the vehicle, or the light intensity corresponding to the current environment where the vehicle is located can be determined through the vehicle's photosensitive sensor, or the current time corresponding to the current environment where the vehicle is located can be obtained through the vehicle's in-vehicle server, and the location information of the current environment where the vehicle is located can be determined through the vehicle's Global Positioning System (GPS).

[0029] Optionally, the currently logged-in user account can be determined first in the vehicle's in-vehicle server, and based on the user account, the attribute data of the user corresponding to the user account and / or the attribute data of the vehicle can be obtained in the in-vehicle server or a cloud server associated with the user account. Among them, the attribute data of the user may include preference information set by the user himself (for example, the volume size of the in-vehicle entertainment system, etc.), and the attribute data of the vehicle includes the brightness of the vehicle's headlights, the position and height of the vehicle's seats, etc.

[0030] Step 120, input the usage data into a pre-trained long short-term memory network for user behavior feature extraction to obtain user behavior features.

[0031] As a way, the user usage features of the input usage data can be extracted through a long short-term memory network. In the long short-term memory network, the maximum value of the input sequence of the input data can be preset in advance, and the input data of the long short-term memory network can be screened by the maximum value of the input sequence, so as to avoid the reduction of the feature extraction efficiency of the long short-term memory network caused by too much input data.

[0032] Optionally, in order to avoid overfitting in the process of feature extraction by the long short-term memory network, unit data can be preset in the long short-term memory network to determine the corresponding network complexity of the long short-term memory network, so as to affect the overfitting and performance of the long short-term memory network. Optionally, the hidden state dimension (such as 64, 128, etc.) corresponding to the long short-term memory network can also be preset in advance. Among them, the hidden state is the internal state used by the long short-term memory network to store and transmit information when processing the input sequence, which is updated at each time step to capture the information and context relationship in the input sequence. The dimension of the features obtained by feature extraction can be set by setting the hidden state dimension.

[0033] As a way, the long short-term memory network to be trained can be pre-trained in advance to obtain a pre-trained long short-term memory network for extracting user behavior features from usage data. Optionally, the historical usage data of the vehicle can be obtained, and then the historical usage data can be input into the long short-term memory network to be trained, so that the historical usage data is used as sample data to train the long short-term memory network to be trained, so as to obtain a pre-trained long short-term memory network for offline application. Optionally, the historical usage data may include the first feature identifier corresponding to the user behavior feature. In this way, the loss function of the long short-term memory network can be determined by comparing the user behavior features obtained by feature extraction of the historical usage data by the long short-term memory network to be trained with the user behavior features indicated by the first feature identifier corresponding to the historical usage data. Based on the loss function, the parameters of the long short-term memory network can be adjusted when it does not converge, and the long short-term memory network after parameter adjustment continues to extract features from the input historical usage data until the loss function determined by the first feature identifier and the user behavior features extracted by the long short-term memory network after parameter adjustment converges, and the training of the long short-term memory network ends to obtain a pre-trained long short-term memory network.

[0034] Step 130, input the attribute data into the pre-trained fully connected neural network for auxiliary feature extraction to obtain auxiliary features.

[0035] As a way, the auxiliary feature extraction can be performed on the attribute data through a fully-connected neural network to obtain auxiliary features that have a certain impact on the prediction of user behavior. Among them, the features corresponding to the data in the attribute data that affect the prediction of user behavior indicated by the auxiliary features. For example, different heights of users in the attribute data will cause user behaviors such as adjusting the height of the seat.

[0036] Optionally, in order to ensure the effectiveness of the features obtained by the fully-connected neural network for auxiliary feature extraction, the number of units in the hidden layer corresponding to the fully-connected neural network can be preset to ensure the complexity of the fully-connected neural network. Optionally, in order to ensure the accuracy of the fully-connected neural network for auxiliary feature extraction of attribute data, the activation function of the fully-connected neural network can be preset, so as to map the input to the output end through a suitable activation function, thereby ensuring the accuracy of the auxiliary features. Among them, the activation function can be ReLU, Sigmoid, or other activation functions, which can be set according to actual needs.

[0037] As a way, the fully-connected neural network to be trained can be pre-trained to obtain a pre-trained fully-connected neural network for auxiliary feature extraction of attribute data. Optionally, the historical attribute data of the vehicle or user can be obtained, and then the historical attribute data is input into the fully-connected neural network to be trained, so as to use the historical attribute data as sample data to train the fully-connected neural network to be trained, so as to obtain a pre-trained fully-connected neural network for offline application. Optionally, the historical attribute data may include a second feature identifier corresponding to the auxiliary feature, so that the loss function of the long short-term memory network can be determined by the auxiliary feature obtained by the fully-connected neural network for feature extraction of the historical attribute data and the auxiliary feature indicated by the second feature identifier corresponding to the historical attribute data. Based on this, when the loss function does not converge, the parameters of the fully-connected neural network can be adjusted, and the input historical attribute data is continuously subjected to auxiliary feature extraction according to the fully-connected neural network after parameter adjustment until the loss function determined according to the second feature identifier and the auxiliary feature extracted by the fully-connected neural network after parameter adjustment converges, and the training of the fully-connected neural network is ended to obtain a pre-trained fully-connected neural network.

[0038] Step 140, input the environmental data into the pre-trained convolutional neural network for context feature extraction to obtain environmental factor features.

[0039] As a way, the context features of the environmental data can be extracted through a convolutional neural network to analyze the environmental data and determine the environmental factor features that affect the user's behavior in the environmental data. Among them, the context features can be the features corresponding to the data that affect the behavior prediction of the user in the environmental data. For example, since it is raining in the environmental data, it affects the user to turn on the windshield wiper. Therefore, the features corresponding to the rainy weather can be determined as the environmental factor features.

[0040] Optionally, in order to ensure the accuracy of the context feature extraction of the convolutional neural network for the environmental data, the number of convolutional layers of the convolutional neural network can be preset, and the number of convolutional layers of the convolutional neural network can be set according to actual needs to ensure the depth and feature extraction ability of the convolutional neural network. Optionally, the type of pooling layer of the convolutional neural network can also be preset, and the pooling layer type can be max pooling or average pooling to determine the dimension of the extracted features.

[0041] As a way, the convolutional neural network to be trained can be pre-trained to obtain a pre-trained convolutional neural network to extract the environmental factor features of the environmental data. Optionally, the historical environmental data of the vehicle can be obtained, and then the historical environmental data is input into the convolutional neural network to be trained. In this way, the historical environmental data is used as sample data to train the convolutional neural network to be trained, so as to obtain a pre-trained convolutional neural network for offline application. Optionally, the historical environmental data can include the third feature identifier corresponding to the environmental factor features. In this way, the loss function of the convolutional neural network can be determined by comparing the environmental factor features obtained by the convolutional neural network through feature extraction of the historical environmental data with the environmental factor features indicated by the third feature identifier corresponding to the historical environmental data. Based on this, the parameters of the convolutional neural network can be adjusted when the loss function does not converge, and the convolutional neural network after parameter adjustment continues to extract features from the input historical environmental data until the loss function determined by the environmental factor features extracted according to the third feature identifier and the convolutional neural network after parameter adjustment converges, and the training of the convolutional neural network ends to obtain a pre-trained convolutional neural network.

[0042] Step 150, determine the target feature according to the user behavior feature, the auxiliary feature and the environmental factor feature.

[0043] As a way, after determining the user behavior feature, the auxiliary feature and the environmental factor feature, in order to enable the pre-trained behavior prediction model to predict the user's behavior based on the user behavior feature, the auxiliary feature and the environmental factor feature, feature fusion and feature interaction can be performed through the user behavior feature, the auxiliary feature and the environmental factor feature to obtain the target feature.

[0044] Optionally, in the process of feature fusion and feature interaction of user behavior features, auxiliary features, and environmental factor features, non-linear relationships between different features can be captured, the information integration efficiency between different features can be improved, so as to ensure that the pre-trained behavior prediction model can comprehensively consider multi-dimensional factors and thus improve the overall prediction ability.

[0045] Optionally, it can be to first perform feature interaction calculation on user behavior features, auxiliary features, and environmental factor features to obtain interaction features, and then fuse the interaction features with user behavior features, auxiliary features, and environmental factor features respectively to obtain target features. Optionally, the interaction features can be calculated by the method of Factorization Machines (FM), or can be calculated by other methods. Optionally, the target feature can be a feature set, and the feature set can include original features (user behavior features, auxiliary features, and environmental factor features), interaction features, and fusion features.

[0046] Step 160, input the target feature into the pre-trained behavior prediction model for behavior prediction to obtain the behavior prediction result of the user on the vehicle.

[0047] As a way, after obtaining the target feature, the pre-trained behavior prediction model can be used to predict the user's behavior based on the respective features of the actually collected data to obtain the behavior prediction result of the user on the vehicle, so as to control the vehicle based on the behavior prediction result. Optionally, the behavior prediction result can indicate the operation that the user will perform on the vehicle next. Among them, the behavior prediction model can be a model composed of at least one neural network, and the specific type of neural network can be set according to actual needs.

[0048] In an embodiment of the present application, first, the pre-trained long short-term memory network is used to extract user behavior features from the acquired usage data of the user in the vehicle, obtaining user behavior features; the pre-trained fully connected neural network is used to extract auxiliary features from the attribute data of the vehicle and / or the user, obtaining auxiliary features; and the pre-trained convolutional neural network is used to extract context features from the environmental data, obtaining environmental factor features. Then, the target features are determined based on the user behavior features, the auxiliary features, and the environmental factor features. Finally, the pre-trained behavior prediction model is used to perform behavior prediction on the target features, obtaining the behavior prediction result of the user in the vehicle. By separately processing different types of data in the embodiment of the present application, the data information of each can be more deeply mined, while avoiding interference between data. Moreover, determining the target features based on the user behavior features, the auxiliary features, and the environmental factor features can comprehensively capture the user behavior, the vehicle state, and the environmental factors, thereby generating a richer feature representation, improving the prediction accuracy, and enabling the model to provide a clearer explanation to understand the driving factors behind the user behavior, facilitating subsequent optimization and improvement.

[0049] Please refer to Figure 2 , Figure 2 which shows a method for predicting user behavior provided by an embodiment of the present application. The following will elaborate in detail on the Figure 2 process shown below. The method for predicting user behavior may specifically include the following steps:

[0050] Step 210, obtain the usage data of the user in the vehicle, the environmental data of the environment where the vehicle is located, and the attribute data of the vehicle and / or the user.

[0051] Step 220, input the usage data into the pre-trained long short-term memory network for user behavior feature extraction, obtaining user behavior features.

[0052] Step 230, input the attribute data into the pre-trained fully connected neural network for auxiliary feature extraction, obtaining auxiliary features.

[0053] Step 240, input the environmental data into the pre-trained convolutional neural network for context feature extraction, obtaining environmental factor features.

[0054] Among them, the specific step descriptions of steps 210 - 240 can refer to steps 110 - 140, and will not be elaborated here.

[0055] Step 250, perform feature interaction calculation on the user behavior features, the auxiliary features, and the environmental factor features, obtaining interaction features.

[0056] As a way, feature interaction refers to the relationship or combined effect between different features in a dataset. It involves how features interact with each other to affect the results of a machine learning model. That is, in this embodiment, feature interaction can be performed on user behavior features, auxiliary features, and environmental factor features, so that when predicting user behavior using the pre-trained user behavior prediction model later, user behavior prediction can be based on the features after feature interaction that influence each other, improving the accuracy of the behavior prediction result.

[0057] Optionally, feature interaction calculation can be performed based on user behavior features, auxiliary features, and environmental factor features through simple logical operations. For example, the feature vectors corresponding to the feature interaction of user behavior features, auxiliary features, and environmental factor features are multiplied pairwise to obtain interaction features. Optionally, it can also be by combining user behavior features, auxiliary features, and environmental factor features into a feature matrix, and then decomposing the feature matrix to capture the second-order interaction effect between features, thus solving the problem of being unable to effectively process high-dimensional sparse data.

[0058] In some embodiments, as Figure 3 shown, step 250 includes:

[0059] Step 251, determining the product features between every two of the user behavior features, the auxiliary features, and the environmental factor features.

[0060] As a way, the product features can be obtained by multiplying every two of the user behavior features, auxiliary features, and environmental factor features. As shown in the formula: y1 = w 1 *w 2 +w 2 *w 3 +…+w n―1 *w n , where n is the number of features and w are different features.

[0061] Step 252, determining the power-added features corresponding to the user behavior features, the auxiliary features, and the environmental factor features respectively, where the power-added features indicate the features obtained after a feature multiplies itself.

[0062] As a way, to ensure the richness of interaction features, the power-added features obtained after each feature multiplies itself can also be determined. As shown in the formula: y2 = w 1 *w 1 +w 2 *w 2 +…+w n *w n .

[0063] Step 253: Obtain the interaction feature based on the product feature and the power addition feature.

[0064] As a way, after obtaining the product feature and the power addition feature, the matrix obtained by adding the power addition feature and the product feature can be used as the interaction feature.

[0065] Please continue to refer to Figure 2 , Step 260: Perform feature fusion on the interaction feature, the user behavior feature, the auxiliary feature, and the environmental factor feature to obtain the target feature.

[0066] As a way, to ensure the richness of the obtained target feature, the interaction feature can be respectively fused with the original features (i.e., the user behavior feature, the auxiliary feature, and the environmental factor feature), so that the target feature is a rich feature set, and the corresponding features in this feature set can effectively represent the influence of different features on the prediction result.

[0067] Step 270: Input the target feature into the pre-trained behavior prediction model for behavior prediction to obtain the behavior prediction result of the user in the vehicle.

[0068] In this embodiment, by performing feature interaction calculation on the user behavior feature, the auxiliary feature, and the environmental factor feature to obtain the interaction feature, and then fusing the interaction feature with the user behavior feature, the auxiliary feature, and the environmental factor feature to obtain the target feature, the relationship between features can be captured and the prediction accuracy can be improved.

[0069] Please refer to Figure 4 , Figure 4 shows a prediction method for user behavior provided by an embodiment of the present application. The following will elaborate in detail on the Figure 4 shown process. The prediction method for user behavior may specifically include the following steps:

[0070] Step 310: Obtain the usage data of the user in the vehicle, the environmental data of the environment where the vehicle is located, and the attribute data of the vehicle and / or the user.

[0071] Step 320: Input the usage data into the pre-trained long short-term memory network for user behavior feature extraction to obtain the user behavior feature.

[0072] Step 330: Input the attribute data into the pre-trained fully connected neural network for auxiliary feature extraction to obtain the auxiliary feature.

[0073] Step 340: Input the environmental data into the pre-trained convolutional neural network for context feature extraction to obtain the environmental factor feature.

[0074] Step 350: Determine the target feature according to the user behavior feature, the auxiliary feature, and the environmental factor feature.

[0075] Step 360: Input the target feature into a pre-trained behavior prediction model for behavior prediction to obtain the behavior prediction result of the user on the vehicle.

[0076] Among them, for the specific step descriptions of steps 310 - 360, please refer to steps 110 - 160, which will not be elaborated here.

[0077] Step 370: Determine the corresponding target application in the vehicle indicated by the behavior prediction result, and generate a control instruction according to the behavior prediction result.

[0078] As a way, since the behavior prediction result indicates the operation that the user is about to perform on the vehicle, in order to improve the user experience, the target application that the user is about to operate on the vehicle indicated in the behavior prediction result can be determined first. For example, if the behavior prediction result indicates that the user is about to adjust the brightness of the vehicle's headlights, then determine that the corresponding target application in the vehicle is the headlight brightness adjustment application.

[0079] Optionally, after determining the target application, generate a control instruction for controlling the target application based on the behavior prediction result, where the control instruction can reflect the expectation of the user to operate the target application in the behavior prediction result, that is, the function that the user wants to achieve by operating the target application. For example, when the target application is the headlight brightness adjustment application, a control instruction for indicating the headlight brightness adjustment can be generated based on the behavior prediction result, and the control instruction can be an instruction including the brightness value of the headlights.

[0080] Step 380: Control the target application based on the control instruction.

[0081] As a way, after generating the control instruction, the control instruction can be sent to the vehicle's in-vehicle processor, and the in-vehicle processor can execute the corresponding control instruction on the target application according to the control instruction, so as to avoid traffic accidents caused by the user being distracted while operating the target application during driving, thereby ensuring driving safety and improving the user experience.

[0082] In some embodiments, after step 380, the method further includes: obtaining the control data of the vehicle, and determining the control result of the vehicle according to the control data; adjusting the parameters of the pre-trained behavior prediction model according to the control result.

[0083] As a way to ensure that the behavior prediction model can be adapted to different users or on different vehicles, after each time the vehicle is controlled based on the behavior prediction result, the control result of the vehicle can be determined through the obtained control data of the vehicle, so as to adjust the parameters of the pre-trained behavior prediction model according to the control result, and re-train based on the control result and the user data corresponding to the control result, and update the behavior prediction model based on the training result, thereby ensuring the adaptability and accuracy of the behavior prediction model. Optionally, the behavior prediction model can be updated by adjusting the learning rate (such as the decay strategy) of the behavior prediction model.

[0084] In this embodiment, after determining the behavior prediction result of the user in the vehicle, the target application indicated by the behavior prediction result can be determined and a control instruction can be generated, so as to control the target application based on the control instruction, thereby improving the user experience.

[0085] Please refer to Figure 5 , Figure 5 which shows the prediction method of user behavior provided by an embodiment of the present application. The following will elaborate in detail on the Figure 5 process shown. The prediction method of user behavior may specifically include the following steps:

[0086] Step 410, obtain the usage data of the user in the vehicle, the environmental data of the environment where the vehicle is located, and the attribute data of the vehicle and / or the user.

[0087] Step 420, input the usage data into the pre-trained long short-term memory network for user behavior feature extraction to obtain user behavior features.

[0088] Step 430, input the attribute data into the pre-trained fully connected neural network for auxiliary feature extraction to obtain auxiliary features.

[0089] Step 440, input the environmental data into the pre-trained convolutional neural network for context feature extraction to obtain environmental factor features.

[0090] Step 450, determine the target features according to the user behavior features, the auxiliary features, and the environmental factor features.

[0091] Step 460, input the target features into the pre-trained behavior prediction model for behavior prediction to obtain the behavior prediction result of the user in the vehicle.

[0092] Among them, the specific step descriptions of steps 410 - 460 can refer to steps 110 - 160, and will not be elaborated here.

[0093] Step 470: Determine the target application corresponding to the vehicle indicated by the behavior prediction result, and generate a prompt message according to the behavior prediction result, where the prompt message is used to prompt the startup or shutdown of the target application.

[0094] As a way, in order to improve the user experience, before controlling the target application on the vehicle through the behavior prediction result, the target application corresponding to the vehicle indicated by the behavior prediction result can be determined first, and a prompt message can be generated based on the behavior prediction result, so as to prompt the user to start or shut down the target application through the prompt message.

[0095] Optionally, prompting the startup or shutdown of the target application based on the prompt message can be through pop-up text prompts on the vehicle display or an electronic device communicatively connected to the vehicle, or through audio prompts by the vehicle audio playback application or an electronic device communicatively connected to the vehicle, or other prompting methods, which can be set according to actual needs and are not specifically limited here.

[0096] Optionally, the user can determine whether the startup or shutdown control of the target application by the vehicle based on the control instruction meets the user's expectations based on the prompt message.

[0097] In this embodiment, after determining the behavior prediction result of the user on the vehicle, the target application corresponding to the vehicle can be determined according to the behavior prediction result, and a prompt message for prompting the startup or shutdown of the target application can be determined and generated based on the behavior prediction result, so as to improve the user experience.

[0098] Please refer to Figure 6 , Figure 6 which shows a method for predicting user behavior provided by an embodiment of the present application. The following will elaborate in detail on the Figure 6 flow shown. The method for predicting user behavior may specifically include the following steps:

[0099] Step 510: Obtain the driving data of the vehicle.

[0100] As a way, the driving data of the vehicle may include the driving path of the vehicle, the duration from power-on to starting the behavior prediction model, the current speed of the vehicle, the current acceleration of the vehicle, and the battery temperature of the vehicle, etc. Optionally, the driving data of the vehicle can be obtained through the vehicle terminal server of the vehicle.

[0101] Step 520: Determine the driving duration and the driving distance of the vehicle according to the driving data.

[0102] As a way, after obtaining the driving data of the vehicle, the duration representation representing the driving duration and the distance identifier of the driving distance can be determined from the driving data, so that the driving duration can be determined based on the duration identifier and the driving distance can be determined based on the distance identifier.

[0103] Step 530, if the driving duration is greater than the duration threshold and / or the driving distance is greater than the distance threshold, obtain the usage data of the user in the vehicle, the environmental data of the current environment where the vehicle is located, and the attribute data of the vehicle and / or the user.

[0104] As a way, after determining the driving duration and the driving distance, the driving duration can be compared with the duration threshold to determine the first relationship between the driving duration and the duration threshold, and the driving distance can be compared with the distance threshold to determine the second relationship between the driving distance and the distance threshold.

[0105] Optionally, if the first relationship indicates that the driving duration is greater than the duration threshold and / or the second relationship indicates that the driving distance is greater than the distance threshold, it is determined that the driver's current operation on the target application of the vehicle may cause distraction and lead to a traffic accident. Thus, when it is determined that the driving duration is greater than the duration threshold and / or the driving distance is greater than the distance threshold, by obtaining the usage data of the user in the vehicle, the environmental data of the current environment where the vehicle is located, and the attribute data of the vehicle and / or the user, the behavior of the user can be predicted, so that the vehicle can be controlled based on the behavior prediction result output by the behavior prediction model.

[0106] In this embodiment, the driving duration and the driving distance of the vehicle are determined by obtaining the driving data of the vehicle. Thus, when it is determined that the driving duration is greater than the duration threshold and / or the driving distance is greater than the distance threshold, the usage data of the user in the vehicle, the environmental data of the current environment where the vehicle is located, and the attribute data of the vehicle and / or the user are obtained to ensure the timing accuracy of the behavior prediction of the user in the vehicle.

[0107] Figure 7 It is a flowchart of a method for predicting user behavior shown in an embodiment of the present application, as Figure 7As shown, in the embodiment of the present application, data preparation is first performed. By collecting the usage data of the user in the vehicle, the environmental data of the vehicle's location environment, and the attribute data of the vehicle and / or the user, and then module design is carried out to obtain a user behavior feature module, an auxiliary feature module, a context feature module, and an interaction feature module. And the usage data of the user in the vehicle, the environmental data of the vehicle's location environment, and the attribute data of the vehicle and / or the user collected are stored in the cloud server, and cloud data preprocessing is performed. Among them, cloud data preprocessing includes handling missing values and outliers, performing data preprocessing. At the same time, based on the user behavior features, auxiliary features, and context features obtained from the user behavior feature module, auxiliary feature module, and context feature module, feature engineering is carried out in the interaction feature module. Among them, feature engineering can be feature selection and construction, especially constructing possible interaction features. Optionally, constructing interaction features can include two parts: constructing a protection layer and fusing features. Among them, constructing interaction features can take the outputs of each module as inputs and calculate the interaction features through FM or other interaction methods. Fusing features can combine the interaction features with the original features to form a rich feature set.

[0108] Then, the obtained interaction features are used to train and optimize the behavior prediction model. Among them, the process of training and optimizing the behavior prediction model can include steps such as selecting a loss function, an optimization algorithm, and training iterations. Optionally, the loss function can be selected based on the specific task of the behavior prediction model to select a suitable loss function; the optimization algorithm can be to use an appropriate optimization algorithm (such as Adm) to adjust and update the parameters of the behavior prediction model; training iterations can be to make the loss function converge through multiple rounds of training. Optionally, the sample data for training the behavior prediction model can also be divided into a sample set and a validation set, so as to evaluate the performance of the behavior prediction model during the training process by using the validation set, and then adjust the hyperparameters and model structure of the behavior prediction model according to the evaluation results of the performance, so as to improve the prediction effect of the behavior prediction model. Among them, the indicators for evaluating the performance of the behavior prediction model can be, for example, AUC, F1-score, RMSE, etc., or other indicators, which can be set according to actual needs and are not specifically limited here.

[0109] After training the behavior prediction model, the trained behavior prediction model can be first deployed to the cloud server, and then distributed by the cloud server to the local server of the vehicle, so as to predict the user's behavior through the behavior prediction model in the local server.

[0110] In this way, the behavior prediction model distributed from the cloud server to the local database can be used to predict the user behavior of the real-time collected data, so as to obtain the behavior prediction result. Then, the behavior prediction model can be updated in real time according to the behavior prediction result, and the behavior prediction model can be retrained according to the latest data to maintain the adaptability and accuracy of the behavior prediction model. Optionally, the process of using the behavior prediction model to predict the user behavior of the real-time collected data can be as follows: First, the user behavior feature module uses sequence models such as LSTM or GRU to process the historical behavior data of the user in the in-vehicle entertainment system, such as frequently used applications, playing time, and interaction methods, and extracts the user's habits and preferences from them. At the same time, the auxiliary feature module analyzes the static information of the user through a fully connected neural network, such as the driver's basic information and vehicle status (such as vehicle speed, fuel consumption), and generates relevant feature vectors. These features help to understand the user's driving habits and needs. Next, the context feature module uses a convolutional neural network (CNN) to analyze the current environmental factors, such as weather conditions, current location, and time, to capture the impact of these factors on the user behavior. After the feature extraction is completed, the output feature vectors of these modules are fused into a comprehensive feature representation. Finally, in the feature interaction session, the interaction relationship between the features is calculated through FM, the non-linear interaction between the features is captured to obtain the target features, and finally the behavior prediction model is used to predict the user behavior of the target features to obtain the behavior prediction result. In this way, the behavior prediction model can accurately predict the applications that the user may start in a specific driving scenario, thereby improving the personalized experience and response speed of the in-vehicle entertainment system.

[0111] Figure 8 is a block diagram of a prediction device for user behavior shown according to an embodiment of the present application, as Figure 8 shown, the prediction device 600 for user behavior includes: a data acquisition module 610, a user behavior feature determination module 620, an auxiliary feature determination module 630, an environmental factor feature determination module 640, a target feature determination module 650, and a behavior prediction result determination module 660.

[0112] A data acquisition module 610 is configured to acquire usage data of a user in a vehicle, environmental data of an environment where the vehicle is located, and attribute data of the vehicle and / or the user; a user behavior feature determination module 620 is configured to input the usage data into a pre-trained long short-term memory network to extract user behavior features, thereby obtaining user behavior features; an auxiliary feature determination module 630 is configured to input the attribute data into a pre-trained fully connected neural network to extract auxiliary features, thereby obtaining auxiliary features; an environmental factor feature determination module 640 is configured to input the environmental data into a pre-trained convolutional neural network to extract context features, thereby obtaining environmental factor features; a target feature determination module 650 is configured to determine target features based on the user behavior features, the auxiliary features, and the environmental factor features; a behavior prediction result determination module 660 is configured to input the target features into a pre-trained behavior prediction model to perform behavior prediction, thereby obtaining a behavior prediction result of the user on the vehicle.

[0113] In some embodiments, the target feature determination module 650 includes: an interaction feature determination sub-module configured to perform feature interaction calculation on the user behavior features, the auxiliary features, and the environmental factor features to obtain interaction features; a target feature determination sub-module configured to perform feature fusion on the interaction features, the user behavior features, the auxiliary features, and the environmental factor features to obtain the target features.

[0114] In some embodiments, the interaction feature determination sub-module includes: a product feature determination unit configured to determine product features between every two of the user behavior features, the auxiliary features, and the environmental factor features; a power addition feature determination unit configured to determine power addition features corresponding to the user behavior features, the auxiliary features, and the environmental factor features respectively, where the power addition feature indicates a feature obtained by multiplying the feature by itself; an interaction feature determination unit configured to obtain the interaction features based on the product features and the power addition features.

[0115] In some embodiments, the user behavior prediction device 600 further includes: a control instruction determination module configured to determine a target application corresponding to the vehicle indicated by the behavior prediction result and generate a control instruction according to the behavior prediction result; a control module configured to control the target application based on the control instruction.

[0116] In some other embodiments, the user behavior prediction device 600 further includes: a control result determination module configured to acquire control data of the vehicle and determine a control result of the vehicle according to the control data; an adjustment module configured to adjust parameters of the pre-trained behavior prediction model according to the control result.

[0117] In still other embodiments, the user behavior prediction device 600 further includes: a prompt information generation module, configured to determine a target application corresponding to the vehicle indicated by the behavior prediction result, and generate prompt information according to the behavior prediction result, where the prompt information is used to prompt the startup or shutdown of the target application.

[0118] In still other embodiments, the user behavior prediction device 600 further includes: a driving data determination module, configured to obtain driving data of the vehicle; a first determination module, configured to determine a driving duration of the vehicle and a driving distance of the vehicle according to the driving data; a second determination module, configured to, if the driving duration is greater than a duration threshold and / or the driving distance is greater than a distance threshold, obtain usage data of the user in the vehicle, environmental data of the current environment where the vehicle is located, and attribute data of the vehicle and / or the user.

[0119] According to one aspect of the embodiments of the present application, there is also provided an electronic device, as Figure 9 shown. The vehicle 700 includes a processor 710 and one or more memories 720. The one or more memories 720 are used to store program instructions executed by the processor 710. When the processor 710 executes the program instructions, the above-mentioned user behavior prediction method is implemented.

[0120] Further, the processor 710 may include one or more processing cores. The processor 710 runs or executes instructions, programs, code sets or instruction sets stored in the memory 720, and calls data stored in the memory 720. Optionally, the processor 710 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 710 may integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing display content; the modem is used to process wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor and may be implemented separately by a communication chip.

[0121] According to one aspect of the present application, the present application further provides a computer-readable storage medium. This computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist separately without being assembled into the electronic device. The above computer-readable storage medium carries computer-readable instructions. When the computer-readable storage instructions are executed by a processor, the method in any of the above embodiments is implemented.

[0122] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. And in the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0123] The units involved in the embodiments described in the present application can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not constitute a limitation to the unit itself in some cases.

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Among them, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0125] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include known common knowledge or conventional technical means in the technical field not disclosed in the present application.

[0126] It should be understood that the present application is not limited to the exact structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A method for predicting user behavior, characterized in that: The method comprises: Acquire usage data of the user in the vehicle, environmental data of the environment where the vehicle is located, and attribute data of the vehicle and / or the user; Inputting the usage data into a pre-trained long short-term memory network to extract user behavior features to obtain user behavior features; Inputting the attribute data into a pre-trained fully connected neural network to extract auxiliary features to obtain auxiliary features; Inputting the environmental data into a pre-trained convolutional neural network to extract context features and obtain environmental factor features; Determine a target feature according to the user behavior feature, the auxiliary feature and the environmental factor feature; The target feature is input into a pre-trained behavior prediction model to perform behavior prediction, and obtain a behavior prediction result of the user on the vehicle.

2. The method according to claim 1, characterized in that The determining of the target feature according to the user behavior feature, the auxiliary feature and the environmental factor feature includes: Performing feature interaction calculation on the user behavior feature, the auxiliary feature, and the environmental factor feature to obtain an interaction feature; The interaction feature is fused with the user behavior feature, the auxiliary feature and the environmental factor feature to obtain the target feature.

3. The method according to claim 2, characterized in that The performing feature interaction calculation on the user behavior feature, the auxiliary feature and the environmental factor feature to obtain the interaction feature includes: Determine a product feature between each two of the user behavior feature, the auxiliary feature, and the environmental factor feature; Determine the power addition features corresponding to the user behavior feature, the auxiliary feature, and the environmental factor feature, respectively, wherein the power addition feature indicates a feature obtained by multiplying the feature itself; The interaction feature is obtained based on the product feature and the power feature.

4. The method according to claim 1, characterized in that: After inputting the target feature into the pre-trained behavior prediction model to perform behavior prediction and obtaining the behavior prediction result of the user on the vehicle, the method further includes: Determining a target application in the vehicle corresponding to the behavior prediction result, and generating a control instruction according to the behavior prediction result; The target application is controlled based on the control instruction.

5. The method according to claim 4, characterized in that After controlling the target application based on the control instruction, the method further includes: Acquiring control data of the vehicle, and determining a control result of the vehicle according to the control data; The parameters of the pre-trained behavior prediction model are adjusted according to the control result.

6. The method according to claim 1, characterized in that After inputting the target feature into the pre-trained behavior prediction model to perform behavior prediction and obtaining the behavior prediction result of the user on the vehicle, the method further includes: Determine a target application in the vehicle corresponding to the behavior prediction result, and generate prompt information according to the behavior prediction result, wherein the prompt information is used to prompt the start or shutdown of the target application.

7. The method according to any one of claims 1 to 6, characterized in that: Before acquiring the user's usage data in the vehicle, the environmental data of the vehicle's current environment, and the attribute data of the vehicle and / or the user, the method further includes: Acquiring driving data of the vehicle; Determine the driving time of the vehicle and the driving distance of the vehicle according to the driving data; If the driving time is greater than a time threshold and / or the driving distance is greater than a distance threshold, the user's usage data in the vehicle, the environmental data of the vehicle's current environment, and the attribute data of the vehicle and / or the user are obtained.

8. A user behavior prediction device, characterized in that: The device comprises: A data acquisition module, used to acquire the user's usage data in the vehicle, the environmental data of the environment where the vehicle is located, and the attribute data of the vehicle and / or the user; A user behavior feature determination module, used to input the usage data into a pre-trained long short-term memory network to extract user behavior features and obtain user behavior features; An auxiliary feature determination module is used to input the attribute data into a pre-trained fully connected neural network to extract auxiliary features and obtain auxiliary features; An environmental factor feature determination module is used to input the environmental data into a pre-trained convolutional neural network to extract context features and obtain environmental factor features; A target feature determination module, used to determine the target feature according to the user behavior feature, the auxiliary feature and the environmental factor feature; The behavior prediction result determination module is used to input the target feature into a pre-trained behavior prediction model to perform behavior prediction and obtain the behavior prediction result of the user on the vehicle.

9. An electronic device, characterized in that: The electronic device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images