Information recommendation method, device and equipment based on vehicle-mounted specification, and vehicle

Abstract

The invention relates to an information recommendation method and device based on a vehicle-mounted specification, equipment and a vehicle. The method comprises the following steps: acquiring vehicle state data of a vehicle, environment data of a driving environment and user behavior data in real time; determining a driving scene label and a user query intention corresponding to the vehicle according to the vehicle state data, the environment data and the user behavior data of the vehicle; retrieving to-be-recommended information from a preset knowledge base according to the driving scene label and the user query intention; determining a recommendation mode of the to-be-recommended information according to the priority of the to-be-recommended information; and recommending the to-be-recommended information to the user of the vehicle according to the recommendation mode of the to-be-recommended information. According to the invention, the user using the vehicle can quickly and accurately query the vehicle-mounted specification, and the query operation convenience can be remarkably improved on the premise of ensuring the driving safety.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to an information recommendation method and device based on vehicle-mounted instructions, an electronic device and a vehicle. BACKGROUND

[0002] With the rapid development of intelligent technology for vehicles, vehicle-mounted electronic instruction systems have gradually become an important part of intelligent cockpits in vehicles. The vehicle-mounted electronic instructions can help users quickly master the intelligent functions of the vehicle to improve the convenience of driving for users.

[0003] In the prior art, the vehicle-mounted electronic instructions are stored in the memory of the vehicle system in a static manner, and the user needs to actively search for the vehicle-mounted electronic instructions in the vehicle system. However, in this manner, the driver cannot safely perform manual queries during driving, and thus cannot quickly and accurately query the vehicle-mounted electronic instructions.

[0004] In another manner, an automotive electronic instruction system is constructed based on a voice recognition scheme of cloud computing, and the user can query the instructions through voice interaction. However, in this manner, the instruction content cannot be dynamically recommended to adapt to the driving scene, resulting in a low matching accuracy of the information queried in an emergency working condition. SUMMARY

[0005] One of the purposes of the present application is to provide an information recommendation method based on vehicle-mounted instructions, which can remind traffic participants around the vehicle to improve attention and control driving, reduce traffic accidents, and improve traffic safety. The second purpose is to provide an information recommendation device based on vehicle-mounted instructions. The third purpose is to provide an electronic device. The fourth purpose is to provide a vehicle. The fifth purpose is to provide a computer-readable storage medium. The sixth purpose is to provide a computer program product.

[0006] To achieve the above purposes, the technical solutions adopted by the present application are as follows:

[0007] An information recommendation method based on vehicle-mounted instructions, comprising:

[0008] real-time acquisition of vehicle state data of a vehicle, environmental data of a driving environment in which the vehicle is located, and user behavior data; and determination of a driving scene label corresponding to the vehicle and a user query intention according to the vehicle state data of the vehicle, the environmental data, and the user behavior data;

[0009] retrieval of to-be-recommended information from a preset knowledge base according to the driving scene label and the user query intention; wherein the preset knowledge base includes text units corresponding to instruction content of each chapter in the vehicle-mounted instructions;

[0010] determine a recommendation manner of the to-be-recommended information according to the priority of the to-be-recommended information; wherein the priority represents importance and urgency of the user obtaining the to-be-recommended information;

[0011] recommend the to-be-recommended information to the user of the vehicle according to the recommendation manner of the to-be-recommended information.

[0012] Further, the retrieving the to-be-recommended information from the preset knowledge base according to the driving scene label and the user query intention comprises:

[0013] generating a retrieval condition according to the driving scene label and the user query intention;

[0014] determining at least one candidate text unit from the preset knowledge base according to the retrieval condition;

[0015] reordering and processing the at least one candidate text unit to obtain the to-be-recommended information.

[0016] Further, the retrieval condition comprises a query vector representation representing the user query intention; and the determining the at least one candidate text unit from the preset knowledge base according to the retrieval condition comprises:

[0017] vectorizing each text unit in the preset knowledge base to obtain a text vector representation corresponding to the text unit;

[0018] determining the at least one candidate text unit from the preset knowledge base according to a vector similarity between the query vector representation in the retrieval condition and the text vector representation.

[0019] Further, the reordering and processing the at least one candidate text unit to obtain the to-be-recommended information comprises:

[0020] determining a first relevance degree corresponding to the candidate text unit according to the driving scene label; wherein the first relevance degree represents relevance of the candidate text unit to the driving scene in which the current vehicle is located;

[0021] determining a query priority corresponding to the candidate text unit according to the user query intention; wherein the query priority represents urgency of the user querying the candidate text unit;

[0022] reordering and processing each of the candidate text units according to the first relevance degree corresponding to each of the candidate text units and the query priority to obtain the to-be-recommended information.

[0023] Further, the determining the driving scene label corresponding to the vehicle and the user query intention according to the vehicle state data, the environment data and the user behavior data of the vehicle comprises:

[0024] performing feature processing on the vehicle state data and the environment data of the vehicle to obtain vehicle feature data of the vehicle;

[0025] performing feature recognition processing on the vehicle feature data to obtain the driving scene label corresponding to the vehicle;

[0026] performing recognition on the user behavior data of the vehicle to obtain the user query intention corresponding to the vehicle.

[0027] Further, the determining the recommendation mode of the to-be-recommended information according to the priority of the to-be-recommended information comprises:

[0028] determining the recommendation mode of the to-be-recommended information according to the priority of the to-be-recommended information and the user preference data of the vehicle; wherein the user preference data represents the information receiving mode preferred by the user.

[0029] Further, before the determining the recommendation mode of the to-be-recommended information according to the priority of the to-be-recommended information, the method further comprises:

[0030] determining a second correlation degree corresponding to the to-be-recommended information; and determining a complexity corresponding to the to-be-recommended information; wherein the second correlation degree represents the correlation between the to-be-recommended information and the driving scene in which the current vehicle is located; and the complexity represents the operation complexity degree corresponding to the instruction content represented by the to-be-recommended information;

[0031] determining the priority of the to-be-recommended information according to the second correlation degree and the complexity corresponding to the to-be-recommended information.

[0032] Further, the method further comprises:

[0033] if it is determined that no interaction operation of the user is received within a preset time period, or if it is determined that an exit instruction input by the user is received, controlling a vehicle-mounted display device in the vehicle to restore an original display state; wherein the vehicle-mounted display device is configured to display the to-be-recommended information to the user of the vehicle.

[0034] An information recommendation device based on vehicle-mounted instructions, comprising:

[0035] a first determining module configured to acquire vehicle state data of a vehicle, environment data of a driving environment in which the vehicle is located and user behavior data in real time; and determine a driving scene label corresponding to the vehicle and a user query intention according to the vehicle state data, the environment data and the user behavior data of the vehicle;

[0036] retrieving a to-be-recommended information from a preset knowledge base according to the driving scene label and the user query intention; wherein the preset knowledge base comprises text units corresponding to the content of each chapter of a vehicle-mounted instruction manual;

[0037] determining a recommendation mode of the to-be-recommended information according to a priority of the to-be-recommended information; wherein the priority represents the importance and urgency of the user obtaining the to-be-recommended information;

[0038] recommending the to-be-recommended information to the user of the vehicle according to the recommendation mode of the to-be-recommended information.

[0039] Further, the retrieval module is specifically configured to: generate a retrieval condition according to the driving scene label and the user query intention; determine at least one candidate text unit from the preset knowledge base according to the retrieval condition; and perform reordering processing on the at least one candidate text unit to obtain the to-be-recommended information.

[0040] Further, the retrieval condition comprises a query vector representation representing the user query intention; and the retrieval module is specifically configured to: perform vectorization processing on each text unit in the preset knowledge base to obtain a text vector representation corresponding to the text unit; and determine the at least one candidate text unit from the preset knowledge base according to the vector similarity between the query vector representation in the retrieval condition and the text vector representation.

[0041] Further, the retrieval module is specifically configured to: determine a first relevance corresponding to the candidate text unit according to the driving scene label; wherein the first relevance represents the relevance of the candidate text unit to the driving scene in which the current vehicle is located; determine a query priority corresponding to the candidate text unit according to the user query intention; wherein the query priority represents the urgency of the user querying the candidate text unit; and perform reordering and screening processing on each of the candidate text units according to the first relevance corresponding to each of the candidate text units and the query priority to obtain the to-be-recommended information.

[0042] Further, the first determination module is specifically configured to: perform feature processing on vehicle state data and environment data of the vehicle to obtain vehicle feature data of the vehicle; perform feature recognition processing on the vehicle feature data to obtain the driving scene label corresponding to the vehicle; and perform recognition on user behavior data of the vehicle to obtain the user query intention corresponding to the vehicle.

[0043] Further, the second determining module is specifically configured to determine the recommendation mode of the to-be-recommended information according to the priority of the to-be-recommended information and user preference data of the vehicle; wherein the user preference data represents a user preferred information receiving mode.

[0044] Further, before the second determining module is configured to determine the recommendation mode of the to-be-recommended information according to the priority of the to-be-recommended information, the second determining module is further configured to determine a second correlation degree corresponding to the to-be-recommended information and determine a complexity corresponding to the to-be-recommended information; wherein the second correlation degree represents the correlation between the to-be-recommended information and a current driving scene in which the vehicle is located; the complexity represents the operation complexity degree corresponding to the instruction content represented by the to-be-recommended information; and the priority of the to-be-recommended information is determined according to the second correlation degree and the complexity corresponding to the to-be-recommended information.

[0045] Further, the device is further configured to control a vehicle-mounted display device in the vehicle to restore an original display state if it is determined that no interaction operation of a user is received within a preset time period or if it is determined that a user input exit instruction is received; wherein the vehicle-mounted display device is configured to display the to-be-recommended information to a user of the vehicle.

[0046] An electronic device, comprising: a memory, a processor;

[0047] The memory stores computer execution instructions;

[0048] The processor executes the computer execution instructions stored in the memory, so that the processor executes the first aspect and / or various possible implementation manners of the first aspect.

[0049] A vehicle, comprising an electronic device; the electronic device is configured to execute the first aspect and / or various possible implementation manners of the first aspect.

[0050] A computer readable storage medium, the computer readable storage medium stores computer execution instructions, the computer execution instructions are executed by a processor to implement the first aspect and / or various possible implementation manners of the first aspect.

[0051] A computer program product, comprising a computer program, the computer program is executed by a processor to implement the first aspect and / or various possible implementation manners of the first aspect.

[0052] The beneficial effects of the present application are as follows:

[0053] The application determines the current driving scene label and user query intention through real-time vehicle state data of the vehicle, environment data of the driving environment and user behavior data, and retrieves the to-be-recommended information from a preset knowledge base according to the current driving scene label and the user query intention, determines the recommendation mode of the to-be-recommended information according to the importance and urgency of the to-be-recommended information obtained by the user, and recommends the specification content corresponding to the to-be-recommended information to the user through the recommendation mode, so that the user can quickly and accurately realize the query of the vehicle-mounted specification. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 An application scenario provided by an embodiment of the application is shown in the figure.

[0055] Figure 2 A flow of the information recommendation method based on the vehicle-mounted specification provided by an embodiment of the application is shown in the figure. Figure 1

[0056] Figure 3 An application flow of the intelligent automobile electronic specification system and method supporting scene perception and voice recognition provided by an embodiment of the application is shown in the figure.

[0057] Figure 4 A flow of the information recommendation method based on the vehicle-mounted specification provided by an embodiment of the application is shown in the figure. Figure 2

[0058] Figure 5 An application flow of the scene perception type automobile specification intelligent retrieval system based on multi-modal interaction provided by an embodiment of the application is shown in the figure.

[0059] Figure 6 A structural schematic diagram of the information recommendation device based on the vehicle-mounted specification provided by an embodiment of the application is shown in the figure.

[0060] Figure 7 A structural schematic diagram of the electronic device provided by an embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0061] The embodiments of the application will be described below with reference to the accompanying drawings and preferred embodiments, and other advantages and effects of the application can be easily understood by those skilled in the art from the disclosure in the specification. The application can also be implemented or applied by different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the application. It should be understood that the preferred embodiments are only for illustrating the application, and are not intended to limit the protection scope of the application.

[0062] ​​It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concepts of the present application, and only the components related to the present application are shown in the diagrams, rather than being drawn according to the number, shape and size of the components in actual implementation. The actual implementation of each component may be a random change, and the component layout pattern may be more complex.

[0063] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards, and provide corresponding operation portal for user to choose authorization or refusal.

[0064] Figure 1 The application scenario diagram provided by an embodiment of the present application is shown in FIG. 1. Figure 1 As shown in FIG. 1, with the rapid development of intelligent technology of the automobile, the vehicle-mounted electronic manual system gradually becomes an important part of the intelligent cockpit. Based on the vehicle-mounted electronic manual system, the vehicle-mounted electronic manual can be queried to meet the operation guidance needs of complex functions of modern automobiles.

[0065] In view of this, the embodiment of the present application provides an information recommendation method based on vehicle-mounted manual. The current driving scene label of the vehicle and the user query intention are used to retrieve the to-be-recommended information from a preset knowledge base. According to the importance and urgency of the to-be-recommended information obtained by the user, the corresponding recommendation mode is determined, and the manual content corresponding to the to-be-recommended information is recommended to the user. Thus, the user can quickly and accurately realize the vehicle-mounted manual query.

[0066] The technical solutions of the present application will be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0067] Figure 2 The flow of the information recommendation method based on vehicle-mounted manual provided by an embodiment of the present application is shown in FIG. 2. Figure 1 As shown in FIG. 2, the method comprises the following steps. Figure 2

[0068] 201, real-time acquisition of vehicle state data of the vehicle, environment data of the driving environment and user behavior data; and according to the vehicle state data of the vehicle, the environment data and the user behavior data, the driving scene label corresponding to the vehicle and the user query intention are determined.

[0069] ​Exemplarily, the execution subject of the embodiment can be an electronic device, hereinafter referred to as a device. The device can be a virtual device or a physical device that executes the information recommendation method based on the vehicle specification. Figure 3 The application flowchart of the intelligent vehicle electronic specification system and method supporting scene perception and voice recognition provided by an embodiment of the present application is shown in Figure 3 As shown in the figure, the user in the vehicle can input a voice instruction (such as "how to start automatic driving") or a touch operation (such as clicking the "help" button on the center control screen) to the vehicle machine to activate the intelligent vehicle electronic specification system. The intelligent vehicle electronic specification system will start the intelligent specification mode and collect the multi-source sensor data of the vehicle, that is, the real-time vehicle state data, the environmental data of the driving environment, and the user behavior data, wherein the vehicle state data represents the driving state of the vehicle; the environmental data represents the environmental situation of the driving environment; and the user behavior data represents the query behavior of the user to the vehicle machine. The device inputs the obtained vehicle state data, environmental data, and user behavior data into a pre-trained multi-modal large model based on the Flink engine, such as a hybrid deep learning model that combines the Transformer architecture and the Graph Neural Network (GNN), to dynamically identify and process the vehicle state data and the environmental data, identify the driving scene where the vehicle is currently located, classify the driving scene, and obtain the classification label of the driving scene, that is, obtain the driving scene label corresponding to the vehicle. At the same time, based on the pre-trained multi-modal large model, the user voice or touch interaction of the user behavior data is identified and processed, which can accurately analyze the operation intention to obtain the user query intention of the user in the vehicle (such as "how to adjust the windshield wiper" or "set the air conditioning cooling mode").

[0070] The device can be the cloud in a smart automotive electronic manual system, which also includes multiple sensors and a vehicle computer installed in the vehicle. The vehicle computer can obtain real-time vehicle status data (such as vehicle speed, engine speed, fault codes, etc.) via a Controller Area Network (CAN) bus or a Local Interconnect Network (LIN) bus at a sampling frequency ≥ 100Hz. The vehicle computer can also determine the driving environment using various environmental sensors installed in the vehicle to obtain environmental data, such as visual data collected by a camera, distance data collected by a millimeter-wave radar, weather data and lighting data collected by other sensors, and temperature data collected by an in-vehicle temperature sensor. The vehicle computer can also obtain user behavior data based on user interactions within the vehicle, including voice input commands and touch operation habits. Voice input commands are collected by a distributed microphone array, and touch operation habits based on touch interaction can be captured by the central control screen's capacitive touch module to identify taps, swipes, and other operations (such as clicking the "Help" icon or swiping to turn pages). After the vehicle computer obtains vehicle status data, environmental data, user behavior data and other vehicle-side data, it uploads these vehicle-side data to the cloud. Figure 3 , it can achieve high-throughput, low-latency storage and transmission of real-time data streams through Kafka message queues, ensuring the temporal consistency of the data uploaded to the cloud.

[0071] 202. Retrieve information to be recommended from a preset knowledge base based on the driving scenario label and the user's query intent; wherein the preset knowledge base includes text units corresponding to the instruction manual content of each chapter in the vehicle manual.

[0072] For example, the device can call a preset knowledge base stored locally in the vehicle, which includes text units corresponding to the contents of each chapter in the vehicle manual, and the preset knowledge base also includes each preset driving scene label, each preset user query intention, and each preset driving scene label. Figure 2 The device uses the scenario-based enhanced retrieval generation (RAG) technology, that is, according to the corresponding relationship in the preset knowledge base, it can retrieve the corresponding driving scene label and user query intention from the preset knowledge base. Figure 2 For example, when the outdoor temperature is above 30°C and the vehicle user asks about "air conditioning settings", the "temperature control system operating instructions" is recommended first.

[0073] The preset knowledge base is constructed based on an electronic specification in a Markdown format easy to maintain. Specifically, the electronic specification in the Markdown format is acquired, the electronic specification includes a multi-level title structure organization, text slicing is performed on the specification content under each level of the title structure organization, each chapter or paragraph under each level of the title structure organization is taken as an independent text unit, and each text unit obtained is stored in the preset knowledge base, facilitating subsequent retrieval and generation.

[0074] 203. Determine a recommendation mode of the to-be-recommended information according to a priority of the to-be-recommended information, wherein the priority represents importance and urgency of the user obtaining the to-be-recommended information.

[0075] For example, for each text unit in the preset knowledge base, a priority of each text unit can be set in advance to represent importance and urgency of the user obtaining the to-be-recommended information. For example, the specification content corresponding to each text unit is analyzed manually, and each text unit is marked with a priority to obtain a priority label of each text unit, such as urgent, non-urgent, important, or non-important. Alternatively, the priority label of each text unit is determined according to the attribute (safety warning, operation guidance, or function description) of each text unit, and the priority label of each text unit is stored in the preset knowledge base. When the device acquires the to-be-recommended information, the priority of the text unit included in the to-be-recommended information can be acquired from the preset knowledge base as the priority of the to-be-recommended information. The device can call a preset mapping table including a correspondence between each preset priority and each preset recommendation mode, determine the preset recommendation mode corresponding to the priority of the current to-be-recommended information according to the preset mapping table, and determine the preset recommendation mode as the recommendation mode of the to-be-recommended information.

[0076] For example, in combination with Figure 3Based on the preset adaptive output arbitration strategy, the priority of recommended information belonging to the safety warning category is higher than the priority of recommended information belonging to the operation guidance category and higher than the priority of recommended information belonging to the function description category, with the priorities being high, medium, and low respectively. For recommended information including emergency instructions (such as fault alarms), the recommended method is flashing warning + voice output based on the high priority of the emergency instructions; for recommended information including complex operations (such as maintenance steps), the recommended method is step-by-step graphic display + voice-assisted explanation based on the medium priority of the complex operations; for recommended information including general queries (such as function usage), the recommended method is voice broadcast + floating pop-up window prompt based on the low priority of the general queries. Furthermore, through the Augmented Reality Head-Up Display (AR-HUD), a full-screen flashing red warning and high-priority voice broadcast are used to instantly transmit emergency information such as collision warnings and system failures, ensuring the driver's quick response, reducing accident risks and improving driving safety. At the same time, an adaptive output strategy is adopted, and complex operations are guided by step-by-step graphics and text on the central control screen. Routine information is combined with voice and floating prompts to reduce driver distraction and improve operating efficiency.

[0077] 204. Recommend the information to be recommended to the user of the vehicle according to the recommendation method of the information to be recommended.

[0078] Exemplarily, the device determines a presentation method corresponding to the recommendation method of the information to be recommended based on the recommendation method of the information to be recommended, and determines an output device corresponding to the presentation method based on the presentation method, and recommends the information to be recommended to the user of the vehicle through the output device.

[0079] For example, for recommended information involving complex operations (such as repair procedures), the recommended method is a full-screen red flashing warning combined with voice output. The AR-HUD flashes a red warning and announces the recommended information via voice output. Conventional operating instructions are displayed on the central control screen in graphic form, supplemented by voice instructions; basic function instructions are explained using concise voice prompts.

[0080] In this embodiment, a method for recommending information based on vehicle manuals is provided. This method intelligently optimizes the content recommendation method and interaction mode of electronic manuals by dynamically adapting to driving scenarios and user query intentions. This not only allows users to quickly and accurately query manuals, but also significantly improves the convenience of query operations while ensuring driving safety.

[0081] Figure 4 The process of the information recommendation method based on the vehicle manual provided by an embodiment of the present invention Figure 2 ,likeFigure 4 The method comprises:

[0082] 301. Real-time acquisition of vehicle state data of the vehicle, environment data of the driving environment, and user behavior data, and determination of the driving scene label corresponding to the vehicle and the user query intention according to the vehicle state data of the vehicle, the environment data, and the user behavior data.

[0083] By way of example, this step can refer to step 201, which will not be described here again.

[0084] In one example, step 301 comprises: performing feature processing on the vehicle state data and the environment data of the vehicle to obtain vehicle feature data of the vehicle; performing feature recognition processing on the vehicle feature data to obtain the driving scene label corresponding to the vehicle; and performing recognition on the user behavior data of the vehicle to obtain the user query intention corresponding to the vehicle.

[0085] Specifically, after starting the intelligent query electronic manual mode, the vehicle machine acquires vehicle end multi-source sensor data, including the vehicle state data and the environment data of the vehicle and the user behavior data, and transmits the data to the device in real time through Kafka real-time data flow. The device calls a preset Flink stream computing engine, and based on the Flink stream computing engine, performs data synchronization processing on the vehicle state data and the environment data and the user behavior data, that is, unifies the timestamps of all data, and performs batch processing according to a preset time window such as a 3-second window; for each batch of data obtained, according to the dimensions of different sensor data, a first custom algorithm is used to extract features from the vehicle state data to obtain first feature data representing vehicle state features, and a second custom algorithm is used to extract features from the environment data to obtain second feature data representing environment features, and the first feature data and the second feature data are arranged to obtain vehicle feature data.

[0086] Among them, for the CAN / LIN signal in each batch of vehicle state data, the first custom algorithm can detect abnormal signals of devices such as engines and motors, and extract signal features such as signal frequency and phase of the CAN / LIN signal as first feature data; through the second custom algorithm such as the target detection algorithm, the target detection is performed on the data collected by the camera in the environment data, and the position data or contour data of the lane line, traffic sign, and pedestrian target are recognized as second feature data.

[0087] Further, Figure 5 The application flowchart of the scene-aware automobile manual intelligent retrieval system based on multi-modal interaction provided by an embodiment of the present application is as follows: Figure 5As shown, the device calls a pre-trained graph neural network. In order to make the graph neural network better understand the importance of different features, an attention mechanism (Attention) is introduced, which can help the model pay more attention to information more important for scene recognition. The obtained vehicle feature data is input into the pre-trained graph neural network, and based on the attention mechanism (Attention), the vehicle feature data is attention processed, and the driving scene label of the current vehicle is recognized and output, such as the driving scene label of "high-speed cruising on a sunny day with normal battery" or "urban congestion on a rainy day with abnormal tire pressure". At the same time, for the same batch of user behavior data, the user behavior data is analyzed for voice / touch interaction. Specifically, in terms of voice interaction, the voice command input by the user is picked up through the vehicle microphone, and a third-party cloud / offline speech recognition application programming interface (API) is called to use automatic speech recognition (ASR) technology to recognize the voice command and obtain the user query intent; and / or, by monitoring the click / slide events input by the user on the central control screen, such as clicking the "help" button of a specific function area or sliding to select a menu item, the user operation is mapped to the corresponding user query intent, and then the user demand is accurately understood through voice recognition and touch operation.

[0088] 302. Generate a retrieval condition according to the driving scene label and the user query intent.

[0089] For example, the device comprehensively considers the current driving scene label and the user interaction intent. Specifically, according to a preset weighting algorithm, the current driving scene label and the user query intent are weighted and analyzed, wherein the driving scene label can account for 30% of the weight, and the user query intent can account for 70% of the weight, to generate a retrieval condition; for example, the driving scene label is identified as "rainy day", and the user query intent is "wiper usage method", to generate the retrieval keyword "rainy day usage" as the retrieval condition.

[0090] 303. Determine at least one candidate text unit from the pre-set knowledge base according to the retrieval condition.

[0091] For example, in combination with Figure 5According to the retrieval condition obtained based on the scene feature fusion, the device uses the retrieval-augmented generation (RAG) technology to retrieve matching content from the structured electronic manual knowledge base (i.e., the preset knowledge base), including multiple text units strongly related to the driving scene label and the user query intention, and additional context information. The multiple text units and the context information are input into a preset natural language processing model (Bidirectional Encoder Representations from Transformers, BERT for short), such as a lightweight BERT model, for model processing, and at least one candidate text unit with more abundant and accurate information is output. The lightweight BERT model is obtained by adjusting and training the model parameters of the initial BERT model through a likelihood function (maximum likelihood estimation algorithm).

[0092] In combination with Figure 5 The preset knowledge base is constructed based on the original manual in the Markdown format or the Portable Document Format (PDF) format that is easy to maintain. Specifically, an electronic manual is obtained, the electronic manual is subjected to text analysis and extraction and structured segmentation processing according to function modules, to obtain manual texts under multiple function modules, and the manual texts under each function module are subjected to text slicing processing, each chapter or paragraph in the manual texts under each function module is taken as an independent text unit, and each text unit obtained is stored in the preset knowledge base, for subsequent retrieval and generation.

[0093] In one example, step 303 includes: performing vectorization processing on each text unit in the preset knowledge base to obtain a text vector representation corresponding to the text unit; and determining at least one candidate text unit from the preset knowledge base according to the vector similarity between the query vector representation in the retrieval condition and the text vector representation.

[0094] Specifically, in combination with Figure 5The device obtains a query vector representation in a search condition, where the device obtains a query semantic text by performing semantic recognition on the obtained driving scene label and user query intention, and obtains the query vector representation of the query semantic text by performing vectorization processing on the query semantic text in the preset knowledge base by using an encoder. The device performs vectorization processing on each text unit in the preset knowledge base by using the encoder to obtain a text vector representation corresponding to each text unit, and stores the text vector representation in the form of a vector data block for processing. The device calculates the similarity between the query vector representation and each text vector representation based on a cosine similarity calculation formula to obtain a cosine similarity as a vector similarity between the query vector representation and each text vector representation, and sorts all the text vector representations according to the vector similarity from high to low according to each vector similarity to recall text units corresponding to the top N text vector representations as candidate text units.

[0095] 304. Reordering the at least one candidate text unit to obtain the recommended information.

[0096] For example, the device calls a preset reordering processing technology, such as a term frequency (TF) and inverse document frequency (IDF) based information retrieval algorithm, to reorder the at least one candidate text unit. Specifically, the device performs semantic recognition on the obtained driving scene label and user query intention to obtain a query semantic text, extracts keywords such as “rainy day operation” from the query semantic text, calculates the term frequency (TF) of the keywords in each candidate text unit, calculates the inverse document frequency (IDF) of the keywords in each candidate text unit, performs weighted summation calculation on the term frequency (TF) of the keywords in each candidate text unit and the inverse document frequency (IDF) of the keywords in each candidate text unit to obtain a relevance score of each candidate text unit with respect to the query semantic text, sorts the candidate text units according to the relevance score, and the candidate text unit with a higher relevance score is more relevant, and then the candidate text unit with the highest relevance score is taken as the recommended information.

[0097] In one example, step 304 includes determining a first relevance degree of the candidate text unit corresponding to the driving scene label, where the first relevance degree represents the relevance of the candidate text unit to the driving scene in which the current vehicle is located; determining a query priority of the candidate text unit corresponding to the user query intention, where the query priority represents the urgency of the user query for the candidate text unit; and performing reordering and screening processing on the candidate text units according to the first relevance degree and the query priority of each candidate text unit to obtain the recommended information.

[0098] Specifically, the device calculates the vector similarity between the semantic vector corresponding to the driving scene label and the vector corresponding to each candidate text unit according to a vector similarity calculation formula, such as a cosine similarity calculation formula, as the first correlation degree, to represent the relevance of the candidate text unit to the driving scene in which the current vehicle is located. According to the matching degree between the current user query intention and the preset user query intention corresponding to each candidate text unit, the query priority corresponding to each candidate text unit is determined to represent the urgency of the user querying the candidate text unit. The first correlation degree and the query priority corresponding to each candidate text unit are processed, such as weighted calculation of the first correlation degree and the query priority corresponding to each candidate text unit, to obtain a ranking score corresponding to each candidate text unit. According to the ranking score from high to low, the various candidate text units are sorted and optimized. According to the obtained ranking result, the most relevant (ranking first) candidate text unit is selected, and the generative pre-training transformer (GPT) is used to further optimize and polish the most relevant candidate text unit, to generate the final content recommended to the user, and obtain the information to be recommended.

[0099] For example, when the system identifies the "rainy day" driving scene, and the user asks "rain wiper usage method" through voice, the relevant operation instruction chapter in the "rainy day driving guide" will be automatically retrieved and returned. This scenario-based RAG implementation mechanism not only ensures the accuracy of content recommendation, but also improves the operation efficiency of the user.

[0100] 305、According to the priority of the information to be recommended and the user preference data of the vehicle, the recommendation mode of the information to be recommended is determined; wherein the user preference data represents the user's preferred information receiving mode.

[0101] Illustratively, the device can automatically generate and remember the user's preferred information receiving mode based on the user's input information receiving mode setting record, to obtain the user preference data. The device can determine whether the information to be recommended can be received in a self-defined mode according to the priority of the information to be recommended, for example, if the priority is within a preset range, the information to be recommended can be received in a self-defined mode, otherwise it cannot be received in a self-defined mode. If it is determined that the information to be recommended can be received in a self-defined mode, the recorded user's preferred information receiving mode is determined as the recommendation mode of the information to be recommended.

[0102] For example, based on the adaptive strategy of the user preference, the device automatically memorizes the information receiving mode preferred by the user, and for the to-be-recommended information belonging to the routine operation guide category, the priority is medium, which meets the preset range (medium / low), and if the user selects "text prompt only" multiple times, the subsequent to-be-recommended information belonging to the routine operation guide category will adopt the recommended mode of "text prompt only" by default.

[0103] In one example, before step 305, it further includes: determining a second relevance corresponding to the to-be-recommended information; and determining a complexity corresponding to the to-be-recommended information; wherein the second relevance represents the relevance between the to-be-recommended information and the driving scene where the current vehicle is located; the complexity represents the operation complexity corresponding to the instruction content represented by the to-be-recommended information; and the priority of the to-be-recommended information is determined according to the second relevance and the complexity corresponding to the to-be-recommended information.

[0104] Specifically, the device calculates the vector similarity between the semantic vector corresponding to the driving scene label of the driving scene where the current vehicle is located and the vector corresponding to the to-be-recommended information as the second relevance according to the vector similarity calculation formula, such as the cosine similarity calculation formula, to represent the relevance between the to-be-recommended information and the driving scene where the current vehicle is located. The complexity corresponding to the to-be-recommended information is obtained by performing semantic recognition on the instruction content represented by the to-be-recommended information, to represent the operation complexity corresponding to the instruction content represented by the to-be-recommended information; for example, the operation complexity in "Rainy Day Driving Guide" is high, and the operation complexity in "Sunny Day Driving Guide" is low. Based on the preset mapping relationship between each second relevance, each complexity, and each preset priority, the preset priority corresponding to the second relevance and the complexity of the current to-be-recommended information can be determined as the priority of the to-be-recommended information.

[0105] 306、According to the recommendation mode of the to-be-recommended information, the to-be-recommended information is recommended to the user of the vehicle.

[0106] For example, this step can refer to step 204, which will not be described here.

[0107] In one example, after step 305, it further includes: if it is determined that no interaction operation of the user is received within a preset time period, or if it is determined that an exit instruction input by the user is received, the vehicle-mounted display device in the vehicle is controlled to restore the original display state; wherein the vehicle-mounted display device is used to display the to-be-recommended information to the user of the vehicle.

[0108] Specifically, based on the timeout recovery and state management strategy, if the device determines that no interaction operation of the user is received within a preset time period or determines that an exit instruction of the user input is received, the device sends a reply instruction to the in-vehicle display device in the vehicle to control the display interface of the in-vehicle display device to recover to an original display state (for example, to recover to a navigation interface), so as to ensure driving continuity. When the display interface of the in-vehicle display device automatically recovers to the original interface state, that is, after exiting the instruction manual mode, all personalized settings are saved and automatically loaded next time, thereby providing a continuously optimized user experience.

[0109] In the embodiment, based on the above-mentioned embodiment, on the one hand, based on driving scene classification and multi-modal data analysis, high-relevance content is preferentially pushed in combination with the RAG (reinforced attentional gating) technology, the time for obtaining key information is shortened, and information accuracy is enhanced. On the other hand, user high-frequency operation preferences (for example, voice or text preference) are automatically remembered, and output modes are adaptively adjusted in similar scenes. After exiting, the original interface is automatically recovered, manual switching steps are reduced, and user experience is optimized.

[0110] Figure 6 A structure schematic diagram of an information recommendation device based on an in-vehicle instruction manual provided by an embodiment of the present application is shown in FIG. 1. Figure 6 As shown in the figure, the device comprises:

[0111] A first determination module 401 is configured to acquire vehicle state data of a vehicle, environmental data of a driving environment in which the vehicle is located, and user behavior data in real time, and determine a driving scene label corresponding to the vehicle and a user query intention according to the vehicle state data, the environmental data, and the user behavior data.

[0112] A retrieval module 402 is configured to retrieve to-be-recommended information from a preset knowledge base according to the driving scene label and the user query intention, wherein the preset knowledge base comprises text units corresponding to instruction manual contents of each chapter in the in-vehicle instruction manual.

[0113] A second determination module 403 is configured to determine a recommendation mode of the to-be-recommended information according to a priority of the to-be-recommended information, wherein the priority represents importance and urgency of the user obtaining the to-be-recommended information.

[0114] A recommendation module 404 is configured to recommend the to-be-recommended information to a user of the vehicle according to the recommendation mode of the to-be-recommended information.

[0115] Further, the retrieval module 402 is specifically configured to: generate a retrieval condition according to the driving scene label and the user query intention; determine at least one candidate text unit from the preset knowledge base according to the retrieval condition; and perform reordering processing on the at least one candidate text unit to obtain the to-be-recommended information.

[0116] Further, the search condition includes a query vector representation representing a user query intention; the search module 402 is specifically configured to: perform vectorization processing on each text unit in the preset knowledge base to obtain a text vector representation corresponding to the text unit; and determine at least one candidate text unit from the preset knowledge base according to a vector similarity between the query vector representation in the search condition and the text vector representation.

[0117] Further, the search module 402 is specifically configured to: determine a first relevance corresponding to the candidate text unit according to the driving scene label; wherein the first relevance represents the relevance of the candidate text unit to the driving scene in which the current vehicle is located; determine a query priority corresponding to the candidate text unit according to the user query intention; wherein the query priority represents the urgency of the user query for the candidate text unit; and perform reordering and filtering processing on each candidate text unit according to the first relevance and the query priority corresponding to each candidate text unit to obtain the information to be recommended.

[0118] Further, the first determination module 401 is specifically configured to: perform feature processing on vehicle state data and environment data of the vehicle to obtain vehicle feature data of the vehicle; perform feature recognition processing on the vehicle feature data to obtain a driving scene label corresponding to the vehicle; and perform recognition on user behavior data of the vehicle to obtain a user query intention corresponding to the vehicle.

[0119] Further, the second determination module 403 is specifically configured to: determine a recommendation mode of the information to be recommended according to the priority of the information to be recommended and user preference data of the vehicle; wherein the user preference data represents a preferred information receiving mode of the user.

[0120] Further, the second determination module 403 is specifically configured to: before determining the recommendation mode of the information to be recommended according to the priority of the information to be recommended, the second determination module 403 is further configured to: determine a second relevance corresponding to the information to be recommended; and determine a complexity corresponding to the information to be recommended; wherein the second relevance represents the relevance of the information to be recommended to the driving scene in which the current vehicle is located; and the complexity represents an operation complexity degree corresponding to the instruction content represented by the information to be recommended; and determine the priority of the information to be recommended according to the second relevance and the complexity corresponding to the information to be recommended.

[0121] Further, the device is further configured to: if it is determined that no interaction operation of the user is received within a preset time period, or if it is determined that an exit instruction input by the user is received, control a vehicle-mounted display device in the vehicle to restore an original display state; wherein the vehicle-mounted display device is configured to display the information to be recommended to the user of the vehicle.

[0122] The device of the embodiment can execute the technical solutions in the above method, and the specific implementation process and technical principles are the same, which will not be described here.

[0123] Figure 7 A structural schematic diagram of an electronic device is provided for an embodiment of the present application. As shown in the figure, the electronic device can include a memory 501 and a processor 502. Figure 7

[0124] The memory 501 is configured to store a program. Specifically, the program can include program codes, and the program codes include computer execution instructions.

[0125] The memory 501 can include a high-speed random access memory (RAM), and can also include a non-volatile memory, for example, at least one disk memory.

[0126] The processor 502 is configured to execute the computer execution instructions stored in the memory 501, so as to implement the method described in the foregoing method embodiments. The processor 502 can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0127] Optionally, the electronic device can further include a receiver 503 and a transmitter 504. In a specific implementation, if the receiver 503, the transmitter 504, the memory 501 and the processor 502 are independently implemented, the receiver 503, the transmitter 504, the memory 501 and the processor 502 can be connected with each other through a bus and complete communication among each other. The bus can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc., but does not mean that there is only one bus or only one type of bus.

[0128] The present application also provides a vehicle, and the vehicle includes an electronic device. The electronic device is configured to execute the method in the above embodiments.

[0129] The present application also provides a computer readable storage medium, and the computer readable storage medium stores computer program instructions. When a processor executes the computer program instructions, the scheme in the above embodiments is implemented.

[0130] ​The computer readable storage medium described above can be realized by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0131] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in a special integrated circuit.

[0132] The present application also provides a computer program product, comprising a computer program which, when executed by a processor, implements the solutions in the above embodiments.

[0133] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The foregoing program can be stored in a computer readable storage medium. The program, when executed, executes steps including the above-mentioned method embodiments; and the foregoing storage medium includes various storage media that can store program codes, such as magnetic disks or optical disks.

[0134] Finally, it should be noted that the above embodiments are only preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Any equivalent replacement or transformation of the present application based on the present application is within the protection scope of the present application.

Claims

1. An information recommendation method based on vehicle manuals, characterized in that: include: Real-time acquisition of vehicle status data, environmental data of the driving environment, and user behavior data; and determining a driving scene label corresponding to the vehicle and a user query intention based on the vehicle status data, environmental data, and user behavior data of the vehicle; Retrieving information to be recommended from a preset knowledge base based on the driving scenario tag and the user's query intent; wherein the preset knowledge base includes text units corresponding to the contents of each chapter in the vehicle manual; Determining a recommendation method for the information to be recommended based on the priority of the information to be recommended; wherein the priority represents the importance and urgency of the user obtaining the information to be recommended; The information to be recommended is recommended to the user of the vehicle according to the recommendation method of the information to be recommended.

2. The method according to claim 1, characterized in that The step of retrieving information to be recommended from a preset knowledge base based on the driving scene label and the user query intention includes: Generate a search condition based on the driving scene label and the user's query intention; Determining at least one candidate text unit from the preset knowledge base according to the search condition; The at least one candidate text unit is reordered to obtain the information to be recommended.

3. The method according to claim 2, characterized in that The retrieval condition includes a query vector representation representing the user's query intention; and determining at least one candidate text unit from the preset knowledge base based on the retrieval condition includes: Performing vectorization processing on each text unit in the preset knowledge base to obtain a text vector representation corresponding to the text unit; The at least one candidate text unit is determined from the preset knowledge base according to the vector similarity between the query vector representation in the retrieval condition and the text vector representation.

4. The method according to claim 2, characterized in that The reordering of the at least one candidate text unit to obtain the information to be recommended includes: Determining a first relevance corresponding to the candidate text unit based on the driving scene label; wherein the first relevance represents a relevance between the candidate text unit and the driving scene in which the current vehicle is located; Determining the query priority corresponding to the candidate text unit according to the user's query intention; wherein the query priority represents the urgency of the user's query for the candidate text unit; According to the first relevance corresponding to each candidate text unit and the query priority, each candidate text unit is reordered and screened to obtain the information to be recommended.

5. The method according to claim 1, wherein The determining, based on the vehicle state data, environment data, and user behavior data of the vehicle, a driving scene label corresponding to the vehicle and a user query intention, includes: Performing feature processing on the vehicle state data and the environment data of the vehicle to obtain vehicle feature data of the vehicle; Performing feature recognition processing on the vehicle feature data to obtain a driving scene label corresponding to the vehicle; Identify the user behavior data of the vehicle to obtain the user query intention corresponding to the vehicle.

6. The method according to claim 1, characterized in that The determining, based on the priority of the information to be recommended, a recommendation method for the information to be recommended, includes: A recommendation method for the information to be recommended is determined according to the priority of the information to be recommended and user preference data of the vehicle; wherein the user preference data represents a user preferred information receiving method.

7. The method according to claim 1, characterized in that Before determining the recommendation method of the information to be recommended according to the priority of the information to be recommended, the method further includes: Determining a second relevance corresponding to the information to be recommended; and determining a complexity corresponding to the information to be recommended; wherein the second relevance represents the relevance of the information to be recommended to the current driving scenario of the vehicle; and the complexity represents the operational complexity corresponding to the instruction manual represented by the information to be recommended; The priority of the information to be recommended is determined according to the second relevance and complexity corresponding to the information to be recommended.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: If it is determined that no user interaction operation is received within a preset time period, or if it is determined that an exit command input by the user is received, the vehicle-mounted display device in the vehicle is controlled to restore the original display state; wherein, the vehicle-mounted display device is used to display the information to be recommended to the user of the vehicle.

9. An information recommendation device based on vehicle manuals, characterized in that: include: A first determination module is used to obtain vehicle status data of the vehicle, environmental data of the driving environment, and user behavior data in real time; and determining a driving scene label corresponding to the vehicle and a user query intention based on the vehicle status data, environmental data, and user behavior data of the vehicle; a retrieval module, configured to retrieve information to be recommended from a preset knowledge base based on the driving scenario tag and the user's query intent; wherein the preset knowledge base includes text units corresponding to the contents of each chapter of the vehicle manual; A second determining module is configured to determine a recommendation method for the information to be recommended based on a priority of the information to be recommended; wherein the priority represents the importance and urgency of the user obtaining the information to be recommended; A recommendation module is used to recommend the information to be recommended to the user of the vehicle according to the recommendation method of the information to be recommended.

10. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

11. A vehicle, characterized in that: The vehicle comprises an electronic device; the electronic device is configured to execute the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 8 when executed by a processor.

Images