Human-cabin interaction scene identification method based on intelligent cabin and related device

Abstract

The invention relates to the technical field of vehicle scene interaction, in particular to a human cabin interaction scene recognition method based on an intelligent cabin and a related device. The method comprises the following steps: constructing a human-cabin interaction scene index system; obtaining operation information of a plurality of vehicles, and determining a target scene index; generating a score sequence of the target scene indexes; obtaining a human-cabin interaction scene label according to the data features of each score sequence; training a preset human-cabin interaction high-value scene recognition model according to each human-cabin interaction scene label and the operation information to obtain a trained human-cabin interaction high-value scene recognition model; and inputting operation information of a to-be-analyzed vehicle into the trained human-cabin interaction high-value scene recognition model, and outputting a human-cabin interaction scene label corresponding to the to-be-analyzed vehicle. According to the invention, the method can achieve the online recognition of the type of the human-cabin interaction scene in the vehicle driving process, and greatly improves the recognition efficiency and accuracy.

Description

Technical Field

[0001] This application relates to the field of vehicle scene interaction technology, and in particular to a method and related device for human-cabin interaction scene recognition based on a smart cockpit. Background Technology

[0002] In recent years, car cockpits have gradually transformed from traditional cockpits to intelligent cockpits, and users' perception of the value of cars has shifted from a means of transportation to a "third living space." Based on digital technology, various functions, systems, and devices within the intelligent cockpit are integrated and optimized to form a digital cockpit that displays diverse data such as vehicle status, environmental information, and entertainment content in real time, enabling drivers and passengers to enjoy a more intelligent and convenient service experience. Human-cockpit interaction scenarios are the foundation for the realization of digital cockpits. Combining multimodal data from various sources inside and outside the cockpit with artificial intelligence technology, intelligent cockpits will be able to meet users' needs in various human-cockpit interaction scenarios, including entertainment, social interaction, and work.

[0003] However, tens of thousands of human-cabin interaction scenarios are generated during daily driving. Mining user needs and intentions from these massive amounts of human-cabin interaction scenarios not only requires high computing power, but also struggles to guarantee the accuracy of user needs and intention recognition. The recognition results often only capture the general driving needs of all users, failing to delve into users' personalized preferences and specific needs. Therefore, identifying high-value human-cabin interaction scenarios is a crucial step for intelligent cockpits to provide accurate personalized recommendations. This allows for further in-depth mining of users' surface and deeper needs within these high-value scenarios, satisfying the unique driving needs of each individual.

[0004] In existing technologies, there is limited research on identifying high-value scenarios of human-cabin interaction in smart cockpits. Current solutions propose conducting surveys or on-vehicle interviews with target users to deeply explore their pain points and emotional changes in different driving scenarios, and further improve the functions and services of smart cockpits through in-depth expert interviews. However, these methods require researchers to communicate with a large number of users and analyze a large amount of data, requiring significant investment of human and time resources. This approach is not only inefficient but also susceptible to subjective factors, leading to biased results. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a method and related apparatus for identifying human-cabin interaction scenarios in a smart cockpit. This method enables automated classification of human-cabin interaction scenarios in a smart cockpit and allows for online identification of human-cabin interaction scenario categories during vehicle operation. This significantly improves the efficiency and accuracy of identification, thereby enhancing the smart cockpit's ability to provide personalized and intelligent proactive recommendation services.

[0006] According to one aspect of the embodiments of this application, a method for recognizing human-cabin interaction scenes based on a smart cockpit is proposed, the method comprising: A human-cabin interaction scenario indicator system is constructed, which includes N scenario indicators, each of which corresponds to a weight value. The operation information of multiple vehicles is obtained, and target scene indicators corresponding to each vehicle are determined based on the operation information of each vehicle. Each target scene indicator corresponds to M scene indicators, where M≤N. For each vehicle, the score of the target scenario indicator and the travel purpose of the vehicle are determined based on the vehicle's operation information, and a score sequence of the target scenario indicator is generated based on the score of the target scenario indicator, the weight value of the target scenario indicator, and the hotspot frequency of the travel purpose. Based on the data characteristics of each score sequence, human-cabin interaction scenario labels are divided to obtain human-cabin interaction scenario labels corresponding to each vehicle. The preset high-value human-cabin interaction scene recognition model is trained based on the human-cabin interaction scene labels corresponding to each vehicle and the operation information of each vehicle to obtain the trained high-value human-cabin interaction scene recognition model. The operating information of the vehicle to be analyzed is input into the trained human-cabin interaction high-value scene recognition model, and the human-cabin interaction scene label corresponding to the vehicle to be analyzed is output, so as to determine the human-cabin interaction scene information of the vehicle to be analyzed based on the human-cabin interaction scene label corresponding to the vehicle to be analyzed.

[0007] In the above scheme, the scenario indicators include primary indicators and secondary indicators. The primary indicators are related to time elements, user elements, vehicle elements, and environmental elements. The secondary indicators are related to the date attribute and time period corresponding to the time element. The secondary indicators are also related to the user attributes, user status, and non-vehicle behavior corresponding to the user element. The secondary indicators are also related to the vehicle status corresponding to the vehicle element. The secondary indicators are also related to the road conditions, distance, weather, and in-vehicle temperature range corresponding to the environmental element. Each primary indicator corresponds to a single weight value, and each secondary indicator corresponds to a single weight value.

[0008] In the above scheme, determining the target scenario indicators corresponding to each of the vehicles based on their operational information includes: For each vehicle, multiple scene factors corresponding to the vehicle are determined based on the vehicle's operating information, and scene indicators corresponding to each scene factor are determined one by one. The scene indicators corresponding to each scene factor are used as the target scene indicators.

[0009] In the above scheme, the score sequence includes multiple scores. Generating the score sequence of the target scenario indicator based on the score of the target scenario indicator, the weight value of the target scenario indicator, and the hotspot frequency of the travel destination includes: For each scenario indicator in the target scenario indicators, the product of the scenario indicator score, the scenario indicator weight value, and the hotspot frequency of the travel destination is used as the score of the scenario indicator. The scores of each scenario indicator are integrated to obtain the score sequence.

[0010] In the above scheme, each scenario indicator corresponds to a scenario factor, each scenario factor corresponds to a single attribute value, and the score of the scenario indicator is determined based on the attribute value corresponding to the scenario factor.

[0011] In the above scheme, the purpose of travel includes commuting, pick-up and drop-off, leisure and entertainment, business activities and other purposes, and the sum of the hotspot frequency of commuting, the hotspot frequency of pick-up and drop-off, the hotspot frequency of leisure and entertainment, the hotspot frequency of business activities and the hotspot frequency of other purposes is 1.

[0012] In the above scheme, training a preset high-value human-cabin interaction scenario recognition model based on the human-cabin interaction scenario labels corresponding to each vehicle and the operating information of each vehicle includes: The operation information of each vehicle is labeled to obtain a verification set, which includes real human-cabin interaction scene labels corresponding to each vehicle. Input the human-cabin interaction scene label corresponding to each of the vehicles into the preset human-cabin interaction high-value scene recognition model to obtain the human-cabin interaction scene prediction label output by the preset human-cabin interaction high-value scene recognition model; For each of the aforementioned vehicles, the error value between the real human-cabin interaction scene label corresponding to the vehicle and the predicted human-cabin interaction scene label is determined; The model weights of the preset high-value human-cabin interaction scene recognition model are adjusted according to the error values ​​corresponding to each of the vehicles, so as to train the preset high-value human-cabin interaction scene recognition model.

[0013] According to one aspect of the embodiments of this application, a human-cabin interaction scene recognition device based on a smart cockpit is proposed, the device comprising: A construction unit is used to construct a human-cabin interaction scenario indicator system, which includes N scenario indicators, each of which corresponds to a weight value. An acquisition unit is used to acquire the operating information of multiple vehicles and determine the target scene indicators corresponding to each vehicle based on the operating information of each vehicle. Each target scene indicator corresponds to M scene indicators, where M≤N. The determining unit is configured to, for each vehicle, determine the score of the target scenario indicator and the travel purpose of the vehicle based on the vehicle's operating information, and generate a score sequence of the target scenario indicator based on the score of the target scenario indicator, the weight value of the target scenario indicator, and the hotspot frequency of the travel purpose. The segmentation unit is used to segment human-cabin interaction scene labels according to the data features of each score sequence, so as to obtain human-cabin interaction scene labels corresponding to each vehicle. The training unit is used to train a preset high-value human-cabin interaction scene recognition model based on the human-cabin interaction scene labels corresponding to each vehicle and the operation information of each vehicle, so as to obtain the trained high-value human-cabin interaction scene recognition model. The output unit is used to input the operating information of the vehicle to be analyzed into the trained human-cabin interaction high-value scene recognition model, and output the human-cabin interaction scene label corresponding to the vehicle to be analyzed, so as to determine the human-cabin interaction scene information of the vehicle to be analyzed based on the human-cabin interaction scene label corresponding to the vehicle to be analyzed.

[0014] According to one aspect of the embodiments of this application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the human-cabin interaction scene recognition method based on a smart cockpit as described above. According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including a computer program, the computer program being read and executed by a processor of an electronic device, causing the electronic device to perform the human-cabin interaction scene recognition method based on a smart cockpit as described above.

[0015] The beneficial effects of this application are as follows: A human-cabin interaction scenario indicator system is constructed. By acquiring vehicle operation information, the travel purpose of the vehicle and the score of the corresponding target scenario are determined. A score sequence of target scenario indicators is generated by combining the weight value of the target scenario and the frequency of travel purpose hotspots. Then, human-cabin interaction scenario labels are segmented based on the data characteristics corresponding to the score sequence, thereby generating human-cabin interaction scenario labels for each vehicle. While automatically segmented labels may have insufficient accuracy, a pre-set high-value human-cabin interaction scenario recognition model can be trained using vehicle operation information combined with the segmented human-cabin interaction scenario labels. The trained high-value human-cabin interaction scenario recognition model has more accurate recognition of human-cabin interaction scenarios. Therefore, inputting the operation information of the vehicle to be analyzed into the trained high-value human-cabin interaction scenario recognition model results in more authentic human-cabin interaction scenario labels for the vehicle to be analyzed. Thus, the human-cabin interaction scenario information of the vehicle to be analyzed can be obtained through these labels. This application enables automated classification of human-cabin interaction scenarios in smart cockpits, allowing for online identification of human-cabin interaction scenario categories during vehicle operation. This significantly improves the efficiency and accuracy of identification, thereby enhancing the smart cockpit's ability to provide personalized and intelligent proactive recommendation services. Attached Figure Description

[0016] Figure 1 This is a system architecture diagram of the human-cabin interaction scene recognition method based on the intelligent cockpit provided in the embodiments of this application; Figure 2 A flowchart illustrating the human-cabin interaction scene recognition method based on an embodiment of this application; Figure 3 An architecture diagram of the high-value human-cabin interaction scene recognition model provided in the embodiments of this application; Figure 4 A block diagram of a human-cabin interaction scene recognition device based on an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0017] To enable those skilled in the art to better understand the solutions of this application, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] It should be noted that while some processes described in the specification, claims, and accompanying drawings include multiple steps appearing in a specific order, it should be clearly understood that these steps may not be performed in the order they appear herein, or may be performed in parallel. The step numbers are merely used to distinguish different steps and do not themselves represent any execution order. Furthermore, descriptions such as "first," "second," or "objective" in this document are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. "Multiple" in this document refers to at least two.

[0019] It is worth noting that in the specific embodiments of this application, vehicle operation information and other related data are involved. When the above embodiments of this application are applied to specific products or technologies, permission or consent from the target object is required, and the collection, use, and processing of related data must comply with relevant laws, regulations, and standards. For example, when this application embodiment needs to obtain vehicle operation information and other related data, separate permission or consent from the target object can be obtained through pop-up windows or redirection to a confirmation page. After obtaining the separate permission or consent from the target object, the necessary vehicle operation information and other related data for enabling the embodiments of this application to operate normally can then be obtained.

[0020] Please see Figure 1 , Figure 1 This is a system architecture diagram of the human-cabin interaction scene recognition method based on the intelligent cockpit provided in this application embodiment. It includes a terminal 140, an Internet connection 130, a gateway 120, a server 110, etc.

[0021] Terminal 140 can take various forms, including desktop computers, laptops, PDAs (personal digital assistants), mobile phones, vehicle terminals, and dedicated terminals. Furthermore, it can be a single device or a collection of multiple devices. For example, multiple desktop computers can be interconnected via a local area network, sharing a single monitor to work collaboratively, forming a single terminal 140. Terminal 140 can communicate with the Internet 130 via wired or wireless means to exchange data.

[0022] Server 110 refers to a computer system capable of providing certain services to terminal 140. Compared to ordinary terminal 140, server 110 has higher requirements in terms of stability, security, and performance. Server 110 can be a single high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a virtual machine), or a combination of portions of multiple high-performance computers (e.g., virtual machines). Server 110 can also communicate with the Internet 130 via wired or wireless means to exchange data.

[0023] Gateway 120, also known as an internetwork connector or protocol converter, is a computer system or device that acts as a translator, enabling network interconnection at the transport layer. It bridges the gap between two systems using different communication protocols, data formats, languages, or even completely different architectures. Gateways can also provide filtering and security functions. Messages sent from terminal 140 to server 110 are forwarded to the corresponding server 110 via gateway 120. Messages sent from server 110 to terminal 140 are also forwarded to the corresponding terminal 140 via gateway 120.

[0024] The following provides a detailed description of the specific implementation methods of the embodiments of this application: Please see Figure 2 , Figure 2 This is a flowchart illustrating the human-cabin interaction scene recognition method based on an embodiment of the present application. The human-cabin interaction scene recognition method based on an intelligent cockpit can be implemented by server 110 and / or terminal 140. Figure 2 The demonstrated method for recognizing human-cabin interaction scenarios based on smart cockpits includes: Step 210: Construct a human-cabin interaction scenario indicator system, which includes N scenario indicators, each of which corresponds to a weight value. Step 220: Obtain the operation information of multiple vehicles, and determine the target scene indicators corresponding to each vehicle based on the operation information of each vehicle. Each target scene indicator corresponds to M scene indicators, where M≤N. Step 230: For each vehicle, determine the score of the target scenario indicator and the travel purpose of the vehicle based on the vehicle's operation information, and generate a score sequence of the target scenario indicator based on the score of the target scenario indicator, the weight value of the target scenario indicator, and the hotspot frequency of the travel purpose. Step 240: Divide the human-cabin interaction scene labels according to the data characteristics of each score sequence to obtain the human-cabin interaction scene labels corresponding to each vehicle; Step 250: Train the preset high-value human-cabin interaction scene recognition model based on the human-cabin interaction scene labels corresponding to each vehicle and the operation information of each vehicle to obtain the trained high-value human-cabin interaction scene recognition model. Step 260: Input the operating information of the vehicle to be analyzed into the trained human-cabin interaction high-value scene recognition model, and output the human-cabin interaction scene label corresponding to the vehicle to be analyzed, so as to determine the human-cabin interaction scene information of the vehicle to be analyzed based on the human-cabin interaction scene label corresponding to the vehicle to be analyzed.

[0025] The complete embodiment of this application will be explained in detail below with reference to steps 210-260: In step 210, the control logic of the human-cabin interaction scene recognition method based on the intelligent cockpit proposed in this application can be deployed in any vehicle. The high-value human-cabin interaction scene proposed in this application refers to the key scene that can improve the driving experience to a greater extent through the adjustment of the intelligent cockpit interaction device among all possible scenarios generated by the interaction between the user and the intelligent cockpit, or the scene that is of high importance in the user's daily driving process.

[0026] The data is sliced ​​according to different VINs and different trips. We then form a standardized and regulated human-cabin interaction scenario for each of these trip segments, which we define as S:

[0027] In step 220, the human-cabin interaction scenario includes, but is not limited to, the following scenario factors: obtained based on GPS geographic location information. Purpose of vehicle use; obtained based on GPS time information Date attribute, Time period; other information obtained directly from the vehicle system. Childhood status, Fatigue state Distraction Emotional state Non-vehicle use behavior Remaining battery power Fault status, Road conditions, Distance, weather, In-vehicle temperature range. Attribute values ​​for each scenario factor are obtained based on vehicle operation information. The combination of all scenario factors and their attribute values ​​forms human-cabin interaction scenario information, used to describe the environmental state during vehicle operation.

[0028] Construct a human-cabin interaction scenario indicator system, and calculate the weight allocation of each indicator using the analytic hierarchy process: remove In addition to the scenario factors of vehicle usage purpose, other factors will be considered. ~ After classifying the scenario factors, a human-cabin interaction scenario indicator system was constructed, including primary and secondary indicators, as shown in Table 1. Primary indicators include time elements, user elements, vehicle elements, and environmental elements; secondary indicators include, but are not limited to, those corresponding to the time element. Date attribute, Time period, corresponding to user element User attributes, User status, Non-vehicle usage behavior, vehicle element corresponding Vehicle status, corresponding environmental elements Road conditions, Distance, weather, Temperature range inside the vehicle.

[0029]

[0030] Table 1 Based on the Analytic Hierarchy Process (AHP), and through expert scoring, a judgment matrix is ​​constructed by comparing the elements of each indicator in the primary and secondary indicators using a matrix element scaling method.

[0031] The elements in matrix A satisfy: , , ; The weight vector and the largest eigenvalue of the judgment matrix A are calculated using the arithmetic mean method:

[0032]

[0033] Perform a consistency check on the constructed judgment matrix:

[0034]

[0035] If CR < 0.1, the consistency of the judgment matrix A is considered to be within the acceptable range; if CR ≥ 0.1, there is a logical error in the construction of the judgment matrix, and the element values ​​of the judgment matrix need to be readjusted and the weight vector calculated; after hierarchical overall sorting, the weight values ​​of all factors of the secondary indicators relative to the overall goal are calculated.

[0036] In step 230, based on vehicle operation information, each secondary indicator in the human-cabin interaction scenario indicator system has different values. For special vehicle use scenarios that appear in the indicator values, such as weekday travel, nighttime travel, driver fatigue, insufficient vehicle range, and rainy travel, the pain points and needs that drivers and passengers may experience when such situations occur in actual vehicle use are analyzed, which leads to a poor vehicle use experience.

[0037] Based on pain point needs analysis, a subjective evaluation questionnaire was designed using methods such as on-site inspections, expert consultations, and field surveys to obtain the urgency of user needs and the degree of negative experience when a certain situation occurs during vehicle use. Referring to the Likert scale structure, a five-level evaluation result was designed, corresponding to scores of 100, 85, 70, 55, and 40. The average of all expert scores was calculated as the score corresponding to different values ​​of the secondary indicators. The secondary indicator values ​​are allowed to overlap; for example, if a user is both fatigued and distracted, the corresponding scores are added together to obtain the indicator score. For other values ​​of the secondary indicators corresponding to daily driving scenarios, such as daytime or sunny travel, or when the user is in a normal state, the indicator score is recorded as 0.

[0038] Furthermore, based on the travel purposes collected from ordinary users through the online car information platform, the frequency of hot spots for each travel purpose was calculated to indicate the importance of the user's travel; the frequency of travel purposes for 40 car models, including different models, brands, and price ranges, was statistically analyzed, as shown in Table 2.

[0039]

[0040] Table 2 Calculate the score sequence of human-cabin interaction scenarios m is the number of secondary indicators in the human-cabin interaction scenario indicator system. This represents the score for a specific scenario indicator that belongs to a secondary indicator for a particular vehicle. ; Frequency of hotspots for travel purposes; These are the weight values ​​corresponding to the secondary indicators; The score corresponds to the secondary indicator.

[0041] In step 240, all scene data are divided into two categories using K-means clustering: high-value scenes and ordinary scenes. The score sequence of the human-cabin interaction scenario was used as a clustering sample. Select two cluster centers, calculate the distance between the sample data and the cluster centers, and assign the sample data to the j-th category represented by the nearest cluster center. middle:

[0042] in, This indicates the calculation of the Euclidean distance between a sample and its cluster center. Indicates sample The class that is closest to the other class. This represents the i-th sample data. This represents the j-th cluster center; Take the mean of the categories and use the mean as the new cluster center:

[0043] Repeat the above process to continuously iterate and update the cluster centers; analyze the characteristic parameters of the cluster centers, and classify the human-cabin interaction scenario labels according to the data characteristics, defining them as high-value scenarios and ordinary scenarios respectively.

[0044] A high-value human-cabin interaction scene recognition model based on DNN deep neural network: K-means clustering was used to generate category labels for human-cabin interaction scenarios. Vehicle operation information was labeled to form a dataset, which was then divided into training, validation, and test sets according to a certain ratio (e.g., 8:1:1). Max-min normalization was performed on the sample data to map the values ​​to the [0,1] interval, thereby achieving fast and stable convergence of the model during training.

[0045] Using experimental data from the training set, an offline binary classification DNN network is trained to establish a high-value human-cabin interaction scenario recognition model. The data to be classified is processed by the DNN to output a binary classification prediction result, which indicates the type of human-cabin interaction scenario during vehicle operation, facilitating subsequent judgment of the type and timing of proactive recommendation services in the intelligent cockpit.

[0046] Given a DNN network output function f, inputting data into the network yields a classification result:

[0047] in, It is the model's predicted classification result. These are the parameters of the DNN network.

[0048] The initial DNN network structure consists of two hidden layers, one with 24 neurons and the other with 10 neurons. The hidden layers feed the extracted features into the output layer, which contains two neurons, corresponding to high-value and ordinary scenes. Finally, the softmax layer outputs the classification result with the highest confidence. The DNN network structure is as follows: Figure 3 As shown.

[0049] DNN networks use stochastic gradient descent (SGD) as the optimizer, and this optimization method can be expressed as:

[0050] Where t represents a single iteration; The weights are optimized, generally referred to as the learning rate, initially chosen to be 0.01; g represents the stochastic gradient, whose expectation is the gradient of f, i.e., satisfying... The DNN network is trained iteratively a certain number of times using the above algorithm to fully learn the features of the input data, thereby accurately classifying the vehicle online operation data. It should be noted that the DNN network mentioned in this embodiment refers to the high-value human-cabin interaction scene recognition model described in this application.

[0051] The loss function for the DNN network is set to the cross-entropy loss function, and the calculation formula is as follows:

[0052] The training process of the DNN network (i.e., the human-cabin interaction high-value scene recognition model described in this application) is an optimization process, aiming to minimize the loss function and make the network model's predictions as close as possible to the true labels. In the trained model, the accuracy metric is used to evaluate model performance, calculated as follows:

[0053] Train the DNN model using the method described above, initially selecting 300 epochs and a batch size of 64. The DNN network can be adjusted and optimized by adjusting multiple parameters such as the number of neural network layers, the number of neural network nodes, the ratio of training set to test set, the batch size, the number of epochs, and the optimization algorithm to improve the model's recognition accuracy.

[0054] Furthermore, the trained DNN network model was tested using data from the test set. The DNN network model that passed the test accuracy was used to identify the category of human-cabin interaction scenario in which the vehicle was driving online.

[0055] In step 260, the operating information of the vehicle to be analyzed is input into the human-cabin interaction high-value scenario recognition model, and the human-cabin interaction high-value scenario classification is identified online: After receiving vehicle operation information data, the cloud server preprocesses the vehicle operation information data into a standard input format based on the human-cabin interaction high-value scenario recognition model trained offline in step five. The recognition model then determines the type of human-cabin interaction scenario in this time series, realizing online recognition of high-value human-cabin interaction scenarios. The recognition results can be further analyzed in conjunction with the intelligent cockpit proactive recommendation service module to achieve personalized recommendation services.

[0056] In summary, this application proposes a high-value scenario identification scheme for human-cabin interaction in intelligent cockpits, which can classify human-cabin interaction scenarios in intelligent cockpits, providing support for subsequent intelligent cockpit proactive recommendation service modules and enabling accurate and personalized recommendations for users.

[0057] This application proposes a human-cabin interaction scenario indicator system. Through surveys and interviews with industry experts, it is possible to obtain in-depth insights and experience from industry experts with relatively little effort, resulting in more targeted and accurate evaluation results. Furthermore, the proposed indicator system can be quickly and easily expanded when new scenario elements emerge.

[0058] This application proposes a method for establishing a high-value human-cabin interaction scenario recognition model by combining K-means clustering and DNN deep neural networks. This method enables online recognition of human-cabin interaction scenario categories during vehicle operation, significantly improving the efficiency and accuracy of recognition, thereby enhancing the ability of smart cockpits to provide personalized and intelligent proactive recommendation services.

[0059] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a cockpit-based human-cabin interaction scene recognition device provided in an embodiment of this application. The cockpit-based human-cabin interaction scene recognition device is applied to a computer device, and may include: The construction unit 401 is used to construct a human-cabin interaction scenario indicator system, which includes N scenario indicators, each of which corresponds to a weight value. The acquisition unit 402 is used to acquire the operation information of multiple vehicles and determine the target scene indicators corresponding to each vehicle based on the operation information of each vehicle. Each target scene indicator corresponds to M scene indicators, where M≤N. The determining unit 403 is used to determine the score of the target scenario indicator and the travel purpose of the vehicle based on the vehicle's operation information for each vehicle, and to generate a score sequence of the target scenario indicator based on the score of the target scenario indicator, the weight value of the target scenario indicator and the hotspot frequency of the travel purpose. The segmentation unit 404 is used to segment human-cabin interaction scene labels according to the data features of each score sequence, so as to obtain human-cabin interaction scene labels corresponding to each vehicle. The training unit 405 is used to train a preset high-value human-cabin interaction scene recognition model based on the human-cabin interaction scene labels corresponding to each vehicle and the operation information of each vehicle, so as to obtain the trained high-value human-cabin interaction scene recognition model. The output unit 406 is used to input the operation information of the vehicle to be analyzed into the trained human-cabin interaction high-value scene recognition model, and output the human-cabin interaction scene label corresponding to the vehicle to be analyzed, so as to determine the human-cabin interaction scene information of the vehicle to be analyzed based on the human-cabin interaction scene label corresponding to the vehicle to be analyzed.

[0060] Reference Figure 5 , Figure 5 To implement the structural block diagram of a portion of the terminal 140 in this application embodiment, the terminal 140 includes: a radio frequency (RF) circuit 710, a memory 715, an input unit 730, a display unit 740, a sensor 750, an audio circuit 760, a wireless fidelity (WiFi) module 770, a processor 780, and a power supply 790, among other components. Those skilled in the art will understand that... Figure 5 The terminal 140 structure shown does not constitute a limitation on a mobile phone or computer, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0061] The RF circuit 710 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 780; in addition, it transmits uplink data to the base station.

[0062] The memory 715 can be used to store software programs and modules. The processor 780 executes various functional applications of the terminal and human-cabin interaction scene recognition processing based on the smart cockpit by running the software programs and modules stored in the memory 715.

[0063] The input unit 730 can be used to receive input numeric or character information, and to generate key signal inputs related to the terminal's settings and function control. Specifically, the input unit 730 may include a touch panel 731 and other input devices 732.

[0064] The display unit 740 can be used to display input or provided information, as well as various menus of the terminal. The display unit 740 may include a display panel 741.

[0065] Audio circuitry 760, speaker 761, and microphone 762 provide an audio interface.

[0066] In this embodiment, the processor 780 included in the terminal 140 can execute the human-cabin interaction scene recognition method based on the smart cockpit described in the previous embodiment.

[0067] The terminal 140 in this application embodiment includes, but is not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, and aircraft. This application embodiment can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0068] Figure 6 This is a partial structural block diagram of a server 110 implementing an embodiment of this application. The server 110 can vary significantly due to different configurations or performance characteristics, and may include one or more central processing units (CPUs) 822 (e.g., one or more processors) and memory 832, and one or more storage media 830 (e.g., one or more mass storage devices) for storing application programs 842 or data 844. The memory 832 and storage media 830 can be temporary or persistent storage. The program stored in the storage media 830 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server 110. Furthermore, the CPU 822 may be configured to communicate with the storage media 830 and execute the series of instruction operations in the storage media 830 on the server 110.

[0069] Server 110 may also include one or more power supplies 826, one or more wired or wireless network interfaces 850, one or more input / output interfaces 858, and / or one or more operating systems 841, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0070] The central processing unit 822 in server 110 can be used to execute the human-cabin interaction scene recognition method based on the intelligent cockpit of this application embodiment.

[0071] This application also provides a computer-readable storage medium for storing program code, which is used to execute the human-cabin interaction scene recognition method based on the aforementioned embodiments.

[0072] This application also provides a computer program product, which includes a computer program. The processor of a computer device reads and executes the computer program, causing the computer device to perform the aforementioned human-cabin interaction scene recognition method based on an intelligent cockpit.

[0073] Furthermore, the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or apparatus.

[0074] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0075] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0076] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0077] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0078] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0079] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0080] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0081] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0082] The above is a detailed description of the embodiments of this application. However, this application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for identifying a human-vehicle interaction scene based on an intelligent cockpit, characterized in that, The method comprises: constructing a human-cabin interaction scene index system, the human-cabin interaction scene index system comprising N scene indexes, each scene index corresponding to a weight value; obtaining running information of a plurality of vehicles and determining, based on the running information of each vehicle, a target scene index corresponding to each vehicle respectively, each target scene index corresponding to M scene indexes, M≤N; for each vehicle, determining a score of the target scene index and a travel purpose of the vehicle according to the running information of the vehicle, and generating a score sequence of the target scene index based on the score of the target scene index, the weight value of the target scene index, and the hotspot frequency of the travel purpose; performing human-cabin interaction scene label division according to the data characteristics of each score sequence to obtain a human-cabin interaction scene label corresponding to each vehicle; training a preset human-cabin interaction high-value scene recognition model according to the human-cabin interaction scene label corresponding to each vehicle and the running information of each vehicle to obtain a trained human-cabin interaction high-value scene recognition model; inputting the running information of a vehicle to be analyzed into the trained human-cabin interaction high-value scene recognition model to output a human-cabin interaction scene label corresponding to the vehicle to be analyzed, so as to determine human-cabin interaction scene information of the vehicle to be analyzed according to the human-cabin interaction scene label corresponding to the vehicle to be analyzed.

2. The smart cockpit-based cabin interaction scenario recognition method according to claim 1, characterized in that, The scene indexes comprise primary indexes and secondary indexes, the primary indexes are related to time elements, user elements, vehicle elements, and environment elements, the secondary indexes are related to date attributes and time periods corresponding to the time elements, the secondary indexes are also related to user attributes, user states, and non-vehicle-use behaviors corresponding to the user elements, the secondary indexes are also related to vehicle states corresponding to the vehicle elements, and the secondary indexes are also related to road conditions, distances, weather, and temperature intervals in vehicles corresponding to the environment elements; the primary indexes correspond to single weight values, and the secondary indexes correspond to single weight values.

3. The smart cockpit-based cabin interaction scenario recognition method of claim 1, wherein, The determination of the target scene index corresponding to each vehicle based on the running information of each vehicle comprises: for each vehicle, determining a plurality of scene factors corresponding to the vehicle according to the running information of the vehicle, and determining a scene index corresponding to each scene factor respectively, and taking the scene index corresponding to each scene factor as the target scene index.

4. The smart cockpit-based cabin interaction scenario recognition method according to claim 3, characterized in that, The score sequence comprises a plurality of scores, and the generation of the score sequence of the target scene index based on the score of the target scene index, the weight value of the target scene index, and the hotspot frequency of the travel purpose comprises: for each scene index in the target scene index, taking the product of the score of the scene index, the weight value of the scene index, and the hotspot frequency of the travel purpose as the score of the scene index; integrating the scores of each scene index to obtain the score sequence.

5. The smart cockpit-based cabin interaction scenario recognition method according to claim 4, characterized in that, Each of the scene indicators corresponds to a scene factor, each of the scene factors corresponds to a single attribute value, and a score of the scene indicator is determined according to the attribute value corresponding to the scene factor.

6. The smart cockpit-based cabin interaction scenario recognition method of claim 4, wherein The travel purposes include commuting, pick-up and drop-off, leisure and entertainment, business activities, and other purposes, and a sum of a hot spot frequency of the commuting, a hot spot frequency of the pick-up and drop-off, a hot spot frequency of the leisure and entertainment, a hot spot frequency of the business activities, and a hot spot frequency of the other purposes is 1.

7. The smart cockpit-based cabin interaction scenario recognition method according to claim 1, characterized in that, The training of the preset human-cabin interaction high-value scene recognition model according to the human-cabin interaction scene labels corresponding to each of the vehicles and the operation information of each of the vehicles comprises: annotating the operation information of each of the vehicles to obtain a verification set, the verification set comprising the real human-cabin interaction scene labels corresponding to each of the vehicles; inputting the human-cabin interaction scene labels corresponding to each of the vehicles into the preset human-cabin interaction high-value scene recognition model to obtain human-cabin interaction scene prediction labels output by the preset human-cabin interaction high-value scene recognition model; for each of the vehicles, determining an error value of the real human-cabin interaction scene label corresponding to the vehicle and the human-cabin interaction scene prediction label; adjusting model weights of the preset human-cabin interaction high-value scene recognition model according to the error values corresponding to each of the vehicles to train the preset human-cabin interaction high-value scene recognition model.

8. An intelligent cockpit-based cabin interaction scene recognition device, characterized in that, The device comprises: a construction unit configured to construct a human-cabin interaction scene indicator system, the human-cabin interaction scene indicator system comprising N scene indicators, each of the scene indicators corresponding to a weight value; an acquisition unit configured to acquire operation information of a plurality of vehicles, and determine target scene indicators corresponding to each of the vehicles based on the operation information of each of the vehicles, each of the target scene indicators corresponding to M of the scene indicators, M≤N; a determination unit configured to, for each of the vehicles, determine scores of the target scene indicators and travel purposes of the vehicle according to the operation information of the vehicle, and generate a score sequence of the target scene indicators based on the scores of the target scene indicators, the weight values of the target scene indicators, and hot spot frequencies of the travel purposes; a division unit configured to divide human-cabin interaction scene labels according to data features of each of the score sequences to obtain human-cabin interaction scene labels corresponding to each of the vehicles; a training unit configured to train a preset human-cabin interaction high-value scene recognition model according to the human-cabin interaction scene labels corresponding to each of the vehicles and the operation information of each of the vehicles to obtain a trained human-cabin interaction high-value scene recognition model; an output unit configured to input operation information of a vehicle to be analyzed into the trained human-cabin interaction high-value scene recognition model, and output human-cabin interaction scene labels corresponding to the vehicle to be analyzed to determine human-cabin interaction scene information of the vehicle to be analyzed according to the human-cabin interaction scene labels corresponding to the vehicle to be analyzed. 9.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor implements the computer program to realize the intelligent cockpit-based human cabin interaction scene identification method in any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that, The computer program is read and executed by the processor of the electronic device, so that the electronic device executes the intelligent cockpit-based human cabin interaction scene identification method in any one of claims 1 to 7.

Images