A vehicle cockpit interaction system and method based on cognitive big model

Abstract

The present invention discloses a vehicle-mounted cockpit interaction system and method based on a cognitive big model. The vehicle-mounted cockpit interaction system based on a cognitive big model includes an interaction data acquisition module, an interaction data analysis module and an interaction execution module. The present invention obtains voice interaction data, gesture interaction data and real-time interaction data of the vehicle to be tested through the interaction data acquisition module, and then the interaction data analysis module constructs an interaction recognition data set in combination with the received data, and constructs a first interaction recognition model, and then screens the first interaction recognition model according to the recognition performance index to obtain an interaction recognition model, and obtains the cockpit interaction analysis results corresponding to the real-time interaction data according to the interaction recognition model, and finally the interaction execution module provides corresponding functional services according to the received cockpit interaction analysis results, thereby reducing the influence of the in-vehicle environment on the accuracy of the in-vehicle cockpit interaction recognition, and solving the problem in the prior art that the in-vehicle environment has a great influence on the accuracy of the in-vehicle cockpit interaction recognition.

Description

Technical Field

[0001] The present invention relates to the field of intelligent cockpit technology, and in particular to a vehicle-mounted cockpit interaction system and method based on a cognitive large model. Background Art

[0002] In-vehicle cockpit interaction refers to the process of information exchange and control operations between the driver and passengers and the vehicle system, entertainment equipment and other information sources inside the car. In-vehicle cockpit interaction can improve user experience and driving safety. At the same time, the in-vehicle cockpit interaction system can integrate various functions to provide more convenience for users. Cognitive big models refer to a class of complex computational models that are dedicated to simulating the complex psychological processes of humans when processing information in order to better understand and respond to various inputs. Cognitive big models are mainly used in natural language processing, computer vision, intelligent interactive systems and autonomous driving. For example, in the field of autonomous driving, traffic safety can be improved by understanding the traffic environment and making real-time decisions.

[0003] In the prior art, common methods for realizing vehicle-mounted cockpit interaction include touch screen interface interaction, voice interaction, gesture interaction and physical button interaction. Touch screen interface interaction controls various functions of the vehicle through the touch screen, voice interaction is to control various functions of the vehicle through voice commands, gesture interaction is to control by recognizing gestures, and physical button interaction is used to adjust the temperature and volume in the vehicle, etc.

[0004] For example, the invention patent with announcement number: CN112241204B announces a gesture interaction method and system for a vehicle-mounted AR-HUD, including: processing images through a gesture recognition model to output hand position information and gesture categories; obtaining point cloud information in the vehicle cabin; determining gesture coordinates based on the hand position information and the point cloud information; determining the AR screen corresponding to the gesture interaction operation based on the gesture coordinates and the coordinates of the AR-HUD projected screen; and performing corresponding interaction operations based on the gesture category and the AR screen corresponding to the gesture interaction operation.

[0005] For example, the invention patent application with publication number: CN115657922A discloses a method and device for comprehensively evaluating the safety of human-computer interaction in an intelligent cockpit, including: collecting information on the time of sight away from the road, operation frequency, subjective load and / or pupil diameter change during the interaction with the cockpit interaction device, obtaining multiple sub-safety levels and the limit threshold of each sub-safety level under the visual distraction dimension, operation distraction dimension and / or cognitive distraction dimension of human-computer interaction, determining multiple safety levels and the limit threshold of each safety level under the comprehensive evaluation dimension, generating an evaluation standard for the comprehensive evaluation dimension, and constructing an evaluation model to obtain a comprehensive evaluation model for human-computer interaction safety.

[0006] However, in the process of implementing the technical solution of the invention in the embodiments of the present application, the present application found that the above technology has at least the following technical problems:

[0007] In the prior art, voice interaction and gesture interaction methods require accurate recognition of interaction information, but the in-vehicle environment has a great influence on the accurate acquisition and recognition of interaction information. There is a problem that the in-vehicle environment has a great influence on the accuracy of vehicle-mounted cockpit interaction recognition. Summary of the invention

[0008] The embodiments of the present application provide a vehicle-cockpit interaction system and method based on a cognitive big model, thereby solving the problem in the prior art that the in-vehicle environment has a great influence on the accuracy of vehicle-cockpit interaction recognition, and reducing the influence of the in-vehicle environment on the accuracy of vehicle-cockpit interaction recognition.

[0009] The embodiment of the present application provides a vehicle-mounted cockpit interaction system based on a cognitive big model, and the vehicle-mounted cockpit interaction system based on a cognitive big model includes: an interaction data acquisition module, an interaction data analysis module and an interaction execution module; wherein the interaction data acquisition module is used to acquire the cockpit interaction data of the vehicle to be tested, and send the acquired cockpit interaction data to the interaction data analysis module, the cockpit interaction data is used to describe the cockpit interaction commands sent by the user of the vehicle to be tested, and the cockpit interaction commands are used to describe the functional service requirements expressed by the user of the vehicle to be tested through a preset cockpit interaction method during the ride, and the preset cockpit interaction method includes a voice interaction method and a gesture interaction method; the interaction data analysis module is used to analyze the received cockpit interaction data in combination with the trained interaction recognition model to obtain a cockpit interaction analysis result, and send the obtained result to the interaction execution module, the interaction recognition model is used to identify the functional service requirements corresponding to the input cockpit interaction data; the interaction execution module is used to receive the cockpit interaction analysis result sent by the interaction data analysis module, and provide corresponding functional services according to the cockpit interaction analysis result.

[0010] Furthermore, the interaction data acquisition module includes a voice interaction data acquisition unit, a gesture interaction data acquisition unit and a real-time interaction data acquisition unit; the voice interaction data acquisition unit is used to: acquire the voice interaction data of the vehicle to be tested, the voice interaction data is used to describe the voice interaction commands in the voice interaction simulation experiment, and the voice interaction commands are used to describe the functional service requirements expressed by the user of the vehicle to be tested through voice interaction during the ride; the gesture interaction data acquisition unit is used to: acquire the gesture interaction data of the vehicle to be tested, the gesture interaction data is used to describe the gesture interaction commands in the designed gesture interaction simulation experiment, and the gesture interaction commands are used to describe the functional service requirements expressed by the user of the vehicle to be tested through gesture interaction in the reference gesture interaction area; the real-time interaction data acquisition unit is used to: acquire the real-time interaction data of the vehicle to be tested, the real-time interaction data is used to describe the real-time interaction commands sent by the user of the vehicle to be tested within a preset time period, and the real-time interaction commands are used to describe the functional service requirements expressed by the user of the vehicle to be tested through a preset cockpit interaction method.

[0011] Furthermore, the voice interaction simulation experiment includes a noise intensity simulation experiment, a voice intensity simulation experiment and a voice interaction distance simulation experiment, and the specific experimental method is as follows: obtaining voice interaction control parameters, the voice interaction control parameters including noise intensity, interactive voice intensity and interactive voice distance, the noise intensity is used to describe the noise decibels in the vehicle to be tested when the voice interaction command is sent, the interactive voice intensity is used to describe the voice decibels when the voice interaction command is sent, and the interactive voice distance is used to describe the distance between the voice interaction command sending position and the reference position of the voice interaction command receiving device; obtaining first voice interaction control data, the first voice interaction control data is a reference value corresponding to the voice interaction control parameter, specifically including the first noise intensity, the first interactive voice intensity and the interactive voice distance. degree and the first interactive speech distance; conduct a noise intensity simulation experiment, the noise intensity simulation experiment is used to describe the experiment of changing the noise intensity under the first interactive speech intensity and the first interactive speech distance, and the noise intensity varies within the reference noise intensity range; conduct a speech intensity simulation experiment, the speech intensity simulation experiment is used to describe the experiment of changing the interactive speech intensity under the first noise intensity and the first interactive speech distance, and the interactive speech intensity varies within the reference interactive speech intensity range; conduct a speech interaction distance simulation experiment, the speech interaction distance simulation experiment is used to describe the experiment of changing the interactive speech distance under the condition that the first noise intensity and the first interactive speech intensity remain unchanged, and the interactive speech distance varies within the reference interactive speech distance range.

[0012] Furthermore, the gesture interaction simulation experiment includes a light intensity simulation experiment, a gesture interaction angle simulation experiment and a gesture interaction distance simulation experiment, and the specific design method is as follows: obtain gesture interaction control parameters, the gesture interaction control parameters include light intensity, gesture interaction angle and gesture interaction distance, the light intensity is used to describe the light intensity in the vehicle to be tested when the gesture interaction command is sent, the gesture interaction angle is used to describe the angle formed by the position where the gesture interaction command is issued and the reference position of the gesture interaction command receiving device, and the gesture interaction distance is used to describe the distance between the position where the gesture interaction command is issued and the reference position of the gesture interaction command receiving device; obtain first gesture interaction control data, the first gesture interaction control data is a reference value corresponding to the gesture interaction control parameter, specifically including the first light intensity, a first gesture interaction angle and a first gesture interaction distance; conducting a light intensity simulation experiment, wherein the light intensity simulation experiment is used to describe an experiment of changing the light intensity at a first gesture interaction angle and a first gesture interaction distance, and the light intensity varies within a reference light intensity range; conducting a gesture interaction angle simulation experiment, wherein the gesture interaction angle simulation experiment is used to describe an experiment of changing the gesture interaction angle at a first light intensity and a first gesture interaction distance, and the gesture interaction angle varies within a reference gesture interaction angle range; conducting a gesture interaction distance simulation experiment, wherein the gesture interaction distance simulation experiment is used to describe an experiment of changing the gesture interaction distance at a first light intensity and a first gesture interaction angle, and the gesture interaction distance varies within a reference gesture interaction distance range.

[0013] Furthermore, the interaction data analysis module includes a model building unit, a model screening unit and an interaction recognition analysis unit; the model building unit is used to: construct an interaction recognition data set in combination with the received voice interaction data and gesture interaction data, and construct a first interaction recognition model through the interaction recognition data set, and the interaction recognition data set is used to describe the set composed of voice interaction data and gesture interaction data; the model screening unit is used to: screen the constructed first interaction recognition model to obtain the interaction recognition model; the interaction recognition analysis unit is used to: input the received real-time interaction data into the interaction recognition model for recognition to obtain the interaction recognition result, and analyze the interaction recognition result to obtain the cockpit interaction analysis result.

[0014] Furthermore, the specific method for obtaining the interactive recognition model is as follows: step one, dividing the constructed interactive recognition data set according to a preset ratio to obtain an interactive recognition training set and an interactive recognition verification set, and performing data preprocessing on the interactive recognition training set, specifically including data enhancement and data labeling; step two, training a first interactive recognition model, and verifying the first interactive recognition model through an interactive recognition verification set, the first interactive recognition model being used to describe a converged model obtained by training and optimizing a preset recognition model through the interactive recognition training set after data preprocessing; step three, constructing a recognition performance index set, the recognition performance index set being used to describe an ordered set of recognition performance indices of the first interactive recognition model, and dividing the recognition performance indices in combination with the reference recognition performance index to obtain a first recognition performance index set and a second recognition performance index set. A recognition performance index set, wherein the recognition performance index is used to describe the recognition performance of the first interactive recognition model, and the first recognition performance index set and the second recognition performance index set are respectively used to describe an ordered set of data whose recognition performance index is less than a reference recognition performance index and data whose recognition performance index is not less than a reference recognition performance index; step four, determining whether the second recognition performance index set is an empty set, if it is an empty set, returning to step two, otherwise obtaining the maximum value in the second recognition performance index set, and determining whether the maximum value is unique, if it is unique, obtaining the first interactive recognition model corresponding to the maximum value and defining it as an interactive recognition model, otherwise executing step five; step five, screening the model corresponding to the maximum value according to the model training time, the model verification time and the model complexity until a unique model is obtained, and defining the obtained model as an interactive recognition model.

[0015] Furthermore, a specific method for obtaining the recognition performance index is as follows: obtaining performance evaluation data of the first interactive recognition model, wherein the performance evaluation data is used to evaluate the voice interaction recognition performance and gesture interaction recognition performance of the first interactive recognition model, specifically including voice interaction performance evaluation data and gesture interaction performance evaluation data; analyzing the obtained voice interaction performance evaluation data to obtain a voice interaction performance index of the first interactive recognition model, wherein the voice interaction performance index is used to evaluate the voice interaction performance of the first interactive recognition model; analyzing the obtained gesture interaction performance evaluation data to obtain a gesture interaction performance index of the first interactive recognition model, wherein the gesture interaction performance index is used to evaluate the gesture interaction performance of the first interactive recognition model; and obtaining the recognition performance index of the first interactive recognition model by combining the obtained voice interaction performance index and gesture interaction performance index.

[0016] Furthermore, the specific calculation method of the recognition performance index is as follows: Wherein, e is a natural constant, h is the number of the first interactive recognition model, h = 1, 2, ..., H, H is the total number of first interactive recognition models, IPh is the recognition performance index of the h-th first interactive recognition model, SRh and GPh are the speech interaction performance index and gesture interaction performance index of the h-th first interactive recognition model, SR0 and GR0 are the reference speech interaction performance index and the reference gesture interaction performance index, ω1 and ω2 are the weights of the relative deviation of the speech interaction performance index and the relative deviation of the gesture interaction performance index relative to the recognition performance index.

[0017] Furthermore, the voice interaction performance index is calculated using the following formula: Wherein, j is the group number of the voice interaction simulation experiment, j=1 is the noise intensity simulation experiment, j=2 is the voice intensity simulation experiment, j=3 is the voice interaction distance simulation experiment, is the F1 score corresponding to the speech interaction data corresponding to the jth group of speech interaction simulation experiments in the interaction recognition verification set recognized by the hth first interaction recognition model, λj is the proportion of speech interaction data corresponding to the jth group of speech interaction simulation experiments, STh is the speech recognition timeliness score of the hth first interaction recognition model, and χ is the correction factor of the speech recognition timeliness score; the speech interaction data proportion is used to describe the proportion of speech interaction data corresponding to each group of speech interaction simulation experiments in the speech interaction data in the interaction recognition verification set; the speech recognition timeliness score is used to describe the real-time degree of the speech interaction command corresponding to the speech interaction data accurately recognized by the first interaction recognition model; the gesture interaction performance index is calculated using the following formula: Where g is the group number of the gesture interaction simulation experiment, g=1, g=2 and g=3 are the light intensity simulation experiment, gesture interaction angle simulation experiment and gesture interaction distance simulation experiment respectively. and are respectively the gesture interaction accuracy score and the gesture interaction timeliness score of the gesture interaction data corresponding to the g-th group of gesture interaction simulation experiments in the h-th first interaction recognition model recognition interaction recognition verification set, μ and η are respectively the correction factors of the gesture interaction accuracy score and the gesture interaction timeliness score, W1 and W2 are respectively the weights of the gesture interaction accuracy score and the gesture interaction real-time score for the gesture performance interaction index; the gesture interaction accuracy score is used to evaluate the accuracy of the gesture interaction commands corresponding to the gesture interaction data recognized by the first interaction recognition model; the gesture interaction timeliness score is used to evaluate the real-time degree of the gesture interaction commands corresponding to the gesture interaction data accurately recognized by the first interaction recognition model.

[0018] An embodiment of the present application provides a vehicle-mounted cockpit interaction method based on a cognitive big model, comprising the following steps: obtaining cockpit interaction data of a vehicle to be tested, and sending the obtained cockpit interaction data to an interaction data analysis module, wherein the cockpit interaction data is used to describe the cockpit interaction commands sent by a user of the vehicle to be tested, and the cockpit interaction commands are used to describe the functional service requirements expressed by the user of the vehicle to be tested through a preset cockpit interaction mode during the ride, and the preset cockpit interaction mode includes a voice interaction mode and a gesture interaction mode; analyzing the received cockpit interaction data in combination with a trained interaction recognition model to obtain a cockpit interaction analysis result, and sending the obtained result to an interaction execution module, wherein the interaction recognition model is used to identify the functional service requirements corresponding to the input cockpit interaction data; receiving the cockpit interaction analysis result sent by the interaction data analysis module, and providing corresponding functional services according to the cockpit interaction analysis result.

[0019] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0020] 1. By acquiring the voice interaction data, gesture interaction data and real-time interaction data of the vehicle to be tested, and then combining the voice interaction data and gesture interaction data to build an interaction recognition data set, and obtaining an interaction recognition model through model training, optimization and screening, and then obtaining the cockpit interaction analysis results corresponding to the real-time interaction data according to the interaction recognition model, and finally providing corresponding functional services according to the received cockpit interaction analysis results, it is possible to obtain voice interaction data and gesture interaction data more comprehensively, and then reduce the influence of the in-vehicle environment on the accuracy of vehicle-mounted cockpit interaction recognition, and effectively solve the problem in the prior art that the in-vehicle environment has a great influence on the accuracy of vehicle-mounted cockpit interaction recognition.

[0021] 2. Train the first interactive recognition model with the constructed interactive recognition data set and verify it, then obtain the recognition performance index of the first interactive recognition model to construct a recognition performance index set, and divide the recognition performance index set to obtain a first recognition performance index set and a second recognition performance index set. Finally, judge whether the second recognition performance index set is an empty set and whether the maximum value of the corresponding data is unique, and obtain the interactive recognition model in combination with the model training time, model verification time and model complexity, thereby achieving accurate screening of the first interactive recognition model and further achieving more accurate cockpit interaction.

[0022] 3. By obtaining speech interaction performance evaluation data and gesture interaction performance evaluation data of the first interaction recognition model, and then obtaining the speech interaction performance index of the first interaction recognition model according to the speech interaction performance evaluation data, and then obtaining the gesture interaction performance index of the first interaction recognition model according to the gesture interaction performance evaluation data, and finally combining the obtained speech interaction performance index and gesture interaction performance index to obtain the recognition performance index of the first interaction recognition model, the recognition performance of the first interaction recognition model is quantified, thereby achieving a more accurate acquisition of the interaction recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A structural flow chart of a vehicle cockpit interaction system based on a cognitive large model provided in an embodiment of the present application;

[0024] Figure 2 A structural flow chart of an interactive data acquisition module in a vehicle cockpit interactive system based on a cognitive big model provided in an embodiment of the present application;

[0025] Figure 3 An embodiment of the present application provides a structural flow chart of an interactive data analysis module in a vehicle-mounted cockpit interactive system based on a cognitive big model. DETAILED DESCRIPTION

[0026] The embodiment of the present application solves the problem in the prior art that the in-vehicle environment has a great influence on the accuracy of in-vehicle cockpit interaction recognition by providing a vehicle-mounted cockpit interaction system and method based on a cognitive big model. The voice interaction data acquisition unit in the interaction data acquisition module designs a voice interaction simulation experiment according to noise intensity, interaction voice intensity and interaction voice distance, and obtains voice interaction data of the vehicle to be tested according to the designed voice interaction simulation experiment. Then, the gesture interaction data acquisition unit in the interaction data acquisition module obtains corresponding gesture interaction data according to the gesture interaction simulation experiment designed according to light intensity, gesture interaction angle and gesture interaction distance. The real-time interaction data acquisition unit in the interaction data acquisition module obtains corresponding real-time interaction data, and then the interaction data analysis module obtains the corresponding gesture interaction data. The model building unit in the method builds an interactive recognition data set in combination with the received voice interaction data and gesture interaction data, and trains, optimizes and verifies the preset recognition model through the interactive recognition data set to obtain a first interactive recognition model. The model screening unit then obtains the recognition performance index of the first interactive recognition model, and screens the first interactive recognition model in combination with the reference recognition performance index to obtain the interactive recognition model. The interactive recognition analysis unit inputs the received real-time interactive data into the interactive recognition model for identification and analysis to obtain the cockpit interaction analysis result. Finally, the interactive execution module receives the cockpit interaction analysis result sent by the interactive data analysis module, and provides corresponding functional services according to the cockpit interaction analysis result, thereby reducing the influence of the in-vehicle environment on the accuracy of the vehicle-mounted cockpit interaction recognition.

[0027] The technical solution in the embodiment of the present application is to solve the problem that the in-vehicle environment has a great influence on the accuracy of vehicle-mounted cockpit interaction recognition. The overall idea is as follows:

[0028] By acquiring voice interaction data, gesture interaction data and real-time interaction data of the vehicle to be tested, and then combining the voice interaction data and gesture interaction data to construct an interaction recognition data set, and obtaining a first interaction recognition model through the interaction recognition data set, and then screening the first interaction recognition model according to the obtained recognition performance index of the first interaction recognition model to obtain an interaction recognition model, and obtaining the cockpit interaction analysis results corresponding to the real-time interaction data according to the interaction recognition model, and finally providing corresponding functional services according to the received cockpit interaction analysis results, the effect of reducing the influence of the in-vehicle environment on the accuracy of vehicle-mounted cockpit interaction recognition is achieved.

[0029] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0030] like Figure 1 As shown, it is a structural flow chart of a vehicle-mounted cockpit interaction system based on a cognitive big model provided in an embodiment of the present application. The vehicle-mounted cockpit interaction system based on a cognitive big model provided in an embodiment of the present application includes: an interaction data acquisition module, an interaction data analysis module and an interaction execution module; wherein the interaction data acquisition module is used to acquire the cockpit interaction data of the vehicle to be tested, and send the acquired cockpit interaction data to the interaction data analysis module, the cockpit interaction data is used to describe the cockpit interaction commands sent by the user of the vehicle to be tested, and the cockpit interaction commands are used to describe the functional service requirements expressed by the user of the vehicle to be tested through a preset cockpit interaction mode during the ride, and the preset cockpit interaction mode includes a voice interaction mode and a gesture interaction mode; the interaction data analysis module is used to analyze the received cockpit interaction data in combination with the trained interaction recognition model to obtain the cockpit interaction analysis results, and send the obtained results to the interaction execution module, the interaction recognition model is used to identify the functional service requirements corresponding to the input cockpit interaction data; the interaction execution module is used to receive the cockpit interaction analysis results sent by the interaction data analysis module, and provide corresponding functional services according to the cockpit interaction analysis results.

[0031] In this embodiment, the interaction execution module can be connected to the voice interaction receiving device and the gesture interaction receiving device to provide corresponding functional services; the gesture interaction mode refers to expressing functional service requirements mainly through gestures, but the activation of the gesture interaction mode may still rely on voice activation, which is specifically judged in combination with the actual functions of the vehicle to be tested; the interaction recognition model recognizes the voice and gestures of the user of the vehicle to be tested, which belongs to the application scope of the cognitive large model; it realizes faster identification of the functional service requirements of the user of the vehicle to be tested.

[0032] Further, such as Figure 2 As shown, it is a structural flow chart of an interaction data acquisition module in a vehicle-mounted cockpit interaction system based on a cognitive big model provided by an embodiment of the present application, the interaction data acquisition module includes a voice interaction data acquisition unit, a gesture interaction data acquisition unit and a real-time interaction data acquisition unit; the voice interaction data acquisition unit is used to: acquire the voice interaction data of the vehicle to be tested, the voice interaction data is used to describe the voice interaction commands in the voice interaction simulation experiment, and the voice interaction commands are used to describe the functional service requirements expressed by the user of the vehicle to be tested through voice interaction during the ride; the gesture interaction data acquisition unit is used to: acquire the gesture interaction data of the vehicle to be tested, the gesture interaction data is used to describe the gesture interaction commands in the designed gesture interaction simulation experiment, and the gesture interaction commands are used to describe the functional service requirements expressed by the user of the vehicle to be tested through gesture interaction in the reference gesture interaction area; the real-time interaction data acquisition unit is used to: acquire the real-time interaction data of the vehicle to be tested, the real-time interaction data is used to describe the real-time interaction commands sent by the user of the vehicle to be tested within a preset time period, and the real-time interaction commands are used to describe the functional service requirements expressed by the user of the vehicle to be tested through a preset cockpit interaction method.

[0033] In this embodiment, the actual cockpit interaction environment of the vehicle to be tested is very complex. Noise, light and cockpit interaction position will have a great impact on the results of the cockpit interaction. According to these influences, voice interaction simulation experiments and gesture interaction simulation experiments are designed. Then, the interaction recognition model trained based on these data will have better robustness, and the corresponding recognition performance will also be improved; the robustness of the interaction recognition model is improved.

[0034] Furthermore, the voice interaction simulation experiment includes a noise intensity simulation experiment, a voice intensity simulation experiment and a voice interaction distance simulation experiment. The specific experimental method is as follows: obtain voice interaction control parameters, the voice interaction control parameters include noise intensity, interactive voice intensity and interactive voice distance, the noise intensity is used to describe the noise decibels in the vehicle to be tested when the voice interaction command is sent, the interactive voice intensity is used to describe the voice decibels when the voice interaction command is sent, and the interactive voice distance is used to describe the distance between the voice interaction command sending position and the reference position of the voice interaction command receiving device; obtain first voice interaction control data, the first voice interaction control data is the reference value corresponding to the voice interaction control parameter, specifically including the first noise intensity, the first interactive voice intensity and the interactive voice distance. degree and the first interactive speech distance; conduct a noise intensity simulation experiment, the noise intensity simulation experiment is used to describe the experiment of changing the noise intensity under the first interactive speech intensity and the first interactive speech distance, and the noise intensity varies within the reference noise intensity range; conduct a speech intensity simulation experiment, the speech intensity simulation experiment is used to describe the experiment of changing the interactive speech intensity under the first noise intensity and the first interactive speech distance, and the interactive speech intensity varies within the reference interactive speech intensity range; conduct a speech interaction distance simulation experiment, the speech interaction distance simulation experiment is used to describe the experiment of changing the interactive speech distance under the condition that the first noise intensity and the first interactive speech intensity remain unchanged, and the interactive speech distance varies within the reference interactive speech distance range.

[0035] In this embodiment, for the voice interaction method, the noise level in the vehicle to be tested, the sound level when the voice interaction command is sent, and the distance from the voice interaction command receiving device will all affect the voice interaction recognition. By controlling two of the voice interaction control parameters to remain unchanged and changing the other parameter within the corresponding reference range, the corresponding voice interaction data is obtained; a more comprehensive acquisition of voice interaction data is achieved.

[0036] Furthermore, the gesture interaction simulation experiment includes a light intensity simulation experiment, a gesture interaction angle simulation experiment and a gesture interaction distance simulation experiment. The specific design method is as follows: obtain gesture interaction control parameters, the gesture interaction control parameters include light intensity, gesture interaction angle and gesture interaction distance, the light intensity is used to describe the light intensity in the vehicle to be tested when the gesture interaction command is sent, the gesture interaction angle is used to describe the angle formed by the position where the gesture interaction command is issued and the reference position of the gesture interaction command receiving device, and the gesture interaction distance is used to describe the distance between the position where the gesture interaction command is issued and the reference position of the gesture interaction command receiving device; obtain first gesture interaction control data, the first gesture interaction control data is the reference value corresponding to the gesture interaction control parameter, specifically including the first light intensity, the first gesture interaction angle and the first gesture interaction distance; conduct a light intensity simulation experiment, the light intensity simulation experiment is used to describe the experiment of changing the light intensity at the first gesture interaction angle and the first gesture interaction distance, and the light intensity varies within the reference light intensity range; conduct a gesture interaction angle simulation experiment, the gesture interaction angle simulation experiment is used to describe the experiment of changing the gesture interaction angle at the first light intensity and the first gesture interaction distance, and the gesture interaction angle varies within the reference gesture interaction angle range; conduct a gesture interaction distance simulation experiment, the gesture interaction distance simulation experiment is used to describe the experiment of changing the gesture interaction distance at the first light intensity and the first gesture interaction angle, and the gesture interaction distance varies within the reference gesture interaction distance range.

[0037] In this embodiment, for the gesture interaction method, the light intensity in the vehicle to be tested, the distance and angle between the position when the gesture interaction command is sent and the gesture interaction command receiving device have a direct impact on the recognition of the gesture interaction command. By controlling the gesture interaction control parameters to obtain gesture interaction data, more comprehensive acquisition of gesture interaction data is achieved.

[0038] Further, such as Figure 3 As shown, it is a structural flow chart of an interaction data analysis module in an in-vehicle cockpit interaction system based on a cognitive big model provided in an embodiment of the present application, the interaction data analysis module includes a model building unit, a model screening unit and an interaction recognition analysis unit; the model building unit is used to: construct an interaction recognition data set in combination with the received voice interaction data and gesture interaction data, and construct a first interaction recognition model through the interaction recognition data set, and the interaction recognition data set is used to describe the set composed of voice interaction data and gesture interaction data; the model screening unit is used to: screen the constructed first interaction recognition model to obtain the interaction recognition model; the interaction recognition analysis unit is used to: input the received real-time interaction data into the interaction recognition model for recognition to obtain the interaction recognition result, and analyze the interaction recognition result to obtain the cockpit interaction analysis result.

[0039] In this embodiment, the interaction recognition model recognizes the real-time interaction data, and the obtained interaction recognition result is not necessarily correct, so further analysis is required to obtain the cockpit interaction analysis result to achieve more accurate cockpit interaction.

[0040] Furthermore, a specific method for obtaining the interactive recognition model is as follows: step one, dividing the constructed interactive recognition data set according to a preset ratio to obtain an interactive recognition training set and an interactive recognition verification set, and performing data preprocessing on the interactive recognition training set, specifically including data enhancement and data labeling; step two, training a first interactive recognition model, and verifying the first interactive recognition model through the interactive recognition verification set, the first interactive recognition model is used to describe the converged model obtained by training and optimizing the preset recognition model with the interactive recognition training set after data preprocessing; step three, constructing a recognition performance index set, the recognition performance index set is used to describe an ordered set of recognition performance indices of the first interactive recognition model, and dividing the recognition performance indices in combination with the reference recognition performance index to obtain a first recognition performance index set and a second recognition performance index set. A recognition performance index set is defined, wherein the recognition performance index is used to describe the recognition performance of the first interactive recognition model. The first recognition performance index set and the second recognition performance index set are respectively used to describe the ordered sets of data whose recognition performance index is less than the reference recognition performance index and data whose recognition performance index is not less than the reference recognition performance index; step four, determining whether the second recognition performance index set is an empty set, if it is an empty set, returning to step two, otherwise obtaining the maximum value in the second recognition performance index set, and determining whether the maximum value is unique, if it is unique, obtaining the first interactive recognition model corresponding to the maximum value and defining it as an interactive recognition model, otherwise executing step five; step five, screening the model corresponding to the maximum value according to the model training time, the model verification time and the model complexity until a unique model is obtained, and defining the obtained model as an interactive recognition model.

[0041] In this embodiment, data enhancement is used to enhance the features of voice interaction data and gesture interaction data to improve the accuracy of the interaction recognition model. Specifically, methods such as noise reduction are used for data enhancement of voice interaction data, and methods such as deblurring and continuous frame analysis are used for data enhancement of gesture interaction data. If the second recognition performance index set is an empty set, the model needs to be retrained, that is, the reference recognition performance index is the minimum value of the recognition performance index of the interaction recognition model; thereby achieving more accurate acquisition of the interaction recognition model.

[0042] Furthermore, a specific method for obtaining the recognition performance index is as follows: obtaining performance evaluation data of the first interactive recognition model, the performance evaluation data being used to evaluate the voice interaction recognition performance and gesture interaction recognition performance of the first interactive recognition model, specifically including voice interaction performance evaluation data and gesture interaction performance evaluation data; analyzing the obtained voice interaction performance evaluation data to obtain the voice interaction performance index of the first interactive recognition model, the voice interaction performance index being used to evaluate the voice interaction performance of the first interactive recognition model; analyzing the obtained gesture interaction performance evaluation data to obtain the gesture interaction performance index of the first interactive recognition model, the gesture interaction performance index being used to evaluate the gesture interaction performance of the first interactive recognition model; and obtaining the recognition performance index of the first interactive recognition model by combining the obtained voice interaction performance index and gesture interaction performance index.

[0043] In this embodiment, the voice interaction performance evaluation data is used to obtain a voice interaction performance index, which is then combined with the gesture interaction performance index obtained from the gesture interaction performance evaluation data to obtain the recognition performance index of the first interaction recognition model, thereby achieving comprehensive quantification of the recognition performance of the first interaction recognition model.

[0044] Furthermore, the specific calculation method of the recognition performance index is as follows: Wherein, e is a natural constant, h is the number of the first interactive recognition model, h = 1, 2, ..., H, H is the total number of first interactive recognition models, IPh is the recognition performance index of the h-th first interactive recognition model, SRh and GPh are the speech interaction performance index and gesture interaction performance index of the h-th first interactive recognition model, SR0 and GR0 are the reference speech interaction performance index and the reference gesture interaction performance index, ω1 and ω2 are the weights of the relative deviation of the speech interaction performance index and the relative deviation of the gesture interaction performance index relative to the recognition performance index.

[0045] In this embodiment, the recognition performance index is obtained by combining the voice interaction performance index and the gesture interaction performance index to evaluate the recognition performance of the trained first interaction recognition model; the interaction recognition performance index increases with the increase of the voice interaction performance index and the gesture interaction performance index; the weights of the relative deviation of the voice interaction performance index and the relative deviation of the gesture interaction performance index relative to the recognition performance index can be obtained through subjective weighting or objective weighting, which is specifically obtained in combination with the actual situation; more accurate screening of the first interaction recognition model is achieved.

[0046] Furthermore, the voice interaction performance index is calculated using the following formula: Wherein, j is the group number of the voice interaction simulation experiment, j=1 is the noise intensity simulation experiment, j=2 is the voice intensity simulation experiment, j=3 is the voice interaction distance simulation experiment, is the F1 score corresponding to the speech interaction data corresponding to the jth group of speech interaction simulation experiments in the interaction recognition verification set recognized by the hth first interaction recognition model, λj is the proportion of speech interaction data corresponding to the jth group of speech interaction simulation experiments, STh is the speech recognition timeliness score of the hth first interaction recognition model, and χ is the correction factor of the speech recognition timeliness score; the speech interaction data proportion is used to describe the proportion of speech interaction data corresponding to each group of speech interaction simulation experiments in the speech interaction data in the interaction recognition verification set; the speech recognition timeliness score is used to describe the real-time degree of the speech interaction command corresponding to the speech interaction data accurately recognized by the first interaction recognition model; the gesture interaction performance index is calculated using the following formula: Where g is the group number of the gesture interaction simulation experiment, g=1, g=2 and g=3 are the light intensity simulation experiment, gesture interaction angle simulation experiment and gesture interaction distance simulation experiment respectively. and are the gesture interaction accuracy score and gesture interaction timeliness score of the gesture interaction data corresponding to the g-th group of gesture interaction simulation experiments in the h-th first interaction recognition model recognition interaction recognition verification set, μ and η are the correction factors of the gesture interaction accuracy score and the gesture interaction timeliness score, respectively, W1 and W2 are the weights of the gesture interaction accuracy score and the gesture interaction real-time score for the gesture performance interaction index, respectively; the gesture interaction accuracy score is used to evaluate the accuracy of the gesture interaction commands corresponding to the gesture interaction data recognized by the first interaction recognition model; the gesture interaction timeliness score is used to evaluate the real-time degree of the gesture interaction commands corresponding to the gesture interaction data accurately recognized by the first interaction recognition model.

[0047] In this embodiment, the speech recognition timeliness score can be evaluated by the recognition time of the interactive recognition model input, or it can be more accurately calculated by the time of accurately recognizing the speech interaction command corresponding to the speech interaction data. Because there may be inaccurate recognition, the corresponding time may be shorter when the recognition is inaccurate. The speech recognition timeliness score is calculated using the following formula. The inaccurate recognition is defined as the time corresponding to the time greater than T 2 : Where n is the number of the speech interaction data in the interactive recognition verification set, n = 1, 2, ..., N, N is the total number of speech interaction data in the interactive recognition verification set, is the time for the hth first interaction recognition model to accurately recognize the voice interaction command corresponding to the nth voice interaction data, T1 and T2 are the first voice interaction recognition time and the second voice interaction recognition time respectively, and the first voice interaction recognition time and the second voice interaction recognition time are used to describe the reference time variation range for the first interaction recognition model to accurately recognize the voice interaction commands corresponding to each voice interaction data in the interaction recognition verification set; similarly, the gesture interaction accuracy score can be evaluated by F1 score, accuracy, recall rate and other data, or by kappa coefficient for a more accurate numerical evaluation, and the gesture interaction accuracy score is calculated using the following formula: In the formula, is the Kappa coefficient of the gesture interaction data corresponding to the g-th group of gesture interaction simulation experiments in the h-th first interaction recognition model recognition interaction recognition verification set. K1 and K2 are the first kappa coefficient and the second kappa coefficient respectively. The first kappa coefficient and the second kappa coefficient are used to describe the reference variation range of the Kappa coefficient. The gesture interaction timeliness score can be evaluated by the verification time entered during model verification, or it can be more accurately calculated by accurately identifying the time of the gesture interaction command corresponding to the gesture interaction data. In the case of inaccurate recognition, the corresponding time is defined as greater than T2. ​​The gesture interaction timeliness score is calculated using the following formula:

[0048] Wherein, gm is the number of the gesture interaction data obtained from the g-th group of gesture interaction simulation experiments in the interaction recognition verification set, gm = g.1, g.2, ..., gM, gM is the total number of gesture interaction data obtained from the g-th group of gesture interaction simulation experiments in the interaction recognition verification set, It is the time for the hth first interaction recognition model to accurately recognize the gesture interaction command corresponding to the gmth gesture interaction data in the gth group of gesture interaction simulation experiments. T1 and T2 are the first gesture interaction recognition time and the second gesture interaction recognition time, respectively. The first gesture interaction recognition time and the second gesture interaction recognition time are used to describe the reference time variation range for the first interaction recognition model to accurately recognize the gesture interaction command corresponding to the gesture interaction data in the gesture interaction simulation experiment in the interaction recognition verification set. This achieves a more accurate and comprehensive evaluation of the speech interaction recognition performance and gesture interaction recognition performance of the first interaction recognition model.

[0049] An in-vehicle cockpit interaction method based on a cognitive big model provided in an embodiment of the present application comprises the following steps: obtaining cockpit interaction data of a vehicle to be tested, and sending the obtained cockpit interaction data to an interaction data analysis module, wherein the cockpit interaction data is used to describe the cockpit interaction commands sent by the user of the vehicle to be tested, and the cockpit interaction commands are used to describe the functional service requirements expressed by the user of the vehicle to be tested through a preset cockpit interaction mode during the ride, and the preset cockpit interaction mode includes a voice interaction mode and a gesture interaction mode; analyzing the received cockpit interaction data in combination with a trained interaction recognition model to obtain a cockpit interaction analysis result, and sending the obtained result to an interaction execution module, wherein the interaction recognition model is used to identify the functional service requirements corresponding to the input cockpit interaction data; receiving the cockpit interaction analysis result sent by the interaction data analysis module, and providing corresponding functional services according to the cockpit interaction analysis result.

[0050] In this embodiment, the functional service requirements include many aspects, such as navigation, in-car temperature control, or the use of entertainment equipment, etc.; in actual applications, it may also involve the handling of dangerous situations, etc.; the impact of the environment on the interactive performance of the cabin is reduced.

[0051] The technical solution in the above-mentioned embodiment of the present application has at least the following technical effects or advantages: relative to a gesture interaction method and system for a vehicle-mounted AR-HUD announced with announcement number: CN112241204B, the embodiment of the present application trains a first interactive recognition model through a constructed interactive recognition data set and verifies it, then obtains the recognition performance index of the first interactive recognition model to construct a recognition performance index set, and divides the recognition performance index to obtain a first recognition performance index set and a second recognition performance index set, and finally judges whether the second recognition performance index set is an empty set and whether the maximum value of the corresponding data is unique, and obtains the interactive recognition model in combination with the model training time, model verification time and model complexity, thereby realizing accurate screening of the first interactive recognition model. model, thereby achieving more accurate cockpit interaction; relative to the human-computer interaction safety comprehensive evaluation method and device for an intelligent cockpit disclosed in the publication number: CN115657922A, the embodiment of the present application obtains voice interaction performance evaluation data and gesture interaction performance evaluation data of a first interaction recognition model, and then obtains a voice interaction performance index of the first interaction recognition model according to the voice interaction performance evaluation data, and then obtains a gesture interaction performance index of the first interaction recognition model according to the gesture interaction performance evaluation data, and finally obtains a recognition performance index of the first interaction recognition model by combining the obtained voice interaction performance index and gesture interaction performance index, thereby achieving the digitization of the recognition performance of the first interaction recognition model, and thereby achieving the acquisition of a more accurate interaction recognition model.

[0052] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0053] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0054] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0055] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0056] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0057] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A vehicle cockpit interaction system based on a cognitive large model, characterized in that: include: Interaction data acquisition module, interaction data analysis module and interaction execution module; The interactive data acquisition module is used to acquire cockpit interaction data of the vehicle to be tested, and send the acquired cockpit interaction data to the interactive data analysis module, wherein the cockpit interaction data is used to describe the cockpit interaction commands sent by the user of the vehicle to be tested, and the cockpit interaction commands are used to describe the functional service requirements expressed by the user of the vehicle to be tested through a preset cockpit interaction mode during the ride, and the preset cockpit interaction mode includes a voice interaction mode and a gesture interaction mode; The interaction data analysis module is used to analyze the received cockpit interaction data in combination with the trained interaction recognition model to obtain cockpit interaction analysis results, and send the obtained results to the interaction execution module, and the interaction recognition model is used to identify the functional service requirements corresponding to the input cockpit interaction data; The interaction execution module is used to receive the cockpit interaction analysis result sent by the interaction data analysis module, and provide corresponding functional services according to the cockpit interaction analysis result; The specific method for obtaining the interactive recognition model is as follows: Step 1: divide the constructed interactive recognition data set into an interactive recognition training set and an interactive recognition verification set according to a preset ratio, and perform data preprocessing on the interactive recognition training set, specifically including data enhancement and data labeling; Step 2: training a first interactive recognition model and verifying the first interactive recognition model through an interactive recognition verification set, wherein the first interactive recognition model is used to describe a converged model obtained by training and optimizing a preset recognition model through an interactive recognition training set after data preprocessing; Step 3: construct a recognition performance index set, the recognition performance index set is used to describe the ordered set of recognition performance indexes of the first interactive recognition model, and the recognition performance index is divided into a first recognition performance index set and a second recognition performance index set in combination with the reference recognition performance index, the recognition performance index is used to describe the recognition performance of the first interactive recognition model, and the first recognition performance index set and the second recognition performance index set are respectively used to describe the ordered set of data whose recognition performance index is less than the reference recognition performance index and the data whose recognition performance index is not less than the reference recognition performance index; Step 4: determine whether the second recognition performance index set is an empty set. If it is an empty set, return to step 2. Otherwise, obtain the maximum value in the second recognition performance index set and determine whether the maximum value is unique. If it is unique, obtain the first interactive recognition model corresponding to the maximum value and define it as the interactive recognition model. Otherwise, execute step 5. Step 5: Screen the model corresponding to the maximum value according to the model training time, model verification time and model complexity until a unique model is obtained, and define the obtained model as an interactive recognition model; The specific method for obtaining the recognition performance index is as follows: Acquire performance evaluation data of the first interaction recognition model, where the performance evaluation data is used to evaluate the voice interaction recognition performance and gesture interaction recognition performance of the first interaction recognition model, and specifically includes voice interaction performance evaluation data and gesture interaction performance evaluation data; Analyze the acquired speech interaction performance evaluation data to obtain a speech interaction performance index of the first interaction recognition model, wherein the speech interaction performance index is used to evaluate the speech interaction performance of the first interaction recognition model; Analyze the acquired gesture interaction performance evaluation data to obtain a gesture interaction performance index of the first interaction recognition model, wherein the gesture interaction performance index is used to evaluate the gesture interaction performance of the first interaction recognition model; Acquire a recognition performance index of the first interaction recognition model by combining the acquired voice interaction performance index and gesture interaction performance index; The specific calculation method of the recognition performance index is as follows: Wherein, e is a natural constant, h is the number of the first interactive recognition model, h = 1, 2, ..., H, H is the total number of first interactive recognition models, IPh is the recognition performance index of the h-th first interactive recognition model, SRh and GPh are the speech interaction performance index and gesture interaction performance index of the h-th first interactive recognition model, SR0 and GR0 are the reference speech interaction performance index and the reference gesture interaction performance index, ω1 and ω2 are the weights of the relative deviation of the speech interaction performance index and the relative deviation of the gesture interaction performance index relative to the recognition performance index.

2. The vehicle cockpit interaction system based on cognitive big model as claimed in claim 1, characterized in that: The interaction data acquisition module includes a voice interaction data acquisition unit, a gesture interaction data acquisition unit and a real-time interaction data acquisition unit; The voice interaction data acquisition unit is used to: acquire voice interaction data of the vehicle to be tested, wherein the voice interaction data is used to describe voice interaction commands in the voice interaction simulation experiment, and the voice interaction commands are used to describe functional service requirements expressed by users of the vehicle to be tested through voice interaction during the ride; The gesture interaction data acquisition unit is used to: acquire gesture interaction data of the vehicle to be tested, wherein the gesture interaction data is used to describe gesture interaction commands in the designed gesture interaction simulation experiment, and the gesture interaction commands are used to describe functional service requirements expressed by users of the vehicle to be tested in a reference gesture interaction area through gesture interaction; The real-time interaction data acquisition unit is used to: acquire real-time interaction data of the vehicle to be tested, wherein the real-time interaction data is used to describe the real-time interaction commands sent by the user of the vehicle to be tested within a preset time period, and the real-time interaction commands are used to describe the functional service requirements expressed by the user of the vehicle to be tested through a preset cockpit interaction method.

3. The vehicle cockpit interaction system based on cognitive big model as claimed in claim 2, characterized in that: The speech interaction simulation experiment includes a noise intensity simulation experiment, a speech intensity simulation experiment and a speech interaction distance simulation experiment. The specific experimental methods are as follows: Acquire voice interaction control parameters, wherein the voice interaction control parameters include noise intensity, interactive voice intensity, and interactive voice distance, wherein the noise intensity is used to describe the noise decibel in the vehicle to be tested when the voice interaction command is sent, the interactive voice intensity is used to describe the voice decibel when the voice interaction command is sent, and the interactive voice distance is used to describe the distance between the voice interaction command sending position and the reference position of the voice interaction command receiving device; Acquire first voice interaction control data, where the first voice interaction control data is a reference value corresponding to a voice interaction control parameter, specifically including a first noise intensity, a first interaction voice intensity, and a first interaction voice distance; Conducting a noise intensity simulation experiment, wherein the noise intensity simulation experiment is used to describe an experiment of changing the noise intensity at a first interactive voice intensity and a first interactive voice distance, wherein the noise intensity varies within a reference noise intensity range; Conducting a speech intensity simulation experiment, wherein the speech intensity simulation experiment is used to describe an experiment of changing the interactive speech intensity under a first noise intensity and a first interactive speech distance, wherein the interactive speech intensity varies within a reference interactive speech intensity range; A speech interaction distance simulation experiment is performed, wherein the speech interaction distance simulation experiment is used to describe an experiment in which the interaction speech distance is changed under the condition that the first noise intensity and the first interaction speech intensity remain unchanged, and the interaction speech distance varies within a reference interaction speech distance range.

4. The vehicle cockpit interaction system based on cognitive large model as claimed in claim 2, characterized in that: The gesture interaction simulation experiment includes a light intensity simulation experiment, a gesture interaction angle simulation experiment and a gesture interaction distance simulation experiment. The specific design method is as follows: Acquire gesture interaction control parameters, where the gesture interaction control parameters include light intensity, gesture interaction angle, and gesture interaction distance. The light intensity is used to describe the light intensity in the vehicle to be tested when the gesture interaction command is sent. The gesture interaction angle is used to describe the angle formed by the position where the gesture interaction command is issued and the reference position of the gesture interaction command receiving device. The gesture interaction distance is used to describe the distance between the position where the gesture interaction command is issued and the reference position of the gesture interaction command receiving device. Acquire first gesture interaction control data, where the first gesture interaction control data is a reference value corresponding to a gesture interaction control parameter, specifically including a first light intensity, a first gesture interaction angle, and a first gesture interaction distance; Performing a light intensity simulation experiment, wherein the light intensity simulation experiment is used to describe an experiment of changing the light intensity at a first gesture interaction angle and a first gesture interaction distance, wherein the light intensity varies within a reference light intensity range; Performing a gesture interaction angle simulation experiment, wherein the gesture interaction angle simulation experiment is used to describe an experiment of changing a gesture interaction angle under a first light intensity and a first gesture interaction distance, wherein the gesture interaction angle varies within a reference gesture interaction angle range; A gesture interaction distance simulation experiment is performed, wherein the gesture interaction distance simulation experiment is used to describe an experiment of changing the gesture interaction distance under a first light intensity and a first gesture interaction angle, and the gesture interaction distance varies within a reference gesture interaction distance range.

5. The vehicle cockpit interaction system based on cognitive large model as claimed in claim 2, characterized in that: The interactive data analysis module includes a model building unit, a model screening unit and an interactive identification analysis unit; The model building unit is used to: build an interaction recognition data set in combination with the received voice interaction data and gesture interaction data, and build a first interaction recognition model through the interaction recognition data set, wherein the interaction recognition data set is used to describe a set composed of the voice interaction data and the gesture interaction data; The model screening unit is used to: screen the constructed first interaction recognition model to obtain an interaction recognition model; The interaction recognition and analysis unit is used to: input the received real-time interaction data into the interaction recognition model for recognition to obtain an interaction recognition result, and analyze the interaction recognition result to obtain a cockpit interaction analysis result.

6. The vehicle cockpit interaction system based on cognitive big model as claimed in claim 1, characterized in that: The voice interaction performance index is calculated using the following formula: Wherein, j is the group number of the voice interaction simulation experiment, j=1 is the noise intensity simulation experiment, j=2 is the voice intensity simulation experiment, j=3 is the voice interaction distance simulation experiment, is the F1 score corresponding to the speech interaction data corresponding to the j-th group of speech interaction simulation experiments in the interaction recognition validation set recognized by the h-th first interaction recognition model, λj is the proportion of speech interaction data corresponding to the j-th group of speech interaction simulation experiments, STh is the speech recognition timeliness score of the h-th first interaction recognition model, and χ is the correction factor for the speech recognition timeliness score; The voice interaction data ratio is used to describe the proportion of the voice interaction data corresponding to each group of voice interaction simulation experiments to the voice interaction data in the interaction recognition verification set; The speech recognition timeliness score is used to describe the real-time degree of the first interaction recognition model accurately recognizing the speech interaction command corresponding to the speech interaction data; The gesture interaction performance index is calculated using the following formula: Where g is the group number of the gesture interaction simulation experiment, g=1, g=2 and g=3 are the light intensity simulation experiment, gesture interaction angle simulation experiment and gesture interaction distance simulation experiment respectively. and are the gesture interaction accuracy score and gesture interaction timeliness score of the gesture interaction data corresponding to the g-th group of gesture interaction simulation experiments in the h-th first interaction recognition model recognition interaction recognition validation set, μ and η are the correction factors of the gesture interaction accuracy score and the gesture interaction timeliness score, respectively, and W1 and W2 are the weights of the gesture interaction accuracy score and the gesture interaction real-time score for the gesture performance interaction index, respectively; The gesture interaction accuracy score is used to evaluate the accuracy of the gesture interaction command corresponding to the gesture interaction data recognized by the first interaction recognition model; The gesture interaction timeliness score is used to evaluate the real-time degree of the first interaction recognition model accurately recognizing the gesture interaction command corresponding to the gesture interaction data.

7. A method for a vehicle cockpit interaction system based on a cognitive large model as claimed in any one of claims 1 to 6, characterized in that: The following steps are involved: Acquire cockpit interaction data of the vehicle to be tested, and send the acquired cockpit interaction data to an interaction data analysis module, wherein the cockpit interaction data is used to describe cockpit interaction commands sent by a user of the vehicle to be tested, and the cockpit interaction commands are used to describe functional service requirements expressed by the user of the vehicle to be tested through a preset cockpit interaction mode during the ride, wherein the preset cockpit interaction mode includes a voice interaction mode and a gesture interaction mode; Analyze the received cockpit interaction data in combination with the trained interaction recognition model to obtain cockpit interaction analysis results, and send the obtained results to the interaction execution module, wherein the interaction recognition model is used to identify the functional service requirements corresponding to the input cockpit interaction data; Receive the cockpit interaction analysis results sent by the interactive data analysis module, and provide corresponding functional services based on the cockpit interaction analysis results.

Images