Vehicle computer and 3Dof controller fusion control method, system, storage medium and vehicle

Abstract

The embodiment of the present invention provides a vehicle computer and 3DOF controller fusion control method, system, storage medium and vehicle. The method includes: obtaining the first vehicle computer inertia data of the vehicle computer, the first controller inertia data of the 3DOF controller, and the first vehicle speed data, the first steering wheel angle data and the first positioning data during driving; performing a first transformation on the first vehicle computer inertia data and the first controller inertia data to obtain the second vehicle computer inertia data and the second controller inertia data; performing a second transformation on the first vehicle speed data, the first steering wheel angle data and the first positioning data to obtain the second vehicle speed data, the second steering wheel angle data and the second positioning data; performing noise processing on the second vehicle computer inertia data and the second controller inertia data through a first time series analysis model to obtain the third vehicle computer inertia data and the third controller inertia data, performing data fusion processing, and obtaining the 3DOF posture information of the controller in the driving state.

Description

Technical Field

[0001] The present application relates to the field of data vehicle automation control technology, and more specifically, to a vehicle computer and 3Dof controller fusion control method, system, storage medium and vehicle. Background Art

[0002] With the development of new energy vehicles, smart cockpits are gradually replacing traditional cockpits. Among them, XR (Extended Reality) technology is one of the indispensable components of smart cockpits. It combines reality and virtuality through computers to create a virtual environment for human-computer interaction for smart cockpits. 3DOF (Three Degrees Of Freedom) interaction is an interaction method based on the rotational freedom of the three coordinate axes of X, Y, and Z. Generally, inertial sensors are used to detect free rotation in different directions. Since only inertial sensors are needed to detect the axial free rotation of the three coordinate axes, and the equipment used is small in size and has few restrictions on usage conditions, 3DOF interaction has become a basic interaction method for XR applications.

[0003] However, new problems have arisen after 3DOF interaction was used in smart cockpits. Compared with a static environment, the driving environment is more complex. There is a lot of noise in the inertial data obtained by the smart cockpit's car computer and 3DOF controller, which makes it impossible to properly analyze the posture changes of the 3DOF controller in the driving state, resulting in the failure of the 3DOF controller to control the XR application. Summary of the invention

[0004] In view of this, embodiments of the present application provide a vehicle computer and 3Dof controller fusion control method, system, storage medium and vehicle to at least partially solve the above problems.

[0005] According to a first aspect of an embodiment of the present application, a method for fusion control of a vehicle computer and a 3Dof controller is provided, characterized by comprising:

[0006] Acquire first vehicle machine inertia data of a vehicle machine during driving, first controller inertia data of a 3DOF controller paired with the vehicle machine, and first vehicle speed data, first steering wheel angle data, and first positioning data of the vehicle machine; perform a first transformation on the first vehicle machine inertia data and the first controller inertia data to obtain second vehicle machine inertia data and second controller inertia data; perform a second transformation on the first vehicle speed data, the first steering wheel angle data, and the first positioning data to obtain second vehicle speed data, second steering wheel angle data, and second positioning data; perform a second transformation on the first vehicle speed data, the first steering wheel angle data, and the first positioning data based on the second vehicle machine inertia data and the first timing analysis model; According to the timing correspondence between the second vehicle speed data, the second steering wheel angle data, and the second positioning data, the second vehicle computer inertia data is subjected to noise processing to obtain third vehicle computer inertia data; according to the timing correspondence between the second controller inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data, the second vehicle computer inertia data is subjected to noise processing to obtain third controller inertia data; and based on the third vehicle computer inertia data and the third controller inertia data, fusion processing is performed to obtain 3Dof posture information of the controller in a driving state.

[0007] According to a second aspect of an embodiment of the present application, a vehicle computer and a 3DOF controller fusion control system is provided, including: a vehicle computer and a 3DOF controller. The vehicle computer includes a first inertial measurement unit, a first communication unit, a speed detection unit, a steering wheel angle detection unit, and a positioning device, wherein the first inertial measurement unit, the first communication unit, the speed detection unit, the steering wheel angle detection unit, and the positioning device are respectively used to collect first vehicle computer inertial data, first vehicle speed data, first steering wheel angle data, and first positioning data, and the first communication unit is used to transmit data; the vehicle computer also includes: a processor, a memory, a communication interface, and a communication bus, the first inertial measurement unit, the first communication unit, the speed detection unit, the steering wheel angle detection unit, the positioning device, the processor, the memory, and the communication interface communicate with each other through the communication bus, and the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation corresponding to the method provided in the first aspect to realize 3Dof control in a driving state; the 3DOF controller is paired with the vehicle computer, and the 3DOF controller includes a second inertial measurement unit and a second communication unit, and the second communication unit can transmit the data collected by the second inertial measurement unit to the first communication unit.

[0008] According to a third aspect of an embodiment of the present application, a vehicle is provided, comprising the above control system.

[0009] According to a fourth aspect of an embodiment of the present application, a computer storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method provided in the first aspect is implemented.

[0010] The method of this embodiment uses the first transformation to remove the noise with a frequency greater than the first threshold in the inertial data of the first vehicle machine and the inertial data of the first controller respectively; the first time series analysis model is used to automatically learn the correlation between the inertial data of the second vehicle machine and the second vehicle speed data, the second steering wheel angle data, and the second positioning data according to the time series correspondence, and denoise is performed based on this correlation, so that the noise with a frequency less than the first threshold and the noise with a frequency equal to the first threshold in the inertial data of the second vehicle machine can be removed. Similarly, the noise with a frequency less than the first threshold and the noise with a frequency equal to the first threshold in the inertial data of the second controller are removed by the second time series analysis model. The above process can remove the noise with a frequency greater than, less than, and equal to the first threshold in the inertial data obtained by the vehicle machine and the 3DOF controller during driving, and achieve comprehensive noise reduction. The 3dof posture information of the controller in the driving state can be obtained by fusing the inertial data of the third vehicle machine and the inertial data of the third controller, so as to obtain the instructions issued by the user using the controller. To sum up, after being processed by the vehicle computer and 3Dof controller fusion control method of the embodiment of the present application, the 3dof posture information of the controller in the driving state can be obtained, that is, the posture changes of the 3DOF controller in the driving state can be normally analyzed, so that the 3DOF controller can also normally control the XR application in the driving state. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0012] Figure 1 It is a flow chart of a method for integrating a vehicle computer and a 3Dof controller according to an embodiment of the present application;

[0013] Figure 2 A data processing diagram of a vehicle computer and 3Dof controller fusion control method according to an embodiment of the present application;

[0014] Figure 3 It is a structural schematic diagram of a vehicle computer and 3Dof controller fusion control system according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0016] In order to solve the problem that due to the complex driving environment, there is a lot of noise in the inertial data obtained by the vehicle computer and 3DOF controller of the smart cockpit, and the posture changes of the 3DOF controller under the driving state cannot be normally analyzed, resulting in the failure of the 3DOF controller to control the XR application, according to the first aspect of the embodiment of the present application, a method for fusion control of the vehicle computer and the 3DoF controller is provided.

[0017] Figure 1 The flowchart of the vehicle computer and 3Dof controller fusion control method of the embodiment of the present application is shown. In this embodiment, the vehicle computer and 3Dof controller fusion control method includes the following steps:

[0018] S101: Acquire first vehicle computer inertia data of a vehicle computer during driving, first controller inertia data of a 3DOF controller paired with the vehicle computer, and first vehicle speed data, first steering wheel angle data, and first positioning data of the vehicle computer.

[0019] In this embodiment, the car computer refers to the abbreviation of the in-vehicle infotainment system installed in the car, which includes two parts: hardware and software. The hardware usually includes a processor, memory, display, positioning device, IMU (Inertial Measurement Unit), instrumentation, and other vehicle sensors, etc. The software includes an operating system and application programs. The interaction between software and hardware modules supports the complete operation of the entire car computer system, realizing information communication between people and cars, and between cars and the outside world (cars and cars).

[0020] The 3DOF controller is a 3DOF interactive controller that integrates an MCU (Microcontroller Unit), IMU, and RF (Radio Frequency) module. It captures user gestures based on the inertial sensor in the IMU, allowing users to manipulate objects in the XR world, thereby enabling users to interact with the surrounding environment. The 3DOF controller can specifically be a 3DOF ring, 3DOF handle, etc.

[0021] The "pairing" in "pairing with the vehicle computer" refers to establishing a connection between devices by matching the registration information between the devices.

[0022] The first vehicle computer inertial data is the vehicle motion data collected by the vehicle computer IMU, and the first controller inertial data is the user gesture action data collected by the IMU in the 3DOF controller. Usually, the inertial sensors in the vehicle computer and the IMU in the 3DOF controller include at least an accelerometer and a gyroscope.

[0023] S102: Perform a first transformation on the first vehicle machine inertia data and the first controller inertia data to obtain second vehicle machine inertia data and second controller inertia data.

[0024] In this embodiment, the first transformation is used to remove the noise with a frequency greater than the first threshold from the first vehicle inertia data and the first controller inertia data, so as to obtain the second vehicle inertia data and the second controller inertia data. Specifically, the first transformation can be a transformation analysis method such as wavelet transformation that can separate noise. As a specific implementation method, the first (wavelet) transformation can be used to separate the high-frequency band whose main component is noise from the first vehicle inertia data and the first controller inertia data according to the first threshold. After removing the high-frequency band, the second vehicle inertia data and the second controller inertia data that have undergone preliminary denoising can be obtained. Among them, wavelets are waves in a small area, which have non-zero values ​​only in a very limited interval, rather than being endless like sine waves and cosine waves. Wavelets can be translated forward and backward along the time axis, and can also be stretched and compressed proportionally to obtain low-frequency and high-frequency wavelets. Wavelet transformation can be used to filter or compress signals by constructing wavelet functions, so that useful signals can be extracted from noisy signals.

[0025] It should be understood that the first threshold value can be obtained by conventional means in the field, and the present application does not limit the acquisition of the first threshold value. Specifically, when collecting the inertial data of the first vehicle computer and the inertial data of the first controller, in order to ensure sufficient collection of valid data, the data is usually oversampled, so that a part of the high-frequency frequency band with the highest frequency in the collected data basically does not contain valid data, that is, the main component of the high-frequency frequency band is noise. After the data collection is completed, the collected data frequency is generally analyzed to obtain the approximate distribution of the noise frequency for denoising. According to the approximate distribution of the noise frequency, the frequency lower limit of the above-mentioned high-frequency frequency band can be determined, and the frequency lower limit is determined as the first threshold value.

[0026] S103: Perform a second transformation on the first vehicle speed data, the first steering wheel angle data and the first positioning data to obtain second vehicle speed data, second steering wheel angle data and second positioning data.

[0027] Similar to the first transformation, the second transformation can also be a transformation analysis method that can separate noise, such as wavelet transformation. In this process, the second transformation is used to remove noise with a frequency greater than a second threshold from the first vehicle speed data, the first steering wheel angle data, and the first positioning data, respectively, and valid data is retained to obtain the second vehicle speed data, the second steering wheel angle data, and the second positioning data.

[0028] The second threshold value can be determined by referring to the above-mentioned first threshold value determination process. Specifically, the frequency lower limit of the high-frequency band whose main component is noise in the first vehicle speed data, the frequency lower limit of the high-frequency band whose main component is noise in the first steering wheel angle data, and the frequency lower limit of the high-frequency band whose main component is noise in the first positioning data are determined respectively, and the one with the highest frequency among these three frequency lower limits is determined as the second threshold value. As an optional embodiment, the high-frequency band whose main component is noise can be separated from the second vehicle speed data, the second steering wheel angle data, and the second positioning data according to the second threshold value by using a second wavelet transform to achieve denoising.

[0029] The first transformation is for the first vehicle inertial data collected by the vehicle IMU or the first controller inertial data of the 3DOF controller IMU, and the second transformation is for the vehicle speed data, steering wheel angle data and positioning data collected by other sensors or detection devices in the vehicle. The first transformation and the second transformation are aimed at different sources and types of data that need to be denoised, and the transformation methods and transformation functions used may also be different.

[0030] S104: performing noise processing on the second vehicle computer inertia data based on the timing correspondence between the second vehicle computer inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data using the first timing analysis model to obtain third vehicle computer inertia data; performing noise processing on the second vehicle computer inertia data based on the timing correspondence between the second controller inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data using the second timing analysis model to obtain third controller inertia data.

[0031] The second vehicle inertia data obtained by denoising the first transformation and the second vehicle speed data, the second steering wheel angle data, and the second positioning data obtained by denoising the second transformation are input into the first time series analysis model, and the second controller inertia data obtained by denoising the first transformation and the second transformation are input into the second time series analysis model. The first and second time series analysis models are models for obtaining the development law of things by analyzing the time series data of the past development of things. In this embodiment, the first time series analysis model can automatically learn the correlation between the second vehicle inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data according to the time series correspondence between the second vehicle inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data, and perform denoising based on this correlation, so as to remove the noise with a frequency less than the first threshold and the noise with a frequency equal to the first threshold in the second vehicle inertia data, and the noise includes the zero bias noise generated by the IMU, that is, this process can denoise the zero bias noise, wherein the zero bias noise is the noise generated by the fluctuation or fluctuation of the output signal of the sensor such as the IMU around its mean. Similarly, based on the timing correspondence between the second controller inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data, the second timing analysis model can remove noise with a frequency less than the first threshold and noise with a frequency equal to the first threshold in the second vehicle machine inertia data.

[0032] S105: Based on the inertial data of the third vehicle computer and the inertial data of the third controller, a fusion process is performed to obtain 3dof posture information of the controller in the driving state.

[0033] The processor in the vehicle computer is used to fuse the inertial data of the third vehicle computer and the inertial data of the third controller. The fusion processing can build a reference system with the vehicle computer as the reference object, and then use the inertial data of the third vehicle computer as the benchmark to map the inertial data of the third controller to the reference system built with the vehicle computer as the reference object to obtain the inertial data of the fourth controller. The inertial data of the fourth controller can be used to parse the posture change of the controller relative to the vehicle computer, thereby obtaining the 3dof posture information of the controller in the driving state, and obtaining the instructions issued by the user using the controller. As an optional embodiment, the fusion processing can also use other methods for calculating relative motion, which are not limited in this application.

[0034] The method of this embodiment uses the first transformation to remove the noise with a frequency greater than the first threshold in the inertial data of the first vehicle machine and the inertial data of the first controller respectively; the first time series analysis model is used to automatically learn the correlation between the inertial data of the second vehicle machine and the second vehicle speed data, the second steering wheel angle data, and the second positioning data according to the time series correspondence, and denoise is performed based on this correlation, so that the noise with a frequency less than the first threshold and the noise with a frequency equal to the first threshold in the inertial data of the second vehicle machine can be removed. Similarly, the noise with a frequency less than the first threshold and the noise with a frequency equal to the first threshold in the inertial data of the second controller are removed by the second time series analysis model. The above process can remove the noise with a frequency greater than, less than, and equal to the first threshold in the inertial data obtained by the vehicle machine and the 3DOF controller during driving, and achieve comprehensive noise reduction. The 3dof posture information of the controller in the driving state can be obtained by fusing the inertial data of the third vehicle machine and the inertial data of the third controller, so as to obtain the instructions issued by the user using the controller. To sum up, after being processed by the vehicle computer and 3DOF controller fusion control method of the embodiment of the present application, the 3DOF posture information of the controller in the driving state can be obtained, that is, the posture changes of the 3DOF controller in the driving state can be normally analyzed, so that the 3DOF controller can normally control the XR application.

[0035] In some possible embodiments, the first vehicle machine inertial data and the first controller inertial data include 3-axis acceleration data collected by an accelerometer and 3-axis angular velocity data collected by a gyroscope (all data share a set of X, Y, and Z coordinate axes).

[0036] As a possible implementation mode of the present invention, the IMU sampling frame rate in the vehicle computer and the 3DOF controller is 512 Hz, and data with a frequency of 0-256 Hz can be collected as the first vehicle computer inertial data and the first controller inertial data, wherein the high frequency band of 128-256 Hz mainly appears as noise.

[0037] In the process of performing a first transformation on the first vehicle machine inertia data and the first controller inertia data to obtain second vehicle machine inertia data and second controller inertia data, a first wavelet transform may be used as the first transformation, and based on the frequency of the above noise, 128 Hz may be set as the first threshold.

[0038] On the one hand, the first wavelet transform is used to decompose the acceleration data of each axis in the three-axis acceleration data in the first vehicle inertial data into a high-frequency part with a frequency interval of (128, 256] and a low-frequency part with a frequency interval of (0, 128] according to the first threshold 128Hz, and the high-frequency part is removed, and the low-frequency parts corresponding to the three axes are respectively decomposed into 8 acceleration low-frequency bands (the frequency intervals of each band are (0, 16], (16, 32], (32, 48], (48, 64], (64, 80], (80, 96], (96, 112], (112, 128]), and 3 groups of frequency bands containing 8 acceleration low-frequency bands are obtained, wherein the low-frequency parts can also be decomposed into 32 or 16 segments of other acceleration low-frequency bands according to the computer performance. In addition, the first wavelet transform is used to decompose the three-axis angular velocity data in the first vehicle inertial data. The angular velocity data of each axis in the degree data is decomposed into a high frequency part with a frequency interval of (128, 256] and a low frequency part with a frequency interval of (0, 128] according to the first threshold value, and the high frequency part is removed, and the low frequency parts corresponding to the three axes are respectively decomposed into 8 angular velocity low frequency bands (the frequency intervals of each band are (0, 16], (16, 32], (32, 48], (48, 64], (64, 80], (80, 96], (96, 112], (112, 128]), and 3 groups of frequency bands containing 8 angular velocity low frequency bands are obtained. Among them, the low frequency parts can also be decomposed into 32 or 16 acceleration low frequency bands of other numbers according to the computer performance. The above process can obtain the second vehicle inertia data including 3 groups of frequency bands containing 8 acceleration low frequency bands and 3 groups of frequency bands containing 8 angular velocity low frequency bands.

[0039] On the other hand, the first wavelet transform is used to decompose the acceleration data of each axis in the three-axis acceleration data in the inertial data of the first controller into a high-frequency part with a frequency interval of (128, 256] and a low-frequency part with a frequency interval of (0, 128] according to the first threshold value 128Hz, and the high-frequency part is removed, and the low-frequency parts corresponding to the three axes are respectively decomposed into 8 acceleration low-frequency bands (the frequency intervals of each band are respectively (0, 16], (16, 32], (32, 48], (48, 64], (64, 80], (80, 96], (96, 112], (112, 128]), and 3 groups of frequency bands containing 8 acceleration low-frequency bands are obtained, wherein the low-frequency parts can also be decomposed into 32 or 16 acceleration low-frequency bands of other numbers according to the computer performance. In addition, the first wavelet transform is used to decompose the three-axis angle data in the inertial data of the first controller. The angular velocity data of each axis in the velocity data is decomposed into a high frequency part with a frequency interval of (128, 256] and a low frequency part with a frequency interval of (0, 128] according to the first threshold value, and the high frequency part is removed, and the low frequency parts corresponding to the three axes are respectively decomposed into 8 angular velocity low frequency bands (the frequency intervals of each band are (0, 16], (16, 32], (32, 48], (48, 64], (64, 80], (80, 96], (96, 112], (112, 128]), and 3 groups of frequency bands containing 8 angular velocity low frequency bands are obtained. The low frequency parts can also be decomposed into 32 or 16 acceleration low frequency bands of other numbers according to the computer performance. The above process can obtain the second controller inertia data including 3 groups of frequency bands containing 8 acceleration low frequency bands and 3 groups of frequency bands containing 8 angular velocity low frequency bands.

[0040] As can be seen from the above, the high-frequency part containing noise with a frequency greater than the first threshold value can be separated from the first vehicle machine inertia data and the first controller inertia data through the first transformation. The noise with a frequency greater than the first threshold value in the first vehicle machine inertia data and the first controller inertia data can be removed by simply clearing the high-frequency part or not outputting it.

[0041] In some embodiments of the present invention, the sampling frame rate of the first vehicle speed data, the first steering wheel angle data, and the first positioning data is 20 Hz, and data with a frequency of 0-10 Hz can be collected. Among the collected first vehicle speed data, the first steering wheel angle data, and the first positioning data, the data corresponding to the frequency band of 5-10 Hz mainly appears as noise.

[0042] In the process of performing a second transformation on the first vehicle speed data, the first steering wheel angle data and the first positioning data to obtain second vehicle speed data, second steering wheel angle data and second positioning data, a second wavelet transform can be used as the second transformation, with 5 Hz as the second threshold based on the above noise frequency.

[0043] The first vehicle speed data is decomposed into a high frequency part with a frequency interval of (5,10] and a low frequency part with a frequency interval of (0,5] according to the second threshold value 5Hz by using the second wavelet transform, and the high frequency part is removed, and the low frequency part decomposed from the first vehicle speed data is used as the second vehicle speed data of the vehicle computer; the first steering wheel angle data is decomposed into a high frequency part with a frequency interval of (5,10] and a low frequency part with a frequency interval of (0,5] according to the second threshold value 5Hz by using the second wavelet transform, and the high frequency part is removed, and the low frequency part decomposed from the first steering wheel angle data is used as the second steering wheel angle data of the vehicle computer; the first positioning data is decomposed into a high frequency part with a frequency interval of (5,10] and a low frequency part with a frequency interval of (0,5] according to the second threshold value 5Hz by using the second wavelet transform, and the high frequency part is removed, and the low frequency part decomposed from the first positioning data is used as the second positioning data of the vehicle computer.

[0044] As can be seen from the above, through the second transformation, the high-frequency part containing noise with a frequency greater than the second threshold can be separated from the first vehicle speed data, the first steering wheel angle data and the first positioning data. The noise with a frequency greater than the second threshold in the first vehicle speed data, the first steering wheel angle data and the first positioning data can be removed by simply clearing the high-frequency part or not outputting it.

[0045] In some embodiments, the process of performing a second transformation on the first vehicle speed data, the first steering wheel angle data and the first positioning data to obtain the second vehicle speed data, the second steering wheel angle data and the second positioning data further includes: removing the outliers in the low-frequency part decomposed from the first vehicle speed data, the first steering wheel angle data and the first positioning data according to a preset offset range. Among them, outliers (Outliers) are also called escape points, which refer to data with a large difference compared with other data in the data. In this embodiment, 3 times the standard deviation is used as the preset offset range to remove the outliers in each low-frequency part decomposed from the first vehicle speed data, the first steering wheel angle data and the first positioning data that exceed the preset offset range. In other feasible embodiments, 4, 5 or other multiples of the standard deviation can also be used as the preset offset range. In other implementations, variance or other indicators can also be used as the unit of the preset offset range. Removing the outliers in the low-frequency band decomposed from the first vehicle speed data, the first steering wheel angle data and the first positioning data according to the preset offset range can further denoise them, thereby improving the accuracy of the first vehicle speed data, the first steering wheel angle data and the first positioning data.

[0046] In some optional embodiments, the second vehicle machine inertia data and the second controller inertia data include an acceleration low frequency band and an angular velocity low frequency band. At this time, the second vehicle machine inertia data is subjected to noise processing based on the time series correspondence between the second vehicle machine inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data through the first time series analysis model, and the third vehicle machine inertia data is obtained, including:

[0047] First, the second vehicle speed data, the second steering wheel angle data, and the second positioning data are taken as three input data, and each acceleration low-frequency band and each angular velocity low-frequency band included in the second vehicle inertia data are taken as one input data. In this embodiment, the second vehicle inertia data includes three frequency band groups each including eight acceleration low-frequency bands and three frequency band groups each including eight angular velocity low-frequency bands, that is, the second vehicle inertia data is divided into 48 input data according to the low-frequency bands, and 51 input data for the first timing analysis model are obtained.

[0048] Afterwards, the 51 input data for the first timing analysis model are input into the first timing analysis model together. Then, based on the timing correspondence between each low-frequency band in the second vehicle machine inertial data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data, the first timing analysis model is used to remove the noise with a frequency less than the first threshold and the noise with a frequency equal to the first threshold in each low-frequency band of the second vehicle machine inertial data. In this way, the first acceleration data corresponding to the acceleration low-frequency band and the first angular velocity data corresponding to the angular velocity low-frequency band as the input data of the first timing analysis model can be obtained and output, and the third vehicle machine inertial data composed of the first acceleration data and the first angular velocity data can be obtained.

[0049] Furthermore, the process of performing noise processing on the second vehicle machine inertia data based on the time series correspondence between the second controller inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data through the second time series analysis model to obtain the inertia data of the third controller includes:

[0050] First, the second vehicle speed data, the second steering wheel angle data, and the second positioning data are taken as three input data, and each acceleration low-frequency band and each angular velocity low-frequency band included in the second controller inertial data are taken as one input data. In this embodiment, the second controller inertial data includes 3 frequency band groups each including 8 acceleration low-frequency bands and 3 frequency band groups each including 8 angular velocity low-frequency bands, that is, the second controller inertial data is divided into 48 input data according to the low-frequency bands, and 51 input data for the second timing analysis model are obtained.

[0051] The 51 input data for the second timing analysis model are input into the second timing analysis model together, and then the second timing analysis model is used to remove the noise with a frequency less than the first threshold and the noise with a frequency equal to the first threshold in each low-frequency band of the inertial data of the second controller based on the timing correspondence between each low-frequency band in the inertial data of the second controller and the second vehicle speed data, the second steering wheel angle data, and the second positioning data. In this way, the second acceleration data corresponding to the acceleration low-frequency band and the second angular velocity data corresponding to the angular velocity low-frequency band as the input data of the second timing analysis model can be obtained and output, and the third controller inertial data composed of the second acceleration data and the second angular velocity data can be obtained.

[0052] In the above process, the noise with a frequency less than the first threshold and the noise with a frequency equal to the first threshold in the second vehicle machine inertial data and the second controller inertial data can be removed through the first timing analysis model and the second timing analysis model based on the timing correspondence between each low-frequency band in the second vehicle machine inertial data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data, and the timing correspondence between each low-frequency band in the second controller inertial data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data. This process further removes the noise with a frequency less than the first threshold and the noise with a frequency equal to the first threshold on the basis of removing the noise with a frequency greater than the first threshold, and can achieve comprehensive noise reduction, so that the posture change of the vehicle machine can be parsed from the third vehicle machine inertial data, and the posture change of the 3DOF controller can be parsed from the third controller inertial data. After being processed by the fusion control method of the embodiment of the present application, the posture information of the 3DOF controller in the driving state can be normally parsed.

[0053] As an optional embodiment of the present invention, the first time series analysis model and the second time series analysis model are LSTM (Long Short-term Memory) models. The LSTM model is a time recurrent neural network model that can solve the long-term dependency problem of the ordinary RNN (Recurrent Neural Network) model, and can be used to process and predict important events with very long intervals and delays in time series. It can analyze and express data or signals from a longer time dimension, so it can remove noise in the data. In this embodiment, it is used to calculate the correlation between the second vehicle inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data, and calculate the correlation between the second controller inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data, and denoise based on the calculated correlation, respectively, to remove the noise with a frequency less than the first threshold and the noise with a frequency equal to the first threshold in the second vehicle inertia data and the second controller inertia data.

[0054] like Figure 2As shown, in this embodiment, the first transformation adopts the first wavelet transformation, and the second transformation adopts the second wavelet transformation. The 3-axis acceleration data and the 3-axis angular velocity data in the first vehicle computer inertial data or the first controller inertial data are respectively processed by the first wavelet transformation to obtain a series of acceleration low-frequency bands and angular velocity low-frequency bands, wherein the acceleration low-frequency band can be divided into 3 groups corresponding to the 3 axes, each of which contains 8 segments of acceleration low-frequency bands, and the angular velocity low-frequency band can be divided into 3 groups corresponding to the 3 axes, each of which contains 8 segments of angular velocity low-frequency bands.

[0055] Considering the balance between network expression ability and computing power requirements, the LSTM model 300 includes three layers of LSTM networks, two fully connected layers and one output layer, wherein the first layer of LSTM network 311 is the input layer with a dimension of 51; the number of nodes of the second layer of LSTM network 312 and the third layer of LSTM network 313 is set to 128, which is used to extract the time series features of the input data; the number of nodes of the first layer of fully connected layer 321 and the second layer of fully connected layer 322 are 128 and 64 respectively, which are used to receive the time series features extracted by the third layer of LSTM network 313. The output layer 330 has 6 nodes, which are used to output 3-way acceleration data (corresponding to 3 coordinate axes) and 3-way angular velocity data (corresponding to 3 coordinate axes) after denoising, wherein the 3-way acceleration data outputted constitute the first acceleration data or the second acceleration data, and the 3-way angular velocity data constitute the first angular velocity data or the second angular velocity data. The third vehicle inertia data can be obtained from the first acceleration data and the first angular velocity data, and the third controller inertia data can be obtained from the second acceleration data and the second angular velocity data. After that, the third vehicle inertia data and the third controller inertia data are fused to obtain the 3DOF controller attitude information used to characterize the user command.

[0056] This embodiment utilizes the characteristics of the LSTM model that can be used to process and predict important events with long intervals and delays in a time series. The LSTM model is used to calculate the correlation between the second vehicle computer inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data, as well as the correlation between the second controller inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data, and denoising is performed based on the calculated correlations. On the basis of the above-mentioned removal of noise greater than the first threshold, the noise with a frequency less than the first threshold and the noise with a frequency equal to the first threshold in the second vehicle computer inertia data and the second controller inertia data can be removed to obtain fully denoised third vehicle computer inertia data and third controller inertia data, thereby avoiding the influence of noise on the posture analysis of the vehicle computer and the 3DOF controller.

[0057] Furthermore, in order to prevent the LSTM model from overfitting, the random deactivation parameter can be set to 0.5 in the LSTM model, so that the LSTM model can be randomly deactivated during training. Among them, the overfitting phenomenon refers to the phenomenon that the model fits the training set too well during training, which makes it unable to correctly predict unknown data. Overfitting will lead to poor generalization ability of the model; random deactivation processing is dropout processing, which means that during the training process of the deep learning network, a part of the neural network units are temporarily discarded from the network according to the probability specified by the random deactivation parameter, which is equivalent to finding a thinner network from the original network. In this application, the random deactivation parameter is set to 0.5, and during the training process of the LSTM network, a part of the neural network units can be temporarily discarded from the network according to the probability of 0.5, so as to prevent the LSTM model from overfitting, thereby making the analysis of the LSTM model more accurate.

[0058] According to the second aspect of the embodiment of the present application, Figure 3 As shown, a vehicle computer and 3DOF controller fusion control system is provided, including: a vehicle computer and a 3DOF controller.

[0059] The vehicle computer includes a first inertial measurement unit 811, a first communication unit 812, a speed detection unit 813, a steering wheel angle detection unit 814, and a positioning device 815, wherein the first inertial measurement unit 811, the first communication unit 812, the speed detection unit 813, the steering wheel angle detection unit 814, and the positioning device 815 are respectively used to collect first vehicle computer inertial data, first vehicle speed data, first steering wheel angle data, and first positioning data, and the first communication unit 812 is used for data transmission; in this embodiment, the first inertial measurement unit 811 includes an accelerometer and a gyroscope, the first communication unit 812 includes an RF module, the speed detection unit 813 can be a speed measuring device provided by the vehicle or an additional speed sensor, the steering wheel angle detection unit 814 can be an angle measuring device provided by the vehicle or an additional steering angle sensor, and the positioning device 815 can be a suitable positioning device such as a GPS device or a Beidou navigation device.

[0060] The vehicle computer may include: a processor 816 , a memory 817 , a communication interface 818 , and a communication bus 819 .

[0061] Among them, the communication interface 818 is used to communicate with other electronic devices or servers, and the above-mentioned first inertial measurement unit, first communication unit, speed detection unit, steering wheel angle detection unit, positioning device, processor, memory and communication interface communicate with each other through a communication bus.

[0062] The processor 816 is used to execute the program 820, and specifically can execute the relevant steps in the above method embodiment.

[0063] Specifically, the program 820 may include program codes, which include computer operation instructions.

[0064] The processor 816 may be a CPU, or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0065] The memory 817 is used to store the program 820. The memory 817 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0066] The program 820 can be specifically used to enable the processor 816 to execute operations corresponding to the method described in any one of the aforementioned method embodiments.

[0067] The specific implementation of each step in program 820 can refer to the corresponding description of the corresponding steps and units in the above method embodiment, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above method embodiment, which will not be repeated here.

[0068] The above-mentioned 3DOF controller is paired with the vehicle computer. The 3DOF controller includes a second inertial measurement unit and a second communication unit. The second communication unit can transmit the data collected by the second inertial measurement unit to the first communication unit, so that the processor in the vehicle computer can execute the operations corresponding to the above-mentioned vehicle computer and 3DoF controller fusion control method. In this embodiment, the first inertial measurement unit includes an accelerometer and a gyroscope, and the second communication unit includes an RF module.

[0069] Using the system provided in this embodiment, the processor can execute operations corresponding to the vehicle computer and 3DoF controller fusion control method in any embodiment provided in the first aspect of this application, so the system can achieve the effect achieved by the embodiment provided in the first aspect of this application, so that the 3DOF controller can normally control the XR application even in the driving state.

[0070] According to a third aspect of the embodiment of the present application, a vehicle is provided, the vehicle comprising the vehicle-machine and 3Dof controller fusion control system provided in the second aspect of the present application. Referring to the embodiment of the second aspect of the present application, the vehicle can also achieve the effect achieved by the first aspect of the present application by using the vehicle-machine and 3Dof controller fusion control system provided in the second aspect of the present application, so that the 3DOF controller can also normally control the XR application in the driving state.

[0071] According to the fourth aspect of the embodiments of the present application, a computer storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, a vehicle-machine and 3Dof controller fusion control method as in any embodiment provided in the first aspect of the present application is implemented. By implementing the vehicle-machine and 3Dof controller fusion control method as in any embodiment provided in the first aspect of the present application when the program is executed by a processor, the noise reduction system can achieve the effect achieved by the embodiment provided in the first aspect of the present application, so that the 3DOF controller can normally control the XR application even in the driving state.

[0072] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0073] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0074] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.

Claims

1. A vehicle computer and 3DOF controller fusion control method, characterized in that: include: Acquire first vehicle computer inertia data of the vehicle computer during driving, first controller inertia data of a 3DOF controller paired with the vehicle computer, and first vehicle speed data, first steering wheel angle data, and first positioning data of the vehicle computer; Performing a first transformation on the first vehicle machine inertia data and the first controller inertia data to obtain second vehicle machine inertia data and second controller inertia data; Performing a second transformation on the first vehicle speed data, the first steering wheel angle data and the first positioning data to obtain second vehicle speed data, second steering wheel angle data and second positioning data; performing noise processing on the second vehicle computer inertia data based on the time series correspondence between the second vehicle computer inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data through a first time series analysis model to obtain third vehicle computer inertia data; Performing noise processing on the second vehicle computer inertia data based on the time series correspondence between the second controller inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data through a second time series analysis model to obtain third controller inertia data; Based on the inertial data of the third vehicle computer and the inertial data of the third controller, a fusion process is performed to obtain 3DOF posture information of the controller in a driving state; The first vehicle machine inertia data and the first controller inertia data include 3-axis acceleration data and 3-axis angular velocity data, and performing a first transformation on the first vehicle machine inertia data and the first controller inertia data to obtain the second vehicle machine inertia data and the second controller inertia data includes: Decomposing the acceleration data of each axis in the three-axis acceleration data in the first vehicle machine inertial data into a high-frequency part and a low-frequency part according to a first threshold value by using the first transformation, removing the high-frequency part, and decomposing the low-frequency parts corresponding to the three axes into N acceleration low-frequency bands, to obtain three frequency band groups including the N acceleration low-frequency bands; and decomposing the angular velocity data of each axis in the three-axis angular velocity data in the first vehicle machine inertial data into a high-frequency part and a low-frequency part according to the first threshold value by using the first transformation, removing the high-frequency part, and decomposing the low-frequency parts corresponding to the three axes into N angular velocity low-frequency bands, to obtain three frequency band groups including the N angular velocity low-frequency bands; obtaining the second vehicle machine inertial data including the three frequency band groups including the N acceleration low-frequency bands and the three frequency band groups including the N angular velocity low-frequency bands; Decomposing the acceleration data of each axis in the 3-axis acceleration data in the inertial data of the first controller into a high-frequency part and a low-frequency part according to the first threshold value by using the first transformation, removing the high-frequency part, and decomposing the low-frequency parts corresponding to the three axes into M acceleration low-frequency bands, so as to obtain three frequency band groups including the M acceleration low-frequency bands; and decomposing the angular velocity data of each axis in the 3-axis angular velocity data in the inertial data of the first controller into a high-frequency part and a low-frequency part according to the first threshold value by using the first transformation, removing the high-frequency part, and decomposing the low-frequency parts corresponding to the three axes into M angular velocity low-frequency bands, so as to obtain three frequency band groups including the M angular velocity low-frequency bands; obtaining the inertial data of the second controller including the three frequency band groups including the M acceleration low-frequency bands and the three frequency band groups including the M angular velocity low-frequency bands; Wherein, N and M are positive integers.

2. The fusion control method according to claim 1, characterized in that If the second vehicle machine inertia data and the second controller inertia data include an acceleration low-frequency band and an angular velocity low-frequency band, the first timing analysis model is used to perform noise processing on the second vehicle machine inertia data based on the timing correspondence between the second vehicle machine inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data to obtain the third vehicle machine inertia data, including: The second vehicle speed data, the second steering wheel angle data, and the second positioning data are used as three input data, and each section of the acceleration low-frequency band and each section of the angular velocity low-frequency band included in the second vehicle inertial data are used as one input data respectively, to obtain a plurality of input data for the first timing analysis model; Inputting a plurality of input data for the first timing analysis model into the first timing analysis model together, removing noise with a frequency less than a first threshold and noise with a frequency equal to the first threshold in each low-frequency band in the second vehicle machine inertial data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data through the first timing analysis model based on the timing correspondence between each low-frequency band in the second vehicle machine inertial data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data, and obtaining and outputting first acceleration data corresponding to the acceleration low-frequency band and first angular velocity data corresponding to the angular velocity low-frequency band as input data for the first timing analysis model; The step of performing noise processing on the second vehicle computer inertia data based on the time series correspondence between the second controller inertia data and the second vehicle speed data, the second steering wheel angle data, and the second positioning data through the second time series analysis model to obtain the third controller inertia data includes: The second vehicle speed data, the second steering wheel angle data, and the second positioning data are used as three input data, and each section of the acceleration low-frequency band and each section of the angular velocity low-frequency band included in the second controller inertial data are used as one input data respectively, to obtain a plurality of input data for the second timing analysis model; Several input data for the second timing analysis model are input into the second timing analysis model together. The second timing analysis model removes noise with a frequency less than the first threshold and noise with a frequency equal to the first threshold in each low-frequency band of the inertial data of the second controller based on the timing correspondence between each low-frequency band in the inertial data of the second controller and the second vehicle speed data, the second steering wheel angle data, and the second positioning data, and obtains and outputs second acceleration data corresponding to the acceleration low-frequency band and second angular velocity data corresponding to the angular velocity low-frequency band as input data of the second timing analysis model.

3. The fusion control method according to claim 1, characterized in that: The performing a second transformation on the first vehicle speed data, the first steering wheel angle data and the first positioning data to obtain second vehicle speed data, second steering wheel angle data and second positioning data comprises: Decomposing the first vehicle speed data into a high-frequency part and a low-frequency part according to a second threshold value by using the second transformation, removing the high-frequency part, and using the low-frequency part decomposed from the first vehicle speed data as the second vehicle speed data of the vehicle computer; Decomposing the first steering wheel angle data into a high-frequency part and a low-frequency part according to a second threshold value by using the second transformation, removing the high-frequency part, and using the low-frequency part decomposed from the first steering wheel angle data as the second steering wheel angle data of the vehicle computer; The first positioning data is decomposed into a high-frequency part and a low-frequency part according to a second threshold value by using the second transformation, the high-frequency part is removed, and the low-frequency part decomposed from the first positioning data is used as the second positioning data of the vehicle computer.

4. The fusion control method according to claim 3, characterized in that: The second transformation of the first vehicle speed data, the first steering wheel angle data and the first positioning data to obtain second vehicle speed data, second steering wheel angle data and second positioning data also includes: removing external points in the low-frequency part decomposed from the first vehicle speed data, the first steering wheel angle data and the first positioning data according to a preset offset range.

5. The fusion control method according to any one of claims 1 to 4, characterized in that: The first time series analysis model and / or the second time series analysis model is an LSTM model.

6. The fusion control method according to claim 5, characterized in that: The random dropout parameter is set to 0.5 in the LSTM model.

7. A vehicle computer and 3DOF controller fusion control system, characterized in that: include: Car computer and 3DOF controller, The vehicle computer includes a first inertial measurement unit, a first communication unit, a speed detection unit, a steering wheel angle detection unit, and a positioning device, wherein the first inertial measurement unit, the first communication unit, the speed detection unit, the steering wheel angle detection unit, and the positioning device are respectively used to collect first vehicle computer inertial data, first vehicle speed data, first steering wheel angle data, and first positioning data, and the first communication unit is used to perform data transmission; The vehicle computer further includes: a processor, a memory, a communication interface and a communication bus, wherein the first inertial measurement unit, the first communication unit, the speed detection unit, the steering wheel angle detection unit, the positioning device, the processor, the memory and the communication interface communicate with each other through the communication bus, and the memory is used to store at least one executable instruction, wherein the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1 to 6 to realize 3DOF control in a driving state; The 3DOF controller is paired with the vehicle computer. The 3DOF controller includes a second inertial measurement unit and a second communication unit. The second communication unit can transmit data collected by the second inertial measurement unit to the first communication unit.

8. A vehicle, characterized in that: The vehicle comprises the control system of claim 7.

9. A computer storage medium, characterized in that The computer storage medium stores a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images