Man-machine interaction system and method based on two-wheeled vehicle

Abstract

The invention discloses a man-machine interaction system and method based on a two-wheeled vehicle, and relates to the field of man-machine interaction, and the man-machine interaction system based on the two-wheeled vehicle comprises a microphone array module which is used for obtaining a voice signal of a surrounding environment; wherein the microphone array module comprises a microphone array, the microphone array comprises a plurality of microphone units, the plurality of microphone units are uniformly distributed in the peripheral area of the human-computer interaction interface, and the microphone array forms a multi-MIC matrix; the voice data processing module is used for processing the voice signals transmitted by the microphone array module; the voice data processing module comprises a preprocessing unit, a time difference estimation unit, a direction calculation unit and a beam forming unit; and the control module is used for converting the processed voice signal into a control signal so as to realize voice control of the two-wheeled vehicle function. According to the method and the device, potential safety hazards caused by manual operation of a driver in the driving process can be avoided.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction, and in particular to a human-computer interaction system and method based on a two-wheeled vehicle. Background Art

[0002] As people's living standards continue to improve, two-wheeled vehicles have been widely used as a convenient means of transportation in short-distance urban travel. With its lightness, flexibility and easy parking, it has become the first choice for many people's daily travel. At the same time, with the continuous advancement of artificial intelligence technology, the intelligence level of two-wheeled vehicles is also constantly improving. Among them, the human-computer interaction system is crucial. It is not only the core of the intelligence of two-wheeled vehicles, but also enables the interaction between vehicles and users to improve user experience.

[0003] However, the existing human-computer interaction system for two-wheeled vehicles can only interact with the two-wheeled vehicle by touching the screen. This operation method may distract the driver's attention and thus affect driving safety. Summary of the invention

[0004] The purpose of the present application is to provide a human-computer interaction system and method based on a two-wheeled vehicle, which can avoid manual operation of the driver during the operation process.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a human-computer interaction system based on a two-wheeled vehicle, comprising:

[0007] Microphone array module, used to obtain voice signals from the surrounding environment;

[0008] Wherein, the microphone array module includes a microphone array, the microphone array includes a plurality of microphone units, the plurality of microphone units are evenly distributed in the surrounding area of ​​the human-computer interaction interface, and the microphone array forms a multi-MIC matrix;

[0009] A voice data processing module, used for processing the voice signal transmitted by the microphone array module;

[0010] Wherein, the voice data processing module includes:

[0011] A preprocessing unit, used for preprocessing the speech signal;

[0012] A time difference estimation unit, used for estimating time difference information of the speech signals received by different microphone units;

[0013] A direction calculation unit, used to obtain the direction of the sound source according to the time difference information and the layout of the microphone array;

[0014] A beam forming unit, used to adjust the weight according to the direction of the sound source in different usage scenarios to form a beam in a specified range, thereby enhancing the speech signal in the target direction;

[0015] The control module is used to convert the processed voice signal into a control signal, and control at least one functional module to perform a specified operation through the control signal, thereby realizing voice control of the two-wheeled vehicle function, and displaying the operation result on the human-computer interaction interface.

[0016] In a second aspect, the present application provides a human-computer interaction method based on a two-wheeled vehicle, comprising:

[0017] Acquire the voice signal of the surrounding environment based on the microphone array module;

[0018] Wherein, the microphone array module includes a microphone array, the microphone array includes a plurality of microphone units, the plurality of microphone units are evenly distributed in the surrounding area of ​​the human-computer interaction interface, and the microphone array forms a multi-MIC matrix;

[0019] Processing the voice signal transmitted by the microphone array module;

[0020] Wherein, processing the voice signal transmitted by the microphone array module includes:

[0021] Preprocessing the voice signal transmitted by the microphone array module;

[0022] estimating time difference information of the speech signals received by different microphone units;

[0023] Obtaining a sound source direction according to the time difference information and the layout of the microphone array;

[0024] In different usage scenarios, the weight is adjusted according to the direction of the sound source to form a beam in a specified range, thereby enhancing the speech signal in the target direction;

[0025] The processed voice signal is converted into a control signal, and at least one functional module is controlled by the control signal to perform a specified operation, thereby realizing voice control of the two-wheeled vehicle function, and the operation result is displayed on the human-computer interaction interface.

[0026] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0027] The present application provides a human-computer interaction system and method based on a two-wheeled vehicle. The human-computer interaction system based on a two-wheeled vehicle includes: a microphone array module for acquiring voice signals in the surrounding environment; wherein the microphone array module includes a microphone array, the microphone array includes a plurality of microphone units, the plurality of microphone units are evenly distributed in the surrounding area of ​​the human-computer interaction interface, and the microphone array forms a multi-MIC matrix for acquiring voice signals in the surrounding environment; in order to process these voice signals, the system further includes a voice data processing module for processing the voice signals transmitted by the microphone array module; wherein the voice data processing module includes: a preprocessing unit for preprocessing the voice signals; a time difference estimation unit for estimating the time difference between different microphones; The microphone unit receives the time difference information of the voice signal; the direction calculation unit is used to obtain the sound source direction according to the time difference information and the layout of the microphone array; the beam forming unit adjusts the weight according to the sound source direction according to different usage scenarios to form a beam within a specified range. This beam can effectively enhance the voice signal in the target direction and suppress interference from other directions, thereby improving the accuracy of voice recognition; the control module is used to convert the processed voice signal into a control signal, and control one or more functional modules on the two-wheeled vehicle to perform specified operations through the control signal, thereby realizing voice control of the two-wheeled vehicle function, and displaying the operation results on the human-computer interaction interface, so that the user can intuitively see the results of the operation and the status of the vehicle. Manual operation by the driver during the operation is avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 This is a schematic diagram of the structure of a human-computer interaction system based on a two-wheeled vehicle according to an embodiment of the present application;

[0030] Figure 2 A schematic flow chart of a human-computer interaction method based on a two-wheeled vehicle according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0033] Figure 1 is a schematic diagram of the structure of a human-computer interaction system based on a two-wheeled vehicle provided by an embodiment of the present invention. Figure 1 As shown, the human-computer interaction system includes: a microphone array module, a voice data processing module and a control module.

[0034] Specifically, the microphone array module is used to obtain voice signals from the surrounding environment and accurately locate the sound source. The microphone array module includes a microphone array, and the microphone array includes a plurality of microphone units, and the plurality of microphone units are evenly distributed in the surrounding area of ​​the human-computer interaction interface. For example, for the top edge or side edge of the human-computer interaction interface, the microphone units are arranged equidistantly along the top edge or the side edge to form an array layout of a specific geometric shape, such as a linear array, a circular array or a planar array. When the array layout is a linear array, the microphones are arranged along a straight line, which is suitable for sound source localization in one dimension. When the array layout is a circular array, the microphones are arranged around a circle, which is suitable for sound source localization in two dimensions. When the array layout is a planar array, the microphones are distributed on a two-dimensional plane, which is suitable for sound source localization in three dimensions. Together, they constitute a multi-MIC matrix, through which voice signals from different directions can be captured.

[0035] The voice data processing module is used to process the voice signal transmitted by the microphone array module. The voice data processing module includes a preprocessing unit, a time difference estimation unit, a direction calculation unit and a beam forming unit. The preprocessing unit is used to preprocess the voice signal; the time difference estimation unit is used to estimate the time difference information of the voice signal received by different microphone units; the direction calculation unit is used to obtain the sound source direction according to the time difference information and the layout of the microphone array; the beam forming unit is used to adjust the weight according to the sound source direction in different usage scenarios to form a beam of a specified range, thereby enhancing the voice signal in the target direction.

[0036] Specifically, processing the voice signal transmitted by the microphone array module by the voice data processing module includes the following steps:

[0037] A1. Preprocess the speech signal, including signal noise reduction, signal filtering and normalization.

[0038] Specifically, in the process of preprocessing the speech signal, first, the background noise is removed by noise reduction to improve the signal-to-noise ratio, ensure the clarity of the speech signal, and reduce the impact of environmental interference on the speech signal. By using a bandpass filter to remove unnecessary frequency components, the frequency components that may interfere with speech recognition or analysis are eliminated, thereby further improving the quality of the signal. The amplitude of the signal is normalized through normalization processing to keep it within a standard range to avoid excessive differences in signal amplitude.

[0039] A2. Estimate the time difference between different microphones through the delay estimation algorithm. The delay estimation algorithm is expressed by the following formula:

[0040]

[0041] Among them, τ ij represents the time difference between the i-th and j-th channels; k represents a discrete time point or an index in the frequency domain; N represents the length of the Fourier transform; e -j2πkτ / N represents the operation of shifting the signal in the frequency domain, j is an imaginary unit, and τ is the delay position of the maximum correlation to be found; argmax τ It means to find the value that makes the expression reach the maximum value; X i (k) represents the signal value of the i-th channel at the k-th time; Represents the conjugate signal value of the j-th channel at the k-th time.

[0042] The delay estimation algorithm performs Fourier transform on the signal received by each microphone, calculates its cross-power spectrum density, performs phase transform in the frequency domain, and finally obtains the cross-correlation function in the time domain through inverse Fourier transform. By analyzing the peak position of the cross-correlation function, the time difference between different microphones can be accurately estimated.

[0043] A3. Calculate the sound source direction based on the time difference information and the layout of the microphone array. The sound source direction is reflected by the angle. The angle calculation formula is as follows:

[0044]

[0045] Where c is the speed of sound and d is the distance between microphones.

[0046] A4. According to the calculated sound source direction, adjust the weight of the microphone array to form a beam pointing in that direction.

[0047] The weight of each microphone is calculated as follows:

[0048]

[0049] Among them, d irepresents the distance from the ith microphone to the center of the array; f represents the Doppler frequency; and c represents the speed of sound.

[0050] The calculated weights are applied to the signal from each microphone to form a beam signal:

[0051]

[0052] Among them, x i (t) represents the signal of the i-th microphone, and y(t) represents the synthesized beam signal.

[0053] Beamforming is achieved by adjusting the weight of each microphone in the microphone array so that the synthesized beam has the highest sensitivity in a specific direction and lower sensitivity in other directions. Common beamforming methods include fixed beamforming and adaptive beamforming. The weight of each microphone is calculated based on the direction of the sound source and the geometric layout of the microphone array. The calculated weight is applied to the signal of each microphone to synthesize a beam pointing in a specific direction. In order to ensure the quality and clarity of the signal, the formed beam signal is subjected to noise reduction processing to remove unnecessary background noise in the signal and improve the purity of the signal. The automatic gain control method automatically adjusts the gain according to the strength of the signal to avoid the signal being too strong or too weak, ensuring a stable output signal.

[0054] When in a riding state, the weight is adjusted according to the direction of the sound source to form a beam within a first preset distance within a range of 30° directly behind the human-computer interaction interface; when in a parking state, the weight is adjusted according to the direction of the sound source to form a beam within a second preset distance within a range of 360° with the human-computer interaction interface as the center.

[0055] Specifically, when the vehicle is in a driving state, by using the integrated multi-MIC matrix and the above-mentioned voice signal processing algorithm, the system will only recognize voice commands within a range of 1-2 meters behind the device. Usually, the first preset distance is related to the length of the two-wheeled vehicle along the body direction. The first preset distance can be set to about 1.5 meters, and these voice commands come from the 30° angle directly behind the multi-MIC matrix. The selection of the 30° angle can be centered on the multi-MIC matrix, with the length direction of the two-wheeled vehicle as the angle bisector, and the angle bisector is set to about 15° on both sides. Ensure that only the driver or passenger on the vehicle can voice control the system, thereby avoiding misoperation caused by interfering sounds in the external environment. When the vehicle is stopped, the system will switch to a wider 360°, 10-15 meters away voice recognition mode, and the second preset distance can also be set according to actual needs. In this way, it is not limited to the people in the car, but also allows users within a certain range outside the car to interact with the system through voice, thereby improving the flexibility and convenience of use.

[0056] The control module is used to convert the processed voice signal into a control signal, and control at least one functional module to perform a specified operation through the control signal. The voice control capability of various functions of the two-wheeled vehicle is realized through the function of the control module. In order to ensure that the user can clearly understand the operation results, the control module will also display these results on the human-computer interaction interface to provide intuitive feedback information. In the process of processing the voice signal, the control module will extract useful voice features from the voice signal, such as Mel Frequency Cepstral Coefficients (MFCC). The control module parses the voice command through the voice recognition engine. The voice recognition engine will match the extracted features with the preset command library to accurately parse the user's voice command. Once a specific command is recognized, the control module will quickly control the corresponding functional module, and the functional module will perform the corresponding operation according to the content of the audio signal. These operations may include starting the motor, adjusting the speed, changing the driving direction, turning on the navigation, adjusting the volume, making a call, turning on the camera, playing music, etc., depending on the content of the voice command issued by the user. In this way, the user can operate the two-wheeled vehicle with simple voice commands. The control module ensures that the two-wheeled vehicle can flexibly perform various functions according to the user's voice command, improving the user's operating convenience and interactive experience.

[0057] Functional modules may include navigation modules, entertainment modules, safety modules, volume adjustment modules, phone dialing modules, and camera control modules. After receiving the control command, each module will perform corresponding operations, which may include but are not limited to opening applications, sending messages, adjusting system settings, etc. For example, the navigation module is controlled to plan routes, the entertainment module is controlled to play music, the safety module is controlled to perform emergency braking or warning, the volume adjustment module is controlled to adjust the volume, the phone dialing module is controlled to make or answer calls, and the camera module is controlled to capture images or monitor. The results of these operations will be fed back to the human-computer interaction interface and displayed to the user in the form of graphics, text or sound, so that the user can intuitively see the effect of the execution of the command. In this way, the voice data acquisition module not only improves the convenience of the user's operation of the two-wheeled vehicle, but also enhances the naturalness and intuitiveness of human-computer interaction, making the driving experience smoother and more pleasant, while avoiding the driver's distraction during driving, thereby affecting driving safety.

[0058] The human-computer interaction system based on the two-wheeled vehicle also includes a gyroscope. The system can accurately determine whether the two-wheeled vehicle is currently in a riding state or a stopped state by obtaining the state signal of the two-wheeled vehicle through the gyroscope. The gyroscope is connected to the sensor module, and the sensor module is used to obtain the state signal of the gyroscope. The state signal is then transmitted to the control module, and the control module transmits the state signal to the voice data processing module. The voice data processing module selects the corresponding voice recognition mode according to the state of the two-wheeled vehicle.

[0059] Specifically, the gyroscope can detect the change in angular velocity of the vehicle in different axial directions, and the angular velocity change signal is the state signal of the two-wheeled vehicle, thereby determining whether the vehicle is in motion. The gyroscope obtains three-axis angular velocity, which refers to the angular velocity value generated when the object rotates in the three mutually perpendicular directions of the X-axis, Y-axis and Z-axis. When the vehicle is stationary, the reading of the gyroscope is usually close to zero, indicating that the vehicle has not rotated. However, once the vehicle starts to move, the gyroscope will immediately detect additional angular velocity changes, especially in the X-axis and Y-axis directions, because the angular velocity changes in these two directions are usually closely related to the steering and tilting movements of the vehicle. The gyroscope usually exchanges data with the vehicle's main control system through two communication interfaces, I2C (Inter-Integrated Circuit) or SPI (Serial Peripheral Interface), and transmits angular velocity data in real time, so that the vehicle's control system can accurately obtain the vehicle's dynamic information.

[0060] In one practicable manner, the human-machine interaction interface is used to display the driving status information and various warning information of the two-wheeled vehicle so that the user can understand the running status of the vehicle in real time. The human-machine interaction interface includes a liquid crystal display screen, and the human-machine interaction interface is also used to control at least one function of the two-wheeled vehicle by touching the liquid crystal display screen; the liquid crystal display screen is arranged above the instrument panel housing of the two-wheeled vehicle.

[0061] The human-machine interaction interface includes an LCD screen, which is used to display various data and information. It also has a touch function, allowing users to control various functions of the vehicle through simple touch operations, improving the user's operational convenience and interactive experience. The LCD screen can be embedded above the dashboard shell of the two-wheeled vehicle. During riding, users can easily switch between different function options by simply touching the screen, such as viewing navigation information, playing music, adjusting the volume, making calls, etc.

[0062] Specifically, the human-machine interface is connected and communicated with the vehicle's T-BOX module via the CAN bus. This connection method enables the intelligent LCD display device to obtain the vehicle's driving status information in real time, such as the vehicle's speed, the remaining battery power, the mileage, and road condition warnings. Through these real-time data, the human-machine interface can intelligently adjust the content on the display screen, thereby significantly improving the safety and convenience of riding. The ambient light sensor monitors the light intensity of the surrounding environment in real time, obtains the corresponding brightness lumen value, and dynamically adjusts the display brightness of the LCD screen according to these ambient light data. In this way, whether it is during direct sunlight during the day or in the dim night, the display screen can automatically adjust to the appropriate brightness to ensure that the screen content is clearly visible. This adaptive brightness adjustment function not only improves the visual experience during riding, but also reduces visual fatigue caused by screen reflection or too dark, further improving the safety and comfort of riding.

[0063] In one practicable manner, the image processing module is used to process and analyze the real-time image of the environment in front of and behind the two-wheeled vehicle to ensure that changes in the surrounding environment can be captured in a timely manner. The real-time image of the environment in front of and behind the two-wheeled vehicle is obtained through a camera module.

[0064] Specifically, the camera module is used to obtain real-time image information of the front and rear environment of the two-wheeled vehicle, which can be understood as the surrounding environment of the two-wheeled vehicle. The camera module is connected to the image processing module through a video interface, and the camera module sends the acquired real-time image of the front and rear environment of the two-wheeled vehicle to the image processing module, and the image processing module sends the processing result of the real-time image to the control module, and the control module generates risk level information according to the processing result of the real-time image data, and the control module sends the risk level information to the early warning module.

[0065] In order to ensure the quality and effect of image acquisition, the resolution of the camera module needs to be greater than or equal to 1080p, and the frame rate needs to be greater than or equal to 30fps to ensure the smoothness and clarity of the image. In addition, in order to further expand the field of view, the camera module adopts a wide-angle lens to cover a wider field of view. The camera module is connected to the image processing module through MIPI, USB, and DVP interfaces. The above-mentioned camera module and image processing module are powered by a vehicle power supply or a dedicated power adapter to ensure the stable operation of the system. After receiving the real-time image data transmitted by the camera module, the image processing module will perform image recognition and analysis on obstacles in the image through a deep learning algorithm. The image recognition process in this application includes the following steps:

[0066] B1. Obtain image data containing various obstacles and mark the target objects.

[0067] B2. Train the deep learning model with labeled data sets to optimize the accuracy of the model.

[0068] B3. Deploy the trained model to the image processing module to ensure that the model can run in real time on the embedded platform.

[0069] B4. Receive image data transmitted by the camera module in real time and use the trained model to identify obstacles.

[0070] The sensor module is used to monitor the status of the two-wheeler as well as the environmental conditions around the two-wheeler to ensure that the vehicle can function properly in various environments.

[0071] Specifically, the sensor module includes the following:

[0072] Tire pressure sensors are used to obtain tire pressure information of a moving vehicle. The tire pressure information includes tire position identification and tire pressure value, thereby ensuring that the tire can maintain the best condition under any circumstances.

[0073] The fuel level sensor is used to obtain fuel level information in the fuel tank to ensure the endurance of the two-wheeled vehicle. By understanding the fuel level in real time, the user can refuel in time to avoid affecting the journey due to insufficient fuel.

[0074] The power sensor is used to monitor the remaining power of the battery and display it to the user in real time through the human-computer interaction interface, so that the user can understand the battery usage at any time and arrange the usage time and charging time reasonably.

[0075] The rain sensor is used to detect the rainfall in the external environment of the two-wheeled vehicle to prompt the user to take anti-skid measures to ensure driving safety.

[0076] The radar sensor is used to detect obstacle information around the vehicle to enhance the driving safety of the two-wheeled vehicle. By accurately detecting surrounding obstacles, the vehicle can respond in time to avoid collision.

[0077] The temperature sensor is used to monitor the temperature change information of the external environment of the two-wheeled vehicle and to maintain the normal operation of the vehicle under extreme temperature conditions.

[0078] An ambient light sensor, used to adjust the brightness of the human-computer interaction interface according to the ambient brightness to meet visual requirements under different lighting conditions;

[0079] The vehicle speed sensor is used to measure the speed of the two-wheeled vehicle in real time to help the user control the vehicle speed and ensure driving safety.

[0080] Accelerometers for motion detection, collision detection, tilt detection and vibration detection of two-wheeled vehicles.

[0081] The warning module generates warning information according to the vehicle status and driving environment of the two-wheeled vehicle, and sends the warning information to the human-computer interaction interface for display. The warning information can also be sent to the voice prompt module for alarm prompts. The voice prompt module will generate corresponding voice prompts based on the real-time processed image data and sensor information to guide the user to perform corresponding operations and ensure that the user can drive the two-wheeled vehicle safely and accurately.

[0082] Specifically, the vehicle state includes the tire pressure information, the fuel level information, the power level information, and the speed of the two-wheeled vehicle. The driving environment includes rainfall, obstacle information, and temperature change information. Among them, the obstacle information includes the road condition ahead, the distance to the vehicle ahead, the roadblock ahead, the distance to the vehicle behind, and the speed of the vehicle behind.

[0083] The warning information includes the following:

[0084] Road condition warning ahead: Combined with the obstacle recognition results of the image processing module, determine whether the road condition ahead is safe.

[0085] Distance warning for the vehicle ahead: The radar / lidar is used to measure the distance to the vehicle ahead and calculate the safe distance based on the vehicle's current speed. If the distance is too close, a warning is generated.

[0086] Roadblock warning ahead: Identify whether there are fixed or moving obstacles ahead and generate a warning.

[0087] Rear vehicle distance warning: The rear camera and radar / lidar are used to measure the distance to the rear vehicle, and the safe distance is calculated based on the vehicle's current speed. If the distance is too close, a warning is generated.

[0088] Rear vehicle speed warning: Identifies the speed of the vehicle behind you and generates a warning if the speed is too fast.

[0089] Tire pressure warning: monitors tire pressure through tire pressure sensors and generates a warning if the tire pressure is too low.

[0090] Low battery warning: The battery level is monitored through the power sensor, and a warning is generated if the battery level is too low.

[0091] Low oil warning: The oil level is monitored by the oil level sensor and a warning is generated if the oil level is too low.

[0092] Vehicle equipment abnormality warning: The vehicle equipment status sensor monitors the working status of the equipment and generates a warning if an abnormality is found.

[0093] In order to ensure driving safety and improve the driving experience, special tire pressure monitoring modules are usually installed on the two tires of the vehicle. These tire pressure detection modules can monitor and obtain the tire pressure information of the moving vehicle in real time. The tire pressure detection module usually includes a pressure sensor, a signal processor, a signal transmitter, and a signal receiver. First, the pressure sensor will be installed on the two tires to collect tire pressure information in real time. This tire pressure information not only includes the tire pressure data, but also covers the specific information of the tire and its position marking on the vehicle. The position marking refers to the specific position of the tire on the vehicle, such as the front wheel and the rear wheel.

[0094] The collected tire pressure information is sent to the signal processor by the pressure sensor. The signal processor performs necessary processing on these signals to ensure the accuracy and reliability of the information. The processed signals are then sent out through the signal transmitter. The signal receiver can receive these signals in a variety of ways. A common connection method is to connect directly to the vehicle's control system via the CAN bus. In this case, the signal receiver will send the received tire pressure information directly to the vehicle control system for real-time monitoring and analysis. In addition, the signal receiver can also be connected to the vehicle server via wireless network communication. In this case, the tire pressure information will be sent to the vehicle server, and the server will forward this information to the vehicle control system via wireless network communication. This method not only improves the flexibility of data transmission, but also facilitates the vehicle server to perform more complex data processing and analysis.

[0095] In order to further improve the safety performance of the vehicle, the tire pressure detection module can also be equipped with a detection device that can sense tire bursts. This device indirectly detects tire pressure by obtaining changes in the tire rolling radius. When a tire bursts or other abnormal conditions occur, the detection device can quickly capture these changes and send relevant information to the signal processor, thereby promptly reminding the driver to take appropriate measures.

[0096] In the vehicle control system, the received tire pressure information will be used to monitor the tire status in real time. The system will continuously analyze this data to ensure that the tire is in the best working condition. Once abnormal tire pressure is detected, the system will automatically send a warning signal to remind the driver to adjust the tire pressure or stop for inspection in time. In addition, the vehicle server can also record historical tire pressure data and predict tire wear and maintenance cycles through data analysis. In this way, users can perform preventive maintenance on tires in advance to avoid accidents caused by tire problems. In this way, the operating efficiency and safety of the vehicle have been significantly improved. Real-time monitoring and preventive maintenance not only reduce maintenance costs, but also reduce the risk of accidents caused by tire problems.

[0097] The control module receives the recognition results of the image processing module through a high-speed data interface, such as PCle. At the same time, the control module will also judge the degree of danger based on the vehicle speed information obtained by the vehicle's speed sensor and the distance information of the front and rear vehicles obtained by the radar. Among them, the judgment of the degree of danger includes speed judgment, distance judgment and comprehensive judgment. Speed ​​judgment is to judge whether it has enough reaction time based on the current speed of the two-wheeled vehicle. Distance judgment is to judge whether there is enough safe distance based on the distance between the front and rear vehicles. Combined with speed and distance information, the risk level is calculated. The calculation formula of the risk level is as follows:

[0098]

[0099] Finally, the control module sends the risk level information to the warning module, which displays the risk level information through the human-computer interaction interface, such as flashing warning lights on the dashboard and sounding an alarm to remind the driver of the current risk situation. The warning module can also provide corresponding driving suggestions according to different risk levels, such as slowing down, keeping a safe distance or changing lanes.

[0100] Specifically, the control module establishes a connection with the image processing module through a high-speed data interface technology, such as PCI Express (PCIe) or Universal Serial Bus 3.0 (USB 3.0). This connection mode enables the control module to efficiently receive image data from the image processing module and perform necessary processing operations on the image data. The control module is connected to the sensor module through a wired interface technology, such as a controller area network (CAN) bus, so that it can receive sensor data transmitted by the sensor module in real time, and analyze and process these data. The control module is connected to the voice data acquisition module through a wired interface technology, such as a universal asynchronous receiver-transmitter (UART) or a serial peripheral interface (SPI), so that it can receive voice data. At the same time, the control module also establishes a connection with the human-computer interaction interface through a wired interface technology, such as a serial peripheral interface (SPI) or an I2C (Inter-Integrated Circuit) bus, so that it can send display information to the user, thereby providing an intuitive user interface experience. In order to enhance the interactivity and prompt function of the system, the control module is also connected to the voice prompt module through a wired interface technology, such as a universal asynchronous receiver / transmitter (UART) or an I2C bus, so that voice prompt information can be sent to the user. These voice prompt information can be used to guide user operations or provide system status feedback. Finally, the control module is connected to the early warning module through a wired interface technology, such as a UART or an I2C bus, so that early warning information can be sent to the user in a timely manner, especially when a potential safety risk is encountered during the operation of the two-wheeled vehicle, the early warning module can respond quickly and notify the user to take corresponding safety measures.

[0101] In one achievable manner, the human-computer interaction system further includes a positioning module and an anomaly detection module.

[0102] The positioning module is used to obtain the position information of the two-wheeled vehicle in real time and transmit the obtained position information to the control module. The positioning module includes at least one positioning system among GPS, Beidou, Galileo and GLONASS.

[0103] Specifically, the main function of the positioning module is to obtain the current specific location information of the two-wheeled vehicle in real time. The positioning module is equipped with a variety of positioning systems, including but not limited to the Global Positioning System (GPS), Beidou Satellite Navigation System, Galileo System and GLONASS System. These systems work together to ensure that accurate location information can be obtained no matter where the vehicle is. The acquired location information is transmitted to the control module, and then sent to the terminal device, such as a mobile phone, computer, watch, etc., through the control module, so as to realize real-time monitoring and management of the two-wheeled vehicle. The positioning module is connected to the control module through a wired interface (such as UART, SPI, I2C) or a wireless interface (such as Bluetooth, Wi-Fi), and transmits the location information to the control module.

[0104] The abnormality detection module is used to generate vehicle abnormality information when the two-wheeled vehicle moves abnormally in a parked or closed state, and send the vehicle abnormality information to the early warning module, or send the vehicle abnormality information to the terminal device through the wireless communication module. At the same time, the terminal device can control the opening of the camera to record the environmental image in real time.

[0105] Specifically, the abnormal detection module is used to ensure that any abnormal movement of the two-wheeled vehicle can be detected and handled in time when the two-wheeled vehicle is parked or the engine is turned off. When the vehicle is in a stationary state, the module will continue to monitor the movement state of the vehicle. Once any abnormal movement is detected, for example, the vehicle is illegally moved or hit, the abnormal detection module will be immediately activated and generate vehicle abnormal information, recording the time, location and specific description of the abnormality. The generated vehicle abnormal information is sent to the early warning module. The early warning module will analyze the received information and determine whether further early warning measures need to be initiated, such as sounding an alarm, flashing car lights or reminding surrounding people in other ways. At the same time, the abnormal detection module will also transmit the vehicle abnormal information to the terminal device, such as the owner's smartphone or other monitoring device, through the wireless communication module. In this way, even if the owner is not on site, he can receive notifications of vehicle abnormalities in real time, so as to quickly take corresponding measures, such as contacting security personnel or calling the police. The above method ensures that when the vehicle moves abnormally when it is parked or turned off, the relevant personnel can be notified in a timely and effective manner, thereby minimizing possible losses or damages.

[0106] For example, users can remotely control the camera on the vehicle through the terminal device to turn it on and record the surrounding environment in real time. In this way, users can not only monitor the environment around the vehicle in real time, but also save relevant video evidence when necessary for subsequent investigation and processing.

[0107] In one practicable manner, the wireless communication module includes a WIFI module and a mobile communication module, the mobile communication module includes a 4G network module and / or a 5G network module, and the communication module is connected to the control module to adapt to different communication environments.

[0108] In an exemplary embodiment, Figure 2 As shown, a human-computer interaction method based on a two-wheeled vehicle is also provided, comprising:

[0109] Step S101: Acquire a voice signal from the surrounding environment based on a microphone array module;

[0110] Wherein, the microphone array module includes a microphone array, the microphone array includes a plurality of microphone units, the plurality of microphone units are evenly distributed in the surrounding area of ​​the human-computer interaction interface, and the microphone array forms a multi-MIC matrix;

[0111] Processing the voice signal transmitted by the microphone array module.

[0112] Wherein, processing the voice signal transmitted by the microphone array module includes:

[0113] Step S102: pre-process the voice signal transmitted by the microphone array module.

[0114] Step S103: estimating time difference information of the speech signals received by different microphone units.

[0115] Step S104: Obtain the sound source direction according to the time difference information and the layout of the microphone array.

[0116] Step S105: In different usage scenarios, the weight is adjusted according to the direction of the sound source to form a beam in a specified range, thereby enhancing the voice signal in the target direction.

[0117] Step S106, converting the processed voice signal into a control signal, controlling at least one functional module to perform a specified operation through the control signal, thereby realizing voice control of the two-wheeled vehicle function, and displaying the operation result on the human-computer interaction interface.

[0118] As an optional implementation, step S105 includes:

[0119] When in a riding state, the weight is adjusted according to the direction of the sound source to form a beam within a first preset distance within a range of 30° directly behind the human-computer interaction interface;

[0120] When the vehicle is in a parking state, the weight is adjusted according to the direction of the sound source, and a beam within a second preset distance is formed within a 360° angle range with the human-computer interaction interface as the center.

[0121] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A human-computer interaction system based on a two-wheeled vehicle, characterized in that: The human-computer interaction system based on a two-wheeled vehicle comprises: Microphone array module, used to obtain voice signals from the surrounding environment; Wherein, the microphone array module includes a microphone array, the microphone array includes a plurality of microphone units, the plurality of microphone units are evenly distributed in the surrounding area of ​​the human-computer interaction interface, and the microphone array forms a multi-MIC matrix; A voice data processing module, used for processing the voice signal transmitted by the microphone array module; Wherein, the voice data processing module includes: A preprocessing unit, used for preprocessing the speech signal; A time difference estimation unit, used for estimating time difference information of the speech signals received by different microphone units; A direction calculation unit, used to obtain the direction of the sound source according to the time difference information and the layout of the microphone array; A beam forming unit, used to adjust the weight according to the direction of the sound source in different usage scenarios to form a beam in a specified range, thereby enhancing the speech signal in the target direction; The control module is used to convert the pre-processed voice signal into a control signal, and control at least one functional module to perform a specified operation through the control signal, thereby realizing voice control of the two-wheeled vehicle function, and displaying the operation result on the human-computer interaction interface.

2. The human-computer interaction system based on a two-wheeled vehicle according to claim 1, characterized in that: The method of adjusting the weight according to the direction of the sound source in different usage scenarios to form a beam within a specified range includes: When in a riding state, the weight is adjusted according to the direction of the sound source to form a beam within a first preset distance within a range of 30° directly behind the human-computer interaction interface; When the vehicle is in a parking state, the weight is adjusted according to the direction of the sound source, and a beam within a second preset distance is formed within a 360° angle range with the human-computer interaction interface as the center.

3. The human-computer interaction system based on a two-wheeled vehicle according to claim 2, characterized in that: The human-computer interaction system based on the two-wheeled vehicle also includes: A gyroscope is used to obtain a status signal of the two-wheeled vehicle, wherein the status signal is used to determine whether the two-wheeled vehicle is in the riding state or the parking state. The gyroscope is connected to a sensor module, and the sensor module transmits the obtained status signal to a beam forming unit through the control module.

4. The human-computer interaction system based on a two-wheeled vehicle according to claim 1, characterized in that: The human-computer interaction system based on a two-wheeled vehicle further includes: An image processing module, used for processing real-time image data of the environment in front and behind the two-wheeled vehicle, and sending the processed real-time image data to the control module; A sensor module, used for monitoring a status signal of the two-wheeled vehicle; The early warning module is used to generate early warning information according to the front and rear environment of the two-wheeled vehicle and the status signal, and send the early warning information to the human-computer interaction interface for display, or the early warning module sends the early warning information to the voice prompt module for alarm prompt.

5. The human-computer interaction system based on a two-wheeled vehicle according to claim 4, characterized in that: The sensor module comprises: A tire pressure sensor, used to obtain tire pressure information of a moving vehicle, wherein the tire pressure information includes a tire position identifier; Fuel level sensor, used to obtain information about the amount of fuel in the fuel tank; A power sensor, used to monitor the remaining power of the battery and display it to the user in real time through the human-computer interaction interface; A rain sensor, used to monitor the rainfall in the external environment of the two-wheeled vehicle, so as to prompt the user to take anti-skid measures; Radar sensor, used to detect obstacle information around the vehicle; A temperature sensor, used to monitor temperature change information of the external environment of the two-wheeled vehicle; An ambient light sensor, used to adjust the brightness of the human-computer interaction interface according to the ambient brightness to meet visual requirements under different lighting conditions; The vehicle speed sensor is used to measure the speed of the two-wheeled vehicle in real time.

6. The human-computer interaction system based on a two-wheeled vehicle according to claim 4, characterized in that: The human-computer interaction system based on a two-wheeled vehicle further includes: A camera module connected to the image processing module via a video interface, used to obtain the real-time image data of the front and rear environment of the two-wheeled vehicle and send it to the image processing module; The control module is further used to generate risk level information according to the processed real-time image data, and send the risk level information to the early warning module.

7. The human-computer interaction system based on a two-wheeled vehicle according to claim 4, characterized in that: The human-computer interaction system further includes: A positioning module, used for acquiring the position information of the two-wheeled vehicle in real time and transmitting the acquired position information to the control module, wherein the positioning module includes at least one positioning system selected from GPS, Beidou, Galileo and GLONASS; The abnormality detection module is used to generate vehicle abnormality information when the two-wheeled vehicle moves abnormally in a parked or closed state, and send the vehicle abnormality information to the early warning module, or send the vehicle abnormality information to the terminal device through the wireless communication module.

8. A human-computer interaction method based on a two-wheeled vehicle, characterized in that: The human-computer interaction method based on a two-wheeled vehicle comprises: Acquire the voice signal of the surrounding environment based on the microphone array module; wherein the microphone array module includes a microphone array, the microphone array includes a plurality of microphone units, the plurality of microphone units are evenly distributed in the surrounding area of ​​the human-computer interaction interface, and the microphone array forms a multi-MIC matrix; Processing the voice signal transmitted by the microphone array module; The voice signal transmitted by the microphone array module is processed, including: Preprocessing the voice signal transmitted by the microphone array module; estimating time difference information of the speech signals received by different microphone units; Obtaining a sound source direction according to the time difference information and the layout of the microphone array; In different usage scenarios, the weight is adjusted according to the direction of the sound source to form a beam in a specified range, thereby enhancing the speech signal in the target direction; The processed voice signal is converted into a control signal, and at least one functional module is controlled by the control signal to perform a specified operation, thereby realizing voice control of the two-wheeled vehicle function, and the operation result is displayed on the human-computer interaction interface.

9. The human-computer interaction method based on a two-wheeled vehicle according to claim 8, characterized in that: The method of adjusting the weight according to the sound source direction in different usage scenarios to form a beam in a specified range, thereby enhancing the speech signal in the target direction, includes: When in a riding state, the weight is adjusted according to the direction of the sound source to form a beam within a first preset distance within a range of 30° directly behind the human-computer interaction interface; When the vehicle is in a parking state, the weight is adjusted according to the direction of the sound source, and a beam within a second preset distance is formed within a 360° angle range with the human-computer interaction interface as the center.

Images