Fatigue prompting method and device, vehicle, medium and program product

Abstract

The invention relates to a fatigue prompting method and device, a vehicle, a medium and a program product in the technical field of intelligent cabins, and the method comprises the steps that a fatigue detection result is determined according to a sound signal of a target object, and the sound signal is obtained through a sound collection device located in the vehicle; and under the condition that the fatigue detection result represents that the target object has fatigue driving, executing fatigue prompt. Thus, fatigue detection is carried out without depending on images acquired by a camera of a driver monitoring system, the fatigue detection result is determined through the sound signal, the privacy of the driver can be protected, the situation that the detection accuracy and reliability are reduced due to wearing of sunglasses, shielding objects and the like is avoided, and meanwhile the fatigue state of the driver is monitored in real time; and the accuracy and reliability of driver fatigue detection are improved, so that the driving safety is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of smart cockpit technology, and in particular to a fatigue prompt method, device, vehicle, medium, and program product. Background Art

[0002] In scenarios requiring high concentration, such as driving, a driver in a fatigued state can pose uncontrollable safety risks. Related technologies can detect driver fatigue based on images captured by cameras, but this cannot be performed if the camera is damaged or not configured. Summary of the Invention

[0003] To overcome the problems existing in the related art, the present disclosure provides a fatigue prompt method, device, vehicle, medium and program product.

[0004] According to a first aspect of an embodiment of the present disclosure, a fatigue prompt method is provided, comprising: A fatigue detection result is determined according to a sound signal of the target object; and a fatigue prompt is executed if the fatigue detection result indicates that the target object is driving fatigued.

[0005] The above technical solution does not rely on the camera of the driver monitoring system to capture images for fatigue detection, but determines the fatigue detection results through sound signals. It can protect the driver's privacy and avoid wearing sunglasses, obstructions, etc., which may reduce the detection accuracy and reliability. At the same time, it can monitor the driver's fatigue status in real time, improve the accuracy and reliability of driver fatigue detection, and thus enhance driving safety.

[0006] In some possible implementations, the sound signal includes a multi-dimensional sound signal, and the method of fusing the multi-dimensional sound signals of the target object to determine the fatigue detection result includes: determining the fatigue monitoring information corresponding to each dimension based on the sound signal of each dimension of the target object; and fusing the fatigue monitoring information corresponding to each dimension to determine the fatigue detection result.

[0007] By analyzing the multi-dimensional sound signals of the target object, the above technical solution can accurately determine the fatigue monitoring information corresponding to each dimension. The fatigue monitoring information corresponding to each dimension is integrated, utilizing the complementarity between sound signals of different dimensions to avoid the influence of single-dimensional sound signals on factors such as environmental noise and individual differences. Multi-dimensional fusion can effectively reduce these interferences and significantly improve the accuracy and reliability of fatigue detection results. Because it does not rely on a fixed image acquisition environment, it has stronger adaptability in dynamically changing driving scenarios and can maintain stable detection performance whether it is daytime, nighttime, or in different weather conditions.

[0008] In some possible implementations, determining the fatigue monitoring information corresponding to each dimension based on the sound signal of each dimension of the target object includes: determining the fatigue characteristic value corresponding to the dimension based on the multi-dimensional sound signal of the target object; determining the fatigue monitoring information corresponding to each dimension based on the fatigue characteristic value corresponding to each dimension and the fatigue trigger threshold corresponding to the dimension.

[0009] The above-mentioned technical solution can accurately extract and determine the fatigue characteristic value corresponding to each dimension by analyzing the multi-dimensional sound signals of the target object, thereby directly reflecting the driver's fatigue performance in different sound dimensions. The corresponding fatigue trigger threshold is set for each dimension, and the influence of individual differences, characteristics of different sound dimensions, and actual driving environment on fatigue judgment is fully considered. It enables more accurate identification of fatigue status according to the actual situation of the driver and the characteristics of different sound dimensions, thereby improving the sensitivity and specificity of fatigue monitoring.

[0010] In some possible implementations, the multidimensional sound signal includes a breathing sound signal, and determining the fatigue characteristic value corresponding to the dimension based on the multidimensional sound signal of the target object includes: determining the breathing frequency of the target object based on the breathing sound signal in the multidimensional sound signal; and determining the fatigue characteristic value corresponding to the breathing sound signal based on the breathing frequency of the target object.

[0011] The above technical solution determines the respiratory rate through the respiratory sound signal, which can accurately capture the changes in the driver's breathing rhythm. Compared with subjective judgment or other ambiguous monitoring methods, this quantification method provides a more objective and accurate data basis for fatigue detection. Establishing a correlation between the respiratory frequency and the fatigue characteristic value can intuitively reflect the intrinsic connection between the change in respiratory frequency and the fatigue state. It can more clearly identify the degree of fatigue corresponding to different respiratory frequencies, providing a more intuitive and quantifiable expression for fatigue detection results, thereby improving the adaptability and accuracy of fatigue detection for different drivers.

[0012] In some possible implementations, the multidimensional sound signal includes a yawn sound signal, and determining the fatigue characteristic value corresponding to the dimension based on the multidimensional sound signal of the target object includes: determining the yawn feature information of the target object based on the yawn sound signal in the multidimensional sound signal; and determining the fatigue characteristic value corresponding to the yawn sound signal based on the yawn feature information of the target object.

[0013] The yawning sound of the above technical solution has a unique waveform and rhythm, and can still be accurately extracted in the complex driving environment noise due to its own characteristics, reducing the impact of external interference on fatigue detection and improving stability.

[0014] In some possible implementations, the yawn feature information includes at least one of the following: the number of yawns, the frequency of yawns, the volume of yawns, and the duration of yawns.

[0015] The above technical solution quantifies yawn characteristic information by yawning number, yawn frequency, yawn volume, and yawn duration, so as to clearly judge the degree of fatigue rather than simply qualitatively, thereby improving the accuracy of determining the fatigue characteristic value.

[0016] In some possible implementations, the fatigue trigger threshold is obtained through self-learning in the following manner: determining the key parameters of the fatigue feature values ​​of each dimension based on the fatigue feature values ​​corresponding to the historical sound signals of each person of the target object; determining the fatigue feature label of the target object based on the key parameters of the fatigue feature values ​​of each dimension; determining the fatigue trigger threshold of each dimension based on the baseline fatigue threshold and the fatigue feature label corresponding to each dimension.

[0017] The above technical solution can automatically adjust the fatigue threshold based on the long-term usage data of the target object to ensure the accuracy and adaptability of fatigue detection.

[0018] In some possible implementations, the fatigue monitoring information corresponding to each dimension is integrated to determine the fatigue detection result, including: determining cumulative fatigue information based on the fatigue monitoring information corresponding to each dimension; and determining the fatigue detection result based on the cumulative fatigue information.

[0019] The above technical solution effectively integrates fatigue information from various dimensions by determining the comprehensive indicator of cumulative fatigue information, fully utilizes the information provided by different dimensions, compensates for the limitations that may exist in a single dimension, more comprehensively reflects the fatigue status of the object being tested, and effectively improves the accuracy and reliability of fatigue detection.

[0020] In some possible implementations, the fatigue detection result is determined by fusing the multi-dimensional sound signals of the target object, including: determining the target sound signal corresponding to the dimension based on the weight vector and the sound signal of each dimension of the target object; and determining the fatigue detection result by fusing the target sound signal corresponding to each dimension.

[0021] This technical solution utilizes weight vectors to optimize the sound signals in each dimension to obtain a target sound signal, ensuring the quality and relevance of the input signal and more accurately extracting key fatigue-related information. The target sound signals from each dimension are then combined to determine the fatigue detection result. This leverages the advantages of multidimensional information, integrating fatigue characteristics reflected by different dimensions, reducing the uncertainty of single-dimensional detection and improving the accuracy of fatigue detection results to the actual fatigue state.

[0022] In some possible implementations, determining the target sound signal corresponding to the dimension based on the weight vector and the sound signal of each dimension of the target object includes: determining the error signal corresponding to each dimension based on the weight vector and the sound signal of each dimension of the target object; determining a new weight vector based on the weight vector, the sound signal and the error signal; re-executing the steps of determining the error signal corresponding to each dimension to determining a new weight vector based on the sound signal and the new weight vector until the error signal meets a preset condition; and determining the error signal corresponding to each dimension that meets the preset condition as the target sound signal of that dimension.

[0023] During the iterative process of determining the error signal and updating the weight vector, this technical solution continuously learns and adapts to the relationship between different dimensional sound signals and fatigue characteristics, automatically adjusting the weight of each dimensional sound signal in fatigue detection. This effectively reduces the impact of factors such as noise interference and individual differences on the detection results. The resulting target sound signal can more accurately reflect the target subject's fatigue state.

[0024] In some possible implementations, determining a new weight vector based on the weight vector, the sound signal, and the error signal includes: determining an autocorrelation matrix based on the sound signal within a time range of a preset length; determining a step factor based on the autocorrelation matrix; and determining a new weight vector based on the step factor, the weight vector, the sound signal, and the error signal.

[0025] This technical solution calculates the autocorrelation matrix based on the sound signal within a preset time range, fully exploiting the characteristic information of the sound signal and improving the accuracy of the step-size factor determination. The step-size factor is then dynamically adjusted based on the autocorrelation matrix, making the weight vector update process more flexible and efficient, and able to quickly adapt to changes in different sound signals. Finally, a new weight vector is determined by comprehensively considering the step-size factor, weight vector, sound signal, and error signal. This achieves precise updating of the weight vector, more accurately extracting fatigue-related features from the sound signal, reducing the influence of factors such as noise interference and individual differences, and significantly improving the accuracy and reliability of fatigue detection results.

[0026] In some possible implementations, determining the error signal corresponding to each dimension based on the weight vector and the sound signal of each dimension of the target object includes: determining the adjusted sound signal corresponding to the dimension based on the sound signal of each dimension and the weight vector; and determining the error signal corresponding to the dimension based on the adjusted sound signal and the sound signal corresponding to the dimension.

[0027] This technical solution uses weight vectors to generate and adjust sound signals, achieving preliminary optimization of each dimension's sound signal, bringing it closer to fatigue-related characteristics and helping to improve the specificity of fatigue detection. Then, by comparing the adjusted sound signal with the original sound signal to generate an error signal, it is possible to quantitatively evaluate the processing effect of the current weight vector and clearly identify the difference between the performance of each dimension's sound signal in fatigue detection and the expected performance. Based on these error signals, the weight vector can be further adjusted to continuously optimize the sound signal processing process, ultimately improving the accuracy and reliability of fatigue detection results.

[0028] In some possible implementations, the sound collection device includes at least one of the following: a sound collection device configured in the vehicle itself, or a sound collection device of a terminal communicating with the vehicle.

[0029] The above technical solution obtains sound signals through the sound collection device configured by the vehicle itself, realizes the reuse of the sound collection device on the vehicle, and reduces manufacturing costs; obtains sound signals through the sound collection device of the terminal communicating with the vehicle, thereby improving the flexibility of sound signal collection.

[0030] In some possible implementations, the sound signal includes at least one of the following: a breathing sound signal, a yawning sound signal, and a coughing sound signal.

[0031] The above technical solution introduces breathing sound signals and yawning sound signals as the basis for fatigue detection, fully utilizing the characteristics and advantages of different sound signals, and significantly improving the accuracy, sensitivity and reliability of fatigue detection.

[0032] In some possible implementations, the fatigue prompt includes at least one of the following: an auditory prompt, a visual prompt, and a vibration prompt.

[0033] The above technical solution significantly enhances the effectiveness and adaptability of fatigue notifications by providing at least one of auditory, visual, and vibration cues. Auditory cues can attract users' attention in a variety of environments, visual cues can clearly and intuitively present information, and vibration cues can provide warnings despite auditory and visual interference.

[0034] In some possible implementations, the fusing and determining of fatigue detection results based on multi-dimensional sound signals of the target object includes: fusing and determining an initial detection result based on multi-dimensional sound signals of the target object; and fusing and determining the fatigue detection result based on the initial detection result and target information.

[0035] This technical solution determines the initial detection result by fusing multidimensional sound signals, fully utilizing the multidimensional information in the sound signals to accurately capture the target subject's fatigue state. This initial detection result is then combined with the target information to determine the final fatigue detection result. This optimization and correction of the initial result takes into account individual differences in the target subject and the impact of external environmental factors on fatigue status. This makes the fatigue detection result more consistent with actual conditions and effectively reduces the false positive rate.

[0036] In some possible implementations, the target information includes at least one of the following: driving time information, vehicle speed information, and steering wheel operation information.

[0037] This technical solution integrates at least one of driving duration, vehicle speed, and steering wheel operation information as target information, integrating it with initial detection results determined based on multi-dimensional sound signals. This significantly enhances the accuracy and reliability of fatigue detection. Driving duration information provides a temporal basis for fatigue assessment, while vehicle speed and steering wheel operation information provide key information for determining fatigue status from the perspective of dynamic driving behavior. By comprehensively considering these target information, a more comprehensive and accurate assessment of the driver's fatigue level can be achieved, enabling the timely identification of potential fatigue driving risks and effectively preventing traffic accidents caused by fatigue driving.

[0038] According to a second aspect of an embodiment of the present disclosure, a fatigue prompting device is provided, comprising: a fusion determination module configured to fuse and determine a fatigue detection result based on a sound signal of a target object, wherein the sound signal is obtained by a sound collection device located in a vehicle; and a prompting module configured to perform a fatigue prompting if the fatigue detection result indicates that the target object is fatigue driving.

[0039] In some possible implementations, the sound signal includes a multi-dimensional sound signal, and the result determination unit includes: a monitoring value determination submodule, configured to determine the fatigue monitoring information corresponding to each dimension based on the sound signal of each dimension of the target object; and a fusion determination submodule, configured to fuse the fatigue monitoring information corresponding to each dimension to determine the fatigue detection result.

[0040] In some possible implementations, the monitoring value determination submodule includes: a characteristic value determination unit, configured to determine the fatigue characteristic value corresponding to the dimension based on the multi-dimensional sound signal of the target object; a monitoring value determination unit, configured to determine the fatigue monitoring information corresponding to each dimension based on the fatigue characteristic value corresponding to each dimension and the fatigue trigger threshold corresponding to the dimension.

[0041] In some possible implementations, the multi-dimensional sound signal includes at least one of the following: a breathing sound signal, a yawning sound signal, and a coughing sound signal.

[0042] In some possible implementations, the multidimensional sound signal includes a breathing sound signal, and the characteristic value determination unit is configured to: determine the breathing frequency of the target object based on the breathing sound signal in the multidimensional sound signal; and determine the fatigue characteristic value corresponding to the breathing sound signal based on the breathing frequency of the target object.

[0043] In some possible implementations, the multidimensional sound signal includes a yawn sound signal, and the characteristic value determination unit is configured to: determine the yawn characteristic information of the target object based on the yawn sound signal in the multidimensional sound signal; and determine the fatigue characteristic value corresponding to the yawn sound signal based on the yawn characteristic information of the target object.

[0044] In some possible implementations, the yawn feature information includes at least one of the following: the number of yawns, the frequency of yawns, the volume of yawns, and the duration of yawns.

[0045] In some possible implementations, the fusion determination submodule includes: a fusion determination unit, configured to determine the cumulative fatigue information based on the fatigue monitoring information corresponding to each dimension; and a result determination unit, configured to determine the fatigue detection result based on the cumulative fatigue information.

[0046] In some possible implementations, the result determination unit includes: a signal fusion determination submodule, configured to determine the target sound signal corresponding to the dimension based on the weight vector and the sound signal of each dimension of the target object; a fatigue result determination submodule, configured to fuse and determine the fatigue detection result based on the target sound signal corresponding to each dimension.

[0047] In some possible implementations, the sound collection device includes at least one of the following: a sound collection device configured in the vehicle itself, or a sound collection device of a terminal communicating with the vehicle.

[0048] In some possible implementations, the sound signal includes at least one of the following: a breathing sound signal, a yawning sound signal, and a coughing sound signal.

[0049] In some possible implementations, the fatigue prompt includes at least one of the following: an auditory prompt, a visual prompt, and a vibration prompt.

[0050] In some possible implementations, the determination module includes: an initial result determination submodule, configured to determine the initial detection result based on the sound signal of the target object; and a fusion result determination submodule, configured to fuse and determine the fatigue detection result based on the initial detection result and target information.

[0051] In some possible implementations, the target information includes at least one of the following: driving time information, vehicle speed information, steering wheel operation information, breathing frequency in a sleeping state, and heart rate in a sleeping state.

[0052] According to a third aspect of an embodiment of the present disclosure, there is provided a vehicle, comprising: A processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions stored in the memory to implement any one of the methods of the first aspect.

[0053] According to a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, which implements the steps of any one of the methods described in the first aspect when executed by a processor.

[0054] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which implements the steps of any one of the methods in the first aspect when executed by a processor.

[0055] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0057] Figure 1 The figure is a flowchart of a fatigue prompt method according to an exemplary embodiment.

[0058] Figure 2 An implementation according to an exemplary embodiment is shown Figure 1 Flowchart of step S11 in FIG.

[0059] Figure 3 An implementation according to an exemplary embodiment is shown Figure 2 Flowchart of step S111 in FIG.

[0060] Figure 4 Another implementation according to an exemplary embodiment is shown Figure 1 Flowchart of step S11 in FIG.

[0061] Figure 5 Another implementation according to an exemplary embodiment is shown Figure 1 Flowchart of step S11 in FIG.

[0062] Figure 6 The figure is a block diagram of a fatigue prompting device according to an exemplary embodiment.

[0063] Figure 7 is a block diagram of a vehicle according to an exemplary embodiment. DETAILED DESCRIPTION

[0064] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0065] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the corresponding data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0066] Before introducing the fatigue detection method provided by this disclosure, let's first describe the technologies involved in the relevant scenario. A driver monitoring system (DMS) captures a driver's facial image and then uses facial recognition technology to analyze the facial image to determine the driver's eye movements, blinking frequency, yawning, and other behavioral information. Based on this information, the driver's fatigue is determined. However, the DMS relies on facial detection and tracking technology to detect eye movements, blinking frequency, head posture, yawning, and mouth movements. It also requires attention distraction detection, deep learning, and behavior prediction. The configuration of image processing units and models is not only costly, but many drivers block the camera for privacy reasons, causing the DMS to fail. Furthermore, the camera does not function effectively in low light or when the driver is wearing sunglasses, which can also cause the DMS to fail. This results in low accuracy and stability in fatigue detection.

[0067] In view of this, the present disclosure provides a fatigue prompt method, which aims to reduce the cost of fatigue detection while improving the accuracy and stability of fatigue detection.

[0068] Figure 1 This is a flowchart of a fatigue prompt method according to an exemplary embodiment. The fatigue prompt method in the embodiment of the present disclosure can be applied to a vehicle's cockpit domain controller or an autonomous driving domain controller, such as Figure 1 The method shown includes the following steps.

[0069] In step S11 , a fatigue detection result is determined based on the sound signal of the target object.

[0070] Among them, sound signals are information of various types or dimensions presented by the target object (such as the vehicle driver) in terms of sound. Sound signals of different dimensions can reflect the driver's physiological and psychological state from different aspects.

[0071] In the disclosed embodiment, an onboard microphone continuously monitors the vehicle's interior environment, capturing acoustic signals such as the driver's breathing rate, yawning, and coughing. The microphone collects these acoustic signals in real time and pre-processes them using an embedded software system. For example, noise reduction techniques are used to remove background noise (such as wind noise, road noise, and vehicle engine noise), ensuring that the extracted acoustic signals are clear enough to reflect the driver's actual state.

[0072] In one embodiment, in response to vehicle startup, the system initializes, activating the onboard microphone and establishing a connection with the onboard audio processing unit via a controller to prepare for audio capture. Furthermore, the microphone begins continuously collecting the driver's voice signal. The voice signal enters the audio processing module, where adaptive filtering technology is used to remove background noise and extract low-frequency signals associated with breathing, yawning, coughing, etc. The processed voice signal is then temporarily stored in a buffer, awaiting further analysis and processing.

[0073] The collected sound signals are then preprocessed, including noise reduction, filtering, and feature extraction. Noise reduction removes environmental noise interference and improves the quality of the sound signal. Filtering extracts effective frequency components based on the frequency characteristics of sound signals of different dimensions. Feature extraction extracts key fatigue-related features from the sound signal, such as the frequency of voice intonation changes and the duration of breathing sounds.

[0074] Furthermore, the frequency features of speech are extracted. When people are tired, the frequency of their speech may decrease; the intonation features are extracted. When people are tired, the intonation may become flat and lack fluctuations; the speaking speed features are extracted. When people are tired, the speaking speed may slow down; the volume change features are extracted. When people are tired, the volume may become unstable; at the same time, the presence of specific sounds such as coughing and yawning is detected.

[0075] The extracted sound features are then processed. Machine learning algorithms (such as support vector machines and neural networks) can be used to analyze and learn these features to establish a fatigue detection model. By inputting the real-time collected sound features into this model, the model determines whether the target driver is experiencing fatigue driving based on pre-learned patterns, thereby generating fatigue detection results.

[0076] In step S12, when the fatigue detection result indicates that the target object is driving in a fatigue state, a fatigue prompt is performed.

[0077] In the disclosed embodiment, after the fatigue detection result is obtained, the result is judged. If the detection result is "fatigue driving present", the fatigue prompt mechanism is triggered. Based on the preset prompt method, an appropriate prompt method is selected to remind the target person.

[0078] In the disclosed embodiments, fatigue reminders can be provided through audio alerts (e.g., playing a specific alarm sound), vibration reminders (e.g., seat vibration), and displayed prompts (e.g., displaying a text message such as "You are tired, please take a rest" on the vehicle's instrument panel or central control screen). The selected reminder method is implemented through corresponding hardware devices (e.g., speakers, vibration motors, display screens, etc.) to ensure that the target person receives the reminder information in a timely manner.

[0079] For example, if the fatigue detection result is "fatigue driving present," the fatigue warning mechanism is immediately triggered. Assume that the preset warning method is a combination of an audible alarm and seat vibration. The vehicle's internal speakers are controlled to play a sharp warning sound, while the seat's vibration motor is activated, causing the seat to vibrate noticeably. After hearing the alarm and feeling the seat vibration, the driver can quickly realize that they are driving fatigued and take appropriate measures, such as stopping to rest, to avoid danger.

[0080] This technical solution does not rely on the driver monitoring system's camera to capture images for fatigue detection. Instead, it determines fatigue detection results through sound signals. This allows real-time monitoring of the driver's fatigue state while protecting the driver's privacy and avoiding detection accuracy and reliability reduced by sunglasses, obstructions, and other factors. This improves the accuracy and reliability of driver fatigue detection, thereby enhancing driving safety. Furthermore, compared to existing fatigue detection methods that capture images through cameras, sound signals do not involve the driver's image, thus protecting the driver's privacy. Furthermore, since vehicles are already equipped with microphones, such as those used for answering phone calls on trams, collecting sound signals does not require additional hardware, reducing vehicle production costs.

[0081] In some possible implementations, the sound signal includes a multi-dimensional sound signal, see Figure 2 As shown, in step S11, the fatigue detection result is determined by fusing the multi-dimensional sound signal of the target object, including: In step S111 , fatigue monitoring information corresponding to each dimension is determined based on the sound signal of each dimension of the target object.

[0082] Fatigue monitoring information is a quantitative indicator derived from analyzing each dimension (such as speech frequency, intonation, and speaking rate) of the target subject's (driver's) multidimensional sound signal. It measures the degree of fatigue exhibited by the target subject within that dimension. Fatigue monitoring information calculation methods may vary for different dimensions, but all are based on the correlation between the sound signal and fatigue characteristics in that dimension.

[0083] In an embodiment of the present disclosure, the multi-dimensional sound signal of the target object is divided into different dimensions, such as signals of multiple dimensions such as coughing, yawning, and breathing in the sound signal. For each dimension, corresponding signal processing and analysis methods are used to extract features. For example, for a breathing sound signal, the sound signal can be converted from the time domain to the frequency domain through Fourier transform to analyze its frequency distribution; for a yawning sound signal, the fluctuation of the yawning tone can be measured by calculating the fundamental frequency change rate of the yawning sound signal; for yawning and breathing sound signals, the rate can be calculated by counting the number of specific frequency sounds emitted per unit time; for a yawning sound signal, the standard deviation of the volume of the sound signal within a certain period of time can also be calculated to measure the stability of the yawning volume.

[0084] Furthermore, based on the features extracted from each dimension and combined with a pre-set fatigue feature threshold or model, fatigue monitoring information corresponding to that dimension is calculated. For example, for respiratory rate, if the current respiratory rate is detected to be lower than the average frequency under normal conditions by a certain percentage (such as 10%), the fatigue monitoring information will be increased accordingly. For yawning pitch, if the fundamental frequency change rate is lower than the normal range, the fatigue monitoring information will also be increased.

[0085] In step S112, the fatigue monitoring information corresponding to each dimension is fused to determine the fatigue detection result.

[0086] In the disclosed embodiment, a variety of methods can be used to fuse the fatigue monitoring information of each dimension, such as weighted average method, fuzzy comprehensive evaluation method, neural network fusion method, etc. The weighted average method can assign different weights to each dimension according to its importance to fatigue detection, and then multiply the fatigue monitoring information of each dimension by the corresponding weight and sum them to obtain a comprehensive fatigue detection result; the fuzzy comprehensive evaluation method can use the theory of fuzzy mathematics to convert the fatigue monitoring information of each dimension into fuzzy membership, and then perform a comprehensive evaluation; the neural network fusion method can train a neural network model to allow the model to automatically learn the complex relationship between the fatigue monitoring information of each dimension, thereby outputting the final fatigue detection result.

[0087] By analyzing the multi-dimensional sound signals of the target object, the above technical solution can accurately determine the fatigue monitoring information corresponding to each dimension. The fatigue monitoring information corresponding to each dimension is integrated, utilizing the complementarity between sound signals of different dimensions to avoid the influence of single-dimensional sound signals on factors such as environmental noise and individual differences. Multi-dimensional fusion can effectively reduce these interferences and significantly improve the accuracy and reliability of fatigue detection results. Because it does not rely on a fixed image acquisition environment, it has stronger adaptability in dynamically changing driving scenarios and can maintain stable detection performance whether it is daytime, nighttime, or in different weather conditions.

[0088] In some possible implementations, see Figure 3 As shown, in step S111, the fatigue monitoring information corresponding to each dimension is determined based on the sound signal of each dimension of the target object, including: In step S1111 , fatigue feature values ​​corresponding to the dimensions are determined based on the multi-dimensional sound signals of the target object.

[0089] Fatigue eigenvalues ​​are specific quantitative indicators extracted from sound signals that characterize the target subject's fatigue level. These eigenvalues ​​are derived through specific analysis and processing of the sound signal, such as the rate of change of the fundamental frequency of the speech signal, the degree of energy attenuation, and the intensity changes of specific frequency components. Different fatigue eigenvalues ​​reflect the correlation between sound and fatigue from different perspectives.

[0090] In the disclosed embodiments, when a person is fatigued, the control of the vocal cord muscles decreases, which may cause changes in the fundamental frequency, such as an increase in the fundamental frequency fluctuation range and a decrease in the average fundamental frequency. By analyzing the changes in the fundamental frequency, features such as the fundamental frequency change rate can be extracted. In addition, the energy characteristics of speech can also reflect the fatigue state. When fatigued, the pronunciation force weakens and the sound energy decreases. Features such as the short-term energy and energy attenuation of the speech signal can be calculated to obtain fatigue feature values ​​corresponding to the dimensions.

[0091] In step S1112, fatigue monitoring information corresponding to each dimension is determined according to the fatigue characteristic value corresponding to each dimension and the fatigue trigger threshold corresponding to the dimension.

[0092] The fatigue trigger threshold is a pre-set standard value used to determine whether the sound signal in a certain dimension has reached a fatigue state. This threshold is determined based on a large amount of experimental data, statistical analysis, and the needs of actual application scenarios. It can be set not only by the user but also based on historical information collected in the vehicle or data detected by mobile devices (such as wearable devices). When the fatigue feature value corresponding to a certain dimension exceeds the fatigue trigger threshold of that dimension, it indicates that the sound of the target object in that dimension exhibits obvious fatigue characteristics and may be in a fatigue state.

[0093] In the disclosed embodiments, a self-learning algorithm can be used to gradually optimize the detection model to adapt to individual differences among drivers. For example, for drivers with low breathing rates, the fatigue trigger threshold can be automatically adjusted based on historical data to reduce false positives. The normal breathing rate range can be a system preset value, a customized value actively recorded by the driver, or a characteristic value that has been self-learned and optimized over a long period of time based on the driver's specific breathing characteristics.

[0094] By analyzing the audio characteristics of different drivers, the fatigue detection model can be gradually adjusted to optimize detection results based on individual differences among different drivers. Based on the driver's long-term usage data, the fatigue trigger threshold can be automatically adjusted to ensure the accuracy and adaptability of fatigue detection. Therefore, the various dimensions of the sound signal and fatigue detection results of each test can be stored in the cloud or on-board storage, which can be used as long-term usage data to adjust the fatigue trigger threshold. This gradually improves detection accuracy and reduces sensitivity to individual differences.

[0095] In the disclosed embodiment, the audio analysis module analyzes the preprocessed sound signals for each dimension. For example, the audio analysis module uses signal processing algorithms (such as Fourier transform and short-time energy analysis) to extract audio parameters related to fatigue characteristics, such as breathing rate, yawn frequency, and volume changes.

[0096] Among them, respiratory rate detection can use fast Fourier transform to convert the time domain signal into a frequency domain signal, extract the respiratory frequency from it, obtain the corresponding fatigue characteristic value, and then compare it with the preset fatigue trigger threshold corresponding to the respiratory frequency. For example, the preset fatigue trigger threshold corresponding to the respiratory frequency can be a preset respiratory frequency range (for example, 12 times / minute to 20 times / minute), and then compare the fatigue characteristic value corresponding to the respiratory frequency with the preset respiratory frequency range to determine whether the respiratory frequency is within the preset respiratory frequency range. Among them, the respiratory rate range can be a system preset value, a personalized value actively recorded by the driver, or a characteristic value that is self-learned and optimized based on the driver's breathing characteristics over a long period of time.

[0097] Yawning detection uses short-time energy analysis to examine the driver's mouth sounds and determine whether yawning, a typical fatigue behavior, is present. Fatigue characteristics such as yawn frequency, volume change, and duration are then extracted. Furthermore, fatigue monitoring information corresponding to yawning behavior is identified based on preset fatigue trigger thresholds for yawn frequency, volume change, and duration.

[0098] The above-mentioned technical solution can accurately extract and determine the fatigue characteristic value corresponding to each dimension by analyzing the multi-dimensional sound signals of the target object, thereby directly reflecting the driver's fatigue performance in different sound dimensions. The corresponding fatigue trigger threshold is set for each dimension, and the influence of individual differences, characteristics of different sound dimensions, and actual driving environment on fatigue judgment is fully considered. It enables more accurate identification of fatigue status according to the actual situation of the driver and the characteristics of different sound dimensions, thereby improving the sensitivity and specificity of fatigue monitoring.

[0099] In some possible implementations, in step S1111, determining fatigue feature values ​​corresponding to the dimensions based on the multi-dimensional sound signals of the target object includes: The respiratory frequency of the target object is determined according to the respiratory sound signal in the multi-dimensional sound signal.

[0100] Among them, the respiratory rate is the number of times the human body completes breathing in a unit of time (usually in minutes), that is, the number of breaths per minute.

[0101] In the disclosed embodiment, segmentation can be achieved by detecting the peak value of the respiratory sound signal, and then the segmented respiratory sound signal can be judged by using a sliding window method combined with a threshold value. A sliding window of appropriate size is set, which slides on the signal to calculate the maximum value of the signal within the window. At the same time, a peak threshold is set. When the maximum value within the window exceeds the threshold, it is considered that a peak is detected. In order to avoid repeated detection and missed detection, the detected peaks need to be post-processed, such as setting the minimum time interval between peaks. After the peak value of the respiratory sound signal is detected, the respiratory frequency is calculated based on the time interval between adjacent peaks.

[0102] A fatigue characteristic value corresponding to the breathing sound signal is determined according to the breathing frequency of the target object.

[0103] In the disclosed embodiments, when fatigued, the body's physiological functions change, affecting the respiratory system and causing changes in respiratory parameters such as respiratory rate. For example, fatigue can reduce the body's metabolic rate, leading to corresponding changes in oxygen demand and carbon dioxide excretion, thus causing changes in respiratory rate. By analyzing the difference in respiratory rate from a normal state and quantifying it as a fatigue characteristic value, the fatigue level of the target subject can be assessed.

[0104] In the disclosed embodiments, the obtained respiratory frequency of the target subject can be compared with a normal respiratory frequency range, the deviation of the respiratory frequency from the median of the normal range can be calculated, and a fatigue characteristic value corresponding to the respiratory sound signal can be determined. For example, the fatigue characteristic value corresponding to the respiratory sound signal can be determined based on the magnitude and direction of the respiratory frequency deviation. The respiratory frequency deviation can be converted into a fatigue characteristic value using a linear or nonlinear mapping method.

[0105] The above technical solution determines the respiratory rate through the respiratory sound signal, which can accurately capture the changes in the driver's breathing rhythm. Compared with subjective judgment or other ambiguous monitoring methods, this quantification method provides a more objective and accurate data basis for fatigue detection. Establishing a correlation between the respiratory frequency and the fatigue characteristic value can intuitively reflect the intrinsic connection between the change in respiratory frequency and the fatigue state. It can more clearly identify the degree of fatigue corresponding to different respiratory frequencies, providing a more intuitive and quantifiable expression for fatigue detection results, thereby improving the adaptability and accuracy of fatigue detection for different drivers.

[0106] In some possible implementations, in step S1111, determining fatigue feature values ​​corresponding to the dimensions based on the multi-dimensional sound signals of the target object includes: The yawn feature information of the target object is determined according to the yawn sound signal in the multi-dimensional sound signal.

[0107] Yawn feature information is a set of parameters extracted from yawn sound signals that can describe yawn characteristics. These parameters cover the characteristics of yawn sound in multiple dimensions such as time, frequency, and energy.

[0108] Yawning sounds have a specific duration, a specific frequency distribution, and corresponding energy intensity variations. By analyzing yawning sound signals, we can extract these yawn feature information that reflects the essential characteristics of yawning.

[0109] In the disclosed embodiment, the short-term energy of the sound signal is first calculated and an appropriate energy threshold is set. When the short-term energy of the sound signal exceeds this threshold, a yawn is considered to be present. Furthermore, to eliminate the interference of brief, high-energy noise, a minimum duration threshold can be set. A yawn is only considered to be present when the energy threshold is exceeded continuously for a certain period of time.

[0110] For example, a large number of yawn sound samples can be collected in advance, their features extracted, and a yawn sound template created. The original sound signal is then matched against the template, and the similarity is calculated to determine whether a yawn sound signal is present. When the similarity exceeds a certain threshold, a yawn sound signal is considered detected.

[0111] In one embodiment, the start time and end time of the yawn sound signal are calculated to obtain the yawn duration. The yawn duration can reflect the degree of fatigue, and the length of the yawn duration can be yawn feature information.

[0112] In another embodiment, the time interval between two adjacent yawn sound signal segments is calculated to obtain the yawn frequency, which can reflect the degree of fatigue.

[0113] In another embodiment, the fundamental frequency of the yawn sound signal is calculated using algorithms such as the autocorrelation function method and the cepstrum method, and the short-term energy of the yawn sound signal is calculated to reflect the intensity changes of the sound signal in different time periods. The short-term energy of the yawn can be used as yawn characteristic information.

[0114] In another embodiment, the energy change of the yawn sound signal during the continuous process is analyzed to calculate the energy decay rate, which can be the yawn feature information.

[0115] A fatigue feature value corresponding to the yawn sound signal is determined according to the yawn feature information of the target object.

[0116] A fatigue feature value is extracted from yawn feature information to quantify the target subject's fatigue level. This value, derived through comprehensive analysis and processing of yawn feature information, is used to assess the target subject's fatigue state based on yawning. For example, higher fatigue levels may indicate more frequent, longer-lasting, and louder yawns. By analyzing the differences between yawn feature information and normal conditions and quantifying them as fatigue feature values, a more accurate assessment of the target subject's fatigue level can be achieved.

[0117] In this disclosed embodiment, the obtained yawn feature information of the target subject is compared with a reference range of normal yawn features, and the deviation of each yawn feature from the median of the reference range is calculated. The magnitude and direction of the deviation reflect the degree to which the yawn feature deviates from normal. Furthermore, based on the deviation of each yawn feature from the median of the reference range, the fatigue feature value corresponding to the yawn sound signal is determined.

[0118] For example, by comprehensively considering the deviations of various yawn features and using weighted summation or other comprehensive evaluation methods, the yawn feature deviations are converted into fatigue feature values ​​corresponding to the yawn sound signal. Different yawn features may have different importance for fatigue assessment, so it is necessary to assign appropriate weights to each feature. The weights of various yawn features can be determined through expert experience or machine learning algorithms (such as principal component analysis and linear discriminant analysis). For example, if yawn duration is more important for fatigue assessment, it can be given a larger weight; if the interval between yawns is less important, it can be given a smaller weight.

[0119] The yawning sound of the above technical solution has a unique waveform and rhythm, and can still be accurately extracted in the complex driving environment noise due to its own characteristics, reducing the impact of external interference on fatigue detection and improving stability.

[0120] In some possible implementations, the yawn feature information includes at least one of the following: the number of yawns, the frequency of yawns, the volume of yawns, and the duration of yawns.

[0121] Counting the number of yawns can reflect a subject's fatigue level. More yawns may indicate a higher level of fatigue. Yawn frequency can be used as a key indicator for assessing a subject's fatigue; higher yawns indicate more severe fatigue. Yawn volume can also reflect a subject's fatigue level; louder yawns indicate higher levels of fatigue. Yawn duration can also be used as a reference indicator for assessing a subject's fatigue level; longer yawns indicate higher levels of fatigue.

[0122] The above technical solution quantifies yawn characteristic information by yawning number, yawn frequency, yawn volume, and yawn duration, so as to clearly judge the degree of fatigue rather than simply qualitatively, thereby improving the accuracy of determining the fatigue characteristic value.

[0123] In some possible implementations, the fatigue trigger threshold is obtained through self-learning in the following manner: Determining key parameters of the fatigue characteristic values ​​of each dimension according to the fatigue characteristic values ​​corresponding to the historical sound signals of each of the target objects; In the disclosed embodiments, key parameters can be a set of parameters within each dimension's fatigue eigenvalues ​​that most effectively and representatively reflect changes in fatigue characteristics within that dimension. Key parameters can be statistical quantities (such as mean, variance, median, etc.) or eigenvalues ​​calculated using a specific algorithm (such as peak frequency, spectral center of gravity, etc.).

[0124] In the disclosed embodiment, the fatigue characteristic values ​​corresponding to the historical sound signals cover the fatigue characteristic values ​​corresponding to the target object's vocalization under different fatigue states, so as to ensure the comprehensiveness and representativeness of the data. For the fatigue characteristic values ​​of each dimension, statistical methods or specific data analysis algorithms are used to calculate key parameters. Taking statistical methods as an example, for a set of speech rate characteristic values, its mean can be calculated as the key parameter of the speech rate of this dimension. The mean can reflect the average level of the target object's speech rate over a period of time, and to a certain extent reflects the impact of its fatigue state on the speech rate. In this way, the key parameters of the fatigue characteristic values ​​of each dimension are determined.

[0125] determining a fatigue feature label of the target object according to the key parameters of the fatigue feature values ​​of each dimension; The fatigue feature label is a summary description or classification identifier of the target object's fatigue state. It integrates key parameter information of fatigue feature values ​​in various dimensions to concisely represent the target object's current fatigue level or state type. For example, different fatigue feature labels can be defined, such as "mild fatigue," "moderate fatigue," and "severe fatigue."

[0126] In the disclosed embodiments, a pattern recognition or classification algorithm may be used to determine the fatigue feature label of the target object. For example, a rule-based classification model is established, and the value ranges of the key parameters of each dimension corresponding to different fatigue feature labels are pre-set. For example, for the "mild fatigue" label, the key parameters (mean) of the speech speed dimension are set within a certain range (such as 120-150 words per minute), and the key parameters of the pitch dimension (pitch variation) are set within a smaller range, etc. The key parameters of each dimension actually calculated are compared with these preset ranges. If the key parameters of all or most dimensions meet the preset range of a fatigue feature label, the label is assigned to the target object.

[0127] The fatigue triggering threshold of each dimension is determined according to the baseline fatigue threshold and the fatigue feature label corresponding to each dimension.

[0128] Among them, the baseline fatigue threshold is a pre-set fatigue level limit value used as a reference standard, which is used to reflect the overall characteristic level of the target object when it reaches a certain fatigue state under normal circumstances.

[0129] In this disclosed embodiment, the relationship between different fatigue feature labels and the baseline fatigue threshold can be analyzed. For example, if the baseline fatigue threshold corresponds to a "moderate fatigue" state, and the fatigue feature label for a particular dimension is "mild fatigue," the key parameters of the fatigue feature value for that dimension will be lower than the baseline. Based on this relationship, and taking into account the variation patterns and characteristics of the fatigue feature value for each dimension, the baseline fatigue threshold for that dimension can be adjusted.

[0130] The above technical solution can automatically adjust the fatigue threshold based on the long-term usage data of the target object to ensure the accuracy and adaptability of fatigue detection.

[0131] In some possible implementations, in step S1112, fusing the fatigue monitoring information corresponding to each dimension to determine the fatigue detection result includes: Determine accumulated fatigue information based on the fatigue monitoring information corresponding to each dimension.

[0132] In an embodiment of the present disclosure, fatigue monitoring information corresponding to multiple dimensions is comprehensively analyzed. For example, for the sound signal corresponding to each dimension, fatigue monitoring information can be determined through preset thresholds and models, and then cumulative fatigue information can be determined based on each fatigue monitoring information, for example, by weighting.

[0133] Fatigue monitoring information from different dimensions may have different dimensions and numerical ranges. For example, the fatigue eigenvalue of a breathing sound signal may be between 0 and 1, while that of a yawning sound signal may be between 0 and 10. Directly fusing data with these different dimensions and ranges may cause some dimensions to be over-amplified or under-amplified during the fusion process, affecting the accuracy of the fusion results. Therefore, it is necessary to standardize the fatigue monitoring information from each dimension, converting it into data with the same dimensions and similar numerical ranges.

[0134] Furthermore, after completing the data standardization process and determining the weight of each dimension, weighted summation or other fusion algorithms are used to fuse the fatigue monitoring information of each dimension into a cumulative fatigue information.

[0135] The fatigue detection result is determined according to the accumulated fatigue information.

[0136] For example, when the fatigue monitoring information corresponding to the breathing sound signal indicates a decrease in breathing frequency, and the fatigue monitoring information corresponding to the yawning sound signal indicates an increase in the number of yawns, the accumulated fatigue information obtained can be expressed as a high degree of fatigue, and the fatigue detection result is severe fatigue.

[0137] For example, the accumulated fatigue information can be a continuous value or a classification or grade. For example, the fatigue state can be divided into different levels, such as mild fatigue, moderate fatigue, and severe fatigue. Then, based on the accumulated fatigue information and the graded levels, the fatigue level of the target object can be determined to obtain a fatigue detection result.

[0138] The above technical solution effectively integrates fatigue information from various dimensions by determining the comprehensive indicator of cumulative fatigue information, fully utilizes the information provided by different dimensions, compensates for the limitations that may exist in a single dimension, more comprehensively reflects the fatigue status of the object being tested, and effectively improves the accuracy and reliability of fatigue detection.

[0139] In some possible implementations, see Figure 4 As shown, in step S11, the fatigue detection result is determined by fusing the multi-dimensional sound signal of the target object, including: In step S21 , a target sound signal corresponding to the dimension is determined according to the weight vector and the sound signal of each dimension of the target object.

[0140] The original collected sound signals often contain various noises, such as environmental noise (such as surrounding conversations and device operation sounds) and device noise (such as microphone noise). This noise can interfere with the extraction of valid information from the sound signals, leading to inaccurate determination of fatigue eigenvalues. Therefore, filtering and noise reduction are first necessary for the sound signals.

[0141] In the disclosed embodiments, a bandpass filter can be used, with the filter's passband set based on the frequency range of the sound signal. For example, the frequency range of normal breathing sounds is relatively low. Using a bandpass filter can remove high-frequency environmental noise and low-frequency device interference (such as power supply interference), preserving the effective components of the breathing sound signal.

[0142] In the embodiment of the present disclosure, spectral subtraction or wavelet threshold denoising can be used to subtract the power spectrum of the noise from the power spectrum of the noisy sound signal to obtain a power spectrum estimate of the pure sound signal, and then the denoised target sound signal is obtained through inverse Fourier transform.

[0143] The weight vector is a dynamically adjusted set of parameters used to weight each dimension of the target object's multidimensional sound signal. These weights are continuously updated based on the characteristics of the input signal, the desired output, and preset optimization criteria to adapt to the changing contribution of each dimension to fatigue detection in different environments.

[0144] The effectiveness and reliability of sound signals of different dimensions in fatigue detection are affected by a variety of factors, such as ambient noise, signal strength, and the correlation between dimension and fatigue status. The weight vector adjusts the weights of sound signals of different dimensions in real time based on these factors, highlighting dimensional signals that contribute significantly to fatigue detection and suppressing dimensional signals with high noise interference or low correlation, thereby improving the accuracy and robustness of fatigue detection.

[0145] The weight vector is determined based on an iterative process within an adaptive filtering algorithm. For example, the Least Mean Squared Error (LMS) algorithm calculates the error between the actual output and the desired output and uses this error to update the weight vector, gradually reducing the error.

[0146] In step S22, the fatigue detection result is determined by fusing the target sound signal corresponding to each dimension.

[0147] In the disclosed embodiment, the collected sound signal is filtered and processed to obtain a noise-free target sound signal. Features that can reflect fatigue status are extracted from the target sound signal, such as time domain features (such as short-term energy and zero-crossing rate), frequency domain features (such as fundamental frequency and spectral energy distribution), and cepstrum features (such as Mel-frequency cepstral coefficients (MFCCs). These features are then integrated to determine the fatigue detection result.

[0148] This technical solution utilizes weight vectors to optimize the sound signals in each dimension to obtain a target sound signal, ensuring the quality and relevance of the input signal and more accurately extracting key fatigue-related information. The target sound signals from each dimension are then combined to determine the fatigue detection result. This leverages the advantages of multidimensional information, integrating fatigue characteristics reflected by different dimensions, reducing the uncertainty of single-dimensional detection and improving the accuracy of fatigue detection results to the actual fatigue state.

[0149] In some possible implementations, in step S21, determining the target sound signal corresponding to the dimension based on the weight vector and the sound signal of each dimension of the target object includes: An error signal corresponding to each dimension is determined according to the weight vector and the sound signal of each dimension of the target object.

[0150] The error signal is the difference between the expected output signal and the actual output signal during the adaptive filtering process. It reflects the degree of deviation between the output of the current adaptive filter (determined by the weight vector and the input signal) and the expected output.

[0151] In the disclosed embodiments, the sound signal of each dimension is weighted using the current weight vector to obtain the actual output signal of that dimension. Weighted processing can be viewed as a linear combination of the sound signals, where the weight vector is used to assign different importance to sound signals at different times or with different characteristics.

[0152] The actual output signal is then compared with the expected output signal to generate an error signal. The expected output signal is typically set based on known fatigue state characteristics or a standard signal, representing the expected sound signal output under that fatigue state. The error signal reflects the deviation between the actual output and the expected output.

[0153] A new weight vector is determined according to the weight vector, the sound signal and the error signal.

[0154] In the disclosed embodiments, a suitable weight update rule can be selected based on different adaptive filtering algorithms. Examples of weight update rules include the least mean square error (LMS) algorithm, the normalized least mean square error (NLMS) algorithm, and the recursive least squares (RLS) algorithm.

[0155] According to the sound signal and the new weight vector, the steps of determining the error signal corresponding to each dimension to determining the new weight vector are re-executed until the error signal meets a preset condition.

[0156] The preset condition is the criterion used to determine whether to stop iteration. It can be a threshold of the error signal, an upper limit on the number of iterations, or other indicators related to algorithm performance. For example, if the error signal is less than the preset threshold, it indicates that the actual output is very close to the expected output, the algorithm has converged, and further iterations will not significantly improve the results. The preset condition can also take the number of iterations into account to prevent the algorithm from iterating infinitely if it fails to converge. By setting the preset condition appropriately, you can improve computational efficiency while maintaining algorithm performance.

[0157] In the disclosed embodiment, by repeatedly calculating the error signal and updating the weight vector, the weight vector is gradually adjusted to its optimal state, resulting in a continuous decrease in the error signal. Each iteration uses the new weight vector obtained from the previous iteration and the current sound signal to calculate a new error signal, and then updates the weight vector again based on the new error signal. This process is similar to a feedback control system, which continuously adjusts parameters to approach the target state.

[0158] In each iteration, the error signal and new weight vector are continuously calculated and updated. First, the error signal is calculated based on the current weight vector and the sound signal. Then, the weight vector is updated based on the weight vector, the sound signal, and the error signal. This process is repeated until the error signal meets the preset conditions.

[0159] The error signal corresponding to each dimension that meets the preset conditions is determined as the target sound signal of that dimension.

[0160] In this disclosed embodiment, when the error signal meets the preset conditions, the weight vectors at that point have already achieved a relatively ideal weighted combination of the sound signals in each dimension, and the corresponding error signal can better reflect fatigue characteristics. Therefore, the error signal that meets the preset conditions is determined as the target sound signal for that dimension.

[0161] During the iterative process, after each calculated error signal is obtained, it is checked to see if it meets the preset conditions. If so, the iteration stops and the current error signal is used as the target sound signal for that dimension. For multiple dimensions, the above iterative process is repeated for each dimension to obtain the target sound signal corresponding to each dimension.

[0162] During the iterative process of determining the error signal and updating the weight vector, this technical solution continuously learns and adapts to the relationship between different dimensional sound signals and fatigue characteristics, automatically adjusting the weight of each dimensional sound signal in fatigue detection. This effectively reduces the impact of factors such as noise interference and individual differences on the detection results. The resulting target sound signal can more accurately reflect the target subject's fatigue state.

[0163] In some possible implementations, determining a new weight vector according to the weight vector, the sound signal, and the error signal includes: An autocorrelation matrix is ​​determined according to the sound signal within a time range of a preset length.

[0164] The autocorrelation matrix describes the correlation between sound signals at different time delays. For a given sound signal, the elements of the autocorrelation matrix reflect the degree of similarity between the sound signals at different time points. In adaptive filtering algorithms, the autocorrelation matrix characterizes the statistical properties of the input signal, enabling a better understanding of its structure and dynamics. The autocorrelation matrix, derived by performing correlation calculations on these sampled values, can capture characteristics such as signal periodicity and trends.

[0165] In the disclosed embodiment, the sound signal of the target object is sampled to obtain a discrete signal sequence. The sampled signal sequence is then divided into several segments based on a preset time range. For each signal segment, an autocorrelation function is calculated. The autocorrelation function reflects the degree of similarity between the sound signals at different time delays. The autocorrelation functions of each signal segment are combined to construct an autocorrelation matrix.

[0166] A step size factor is determined according to the autocorrelation matrix.

[0167] The step size factor is used to control the update speed of the weight vector, which determines the magnitude of the adjustment of the weight vector according to the error signal at each iteration.

[0168] In the disclosed embodiments, since the characteristics of the sound signal may change over time, the step factor also needs to be dynamically adjusted. The step factor can be updated in real time based on changes in the autocorrelation matrix to adapt to changes in the signal. During each iteration, the autocorrelation matrix is ​​recalculated, and the step factor is updated based on the new autocorrelation matrix. For example, a sliding window method can be used to calculate the autocorrelation matrix and step factor by considering only the signal data within the most recent period, thereby achieving dynamic adjustment of the step factor.

[0169] A new weight vector is determined according to the step factor, the weight vector, the sound signal, and the error signal.

[0170] In the disclosed embodiment, during each iteration, a new weight vector is calculated according to the weight vector update formula based on the current step size factor, weight vector, sound signal, and error signal. The new weight vector is used for the next weighting of the input signal to further reduce the error signal.

[0171] At the beginning of each iteration, the current weight vector w(n), the sound signal vector s(n), and the error signal e(n) are obtained. Then, based on the selected weight vector update formula, a new weight vector w(n+1) is calculated. This process is repeated until the error signal meets the preset conditions, resulting in the final weight vector.

[0172] This technical solution calculates the autocorrelation matrix based on the sound signal within a preset time range, fully exploiting the characteristic information of the sound signal and improving the accuracy of the step-size factor determination. The step-size factor is then dynamically adjusted based on the autocorrelation matrix, making the weight vector update process more flexible and efficient, and able to quickly adapt to changes in different sound signals. Finally, a new weight vector is determined by comprehensively considering the step-size factor, weight vector, sound signal, and error signal. This achieves precise updating of the weight vector, more accurately extracting fatigue-related features from the sound signal, reducing the influence of factors such as noise interference and individual differences, and significantly improving the accuracy and reliability of fatigue detection results.

[0173] In some possible implementations, determining the error signal corresponding to each dimension based on the weight vector and the sound signal of each dimension of the target object includes: An adjusted sound signal corresponding to each dimension is determined according to the sound signal of each dimension and the weight vector.

[0174] In the disclosed embodiments, during the initialization phase, the weight vector can be set to an initial value, such as an all-zero vector or a random vector. Then, through subsequent iterations, the weight vector is continuously adjusted based on the error signal, gradually adapting it to the target subject's fatigue state and acoustic signal characteristics. In each iteration, the acoustic signal is calculated based on the current weight vector and acoustic signal. The weight vector is then updated using the error between the adjusted acoustic signal and the desired output signal (correlated with the fatigue state).

[0175] The error signal corresponding to the dimension is determined according to the adjusted sound signal and the sound signal corresponding to the dimension.

[0176] In the disclosed embodiment, the difference between the adjusted sound signal and the sound signal corresponding to the dimension is calculated to obtain an error signal, which reflects the difference between the adjusted sound signal and the desired output signal. The desired output signal is typically set based on known fatigue state characteristics or a standard signal and represents the expected sound signal output under that fatigue state. By calculating the error between the adjusted sound signal and the desired output signal, the performance of the current weight vector and signal processing process can be quantified.

[0177] In the disclosed embodiments, by analyzing the magnitude and direction of the error signal, the direction and magnitude of the required weight vector adjustment can be determined. If the error signal is large, it indicates that the current weight vector is not well adapted to the signal and fatigue state, and a larger adjustment is required. If the error signal is small, it indicates that the weight vector is close to the optimal state, and the adjustment can be reduced accordingly.

[0178] This technical solution uses weight vectors to generate and adjust sound signals, achieving preliminary optimization of each dimension's sound signal, bringing it closer to fatigue-related characteristics and helping to improve the specificity of fatigue detection. Then, by comparing the adjusted sound signal with the original sound signal to generate an error signal, it is possible to quantitatively evaluate the processing effect of the current weight vector and clearly identify the difference between the performance of each dimension's sound signal in fatigue detection and the expected performance. Based on these error signals, the weight vector can be further adjusted to continuously optimize the sound signal processing process, ultimately improving the accuracy and reliability of fatigue detection results.

[0179] In some possible implementations, the sound collection device includes at least one of the following: a sound collection device configured in the vehicle itself, or a sound collection device of a terminal communicating with the vehicle.

[0180] In the embodiment of the present disclosure, the sound collection device configured in the vehicle itself may be, for example, a microphone arranged on the steering wheel. The microphone may be a microphone reused with other vehicle systems, or a microphone arranged separately for fatigue monitoring.

[0181] In the embodiments of the present disclosure, terminals communicating with the vehicle may include wearable devices such as headphones, mobile phones, and smart watches, and these terminals may communicate with the vehicle via wireless methods such as Wi-Fi and Bluetooth. Communication with the vehicle may be direct communication via Bluetooth or indirect communication with the vehicle. For example, a sound collection device of a headphone or smart watch collects sound signals from a target object, and the headphone or smart watch communicates with the mobile phone via Bluetooth. The mobile phone establishes Wi-Fi communication with the vehicle, and the sound signals collected by the headphone or smart watch are first transmitted to the mobile phone, which then transmits them to the vehicle.

[0182] The above technical solution obtains sound signals through the sound collection device configured by the vehicle itself, realizes the reuse of the sound collection device on the vehicle, and reduces manufacturing costs; obtains sound signals through the sound collection device of the terminal communicating with the vehicle, thereby improving the flexibility of sound signal collection.

[0183] In some possible implementations, the sound signal includes at least one of the following: a breathing sound signal, a yawning sound signal, and a coughing sound signal.

[0184] In the disclosed embodiments, a person's breathing pattern may change when fatigued. For example, fatigue may cause a slower or faster breathing rate, changes in breathing depth, or irregular breathing rhythms. These changes are reflected in the characteristics of the respiratory sound signal, such as changes in the frequency range of breathing sounds, changes in the intensity of breathing sounds, and uneven intervals between breathing sounds. By analyzing the respiratory sound signal, these fatigue-related features can be extracted and used to determine the target subject's fatigue level.

[0185] In the disclosed embodiments, yawning frequency may increase with increasing fatigue, and the characteristics of the yawning sound signal (such as the intensity and duration) may also change. For example, when extremely tired, yawns may become louder and last longer. Analyzing the characteristics of yawning sound signals can assist in determining the subject's fatigue level.

[0186] In the disclosed embodiment, the frequency of coughing may increase when fatigued, and the coughing sound may become hoarse, weak, or last longer. By analyzing the characteristics of the cough sound signal, the fatigue level of the target subject can be comprehensively determined in combination with other fatigue characteristics.

[0187] The above technical solution introduces breathing sound signals and yawning sound signals as the basis for multi-dimensional fatigue detection, fully utilizing the characteristics and advantages of different sound signals, and significantly improving the accuracy, sensitivity and reliability of fatigue detection.

[0188] In some possible implementations, the fatigue prompt includes at least one of the following: an auditory prompt, a visual prompt, and a vibration prompt.

[0189] In the disclosed embodiment, the auditory prompt may be a sound played through a vehicle-mounted or wearable device. For example, when the fatigue detection result indicates that the driver is driving fatigued, a warning sound may be emitted through the vehicle-mounted speaker to remind the driver to pay attention to the fatigue state.

[0190] In the disclosed embodiment, the visual prompt can be a fatigue prompt displayed through, for example, an on-board instrument or a multimedia display screen or a HUD (Head-Up Display). The visual prompt can prompt the driver to rest by displaying text or images or flashing the display screen or changing the background color of the display screen.

[0191] In the disclosed embodiment, a vibration reminder can be used to indicate fatigue through vibration of the seat or steering wheel. The vibration intensity can be adjusted according to the fatigue index. For example, if the fatigue detection result indicates that the driver is only slightly fatigued, a low-frequency, mild vibration can be used as a fatigue reminder. If the fatigue detection result indicates that the driver is severely fatigued, a high-frequency, heavier vibration can be used as a fatigue reminder.

[0192] The above technical solution significantly enhances the effectiveness and adaptability of fatigue notifications by providing at least one of auditory, visual, and vibration cues. Auditory cues can attract users' attention in a variety of environments, visual cues can clearly and intuitively present information, and vibration cues can provide warnings despite auditory and visual interference.

[0193] In some possible implementations, see Figure 5As shown, in step S11, the fatigue detection result is determined by fusing the multi-dimensional sound signal of the target object, including: In step S31, an initial detection result is determined based on the multi-dimensional sound signal of the target object.

[0194] In step S32, the fatigue detection result is determined by fusing the initial detection result and the target information.

[0195] In the disclosed embodiments, in addition to obtaining initial detection results based on sound signals, target information from other vehicle data sources, such as driving time, vehicle speed, and steering wheel operation, can also be combined. For example, target information can be obtained from other vehicle controllers or domain controllers via the CAN bus and integrated with the initial detection results to obtain fatigue detection results, thereby improving the accuracy of fatigue detection.

[0196] This technical solution determines the initial detection result by fusing multidimensional sound signals, fully utilizing the multidimensional information in the sound signals to accurately capture the target subject's fatigue state. This initial detection result is then combined with the target information to determine the final fatigue detection result. This optimization and correction of the initial result takes into account individual differences in the target subject and the impact of external environmental factors on fatigue status. This makes the fatigue detection result more consistent with actual conditions and effectively reduces the false positive rate.

[0197] In some possible implementations, the target information includes at least one of the following: driving time information, vehicle speed information, steering wheel operation information, breathing frequency in a sleeping state, and heart rate in a sleeping state.

[0198] In the disclosed embodiment, the driving time information can obtain the current driving time length information from the CAN bus to determine whether the driver has been driving for a long time (for example, more than 2 hours) and give a higher weight in fatigue detection.

[0199] In the embodiment of the present disclosure, the vehicle speed information can obtain the current driving speed data of the vehicle, and combine it with the initial detection results obtained by the sound signal to detect whether fatigue exists at different vehicle speeds and different driving times.

[0200] In the disclosed embodiments, the steering wheel operation information may be obtained by monitoring the driver's steering wheel operation frequency through a sensor on a device mounted on the steering wheel, and combined with the initial detection results to further confirm the driver's fatigue status. For example, if the driver has not made any steering wheel adjustments for an extended period of time, and the initial detection results indicate that the driver is in a fatigued driving state, the fatigue detection result may be determined to indicate fatigue.

[0201] In an embodiment of the present disclosure, when the user turns on the terminal to monitor the breathing and heartbeat of the target object in the sleep state, and allows the terminal to upload the breathing frequency and heartbeat frequency of the target object in the sleep state to the vehicle, when the target object carries the terminal into the vehicle, the breathing frequency and heartbeat frequency of the target object in the sleep state collected by the terminal can be uploaded to the vehicle, and then the main sleeping breathing and heartbeat frequency of the target object can be obtained. For example, if the heart rate of the target object in the non-sleep state is 80-90min / time, and the heart rate in the sleep state is 60-65min / time, but the average heart rate is 70-80min / time, it means that the target object's heart rate is low and there may be sleep quality problems, and this is used as a feature value to identify whether the target object has driving fatigue.

[0202] The above technical solution takes at least one of the following information: driving time information, vehicle speed information, steering wheel operation information, respiratory rate in sleep state, and heart rate in sleep state as target information, and fuses it with the initial detection results determined based on multi-dimensional sound signals, significantly enhancing the accuracy and reliability of fatigue detection. Driving time information provides a basis for fatigue judgment from the time dimension, while vehicle speed information and steering wheel operation information supplement the key information for judging fatigue status from the perspective of dynamic driving behavior. By comprehensively considering these target information, the driver's fatigue level can be assessed more comprehensively and accurately, and potential fatigue driving risks can be discovered in a timely manner, thereby effectively preventing traffic accidents caused by fatigue driving. By fusing the results of multiple data sources such as sound signals, driving time information, vehicle speed information, and steering wheel operation information, a comprehensive assessment of whether the driver is fatigued is achieved based on multiple factors, avoiding misjudgments caused by relying on a single data source.

[0203] The present disclosure also provides a fatigue reminder device. Figure 6 Shown, including: The determination module 110 is configured to determine a fatigue detection result based on a sound signal of the target object, wherein the sound signal is obtained by a sound collection device in the vehicle; the prompt module 120 is configured to perform a fatigue prompt if the fatigue detection result indicates that the target object is driving fatigued.

[0204] In some possible implementations, the sound signal includes a multi-dimensional sound signal, and the determination module 110 includes: a monitoring value determination submodule, configured to determine the fatigue monitoring information corresponding to each dimension based on the sound signal of each dimension of the target object; and a fusion determination submodule, configured to fuse the fatigue monitoring information corresponding to each dimension to determine the fatigue detection result.

[0205] In some possible implementations, the monitoring value determination submodule includes: a characteristic value determination unit, configured to determine the fatigue characteristic value corresponding to the dimension based on the multi-dimensional sound signal of the target object; a monitoring value determination unit, configured to determine the fatigue monitoring information corresponding to each dimension based on the fatigue characteristic value corresponding to each dimension and the fatigue trigger threshold corresponding to the dimension.

[0206] In some possible implementations, the multidimensional sound signal includes a breathing sound signal, and the characteristic value determination unit is configured to: determine the breathing frequency of the target object based on the breathing sound signal in the multidimensional sound signal; and determine the fatigue characteristic value corresponding to the breathing sound signal based on the breathing frequency of the target object.

[0207] In some possible implementations, the multidimensional sound signal includes a yawn sound signal, and the characteristic value determination unit is configured to: determine the yawn characteristic information of the target object based on the yawn sound signal in the multidimensional sound signal; and determine the fatigue characteristic value corresponding to the yawn sound signal based on the yawn characteristic information of the target object.

[0208] In some possible implementations, the yawn feature information includes at least one of the following: the number of yawns, the frequency of yawns, the volume of yawns, and the duration of yawns.

[0209] In some possible implementations, the monitoring value determination unit is configured to obtain the fatigue trigger threshold through self-learning in the following manner: determining the key parameters of the fatigue characteristic values ​​of each dimension based on the fatigue characteristic values ​​corresponding to the historical sound signals of each person of the target object; determining the fatigue feature label of the target object based on the key parameters of the fatigue characteristic values ​​of each dimension; determining the fatigue trigger threshold of each dimension based on the baseline fatigue threshold and the fatigue feature label corresponding to each dimension.

[0210] In some possible implementations, the fusion determination submodule includes: a fusion determination unit, configured to determine the cumulative fatigue information based on the fatigue monitoring information corresponding to each dimension; and a result determination unit, configured to determine the fatigue detection result based on the cumulative fatigue information.

[0211] In some possible implementations, the determination module 110 includes: a signal fusion determination submodule, configured to determine the target sound signal corresponding to the dimension based on the weight vector and the sound signal of each dimension of the target object; a fatigue result determination submodule, configured to fuse and determine the fatigue detection result based on the target sound signal corresponding to each dimension.

[0212] In some possible implementations, the signal fusion determination submodule is configured to: determine the error signal corresponding to each dimension based on the weight vector and the sound signal of each dimension of the target object; determine a new weight vector based on the weight vector, the sound signal and the error signal; re-execute the steps of determining the error signal corresponding to each dimension to determining the new weight vector based on the sound signal and the new weight vector until the error signal meets the preset conditions; and determine the error signal corresponding to each dimension that meets the preset conditions as the target sound signal of that dimension.

[0213] In some possible implementations, the signal fusion determination submodule is configured to: determine an autocorrelation matrix based on the sound signal within a time range of a preset length; determine a step factor based on the autocorrelation matrix; and determine a new weight vector based on the step factor, the weight vector, the sound signal, and the error signal.

[0214] In some possible implementations, the signal fusion determination submodule is configured to: determine the adjusted sound signal corresponding to each dimension based on the sound signal and the weight vector of each dimension; and determine the error signal corresponding to the dimension based on the adjusted sound signal and the sound signal corresponding to the dimension.

[0215] In some possible implementations, the sound collection device includes at least one of the following: a sound collection device configured in the vehicle itself, or a sound collection device of a terminal communicating with the vehicle.

[0216] In some possible implementations, the sound signal includes at least one of the following: a breathing sound signal, a yawning sound signal, and a coughing sound signal.

[0217] In some possible implementations, the fatigue prompt includes at least one of the following: an auditory prompt, a visual prompt, and a vibration prompt.

[0218] In some possible implementations, the fusion determination module 110 includes: an initial result determination submodule, configured to determine the initial detection result based on the sound signal of the target object; and a fusion result determination submodule, configured to fuse and determine the fatigue detection result based on the initial detection result and the target information.

[0219] In some possible implementations, the target information includes at least one of the following: driving time information, vehicle speed information, steering wheel operation information, breathing frequency in a sleeping state, and heart rate in a sleeping state.

[0220] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0221] The present disclosure also provides a vehicle, including: A processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions stored in the memory to implement the method of any one of the aforementioned embodiments.

[0222] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method described in any one of the aforementioned embodiments are implemented.

[0223] An embodiment of the present disclosure further provides a computer program product, including a computer program, which implements the steps of any one of the methods in the aforementioned embodiments when executed by a processor.

[0224] Figure 7 FIG6 is a block diagram illustrating a vehicle 600 according to an exemplary embodiment. For example, vehicle 600 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or another type of vehicle. Vehicle 600 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0225] Reference Figure 7 Vehicle 600 may include various subsystems, such as an infotainment system 610, a perception system 620, a decision control system 630, a drive system 640, and a computing platform 650. Vehicle 600 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and each component of vehicle 600 may be interconnected via wired or wireless means.

[0226] In some embodiments, the infotainment system 610 may include a communication system, an entertainment system, a navigation system, and the like.

[0227] The perception system 620 may include several sensors for sensing information about the environment surrounding the vehicle 600. For example, the perception system 620 may include a global positioning system (which may be a GPS system, a BeiDou system, or another positioning system), an inertial measurement unit (IMU), a laser radar, a millimeter-wave radar, an ultrasonic radar, and a camera.

[0228] The decision control system 630 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0229] The drive system 640 may include components that provide power to the vehicle 600. In one embodiment, the drive system 640 may include an engine, an energy source, a transmission system, and wheels. The engine may be an internal combustion engine, an electric motor, an air compression engine, or a combination thereof. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0230] Some or all functions of the vehicle 600 are controlled by a computing platform 650. The computing platform 650 may include at least one processor 651 and a memory 652. The processor 651 may execute instructions 653 stored in the memory 652.

[0231] The processor 651 can be any conventional processor, such as a commercially available CPU. The processor can also include a graphics processor (GPU), a field programmable gate array (FPGA), a system on chip (SOC), an application specific integrated circuit (ASIC), or a combination thereof.

[0232] The memory 652 may be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0233] In addition to instructions 653 , memory 652 may also store data, such as road maps, route information, and vehicle location, direction, speed, etc. The data stored in memory 652 may be used by computing platform 650 .

[0234] In the embodiment of the present disclosure, the processor 651 may execute the instruction 653 to complete all or part of the steps of the above-mentioned fatigue prompt method.

[0235] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the appended claims.

[0236] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A fatigue prompting method, characterized in that: include: Determining a fatigue detection result based on a sound signal of the target object, wherein the sound signal is obtained by a sound collection device in the vehicle; When the fatigue detection result indicates that the target object is driving in a fatigue state, a fatigue prompt is performed.

2. The method according to claim 1, characterized in that The sound signal includes a multi-dimensional sound signal, and determining the fatigue detection result according to the sound signal of the target object includes: Determining fatigue monitoring information corresponding to each dimension according to the sound signal of each dimension of the target object; The fatigue monitoring information corresponding to each dimension is fused to determine the fatigue detection result.

3. The method according to claim 2, characterized in that The step of determining fatigue monitoring information corresponding to each dimension based on the sound signal of each dimension of the target object includes: Determining fatigue characteristic values ​​corresponding to the dimensions based on the multi-dimensional sound signals of the target object; Fatigue monitoring information corresponding to each dimension is determined according to the fatigue characteristic value corresponding to each dimension and the fatigue trigger threshold corresponding to the dimension.

4. The method according to claim 3, characterized in that The multi-dimensional sound signal includes a breathing sound signal, and determining the fatigue characteristic value corresponding to the dimension based on the multi-dimensional sound signal of the target object includes: determining a respiratory frequency of the target subject based on the respiratory sound signal in the multi-dimensional sound signal; A fatigue characteristic value corresponding to the breathing sound signal is determined according to the breathing frequency of the target object.

5. The method according to claim 3, characterized in that The multi-dimensional sound signal includes a yawn sound signal, and determining the fatigue characteristic value corresponding to the dimension based on the multi-dimensional sound signal of the target object includes: determining yawn feature information of the target object according to the yawn sound signal in the multi-dimensional sound signal; A fatigue feature value corresponding to the yawn sound signal is determined according to the yawn feature information of the target object.

6. The method according to claim 5, characterized in that The yawn characteristic information includes at least one of the following: the number of yawns, the frequency of yawns, the volume of yawns, and the duration of yawns.

7. The method according to claim 3, characterized in that The fatigue trigger threshold is obtained by self-learning in the following way: Determining key parameters of the fatigue characteristic values ​​of each dimension according to the fatigue characteristic values ​​corresponding to the historical sound signals of each of the target objects; determining a fatigue feature label of the target object according to the key parameters of the fatigue feature values ​​of each dimension; The fatigue triggering threshold of each dimension is determined according to the baseline fatigue threshold and the fatigue feature label corresponding to each dimension.

8. The method according to claim 2, characterized in that The fusing of the fatigue monitoring information corresponding to each dimension to determine the fatigue detection result includes: Determining accumulated fatigue information according to the fatigue monitoring information corresponding to each dimension; The fatigue detection result is determined according to the accumulated fatigue information.

9. The method according to claim 2, characterized in that The step of fusing and determining fatigue detection results based on multi-dimensional sound signals of the target object includes: Determining a target sound signal corresponding to the dimension according to the weight vector and the sound signal of each dimension of the target object; The fatigue detection result is determined by fusing the target sound signal corresponding to each dimension.

10. The method according to claim 9, characterized in that Determining the target sound signal corresponding to the dimension according to the weight vector and the sound signal of each dimension of the target object includes: Determining an error signal corresponding to each dimension according to the weight vector and the sound signal of each dimension of the target object; determining a new weight vector according to the weight vector, the sound signal, and the error signal; Re-performing the steps of determining the error signal corresponding to each dimension and determining the new weight vector according to the sound signal and the new weight vector until the error signal satisfies a preset condition; The error signal corresponding to each dimension that meets the preset conditions is determined as the target sound signal of that dimension.

11. The method according to claim 10, characterized in that The determining a new weight vector according to the weight vector, the sound signal, and the error signal includes: determining an autocorrelation matrix based on the sound signal within a time range of a preset length; Determining a step size factor according to the autocorrelation matrix; A new weight vector is determined according to the step factor, the weight vector, the sound signal, and the error signal.

12. The method according to claim 10, characterized in that The determining, based on the weight vector and the sound signal of each dimension of the target object, an error signal corresponding to each dimension includes: Determining an adjusted sound signal corresponding to each dimension according to the sound signal of each dimension and the weight vector; The error signal corresponding to the dimension is determined according to the adjusted sound signal and the sound signal corresponding to the dimension.

13. The method according to claim 1, wherein The sound collection device includes at least one of the following: a sound collection device configured by the vehicle itself, and a sound collection device of a terminal communicating with the vehicle.

14. The method according to claim 1, wherein The sound signal includes at least one of the following: a breathing sound signal, a yawning sound signal, and a coughing sound signal.

15. The method according to claim 1, wherein The fatigue prompt includes at least one of the following: an auditory prompt, a visual prompt, and a vibration prompt.

16. The method according to any one of claims 1 to 15, characterized in that Determining the fatigue detection result according to the sound signal of the target object includes: determining an initial detection result according to the sound signal of the target object; The fatigue detection result is determined by fusing the initial detection result and the target information.

17. The method according to claim 16, characterized in that The target information includes at least one of the following: driving time information, vehicle speed information, steering wheel operation information, breathing frequency in a sleeping state, and heart rate in a sleeping state.

18. A fatigue reminder device, characterized in that: include: a determination module configured to determine a fatigue detection result based on a sound signal of the target object, wherein the sound signal is obtained by a sound collection device in the vehicle; The prompt module is configured to perform fatigue prompting when the fatigue detection result indicates that the target object is driving in a fatigued state.

19. The device according to claim 18, characterized in that The sound signal includes a multi-dimensional sound signal, and the determination module includes: a monitoring value determination submodule, configured to determine fatigue monitoring information corresponding to each dimension based on the sound signal of each dimension of the target object; The fusion determination submodule is configured to fuse the fatigue monitoring information corresponding to each dimension to determine the fatigue detection result.

20. The device according to claim 19, characterized in that The monitoring value determination submodule includes: a characteristic value determining unit configured to determine fatigue characteristic values ​​corresponding to the dimensions based on the multi-dimensional sound signal of the target object; The monitoring value determination unit is configured to determine the fatigue monitoring information corresponding to each dimension according to the fatigue characteristic value corresponding to each dimension and the fatigue trigger threshold corresponding to the dimension.

21. The device according to claim 20, characterized in that The multi-dimensional sound signal includes a breathing sound signal, and the characteristic value determination unit is configured to: determining a respiratory frequency of the target subject based on the respiratory sound signal in the multi-dimensional sound signal; A fatigue characteristic value corresponding to the breathing sound signal is determined according to the breathing frequency of the target object.

22. The device according to claim 20, characterized in that The multi-dimensional sound signal includes a yawn sound signal, and the characteristic value determination unit is configured to: determining yawn feature information of the target object according to the yawn sound signal in the multi-dimensional sound signal; A fatigue feature value corresponding to the yawn sound signal is determined according to the yawn feature information of the target object.

23. The device according to claim 22, characterized in that The yawn characteristic information includes at least one of the following: the number of yawns, the frequency of yawns, the volume of yawns, and the duration of yawns.

24. The device according to claim 19, characterized in that The fusion determination submodule includes: a fusion determination unit configured to determine accumulated fatigue information based on the fatigue monitoring information corresponding to each dimension; The result determination unit is configured to determine the fatigue detection result according to the accumulated fatigue information.

25. The device according to claim 19, characterized in that The result determination unit includes: a signal fusion determination submodule, configured to determine a target sound signal corresponding to the dimension based on the weight vector and the sound signal of each dimension of the target object; The fatigue result determination submodule is configured to determine the fatigue detection result by fusing the target sound signal corresponding to each dimension.

26. The device according to claim 18, characterized in that The sound collection device includes at least one of the following: a sound collection device configured by the vehicle itself, and a sound collection device of a terminal communicating with the vehicle.

27. The device according to claim 18, characterized in that The sound signal includes at least one of the following: a breathing sound signal, a yawning sound signal, and a coughing sound signal.

28. The device according to claim 18, wherein The fatigue prompt includes at least one of the following: an auditory prompt, a visual prompt, and a vibration prompt.

29. The device according to any one of claims 18 to 28, characterized in that The determining module includes: an initial result determination submodule, configured to determine an initial detection result based on the sound signal of the target object; The fusion result determination submodule is configured to fuse and determine the fatigue detection result according to the initial detection result and target information.

30. The device according to claim 29, characterized in that The target information includes at least one of the following: driving time information, vehicle speed information, steering wheel operation information, breathing frequency in a sleeping state, and heart rate in a sleeping state.

31. A vehicle, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the executable instructions stored in the memory to implement the method according to any one of claims 1 to 17.

32. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 17 are implemented.

33. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 17.

Images