Intelligent allergen identification and response method, system and device based on in-vehicle respiratory tract sound analysis, processor and storage medium thereof
By analyzing the respiratory sounds of occupants, a convolutional neural network based on the GoogleNet Inception v1 architecture was used to identify in-vehicle air allergens. Combined with a distributed cockpit domain controller, this solved the problems of accuracy and system fragmentation in existing technologies, achieving high-precision allergen identification and intelligent cockpit response.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- DONGFENG ELECTRONICS TECH
- Filing Date
- 2026-02-26
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technologies cannot accurately identify allergens in the air inside vehicles, and the system architecture is fragmented, making it difficult to meet the requirements for real-time performance, accuracy, and functional scalability.
By analyzing the respiratory sounds of occupants, a convolutional neural network based on the GoogleNet Inception v1 architecture is used for spectral image classification. Combined with a distributed cockpit domain controller architecture, allergen identification and intelligent response are achieved.
It achieves high-precision identification of multiple allergens and personalized cabin response, improving health management capabilities and passenger comfort.
Smart Images
Figure CN122050442A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the interdisciplinary field of intelligent vehicles and health monitoring, specifically to an intelligent allergen identification and response method, system, device, processor, and computer-readable storage medium based on in-vehicle respiratory sound analysis. Background Technology
[0002] With the development of intelligent cockpit technology in automobiles, improving the health and comfort of drivers and passengers has become an important direction. Airborne allergens in the enclosed environment of a car, such as pollen, dust mites, mold, animal dander, and pet hair, are common triggers for respiratory discomfort (such as coughing and wheezing) in sensitive individuals. Currently, mainstream in-vehicle environmental monitoring systems typically assess air "quality" based on various physical or chemical sensors (such as PM2.5 and formaldehyde sensors), but they cannot identify specific allergen types, thus making it difficult to provide targeted and personalized interventions.
[0003] In the prior art, there are some schemes that determine health status through biosignals. For example, prior art document CN202211294192.2 discloses a system for detecting colds by analyzing the sound characteristics of coughs and sneezes. However, the goal of this scheme is to determine the disease state of "whether one has a cold," rather than to identify the types of external allergens that trigger symptoms. The feature extraction and classification methods it uses (such as specific event detection and feature splicing) are fundamentally different from the technical path of this invention. Another prior art document CN202110610810.9 discloses a system for adjusting the cabin environment by linking biosignals with environmental parameters, but its environmental detection relies on traditional air quality sensors and lacks the ability to infer allergens by analyzing occupant physiological responses (such as sound).
[0004] In summary, the existing technology has the following shortcomings: Limitations of the identification target: It cannot achieve a precise mapping from the passive physiological reactions of passengers (respiratory sounds) to external sensitizing causes (specific allergens).
[0005] Limited technical means: There is a lack of technical solutions to apply deep convolutional neural networks (especially complex networks such as GoogleNet Inception) to in-vehicle sound spectrum image classification in order to solve the specific problem of allergen identification.
[0006] Fragmented system architecture: The distributed architecture of edge computing (in-vehicle AI inference), cloud-based large-scale model intelligent decision-making, and cockpit domain controller is not organically combined, making it difficult to meet the comprehensive requirements of real-time performance, accuracy, and functional scalability.
[0007] Therefore, there is an urgent need for a complete technical solution that can non-invasively, dynamically, and accurately identify specific air allergens in the vehicle and automatically trigger intelligent and personalized cabin responses. Summary of the Invention
[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, system, device, processor, and computer-readable storage medium for intelligent allergen identification and response based on in-vehicle respiratory sound analysis. Its core objective is to proactively improve the in-vehicle health environment by analyzing the specific respiratory sounds produced by occupants due to allergen stimulation, identifying the type of allergen, and coordinating with the cabin system to perform precise environmental adjustments and information prompts.
[0009] To achieve the above objectives, the present invention provides an intelligent allergen identification and response method, system, device, processor, and computer-readable storage medium based on in-vehicle respiratory sound analysis as follows: The main feature of this intelligent allergen identification and response method based on in-vehicle respiratory sound analysis is that the method includes the following steps: (1) Sound acquisition and preprocessing: The original respiratory sound signal of the occupant is acquired by the in-vehicle audio acquisition device within a preset time period, and the original signal is subjected to noise reduction and frame segmentation. (2) Spectrum image generation: Perform time-frequency analysis on the preprocessed sound signal to generate the corresponding sound spectrum image; (3) Allergen classification and identification: The sound spectrum image is input into a pre-trained convolutional neural network model for reinforcement learning training, and the allergen type classification result of the current passenger's sound is output. (4) Intelligent suggestion generation and fusion: Based on the classification results, generate targeted environmental adjustment and health suggestion instructions; wherein, at least some of the suggestion instructions generate natural language suggestion text by calling the large language model service and taking the classification results as input; (5) Cockpit response execution: Parse and execute the generated suggested instructions, control at least one environmental actuator in the vehicle to perform actions, and / or output the suggested text through the vehicle human-machine interface.
[0010] Preferably, the convolutional neural network model is built based on the GoogleNet Inception v1 architecture, and its training process includes: Construct a training dataset containing multiple labeled sample pairs. Each sample pair includes a respiratory sound signal collected under known allergen exposure conditions and its corresponding allergen category label. The allergen category label includes pollen, dust mites, mold, animal dander, and hair. The acquired sound signal is converted into a spectrum image and used as model input; The Inception v1-based convolutional neural network model was trained in a supervised manner using the training dataset described above, and gradient propagation was performed using an auxiliary classifier in the network to optimize the model parameters.
[0011] Preferably, in the training dataset, the respiratory sound signal acquisition time for each sample is 30-90 seconds, and the known allergen exposure environment is a high-concentration environment of a single allergen created through environmental sensor calibration or controlled experiments.
[0012] Preferably, the suggested instructions generated in step (4) include: generating a first type of basic control instructions based on the mapping relationship between the classification results and preset rules; and combining the classification results and / or occupant status information into a query statement, sending it to the large language model server, and receiving the returned natural language suggested text as a second type of enhanced suggestion.
[0013] Preferably, the actions of the in-vehicle environment actuator in step (5) include at least one of the following: controlling the air purification system to switch to the high-efficiency filtration mode corresponding to the allergen, controlling the air conditioning system to switch between internal and external circulation, controlling the opening and closing of the windows or sunroof, and adjusting the seat ventilation intensity.
[0014] The intelligent allergen identification and response system based on in-vehicle respiratory sound analysis, used to implement the methods described above, is characterized by adopting a distributed cockpit domain controller architecture, including: The rear processing domain includes a first domain controller, a rear audio acquisition device connected to the first domain controller, and a rear display device. The front control domain includes a second domain controller, a front human-machine interface device connected to the second domain controller, and an environmental actuator interface. The first domain controller is used to receive and process the sound signals collected by the rear audio acquisition device, run the pre-trained convolutional neural network model to complete the classification and recognition, and send the classification results through the vehicle communication network. The second domain controller is used to receive the classification results, generate and execute suggestion instructions, control the environmental actuator interface, and drive the front-row human-machine interaction device to output information.
[0015] Preferably, the first domain controller and the second domain controller communicate via an in-vehicle Ethernet based on the MQTT protocol; the pre-trained convolutional neural network model is deployed locally on the first domain controller.
[0016] Preferably, the second domain controller integrates a large language model client module or access interface for interacting with cloud-based or local large language model services to generate the natural language suggestion text.
[0017] Preferably, both the first domain controller and the second domain controller use the MTK2715 chip; the second domain controller runs the instrument operating system and the in-vehicle infotainment system simultaneously through virtualization technology.
[0018] Preferably, the rear-seat audio acquisition device is a microphone array integrated into the rear seat headrest or B-pillar; the environmental actuator interface is connected to at least one of the body control module, air conditioning control module, and window control module.
[0019] The main feature of this intelligent allergen identification and response device based on in-vehicle respiratory sound analysis is that the device includes: An audio input interface is used to receive raw respiratory sound data from an in-vehicle microphone; The signal processing unit is used to perform noise reduction and frame preprocessing on the raw respiratory sound data. The spectrum conversion unit is used to convert the preprocessed audio data into a time-spectrum image; The neural network processing unit integrates a convolutional neural network model based on the GoogleNet Inception v1 architecture and pre-trained on an allergen sound spectrum dataset, which is used to receive the time-spectrum image and output the allergen classification result. The result output interface is used to send the classification results to the vehicle bus or a specified domain controller. A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the allergen intelligent identification and response method based on in-vehicle respiratory sound analysis described above.
[0020] The main feature of this intelligent allergen identification and response device based on in-vehicle respiratory sound analysis is that the convolutional neural network model integrated in the neural network processing unit uses an allergen sound spectrum dataset for training, which includes labeled respiratory sound samples collected in environments with four single allergens: pollen, dust mites, mold, and animal dander and hair.
[0021] The main feature of this intelligent allergen identification and response processor based on in-vehicle respiratory sound analysis is that the processor is configured to execute computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the intelligent allergen identification and response method based on in-vehicle respiratory sound analysis described above are implemented.
[0022] The computer-readable storage medium is characterized in that it stores a computer program that can be executed by a processor to implement the steps of the above-described intelligent allergen identification and response method based on in-vehicle respiratory sound analysis.
[0023] The intelligent allergen identification and response method, system, device, processor, and computer-readable storage medium based on in-vehicle respiratory sound analysis of the present invention have the following significant advantages compared with the prior art: 1. It pioneered the "sound-based source identification" technology approach: breaking through the traditional model that relies on direct environmental sensing, it indirectly and dynamically identifies allergens by analyzing the physiological reactions (sound) of passengers, making it more targeted and forward-looking.
[0024] 2. High-precision recognition was achieved: By innovatively applying the GoogleNet Inception v1 network structure, which has excellent performance in the field of image recognition, to respiratory sound spectrum image classification, and combining it with targeted dataset training, high-precision and fine-grained differentiation of various allergens was achieved.
[0025] 3. A hardware and software integrated intelligent response closed loop has been constructed: The innovative distributed cockpit domain architecture of "rear row edge AI recognition + front row intelligent decision-making and control" has been designed, which seamlessly integrates high-performance neural network inference, large language model intelligent interaction and vehicle environment control, and realizes a complete automated closed loop from perception, cognition to execution.
[0026] 4. Enhanced user experience and health protection: The system can proactively and silently monitor allergy risks and provide a comprehensive response, from automatic environmental adjustment to personalized health advice, significantly improving the health management capabilities and passenger comfort of the smart cockpit. Attached Figure Description
[0027] Figure 1 This is a flowchart of a specific embodiment of the intelligent allergen identification and response method based on in-vehicle respiratory sound analysis of the present invention.
[0028] Figure 2 This is a schematic diagram of the GoogleNet single-module architecture used in a specific embodiment of the present invention.
[0029] Figure 3This is an architecture diagram of the convolutional neural network model used in a specific embodiment of the present invention.
[0030] Figure 4 This is a diagram of the distributed cockpit domain controller architecture used in the intelligent allergen identification and response system based on in-vehicle respiratory sound analysis of the present invention.
[0031] Figure 5 This is a diagram of the overall cabin domain architecture of the intelligent allergen identification and response system based on in-vehicle respiratory sound analysis according to the present invention. Detailed Implementation
[0032] To more clearly describe the technical content of the present invention, the following description is provided in conjunction with specific embodiments.
[0033] Before describing the embodiments of the present invention in detail, it should be noted that, in the following, the terms “comprising,” “including,” or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus.
[0034] This intelligent allergen identification and response method based on in-vehicle respiratory sound analysis includes the following steps: (1) Sound acquisition and preprocessing: The original respiratory sound signal of the occupant is acquired by the in-vehicle audio acquisition device within a preset time period, and the original signal is subjected to noise reduction and frame segmentation. (2) Spectrum image generation: Perform time-frequency analysis on the preprocessed sound signal to generate the corresponding sound spectrum image; (3) Allergen classification and identification: The sound spectrum image is input into a pre-trained convolutional neural network model for reinforcement learning training, and the allergen type classification result of the current passenger's sound is output. (4) Intelligent suggestion generation and fusion: Based on the classification results, generate targeted environmental adjustment and health suggestion instructions; wherein, at least some of the suggestion instructions generate natural language suggestion text by calling the large language model service and taking the classification results as input; (5) Cockpit response execution: Parse and execute the generated suggested instructions, control at least one environmental actuator in the vehicle to perform actions, and / or output the suggested text through the vehicle human-machine interface.
[0035] In a preferred embodiment of the present invention, the convolutional neural network model is built based on the GoogleNet Inception v1 architecture, and its training process includes: Construct a training dataset containing multiple labeled sample pairs. Each sample pair includes a respiratory sound signal collected under known allergen exposure conditions and its corresponding allergen category label. The allergen category label includes pollen, dust mites, mold, animal dander, and hair. The acquired sound signal is converted into a spectrum image and used as model input; The Inception v1-based convolutional neural network model was trained in a supervised manner using the training dataset described above, and gradient propagation was performed using an auxiliary classifier in the network to optimize the model parameters.
[0036] In a preferred embodiment of the present invention, the respiratory sound signal acquisition time for each sample in the training dataset is 30-90 seconds, and the known allergen exposure environment is a single high-concentration allergen environment created through environmental sensor calibration or controlled experiment.
[0037] As a preferred embodiment of the present invention, the suggested instructions generated in step (4) include: generating a first type of basic control instructions based on the mapping relationship between the classification results and preset rules; and combining the classification results and / or occupant status information into a query statement, sending it to the large language model server, and receiving the returned natural language suggested text as a second type of enhanced suggestion.
[0038] As a preferred embodiment of the present invention, the actions of the in-vehicle environment actuator in step (5) include at least one of the following: controlling the air purification system to switch to the high-efficiency filtration mode corresponding to the allergen, controlling the air conditioning system to switch between internal and external circulation, controlling the opening and closing of the windows or sunroof, and adjusting the seat ventilation intensity.
[0039] The intelligent allergen identification and response system based on in-vehicle respiratory sound analysis, used to implement the above-described method, comprises a distributed cockpit domain controller architecture and includes: The rear processing domain includes a first domain controller, a rear audio acquisition device connected to the first domain controller, and a rear display device. The front control domain includes a second domain controller, a front human-machine interface device connected to the second domain controller, and an environmental actuator interface. The first domain controller is used to receive and process the sound signals collected by the rear audio acquisition device, run the pre-trained convolutional neural network model to complete the classification and recognition, and send the classification results through the vehicle communication network. The second domain controller is used to receive the classification results, generate and execute suggestion instructions, control the environmental actuator interface, and drive the front-row human-machine interaction device to output information.
[0040] In a preferred embodiment of the present invention, the first domain controller and the second domain controller communicate via an in-vehicle Ethernet based on the MQTT protocol; the pre-trained convolutional neural network model is deployed locally on the first domain controller.
[0041] In a preferred embodiment of the present invention, the second domain controller integrates a large language model client module or access interface for interacting with a large language model service in the cloud or locally to generate the natural language suggestion text.
[0042] In a preferred embodiment of the present invention, both the first domain controller and the second domain controller use the MTK2715 chip; the second domain controller runs the instrument operating system and the in-vehicle infotainment system simultaneously through virtualization technology.
[0043] In a preferred embodiment of the present invention, the rear-seat audio acquisition device is a microphone array integrated into the rear seat headrest or B-pillar; the environmental actuator interface is connected to at least one of the body control module, air conditioning control module, and window control module.
[0044] This intelligent allergen identification and response device based on in-vehicle respiratory sound analysis includes: An audio input interface is used to receive raw respiratory sound data from an in-vehicle microphone; The signal processing unit is used to perform noise reduction and frame preprocessing on the raw respiratory sound data. The spectrum conversion unit is used to convert the preprocessed audio data into a time-spectrum image; The neural network processing unit integrates a convolutional neural network model based on the GoogleNet Inception v1 architecture and pre-trained on an allergen sound spectrum dataset, which is used to receive the time-spectrum image and output the allergen classification result. The result output interface is used to send the classification results to the vehicle bus or a specified domain controller. A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the allergen intelligent identification and response method based on in-vehicle respiratory sound analysis described above.
[0045] As a preferred embodiment of the present invention, the convolutional neural network model integrated in the neural network processing unit uses an allergen sound spectrum dataset for training, which includes labeled respiratory sound samples collected in environments with four single allergens: pollen, dust mites, mold, and animal dander and hair.
[0046] The implementation of this technical solution will be explained in further detail below: This technical solution aims to build a complete intelligent cockpit domain, which consists of three parts: training of the sound-to-allergen algorithm, integration with the virtualization-based Android platform, and design of the overall cockpit domain architecture.
[0047] To achieve a method for classifying respiratory sounds into allergens, we adopted the GoogleNet open-source model as the basis for the algorithm module. This convolutional neural network includes an input layer, a convolutional layer, an activation layer, a pooling layer, and an output layer. The network adds two auxiliary softmax layers for forward gradient propagation (auxiliary classifiers). The neural network is modularized by encapsulating the pooling layer, convolutional layer, and activation layer. We use Inception V1 to establish the neural network link from fixed respiratory sounds to allergen classification.
[0048] For sample data collection, the GoogleNet convolutional neural network employed a data collection method involving test personnel. Ten thousand respiratory sounds from allergy sufferers were collected in four scenarios: dust mites and their excrement, pollen, mold, and animal dander and hair. Each sample was collected for 60 seconds, and the respiratory sounds were converted into spectral images to establish a training set for allergen classification. This set was then used for reinforcement learning training of GoogleNet. The classification results were used to provide suggestions using a Deepseek model, which combines local and cloud-based algorithms deployed on the rear-seat controller. For example, if the allergen was identified as pollen, the system would combine the question "What should I do if I encounter pollen allergy in the car?" and send it to Deepseek, which would then utilize its large model computing power to generate appropriate suggestions.
[0049] In the cockpit domain of the automotive electronic and electrical architecture, a two-core, four-screen design is implemented using two MTK2715 domain controllers and two screens: One MTK2715 domain controller (the first domain controller) is placed in the rear, responsible for deploying a convolutional neural network and connecting to the two screens behind the front seats. These two screens serve as entertainment screens for rear passengers, providing not only standard entertainment and in-vehicle functions, but also collecting respiratory sounds, converting them into a sound spectrum, deploying DeepSeek, and transmitting the algorithm input via MQTT through the internal LAN to the front MTK2715 (the second domain controller). The second domain controller is deployed below the central control screen, implementing both the instrument cluster and Android dual systems based on virtualization. The two domain controllers communicate with each other via MQTT, enabling communication between them.
[0050] like Figure 1As shown, this intelligent allergen identification and response method based on in-vehicle respiratory sound analysis includes the following steps: S1: Start monitoring. After the vehicle starts or passengers are seated, the system automatically or with user authorization starts the respiratory sound monitoring mode.
[0051] S2: Sound Acquisition and Processing. Audio data of a set duration (e.g., 60 seconds) is acquired via the in-vehicle microphone and converted into a spectrum image in real time.
[0052] S3: Neural Network Recognition. The spectral image is input into the deployed GoogleNet model for forward propagation calculation to obtain the probability distribution of each allergen category. The one with the highest probability is taken as the recognition result.
[0053] S4: Intelligent Suggestion Generation. Using the recognition results as input, the system calls a large language model service to obtain text-based processing and response suggestions.
[0054] S5: Response Execution and Feedback. The system automatically executes vehicle-specific suggestions (such as environmental control) and simultaneously feeds all information (identification results, suggestions) back to the user through the cockpit screen. The entire process is cyclical, enabling continuous monitoring and dynamic adjustment.
[0055] As a preferred embodiment of the present invention, a GoogleNET convolutional neural network is built using the Tensorflow framework. This neural network combines pooling layers and convolutional layers in a modular fashion, and uses 1×1 convolutional kernels for network optimization, reducing the number of input parameters and minimizing overfitting. The single modular structure is as follows: Figure 2 As shown.
[0056] The overall neural network structure consists of five modular blocks. After image data input, the first modular block comprises a 7×7 convolutional layer with a stride of 2 and a 3×3 pooling layer with a stride of 2. After computation and normalization (LocalRespNorm), it is input into the second modular block (64 1×1 convolutional layers and a 3×3 convolutional layer with a stride of 1), which is also normalized before entering a max pooling layer (3×3 with a stride of 2). The third modular block consists of two parts, each with four branches. For the first part: the first branch uses 64 1×1 convolutional kernels; the second branch uses 96 1×1 convolutional kernels followed by 128 3×3 convolutions; the third branch uses 16 1×1 convolutional kernels followed by 32 5×5 convolutional layers; the pooling layer uses 3×3 kernels followed by 32 1×1 convolutions. After the four branches complete their operations, their outputs are concatenated and input into the second part of the third module. The second part also uses four branches, with a structure similar to the first part, but with different convolution scales: the first branch uses 128 1×1 convolution kernels; the second branch uses 128 1×1 convolution kernels followed by 192 3×3 convolutions; the third branch uses 32 1×1 convolution kernels followed by 96 5×5 convolutions; the fourth branch is a pooling layer using 3×3 kernels followed by 64 1×1 convolutions. The four results are then concatenated and input into the fourth module. The structures of the fourth and fifth modules are similar to the third module, differing only in the number of convolution kernels. The number of convolution kernels for all modules is summarized in the table below. The activation function used in each layer is the ReLU activation function.
[0057] Because the intermediate layers of a neural network also possess strong recognition capabilities, features from two layers are extracted in the fourth single module and output to the model result. After training, as shown... Figure 2 As shown, the models from the three sources are fused to avoid overfitting and improve model accuracy. In terms of deployment, the model is deployed to the first domain controller in the cockpit domain (i.e., domain controller A) for use by applications on the domain controller.
[0058] Modular software architecture, such as Figure 4As shown. On the first domain controller (i.e., domain controller A), SoundCollect is an application layer application responsible for collecting sound information and converting it into a spectrogram. After calculation by a convolutional neural network, the predicted allergen results are transmitted to the second domain controller (i.e., domain controller B) via MQTT communication. In domain controller B, the DeepSeek module organizes allergen-related suggestions and then feeds them back to the rear domain controller via the MQTT protocol for output to passengers.
[0059] The system architecture of the cockpit domain is as follows Figure 5 The two domain controllers communicate via Ethernet and MQTT, using long connections and multiple connections to exchange data.
[0060] Due to its two-chip, four-screen system architecture, this system features high stability and high response speed, reducing the computational burden caused by algorithms and enabling long-term stable operation.
[0061] As a preferred embodiment of the present invention, in practical applications, this technical solution can also have both Linux and Android systems on the domain controller. This architecture is the first to be applied to allergen identification, achieving lightweight deployment.
[0062] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.
[0063] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution device.
[0064] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0065] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.
[0066] In the description of this specification, references to terms such as "an embodiment," "some embodiments," "example," "specific example," or "embodiment," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0067] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
[0068] In this specification, the invention has been described with reference to specific embodiments thereof. However, it will be apparent that various modifications and variations can be made without departing from the spirit and scope of the invention. Therefore, the specification and drawings should be considered illustrative rather than restrictive.
Claims
1. A method for intelligent allergen identification and response based on in-vehicle respiratory sound analysis, characterized in that, The method includes the following steps: (1) Sound acquisition and preprocessing: The original respiratory sound signal of the occupant is acquired by the in-vehicle audio acquisition device within a preset time period, and the original signal is subjected to noise reduction and frame segmentation. (2) Spectrum image generation: Perform time-frequency analysis on the preprocessed sound signal to generate the corresponding sound spectrum image; (3) Allergen classification and identification: The sound spectrum image is input into a pre-trained convolutional neural network model for reinforcement learning training, and the allergen type classification result of the current passenger's sound is output. (4) Intelligent suggestion generation and fusion: Based on the classification results, generate targeted environmental adjustment and health suggestion instructions; wherein, at least some of the suggestion instructions generate natural language suggestion text by calling the large language model service and taking the classification results as input; (5) Cockpit response execution: Parse and execute the generated suggested instructions, control at least one environmental actuator in the vehicle to perform actions, and / or output the suggested text through the vehicle human-machine interface.
2. The intelligent allergen identification and response method based on in-vehicle respiratory sound analysis according to claim 1, characterized in that, The convolutional neural network model described is built on the GoogleNet Inception v1 architecture, and its training process includes: Construct a training dataset containing multiple labeled sample pairs. Each sample pair includes a respiratory sound signal collected under known allergen exposure conditions and its corresponding allergen category label. The allergen category label includes pollen, dust mites, mold, animal dander, and hair. The acquired sound signal is converted into a spectrum image and used as model input; The Inception v1-based convolutional neural network model was trained in a supervised manner using the training dataset described above, and gradient propagation was performed using an auxiliary classifier in the network to optimize the model parameters.
3. The intelligent allergen identification and response method based on in-vehicle respiratory sound analysis according to claim 2, characterized in that, In the training dataset, the respiratory sound signal acquisition time for each sample is 30-90 seconds, and the known allergen exposure environment is a high-concentration environment of a single allergen created through environmental sensor calibration or controlled experiments.
4. The intelligent allergen identification and response method based on in-vehicle respiratory sound analysis according to claim 1, characterized in that, The suggested instructions generated in step (4) include: generating a first type of basic control instructions based on the mapping relationship between the classification results and preset rules; and combining the classification results and / or occupant status information into a query statement, sending it to the large language model server, and receiving the returned natural language suggested text as a second type of enhanced suggestion.
5. The intelligent allergen identification and response method based on in-vehicle respiratory sound analysis according to claim 1 or 4, characterized in that, Step (5) involves controlling the actuators in the vehicle interior to perform actions including at least one of the following: controlling the air purification system to switch to the high-efficiency filtration mode corresponding to the allergen, controlling the air conditioning system to switch between internal and external circulation, controlling the opening and closing of windows or sunroof, and adjusting the seat ventilation intensity.
6. An intelligent allergen identification and response system based on in-vehicle respiratory sound analysis for implementing the method of any one of claims 1 to 5, characterized in that, The system adopts a distributed cockpit domain controller architecture, including: The rear processing domain includes a first domain controller, a rear audio acquisition device connected to the first domain controller, and a rear display device. The front control domain includes a second domain controller, a front human-machine interface device connected to the second domain controller, and an environmental actuator interface. The first domain controller is used to receive and process the sound signals collected by the rear audio acquisition device, run the pre-trained convolutional neural network model to complete the classification and recognition, and send the classification results through the vehicle communication network. The second domain controller is used to receive the classification results, generate and execute suggestion instructions, control the environmental actuator interface, and drive the front-row human-machine interaction device to output information.
7. The intelligent allergen identification and response system based on in-vehicle respiratory sound analysis according to claim 6, characterized in that, The first domain controller and the second domain controller communicate via an in-vehicle Ethernet based on the MQTT protocol; the pre-trained convolutional neural network model is deployed locally on the first domain controller.
8. The intelligent allergen identification and response system based on in-vehicle respiratory sound analysis according to claim 6, characterized in that, The second domain controller integrates a large language model client module or access interface for interacting with cloud-based or local large language model services to generate the natural language suggestion text.
9. The intelligent allergen identification and response system based on in-vehicle respiratory sound analysis according to claim 6, characterized in that, Both the first domain controller and the second domain controller use the MTK2715 chip; the second domain controller runs the instrument operating system and the in-vehicle infotainment system simultaneously through virtualization technology.
10. The intelligent allergen identification and response system based on in-vehicle respiratory sound analysis according to claim 6, characterized in that, The rear-seat audio acquisition device is a microphone array integrated into the rear seat headrest or B-pillar; the environmental actuator interface is connected to at least one of the body control module, air conditioning control module, and window control module.
11. An intelligent allergen identification and response device based on in-vehicle respiratory sound analysis, characterized in that, The device includes: An audio input interface is used to receive raw respiratory sound data from an in-vehicle microphone; The signal processing unit is used to perform noise reduction and frame preprocessing on the raw respiratory sound data. The spectrum conversion unit is used to convert the preprocessed audio data into a time-spectrum image; The neural network processing unit integrates a convolutional neural network model based on the GoogleNet Inception v1 architecture and pre-trained on an allergen sound spectrum dataset, which is used to receive the time-spectrum image and output the allergen classification result. The result output interface is used to send the classification results to the vehicle bus or a specified domain controller. A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the allergen intelligent identification and response method based on in-vehicle respiratory sound analysis as described in any one of claims 1 to 5.
12. The intelligent allergen identification and response device based on in-vehicle respiratory sound analysis according to claim 11, characterized in that, The convolutional neural network model integrated in the neural network processing unit uses an allergen sound spectrum dataset for training, which includes labeled respiratory sound samples collected in environments with four single allergens: pollen, dust mites, mold, and animal dander and hair.
13. An intelligent allergen identification and response processor based on in-vehicle respiratory sound analysis, characterized in that, The processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the allergen intelligent identification and response method based on in-vehicle respiratory sound analysis as described in any one of claims 1 to 5.
14. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the allergen intelligent identification and response method based on in-vehicle respiratory sound analysis as described in any one of claims 1 to 5.