A hearing aid usage behavior analysis system based on multi-dimensional time sequence features

Abstract

The application relates to the technical field of user behavior analysis, and discloses a hearing aid use behavior analysis system based on multi-dimensional time sequence characteristics, which comprises a time alignment module, parameter channel data, environment channel data and behavior channel data are acquired, the data are sorted according to server time alignment to form multi-channel time sequence data, asynchronous acquisition is continuously represented on the same time axis, a feature statistical module is used for calculating short-term features of statistical quantities in a preset time window, long-term features of wearing time and average hearing aid power are statistically calculated according to natural days, the features are combined to form a multi-dimensional time sequence feature sequence, scene and operation changes are stably described, a channel modeling module is driven by the sequence to form a multi-channel time sequence model, user behavior feature representation and hearing loss trend prediction results are output, the short-term fluctuation and the long-term evolution are connected, a behavior clustering module is used for clustering to form a user behavior portrait according to the representation, and individualized test and fitting suggestions are generated in combination with the hearing loss trend prediction results and the parameter channel data.

Description

Technical Field

[0001] This invention relates to the field of user behavior analysis technology, and more specifically, to a hearing aid usage behavior analysis system based on multidimensional temporal features. Background Technology

[0002] As hearing aids and their associated terminals are increasingly deployed among hearing-impaired users, the collection and analysis of parameter data, environmental data, and user operation data generated during hearing aid use are gradually being used for assisted fitting and follow-up. Under the constraints of limited terminal computing power, storage capacity, and communication bandwidth, existing technologies mostly use daily or per-use statistics to organize information such as wearing time, volume level distribution, number of program switching times, and acoustic environment type. They then use abstract methods such as rule matching, interval determination, and experience configuration to provide simple usage reports or parameter adjustment suggestions. These solutions typically assume that the data source is relatively singular, the time change is relatively gradual, and the differences in usage scenarios are not drastic. They can provide some reference value under relatively stable acoustic environments and relatively fixed wearing habits, but they are difficult to overcome the limitations of continuous modeling of multi-source time-series data.

[0003] In real-world usage scenarios, the acoustic environment in which hearing aid wearers are located changes frequently over different time periods. Users' actions regarding volume, program settings, and wearing methods also exhibit unstable temporal evolution based on daily activities and subjective auditory perception. These objective factors directly related to the scenario collectively influence existing methods based on simple statistics and rule matching, making it difficult to form a coherent behavioral representation of data generated by the same user at different time periods on a unified timeline. This makes it challenging to reflect subtle changes in hearing status over time. Therefore, the technical problem that needs to be solved is how to achieve unified modeling of hearing aid-related data and obtain reliable representations that can be used to identify user behavior patterns and predict hearing change trends, given the presence of multi-source heterogeneous temporal data and dynamic changes in the acoustic environment and user behavior over time.

[0004] In view of this, the present invention proposes a hearing aid usage behavior analysis system based on multidimensional temporal features to solve the above problems. Summary of the Invention

[0005] To overcome the above-mentioned deficiencies of the prior art, the present invention provides a hearing aid usage behavior analysis system based on multi-dimensional temporal features.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] Firstly, a hearing aid usage behavior analysis system based on multi-dimensional temporal features is provided, including:

[0008] Time alignment module: used to acquire parameter channel data, environmental channel data and behavioral channel data of hearing aid, and align and sort the parameter channel data, environmental channel data and behavioral channel data according to the server time to obtain multi-channel time sequence data;

[0009] Feature statistics module: It is used to calculate the statistics of each channel on multi-channel time series data according to a preset time window to obtain short-term features, and to obtain long-term features by statistically analyzing wearing time and average hearing aid power according to natural days. The short-term features and long-term features are combined into a multi-dimensional time series feature sequence.

[0010] Channel modeling module: Used to input multi-dimensional time series feature sequences into a multi-channel time series model including parameter channels, environmental channels and behavioral channels, so that the multi-channel time series model outputs user behavior feature representation and hearing loss trend prediction results;

[0011] Behavioral clustering module: This module is used to cluster the user behavior feature representations of multiple users to obtain user behavior profiles, and generate personalized fitting suggestions based on user behavior profiles, hearing loss trend prediction results, and parameter channel data.

[0012] In some embodiments, statistics for each channel are calculated on multi-channel time-series data according to a preset time window to obtain short-term characteristics, including:

[0013] For multi-channel time-series data, a sliding time window with a window length of a preset number of minutes and a step size smaller than the window length is used. For the parameter channel data in each time window, the mean, variance, and the difference between the maximum and minimum values ​​of the gain, compression ratio, and noise suppression coefficient are calculated within that time window.

[0014] For the environmental channel data within each time window, calculate the mean and variance of the sound pressure level within that time window. For the behavioral channel data within each time window, calculate the cumulative values ​​of the number of volume adjustments and program switching times within that time window, as well as the differences between adjacent time windows. Then, concatenate the above statistics in a fixed order to form the short-term characteristics of the corresponding time window.

[0015] In some embodiments, long-term characteristics are obtained by statistically analyzing wearing time and average hearing aid power over calendar days, including:

[0016] Multi-channel time-series data are summarized on a daily basis. The cumulative wearing time of hearing aids within each daily day is calculated, and the average hearing aid power corresponding to the parameter channel data within that daily day is calculated. The cumulative wearing time and average hearing aid power are combined to form the long-term characteristics of that daily day.

[0017] The long-term features of each natural day are associated with the short-term features corresponding to each time window within that natural day on the time index, so that the combined multidimensional time series feature sequence contains both short-term and long-term features.

[0018] In some embodiments, the multi-channel timing model includes:

[0019] Independent long short-term memory (LSTM) coding subnetworks are set up for parameter channel features, environment channel features and behavior channel features respectively. In each LSM coding subnetwork, the multidimensional temporal features of the corresponding channel are input in chronological order and the time step hidden state of the corresponding channel is output. The behavior channel features are obtained based on the behavior channel data.

[0020] At each time step, the hidden states of the time steps output by the three long short-term memory network encoding subnetworks are concatenated and input into the subsequent network layers. The concatenation results of all time steps are transformed and aggregated to obtain the user behavior feature representation. The hearing decline trend prediction result is generated based on the user behavior feature representation in the output layer.

[0021] In some embodiments, the training logic for the multi-channel time-series model to output hearing loss trend prediction results includes:

[0022] For each audiometry record with a listening test time, a multidimensional temporal feature sequence within a preset number of days before the listening test time is selected as the training sample input, and the hearing thresholds at each frequency point in the audiometry record and the hearing change categories of hearing loss, hearing stability or hearing improvement determined based on multiple adjacent audiometry records are used as the training sample output.

[0023] When training the multi-channel time series model, the errors between the hearing thresholds predicted by the output layer at each frequency and the hearing thresholds at each frequency in the audiometry record, as well as the errors between the hearing change categories predicted by the output layer and the hearing change categories, are constrained. This ensures that the trained multi-channel time series model reflects both numerical and categorical changes when providing the hearing decline trend prediction results.

[0024] In some embodiments, clustering the user behavior feature representations of multiple users to obtain user behavior profiles includes:

[0025] The user behavior feature representations of multiple users are input into the clustering algorithm, and the users are divided into multiple cluster groups based on the differences between the user behavior feature representations.

[0026] The clustering result corresponding to each cluster group is defined as a user behavior profile.

[0027] In some embodiments, the tagging of user behavior profiles includes:

[0028] Based on the statistical results of the number of times the user adjusted the volume in a noisy environment, the number of times the user switched programs in a noisy environment, and the wearing time for each user behavior profile;

[0029] User profiles that frequently adjust volume and switch programs in noisy environments are labeled as environment-sensitive user profiles. User profiles that wear the device for a longer period of time and have moderate volume and program switching times are labeled as proactively adaptive user profiles. User profiles that wear the device for a shorter period of time and have fewer volume and program switching times are labeled as passively used user profiles.

[0030] In some embodiments, the hearing loss trend prediction results are also used to mark hearing loss warnings, and the markers for hearing loss warnings include:

[0031] For each user, at least two timestamped hearing test records are obtained. Based on the hearing decline trend prediction results given by the multi-channel time series model within a preset month range before each hearing test record, the time interval in which the predicted hearing threshold change direction within the preset month range is consistent with the actual hearing threshold change direction in each hearing test record is determined.

[0032] Within a time interval, the predicted hearing threshold change is calculated relative to the actual hearing threshold change in the hearing test record. When the time advance is greater than or equal to the preset advance period and the predicted hearing threshold change reaches the preset change range threshold, the prediction result for the corresponding user within the time interval is recorded as a hearing loss warning message, which is used to evaluate the advance prediction capability of the multi-channel time series model on historical data.

[0033] In some embodiments, the multi-channel time series model is also evaluated through multidimensional change correlation verification, which includes:

[0034] For target users with multiple consecutive timestamped hearing test records, the parameter channel data, environmental channel data, behavioral channel data, and hearing threshold curves in each hearing test record are aligned on a unified timeline within a preset verification period. The change trajectories of gain and output power are extracted from the parameter channel data, and the change trajectories of volume adjustment and program switching under noisy conditions are extracted from the behavioral channel data.

[0035] Within a preset verification period, the relationship between the hearing loss trend prediction results given by the multi-channel time series model and the actual hearing threshold curve is compared. The time segments corresponding to the rising range of hearing threshold in the gain, output power change trajectory and operation change trajectory under noise environment are identified, and these time segments are recorded as multi-dimensional change records for the target user.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0037] In this invention, the time alignment module acquires parameter channel data, environmental channel data, and behavioral channel data, and sorts them into multi-channel time-series data according to server time alignment, enabling data from different sources and rhythms to be continuously represented on the same time axis. The feature statistics module calculates statistics on the multi-channel time-series data according to a preset time window to obtain short-term features, and calculates wearing time and average hearing aid power according to natural days to obtain long-term features. The two are combined into a multi-dimensional time-series feature sequence, so that scene changes and operational behaviors are presented with stable feature carriers. The channel modeling module inputs the multi-dimensional time-series feature sequence into the multi-channel time-series model, and directly outputs user behavior feature representations and hearing loss trend prediction results, thereby connecting short-term fluctuations and long-term evolution in time. The behavior clustering module clusters the user behavior feature representations of multiple users to obtain user behavior profiles, and generates personalized fitting suggestions based on user behavior profiles, hearing loss trend prediction results, and parameter channel data. Thus, without changing the data acquisition side, it achieves unified modeling of hearing aid-related data, reliable identification of usage behavior patterns, and forward-looking prediction of hearing change trends. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the structure of a hearing aid usage behavior analysis system based on multidimensional temporal features in this invention;

[0039] Figure 2 This is a flowchart illustrating a hearing aid usage behavior analysis method based on multidimensional temporal features in this invention.

[0040] Figure 3 This is a schematic diagram of the multidimensional temporal features in this invention;

[0041] Figure 4 This is a schematic diagram illustrating the prediction of hearing loss trends in this invention;

[0042] Figure 5 This is a schematic diagram of the relationship between hearing aid output and frequency and parameter configuration obtained at the first time point in this invention.

[0043] Figure 6 This is a schematic diagram of the relationship between hearing aid output and frequency and parameter configuration obtained at the second time point in this invention.

[0044] Figure 7 This is a schematic diagram of the relationship between hearing aid output and frequency and parameter configuration obtained at the third time point in this invention.

[0045] Figure 8 This is a schematic diagram of the hearing test record at the first hearing test time point in this invention;

[0046] Figure 9This is a schematic diagram of the audiometry record at the second audiometry time point in this invention;

[0047] Figure 10 This is a schematic diagram of the hearing test record at the third hearing test time point in this invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains. The terms “first,” “second,” and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as “comprising” or “including” mean that the element or object preceding the term covers the element or object listed after the term and its equivalents, without excluding other elements or objects. Terms such as “connection” or “linked” are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0050] In the technical solution of this invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data are all carried out in accordance with relevant laws, regulations, and standards, and necessary confidentiality measures have been taken. They do not violate public order and good morals, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0051] Example 1

[0052] Please see Figure 1 As shown, this embodiment discloses a hearing aid usage behavior analysis system based on multi-dimensional temporal features, characterized in that it includes:

[0053] Time alignment module: used to acquire parameter channel data, environmental channel data and behavioral channel data of hearing aid, and align and sort the parameter channel data, environmental channel data and behavioral channel data according to the server time to obtain multi-channel time sequence data;

[0054] In this embodiment, parameter channel data can be understood as the amplification configuration and processing coefficient records of the hearing aid at different frequency points, including values ​​such as gain, compression ratio, and noise suppression coefficient at each frequency point. Environmental channel data can be understood as the sound pressure level detected by the hearing aid during use and the environmental category code given by the environmental recognition module, such as quiet, noisy, or speech plus noise. Behavioral channel data can be understood as records of user interaction operations during use, such as volume adjustment operations, program switching operations, and the derived wear duration increment. These three types of data are recorded separately by the hearing aid terminal and mobile terminal in the actual system, each with a timestamp. By aligning and sorting using the server time as a unified reference, the three types of records, originally stored separately, can be organized into multi-channel time-series data on the same time axis. Figure 3 As shown, the multi-channel time series data simultaneously presents the trajectory of parameter changes, changes in ambient sound intensity, and changes in user operation frequency on the time axis, providing a foundation for subsequent window statistics and time series modeling on a unified time series.

[0055] It is understandable that the process of aligning and sorting parameter channel data, environmental channel data, and behavioral channel data using server time is mainly to solve the time offset problem caused by inconsistent local time of the terminal and uncertain data reporting delay. In practical application scenarios, the hearing aid itself may record parameter updates at the millisecond level, the mobile terminal may record user operations at the second level, and the cloud may record environmental recognition results at the receiving time. If each local time is used directly, the same auditory event will be split into different time points. By mapping all records to a unified server time axis and sorting them according to the time axis, it can be ensured that there is a clear correspondence between parameter channel data, environmental channel data, and behavioral channel data at any time position. This allows those skilled in the art to construct a sliding time window based on the time index in subsequent steps, thereby stably extracting short-term and long-term features.

[0056] Feature statistics module: It is used to calculate the statistics of each channel on multi-channel time series data according to a preset time window to obtain short-term features, and to obtain long-term features by statistically analyzing wearing time and average hearing aid power according to natural days. The short-term features and long-term features are combined into a multi-dimensional time series feature sequence.

[0057] In this embodiment, short-term features can be understood as statistical results describing the fluctuations in hearing aid usage status at a smaller time scale, including changes in parameter channel data, environmental channel data, and behavioral channel data within a continuous window. Long-term features can be understood as statistical information reflecting the user's overall usage pattern at a natural day scale, including values ​​such as wearing time and average hearing aid power. By combining short-term and long-term features, a multi-dimensional time-series feature sequence covering multiple time levels can be formed, enabling subsequent time-series models to focus on both local change trends and capture more stable daily usage patterns.

[0058] Understandably, the processing of multi-channel time series data, which calculates statistics according to a preset time window, divides the continuous time series into multiple local segments based on a sliding method of window length and step size. Each segment corresponds to a short-term feature, for example, in... Figure 3 In the time axis structure shown, the changes in gain ("↑" and "↓"), the changes in ambient sound pressure level ("high", "medium", and "low"), and the changes in operating frequency ("stable → dense → stable") in the behavior channel can all be reflected by statistical measures such as the mean, variance, and the difference between the maximum and minimum values ​​within the window. For example, when the window contains... Figure 3 When a "intensive operation" section is in progress, the statistical data of the behavior channel within this window will be represented as program switching record C. t and adjustment operation A t The cumulative amount increases, while when the window is in Figure 3 In the "steady-state region", the change amplitude of the corresponding statistics in the short-term characteristics will be significantly reduced. Such short-term characteristics can characterize the actual fluctuation of local behavior.

[0059] It should be noted that the long-term characteristics of wearing time and average hearing aid power calculated on a daily basis summarize the overall patterns of usage behavior over a larger time scale. For example, within a whole day, the longer the wearing time, the stronger the user's dependence on the device, and the higher the average hearing aid power may reflect a higher overall sound pressure level in the environment on that day. By summarizing and calculating on a daily basis, the interference of factors such as instantaneous noise and frequent changes in short-term operation can be reduced, so that the obtained long-term characteristics can serve as a stable supplement to the short-term characteristics, and more representative user daily usage patterns can be obtained in subsequent processing.

[0060] Statistics for each channel are calculated within a preset time window on multi-channel time series data to obtain short-term characteristics, including:

[0061] A sliding time window with a preset window length of minutes and a step size smaller than the window length is used for multi-channel time-series data. For the parameter channel data within each time window, the mean, variance, and difference between the maximum and minimum values ​​of gain, compression ratio, and noise suppression coefficient are calculated. For the environmental channel data within each time window, the mean and variance of sound pressure level are calculated. For the behavioral channel data within each time window, the cumulative values ​​of volume adjustment times and program switching times, as well as the differences between adjacent time windows, are calculated. The above statistics are then spliced ​​together in a fixed order to form the short-term characteristics of the corresponding time window.

[0062] In this embodiment, calculating the statistics of each channel on the multi-channel time-series data according to a preset time window can be understood as setting a fixed-length time segment on the time axis that can slide. Within each time segment, the parameter channel data, environmental channel data, and behavioral channel data are locally statistically summarized to obtain short-term feature vectors that can reflect short-term fluctuations in hearing aid parameters, changes in external sound pressure levels, and changes in user operation frequency. The preset window length and the step size smaller than the window length together determine the temporal resolution of the short-term features. For example, in actual deployment, a ten-minute time window and a five-minute step size can be used, which means that every five minutes, an overlapping ten-minute interval is slid forward, and statistical calculations are performed on each interval. This setting avoids the problem of statistical instability caused by an excessively short window and ensures that enough short-term feature points are obtained within a day, so that the subsequent time-series model can continuously observe the smooth evolution of hearing aid usage behavior over time.

[0063] In this embodiment, to ensure the comparability of parameter channel data, environmental channel data, and behavioral channel data with different dimensions within the same model, the original time-series data can first be standardized. The standardization formula can be expressed as:

[0064] ;

[0065] in, Indicates time index A certain standardized feature at a location, Indicates time index A certain original feature of the place, This represents the average value of the feature over a selected time range. This represents the standard deviation of the feature over the same time period.

[0066] In this embodiment, when constructing short-term features within a sliding time window, the mean and variance can be used to jointly describe the local statistical features within the window, with the window length as... Taking the time window as an example, the short-term statistical feature vector can be defined as:

[0067] ;

[0068] in, For time index The short-term eigenvector at a given point represents a combination of the local mean and local fluctuations of a feature within the sliding window. The length of the sliding window. For time index A certain original feature value at a certain location, This represents the average value of this feature within the window. This represents the average value of this feature within the window. This represents the standard deviation of the feature within the window.

[0069] You can also define the change between adjacent time steps. This change can reflect, within the window overlap interval, as... Figure 3 The direction and magnitude of the numerical changes corresponding to the "↑" and "↓" trajectories of the intermediate parameters are used to transform the symbolic changes in the figure into learnable numerical features.

[0070] Long-term characteristics were obtained by statistically analyzing wearing time and average hearing aid power over calendar days, including:

[0071] Multi-channel time-series data are summarized on a daily basis. The cumulative wearing time of hearing aids within each daily day is calculated, and the average hearing aid power corresponding to the parameter channel data within that daily day is calculated. The cumulative wearing time and average hearing aid power are combined to form the long-term characteristics of that daily day.

[0072] The long-term features of each natural day are associated with the short-term features corresponding to each time window within that natural day on the time index, so that the combined multidimensional time series feature sequence contains both short-term and long-term features.

[0073] In this embodiment, long-term characteristics can be understood as a statistical summary of the overall usage of hearing aids on a daily timescale, used to reflect the user's wearing habits and acoustic exposure level within a day. The cumulative wearing time is used to characterize the total time the user actually wears the hearing aid within that daily time, and the average hearing aid power is used to characterize the overall energy level output by the hearing aid in different acoustic environments within that daily time. The long-term characteristics formed by combining the cumulative wearing time and the average hearing aid power can serve as a stable supplement to the short-term characteristics, enabling those skilled in the art to analyze the user's usage patterns from a multi-scale time perspective.

[0074] In this embodiment, when constructing features reflecting long-term stability, the repeatability of a feature on a daily or weekly cycle can be measured using an autocorrelation function. The autocorrelation function can be expressed as:

[0075] ;

[0076] in, The time lag is The autocorrelation value of this feature at that time. This is the time lag. Time Index The original feature values ​​at time 1. This represents the cumulative value of the similarity between the features at the current time point and the lag time point. This is the sum of squares of the deviations of the feature over a selected time range.

[0077] In this embodiment, when characterizing the environment distribution and interaction intensity, contextual features can be introduced by the distribution probability of environment categories and the operation density of behavioral channel data. The distribution of environment categories can be represented as follows:

[0078] ;

[0079] in, For environmental category The probability of its occurrence, For time index The environment category code at any given time, For a specific environmental category, For the indicator function, when the time index The environmental category is equal to The value is 1 if the condition is met, and 0 otherwise. This represents the total number of time steps within the statistical period. For environmental categories equal to within the statistical period The total number of times.

[0080] Interaction density can be expressed as:

[0081] ;

[0082] in, For interaction density, This represents the total number of time steps within the statistical period. For time index The number of interactive operation records in the time behavior channel.

[0083] The aforementioned autocorrelation features, environmental distribution features, and interaction density features, together with short-term and long-term features, constitute a multidimensional time-series feature sequence, enabling the model to simultaneously learn local changes, daily patterns, and contextual factors such as environmental exposure and user interaction intensity.

[0084] Channel modeling module: Used to input multi-dimensional time series feature sequences into a multi-channel time series model including parameter channels, environmental channels and behavioral channels, so that the multi-channel time series model outputs user behavior feature representation and hearing loss trend prediction results;

[0085] In this embodiment, the multi-channel temporal model can be understood as a type of sequence model that splits the input by channel and models them separately. The parameter channel, environment channel, and behavior channel correspond to the parts of the multi-dimensional temporal feature sequence that are related to the hearing aid output configuration, acoustic environment state, and user interaction operation, respectively. By configuring their respective encoding paths for different channels within the same model, the rhythm of parameter change, environment change, and behavior change can be extracted in the time dimension. These rhythms are then fused into a user behavior feature representation in the middle layer of the model to characterize the user's comprehensive usage pattern over a period of time. At the same time, the model output layer provides a hearing loss trend prediction result to characterize the direction and magnitude of change of hearing thresholds at each frequency point over a future period of time.

[0086] It is understandable that inputting multidimensional temporal feature sequences into a multi-channel time-series model, which includes parameter channels, environment channels, and behavioral channels, is essentially a process of... Figure 3 The three curves showing the evolution of parameters, ambient sound intensity, and operation frequency over time are mapped to a unified high-dimensional time series. Within the model, the parameter channel focuses on capturing the smooth adjustment and abrupt changes of values ​​such as gain, compression ratio, and noise suppression coefficient under different sound fields and usage stages. The environment channel focuses on characterizing the periodic changes of sound pressure level and environmental category within a day or multiple days. The behavior channel focuses on identifying the scenarios in which users frequently adjust or switch programs. After the three channels are aligned in time, they are fed into the same temporal model, which can form a user behavior feature representation that simultaneously contains parameter, environmental, and behavioral information in the hidden layer. Then, the hearing loss trend prediction result is inferred from this representation at the output layer.

[0087] Multi-channel time series models, including parameter channels, environment channels, and behavior channels, include:

[0088] Independent long short-term memory (LSTM) coding subnetworks are set up for parameter channel features, environment channel features and behavior channel features respectively. In each LSM coding subnetwork, the multidimensional temporal features of the corresponding channel are input in chronological order and the time step hidden state of the corresponding channel is output. The behavior channel features are obtained based on the behavior channel data.

[0089] At each time step, the hidden states of the time steps output by the three long short-term memory network encoding subnetworks are concatenated and input into the subsequent network layers. The concatenation results of all time steps are transformed and aggregated to obtain the user behavior feature representation. The hearing decline trend prediction result is generated based on the user behavior feature representation in the output layer.

[0090] In this embodiment, the parameter channel features, environmental channel features, and behavioral channel features in the multi-channel time-series model can be understood as multi-dimensional time series extracted through short-term and long-term features. The parameter channel features correspond to the temporal arrangement of values ​​such as gain, compression ratio, and noise suppression coefficient within different time windows; the environmental channel features correspond to the temporal arrangement of sound pressure level and environmental category changes over time; and the behavioral channel features are the temporal arrangement obtained by statistically analyzing behavioral channel data such as volume adjustment times and program switching times. In terms of model structure, independent long short-term memory (LSTM) network encoding subnetworks are set up for each of these three types of features, which can be used in time... In terms of dimensions, different channels learn their own change patterns. For example, the parameter channel focuses on learning the slow rise or sudden drop of gain when switching between quiet and noisy scenes, the environment channel focuses on learning the periodic changes of sound pressure level during day and night, and weekdays and rest days, and the behavior channel focuses on learning the user's frequent adjustment or stable operating habits in certain scenarios. By inputting the multi-dimensional temporal features of the corresponding channel in chronological order into each coding sub-network and outputting the time step hidden state, a compressed representation of the current state of the channel and its historical dependencies can be formed at each time position, thus laying the foundation for subsequent channel fusion.

[0091] It is understandable that concatenating the time-step hidden states of the outputs of the three long short-term memory network coding subnetworks at each time step and inputting them into the processing actions of subsequent network layers is to jointly represent the temporal coding results of the parameter channel, environment channel, and behavior channel at the same time. In this structure, the concatenation result corresponding to each time step simultaneously contains comprehensive information about the hearing aid output configuration, the acoustic environment, and the user's operation behavior at that time. Subsequent network layers can use one or more nonlinear transformations to further map and compress these concatenation results, and then transform and aggregate the outputs of all time steps to form a user behavior feature representation that describes the overall usage behavior over a period of time. Based on this, the output layer generates a hearing loss trend prediction result according to the user behavior feature representation, so that the future change of hearing threshold does not depend on a single wearing time or single operation statistics, but is jointly determined by the parameter change trajectory, environmental exposure trajectory, and behavior operation trajectory.

[0092] The training logic for the multi-channel time-series model to output hearing loss trend prediction results includes:

[0093] For each audiometry record with a listening test time, a multidimensional temporal feature sequence within a preset number of days before the listening test time is selected as the training sample input, and the hearing thresholds at each frequency point in the audiometry record and the hearing change categories of hearing loss, hearing stability or hearing improvement determined based on multiple adjacent audiometry records are used as the training sample output.

[0094] When training the multi-channel time series model, the errors between the hearing thresholds predicted by the output layer at each frequency and the hearing thresholds at each frequency in the audiometry record, as well as the errors between the hearing change categories predicted by the output layer and the hearing change categories, are constrained. This ensures that the trained multi-channel time series model reflects both numerical and categorical changes when providing the hearing decline trend prediction results.

[0095] In this embodiment, the training logic for the multi-channel temporal model to output the hearing loss trend prediction result can be understood as establishing a correspondence between multi-dimensional temporal features and hearing test records, given real audiometry results. At each audiometry record with a test time, the multi-dimensional temporal feature sequence within a consecutive number of days prior is used as a training sample input, and the hearing threshold at each frequency point in the audiometry record is used as the numerical output. At the same time, the hearing change category of hearing loss, hearing stability, or hearing improvement is marked according to the direction of change of hearing threshold at each frequency point between adjacent audiometry records, and is used as the category output of the same training sample. In this way, a pairwise relationship can be established between "recent usage behavior and acoustic exposure" and "current hearing status" in time.

[0096] Understandably, when constructing training samples, for example, if a user completes multiple hearing tests within three months, then for each hearing test time point, a multi-dimensional temporal feature sequence within a preset number of days can be selected as input. This input covers the change trajectory of parameter channel data, the sound pressure level and environmental category changes of environmental channel data, and the adjustment and switching of behavioral channel data. The output directly uses the hearing threshold curves of each frequency point in the hearing test record, and combines the results of two or more previous hearing tests to determine whether the mid-high frequency or low-mid frequency thresholds are rising overall, remaining basically flat, or declining, and thus label them as hearing loss, hearing stability, or hearing improvement categories, respectively. In this way, a training sample simultaneously contains fine frequency threshold information for numerical prediction and discrete category information for trend judgment.

[0097] In this embodiment, when training a multi-channel time series model for numerical prediction, the user behavior feature representation sequence within a preset number of days can be denoted as (Ht−T+1,…,Ht), where T is the selected time step. The regression function... This yields the predicted hearing threshold values ​​for each frequency point after N time steps in the future. ,in, The results of hearing threshold prediction at each frequency point at time index t+N are as follows. Let T be a continuous sequence of user behavior features from time index t−T+1 to t, where T is the length of the time step window used for prediction. This is the mapping function for the multi-channel timing model.

[0098] During training, mean squared error can be used as the numerical prediction loss, and the loss function is expressed as:

[0099] ;

[0100] To regress the loss, The number of training samples. For the first The actual hearing threshold of each sample For the first The predicted hearing threshold for each sample.

[0101] In this embodiment, when performing classification training on hearing change categories, user behavior features at a certain time step can be represented. Based on this, the hearing change category prediction vector is obtained through linear transformation and normalization function as follows:

[0102] ;

[0103] Let be the probability vector of the predicted hearing change category. The weight matrix of the classification layer. For the bias term of the classification layer, For time index User behavior characteristics at any given moment For normalization function, This is the result of a linear transformation of the classification layer.

[0104] Behavioral clustering module: This module is used to cluster the user behavior feature representations of multiple users to obtain user behavior profiles, and generate personalized fitting suggestions based on user behavior profiles, hearing loss trend prediction results, and parameter channel data.

[0105] Clustering user behavior feature representations from multiple users to obtain user behavior profiles includes:

[0106] The user behavior feature representations of multiple users are input into the clustering algorithm, and the users are divided into multiple cluster groups based on the differences between the user behavior feature representations.

[0107] The clustering result corresponding to each cluster group is defined as a user behavior profile.

[0108] In this embodiment, clustering the user behavior feature representations of multiple users can be understood as follows: after obtaining the user behavior feature representations of multiple users, by comparing the differences between these feature representations in the vector space, users with similar features are automatically divided into several groups. Each group represents a set of users who exhibit similar behavioral patterns during hearing aid use. Figure 3 The parameter trajectory changes, ambient sound intensity changes, and operation frequency changes shown in the diagram can be represented in the user behavior feature representation. For example, users with more "stable → dense → stable" structures may have more frequent operation patterns, while users with more "high, medium, low" sound pressure level changes may be more exposed to noisy environments. By integrating these representations reflecting the dynamic characteristics of parameters, environment, and behavior into the clustering algorithm, each cluster can naturally form a user group with consistent behavioral characteristics, and each such group can be defined as a user behavior profile.

[0109] Understandably, inputting user behavior feature representations from multiple users into a clustering algorithm and dividing them into multiple clusters based on their differences serves two purposes. Firstly, it aims to characterize the statistical patterns of user behavior at a higher level. For example, some users may exhibit short-term characteristics within a day, such as... Figure 3 The multiple fluctuations in the data, along with the high average hearing aid power in its long-term characteristics, suggest that users' behavioral characteristics may be close to each other in the cluster space. On the other hand, this also serves to provide a "group reference baseline" structure for the model when generating personalized fitting recommendations in the future, making the user's position in the overall usage behavior clearer. For example, when a user's behavioral characteristics are highly similar to a certain cluster group, fitting recommendations that are more consistent with their behavioral characteristics can be generated based on the typical parameter channel change patterns and environmental exposure patterns of that cluster group. The user behavior profile obtained in this way enables more refined and targeted personalized processing, which helps to steadily improve the actual user experience of hearing aids.

[0110] The tags for user behavior profiles include:

[0111] Based on the statistical results of the number of times the user adjusted the volume, the number of times the user switched programs, and the wearing time in a noisy environment for each user behavior profile, user behavior profiles with a high number of volume adjustments and program switching in noisy environments are marked as environmentally sensitive user behavior profiles. User behavior profiles with a long wearing time and a moderate number of volume adjustments and program switching are marked as proactively adaptive user behavior profiles. User behavior profiles with a short wearing time and a low number of volume adjustments and program switching are marked as passively used user behavior profiles.

[0112] In this embodiment, the labeling of user behavior profiles can be understood as follows: after obtaining multiple user behavior profiles, based on three statistical indicators—the number of times volume is adjusted in a noisy environment, the number of times the user switches programs in a noisy environment, and the wearing time—each user behavior profile is assigned a semantic name corresponding to its behavioral characteristics. The number of times volume is adjusted in a noisy environment reflects the frequency with which the user adjusts the hearing aid's compensation level in a complex sound field; the number of times the user switches programs in a noisy environment reflects the degree to which the user actively switches listening modes between different scenarios; and the wearing time reflects the user's overall dependence on the hearing aid. When the number of times volume is adjusted and the number of times the user switches programs in a noisy environment are both at a high level, it indicates that this type of user is more inclined to actively intervene with the hearing aid in noisy or speech-plus-noise scenarios. Based on the user's activity status, they can be categorized as environmentally sensitive user profiles. When another user profile corresponds to a user group with a longer wearing time and a moderate number of volume adjustments and program switching times in noisy environments, it indicates that this type of user can appropriately adjust parameters according to changes in the acoustic environment, and they can be categorized as actively adaptive user profiles. When yet another user profile corresponds to a user group with a shorter wearing time and fewer volume adjustments and program switching times in noisy environments, it indicates that this type of user usually maintains the default program and rarely operates actively, and they can be categorized as passively using user profiles. Through the above categorization process, each user profile can obtain a clear description of its usage pattern, providing a clear profile basis for subsequently combining hearing loss trend prediction results and parameter channel data to generate targeted personalized fitting recommendations.

[0113] The results of the hearing loss trend prediction are also used to mark early warning signs of hearing loss, which include:

[0114] For each user, at least two timestamped hearing test records are obtained. Based on the hearing decline trend prediction results given by the multi-channel time series model within a preset month range before each hearing test record, the time interval in which the predicted hearing threshold change direction within the preset month range is consistent with the actual hearing threshold change direction in each hearing test record is determined.

[0115] Within a time interval, the predicted hearing threshold change is calculated relative to the actual hearing threshold change in the hearing test record. When the time advance is greater than or equal to the preset advance period and the predicted hearing threshold change reaches the preset change range threshold, the prediction result for the corresponding user within the time interval is recorded as a hearing loss warning message, which is used to evaluate the advance prediction capability of the multi-channel time series model on historical data.

[0116] In this embodiment, when using the hearing loss trend prediction results for early warning marking, multiple timestamped hearing test records of the same user can be aligned with the prediction curve given by the model onto a unified time axis, such as... Figure 4 As shown, there is a temporal relationship between the prediction curve and the hearing threshold curve obtained from three hearing tests. In practical use, prediction results from several months prior to each hearing test can be selected to determine whether the prediction curve has shown a continuous upward trend in the mid-high frequency or low-mid frequency bands. This can be combined with... Figures 8 to 10 The audiograms shown at different times determine whether the actual hearing threshold also shows a change from more than 30 decibels to more than 40 decibels in the corresponding frequency band. When the predicted trend is consistent with the actual audio measurement change, the time period can be determined as a candidate warning time interval. Within the candidate warning time interval, the predicted time advance of the hearing threshold change relative to the actual change is calculated and compared with the preset advance period and change magnitude threshold. When both the advance and magnitude conditions are met, the prediction result of that time period is recorded as hearing loss warning information to evaluate the model's ability to predict in advance on real historical data.

[0117] The multi-channel time series model is also evaluated through multidimensional change correlation verification, which includes:

[0118] For target users with multiple consecutive timestamped hearing test records, the parameter channel data, environmental channel data, behavioral channel data, and hearing threshold curves in each hearing test record are aligned on a unified timeline within a preset verification period. The change trajectories of gain and output power are extracted from the parameter channel data, and the change trajectories of volume adjustment and program switching under noisy conditions are extracted from the behavioral channel data.

[0119] Within a preset verification period, the relationship between the hearing loss trend prediction results given by the multi-channel time series model and the actual hearing threshold curve is compared. The time segments corresponding to the rising range of hearing threshold in the gain, output power change trajectory and operation change trajectory under noise environment are identified, and these time segments are recorded as multi-dimensional change records for the target user.

[0120] In this embodiment, when performing multidimensional change correlation verification, for target users with three consecutive months of hearing test records and usage data, the parameter channel data, environmental channel data, behavioral channel data, and hearing threshold curves obtained from each hearing test within a preset verification period can be mapped onto a unified time axis, such as... Figures 5 to 7 As shown, the high-frequency gain curves and output power distributions on different dates exhibit a gradually upward trend, consistent with... Figures 8 to 10The hearing threshold curve shown corresponds to a gradual increase from approximately 35.8 dB to over 40 dB or even higher in the mid-to-high frequency band. During the verification process, the time-varying trajectories of high-frequency gain and output power can be extracted from the parameter channel data, and the volume adjustment and program switching trajectories under noise cancellation environment can be extracted from the behavior channel data. These are then compared with the time interval of the rising hearing threshold curve. When an increase in hearing threshold, an increase in high-frequency gain and output power, and an increase in operation frequency under noisy environment are observed simultaneously within a certain time segment, this time segment can be recorded as a multidimensional change record for the target user. Through the analysis of these multidimensional change records, the correspondence between the model prediction results and the actual parameter adjustments and environmental exposure can be verified at the individual level, further demonstrating that the multi-channel time series model has good interpretability and application value in real-world usage scenarios.

[0121] Example 2

[0122] Please see Figure 2 As shown, based on the same inventive concept, this embodiment discloses a method for analyzing hearing aid usage behavior based on multidimensional temporal features. For details not covered in this embodiment, please refer to the relevant sections of Embodiment 1. The method includes:

[0123] S10: Acquire the parameter channel data, environmental channel data, and behavioral channel data of the hearing aid, and align and sort the parameter channel data, environmental channel data, and behavioral channel data according to the server time to obtain multi-channel time-series data;

[0124] S20: Calculate the statistics of each channel on the multi-channel time-series data according to the preset time window to obtain short-term features, and obtain long-term features by statistically analyzing the wearing time and average hearing aid power according to the natural day. Combine the short-term features and long-term features into a multi-dimensional time-series feature sequence.

[0125] S30: Input the multi-dimensional time-series feature sequence into a multi-channel time-series model that includes parameter channels, environmental channels, and behavioral channels, so that the multi-channel time-series model outputs user behavior feature representation and hearing loss trend prediction results;

[0126] S40: Cluster the user behavior feature representations of multiple users to obtain user behavior profiles, and generate personalized fitting suggestions based on user behavior profiles, hearing loss trend prediction results and parameter channel data.

[0127] The accompanying drawings of the embodiments of this invention only involve the structures involved in the embodiments of this invention. Other structures can refer to the general design. In the absence of conflict, the features of the same embodiment and different embodiments of this invention can be combined with each other. The above are only specific implementations of this invention, but the protection scope of this invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this invention should be included within the protection scope of this invention. Therefore, the protection scope of this invention should be determined by the protection scope of the claims.

Claims

1. A hearing aid usage behavior analysis system based on multidimensional temporal features, characterized in that, include: Time alignment module: used to acquire parameter channel data, environmental channel data and behavioral channel data of hearing aid, and align and sort the parameter channel data, environmental channel data and behavioral channel data according to the server time to obtain multi-channel time sequence data; Feature statistics module: It is used to calculate the statistics of each channel on multi-channel time series data according to a preset time window to obtain short-term features, and to obtain long-term features by statistically analyzing wearing time and average hearing aid power according to natural days. The short-term features and long-term features are combined into a multi-dimensional time series feature sequence. Channel modeling module: Used to input multi-dimensional time series feature sequences into a multi-channel time series model including parameter channels, environmental channels and behavioral channels, so that the multi-channel time series model outputs user behavior feature representation and hearing loss trend prediction results; Behavioral clustering module: This module is used to cluster the user behavior feature representations of multiple users to obtain user behavior profiles, and generate personalized fitting suggestions based on user behavior profiles, hearing decline trend prediction results, and parameter channel data. Multi-channel timing models include: Independent long short-term memory (LSTM) coding subnetworks are set up for parameter channel features, environment channel features and behavior channel features respectively. In each LSM coding subnetwork, the multidimensional temporal features of the corresponding channel are input in chronological order and the time step hidden state of the corresponding channel is output. The behavior channel features are obtained based on the behavior channel data. At each time step, the hidden states of the time step outputs of the three long short-term memory network encoding subnetworks are concatenated and input into the subsequent network layer. The concatenation results of all time steps are transformed and aggregated to obtain the user behavior feature representation. The hearing decline trend prediction result is generated based on the user behavior feature representation in the output layer. The training logic for the multi-channel time-series model to output hearing loss trend prediction results includes: For each audiometry record with a listening test time, a multidimensional temporal feature sequence within a preset number of days before the listening test time is selected as the training sample input, and the hearing thresholds at each frequency point in the audiometry record and the hearing change categories of hearing loss, hearing stability or hearing improvement determined based on multiple adjacent audiometry records are used as the training sample output. When training the multi-channel time series model, the errors between the hearing thresholds predicted by the output layer at each frequency and the hearing thresholds at each frequency in the audiometry record, as well as the errors between the hearing change categories predicted by the output layer and the hearing change categories, are constrained. This ensures that the trained multi-channel time series model reflects both numerical and categorical changes when providing the hearing decline trend prediction results.

2. The hearing aid usage behavior analysis system based on multidimensional temporal features according to claim 1, characterized in that, Statistics for each channel are calculated within a preset time window on multi-channel time series data to obtain short-term characteristics, including: For multi-channel time-series data, a sliding time window with a window length of a preset number of minutes and a step size smaller than the window length is used. For the parameter channel data in each time window, the mean, variance, and the difference between the maximum and minimum values ​​of the gain, compression ratio, and noise suppression coefficient are calculated within that time window. For the environmental channel data within each time window, calculate the mean and variance of the sound pressure level within that time window. For the behavioral channel data within each time window, calculate the cumulative values ​​of the number of volume adjustments and program switching times within that time window, as well as the differences between adjacent time windows. Then, concatenate the above statistics in a fixed order to form the short-term characteristics of the corresponding time window.

3. The hearing aid usage behavior analysis system based on multi-dimensional temporal features according to claim 1, characterized in that, Long-term characteristics were obtained by statistically analyzing wearing time and average hearing aid power over calendar days, including: Multi-channel time-series data are summarized on a daily basis. The cumulative wearing time of hearing aids within each daily day is calculated, and the average hearing aid power corresponding to the parameter channel data within that daily day is calculated. The cumulative wearing time and average hearing aid power are combined to form the long-term characteristics of that daily day. The long-term features of each natural day are associated with the short-term features corresponding to each time window within that natural day on the time index, so that the combined multidimensional time series feature sequence contains both short-term and long-term features.

4. The hearing aid usage behavior analysis system based on multi-dimensional temporal features according to claim 1, characterized in that, Clustering user behavior feature representations from multiple users to obtain user behavior profiles includes: The user behavior feature representations of multiple users are input into the clustering algorithm, and the users are divided into multiple cluster groups based on the differences between the user behavior feature representations. The clustering result corresponding to each cluster group is defined as a user behavior profile.

5. The hearing aid usage behavior analysis system based on multidimensional temporal features according to claim 4, characterized in that, The tags for user behavior profiles include: Based on the statistical results of the number of times the user adjusted the volume in a noisy environment, the number of times the user switched programs in a noisy environment, and the wearing time for each user behavior profile; User profiles that frequently adjust volume and switch programs in noisy environments are labeled as environment-sensitive user profiles. User profiles that wear the device for a longer period of time and have moderate volume and program switching times are labeled as proactively adaptive user profiles. User profiles that wear the device for a shorter period of time and have fewer volume and program switching times are labeled as passively used user profiles.

6. The hearing aid usage behavior analysis system based on multi-dimensional temporal features according to claim 1, characterized in that, The results of the hearing loss trend prediction are also used to mark early warning signs of hearing loss, which include: For each user, at least two timestamped hearing test records are obtained. Based on the hearing decline trend prediction results given by the multi-channel time series model within a preset month range before each hearing test record, the time interval in which the predicted hearing threshold change direction within the preset month range is consistent with the actual hearing threshold change direction in each hearing test record is determined. Within a time interval, the predicted hearing threshold change is calculated relative to the actual hearing threshold change in the hearing test record. When the time advance is greater than or equal to the preset advance period and the predicted hearing threshold change reaches the preset change range threshold, the prediction result for the corresponding user within the time interval is recorded as a hearing loss warning message, which is used to evaluate the advance prediction capability of the multi-channel time series model on historical data.

7. A hearing aid usage behavior analysis system based on multidimensional temporal features according to claim 6, characterized in that, The multi-channel time series model is also evaluated through multidimensional change correlation verification, which includes: For target users with multiple consecutive timestamped hearing test records, the parameter channel data, environmental channel data, behavioral channel data, and hearing threshold curves in each hearing test record are aligned on a unified timeline within a preset verification period. The change trajectories of gain and output power are extracted from the parameter channel data, and the change trajectories of volume adjustment and program switching under noisy conditions are extracted from the behavioral channel data. Within a preset verification period, the relationship between the hearing loss trend prediction results given by the multi-channel time series model and the actual hearing threshold curve is compared. The time segments corresponding to the rising range of hearing threshold in the gain, output power change trajectory and operation change trajectory under noise environment are identified, and these time segments are recorded as multi-dimensional change records for the target user.

Images