Device parameter adjustment model training method and device and device parameter adjustment method and device

Abstract

The invention provides a training method of an equipment parameter adjustment model and an equipment parameter adjustment method and device.The method comprises the steps that an initial model is obtained, and initial equipment parameters are determined based on state representation of the initial model; sending the initial equipment parameters to the portable auxiliary equipment to obtain environmental acoustic characteristics and behavior characteristics of the user; inputting the environmental acoustic features and the behavior features into an initial model to obtain action probability distribution of the user; and training the initial model based on the action probability distribution to obtain an equipment parameter adjustment model. In the method, behavior characteristics are used for reflecting personalized habits and preferences of a user, environmental acoustic characteristics are used for reflecting listening environment information of portable auxiliary equipment, and subsequent initial model training is performed based on the environmental acoustic characteristics and the behavior characteristics. The equipment parameter adjusting model obtained through training can adjust the equipment parameters of the portable auxiliary equipment in a personalized mode, the listening effect of a user is improved, and then the satisfaction degree and the use experience of the user are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of portable auxiliary equipment, and in particular to a training method for an equipment parameter adjustment model, and an equipment parameter adjustment method and device. Background Art

[0002] At present, the field of portable assistive devices has ushered in unprecedented development opportunities. Among them, assistive devices for people with hearing impairments, such as hearing aids, have become important auxiliary tools for many people with hearing impairments.

[0003] However, traditional hearing aids usually require manual adjustment of parameters to adapt to the user's hearing condition. The parameters of hearing aids adjusted at the fitting center are set to meet the listening environment of the fitting center. But in fact, the listening environment of the wearer in daily life is not exactly the same as that of the fitting center. When the listening environment changes, the requirements for sound will also be slightly different. This is a huge challenge for those who cannot make accurate judgments or operate.

[0004] In summary, the traditional hearing aid parameter adjustment method reduces the listening effect of the hearing aid and the user's experience. Summary of the invention

[0005] The present invention provides a device parameter adjustment model training method, a device parameter adjustment method and a device, so as to solve the defect that the conventional hearing aid parameter adjustment method in the prior art reduces the listening effect of the hearing aid and the user's experience.

[0006] The present invention provides a method for training a device parameter adjustment model, comprising the following steps: Acquire an initial model, and determine initial device parameters based on a state representation of the initial model; Sending the initial device parameters to a portable auxiliary device to obtain environmental acoustic characteristics and user behavior characteristics; Inputting the environmental acoustic features and the behavioral features into the initial model to obtain the user's action probability distribution output by the initial model; Based on the action probability distribution, the initial model is trained to obtain the device parameter adjustment model.

[0007] According to a training method for a device parameter adjustment model provided by the present invention, the step of inputting the environmental acoustic features and the behavioral features into the initial model to obtain the user's action probability distribution output by the initial model includes: Acquire a fusion feature, where the fusion feature is obtained by the portable auxiliary device fusing the environmental acoustic feature and the behavioral feature; The fused features are input into the initial model to obtain the action probability distribution output by the initial model.

[0008] According to a training method for a device parameter adjustment model provided by the present invention, the step of inputting the fusion feature into the initial model to obtain the action probability distribution output by the initial model includes: Acquire a fused coding feature, where the fused coding feature is obtained by encoding and encrypting the fused feature sent by the portable auxiliary device by the terminal device; The fused coding feature is input into the initial model to obtain the action probability distribution output by the initial model.

[0009] According to a method for training a device parameter adjustment model provided by the present invention, the state representation is determined based on simulated ambient sound and simulated user adjustment behavior.

[0010] According to a method for training a device parameter adjustment model provided by the present invention, the initial model is trained based on the action probability distribution to obtain the device parameter adjustment model, including: Determining a recommended action based on the action probability distribution; Based on the action reward corresponding to the recommended behavior, the initial model is trained to obtain the device parameter adjustment model.

[0011] According to a method for training a device parameter adjustment model provided by the present invention, the training step of the initial model includes: Obtaining an original model, and determining sample device parameters in a simulated scenario, and labeled user behaviors of the sample device parameters; Inputting the sample device parameters into the original model to obtain the predicted user behavior output by the original model; Based on the difference between the predicted user behavior and the labeled user behavior, a target loss is determined, and based on the target loss, the initial model is determined.

[0012] The present invention also provides a method for adjusting device parameters, comprising the following steps: Obtaining initial device parameters of the portable auxiliary device to be registered; Based on the device parameter adjustment model, the initial device parameters are adjusted to obtain target device parameters; The device parameter adjustment model is obtained by executing the training method based on the above-mentioned device parameter adjustment model.

[0013] The present invention also provides a training device for a device parameter adjustment model, comprising the following units: an acquisition unit, configured to acquire an initial model and determine initial device parameters based on a state representation of the initial model; A sending unit, used to send the initial device parameters to a portable auxiliary device to obtain environmental acoustic characteristics and user behavior characteristics; An input unit, used to input the environmental acoustic features and the behavioral features into the initial model to obtain the action probability distribution of the user output by the initial model; A training unit is used to train the initial model based on the action probability distribution to obtain the device parameter adjustment model.

[0014] The present invention also provides a device for adjusting equipment parameters, comprising the following units: An initial device parameter acquisition unit, used to acquire initial device parameters of the portable auxiliary device to be registered; An adjustment unit, configured to adjust the initial device parameters based on a device parameter adjustment model to obtain target device parameters; The device parameter adjustment model is obtained by executing the training method based on the above-mentioned device parameter adjustment model.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements a training method for a device parameter adjustment model as described above, or implements the device parameter adjustment method.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the training method of the device parameter adjustment model as described in any one of the above, or the device parameter adjustment method is implemented.

[0017] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements a training method for a device parameter adjustment model as described above, or implements the device parameter adjustment method.

[0018] The training method, device parameter adjustment method and apparatus of the device parameter adjustment model provided by the present invention obtain an initial model, and determine the initial device parameters based on the state representation of the initial model, and then send the initial device parameters to the portable auxiliary device to obtain environmental acoustic characteristics and user behavior characteristics, and then input the environmental acoustic characteristics and behavior characteristics into the initial model to obtain the user's action probability distribution output by the initial model, and finally, based on the action probability distribution, train the initial model to obtain the device parameter adjustment model. The behavior characteristics are used to reflect the user's personalized habits and preferences, and the environmental acoustic characteristics are used to reflect the listening environment information of the portable auxiliary device. The subsequent initial model training is performed based on the environmental acoustic characteristics and behavior characteristics, so that the trained device parameter adjustment model can adjust the device parameters of the portable auxiliary device in a personalized manner, thereby improving the user's listening effect, and further improving the user's satisfaction and use experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is one of the flow charts of the training method of the device parameter adjustment model provided by the present invention.

[0021] Figure 2 It is a flowchart of step 130 in the training method of the device parameter adjustment model provided by the present invention.

[0022] Figure 3 It is a flowchart of step 132 in the training method of the device parameter adjustment model provided by the present invention.

[0023] Figure 4 It is a flowchart of step 140 in the training method of the device parameter adjustment model provided by the present invention.

[0024] Figure 5 It is a schematic diagram of the process of initial model training provided by the present invention.

[0025] Figure 6 It is a flow chart of the device parameter adjustment method provided by the present invention.

[0026] Figure 7 This is the second flow chart of the training method of the device parameter adjustment model provided by the present invention.

[0027] Figure 8It is a structural schematic diagram of a training device for a device parameter adjustment model provided by the present invention.

[0028] Fig. 9 It is a structural schematic diagram of the equipment parameter adjustment device provided by the present invention.

[0029] Fig.10 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] In the existing technology, hearing aid parameter adjustment solutions are mostly concentrated in the experimental scenario of the fitting center. First, the fitter will perform an otoscope examination and hearing test to understand the user's hearing loss. Then, the hearing aid fitting and speech resolution test will be performed to ensure the effectiveness of the hearing aid and evaluate the improvement of the hearing-impaired after using the hearing aid.

[0032] After obtaining the target curve, the fitter needs to adjust the MPO (Maximum Power Output) according to the Uncomfortable Loudness Level (UCL) to ensure that the user feels comfortable when wearing the hearing aid. In addition, hearing aid effect evaluation is also an important step to ensure that the hearing aid can meet the user's hearing needs.

[0033] Throughout the process, fine-tuning of hearing aid parameters is essential. This step involves aspects such as the volume, tone, and sound quality of the hearing aid to ensure that the hearing aid can provide the user with the best hearing experience.

[0034] In general, the fitting center will make detailed settings of hearing aid parameters based on the user's specific situation and needs, combining professional knowledge and technology to achieve the best auditory rehabilitation effect.

[0035] However, the hearing aid parameter adjustment method in the prior art has the following defects: 1. Adjusting these parameters requires corresponding professional knowledge, which not only involves a deep understanding of the purpose and benefits of adjusting these functions, but also requires mastering the specific methods of how to properly set these function parameters; 2. Debugging hearing aid parameters requires the use of special fitting tools, such as wireless programmers, wired programmers, programming cables, and debugging software. Due to copyright issues, debugging software can only be provided by hearing aid manufacturers, and dealers or hearing aid wearers cannot purchase and use it at will. Debugging equipment is expensive, and purchasing debugging equipment just for debugging hearing aids will invisibly increase the cost of using hearing aids; 3. The setting of hearing aid parameters should also be combined with the wearer's hearing condition, language discrimination ability, daily living environment, and tolerance to sound. Taking into account individual differences, some wearers have used analog hearing aids before and have high requirements for sound loudness, so when compensating the hearing aid sound, more compensation should be given; some wearers have had hearing loss for a long time, and the degree of hearing loss is moderate or severe. The volume of the sounds they hear on a daily basis is small, the types of sounds are small, and they are accustomed to a quiet environment. When setting, less compensation should be given to allow the wearer to adapt first, and then gradually increase the volume after adaptation, and do not blindly pursue excessive volume.

[0036] 4. Listening scenarios are diverse in real life, and it is difficult for one configuration parameter to meet user needs. When the listening scenario changes, it is difficult for users to adjust the parameters individually, and the adjustment cannot be completed efficiently.

[0037] Based on the above problems, the present invention provides a training method for a device parameter adjustment model. Figure 1 is one of the flow charts of the training method of the device parameter adjustment model provided by the present invention, such as Figure 1 As shown, the execution subject of the method is the cloud, and the method includes step 110, step 120, step 130 and step 140.

[0038] Step 110: acquiring an initial model, and determining initial device parameters based on a state representation of the initial model.

[0039] Specifically, DRN (Deep Reinforcement Network) is a recommendation model based on deep reinforcement learning. The following is an overview of the basic principles and training process of the DRN model. For portable assistive devices, hearing aids are used as an example. For the hearing aid parameter self-adjustment system, it is necessary not only to understand the user's own hearing loss, but also to understand the scene sound of the listening environment. Therefore, the user's personalized behavior (user's personal hearing aid adaptability) characteristics and the acoustic characteristics of the ambient sound are required during training. That is, in the case of streaming input, the hearing aid adaptive configuration parameters are pushed in real time according to the user's adjustment behavior and the changes in the ambient sound.

[0040] First, an initial model may be obtained. The initial model here is an initial model for reinforcement learning. The parameters of the initial model may be pre-trained or randomly generated, and the embodiment of the present invention does not specifically limit this.

[0041] After obtaining the initial model, the initial device parameters can be determined based on the state representation of the initial model. Here, the initial device parameters refer to a series of basic device parameters set for the portable assistant device during the initial configuration. The state representation is the agent's (portable assistant device's) internal description of the environment, which contains all the information required for the agent to make decisions. The state representation includes multiple device parameters such as the device's current volume, user preferences, language settings, etc. Therefore, based on the device parameters in the state representation, the initial device parameters can be set in the configuration of the portable assistant device.

[0042] Among them, the state representation is determined based on the simulated environmental sound and the simulated user adjustment behavior. Specifically, the relevant recommended configuration is given under the general scenario and general user. In the experimental scenario of the fitting center, the general environmental scenario of the user and several typical hearing loss situations are simulated. Professionals configure the parameters and record the simulated environmental sound and simulated user adjustment behavior. The simulated environmental sound and simulated user adjustment behavior are used as the state representation of the recommendation system. In addition, the user's possible adjustment behavior after the simulated real-life scenario environment is switched is used as the training data of the initial model, so as to train the initial model.

[0043] Among them, the simulated user adjustment behavior includes volume adjustment, gain compensation, frequency range adjustment, control feedback, adjustment of noise suppression degree, adjustment of microphone array angle (sound reception angle), etc., and the embodiment of the present invention does not make specific limitations on this.

[0044] Here, volume adjustment refers to adjusting the loudness or amplitude of the audio signal to achieve the user's desired auditory level. Gain compensation refers to amplifying or attenuating the audio signal to compensate for losses or imbalances in signal transmission. Frequency range adjustment refers to adjusting the intensity of different frequency components in the audio signal to change the timbre and texture of the sound. Controlling feedback refers to monitoring feedback phenomena (such as howling) in the audio signal and automatically adjusting system parameters to eliminate or reduce feedback. Adjusting the degree of noise suppression refers to reducing background noise in the audio signal to improve speech clarity. Adjusting the microphone array angle refers to adjusting the directivity and coverage of the microphone array to optimize the sound pickup effect.

[0045] Here, the simulated environmental sound may include natural environmental sound, human environmental sound, special effect sound, etc., which is not specifically limited in the embodiment of the present invention.

[0046] It should be noted that the ambient sound state of the simulated ambient sound is represented by the acoustic compression features of the autoencoder, and this information can be used to describe the user's preference for the environment and the current environmental information.

[0047] Step 120: Send the initial device parameters to the portable auxiliary device to obtain environmental acoustic characteristics and user behavior characteristics.

[0048] Specifically, after obtaining the initial device parameters, the initial device parameters can be sent to the portable auxiliary device to obtain the environmental acoustic characteristics and the user's behavioral characteristics. That is, the initial device parameters can be sent to the portable auxiliary device to request the portable auxiliary device to perform parameter settings based on the initial device parameters to obtain the environmental acoustic characteristics and the user's behavioral characteristics.

[0049] After the portable auxiliary device obtains the environmental acoustic features and the user's behavioral features, the environmental acoustic features and the user's behavioral features can be sent to the cloud, or the environmental acoustic features and the user's behavioral features can be fused and then sent to the cloud. The fused features obtained by fusion of the environmental acoustic features and the user's behavioral features can also be sent to the terminal device, and the terminal device can then encode and encrypt the fused features and send them to the cloud. The embodiments of the present invention do not specifically limit this.

[0050] The portable auxiliary device may be a hearing aid, a translation machine, or a smart voice recorder, etc. When the portable auxiliary device is a translation machine, the user's behavioral characteristics may include the user's adjustment of the language and volume; when the portable auxiliary device is a smart voice recorder, the user's behavioral characteristics may include the user's adjustment of the language, signal-to-noise ratio, and volume, etc. The embodiments of the present invention do not specifically limit this.

[0051] The environmental acoustic characteristics refer to the acoustic characteristics of the ambient sound in the portable auxiliary device. Here, the acquisition of the ambient sound can be that the portable auxiliary device acquires the ambient sound through a built-in microphone or other audio input device. These ambient sounds may include voice, music, noise, and reverberation information around the user.

[0052] Ambient acoustic features can be extracted through autoencoders, which are unsupervised learning methods that learn the potential representation of data by compressing and reconstructing input data. They can be used for dimensionality reduction, denoising, and generation tasks. For the encoding of ambient sound, first, the sound signal of the ambient sound is framed, and the Fast Fourier Transform (FFT) is applied to convert each frame into a spectral representation. Then, the spectral representation is converted into Mel Frequency Cepstral Coefficients (MFCC) features using a Mel FilterBank, and then the MFCC features are compressed using an autoencoder. Specifically, the ambient sound can be used as input data and compressed into a smaller dimensional vector Z through an autoencoder, so that a reduced 512-dimensional ambient sound representation can be obtained.

[0053] The user's behavioral characteristics refer to the behavioral characteristics after the user sets the initial device parameters for the portable assistive device, that is, the user's personalized parameter modification behavior characteristics. Among them, the user's behavioral characteristics can be extracted by Conformer, which is a new deep neural network structure based on the attention mechanism, which is used to model and encode sequence data. The Conformer encoder mainly uses the multi-head self-attention mechanism (Multi-Head Self-Attention Mechanism) and CNN (Convolutional Neural Network) structure to encode voice information, which can effectively extract the characteristics and historical correlation of the user's adjustment behavior, and perform low-dimensional vector representation. At the same time, Conformer also has good noise resistance and generalization ability, and is suitable for various listening environment scenarios. Therefore, using the Conformer encoder module as the user's behavioral feature extraction module can more accurately capture the characteristics and historical correlation of personalized behavior, and improve the accuracy and efficiency of the subsequent adaptive reinforcement learning algorithm.

[0054] Step 130: input the environmental acoustic features and the behavioral features into the initial model to obtain the action probability distribution of the user output by the initial model.

[0055] Specifically, after obtaining the environmental acoustic features and the user's behavioral features, the environmental acoustic features and the user's behavioral features may be input into the initial model to obtain the user's action probability distribution output by the initial model.

[0056] During the training process of reinforcement learning, it is necessary to determine the action space of the recommendation system, that is, the parameter adaptive push behavior. The environmental acoustic characteristics and user behavior characteristics are used as input to predict the action probability distribution of each user action under a given state. In addition, when the environmental sound changes significantly or the user adjusts the behavior, the system pushes the adaptive parameter configuration according to the principle of maximizing the probability of the action probability distribution.

[0057] Here, the action probability distribution refers to the probability distribution of the agent's choice of each possible action in a certain state. Specifically, assuming that the agent is in state S, it may choose a variety of different actions A, and the probability of each action being chosen constitutes the action probability distribution. This action probability distribution describes the agent's preference or tendency for different actions in a given state. In reinforcement learning, the agent needs to optimize this distribution through continuous trial and error and learning so that it can obtain the maximum reward in the long run.

[0058] It is understandable that behavioral features are used to reflect the personalized habits and preferences of users, and environmental acoustic features are used to reflect the listening environment information of portable auxiliary devices. The subsequent initial model training is based on these two features, so that the trained device parameter adjustment model can adjust the device parameters of portable auxiliary devices in a personalized way, thereby improving the user's listening effect and further improving the user's experience. Moreover, this ability to adjust device parameters in a personalized way will bring better user stickiness and competitive advantages to commercial applications.

[0059] Step 140: training the initial model based on the action probability distribution to obtain the device parameter adjustment model.

[0060] Specifically, after the action probability distribution is obtained, the initial model can be trained based on the action probability distribution to obtain the device parameter adjustment model.

[0061] For example, based on the action probability distribution, the recommended behavior can be determined, and based on the action reward corresponding to the recommended behavior, the initial model is trained to obtain the device parameter adjustment model.

[0062] Specifically, the agent can update the strategy corresponding to the action reward according to the action probability distribution. The agent is trained multiple times in the environment, and each iteration includes the steps of state perception, action selection, action execution, and receiving action rewards.

[0063] Furthermore, when the agent's strategy does not change significantly within a certain period of time, or the ambient sound does not change significantly and the user does not make any adjustments, it is considered that the parameter iteration of the initial model has converged, and the initial model after the parameter iteration is used as the device parameter adjustment model.

[0064] It can be understood that the method provided by the embodiment of the present invention applies the reinforcement learning algorithm and the user feedback behavior mechanism to the field of portable auxiliary device parameter adjustment, such as the field of hearing aid parameter adaptive adjustment, to achieve the ability of personalized configuration and dynamic scene learning, thereby solving the limitations of traditional hearing aid parameter adjustment. The method can be applied to a wide range of end-side device scenarios, such as smart voice recorders, translation machines, etc., to provide personalized and high-quality recommendation services to meet user needs. That is, the method has good scalability and can handle massive amounts of user data and complex configuration tasks.

[0065] The method provided by the embodiment of the present invention obtains an initial model, determines initial device parameters based on the state representation of the initial model, and then sends the initial device parameters to a portable auxiliary device to obtain environmental acoustic characteristics and user behavior characteristics, and then inputs the environmental acoustic characteristics and behavior characteristics into the initial model to obtain the user's action probability distribution output by the initial model, and finally, based on the action probability distribution, trains the initial model to obtain a device parameter adjustment model. Behavioral characteristics are used to reflect the user's personalized habits and preferences, and environmental acoustic characteristics are used to reflect the listening environment information of the portable auxiliary device. Subsequent initial model training is performed based on environmental acoustic characteristics and behavioral characteristics, so that the trained device parameter adjustment model can adjust the device parameters of the portable auxiliary device in a personalized manner, thereby improving the user's listening effect, and further improving the user's satisfaction and user experience.

[0066] Based on the above embodiments, Figure 2 is a flow chart of step 130 in the training method of the device parameter adjustment model provided by the present invention, such as Figure 2 As shown, step 130 includes: Step 131, obtaining a fusion feature, where the fusion feature is obtained by the portable auxiliary device fusing the environmental acoustic feature and the behavioral feature; Step 132: input the fusion feature into the initial model to obtain the action probability distribution output by the initial model.

[0067] Specifically, a fusion feature can be obtained, wherein the fusion feature is obtained by the portable auxiliary device fusing the environmental acoustic feature and the behavioral feature, and the fusion feature is transmitted to the cloud.

[0068] Here, the feature fusion of the environmental acoustic features and the behavioral features may be performed by concatenating the environmental acoustic features and the behavioral features, or by weighting the environmental acoustic features and the behavioral features using an attention mechanism and then concatenating them, which is not specifically limited in the embodiment of the present invention.

[0069] After obtaining the fused features, the fused features can be input into the initial model to obtain the action probability distribution output by the initial model.

[0070] For example, the 512-dimensional environmental acoustic features and behavioral features are finally merged to form the fusion features of the current training token. As the input of the initial model of reinforcement learning, user behavior information and information of various dimensions of the scene can be effectively utilized to improve the learning discrimination of the initial model.

[0071] The method provided in the embodiment of the present invention obtains fusion features, which are obtained by the portable auxiliary device fusing environmental acoustic features and behavioral features, and then inputs the fusion features into the initial model to obtain the action probability distribution output by the initial model. Since the fusion features can remove redundant information and extract more useful feature combinations, the accuracy and reliability of the initial model's prediction of device parameters can be improved.

[0072] Based on the above embodiments, Figure 3 is a flow chart of step 132 in the training method of the device parameter adjustment model provided by the present invention, such as Figure 3 As shown, step 132 includes: Step 1321, obtaining a fused coding feature, where the fused coding feature is obtained by encoding and encrypting the fused feature sent by the portable auxiliary device by the terminal device; Step 1322: input the fused coding feature into the initial model to obtain the action probability distribution output by the initial model.

[0073] Specifically, considering that the portable auxiliary device may not have a network connection, and the terminal device has the ability to temporarily store the fusion features collected by the portable auxiliary device, the fusion features can be transmitted to the terminal device and then transmitted to the cloud by the terminal device.

[0074] Furthermore, when the fused features are transmitted to the cloud, in order to protect user privacy, the fused features can be encoded and encrypted before transmission.

[0075] First, a fused coding feature is obtained, wherein the fused coding feature is obtained by encoding and encrypting the fused feature sent by the portable auxiliary device by the terminal device.

[0076] It should be noted that the fused coding features are represented only by 512-dimensional features, the feature extraction stage is only performed on portable auxiliary devices, and the data transmission only has 512-dimensional text implicit features, which will not expose the user's private text information. In addition, the feature information is used for subsequent processing in the cloud and cannot be decoded into plain text, which can effectively protect the user's personal privacy.

[0077] It should be noted that the transmission of fusion features between terminal devices, such as mobile phones, and portable auxiliary devices is based on LE Audio (Low Energy Audio), which is a low-power audio technology launched by the Bluetooth SIG to provide higher-quality audio transmission for Bluetooth devices. Compared with traditional Bluetooth audio technology, LE Audio has higher sound quality, lower latency and lower power consumption. In addition, LE Audio also supports multi-device connection, broadcast audio and other functions, making the audio experience richer and more convenient.

[0078] The specific steps to obtain fusion features using LE Audio are as follows: 1. Feature fusion: The extracted fusion features are fused with the LE Audio audio signal (ambient sound). This can be achieved by embedding the fusion features into the LE Audio audio signal.

[0079] 2. Audio transmission: The fused LE Audio audio signal is transmitted wirelessly to the terminal device via Bluetooth.

[0080] 3. Device decoding: After receiving the fused LE Audio signal, the terminal device decodes it and restores the fusion features.

[0081] Here, the terminal device can use Huffman coding compression technology to encode and encrypt the fusion features sent by the portable auxiliary device, which can not only reduce the size of the data, reduce storage and transmission costs, but also protect the privacy of the data to a certain extent. Huffman coding: This is a lossless coding technology that builds a Huffman tree based on the frequency of characters in the data, and then assigns a unique binary code to each character. High-frequency characters are assigned shorter codes, while low-frequency characters are assigned longer codes. This method can effectively reduce the size of the data while maintaining the integrity of the data.

[0082] It is understandable that the fused features are uploaded to the cloud after encoding and compression, and there is double encryption, namely feature extraction encryption and encoding encryption, so as to better protect user privacy and security.

[0083] After obtaining the fused coding features, the fused coding features can be input into the initial model to obtain the action probability distribution output by the initial model.

[0084] The method provided by the embodiment of the present invention obtains fused coding features, which are obtained by encoding and encrypting the fused features sent by the portable auxiliary device by the terminal device, and then inputting the fused coding features into the initial model to obtain the action probability distribution output by the initial model. On the one hand, considering that the portable auxiliary device may not have a network connection, and the terminal device has the ability to temporarily store the fused features collected by the portable auxiliary device, the fused features are transmitted from the portable auxiliary device to the terminal device, thereby avoiding the loss of environmental acoustic features and user behavior features in the portable auxiliary device; on the other hand, the fused coding features obtained after encoding and compressing the fused features are uploaded to the cloud, that is, the user data is compressed locally and the full-link transmission is transmitted in the form of a feature matrix, thereby better protecting the privacy and security of the user and ensuring the confidentiality and integrity of the user data.

[0085] Based on the above embodiments, Figure 4 is a flow chart of step 140 in the training method of the device parameter adjustment model provided by the present invention, such as Figure 4 As shown, step 140 includes: Step 141, determining a recommended behavior based on the action probability distribution; Step 142: training the initial model based on the action reward corresponding to the recommended behavior to obtain the device parameter adjustment model.

[0086] Specifically, the recommended behavior can be determined based on the action probability distribution, that is, the behavior corresponding to the probability distribution with the largest action probability distribution can be used as the recommended behavior, and then the initial model can be trained based on the action reward corresponding to the recommended behavior to obtain the device parameter adjustment model.

[0087] The reward function of action rewards should encourage the recommendation system to provide hearing aid parameter configurations that are more suitable for the user's listening experience based on user feedback (such as satisfaction evaluation and adjustment behavior frequency). Design reward functions to strengthen recommendation goals: discretize user behavior and give different rewards based on different recommended behaviors. For example, in scenarios where the ambient sound changes greatly, users are given positive rewards for not making adjustments to the parameter configurations issued or making very small adjustments, and differential rewards are given based on the adjustment amplitude and frequency. Based on the issued parameters, positive incentives of 1-10 points are given for deviations within 10%, negative incentives of 1-10 points are given for deviations above 10%, and so on. In this way, a balance is found between user adjustment and parameter issuance, and ultimately personalized customization of the system is achieved. This reward function can make the model more adaptable to user preferences.

[0088] In actual operation, it interacts with the environment, observes the status, performs actions, obtains action rewards, and uses these experiences to train and update the policy network. During training, batch updates or online updates can be used to adjust the design and hyperparameters of the policy network by comparing indicators such as recommendation effects, user satisfaction evaluations, user configuration adjustment frequency, and parameter adjustment range.

[0089] The main feature of the DRN algorithm is the use of experience replay and target network (device parameter adjustment model) technology to solve the sample correlation and instability problems in reinforcement learning. Experience replay can reduce the correlation between samples and provide more samples for training. The target network can provide a relatively stable target to reduce fluctuations and instability during training.

[0090] The method provided by the embodiment of the present invention, the device parameter adjustment model is equivalent to a self-learning network model constructed using a reinforcement learning algorithm, which can learn from the user's feedback and input on the effect of the hearing aid in the usage scenario, thereby improving the user's listening clarity and comfort.

[0091] Based on the above embodiments, Figure 5 It is a schematic diagram of the process of initial model training provided by the present invention, such as Figure 5 As shown, the training steps of the initial model include: Step 510, obtaining the original model, and determining sample device parameters in the simulation scenario, and label user behaviors of the sample device parameters; Step 520, inputting the sample device parameters into the original model to obtain the predicted user behavior output by the original model; Step 530, determining a target loss based on the difference between the predicted user behavior and the labeled user behavior, and determining the initial model based on the target loss.

[0092] Specifically, in order to better obtain the initial model, training can be performed based on the following steps: First, the original model is obtained, and the sample device parameters and the labeled user behaviors of the sample device parameters in the simulation scenario are determined. The simulation scenario is the fitting center scenario.

[0093] The parameters of the original model here may be preset or randomly generated, and the embodiment of the present invention does not specifically limit this.

[0094] After obtaining the sample device parameters, the sample device parameters may be input into the original model to obtain the predicted user behavior output by the original model.

[0095] Predicting user behavior may include volume adjustment, gain compensation, frequency range adjustment, control feedback, adjusting noise suppression level, adjusting microphone array angle (sound pickup angle), etc., which is not specifically limited in the embodiments of the present invention.

[0096] After the predicted user behavior is obtained based on the original model, the target loss can be determined based on the difference between the predicted user behavior and the labeled user behavior, and the parameters of the original model can be iterated based on the target loss, and the original model after the parameter iteration is used as the initial model.

[0097] It can be understood that the greater the difference between the predicted user behavior and the labeled user behavior, the greater the target loss; the smaller the difference between the predicted user behavior and the labeled user behavior, the smaller the target loss.

[0098] The method provided by the embodiment of the present invention obtains the original model, determines the sample device parameters in the simulation scenario, and the labeled user behavior of the sample device parameters, and then inputs the sample device parameters into the original model to obtain the predicted user behavior output by the original model. Finally, based on the difference between the predicted user behavior and the labeled user behavior, the target loss is determined, and based on the target loss, the initial model is determined, so that the initial model has the ability to predict user behavior based on the device parameters.

[0099] Based on any of the above embodiments, the present invention provides a method for adjusting device parameters. Figure 6 is a flow chart of the device parameter adjustment method provided by the present invention, such as Figure 6 As shown, the method includes: Step 610, obtaining initial device parameters of the portable auxiliary device to be registered; Step 620, adjusting the initial device parameters based on the device parameter adjustment model to obtain target device parameters; The device parameter adjustment model is obtained by executing the training method based on the above-mentioned device parameter adjustment model.

[0100] Specifically, the initial device parameters of the portable auxiliary device to be registered may be obtained, wherein the initial device parameters refer to a series of basic device parameters set for the portable auxiliary device during initial configuration.

[0101] The portable auxiliary device may be a hearing aid, a translation machine, or a smart voice recorder, etc., which is not specifically limited in the embodiment of the present invention.

[0102] After the initial device parameters are obtained, the initial device parameters may be adjusted based on the device parameter adjustment model to obtain target device parameters, where the target device parameters refer to the device parameters that are finally determined for the setting of the portable auxiliary device.

[0103] The training steps of the device parameter adjustment model are as follows: First, an initial model is obtained. The initial model here is an initial model for reinforcement learning. The parameters of the initial model can be pre-trained or randomly generated, and the embodiment of the present invention does not specifically limit this.

[0104] After obtaining the initial model, the initial device parameters can be determined based on the state representation of the initial model. The state representation is the agent's (portable assistant device's) internal description of the environment, which contains all the information required for the agent to make decisions. The state representation includes multiple device parameters such as the device's current volume, user preferences, language settings, etc. Therefore, based on the device parameters in the state representation, the initial device parameters can be set in the configuration of the portable assistant device.

[0105] Among them, the state representation is determined based on the simulated environmental sound and the simulated user adjustment behavior. Specifically, the relevant recommended configuration is given under the general scenario and general user. In the experimental scenario of the fitting center, the general environmental scenario of the user and several typical hearing loss situations are simulated. Professionals configure the parameters and record the simulated environmental sound and simulated user adjustment behavior. The simulated environmental sound and simulated user adjustment behavior are used as the state representation of the recommendation system. In addition, the user's possible adjustment behavior after the simulated real-life scenario environment is switched is used as the training data of the initial model, so as to train the initial model.

[0106] Among them, the simulated user adjustment behavior includes volume adjustment, gain compensation, frequency range adjustment, control feedback, adjustment of noise suppression degree, adjustment of microphone array angle (sound reception angle), etc., and the embodiment of the present invention does not make specific limitations on this.

[0107] Here, volume adjustment refers to adjusting the loudness or amplitude of the audio signal to achieve the user's desired auditory level. Gain compensation refers to amplifying or attenuating the audio signal to compensate for losses or imbalances in signal transmission. Frequency range adjustment refers to adjusting the intensity of different frequency components in the audio signal to change the timbre and texture of the sound. Controlling feedback refers to monitoring feedback phenomena (such as howling) in the audio signal and automatically adjusting system parameters to eliminate or reduce feedback. Adjusting the degree of noise suppression refers to reducing background noise in the audio signal to improve speech clarity. Adjusting the microphone array angle refers to adjusting the directivity and coverage of the microphone array to optimize the sound pickup effect.

[0108] Here, the simulated environmental sound may include natural environmental sound, human environmental sound, special effect sound, etc., which is not specifically limited in the embodiment of the present invention.

[0109] It should be noted that the ambient sound state of the simulated ambient sound is represented by the acoustic compression features of the autoencoder, and this information can be used to describe the user's preference for the environment and the current environmental information.

[0110] After obtaining the initial device parameters, the initial device parameters can be sent to the portable auxiliary device to obtain the environmental acoustic characteristics and the user's behavioral characteristics. That is, the initial device parameters can be sent to the portable auxiliary device to request the portable auxiliary device to perform parameter settings based on the initial device parameters to obtain the environmental acoustic characteristics and the user's behavioral characteristics.

[0111] After the portable auxiliary device obtains the environmental acoustic features and the user's behavioral features, the environmental acoustic features and the user's behavioral features can be sent to the cloud, or the environmental acoustic features and the user's behavioral features can be fused and then sent to the cloud. The fused features obtained by fusion of the environmental acoustic features and the user's behavioral features can also be sent to the terminal device, and the terminal device can then encode and encrypt the fused features and send them to the cloud. The embodiments of the present invention do not specifically limit this.

[0112] Ambient acoustic features can be extracted through an autoencoder, which is an unsupervised learning method that learns the potential representation of data by compressing and reconstructing input data. It can be used for dimensionality reduction, denoising, and generation tasks. For the encoding of ambient sound, first, the sound signal of the ambient sound is framed, and each frame is converted into a spectral representation using a fast Fourier transform. Then, the spectral representation is converted into a Mel-frequency cepstral coefficient feature using a Mel filter bank, and then the MFCC feature is compressed using an autoencoder. Specifically, the ambient sound can be used as input data and compressed into a smaller dimensional vector Z through an autoencoder, so that a reduced 512-dimensional ambient sound representation can be obtained.

[0113] The user's behavioral characteristics refer to the behavioral characteristics after the user sets the initial device parameters for the portable assistive device. Among them, the user's behavioral characteristics can be extracted through Conformer. Conformer is a new deep neural network structure based on the attention mechanism, which is used to model and encode sequence data. The Conformer encoder mainly uses a multi-head self-attention mechanism and a CNN structure to encode voice information. It can effectively extract the characteristics and historical correlation of the user's adjustment behavior, and perform low-dimensional vector representation. At the same time, Conformer also has good noise resistance and generalization capabilities, and is suitable for a variety of different listening environment scenarios. Therefore, using the Conformer encoder module as a user's behavioral feature extraction module can more accurately capture the characteristics and historical correlation of personalized behaviors, and improve the accuracy and efficiency of subsequent adaptive reinforcement learning algorithms.

[0114] The environmental acoustic features and behavioral features are input into the initial model to obtain the user's action probability distribution output by the initial model.

[0115] Based on the action probability distribution, the recommended behavior is determined, and then based on the action reward corresponding to the recommended behavior, the initial model is trained to obtain the device parameter adjustment model.

[0116] It should be noted that by analyzing the environmental acoustic characteristics and behavioral characteristics of the user's ambient sound, the portable auxiliary device can automatically adjust parameters such as volume and frequency to ensure that the final device parameters make the sound emitted by the portable auxiliary device at a level balanced with the background noise, so that the user can clearly hear the required sound, thereby improving the user's listening effect.

[0117] The method provided by the embodiment of the present invention obtains the initial device parameters of the portable auxiliary device to be registered, and then adjusts the initial device parameters based on the device parameter adjustment model to obtain the target device parameters; the device parameter adjustment model is obtained by executing the training method based on the above-mentioned device parameter adjustment model, thereby improving the accuracy and reliability of the target device parameters, improving the user's satisfaction and listening effect, and thus improving the user's usage experience.

[0118] Based on any of the above embodiments, Figure 7 FIG. 2 is a flow chart of the training method of the device parameter adjustment model provided by the present invention. Figure 7 As shown, the method includes: In step S1, the cloud initializes the reinforcement learning model based on the sample device parameters of the fitting center data to obtain the initial model, then conducts real-time reinforcement learning training based on the initial model, determines the initial device parameters based on the state representation of the initial model, and sends the initial device parameters to the end-side hearing aid.

[0119] In step S2, the end-side hearing aid extracts the acoustic features of the ambient sound to obtain the ambient acoustic features, extracts the user's personalized adjustment behavior to obtain the user's behavior features, and fuses the ambient acoustic features and behavior features to obtain fused features, and fuses the extracted fused features with the LE Audio audio signal. This can be achieved by embedding the fused features into the LE Audio audio signal, and transmitting the fused LE Audio audio signal to the end-side mobile phone via Bluetooth wireless.

[0120] Step S3: After receiving the fused LE Audio audio signal, the mobile phone on the edge decodes it, restores the fused features, encodes and encrypts the fused features to obtain fused coded features, and uploads the fused coded features to the cloud.

[0121] In step S4, the cloud inputs the received fusion coding features into the initial model of reinforcement learning, obtains the user's action probability distribution output by the initial model, and determines the recommended behavior based on the action probability distribution. Finally, based on the action reward corresponding to the recommended behavior, the initial model is trained to obtain the device parameter adjustment model.

[0122] It can be understood that the above process is a reinforcement learning training process. If the ambient sound in the end-side hearing aid changes or the user makes personalized adjustments, the operation process from step S1 to step S4 is performed again until the ambient sound in the end-side hearing aid does not change or the user does not make personalized adjustments, and the reinforcement learning training process ends.

[0123] The method provided by the embodiment of the present invention proposes a method and system for self-adjusting hearing aid parameters based on cloud-edge-end collaboration based on reinforcement learning. An embedded vector representing a common user and common scenario is generated through the systematic configuration of the laboratory scenario. A benchmark version is deployed on the cloud to initialize the hearing aid parameters. The hearing aid, as an end-side device, collects the ambient sound and user parameter adjustment behavior, which are transmitted to the edge mobile phone through LE Audio. The mobile phone encodes and encrypts the embedded vector representation of the fusion feature and sends it to the cloud. The initial model on the cloud performs user personalized adaptation adjustment and sends down the relevant parameter configuration. In this process, the reinforcement learning model obtains incentives based on user feedback and optimizes the strategy by maximizing the user experience score.

[0124] It is understandable that the ambient sound of the user's listening scene and personalized parameter adjustment behavior are collected, and then the features of the ambient sound and the user's personalized adjustment behavior are extracted based on the behavior embedding vector method and acoustic coding, and they are mapped into low-dimensional speech behavior vector representations (fusion features). Since the ambient acoustic features contain the noise and reverberation information of the ambient sound, the behavioral features include the user's personalized parameter modification behavior. Therefore, the fusion features determined based on the ambient acoustic features and behavioral features can be used to adjust the user's satisfaction feedback mechanism. According to the phased product usage satisfaction feedback mechanism, the reinforcement learning model parameter adjustment configuration will be optimized and distributed in a timely manner, thereby establishing a more accurate and real-time updated user personalized hearing aid parameter adaptive adjustment system.

[0125] The method provided by the embodiment of the present invention can learn the user's personal usage habits, dynamically adjust the hearing aid configuration through a reinforcement learning algorithm, thereby improving the listening effect and providing the user with a clearer, more comfortable and convenient hearing aid service.

[0126] In summary, this method performs personalized parameter adaptation according to the user's personal usage habits and usage scenarios, applies the reinforcement learning algorithm to the adaptive parameter adjustment system, improves the user experience, and further optimizes the reinforcement learning system based on user feedback to improve the listening effect.

[0127] The training device for the equipment parameter adjustment model provided by the present invention is described below. The training device for the equipment parameter adjustment model described below and the training method for the equipment parameter adjustment model described above can refer to each other.

[0128] Based on any of the above embodiments, the present invention provides a training device for a device parameter adjustment model. Figure 8 is a structural schematic diagram of a training device for a device parameter adjustment model provided by the present invention, such as Figure 8 As shown, the device comprises: An acquisition unit 810 is used to acquire an initial model and determine initial device parameters based on a state representation of the initial model; A sending unit 820, configured to send the initial device parameters to a portable auxiliary device to obtain environmental acoustic characteristics and user behavior characteristics; An input unit 830, configured to input the environmental acoustic features and the behavioral features into the initial model to obtain the user's action probability distribution output by the initial model; The training unit 840 is used to train the initial model based on the action probability distribution to obtain the device parameter adjustment model.

[0129] The device provided by the embodiment of the present invention obtains an initial model, and determines initial device parameters based on the state representation of the initial model, and then sends the initial device parameters to the portable auxiliary device to obtain environmental acoustic characteristics and user behavior characteristics, and then inputs the environmental acoustic characteristics and behavior characteristics into the initial model to obtain the user's action probability distribution output by the initial model, and finally, based on the action probability distribution, the initial model is trained to obtain a device parameter adjustment model. The behavior characteristics are used to reflect the user's personalized habits and preferences, and the environmental acoustic characteristics are used to reflect the listening environment information of the portable auxiliary device. The subsequent initial model training is performed based on the environmental acoustic characteristics and behavior characteristics, so that the trained device parameter adjustment model can adjust the device parameters of the portable auxiliary device in a personalized manner, thereby improving the user's listening effect, and further improving the user's satisfaction and use experience.

[0130] Based on any of the above embodiments, the input unit 830 specifically includes: A fusion feature acquisition unit, used for acquiring a fusion feature, wherein the fusion feature is obtained by the portable auxiliary device fusing the environmental acoustic feature and the behavioral feature; An input subunit is used to input the fusion feature into the initial model to obtain the action probability distribution output by the initial model.

[0131] Based on any of the above embodiments, the input subunit is specifically used for: Acquire a fused coding feature, where the fused coding feature is obtained by encoding and encrypting the fused feature sent by the portable auxiliary device by the terminal device; The fused coding feature is input into the initial model to obtain the action probability distribution output by the initial model.

[0132] Based on any of the above embodiments, the state representation is determined based on simulated environmental sound and simulated user adjustment behavior.

[0133] Based on any of the above embodiments, the training unit 840 is specifically used for: Determining a recommended action based on the action probability distribution; Based on the action reward corresponding to the recommended behavior, the initial model is trained to obtain the device parameter adjustment model.

[0134] Based on any of the above embodiments, it further includes an initial model training unit, and the initial model training unit is specifically used to: Obtaining an original model, and determining sample device parameters in a simulated scenario, and labeled user behaviors of the sample device parameters; Inputting the sample device parameters into the original model to obtain the predicted user behavior output by the original model; Based on the difference between the predicted user behavior and the labeled user behavior, a target loss is determined, and based on the target loss, the initial model is determined.

[0135] The device parameter adjustment apparatus provided by the present invention is described below. The device parameter adjustment apparatus described below and the device parameter adjustment method described above can be referred to each other.

[0136] Based on any of the above embodiments, the present invention provides a device for adjusting equipment parameters. Fig. 9 is a schematic diagram of the structure of the device parameter adjustment device provided by the present invention, such as Fig. 9 As shown, the device comprises: An initial device parameter acquisition unit 910 is used to acquire initial device parameters of a portable auxiliary device to be registered; An adjustment unit 920, configured to adjust the initial device parameters based on a device parameter adjustment model to obtain target device parameters; The device parameter adjustment model is obtained by executing the training method based on the above-mentioned device parameter adjustment model.

[0137] The device provided by the embodiment of the present invention obtains the initial device parameters of the portable auxiliary device to be registered, and then adjusts the initial device parameters based on the device parameter adjustment model to obtain the target device parameters; the device parameter adjustment model is obtained by executing the training method based on the above-mentioned device parameter adjustment model, thereby improving the accuracy and reliability of the target device parameters, improving the user's satisfaction and listening effect, and thus improving the user's usage experience.

[0138] Fig.10 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Fig.10 As shown, the electronic device may include: a processor 1010, a communication interface 1020, a memory 1030 and a communication bus 1040, wherein the processor 1010, the communication interface 1020 and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 may call the logic instructions in the memory 1030 to execute the training method of the device parameter adjustment model, the method comprising: obtaining an initial model, and determining initial device parameters based on the state representation of the initial model; sending the initial device parameters to a portable auxiliary device to obtain environmental acoustic features and user behavior features; inputting the environmental acoustic features and the behavior features into the initial model to obtain the user's action probability distribution output by the initial model; training the initial model based on the action probability distribution to obtain the device parameter adjustment model.

[0139] The processor 1010 can also call the logic instructions in the memory 1030 to execute the device parameter adjustment method, which includes: obtaining the initial device parameters of the portable auxiliary device to be registered; adjusting the initial device parameters based on the device parameter adjustment model to obtain the target device parameters; the device parameter adjustment model is obtained by executing the training method based on the above-mentioned device parameter adjustment model.

[0140] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0141] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the training method of the device parameter adjustment model provided by the above-mentioned methods, and the method includes: obtaining an initial model, and determining initial device parameters based on the state representation of the initial model; sending the initial device parameters to a portable auxiliary device to obtain environmental acoustic characteristics and user behavior characteristics; inputting the environmental acoustic characteristics and the behavior characteristics into the initial model to obtain the user's action probability distribution output by the initial model; training the initial model based on the action probability distribution to obtain the device parameter adjustment model.

[0142] When the computer program is executed by the processor, the computer can execute the device parameter adjustment method provided by the above methods, which method includes: obtaining the initial device parameters of the portable auxiliary device to be registered; adjusting the initial device parameters based on the device parameter adjustment model to obtain the target device parameters; the device parameter adjustment model is obtained by executing the training method based on the above device parameter adjustment model.

[0143] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the training method of the device parameter adjustment model provided by the above-mentioned methods, the method comprising: obtaining an initial model, and determining initial device parameters based on a state representation of the initial model; sending the initial device parameters to a portable auxiliary device to obtain environmental acoustic characteristics and user behavior characteristics; inputting the environmental acoustic characteristics and the behavior characteristics into the initial model to obtain the user's action probability distribution output by the initial model; and training the initial model based on the action probability distribution to obtain the device parameter adjustment model.

[0144] When the computer program is executed by the processor, it is implemented to execute the device parameter adjustment method provided by the above methods, and the method includes: obtaining initial device parameters of the portable auxiliary device to be registered; adjusting the initial device parameters based on the device parameter adjustment model to obtain target device parameters; the device parameter adjustment model is obtained by executing the training method based on the above device parameter adjustment model.

[0145] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a device parameter adjustment model, characterized in that: include: Acquire an initial model, and determine initial device parameters based on a state representation of the initial model; Sending the initial device parameters to a portable auxiliary device to obtain environmental acoustic characteristics and user behavior characteristics; Inputting the environmental acoustic features and the behavioral features into the initial model to obtain the user's action probability distribution output by the initial model; Based on the action probability distribution, the initial model is trained to obtain the device parameter adjustment model.

2. The training method of the device parameter adjustment model according to claim 1, characterized in that: The step of inputting the environmental acoustic features and the behavioral features into the initial model to obtain the user's action probability distribution output by the initial model includes: Acquire a fusion feature, where the fusion feature is obtained by the portable auxiliary device fusing the environmental acoustic feature and the behavioral feature; The fused features are input into the initial model to obtain the action probability distribution output by the initial model.

3. The training method of the device parameter adjustment model according to claim 2, characterized in that: The step of inputting the fusion feature into the initial model to obtain the action probability distribution output by the initial model includes: Acquire a fused coding feature, where the fused coding feature is obtained by encoding and encrypting the fused feature sent by the portable auxiliary device by the terminal device; The fused coding feature is input into the initial model to obtain the action probability distribution output by the initial model.

4. The method for training a device parameter adjustment model according to any one of claims 1 to 3, characterized in that: The state representation is determined based on simulated environmental sound and simulated user adjustment behavior.

5. The method for training a device parameter adjustment model according to any one of claims 1 to 3, characterized in that: The training of the initial model based on the action probability distribution to obtain the device parameter adjustment model includes: Determining a recommended action based on the action probability distribution; Based on the action reward corresponding to the recommended behavior, the initial model is trained to obtain the device parameter adjustment model.

6. The method for training a device parameter adjustment model according to any one of claims 1 to 3, characterized in that: The training steps of the initial model include: Obtaining an original model, and determining sample device parameters in a simulated scenario, and labeled user behaviors of the sample device parameters; Inputting the sample device parameters into the original model to obtain the predicted user behavior output by the original model; Based on the difference between the predicted user behavior and the labeled user behavior, a target loss is determined, and based on the target loss, the initial model is determined.

7. A method for adjusting device parameters, characterized in that: include: Obtaining initial device parameters of the portable auxiliary device to be registered; Based on the device parameter adjustment model, the initial device parameters are adjusted to obtain target device parameters; The device parameter adjustment model is obtained by executing a training method for a device parameter adjustment model according to any one of claims 1 to 6.

8. A training device for a device parameter adjustment model, characterized in that: include: an acquisition unit, configured to acquire an initial model and determine initial device parameters based on a state representation of the initial model; A sending unit, used to send the initial device parameters to a portable auxiliary device to obtain environmental acoustic characteristics and user behavior characteristics; An input unit, used to input the environmental acoustic features and the behavioral features into the initial model to obtain the action probability distribution of the user output by the initial model; A training unit is used to train the initial model based on the action probability distribution to obtain the device parameter adjustment model.

9. A device parameter adjustment device, characterized in that: include: An initial device parameter acquisition unit, used to acquire initial device parameters of the portable auxiliary device to be registered; An adjustment unit, configured to adjust the initial device parameters based on a device parameter adjustment model to obtain target device parameters; The device parameter adjustment model is obtained by executing a training method for a device parameter adjustment model according to any one of claims 1 to 6.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, it implements the training method of the device parameter adjustment model as described in any one of claims 1 to 6, or implements the device parameter adjustment method as described in claim 7.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the training method of the device parameter adjustment model as described in any one of claims 1 to 6, or implements the device parameter adjustment method as described in claim 7.

Images