Audio playing control system, method and device and storage medium

Abstract

The embodiment of the invention provides an audio playing control system, method and device and a storage medium, and the method comprises the steps: collecting gesture data of a user, and determining a gesture mode based on the gesture data; based on the gesture mode, an audio control instruction is determined through a gesture mapping table, and the gesture mapping table comprises a mapping relation between one or more gesture modes and one or more audio control instructions; training an instruction prediction model based on the historical operation data and the historical control instruction; an audio control instruction is predicted by an instruction prediction model based on gesture operation data including gesture data, and a gesture mapping table is adjusted based on the gesture operation data and the predicted audio control instruction.

Description

Technical Field

[0001] The present invention relates to the field of audio playback, and in particular to an audio playback control system, method, device and storage medium. Background Art

[0002] With the development of science and technology, various players have gradually entered people's lives, and using players to play audio has been loved by consumers. At present, the more common methods of controlling audio playback include using a remote control, using the control center on the mobile phone, voice control, etc. These methods have problems such as requiring users to perform too many operations or inaccurate voice collection during audio playback or affecting normal audio playback.

[0003] Therefore, it is necessary to provide an audio playback control system, method, device and storage medium that can reduce the user's operation workload, achieve intelligent control of audio playback, and improve the accuracy of data collection. Summary of the invention

[0004] One or more embodiments of the present specification provide an audio playback control system, the system comprising: a gesture recognition module, configured to collect user gesture data, and determine a gesture pattern based on the gesture data; a gesture mapping module, configured to determine an audio control instruction based on the gesture pattern through a gesture mapping table, the gesture mapping table comprising a mapping relationship between one or more gesture patterns and one or more audio control instructions; a model training module, configured to train an instruction prediction model based on historical operation data and historical control instructions; the instruction prediction model is a machine learning model; a prediction module, configured to predict the audio control instruction based on gesture operation data including the gesture data through the instruction prediction model, and adjust the gesture mapping table based on the gesture operation data and the predicted audio control instruction.

[0005] One or more embodiments of the present specification provide an audio playback control method, the method comprising: collecting user gesture data, and determining a gesture pattern based on the gesture data; based on the gesture pattern, determining an audio control instruction through a gesture mapping table, the gesture mapping table comprising a mapping relationship between one or more gesture patterns and one or more audio control instructions; based on historical operation data and historical control instructions, training an instruction prediction model; the instruction prediction model is a machine learning model; through the instruction prediction model, predicting the audio control instruction based on gesture operation data including the gesture data, and adjusting the gesture mapping table based on the gesture operation data and the predicted audio control instruction.

[0006] One or more embodiments of the present specification provide an audio playback control device, which includes at least one processor and at least one memory; the at least one memory is used to store computer instructions; the at least one processor is used to execute at least part of the computer instructions to implement an audio playback control method.

[0007] One or more embodiments of the present specification provide a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the audio playback control method. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0009] Figure 1 is a schematic diagram of an application scenario of an audio playback control system according to some embodiments of this specification;

[0010] Figure 2 is an exemplary module diagram of an audio playback control system according to some embodiments of this specification;

[0011] Figure 3 is an exemplary flow chart of an audio playback control method according to some embodiments of this specification;

[0012] Figure 4 is an exemplary schematic diagram of updating a gesture mapping table according to some embodiments of this specification;

[0013] Figure 5 It is an exemplary schematic diagram of updating the instruction prediction model based on the necessity of model update according to some embodiments of this specification. DETAILED DESCRIPTION

[0014] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0015] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0016] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0017] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0018] Figure 1 1 is a schematic diagram of application scenarios of the audio playback control system according to some embodiments of this specification. In some embodiments, the application scenarios of the audio playback control system may include but are not limited to home, theater, conference, etc.

[0019] In some embodiments, Figure 1 As shown, the application scenario 100 of the audio playback control system may include a player 110 , a user 120 , a processor 130 , a network 140 and a storage device 150 .

[0020] The player 110 refers to a device for playing audio or video, such as a media player, a video player, an audio player, etc. As an example, in a home scenario, the player may be a TV, a projector, etc.; in a theater scenario, the player may be a screen, etc.; in a conference scenario, the player may be a computer, a projection screen, etc.

[0021] The user 120 refers to a person who uses the player 110. For example, a tester or user of the audio playback control system. In some embodiments, when the user is a tester of the audio playback control system, the user can obtain historical test data related to the audio playback control system stored in the memory through the network based on the player 110 or a user terminal (not shown in the figure) and other devices, and select different test methods to test the audio playback control system by analyzing the historical test data.

[0022] The processor 130 can process data and / or information obtained from the player 110, the user 120, the network 140 and the storage device 150 and / or other components of the application scenario 100 of the audio playback control system. The processor can execute program instructions based on these data, information and / or processing results to perform one or more functions described in the present application. For example, the processor can determine audio control instructions, etc.

[0023] In some embodiments, the processor 130 may be a computer, a user console, a single processor, a processor group, etc. The processor group may be centralized or distributed. In some embodiments, the processor 130 may be implemented on a cloud platform. For example, the cloud platform may include one or any combination of a private cloud, a public cloud, a hybrid cloud, etc.

[0024] In some embodiments, the modules of the audio playback control system 200 may be partially or completely integrated on the processor 130. For more information about the audio playback control system 200, see Figure 2 and its corresponding description.

[0025] The network 140 includes any suitable network capable of facilitating information and / or data exchange of the application scenario 100 of the audio playback control system. In some embodiments, one or more components of the application scenario 100 of the audio playback control system (e.g., the player 110, the user 120, the processor 130, the storage device 150, etc.) can exchange information and / or data with one or more components of the application scenario 100 of the audio playback control system via the network 140.

[0026] The storage device 150 is used to store data, instructions and / or any other information. For example, the storage device can store user's gesture data.

[0027] The storage device 150 may include one or more storage components, each of which may be an independent device or a part of another device. In some embodiments, the storage device 150 may include a random access memory (RAM), a read-only memory (ROM), a removable memory, etc. or any combination thereof. In some embodiments, the storage device 150 may be connected to the network 140 to communicate with one or more components in the application scenario 100 of the audio playback control system.

[0028] In some embodiments, the player 110, the processor 130, and the storage device 150 may be integrated into one or more devices. For example, the player 110, the processor 130, and the storage device 150 may be integrated into a smart player such as a smart speaker. In another example, the player 110 may be implemented based on a smart speaker, and the processor 130 and the storage device 150 may be implemented based on a cloud server.

[0029] It should be noted that the application scenario is provided only for the purpose of illustration and is not intended to limit the scope of this specification. For those of ordinary skill in the art, various modifications or changes can be made according to the description of this specification. For example, the application scenario may also include a database. For another example, the application scenario may be implemented on other devices to achieve similar or different functions. However, changes and modifications will not deviate from the scope of this specification.

[0030] Figure 2 is an exemplary module diagram of an audio playback control system according to some embodiments of this specification. Figure 2 As shown, the audio playback control system 200 includes a gesture recognition module 210 , a gesture mapping module 220 , a model training module 230 and a prediction module 240 .

[0031] The gesture recognition module 210 refers to a module for determining a gesture pattern. In some embodiments, the gesture recognition module 210 may be configured to collect gesture data of a user and determine a gesture pattern based on the gesture data.

[0032] The gesture mapping module 220 is a module for determining audio control instructions. In some embodiments, the gesture mapping module 220 can be configured to determine the audio control instructions based on the gesture pattern through a gesture mapping table, wherein the gesture mapping table includes a mapping relationship between one or more gesture patterns and one or more audio control instructions.

[0033] The model training module 230 refers to a module for training an instruction prediction model. In some embodiments, the model training module 230 can be configured to train an instruction prediction model based on historical operation data and historical control instructions; the instruction prediction model is a machine learning model.

[0034] The prediction module 240 refers to a module for predicting audio control instructions. In some embodiments, the prediction module 240 can be configured to predict audio control instructions based on gesture operation data including gesture data through an instruction prediction model, and adjust the gesture mapping table based on the gesture operation data and the predicted audio control instructions.

[0035] In some embodiments, Figure 2 As shown, the audio playing control system 200 may further include a data collection module 250 and a feedback module 260 .

[0036] The data collection module 250 is a module for acquiring gesture operation data. In some embodiments, the data collection module 250 may be configured to acquire gesture operation data, where the gesture operation data includes gesture data and gesture patterns and gesture durations corresponding to the gesture data.

[0037] Feedback module 260 refers to a module for updating the gesture mapping table. In some embodiments, feedback module 260 may be configured to periodically update the parameters of the instruction prediction model based on an update cycle, and update the gesture mapping table based on the updated instruction prediction model.

[0038] In some embodiments, the feedback module can be further configured to: determine a first prior intention based on the audio playback data and operation data sequence within a second preset time period; the first prior intention includes the probability distribution of the user's target gesture pattern under the target audio; and determine the update cycle based on the first prior intention.

[0039] In some embodiments, the feedback module can be further configured to: determine a second prior intention; the second prior intention includes a probability distribution of a target gesture pattern of the user under target playback conditions; and determine an update cycle based on the first prior intention and the second prior intention.

[0040] In some embodiments, the feedback module can be further configured to: determine the necessity of model update; in response to the necessity of model update satisfying a preset update condition, update the model parameters of the instruction prediction model based on the operation data sequence and the control instruction sequence of the first preset time period.

[0041] For more information about the gesture recognition module 210, the gesture mapping module 220, the model training module 230, the prediction module 240, the data collection module 250 and the feedback module 260, please refer to Figure 3-Figure 5 The corresponding description.

[0042] It should be noted that the above description of the audio playback control system 200 and its modules is only for convenience of description and does not limit the present specification to the scope of the embodiments. It is understandable that, after understanding the principle of the system, those skilled in the art may arbitrarily combine the modules or form a subsystem to connect with other modules without deviating from the principle. In some embodiments, Figure 2 The gesture recognition module 210, gesture mapping module 220, model training module 230, prediction module 240, data collection module 250 and feedback module 260 disclosed in the specification can be different modules in a system, or a module can realize the functions of two or more modules mentioned above. For example, each module can share a storage module, or each module can have its own storage module. Such variations are all within the protection scope of this specification.

[0043] Figure 3It is an exemplary flow chart of the audio playback control method according to some embodiments of this specification. In some embodiments, process 300 can be executed by the audio playback control system 200 or the processor 130, and the following takes the execution based on the processor 130 as an example.

[0044] Step 310 , collecting gesture data of the user, and determining a gesture mode based on the gesture data.

[0045] Gesture data refers to data related to gesture operations performed by users. Gesture operations refer to operations performed by users to interact with the audio playback control system through gestures.

[0046] In some embodiments, gesture data may include the motion trajectory and force of a user performing a gesture operation on an interactive interface (e.g., a control interface for playing audio). For example, gesture data may be represented based on a time series containing motion trajectory and force information, where the sequence contains the user's touch position and pressing force on the interactive interface at different time points.

[0047] In some embodiments, the processor may obtain gesture data in a variety of ways. For example, the processor may directly obtain gesture data from a device that provides an interactive interface (e.g., a touch screen, a touch pad, etc.). The processor may also obtain gesture data based on any other feasible method such as image recognition.

[0048] The gesture mode can reflect the motion characteristics of the gesture data. In some embodiments, the gesture mode can include but is not limited to sliding, clicking, long pressing, double clicking, slamming, zooming, etc.

[0049] In some embodiments, the processor can determine the gesture pattern in a variety of ways. For example, the processor can obtain the gesture pattern based on the gesture data through a gesture recognition algorithm. The gesture recognition algorithm may include but is not limited to a GestureDetector component, a hidden Markov model, etc.

[0050] For another example, the processor may pre-construct a gesture mode preset table, and determine the gesture mode by querying the gesture mode preset table based on the current gesture data. The gesture mode preset table may be constructed based on historical gesture data and historical gesture patterns. The gesture mode preset table includes the correspondence between historical gesture data and different historical gesture patterns.

[0051] Exemplarily, the corresponding relationship may be: when the movement speed of the touch point in a unit time is greater than the static speed threshold and it is a single-point touch, the gesture mode may be sliding. When the touch point is a single-point touch, the movement speed is less than the static speed threshold, and the pressing force is greater than the heavy pressing pressure threshold, the gesture mode may be double-click, etc. Among them, the unit time, static speed threshold, heavy pressing pressure threshold, etc. may be determined based on prior experience or manual input. The processor may determine the corresponding relationship between historical gesture data and historical gesture modes according to actual application scenarios and requirements.

[0052] Step 320: Based on the gesture pattern, determine the audio control instruction through a gesture mapping table.

[0053] Audio control instructions are instructions for adjusting audio parameters of an audio playback system. Audio parameters are basic parameters for an audio playback system to play audio. By setting different audio parameters, different audio playback effects can be achieved.

[0054] In some embodiments, the audio parameters may include volume parameters, equalizer parameters, etc. Accordingly, the audio control instructions may include volume control instructions, equalizer control instructions, etc.

[0055] The volume control instruction is a control instruction for adjusting the volume parameter. In some embodiments, the volume control instruction can be expressed as the adjustment amplitude and adjustment direction of the current volume, for example, increasing the volume by 10% or decreasing the volume by 3%. In other embodiments, the volume control instruction can also be expressed as the target volume that the audio playback system needs to achieve. For example, adjust the volume to 40dB.

[0056] The equalizer control instruction is a control parameter used to adjust the audio frequency response. By setting different equalizer parameters, the sound effect of the played audio can be adjusted. In some embodiments, the equalizer parameters may include adjustment amplitudes corresponding to different frequencies. In other embodiments, the equalizer control instruction may also be expressed as a target control parameter that the audio playback system needs to achieve, such as a target frequency.

[0057] The gesture mapping table refers to a mapping table for determining audio control instructions, and the gesture mapping table includes a mapping relationship between one or more gesture patterns and one or more audio control instructions. For example, the corresponding relationship may be: when the user long presses, the volume increases, when the user double-clicks, the volume decreases, etc.

[0058] In some embodiments, the processor can construct a gesture mapping table in a variety of ways. For example, the processor can construct a gesture mapping table based on the correspondence between the gesture mode assigned by the system by default and different audio control instructions. For another example, the processor can obtain the correspondence between the gesture mode input by the user and different audio control instructions to construct a gesture mapping table.

[0059] Step 330 , training an instruction prediction model based on historical operation data and historical control instructions.

[0060] Historical operation data refers to gesture operation data of historical periods. In some embodiments, the historical period can be determined according to actual application scenarios and requirements. For example, the past year, the past month, etc. For more information about gesture operation data, see Figure 4 Relevant part.

[0061] The command prediction model refers to a prediction model used to determine the audio control command. In some embodiments, the command prediction model can be a machine learning model. In some embodiments, the command prediction model can be any one or combination of a neural network (NN) model, a deep neural network (DNN) model, a recurrent neural network (RNN) model, or other custom model structures.

[0062] In some embodiments, the input of the instruction prediction model may be gesture operation data, and the output may be an audio control instruction corresponding to the gesture operation data.

[0063] In some embodiments, the processor may train the instruction prediction model based on a large number of first training samples with first labels. In some embodiments, the first training samples may include sample gesture operation data.

[0064] In some embodiments, the first label may be a sample audio control instruction corresponding to the first training sample. In some embodiments, the first training sample and the first label corresponding to the first training sample may be acquired based on historical data. Exemplarily, the processor may acquire the historical gesture operation data of the historical user at the first time point based on the historical data and use it as the first training sample, and determine the historical audio parameters of the historical user at the second time point as the target audio parameters in the historical audio control instruction, thereby obtaining the first label corresponding to the first training sample. The time interval between the second time point and the first time point does not exceed the preset interval threshold, the second time point is later than the first time point, and the second time point and the first time point are both historical time points.

[0065] In some embodiments, in order to avoid using the audio control instruction that is inconsistent with the actual control intention of the user during the process of adjusting the audio parameters as the first label, there should be no other historical audio control instructions or no historical audio control instructions with the same adjustment items within the preset time range after the second time point. This ensures that the historical audio control instructions at the second time point can meet the user's actual audio playback needs, helps reduce the user's frequent adjustment of audio parameters, and improves the user's experience.

[0066] In some embodiments, the processor can be trained by various methods based on the first training sample and the first label. For example, training can be performed based on the gradient descent method. As an example only, multiple first training samples with first labels can be input into the initial instruction prediction model, a loss function can be constructed by the first label and the result of the initial instruction prediction model, and the parameters of the initial instruction prediction model can be iteratively updated based on the loss function. When the loss function of the initial instruction prediction model meets the preset conditions, the model training is completed and a trained instruction prediction model is obtained. Among them, the preset conditions can be that the loss function converges, the number of iterations reaches a threshold, etc.

[0067] Step 340 , predicting the audio control instruction based on the gesture operation data including the gesture data through the instruction prediction model, and adjusting the gesture mapping table based on the gesture operation data and the predicted audio control instruction.

[0068] In some embodiments, the processor can input the gesture operation data into the trained instruction prediction model to obtain the audio control instruction output by the model. For more information about gesture operation data, please refer to Figure 4 and its related parts.

[0069] In some embodiments, the processor can adjust the gesture mapping table based on the gesture operation data and the predicted audio control instructions. For example, the processor can query the audio control instructions corresponding to the gesture operation data in the gesture mapping table and compare them with the audio control instructions predicted by the instruction prediction model. When the two do not match, the processor can update the audio control instructions stored in the gesture mapping table to the audio control instructions predicted by the instruction prediction model, thereby adjusting the gesture mapping table.

[0070] In some embodiments of the present specification, the user's gesture data is collected through a gesture recognition module, and the corresponding audio control instructions are determined through a gesture mapping table. At the same time, the gesture mapping table is continuously updated through an instruction prediction model. This can ensure the accuracy and reliability of the predicted audio control instructions, thereby reducing the frequency of users manually adjusting the audio control instructions, helping to achieve the continuity of audio playback, and enhancing the user's experience and sense of happiness.

[0071] It should be noted that the above description of process 300 is only for example and illustration, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to process 300 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0072] Figure 4 is an exemplary schematic diagram of updating a gesture mapping table according to some embodiments of the present specification.

[0073] In some embodiments, the processor may also obtain gesture operation data 410 based on the data collection module 250 of the audio playback control system 200 .

[0074] The gesture operation data 410 refers to data related to the gesture operation performed by the user. In some embodiments, the gesture operation data may include gesture data 421 and a gesture mode 422 and a gesture duration 423 corresponding to the gesture data.

[0075] The gesture duration 423 refers to the duration of a gesture mode, for example, long press for 2 seconds, etc. In some embodiments, the data collection module 250 can determine the start time and end time of different gesture modes based on the gesture data 421, and then determine the duration of the gesture mode.

[0076] For more information about gesture data 421 and gesture mode 422, see Figure 3 Related content.

[0077] In some embodiments, the processor may also periodically execute parameter updates of the instruction prediction model 440 based on an update cycle through the feedback module 260 of the audio playback control system 200 .

[0078] The update cycle refers to the time period used to update the parameters of the instruction prediction model 440 and the gesture mapping table. In some embodiments, the processor can determine the update cycle in a variety of ways. For example, the processor can preset the update cycle based on historical data or prior experience.

[0079] In some embodiments, the processor may further determine an update period based on the operation data sequence of the second preset period.

[0080] The operation data sequence refers to a sequence composed of a plurality of gesture operation data 410. For more information about the gesture operation data, please refer to the above.

[0081] The second preset period is a period used to determine the update cycle. The second preset period and the first preset period are both historical time periods, and the difference is that the length of the first preset period is not less than the second preset period.

[0082] In some embodiments, the first preset period may include a second preset period. For example, when the first preset period is the past week, the second preset period may be the past day. The length of the second preset period may be determined according to actual application scenarios and requirements.

[0083] The first preset period is a time period for collecting and updating the training data set, and the updated training data set is a data set of the training data 450 used to update the instruction prediction model 440. In some embodiments, the processor can determine the first preset period in a variety of ways. For example, the first preset period can be determined based on prior experience or historical data. For example, the past month, the past week, etc.

[0084] In some embodiments, the duration of the first preset period may also be positively correlated with the available computing power of the current audio playback control system. It is understandable that the stronger the computing power of the current playback control system and the lower the occupancy rate, the more computing power is available. Therefore, the duration of the first preset period may be appropriately increased, thereby increasing the amount of data for updating the training data set, which helps to improve the accuracy and robustness of the instruction prediction model 440. Conversely, when the available computing power is less, the duration of the first preset period may be appropriately reduced, which helps to reduce the computing power load and complete the update training of the instruction prediction model 440 in a shorter time.

[0085] In some embodiments, the processor may determine the update period in a variety of ways based on the sequence of operation data in the second preset period. The processor may divide the gesture operation data with the same gesture mode into the same group based on the sequence of operation data in the second preset period, and obtain n groups of gesture operation data. For the i-th group of gesture operation data, the processor may calculate the similarity between any two gesture operation data, and obtain a pair of gesture operation data with the highest similarity and the interval time T of the pair of gesture operation data. i The similarity calculation method may include but is not limited to the Manhattan distance method, the support vector machine (SVM) method, and the like.

[0086] In some embodiments, the processor may repeat the above similarity calculation method to obtain the interval durations of n pairs of gesture operation data with the highest similarity corresponding to n groups of gesture operation data, that is, to obtain T1, T2, ..., T n The processor can count the interval time T1 to T n The statistical value (for example, mean, median, mode, etc.) of the interval duration is determined as the update period. For example, the processor may determine the interval duration T1 to T n The majority of is determined as the currently required update period.

[0087] In some embodiments of the present specification, the update period is determined by the operation data sequence of the second preset time period, and the update period can be flexibly adjusted based on the user's gesture operation data, thereby helping to obtain an update period result that better meets the user's actual needs.

[0088] In some embodiments, the processor may determine a first a priori intention based on the audio playback data and the operation data sequence within a second preset time period, and determine an update period based on the first a priori intention.

[0089] The first prior intention refers to the probability distribution of the user performing different target gesture patterns when playing different target audios. For example, the first prior intention includes the probability distribution of the user's target gesture pattern under the target audio.

[0090] In some embodiments, the target audio may be the audio currently being played, and the target gesture mode may be the gesture mode of the user when the target audio is being played. For more information about the gesture mode, please refer to the above description.

[0091] In some embodiments, the first prior intent can be expressed as a probability distribution sequence of the user executing different gesture modes when playing different target audios. For example, the first prior intent can be: (target audio A; slide 20%, click 50%, long press 30%), indicating that when the user plays target audio A, the probability of executing a sliding gesture is 20%, the probability of executing a click gesture is 50%, and the probability of executing a long press gesture is 30%.

[0092] The audio playback data can reflect the relevant features of the audio played by the user. In some embodiments, the audio playback data can include the number of audio played by the user and the category corresponding to each audio. In some embodiments, the audio playback data can also include the audio features of the audio played by the user, the playback scene, the audio parameters of the audio playback system, etc.

[0093] In some embodiments, the processor can determine the first prior intention of the target audio in a variety of ways. For example, the processor can build a first vector database based on historical data and determine the first prior intention of the target audio based on vector retrieval.

[0094] In some embodiments, the processor may construct a first feature vector based on the audio playback data within a second preset time period. The elements of the first feature vector may include the audio playback data within the second preset time period, the gesture operation data within the second preset time period, and the target audio at the current moment. The gesture operation data within the second preset time period can be acquired based on the operation data sequence within the second preset time period. There are many ways to construct the first feature vector. For example, the first feature vector is constructed by TF-IDF (Term Frequency-Inverse Document Frequency), One-Hot, Word2Vec and other methods.

[0095] The first vector database may include multiple first reference vectors and corresponding reference gesture operation data. The first reference vector may be constructed based on historical data. The reference gesture operation data corresponding to the first reference vector may be: historical gesture operation data of the last played historical target audio in the historical audio playback data of the first reference vector.

[0096] In some embodiments, in order to avoid the same audio being constructed as the first reference vector and the first feature vector respectively, the time span of the historical period has no intersection with the time span of the second preset period. The historical period can be preset based on prior experience or actual needs. The construction method of the first reference vector is similar to that of the first feature vector. For example, the processor can construct the audio playback data in the second historical period, the gesture operation data in the second historical period, and the historical target audio at the first historical time point as the first reference vector, and the reference gesture operation data can be constructed based on the historical gesture operation data corresponding to the first reference vector.

[0097] In some embodiments, the processor may determine sample gesture operation data based on the similarity between the first feature vector and a plurality of first reference vectors in the first vector database. For example, a first reference vector whose similarity with the first feature vector satisfies a similarity condition is determined as a target vector, and the reference gesture operation data corresponding to the target vector and the gesture operation data within a second preset time period are used together as sample gesture operation data. The similarity condition may be set according to circumstances. For example, the similarity is greater than a first similarity threshold, etc. The first similarity threshold may be determined based on actual application scenarios and requirements.

[0098] In some embodiments, the processor can determine the first prior intention of the target audio based on the gesture mode corresponding to the sample gesture operation data. For example only, when the target audio A is played historically or the historical audio similar to the target audio A is played, there are 5 corresponding sample gesture operation data, and the corresponding gesture modes are slide, slide, thump, thump, and zoom. When the target audio A is played, the proportion of sliding gestures is 40%, the proportion of thumping gestures is 40%, and the proportion of zooming is 20%. Then the first prior intention when playing the target audio A can be expressed as: (target audio A: slide 40%, thump 40%, zoom 20%).

[0099] In some embodiments, the processor may determine the update period based on the first a priori intent.

[0100] In some embodiments, the update period may be positively correlated to the fluctuation amplitude of the probabilities of all gesture patterns within the first priori intention and the shortest update period.

[0101] The fluctuation range of the probability of all gesture modes in the first prior intention can reflect the fluctuation of the probability distribution of executing different gesture modes in the first prior intention. The fluctuation range can be represented by a numerical value. The smaller the numerical value, the more average the probabilities corresponding to the various gesture modes in the first prior intention, and it is difficult to determine the user's true intention through the first prior intention. The lower the reference significance of the first prior intention, the more the instruction prediction model 440 needs to be updated as soon as possible.

[0102] In some embodiments, the processor may calculate the standard deviation and mean of all probabilities of executing different gesture patterns within the first prior intention as the fluctuation range of the probability of all gesture patterns within the first prior intention. For example, the fluctuation range of the probability of all gesture patterns within the first prior intention may be:

[0103] The shortest update cycle refers to the minimum update cycle of the update instruction prediction model 440. In some embodiments, the shortest update cycle may be negatively correlated to the available computing power of the current audio playback control system. It is understood that the more sufficient the available computing power of the current audio playback control system is, the more computing power resources can be used to update the instruction prediction model 440, and the instruction prediction model 440 can be updated frequently, thereby shortening the shortest update cycle.

[0104] For the purpose of convenience of description and explanation only, the update period can be obtained based on the following formula (1):

[0105] Among them, G represents the update period, represents the default update cycle, α represents the shortest update cycle coefficient, β represents the fluctuation range of the probability of all gesture patterns in the first prior intention, and k1 is a coefficient greater than 0. Among them, the shortest update cycle coefficient α can be expressed as The value of coefficient k1 can be preset based on prior experience. Default update cycle It can be the original update cycle, such as the system default update cycle.

[0106] In some embodiments of the present specification, since the distribution probability of gesture patterns can reflect the ability to accurately predict the user's true intention to a certain extent, if the distribution probability of gesture patterns tends to be average, it means that the uncertainty of the current information is relatively large. For example, if the sliding probability is 25%, the clicking probability is 25%, the double-clicking probability is 25%, and the zooming probability is 25%, it is equivalent to being unable to determine the user's true intention to perform touch operations. Therefore, when the fluctuation range of the probability of all gesture patterns in the first prior intention is smaller, the update cycle can be appropriately shortened, which helps to obtain a more suitable update cycle and achieve a balance between computing power consumption and the rationality of the update cycle.

[0107] In some embodiments, the processor may determine a second a priori intent, and determine an update period based on the first a priori intent and the second a priori intent.

[0108] The second prior intention refers to the probability distribution of the user performing different target gesture operations under different target playback conditions. For example, the second prior intention includes the probability distribution of the user's target gesture mode under the target playback condition. The second prior intention is similar to the first prior intention in composition, except that the first prior intention is mainly used to reflect the impact of different target audio on the target gesture mode, while the second prior intention is mainly used to reflect the impact of different target playback conditions on the target gesture mode.

[0109] The target playback condition may reflect the audio parameters when the user plays the target audio. In some embodiments, the target playback condition may include the audio parameters when the user plays the target audio. The audio parameters may include at least one of a volume parameter, an equalizer parameter, and an environmental space feature.

[0110] The environmental space feature can reflect the spatial feature of the user when listening to the audio. In some embodiments, the environmental space feature can include whether the user is indoors or outdoors, and the environmental scene when the user listens to the audio (for example, a cooking environment, a leisure and entertainment environment, a driving environment). The processor can obtain the environmental space feature based on obtaining the user's input or any other feasible method. For more descriptions of environmental scenes, see Figure 5 More description of the historical and current operating environments.

[0111] In some embodiments, the processor may determine the second prior intention based on the target playback condition within a third preset period. The third preset period is similar to the second preset period, both are historical time periods, and the length of the third preset period is less than the first preset period. The third preset period may be determined based on prior experience or manual input.

[0112] In some embodiments, the processor may determine the second prior intention in a variety of ways. For example, the processor may refer to the method for determining the first prior intention above, build a second vector database based on historical data, and determine the second prior intention based on matching vector retrieval.

[0113] In some embodiments, the processor may construct a second feature vector based on the playback conditions within a third preset time period and the target playback conditions at the current moment. The second vector database may include multiple second reference vectors and corresponding reference gesture operation data. The processor may determine the reference gesture operation data corresponding to the second reference vector whose similarity with the second feature vector is greater than the second similarity threshold as sample gesture operation data. The processor may determine the second prior intention based on the sample gesture operation data.

[0114] For the specific content of this part of the operation, please refer to the above description of constructing the first reference vector, performing vector matching in the first vector database, and finally determining the first prior intention.

[0115] In some embodiments, the second similarity threshold may be the same as or different from the first similarity threshold, and the value of the second similarity threshold may be determined based on actual application scenarios and requirements.

[0116] In some embodiments, the processor may determine the update cycle based on the first a priori intent and the second a priori intent. For more description of the first a priori intent, please refer to the above related content.

[0117] In some embodiments, the update period may be positively correlated to the fluctuation amplitude of the probability of all gesture patterns in the first prior intention, the fluctuation amplitude of the probability of all gesture patterns in the second prior intention, and the shortest update period. The fluctuation amplitude of the probability of all gesture patterns in the second prior intention is determined in a similar manner to the fluctuation amplitude of the probability of all gesture patterns in the first prior intention. For more information on determining the fluctuation amplitude of the probability of all gesture patterns in the first prior intention and the shortest update period, please refer to the relevant content above.

[0118] For the purpose of convenience of description and explanation only, the update period can also be obtained based on the following formula (2):

[0119] Where γ represents the fluctuation range of the probability of all gesture patterns in the second prior intention, k2, k3 and k4 are coefficients greater than 0, and k2+k3+α=1. The values ​​of coefficients k2 and k3 can be preset based on prior experience. Regarding the update cycle G, the default update cycle For more information about the shortest update cycle coefficient α and the fluctuation range β of the probability of all gesture patterns in the first prior intention, please refer to the relevant description above. The processor can determine the fluctuation range γ of the probability of all gesture patterns in the second prior intention with reference to the relevant steps of determining the fluctuation range β of the probability of all gesture patterns in the first prior intention.

[0120] In some embodiments of the present specification, by measuring the distribution probability of different gesture modes under the current playback conditions, the relevant effects of different playback conditions and different types of audio on gesture operations can be comprehensively considered, and the first prior intention, the second prior intention, and the gesture mode can be selected. Figure 1 At the same time, the update cycle can be adjusted to reasonably optimize the update cycle.

[0121] In some embodiments, the update period is also related to the historical operating environment and the current operating environment.

[0122] The historical operating environment refers to the environment scene in which the user listened to the audio in the historical time period. In some embodiments, the historical operating environment may include but is not limited to a cooking environment, a leisure and entertainment environment, a driving environment, and the like.

[0123] The current operating environment refers to the environment in which the user listens to the audio at the current moment. The current operating environment is determined in a similar manner to the historical operating environment, except that the corresponding time range is different.

[0124] It is understandable that the frequency of users adjusting audio playback varies in different operating environments. For example, users make fewer adjustments during cooking, but may be more inclined to make frequent adjustments when they are entertaining. Therefore, the update cycle will also be affected by the historical operating environment and the current operating environment. For more information on determining the historical operating environment and the current operating environment, see Figure 5 Relevant part.

[0125] In some embodiments, the processor may determine the environment similarity based on the historical operating environment and the current operating environment, and adjust the update period based on the extent to which the environment similarity deviates from an environment similarity threshold.

[0126] The environmental similarity can reflect the similarity between the historical operating environment and the current operating environment. In some embodiments, the environmental similarity can be represented by a numerical value. The larger the numerical value, the greater the similarity between the historical operating environment and the current operating environment.

[0127] In some embodiments, the processor can determine the similarity between the historical operating environment and the current operating environment in a variety of ways. For example, the processor can determine the similarity between the historical operating environment and the current operating environment by querying a pre-constructed environment similarity determination table based on the historical operating environment and the current operating environment. The environment similarity determination table stores the similarity relationship between different operating environments. The environment similarity determination table can be constructed based on prior experience or manual preset. For another example, the processor can determine the environment similarity based on an environment analysis model. The environment analysis model can be an existing or self-trained and learning model.

[0128] The environment similarity threshold is a threshold used to determine whether the historical operating environment and the current operating environment are similar. In some embodiments, the environment similarity threshold can be determined according to actual application scenarios and requirements.

[0129] In some embodiments, the environment similarity threshold may be positively correlated to the average change amplitude of the gesture operation frequency. The processor may count the mean and standard deviation of the frequency of the user executing different gesture modes within a preset time range, and calculate the average change amplitude of the execution frequency of different gesture modes. For example, the average change amplitude of the execution frequency of different gesture modes may be expressed as the mean change amplitude of the execution frequency of different gesture modes.

[0130] For the purpose of convenience of description and explanation only, the update period can also be obtained based on the following formula (3):

[0131] Where S represents the environmental similarity between the first operating environment and the second operating environment, and Y represents the environmental similarity threshold. For more information about coefficients k2 and k3, please refer to the relevant description above.

[0132] In some embodiments of the present specification, when the environmental similarity between the historical operating environment and the current operating environment exceeds the environmental similarity threshold, it means that the operating environments of the historical operating environment and the current operating environment are very similar, indicating that the environment in which the user listens to the audio has not changed significantly, so the update cycle can be appropriately increased to reduce the consumption of computing power. When the environmental similarity is less than the environmental similarity threshold, it means that the environment in which the user listens to the audio has changed significantly, so the update cycle can be appropriately shortened to timely update the instruction prediction model 440 and the gesture mapping table 460.

[0133] The operation data sequence refers to a data sequence consisting of at least one gesture operation data 410. The control instruction sequence refers to a data sequence consisting of at least one audio control instruction 430. The updated training data set may include the operation data sequence and the control instruction sequence.

[0134] Parameter updating refers to the operation step of updating the parameters of the instruction prediction model 440. In some embodiments, the processor can determine the updated training samples and their labels in the training data 450 based on the operation data sequence and the control instruction sequence of the first preset period. Exemplarily, each gesture operation data and the corresponding control instruction sequence contained in the operation data sequence of the first preset period can be used as an updated training sample and its label. Thus, at least one set of updated training samples and their labels can be obtained.

[0135] In some embodiments, the processor may update and train the current instruction prediction model 440 based on the updated training samples and their labels to obtain an updated instruction prediction model 451. For the specific process of updating and training the instruction prediction model 440, see Figure 3 A description of a prediction model based on a first training sample and a first label training instruction.

[0136] In some embodiments, the processor may update the gesture mapping table 460 based on the updated instruction prediction model 451 through the feedback module 260 of the audio playback control system 200 , thereby obtaining an updated gesture mapping table 461 .

[0137] For more information on how to update the gesture mapping table, see the relevant content above.

[0138] In some embodiments of the present specification, by periodically updating the parameters of the instruction prediction model through an update cycle, it can be ensured that the audio control instructions 430 output by the instruction prediction model always meet the actual needs of the user, which helps to ensure the accuracy and reliability of the output results of the instruction prediction model; and by utilizing the adaptive learning mechanism of the instruction prediction model, unnecessary instruction transmission is reduced, the system delay caused by frequent transmission of instructions is reduced, and the response speed and overall performance of the device are improved.

[0139] Figure 5 It is an exemplary schematic diagram of updating the instruction prediction model based on the necessity of model update according to some embodiments of this specification.

[0140] In some embodiments, the processor may determine the necessity of model update 510; in response to the necessity of model update 510 satisfying a preset update condition 520, the model parameters of the instruction prediction model are updated based on the operation data sequence and the control instruction sequence of the first preset time period.

[0141] Model update necessity 510 may be used to measure the degree of necessity of updating the instruction prediction model. In some embodiments, model update necessity may be represented by a numerical value, for example, a larger numerical value of model update necessity indicates a higher necessity of updating the instruction prediction model.

[0142] The preset update condition 520 refers to a judgment condition for judging whether the model parameters need to be updated. In some embodiments, the preset update condition may be that the necessity of model update is greater than the update necessity threshold. The value of the update necessity threshold may be determined according to the actual application scenario and requirements.

[0143] In some embodiments, the processor may determine the necessity of model update 510 in a variety of ways. For example, the processor may determine the necessity of model update based on the time since the last model update.

[0144] In some embodiments, the model update necessity 510 may be positively correlated to the time L since the last model update.

[0145] For the purpose of convenience of description and explanation only, the necessity of model update can also be obtained based on the following formula (4):

[0146] Among them, B represents the necessity of model update, L represents the time since the last model update, and G represents the update cycle. For more information about the update cycle G, please refer to the above description.

[0147] In some embodiments, when the model update necessity 510 satisfies the preset update condition 520, the processor may update the model parameters of the instruction prediction model based on the operation data sequence and the control instruction sequence of the first preset period. For more information about this part, please refer to Figure 4 and related instructions.

[0148] In some embodiments, the model update necessity 510 may also be related to the operation data sequence and control instruction sequence of the first preset period, and the model parameters of the current instruction prediction model.

[0149] In some embodiments, the processor may construct a sample instruction distribution based on an operation data sequence and a control instruction sequence in a first preset time period.

[0150] The sample command distribution can reflect the probability distribution of executing different audio control commands when the user inputs a specific gesture operation. In some embodiments, the sample command distribution can be expressed as a probability sequence of different audio control commands corresponding to different gesture modes, for example, (slide: increase volume by 20%, decrease volume by 80%; double-click: switch to the previous audio by 40%, switch to the next audio by 60%).

[0151] In some embodiments, the processor can construct the sample instruction distribution in a variety of ways. For example, for gesture operation data with the same gesture mode, the processor can count the types and quantities of actual audio control instructions corresponding to different gesture modes within the first preset time period, and count the proportion of different types of audio control instructions in all audio control instructions to obtain the sample instruction distribution. The sample instruction distribution of all gesture modes can be determined by repeating the above method.

[0152] Exemplarily, within the first preset time period, after the user performs one or more gesture operations with the gesture mode of a heavy tap, there are a total of 10 actual audio control instructions, of which 3 audio control instructions are for doubling the volume and 7 audio control instructions are for muting. For the heavy tap gesture, the proportion of audio control instructions for doubling the volume is 30%, and the proportion of audio control instructions for muting is 70%. The distribution of sample instructions corresponding to the heavy tap gesture is (heavy tap: volume doubling 30%, muting 70%). In some embodiments, the processor can obtain several historical audio control instructions after the user performs a gesture operation, and determine the historical audio control instruction with the longest use time as the actual audio control instruction corresponding to the gesture operation.

[0153] In some embodiments, the processor may construct a distribution database based on historical training data of the instruction prediction model.

[0154] The historical training data refers to the historical data related to the updated training instruction prediction model. In some embodiments, the historical training data may include historical training samples, historical training labels, and trained model parameters for each updated training instruction prediction model.

[0155] In some embodiments, the processor may construct at least one set of historical instruction distributions by referring to the method of constructing sample instruction distributions. A set of historical instruction distributions corresponds to a historical training sample and its corresponding historical training label for updating a training instruction prediction model.

[0156] The distribution database refers to a database used for similarity matching with the sample instruction distribution. In some embodiments, the distribution database may store several groups of historical instruction distributions and their corresponding trained model parameters.

[0157] In some embodiments, the processor may perform similarity matching with several groups of historical instruction distributions in the distribution database based on the sample instruction distribution, and select the historical instruction distribution whose similarity satisfies the similarity preset condition as the target instruction distribution. The trained model parameters corresponding to the target instruction distribution are used as the target model parameters. The similarity preset condition can be set according to the situation. For example, the similarity is maximum, or the similarity is greater than a threshold, etc. For another example, the vector distance is minimum. The similarity matching algorithm may include but is not limited to the Manhattan distance method, the SVM method, etc.

[0158] In some embodiments, the processor may calculate the parameter similarity between the target model parameters and the current model parameters based on the target model parameters and the model parameters of the current instruction prediction model, and determine the necessity of model updating based on the parameter similarity, referring to the above-mentioned similarity matching algorithm.

[0159] In some embodiments, model update necessity 510 may be negatively correlated to parameter similarity.

[0160] In some embodiments of the present specification, the necessity of model update is determined by the operation data sequence and control instruction sequence of the first preset period and the model parameters of the current instruction prediction model. When the change range of the instruction prediction model parameters is small, it means that the user's gesture operation habits are relatively stable, so the necessity of model update can be appropriately reduced. When the change range of the instruction prediction model parameters is large, it means that the user's gesture operation habits change frequently, so the necessity of model update can be appropriately increased, thereby realizing flexible adjustment of the necessity of model update.

[0161] In some embodiments, the processor can obtain historical environment images of historical time periods and current environment images at the current time point; determine the historical operating environment and the current operating environment based on the historical environment images and the current environment images; and determine the necessity of model updating based on the historical operating environment and the current operating environment.

[0162] The environment image refers to the image related to the environment in which the user plays the audio. In some embodiments, the environment image can be divided into a historical environment image of a historical period and a current environment image at a current time point.

[0163] In some embodiments, the current environment image at the current time point can be obtained in a variety of ways. For example, the processor can obtain the current environment image uploaded by the user. The processor can also obtain the current environment image based on any other feasible method. The historical environment image can be obtained based on historical data.

[0164] For more information about the historical operating environment and the current operating environment, see Figure 5 Relevant part.

[0165] In some embodiments, the processor can determine the historical operating environment based on the historical environment image and the current operating environment based on the current environment image in a variety of ways. For example, the processor can obtain the operating environment corresponding to the environment image through an image recognition method. Among them, the image recognition method may include but is not limited to a deep neural network method, an environmental semantic analysis method, etc. For another example, the processor can pre-construct several different template operating environments and corresponding template environment images. Through a similarity matching method, determine the template environment image that is most similar to the environment image. The processor can determine the template operating environment corresponding to the template environment image as the operating environment corresponding to the environment image.

[0166] In some embodiments, the processor may determine the necessity of model update based on the historical operating environment and the current operating environment. For example, the processor may calculate the environment similarity between the historical operating environment and the current operating environment, and determine the necessity of model update based on the environment similarity.

[0167] The processor may determine the environmental similarity in a variety of ways. For example, the processor may determine the environmental similarity based on an environmental similarity determination table or an environmental analysis model. For more information about the environmental similarity determination table or the environmental analysis model, see Figure 5 Relevant part.

[0168] In some embodiments, the model update necessity may be correlated to the environment similarity. For example, the model update necessity may be negatively correlated to the environment similarity.

[0169] In some embodiments of the present specification, the historical operating environment and the current operating environment are determined based on the historical environment image and the current environment image; and the necessity of model update is determined based on the historical operating environment and the current operating environment. When determining the necessity of model update, the influence of the environment in which the user plays the audio can be comprehensively considered. When the change in the user's environment is small, the necessity of model update can be appropriately reduced. When the change in the user's environment is large, it means that the original instruction prediction model may no longer be able to accurately predict the audio control instruction, and the necessity of model update can be appropriately increased to update the instruction prediction model in time.

[0170] In some embodiments, the necessity of model updating is also related to the first a priori intention and the second a priori intention.

[0171] In some embodiments, the processor may determine the fluctuation range of the probability of all gesture patterns within the first prior intent and the fluctuation range of the probability of all gesture patterns within the second prior intent based on the first prior intent and the second prior intent. Based on the fluctuation range of the probability of all gesture patterns within the first prior intent and the fluctuation range of the probability of all gesture patterns within the second prior intent, the necessity of model update is adjusted. For more descriptions of the first prior intent, the second prior intent, and the fluctuation range of the probability of all gesture patterns within the first prior intent and the fluctuation range of the probability of all gesture patterns within the second prior intent, please refer to Figure 4 Relevant part.

[0172] In some embodiments, the adjusted model update necessity may be positively correlated to the fluctuation range of the first prior intention and the fluctuation range of the second prior intention.

[0173] For the purpose of convenience of description and explanation only, the adjusted model update necessity can also be obtained based on the following formula (5):

[0174] in, represents the necessity of model update after adjustment, B represents the necessity of model update, β represents the fluctuation range of the probability of all gesture patterns in the first prior intention, γ represents the fluctuation range of the probability of all gesture patterns in the second prior intention, and k4 and k5 are coefficients greater than 0. Among them, the values ​​of coefficients k4 and k5 can be preset based on prior experience. The necessity of model update B can be the necessity of model update before adjustment, such as the necessity of model update calculated based on formula (4), the necessity of model update based on environmental similarity, the necessity of model update based on parameter similarity, etc. For more information on how to determine the fluctuation range β of the probability of all gesture patterns in the first prior intention and the fluctuation range γ of the probability of all gesture patterns in the second prior intention, please refer to Figure 4 Relevant part.

[0175] In some embodiments of the present specification, adjusting the necessity of model updating through the first prior intention and the second prior intention is beneficial to maintaining the update frequency of the instruction prediction model 440 and the gesture mapping table 460 at a relatively similar level, avoiding a large difference between the update frequency of the gesture mapping table 460 and the update frequency of the instruction prediction model, thereby helping to improve the integrity and unity of the instruction prediction model and the gesture mapping table, and comprehensively improve the accuracy and reliability of predicting audio control instructions.

[0176] In some embodiments, the processor may update the model parameters of the instruction prediction model based on the operation data sequence and the control instruction sequence of the first preset period. For more information about this part, please refer to Figure 4 and related instructions.

[0177] In some embodiments of the present specification, since updating the parameters of the instruction prediction model often requires a lot of computing power and time costs, the model parameters of the instruction prediction model are updated in response to the necessity of model update meeting the preset update conditions. This can avoid parameter updates when the current instruction prediction model is capable of performing the task, thereby achieving a balance between computing power costs and model prediction accuracy.

[0178] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0179] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0180] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0181] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0182] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0183] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0184] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. An audio playback control system, comprising: a gesture recognition module configured to collect gesture data of a user and determine a gesture pattern based on the gesture data; a gesture mapping module, configured to determine the audio control instruction through a gesture mapping table based on the gesture pattern, wherein the gesture mapping table includes a mapping relationship between one or more gesture patterns and one or more audio control instructions; A model training module is configured to train an instruction prediction model based on historical operation data and historical control instructions; the instruction prediction model is a machine learning model; The prediction module is configured to predict the audio control instruction based on gesture operation data including the gesture data through the instruction prediction model, and adjust the gesture mapping table based on the gesture operation data and the predicted audio control instruction.

2. The system according to claim 1, characterized in that The system further comprises: A data collection module is configured to obtain the gesture operation data, wherein the gesture operation data includes the gesture data and the gesture mode and gesture duration corresponding to the gesture data; A feedback module is configured to periodically perform parameter updates of the instruction prediction model based on an update cycle, and update the gesture mapping table based on the updated instruction prediction model; The parameter updating includes: updating the training data of the instruction prediction model based on the operation data sequence and the control instruction sequence of the first preset time period to obtain the updated instruction prediction model.

3. The system according to claim 2, characterized in that The update period is determined based on an operation data sequence of a second preset period, and the duration of the first preset period is not less than the duration of the second preset period.

4. The system according to claim 3, characterized in that The feedback module is further configured to: Determine a first priori intention based on the audio playback data and the operation data sequence within the second preset time period; the first priori intention includes a probability distribution of a target gesture pattern of the user under the target audio; The update period is determined based on the first a priori intention.

5. The system according to claim 4, characterized in that The feedback module is further configured to: Determining a second prior intention; the second prior intention includes the probability distribution of the target gesture pattern of the user under the target playback condition; The update period is determined based on the first a priori intent and the second a priori intent.

6. The system according to claim 2, characterized in that The feedback module is further configured to: Determine the need for model updates; In response to the necessity of the model update satisfying a preset update condition, the model parameters of the instruction prediction model are updated based on the operation data sequence and the control instruction sequence of the first preset time period.

7. An audio playback control method, characterized in that: include: Collecting gesture data of a user, and determining a gesture mode based on the gesture data; Based on the gesture pattern, determining the audio control instruction through a gesture mapping table, wherein the gesture mapping table includes a mapping relationship between one or more gesture patterns and one or more audio control instructions; Based on historical operation data and historical control instructions, training an instruction prediction model; the instruction prediction model is a machine learning model; The audio control instruction is predicted based on the gesture operation data including the gesture data through the instruction prediction model, and the gesture mapping table is adjusted based on the gesture operation data and the predicted audio control instruction.

8. The method according to claim 7, characterized in that The method further comprises: Acquire the gesture operation data, where the gesture operation data includes the gesture data and the gesture mode and gesture duration corresponding to the gesture data; Based on an update cycle, periodically performing parameter updates of the instruction prediction model, and updating the gesture mapping table based on the updated instruction prediction model; The parameter updating includes: updating the training data of the instruction prediction model based on the operation data sequence and the control instruction sequence of the first preset time period to obtain the updated instruction prediction model.

9. The method according to claim 8, characterized in that The update period is determined based on an operation data sequence of a second preset period, and the duration of the first preset period is not less than the duration of the second preset period.

10. The method according to claim 9, characterized in that The method further comprises: Determine a first priori intention based on the audio playback data and the operation data sequence within the second preset time period; the first priori intention includes a probability distribution of a target gesture pattern of the user under the target audio; The update period is determined based on the first a priori intention.

11. The method according to claim 10, characterized in that The determining the update period based on the first prior intention further comprises: Determining a second prior intention; the second prior intention includes the probability distribution of the target gesture pattern of the user under the target playback condition; The update period is determined based on the first a priori intent and the second a priori intent.

12. The method according to claim 8, characterized in that The parameter update includes: Determine the need for model updates; In response to the necessity of the model update satisfying a preset update condition, the model parameters of the instruction prediction model are updated based on the operation data sequence and the control instruction sequence of the first preset time period.

13. An audio playback control device, characterized in that: The apparatus comprises at least one processor and at least one memory; The at least one memory is used to store computer instructions; The at least one processor is configured to execute at least part of the computer instructions to implement the method according to any one of claims 7 to 12.

14. A computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the method according to any one of claims 7 to 12.

Images