Sound playing control method, system and device and storage medium

Abstract

The embodiment of the invention provides a sound equipment playing control method. The method comprises the following steps: acquiring environment information of a monitoring environment in real time or at certain time intervals; determining at least one object and motion information thereof in the monitoring environment based on the environment information; in response to the movement information that movement occurs and the movement distance is greater than a threshold value, executing the following operations on the at least one sound box: based on the at least one object and the movement information thereof, determining at least one group of candidate adjustment amounts of sound box parameters of each sound box in the at least one sound box; determining at least one candidate scheme based on the at least one group of candidate adjustment amounts; performing at least one round of iteration on the at least one candidate scheme until a preset updating condition is met; determining a target scheme based on the at least one round of iteration result; and determining a target adjustment amount of the sound parameters of the sound based on the target scheme.

Description

Case Analysis

[0001] This application is a divisional application of Chinese application filed on December 2, 2022, with application number 202211536644.3 and invention title "A Method for Adjusting Audio Parameters". Technical Field

[0002] This specification relates to the field of intelligent speaker technology, and in particular to a speaker playback control method, system, device, and storage medium. Background Technology

[0003] With the rapid development of digitalization and intelligence, people have higher and higher requirements for the audio-visual experience brought by smart homes. As a common playback device, audio equipment has been widely used in people's daily entertainment activities. How to adjust audio equipment so that users can have a good listening experience when listening to information in different environments has become an urgent problem to be solved.

[0004] Therefore, it is desirable to provide a method for adjusting audio parameters that can adaptively adjust audio parameters based on the real-time usage status of one or more users, thereby optimizing the audio-visual experience for users in different environments and providing better intelligent audio services for users. Summary of the Invention

[0005] This specification provides one or more embodiments of a sound playback control method, comprising: acquiring environmental information of a monitored environment in real time or at intervals; determining at least one object in the monitored environment and its motion information based on the environmental information; in response to the motion information indicating movement and the movement distance being greater than a threshold, performing the following operations on at least one sound: determining at least one set of candidate adjustment values ​​for the sound parameters of each of the at least one sound based on the at least one object and its motion information; determining at least one candidate scheme based on the at least one set of candidate adjustment values; performing at least one iteration on the at least one candidate scheme until a preset update condition is met; and performing at least one iteration on the at least one candidate scheme until a preset update condition is met. The iteration results of each round determine the target scheme; based on the target scheme, the target adjustment amount of the audio parameters of the audio system is determined; wherein, each round of iteration in the at least one round includes: performing a preset change processing on the candidate schemes of the current round to obtain a first alternative scheme; in the first round of iteration, the candidate schemes of the current round are determined based on the at least one set of candidate adjustment amounts; the preset change processing includes at least one of crossover processing, mutation processing, and replication processing; based on a preset evaluation algorithm, the evaluation value of the first alternative scheme and the evaluation value of the candidate schemes of the current round are determined; based on the evaluation value of the first alternative scheme and the evaluation value of the candidate schemes of the current round, the candidate schemes to enter the next round of iteration are determined.

[0006] This specification provides one or more embodiments of an audio playback control system, comprising: an acquisition module configured to acquire environmental information of a monitored environment at regular intervals or in real time; a first determination module configured to determine at least one object in the monitored environment and its motion information based on the environmental information; and an execution module configured to, in response to the motion information satisfying a first preset condition, perform the following operations on at least one audio device: determining at least one set of candidate adjustment values ​​for the audio parameters of each of the at least one audio device based on the at least one object and its motion information; determining at least one candidate scheme based on the at least one set of candidate adjustment values; and performing at least one iteration on the at least one candidate scheme until a preset update condition is met. The conditions are met; a target solution is determined based on the results of at least one round of iterations; a target adjustment amount for the audio parameters of the audio system is determined based on the target solution; wherein each round of iteration includes: performing a preset change process on the candidate solutions of the current round to obtain a first alternative solution; in the first round of iteration, the candidate solutions of the current round are determined based on at least one set of candidate adjustment amounts; the preset change process includes at least one of crossover processing, mutation processing, and replication processing; the evaluation value of the first alternative solution and the evaluation value of the candidate solutions of the current round are determined based on a preset evaluation algorithm; and a candidate solution to enter the next round of iteration is determined based on the evaluation value of the first alternative solution and the evaluation value of the candidate solutions of the current round.

[0007] This specification provides one or more embodiments of an audio playback control device, the device including at least one processor and at least one memory; the at least one memory is used to store computer instructions; the at least one processor is used to execute at least a portion of the computer instructions to implement the aforementioned audio playback control method.

[0008] This specification provides one or more embodiments of a computer-readable storage medium that stores computer instructions that, when executed by a processor, implement the aforementioned audio playback control method. Attached Figure Description

[0009] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein: Figure 1 This is a schematic diagram illustrating an application scenario of the audio parameter adjustment system according to some embodiments of this specification; Figure 2 This is an exemplary flowchart of a method for adjusting audio parameters according to some embodiments of this specification; Figure 3This is a schematic diagram illustrating the process of determining the main speaker list according to some embodiments of this specification; Figure 4 These are exemplary schematic diagrams illustrating the determination of experience based on an experience determination model according to some embodiments of this specification; Figure 5 This is a schematic diagram illustrating the process of determining the target adjustment amount of the audio parameters of a speaker according to some embodiments of this specification; Figure 6 This is an exemplary schematic diagram illustrating the iteration of candidate solutions according to some embodiments of this specification; Figure 7 This is a schematic diagram of a sound parameter adjustment system according to some embodiments of this specification. Detailed Implementation

[0010] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0011] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0012] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0013] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0014] Figure 1 This is a schematic diagram of an application scenario 100 of an audio parameter adjustment system according to some embodiments of this specification.

[0015] like Figure 1 As shown, the application scenario 100 of the audio parameter adjustment system may include a processing device 110, a storage device 130, an image acquisition device 140, a network 150, and a monitoring environment 120. The processing device 110 can acquire environmental information of the monitoring environment 120 through the network 150 and the image acquisition device 140, determine at least one object 120-3 in the monitoring environment 120 and its motion information, and then determine the audio parameters and target adjustment amount of each of the at least one audio device 120-2.

[0016] Processing device 110 can process data and / or information acquired from storage device 130 and / or image acquisition device 140. In some embodiments, processing device 110 may include a computer, user console, a single server, or a group of servers. In some embodiments, processing device 110 may be local or remote. For example, processing device 110 may access information and / or data stored in storage device 130 and / or image acquisition device 140 via a network. In some embodiments, processing device 110 may be directly connected to storage device 130 and / or image acquisition device 140 to access information and / or data stored therein. In some embodiments, processing device 110 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processing device 110 may be local or remote. In some embodiments, processing device 110 may run on a cloud platform. For example, the cloud platform may include one or any combination of private cloud, public cloud, hybrid cloud, etc.

[0017] Storage device 130 can be used to store data and / or instructions. In some embodiments, storage device 130 can store data and / or instructions used by processing device 110 to perform or use in order to complete the exemplary methods described herein. For example, storage device 130 can store images of the monitoring environment 120 obtained by image acquisition device 140, or, for example, storage device 130 can store one or more machine learning models.

[0018] Storage device 130 may include one or more storage components, each of which may be a separate device or part of another device. In some embodiments, storage device 130 may include random access memory (RAM), read-only memory (ROM), mass storage, removable memory, volatile read-write memory, etc., or any combination thereof. Exemplarily, mass storage may include disks, optical disks, solid-state drives, etc. In some embodiments, storage device 130 may be implemented on a cloud platform.

[0019] Image acquisition device 140 can refer to a device capable of acquiring images of the monitoring environment 120. Image acquisition device 140 can be a camera, digital camera, etc. Image acquisition device 140 can receive instructions to acquire images of the monitoring environment 120 via network 150, and based on the instructions, take pictures of the monitoring environment 120, and send the captured images to various components and / or external components of application scenario 100 via network 150. The above example is only used to illustrate the breadth of the range of image acquisition device 140 and is not intended to limit its scope.

[0020] Network 150 can connect the various components of application scenario 100 and / or external resources. Network 150 enables communication between the components, as well as with other parts outside application scenario 100, facilitating the exchange of data and / or information. In some embodiments, one or more components of application scenario 100 (e.g., processing device 110, storage device 130, image acquisition device 140) can exchange information and / or data via network 150. For example, network 150 can send images of the monitoring environment 120 obtained by image acquisition device 140 to processing device 110.

[0021] In some embodiments, network 150 can be any one or more of a wired network or a wireless network. In some embodiments, network 150 can include one or more network access points. For example, network 150 can include wired or wireless network access points, such as base stations and / or network switching points 150-1, 150-2, ... In some embodiments, the network can be a point-to-point, shared, centralized, or other topologies, or a combination of multiple topologies.

[0022] The monitoring environment 120 can refer to the area where audio parameters need to be adjusted, such as an indoor or outdoor area with multiple speakers. Figure 1 As shown, in some embodiments, the monitoring environment 120 may include at least one speaker 120-2 and at least one object 120-3, whereby the object 120-3 may be a user using the speaker 120-2. For further details regarding the object, see [link to relevant documentation]. Figure 2 Each speaker 120-2 has a movable range 120-1. The movable range 120-1 can refer to the area within which the speaker can move and / or rotate. For example, the movable range 120-1 can be a circular area with the speaker as the center and a certain distance as the radius. In some embodiments, the speaker 120-2 can be placed on a rotatable base, which is placed on a track. The shape and range of the track can be preset, allowing the speaker 120-2 to move and rotate within the movable range 120-1.

[0023] It should be noted that the application scenario 100 of the audio parameter adjustment system is provided for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can make various modifications or variations based on the description in this specification. For example, the application scenario 100 of the audio parameter adjustment system may also include a user terminal. However, these changes and modifications will not depart from the scope of this application.

[0024] Figure 2 This is an exemplary flowchart illustrating a method for adjusting audio parameters according to some embodiments of this specification. Figure 2 As shown, process 200 includes the following steps. In some embodiments, Figure 2 One or more operations in the process 200 shown can be performed Figure 1 The audio parameter adjustment system shown is implemented in application scenario 100. For example, Figure 2 The illustrated process 200 can be stored in the storage device 130 as instructions and invoked and / or executed by the processing device 110. In some embodiments, the process 200 can be executed by the audio parameter adjustment system 700.

[0025] Step 210: Obtain environmental information of the monitoring environment based on the first preset rule. Step 210 can be executed by the acquisition module 710.

[0026] The first preset rule refers to the method of acquiring environmental information of the monitored environment. In some embodiments, the first preset rule can be set based on experience. For example, the first preset rule can be to acquire environmental information of the monitored environment in real time. Another example is that the first preset rule can be to acquire environmental information of the monitored environment at regular intervals. For instance, the first preset rule can be to acquire environmental information of the monitored environment every 5 seconds.

[0027] For more information on environmental monitoring, please refer to [link / reference]. Figure 1 And its related descriptions.

[0028] Environmental information can refer to information related to the monitored environment. For example, environmental information can refer to image information of the monitored environment, location coordinates of sound sources in the monitored environment, and the size of the monitored environment.

[0029] In some embodiments, the acquisition module 710 can acquire environmental information of the monitoring environment based on a first preset rule in various ways. For example, the acquisition module 710 can acquire image information of the monitoring environment every 5 seconds or in real time through the image acquisition device 140, based on the first preset rule. As another example, the acquisition module 710 can acquire information such as the location coordinates of a sound source and the size of the monitoring environment through manual input by the user.

[0030] Step 220: Based on environmental information, identify at least one object in the monitoring environment and its motion information. Step 220 can be performed by the determination module 720.

[0031] An object can refer to a person using sound equipment in the monitoring environment; for example, an object can refer to a person in a room where sound parameters need to be adjusted. In some embodiments, an object can be one or more.

[0032] Motion information can refer to information related to the movement of an object. For example, motion information could include the object's motion state, direction of movement, and speed. Motion state can refer to the object's motion-related status at the current point in time; for example, the object's motion state could be stationary or in motion. For instance, motion information could include that object 1's motion state is in motion, its direction of movement is 20° west of north, its speed is 1 m / s, and its distance traveled relative to the last time motion information was acquired is 5 meters.

[0033] Step 230: In response to the motion information satisfying the first preset condition, step 231 is executed for at least one speaker. Step 230 can be executed by execution module 730.

[0034] The first preset condition can be set based on the object motion information in the monitoring environment. For example, the first preset condition can be that the object motion information is that it has moved and the moving distance is greater than a first threshold; or the first preset condition can be that the object motion information is that it has moved and the moving distance is greater than a first threshold, and the time after moving and remaining stationary is greater than a second threshold.

[0035] For example, in the monitoring environment, there are two objects: object 1 and object 2. The motion information of object 1 is that it is stationary, and the motion information of object 2 is that it has moved and the moving distance is 7 meters. The first preset condition is that the motion information of the object is that it has moved and the moving distance is greater than 5 meters. Then the execution module 730 can determine that the motion information of object 1 does not meet the first preset condition, and the motion information of object 2 meets the first preset condition.

[0036] In some embodiments, when the motion information meets a first preset condition, the execution module 730 can perform the following steps: Step 231 involves determining at least one set of candidate adjustment values ​​for the acoustic parameters of each of the at least one sound sources, based on at least one object and its motion information. Step 231 can be performed by execution module 730.

[0037] Audio parameters can refer to data information related to audio equipment, such as the position parameters and effects parameters of the audio equipment.

[0038] In some embodiments, the position parameter can refer to a parameter related to the position of the speaker, such as the three-dimensional coordinate position of the speaker, the tilt angle of the sound-emitting unit in the speaker, etc. For example, the position parameter can be the position coordinates of the speaker as (2, 0, 1), or the position parameter can be the tilt angle of the sound-emitting unit in the speaker as 30° relative to the horizontal ground, etc.

[0039] In some embodiments, effect parameters can refer to parameters related to the sound output effect of the speaker, such as volume adjustment parameters, equalizer adjustment parameters, etc. For example, an effect parameter could be a speaker volume of 50dB, or an equalizer setting of -6dB for the 32Hz band and 6dB for the 2kHz band. In some embodiments, the equalizer adjustment parameters have multiple frequency bands, and the execution module 730 can set these multiple frequency bands simultaneously.

[0040] Candidate adjustment amounts can refer to the adjustment amounts of the audio parameters to be selected. For example, candidate adjustment amounts could be increasing the tilt angle of the sound unit in audio 1 by 30° or decreasing the volume by 10dB.

[0041] In some embodiments, the execution module 730 can randomly generate multiple sets of adjustment values, and calculate the corresponding sound comfort zone of the speaker after adjusting the speaker parameters based on each set of adjustment values. If all objects in the monitoring environment are located within the sound comfort zone of the speaker after the adjustment of that set of adjustment values, then they are determined as a set of candidate adjustment values. The sound comfort zone refers to the area where the volume difference between each speaker heard by the user at any position within this area is within a preset decibel. The preset decibel can be set based on experience.

[0042] In some embodiments, the execution module 730 can obtain the sound comfort zone of each speaker under various sound parameters through pre-testing and store the test results for later use.

[0043] In some embodiments, the execution module 730 may determine a list of main speakers based on at least one object and its motion information; and, based on the list of main speakers, determine at least one set of candidate adjustment amounts for the audio parameters of each of the at least one speakers.

[0044] The master speaker list can refer to a list of information containing the master speaker information for at least one object. For example, the master speaker list can be a list constructed with the object as the first column and the master speaker corresponding to the object as the second column.

[0045] In some embodiments, the main speaker corresponding to an object can refer to the speaker that has the greatest impact on the object's auditory experience. For example, the main speaker corresponding to an object can be the speaker closest to the object; or, for another example, the main speaker corresponding to an object can be the speaker with the loudest volume that the object hears.

[0046] In some embodiments, the execution module 730 can determine a main speaker list based on the distances between objects and speakers in the monitoring environment. For example, if there are objects 1, 2, speakers 1, 2, and 3 in the monitoring environment, and the distances between object 1 and speakers 1-3 are 4m, 6m, and 2m respectively, and the distances between object 2 and speakers 1-3 are 3m, 8m, and 7m respectively, then the main speaker list constructed by the execution module 730 can be as shown in Table 1:

[0047] Based on Table 1, we know that the main speaker for object 1 is speaker 3, and the main speaker for object 2 is speaker 1.

[0048] In some embodiments, the execution module 730 may further determine the distance influence and / or effect influence of each speaker on the object based on the motion information of each object in at least one object; determine the main speaker corresponding to each object based on the distance influence and / or effect influence; and determine a list of main speakers based on the main speakers corresponding to each object. For more information on determining the list of main speakers based on distance influence and / or effect influence, please refer to [link to relevant documentation]. Figure 3 And related descriptions. The determination of the main speaker list can also be based on other rules or methods; the examples given above are not intended to limit this.

[0049] In some embodiments, the main speaker list may include information on multiple main speakers corresponding to an object. These multiple main speakers may include a first main speaker, a second main speaker, ..., an Nth main speaker. The level of each main speaker can be determined based on the degree of influence of the corresponding speaker on the auditory experience of the corresponding object; the greater the influence, the higher the level. For example, the influence of the first main speaker is greater than that of the second main speaker.

[0050] In some embodiments, the main speaker list may include multiple columns of data to record the multi-level main speakers corresponding to each object, such as the first column being the object, the second column being the first main speaker corresponding to the object, the third column being the second main speaker corresponding to the object, and so on.

[0051] As an example only, the monitoring environment contains object 1, speaker 1, speaker 2, and speaker 3. The distance between object 1 and speaker 1 is 10m, the distance between object 1 and speaker 2 is 5m, and the distance between object 1 and speaker 3 is 4m. Based on the preset judgment condition that the main speaker can be a speaker that is less than 6m away from the object, and the smaller the distance, the greater the influence and the higher the level, the more likely it is to be considered. Therefore, the main speakers of object 1 are speaker 2 and speaker 3, and the level of speaker 2 is lower than that of speaker 3. The list of main speakers constructed by the execution module 730 can be shown in Table 2:

[0052] Based on Table 2, we can see that the main speaker list corresponding to object 1 includes two main speakers. Among the two main speakers, the first main speaker is speaker 3 and the second main speaker is speaker 2.

[0053] In some embodiments of this specification, by setting a main speaker list that includes multiple levels of main speakers, it is possible to not only consider adjusting the audio parameters of the speakers with the greatest impact, but also to further consider adjusting the audio parameters of other speakers with slightly weaker impact. This allows for comprehensive adjustment of all speakers to provide users with a better listening experience.

[0054] In some embodiments, the execution module 730 may preferentially generate multiple sets of candidate parameter adjustment values, primarily based on the parameter adjustment values ​​of multiple main speakers. That is, when determining the candidate adjustment values, the execution module 730 may determine the candidate adjustment values ​​for adjusting the audio parameters of the main speakers based on the main speakers corresponding to each object in the monitoring environment. For example, the execution module 730 may randomly generate multiple sets of parameter adjustment values ​​for multiple speakers, and after determining the list of main speakers, preferentially generate multiple sets of candidate parameter adjustment values, primarily based on the parameter adjustment values ​​of multiple main speakers.

[0055] In some embodiments of this specification, by determining a list of main speakers based on the object and its motion information, and then determining candidate adjustment amounts, the role of speakers with a greater impact can be effectively highlighted, and the calculation process of sound propagation can be simplified.

[0056] Step 232: Based on at least one set of candidate adjustment values, determine the target adjustment value for the audio parameters of the audio system. Step 232 can be executed by execution module 730.

[0057] The target adjustment amount can refer to the final determined adjustment amount of the audio system. For example, the target adjustment amount can be the final adjustment amount for adjusting the audio parameters of audio system 1 among multiple candidate adjustment amounts.

[0058] In some embodiments, the execution module 730 can calculate the sum of the auditory experiences of each object in the environment after adjusting the audio parameters based on each set of candidate adjustment amounts; and determine the candidate adjustment amount corresponding to the maximum sum of auditory experiences as the target adjustment amount. Auditory experience can refer to a quantitative parameter used to evaluate the merits of candidate adjustment amounts; the better the auditory experience of the object, the better the corresponding auditory experience. In some embodiments, auditory experience can be quantified into a numerical value; the larger the numerical value, the better the corresponding auditory experience. For more information on determining auditory experience and other auditory experiences, please refer to [link to relevant documentation]. Figure 3 , Figure 4 Related descriptions.

[0059] As an example only, there are three objects in the monitoring environment: Object 1, Object 2, and Object 3. After adjusting the audio parameters of the sound system based on candidate adjustment amount 1, the user experience of Object 1 to Object 3 is 0.2, 0.8, and 0.4, respectively. After adjusting the audio parameters of the sound system based on candidate adjustment amount 2, the user experience of Object 1 to Object 3 is 0.5, 0.1, and 0.6, respectively. The execution module 730 can calculate that the sum of the user experience of all objects in the monitoring environment corresponding to candidate adjustment amount 1 and candidate adjustment amount 2 is 1.4 and 1.2, respectively. Then, the execution module 730 can determine candidate adjustment amount 1 as the target adjustment amount.

[0060] In some embodiments, the execution module 730 may determine at least one candidate scheme based on at least one set of candidate adjustment amounts; perform at least one round of iteration on the at least one candidate scheme until a preset update condition is met; determine a target scheme based on the results of at least one round of iteration; and determine the target adjustment amount of the audio parameters of the speaker based on the target scheme. Each round of iteration includes: performing a preset change process on the candidate schemes of the current round to obtain a first alternative scheme; in the first round of iteration, the candidate schemes of the current round are determined based on at least one set of candidate adjustment amounts; the preset change process includes at least one of crossover processing, mutation processing, and replication processing; determining the evaluation value of the first alternative scheme and the evaluation value of the candidate schemes of the current round based on a preset evaluation algorithm; and determining the candidate scheme to enter the next round of iteration based on the evaluation value of the first alternative scheme and the evaluation value of the candidate schemes of the current round. For more information on determining the target adjustment amount of the audio parameters of the speaker based on the target scheme, please refer to [link to relevant documentation]. Figure 5 And its related descriptions.

[0061] The determination of the target adjustment amount can also be based on other rules or methods, and the examples above are not intended to limit this.

[0062] In some embodiments of this specification, by acquiring environmental information of the monitored environment, determining the object and its motion information, determining the candidate adjustment amount of the sound, and then determining the target adjustment amount, the sound parameters can be adaptively adjusted according to different environmental conditions, thereby optimizing the audiovisual experience of the object in the environment and improving the intelligence of the sound.

[0063] Figure 3 This is a schematic diagram illustrating the process of determining the main speaker list according to some embodiments of this specification. Based on process 300, the main speaker list can be determined so that candidate adjustment amounts and target adjustment amounts of speaker parameters can be subsequently determined based on the main speaker list. In some embodiments, process 300 can be executed by a speaker parameter adjustment system 700 (such as execution module 730).

[0064] Step 310: Based on the motion information of each object in at least one object, determine the distance influence and / or effect influence of each sound on the object.

[0065] The distance influence of sound on an object can be a quantitative representation of the impact of sound on the auditory experience of the object based on the distance between the sound and the object. In some embodiments, the distance influence of sound on an object can be determined by the distance between the sound and the object; for example, the smaller the distance between the sound and the object, the greater the distance influence of the sound on the object.

[0066] The impact of sound on an object can be a quantitative representation of the influence of sound on the auditory experience of the object based on the sound characteristics of the sound. In some embodiments, the impact of sound on an object can be determined by the sound characteristics of the sound. The sound characteristics of the sound can refer to sound-related feature information, such as the sound's distortion, signal-to-noise ratio, etc. In some embodiments, the sound characteristics can be represented by vectors. For example, if the distortion of sound 1 is 2% and the signal-to-noise ratio is 80dB, then the sound characteristics of sound 1 can be represented as (2, 80). In some embodiments, the execution module 730 can determine that at different volume levels, the sound with lower distortion and higher signal-to-noise ratio has a greater impact on the object.

[0067] In some embodiments, the execution module 730 may determine the distance influence degree and / or the effect influence degree based on a second preset rule. The second preset rule may represent the correspondence between different distances between the sound and the object and the distance influence degree, and / or the correspondence between different sound characteristics of the sound and their effect influence degree on the object, etc. The second preset rule can be determined based on historical data, experience, etc.

[0068] In some embodiments, the second preset rule may include the distance influence degree corresponding to different distances when the speaker and the object are at different distances. As an example only, the second preset rule may be that the distance influence degree is 0.9 when the speaker is 0 meters to 3 meters away from the object; the distance influence degree is 0.6 when the speaker is 3 meters to 6 meters away from the object; and the distance influence degree is 0.3 when the speaker is more than 6 meters away from the object.

[0069] In some embodiments, the second preset rule includes the effect influence degree corresponding to different sound characteristics when the speakers have different sound characteristics. As an example only, the second preset rule could be: 0.9 for speaker distortion of 0-1%; 0.6 for distortion of 1-5%; 0.3 for distortion of 5%; 0.3 for signal-to-noise ratio (SNR) of 0-60dB; 0.3 for SNR of 60-90dB; and 0.9 for SNR above 90dB. The effect influence degree can be the mean or weighted sum of the distortion influence degree and the SNR influence degree, and the weight value can be based on a preset or randomly determined value.

[0070] For example, if the distance between speaker 1 and object 1 is 5 meters, the distance influence of speaker 1 on object 1 can be determined to be 0.6 according to the second preset rule in the aforementioned embodiment; if the distortion of speaker 1 is 5.5% and the signal-to-noise ratio is 68dB, the effect influence of speaker 1 on object 1 can be determined to be 0.45 according to the second preset rule in the aforementioned embodiment.

[0071] The determination of the influence of distance and the influence of effect can also be based on other rules or methods, and the examples above are not intended to limit this.

[0072] Step 320: Determine the main speaker for each object based on distance influence and / or effect influence.

[0073] In some embodiments, the execution module 730 may determine the main speaker corresponding to each object based on the aforementioned determined distance influence or effect influence. For example, the execution module 730 may sort the obtained distance influence or effect influence based on the distance influence or effect influence of each speaker, and determine the speaker with the largest distance influence or effect influence as the main speaker.

[0074] As an example only, in the monitoring environment there are object 1, speaker 1, speaker 2 and speaker 3. The distance influence of speaker 1 to speaker 3 on object 1 is 0.7, 0.3 and 0.8 respectively. Then the execution module 730 can sort them according to the distance influence of speaker 1 to speaker 3 on object 1, and determine speaker 3 with the largest distance influence as the main speaker of object 1.

[0075] For example, if there are object 1, speaker 1, speaker 2 and speaker 3 in the monitoring environment, and the influence of speaker 1 to speaker 3 on object 1 is 0.7, 0.5 and 0.6 respectively, then the execution module 730 can sort them according to the influence of speaker 1 to speaker 3 on object 1, and determine speaker 1 with the largest influence as the main speaker of object 1.

[0076] In some embodiments, the execution module 730 can perform a weighted summation based on the distance influence and effect influence of each speaker to obtain the comprehensive influence of each speaker, sort the obtained comprehensive influence, and determine the speaker with the largest comprehensive influence as the main speaker. The weights of the distance influence and effect influence during the weighted summation can be randomly assigned, set based on experience, or determined by the user.

[0077] As an example only, in the monitoring environment there are object 1, speaker 1, speaker 2 and speaker 3. The distance influence of speakers 1 to 3 on object 1 is 0.7, 0.3 and 0.8 respectively, and the effect influence of speakers 1 to 3 on object 1 is 0.2, 0.5 and 0.6 respectively. The execution module 730 can randomly assign weights of 0.4 for distance influence and 0.6 for effect influence. Then the calculated comprehensive influence of speakers 1 to 3 on object 1 is 0.4, 0.42 and 0.68 respectively. The execution module 730 can determine speaker 3 with the largest comprehensive influence as the main speaker of object 1.

[0078] In some embodiments, when the execution module 730 determines the main speaker based on the overall influence, the weights of the distance influence and the effect influence can be related to the sensitivity of the object's experience to changes. The sensitivity of the object's experience to changes can be determined based on the difference between the object's experience under different changes in the audio parameters.

[0079] In some embodiments, the user experience can be obtained based on a user experience determination model.

[0080] The "experience" can refer to a quantitative parameter used to evaluate the merits of candidate adjustment values. After adjusting the audio parameters of the speaker based on the candidate adjustment values, the better the auditory experience of the listener, the better the corresponding "experience". In some embodiments, the "experience" can be represented by a value in the range [0, 1] or by words such as "excellent" or "average". When represented by a value, the larger the value, the better the corresponding "experience".

[0081] In some embodiments, such as Figure 4 As shown, the execution module 730 can determine the user experience of an object through the user experience determination model 420, which can be a machine learning model.

[0082] In some embodiments, the experience determination model 420 may include a deep neural network (DNN) model, a convolutional neural network (CNN) model, a recurrent neural network (RNN) model, a neural network (NN) model, or any combination thereof.

[0083] In some embodiments, the input to the experience determination model 420 can be multiple parameter vectors 410-1 corresponding to the audio parameters of multiple speakers, and multiple features 410-2 of the object. The output of the experience determination model 420 can be the experience 430 of the corresponding object.

[0084] Multiple parameter vectors 410-1 corresponding to the audio parameters of multiple speakers can be represented based on the parameter vector of each speaker. For example, multiple parameter vectors 410-1 can include the parameter vector of speaker 1, the parameter vector of speaker 2, ..., the parameter vector of speaker n, etc. The parameter vector of each speaker can be determined based on its audio parameters. For example, if the parameter vector of speaker 1 is (((2, 0, 1), 30), (50, ((32, 5), (64, -6), ...))), it can represent the audio parameters of speaker 1 as: position coordinates (2, 0, 1), speaker unit tilt angle relative to the horizontal plane 30°, volume level 50dB, equalizer adjustment to 5dB in the 32Hz band and -6dB in the 64Hz band, etc. More information on audio parameters can be found in [link to documentation]. Figure 2 And its related descriptions.

[0085] The multiple features 410-2 of an object can refer to feature information related to the object, which may include the object's location coordinates, gender, age, etc. In some embodiments, one feature of the object can be a single feature of the object, and the collection of all feature information can be considered as the multiple features 410-2 of the object. For example, the multiple features 410-2 of the object may include object feature A, object feature B, ..., object feature N, etc. In some embodiments, each characteristic of the object can be represented based on a feature vector. As an example only, the multiple features of object 1 may include feature A and feature B of object 1, where feature A (2, 1, 0) of object 1 can represent the object's location coordinates as (2, 1, 0), and feature B (1, 31) of object 1 can represent that object 1's gender is male (using the number 1 to represent male and the number 2 to represent female), and its age is 31 years old, etc.

[0086] For example, in a certain monitoring environment there are speakers 1, speakers 2, speakers 3 and object 1. The execution module 730 can input multiple parameter vectors 410-1 corresponding to the audio parameters of speakers 1-3 and multiple features 410-2 of object 1 into the experience determination model 420, and the experience determination model 420 outputs the experience 430 of object 1.

[0087] In some embodiments, the experience determination model 420 can be trained using training data obtained from historical data or simulation data. For example, the training data includes a first training sample with a first label. The first training sample is input into the initial experience determination model, and a loss function is established based on the first label and the output of the initial experience determination model. The parameters of the initial experience determination model are updated. When the loss function of the initial experience determination model meets a preset condition, the model training is complete, and the trained experience determination model 420 is obtained. The preset condition may be that the loss function converges, the number of iterations reaches a threshold, etc.

[0088] In some embodiments, the first training sample may include multiple sets of sub-training samples corresponding to multiple sample objects. Each set of sub-training samples may include sample parameter vectors of multiple sample speakers and multiple features of the sample objects. The label corresponding to each set of sub-training samples is the actual user experience of the sample objects in that set under the speaker parameters corresponding to the sample parameter vectors of the sample speakers. The training samples may be obtained based on historical data or simulation data. The label of each set of sub-training samples may be determined by manual annotation or automatic annotation.

[0089] Sensitivity to changes in user experience refers to a parameter that reflects how sensitive an object is to changes in audio parameters such as volume and sound effects. The more sensitive an object is to changes in volume and sound effects, the higher its sensitivity to changes. In some embodiments, sensitivity to changes can be represented by a value within the range [0, 1], where a larger value indicates higher sensitivity to changes.

[0090] In some embodiments, sensitivity to change can include volume sensitivity and sound effect sensitivity. Volume sensitivity refers to a parameter reflecting how sensitive an object is to changes in volume; the more sensitive the object is to volume changes, the higher the corresponding volume sensitivity. Sound effect sensitivity refers to a parameter reflecting how sensitive an object is to changes in sound effects; the more sensitive the object is to sound effect changes, the higher the corresponding volume sensitivity. In some embodiments, volume sensitivity or sound effect sensitivity can be represented by a value within the range [0, 1], where a larger value indicates higher volume sensitivity or sound effect sensitivity.

[0091] In some embodiments, the execution module 730 can determine the sensitivity of the object to changes in its experience based on the adjustment of the position parameters and effect parameters of the sound in the candidate adjustment amounts and the object's experience with the adjusted sound.

[0092] As an example only, the candidate adjustment amount 1 mainly adjusts the volume parameters of the speaker, so that the volume of speaker 1 changes before and after the adjustment, but the distortion, signal-to-noise ratio, etc. remain basically unchanged. The execution module 730 can determine the user experience of object 1 before and after adjusting the volume parameters of speaker 1 using the candidate adjustment amount 1 based on the user experience determination model, and determine the volume sensitivity of object 1 based on the difference between the user experience after adjustment and the user experience before adjustment.

[0093] Candidate adjustment amount 2 mainly adjusts the effect parameters of the speaker 1 other than the volume parameter. For example, the equalizer adjustment parameter of the 32Hz frequency band is reduced by 6dB, which changes the volume, distortion, signal-to-noise ratio, etc. of speaker 1 before and after the adjustment. The execution module 730 can determine the user experience of object 1 before and after adjusting the effect parameters of speaker 1 using candidate adjustment amount 2 based on the user experience determination model, and determine the sound effect sensitivity of object 1 based on the difference between the user experience before and after the adjustment. The method for determining the user experience of object 1 after adjustment and before adjustment can be found in the relevant description of determining the user experience of object based on the user experience determination model in the previous embodiment.

[0094] The execution module 730 can perform multiple tests on all objects in the monitoring environment based on a large number of candidate adjustment samples to obtain the change sensitivity of each object in the monitoring environment. Among them, if the number of times an object's volume sensitivity is higher than its sound effect sensitivity occurs is greater than a preset threshold, then the object can be considered to have higher volume sensitivity.

[0095] In some embodiments, the execution module 730 can also acquire in advance the object characteristics of objects with higher volume sensitivity and objects with higher sound effect sensitivity. The object characteristics may include the object's age, gender, personality, hobbies, height, weight, etc. Based on the analysis of the object characteristics of each object in the current monitoring environment, it can predict whether each object is more sensitive to sound effects or more sensitive to volume.

[0096] In some embodiments, the execution module 730 may associate and store the object features of each object, the determination result of whether it is more sensitive to sound effects or more sensitive to volume, with the object's identification information (such as face, fingerprint, etc.) (such as storage and storage device 130) so that it can be directly called next time.

[0097] In some embodiments, if the proportion of objects with high volume sensitivity in the monitoring environment is greater, the execution module 730 can set the weight corresponding to the distance influence degree to be greater when determining the comprehensive influence degree based on weighting; correspondingly, if the proportion of objects with high sound effect sensitivity in the monitoring environment is greater, the execution module 730 can set the weight corresponding to the effect influence degree to be greater; and then determine the main speaker of each object based on the comprehensive influence degree.

[0098] In some embodiments of this specification, the user experience of an object is determined through an experience perception determination model. This quantifies the quality of the user experience and makes the process of determining the experience perception more accurate and efficient. Based on the user experience perception, the sensitivity of the object to changes is determined, and then the weighting weights for calculating the overall impact are determined. Based on the overall impact, the main speaker is determined, ensuring that the determined main speaker is the one that best improves the user's auditory experience.

[0099] Step 330: Determine the list of main speakers based on the main speaker corresponding to each object.

[0100] In some embodiments, the execution module 730 may construct a main speaker list based on the main speaker corresponding to each object, with the object as the first column and the corresponding main speaker as the second column.

[0101] In some embodiments, the execution module 730 may sort the speakers by influence and determine the top N speakers ranked from highest to lowest as the first main speaker, ..., the Nth main speaker of the object.

[0102] In some embodiments, the degree of influence can be the degree of influence of distance, the degree of influence of effect, or the combined degree of influence of the foregoing.

[0103] As an example only, the execution module 730 can set the influence degree as the distance influence degree, determine the distance influence degree of multiple speakers based on step 310 as the influence degree, sort the distance influence degree from high to low, and determine the speakers with the highest distance influence degree as the first main speaker, ..., the Nth main speaker of the object.

[0104] For example, in the monitoring environment there are object 1, object 2, speaker 1, speaker 2 and speaker 3. The distance influence of speaker 1 to speaker 3 on object 1 is 0.9, 0.6 and 0.3 respectively, and the distance influence of speaker 1 to speaker 3 on object 2 is 0.6, 0.9 and 0.3 respectively. The execution module 730 can sort the objects corresponding to the influence values ​​respectively, and determine that the first main speaker of object 1 is speaker 1, the second main speaker is speaker 2 and the third main speaker is speaker 3, and the first main speaker of object 2 is speaker 2, the second main speaker is speaker 1 and the third main speaker is speaker 3.

[0105] In some embodiments of this specification, multiple main speakers are determined by ranking their influence, which can reflect the degree of influence of different main speakers on the object, making the determination of multiple main speakers corresponding to the object more reliable.

[0106] In some embodiments of this specification, by determining the distance influence and / or effect influence based on the motion information of the object, and then determining the main speaker and main speaker list corresponding to the object, the auditory experience of the object can be quantified, making the determined main speaker and main speaker list more accurate.

[0107] Figure 5 This is a schematic diagram illustrating a process for determining a target adjustment amount of audio parameters of a sound source according to some embodiments of this specification. In some embodiments, process 500 may be executed by an audio parameter adjustment system 700 (such as execution module 730).

[0108] Step 510: Determine at least one candidate scheme based on at least one set of candidate adjustment amounts.

[0109] A candidate solution can refer to an alternative method for adjusting audio parameters. For example, a candidate solution may include adjustment parameters for at least one of the following: adjusting the speaker's position coordinates, adjusting the tilt angle of the speaker's drivers, adjusting the volume parameters, and adjusting the equalizer parameters. Different candidate solutions may contain the same or different adjustment items, and at least one of them may have different adjustment parameters for the same adjustment item.

[0110] As an example only, the candidate solutions can use the letters A, B, C, D, etc. to represent different adjustment items. For example, A represents adjusting the position coordinates of the speaker, B represents adjusting the tilt angle of the speaker unit, C represents adjusting the volume parameter, and D represents adjusting the equalizer parameter.

[0111] For example, candidate solution 1 is (1, (A x -1, A y -2, A z +1), (B+10), (C+10), (D 32 -6,D 64 +5, ...), where the first element 1 indicates that the adjustment object is speaker 1, (A x -1, A y -2, A z +1) is for adjusting the position coordinates of speaker 1, A x A y A z Let x, y, and y represent the horizontal, vertical, and vertical axes, respectively. (B+10) indicates that the tilt angle of the speaker unit of speaker 1 increases by 10°, (C+10) indicates that the volume of speaker 1 increases by 10dB, and (D... 32 -6,D 64 +5, ... indicates that the equalizer parameters of speaker 1 will be reduced by 6dB in the 32Hz band and increased by 5dB in the 64Hz band, etc.

[0112] In some embodiments, the execution module 730 may generate at least one candidate scheme based on at least one set of candidate adjustment values. A candidate scheme may correspond to a set of candidate adjustment values.

[0113] For example only, candidate adjustment 1 involves moving the horizontal coordinate of speaker 1 2 meters to the right, increasing the tilt angle of speaker 1's driver by 10°, decreasing speaker 1's volume by 5dB, decreasing speaker 1's equalizer parameters in the 64Hz band by 6dB, and keeping other frequency bands unchanged. Similarly, moving the vertical coordinate of speaker 2 downwards by 1 meter results in decreasing speaker 2's driver tilt angle by 5°, while keeping speaker 2's volume and equalizer parameters unchanged. Candidate adjustment 2 involves keeping speaker 1's position coordinates and driver tilt angle unchanged, increasing speaker 1's volume by 5dB, and keeping speaker 1's equalizer parameters unchanged. For speaker 2, keeping speaker 2's position coordinates unchanged, increasing speaker 2's driver tilt angle by 3°, decreasing speaker 2's volume by 10dB, decreasing speaker 2's equalizer parameters in the 250Hz band by 12dB, and keeping other frequency bands unchanged.

[0114] For the two sets of candidate adjustment amounts mentioned above, the execution module 730 can generate candidate scheme 1 ((1, (A)) based on candidate adjustment amount 1. x +2, A y A z ), (B+10), (C-5), (D 32 D 64 -6, ...), (2, (A) x A y A z -1), (B-5), C, (D) 32 D 64 Based on candidate adjustment amount 2, candidate scheme 2 is generated ((1, (A)). x A y A z B, (C+5), (D) 32 D 64 ,…), (2, (A x A y A z ), (B+3), (C-10), (D 32 D 250 -12, ...))).

[0115] Step 520: Perform at least one round of iterations on at least one candidate solution until the preset update conditions are met.

[0116] Preset update conditions refer to the conditions that must be met when iteration stops. For example, preset update conditions could include reaching a preset number of iteration rounds, the evaluation value of a candidate solution exceeding a preset evaluation threshold, or the difference between the evaluation values ​​of candidate solutions obtained from at least two consecutive iterations not exceeding a preset difference threshold. For further explanation of evaluation values, see [link to relevant documentation]. Figure 6 Related descriptions.

[0117] In some embodiments, preset update conditions can be pre-set based on experience and stored in a corresponding device (such as storage device 130). For example, preset evaluation thresholds or preset difference thresholds can be determined based on historical iteration experience.

[0118] In some embodiments, the execution module 730 may perform at least one round of iteration on at least one candidate solution based on a preset algorithm until a preset update condition is met. The preset algorithm may refer to an algorithm that performs multiple rounds of iterative updates on multiple candidate solutions to determine the target solution, as detailed below.

[0119] Figure 6 This is an exemplary schematic diagram illustrating the iteration of candidate solutions according to some embodiments of this specification. Process 600 may include: Step 610: Perform preset change processing on the candidate solutions in this round to obtain the first alternative solution.

[0120] In the first iteration, the candidate solutions are determined based on at least one set of candidate adjustments. In subsequent iterations, candidate solutions can be determined based on the candidate solutions from the previous iteration or the first alternative solution.

[0121] Preset variation processing can refer to a pre-defined method for processing candidate solutions to generate solutions (such as a first alternative solution) that are different from the candidate solutions. In some embodiments, preset variation processing includes at least one of crossover processing, mutation processing, and replication processing.

[0122] The first alternative solution may refer to the solution obtained after the candidate solution has undergone the aforementioned preset change processing. In some embodiments, the adjustment parameters of at least some of the adjustment items in the first alternative solution are different from those of the candidate solution.

[0123] Cross-processing can refer to a method of exchanging the adjustment parameters of one or more adjustment items in at least two candidate solutions. For example, candidate solution 1 is ((1, (A)). x +2, A y A z ), (B+10), (C-5), (D 32 D 64 -6, ...), (2, (A) x A y A z -1), (B-5), C, (D) 32 D 64 Candidate solution 2 is ((1, (A)) x A y A z B, (C+5), (D) 32 D 64 ,…), (2, (Ax A y A z ), (B+3), (C-10), (D 32 D 250 -12, ...))), the execution module 730 can randomly select or based on a preset selection of one or more adjustment parameters for exchange. For example, if the execution module 730 selects to exchange the adjustment parameters of the position of the speaker 1 in candidate scheme 1 and candidate scheme 2 to generate two new first alternative schemes, then the first alternative scheme 1 generated after cross-processing is ((1, (A x A y A z B, (C+5), (D) 32 D 64 ,…), (2, (A x A y A z -1), (B-5), C, (D) 32 D 64 The first alternative, 2, is ((1, (A)). x +2, A y A z ), (B+10), (C-5), (D 32 D 64 -6, ...), (2, (A) x A y A z ), (B+3), (C-10), (D 32 D 250 -12, ...))).

[0124] Mutation processing refers to a method of directly mutating the adjustment parameters of one or more adjustment items in a candidate solution. For example, candidate solution 1 is ((1, (A)). x +2, A y A z ), (B+10), (C-5), (D 32 D 64 -6, ...), (2, (A) x A y A z -1), (B-5), C, (D) 32 D 64 , ...))), the execution module 730 randomly selects or modifies the adjustment parameters of one or more adjustment items based on preset selections. For example, the execution module 730 modifies the adjustment parameters of the position of the speaker 1 in candidate scheme 1 to generate the first alternative scheme 3. The first alternative scheme 3 generated after the modification is ((1, (A)) xA y A z ), (B+4), (C+5), (D 32 D 64 ,…), (2, (A x A y A z -1), (B-5), C, (D) 32 D 64 ,...))).

[0125] Copying can refer to a method of directly copying the adjustment parameters of one or more adjustment items from one or more different candidate solutions to another candidate solution. For example, candidate solution 1 is ((1, (A x +2, A y A z ), (B+10), (C-5), (D 32 D 64 -6, ...), (2, (A) x A y A z -1), (B-5), C, (D) 32 D 64 Candidate solution 2 is ((1, (A)) x A y A z B, (C+5), (D) 32 D 64 ,…), (2, (A x A y A z ), (B+3), (C-10), (D 32 D 250 -12, ...))), the execution module 730 can directly copy all the adjustment items for the audio 2 in candidate scheme 2 to candidate scheme 1 to replace all the adjustment items for the audio 2 in candidate scheme 1, so as to generate the first alternative scheme 4. The first alternative scheme 4 can be ((1, (A x +2, A y A z ), (B+10), (C-5), (D 32 D 64 -6, ...), (2, (A) x A y A z ), (B+3), (C-10), (D 32 D 250 -12, ...))).

[0126] In some embodiments, when the execution module 730 performs preset change processing on the candidate schemes in the current round to generate a first alternative scheme, the selection of which candidate scheme to perform preset change processing and which adjustment items in the candidate scheme to perform preset change processing can be determined based on the main audio list. For example, if the adjusted audio included in the candidate scheme exists in the main audio list of the object, the candidate scheme has a higher probability of being selected for preset change processing. Furthermore, for candidate schemes selected for preset change processing, adjustment items for audio that exist in the main audio list of the corresponding object have a higher probability of being selected for the aforementioned cross-processing, mutation processing, and copying processing, etc.

[0127] As an example only, candidate solution 1 is ((1, (A)). x +2, A y A z ), (B+10), (C-5), (D 32 D 64 -6, ...), (2, (A) x A y A z -1), (B-5), C, (D) 32 D 64 Candidate solution 2 is ((3, (A) , ...))). x A y A z B, (C+5), (D) 32 D 64 (, ...), (4, (A x A y A z ), (B+3), (C-10), (D 32 D 250 -12, ...)))), the main speaker corresponding to object 1 in the main speaker list obtained by the execution module 730 is speaker 2. If the object in the current monitoring environment is object 1, when generating the first alternative solution, the second group of candidate adjustment amounts of candidate solution 1 will be given priority for preset change processing.

[0128] In some embodiments, when performing preset change processing on the adjustment items in the candidate scheme, for multiple speakers in the candidate scheme, the adjustment amount of the volume parameter of the main speaker with a greater effect is more likely to be increased, and correspondingly, the adjustment amount of the volume parameter of the main speaker with a lesser effect is more likely to be decreased.

[0129] As the volume of the speakers increases, distortion increases and the signal-to-noise ratio (SNR) decreases. Furthermore, both the increase in distortion and the decrease in SNR are inversely proportional to the effect impact. For example, if speaker 1 has an effect impact of 0.9 and speaker 2 has an effect impact of 0.3, increasing the volume by the same amount will result in a smaller increase in distortion and a smaller decrease in SNR for speaker 1 compared to speaker 2. In some embodiments, when it is necessary to increase the overall volume of the monitoring environment, the volume of speakers with a higher effect impact can be increased first, resulting in a smaller increase in distortion and a smaller decrease in SNR in the overall monitoring environment. Conversely, when it is necessary to decrease the overall volume of the monitoring environment, the volume of speakers with a lower effect impact can be decreased first, resulting in a larger decrease in distortion and a larger increase in SNR in the overall monitoring environment. More information on effect impact can be found in [link to relevant documentation]. Figure 3 And its related descriptions.

[0130] As an example only, candidate solution 1 is ((1, (A)). x +2, A y A z ), (B+10), (C-5), (D 32 D 64 -6, ...), (2, (A) x A y A z -1), (B-5), C, (D) 32 D 64 The execution module 730 obtains a list of main speakers, where the main speakers corresponding to object 1 are speaker 1 and speaker 2. The effect influence of speaker 1 is 0.6, and the effect influence of speaker 2 is 0.9. Therefore, when performing preset change processing on candidate scheme 1, the adjustment amount of the volume parameter in the adjustment item for speaker 2 in candidate scheme 1 has a higher probability of increasing, and the adjustment amount of the volume parameter in the adjustment item for speaker 1 in candidate scheme 1 has a higher probability of decreasing. The probability can be set based on historical data, human experience, etc.

[0131] In some embodiments of this specification, by setting the adjustment amplification probability of the volume parameter of the main speaker with a greater / less significant effect to be higher / lower, the parameters of the main speaker with a greater impact on the sound effect can be changed first, which helps to obtain the target solution more efficiently and accurately, and improve the listener experience.

[0132] In some embodiments of this specification, by selecting candidate schemes based on a list of main speakers through preset variation processing, the parameters of the main speakers in the monitoring environment can be changed first, thereby making the obtained target scheme more accurate and providing listeners with a better auditory experience.

[0133] Step 620: Based on the preset evaluation algorithm, determine the evaluation value of the first alternative solution and the evaluation value of the candidate solutions in this round.

[0134] A preset evaluation algorithm can refer to a pre-defined algorithm used to determine the evaluation value of the first alternative solution or the candidate solutions in this round. For example, the preset algorithm can be a function, model, etc., used to calculate the evaluation value of the first alternative solution.

[0135] An evaluation value is an indicator used to assess the merits of the first alternative solution or the candidate solutions in this round. For example, an evaluation value can be used to assess the auditory experience of objects in a monitoring environment after adjusting the sound in the monitoring environment according to the first alternative solution or the candidate solutions in this round. In some embodiments, the evaluation value can be the sum of the experience of each object under the adjustment scheme corresponding to the first alternative solution or the candidate solutions in this round. For example, if there are objects 1, 2, and 3 in the monitoring environment, and after executing candidate solution 1, the experience of objects 1-3 is 0.8, 0.3, and 0.5 respectively, then the execution module 730 can calculate the evaluation value of candidate solution 1 as 1.6.

[0136] In some embodiments, the execution module 730 can obtain the user experience of each object under each adjustment scheme (such as the first alternative scheme or the candidate scheme of this round) based on the user experience determination model, and directly add the user experience of each object in the monitoring environment to obtain the evaluation value of the corresponding adjustment scheme. For example, there are sound 1, object 1, object 2 and object 3 in the monitoring environment, and candidate scheme 1 is ((1, (A x +2, A y A z ), (B+10), (C-5), (D 32 D 64 -6, ...))) By inputting the parameter vectors of speaker 1 and the features of object 1 under candidate solution 1 into the experience determination model, the experience of object 1 under candidate solution 1 can be obtained. Similarly, the experience of object 2 and object 3 under candidate solution 1 can be obtained respectively. For example, if the experience of object 1-object 3 under candidate solution 1 is obtained as 0.8, 0.3, and 0.5 respectively, then the execution module 730 can add them together to obtain the evaluation value of candidate solution 1 as 1.6. For more information on the experience determination model, the parameter vectors of the speaker, and the features of the object, please refer to Figure 3 And its related descriptions.

[0137] In some embodiments, the preset evaluation algorithm may be a lookup table-based retrieval method. The lookup table includes multiple different candidate solutions or first alternatives and the corresponding evaluation value for each candidate solution or first alternative. The lookup table can be constructed based on prior knowledge or historical data to establish multiple different candidate solutions or first alternatives and the correspondence between each candidate solution or first alternative and the evaluation value. The multiple different candidate solutions or first alternatives may be historical candidate solutions or historical first alternatives. The evaluation value may be determined by the auditory experience of all objects under each historical candidate solution or historical first alternative. The auditory experience can be quantified by the average score of a large number of sample objects after experiencing the corresponding historical candidate solution or historical first alternative.

[0138] In some embodiments, determining the evaluation value of each candidate display scheme in at least one candidate display scheme based on a preset algorithm includes: searching at least one candidate scheme or first alternative scheme in a lookup table, determining a historical candidate scheme or historical first alternative scheme that is the same as or similar to at least one candidate scheme or first alternative scheme in the lookup table, and then determining the evaluation value corresponding to the historical candidate scheme or historical first alternative scheme as the evaluation value of at least one candidate scheme or first alternative scheme.

[0139] For example, the execution module 730 can perform a matching operation one by one with each set of candidate adjustment amounts in a candidate solution or first alternative solution in the lookup table, based on each set of candidate adjustment amounts in a certain candidate solution or first alternative solution. The evaluation value corresponding to the candidate solution or first alternative solution with the highest matching degree in the lookup table is then used as the evaluation value of that candidate solution or first alternative solution. The solution with the highest matching degree can be one where the adjustment items are the same and the adjustment amounts are the same, or where the difference in adjustment amounts is less than a fourth threshold. The difference can be the ratio of the difference between the candidate adjustment amount of the solution in the lookup table and the candidate adjustment amount in that candidate solution or first alternative solution to the candidate adjustment amount in that candidate solution or first alternative solution.

[0140] In some embodiments, the evaluation value of the adjustment scheme is also related to the updated main speaker list corresponding to the adjustment scheme. The method for obtaining the updated main speaker list can be found in [reference needed]. Figure 3 The description of the determination of the main speaker list.

[0141] The updated main speaker list refers to the list obtained by adjusting the speaker parameters based on a certain adjustment scheme, redetermining the main speaker for each object, and updating the correspondence between objects and main speakers in the original main speaker list. For example, in the original main speaker list, object 1 corresponds to speaker 1 as the main speaker. After adjusting the speaker parameters of speaker 1 and / or speaker 2 based on the first alternative scheme, the main speaker for object 1 becomes speaker 2. Therefore, in the updated main speaker list, object 1 corresponds to speaker 2 as the main speaker.

[0142] In some embodiments, the execution module 730 can determine the distance influence of the corresponding sound on the object after executing the adjustment scheme (such as the first alternative scheme or the current round candidate scheme) based on a second preset rule, and then determine the main sound corresponding to each object based on the distance influence. The correspondence between the changed objects and the main sound is then updated in the original main sound list to obtain the updated main sound list. For methods of determining the distance influence and the main sound list, please refer to [link to relevant documentation]. Figure 3 And its related descriptions.

[0143] In some embodiments, the evaluation value of the adjustment scheme (such as the candidate schemes and the first alternative scheme in each round) is also related to the updated main speaker list corresponding to the adjustment scheme. The more concentrated the distribution of the main speakers in the updated main speaker list (i.e., the fewer different speakers in the updated main speaker list), the more similar the main speakers corresponding to each object in the monitoring environment are. When the adjustment target of the adjustment scheme is the main speakers of multiple objects, the evaluation value of the adjustment scheme is larger. The specific increase in evaluation value can be set based on experience.

[0144] In some embodiments of this specification, by relating the evaluation value of the adjustment scheme to the updated main speaker list corresponding to the adjustment scheme, the relevant parameters of the main speakers of multiple objects can be adjusted first, and a more accurate evaluation value of the adjustment scheme can be obtained according to the actual situation.

[0145] Step 630: Based on the evaluation value of the first alternative and the evaluation value of the candidate solutions in this round, determine the candidate solutions to enter the next iteration.

[0146] In some embodiments, the execution module 730 can compare the evaluation values ​​of the first alternative solution and the current round's candidate solution with preset iteration conditions. If the evaluation values ​​of the first alternative solution and the current round's candidate solution satisfy the preset iteration conditions, then the first alternative solution and the current round's candidate solution are selected as candidate solutions for the next iteration. The preset iteration conditions can be set in advance based on experience; for example, the preset iteration conditions can be that the evaluation value is higher than a fifth threshold.

[0147] For example, the evaluation values ​​of the first alternative solution 1, the first alternative solution 2, and the candidate solution 1 are 1.8, 3.1, and 4.0, respectively. The preset update condition is that if the evaluation value is higher than 3.0, then the first alternative solution 2 and the candidate solution 1 can be considered to meet the preset update condition and can be candidate solutions to enter the next round of iteration.

[0148] In some embodiments, after completing one round of iteration, it can be determined whether the iteration can be stopped based on preset update conditions. If the preset number of iteration rounds is met, at least one of the obtained adjustment schemes has an evaluation value higher than a preset evaluation threshold, or the difference between the maximum values ​​of the evaluation values ​​of the adjustment schemes obtained in two or more consecutive iterations is less than a preset difference threshold, it can be determined that the iteration can be stopped. Otherwise, the candidate schemes that meet the preset update conditions are selected as candidate schemes for the new round and a new round of iteration begins (e.g., return to step 610).

[0149] In some embodiments of this specification, by performing preset change processing on candidate solutions to obtain a first alternative solution, as many adjustment solutions as possible can be obtained for iteration; based on a preset evaluation algorithm, the evaluation value of the first alternative solution and the evaluation value of the candidate solutions in this round are determined, and then the candidate solutions to enter the next round of iteration are determined, which can make the determination of evaluation values ​​more accurate, improve computational efficiency, and help obtain adjustment solutions that provide a better listening experience.

[0150] Step 530: Determine the target solution based on the results of at least one round of iterations.

[0151] The target scheme can refer to a specific scheme for adjusting the audio parameters of a speaker. For example, the target scheme could be to move the horizontal coordinate of speaker 1 2 meters to the right, increase the tilt angle of the driver unit by 10°, decrease the volume by 5dB, decrease the equalizer parameters in the 64Hz frequency band by 6dB, and keep the other frequency bands unchanged; for speaker 2, the position coordinates remain unchanged, the tilt angle of the driver unit remains unchanged, the volume is increased by 5dB, and the equalizer parameters remain unchanged. The vector can be represented as ((1, (A)). x +2, A y A z ), (B+10), (C-5), (D 32 D 64 -6, ...), (2, (A) x A y A z ), (B), (C+5), (D 32 D 64 ,...))).

[0152] In some embodiments, the execution module 730 may select the adjustment scheme with the largest evaluation value among the candidate schemes obtained in each round of iteration after stopping the iteration, and among the first alternative schemes, as the target scheme. For details on determining the evaluation value, please refer to the foregoing embodiments and their related descriptions.

[0153] Step 540: Based on the target scheme, determine the target adjustment amount of the audio parameters of the audio system.

[0154] In some embodiments, the execution module 730 can determine the target adjustment amount of the audio parameters of each speaker based on the target scheme and the adjustment amount of the audio parameters of each speaker corresponding to the target scheme. For example, the target scheme is ((1, (A)). x +2, A y A z ), (B+10), (C-5), (D 32 D 64 -6, ...), (2, (A) x A y A z -1), (B-5), C, (D) 32 D 64 , ...))), the execution module 730 can determine the target adjustment amount of the audio parameters of the speaker 1 as (1, (A x +2, A y A z ), (B+10), (C-5), (D 32 D 64 -6, ...), that is, the horizontal coordinate of speaker 1 is moved 2 meters to the right, the tilt angle of the sound unit is increased by 10°, the volume is reduced by 5dB, the equalizer parameters for the 64Hz frequency band are reduced by 6dB, and the other frequency bands remain unchanged. The target adjustment amount 2 for the audio parameters of speaker 2 is (2, (A x A y A z -1), (B-5), C, (D) 32 D 64 (…), that is, the vertical coordinate of the speaker 2 position coordinate is moved down by 1 meter, the tilt angle of the speaker unit is reduced by 5°, the volume remains unchanged, and the equalizer parameters remain unchanged.

[0155] In some embodiments of this specification, by performing at least one round of iteration on at least one candidate solution, and determining the target solution and target adjustment amount based on the iteration results, more considerations can be incorporated into the determination of the target adjustment amount, such as user experience and evaluation values. This allows for effective adjustment of audio parameters based on the user's real-time experience, making the adjustment process more accurate and efficient.

[0156] Figure 7This is a schematic diagram of a sound parameter adjustment system according to some embodiments of this specification. In some embodiments, the sound parameter adjustment system 700 may include an acquisition module 710, a determination module 720, and an execution module 730.

[0157] The acquisition module 710 can be used to acquire environmental information of the monitoring environment based on a first preset rule.

[0158] The determination module 720 can be used to determine at least one object in the monitoring environment and its motion information based on environmental information.

[0159] The execution module 730 is configured to, in response to motion information satisfying a first preset condition, perform the following operations on at least one speaker: based on at least one object and its motion information, determine at least one set of candidate adjustment amounts for the sound parameters of each speaker in the at least one speaker; and based on the at least one set of candidate adjustment amounts, determine a target adjustment amount for the sound parameters of the speaker. In some embodiments, the execution module 730 may be configured to determine a list of main speakers based on at least one object and its motion information; and based on the list of main speakers, determine at least one set of candidate adjustment amounts for the sound parameters of each speaker in the at least one speaker. In some embodiments, the execution module 730 may also be configured to determine the distance influence and / or effect influence of each speaker on the object based on the motion information of each object in the at least one object; based on the distance influence and / or effect influence, determine the main speaker corresponding to each object; and based on the main speaker corresponding to each object, determine a list of main speakers. In some embodiments, the execution module 730 may be further configured to: determine at least one candidate scheme based on at least one set of candidate adjustment amounts; perform at least one round of iteration on the at least one candidate scheme until a preset update condition is met; determine a target scheme based on the results of at least one round of iteration; and determine a target adjustment amount for the audio parameters of the audio system based on the target scheme; wherein each round of iteration includes: performing a preset change process on the candidate schemes of the current round to obtain a first alternative scheme; in the first round of iteration, the candidate schemes of the current round are determined based on at least one set of candidate adjustment amounts; the preset change process includes at least one of crossover processing, mutation processing, and replication processing; determine the evaluation value of the first alternative scheme and the evaluation value of the candidate schemes of the current round based on a preset evaluation algorithm; and determine the candidate scheme to enter the next round of iteration based on the evaluation value of the first alternative scheme and the evaluation value of the candidate schemes of the current round.

[0160] For more details regarding the acquisition module 710, determination module 720, and execution module 730, please refer to [link to relevant documentation]. Figures 2-6 And its related descriptions.

[0161] It should be understood that Figure 7The system and its modules shown can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented using hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated hardware. Those skilled in the art will understand that the methods and systems described above can be implemented using computer-executable instructions and / or included in processor control code.

[0162] It should be noted that the above description of the audio parameter adjustment system and its modules is for convenience only and should not be construed as limiting this specification to the scope of the illustrated embodiments. It is understood that those skilled in the art, after understanding the principles of the system, may arbitrarily combine the various modules or construct subsystems connected to other modules without departing from these principles. In some embodiments, Figure 7 The acquisition module 710, determination module 720, and execution module 730 disclosed herein can be different modules within a single system, or a single module can implement the functions of two or more of the aforementioned modules. For example, the modules can share a single storage module, or each module can have its own separate storage module. Such variations are all within the scope of protection of this specification.

[0163] This specification provides one or more embodiments of an audio parameter adjustment device, including a processor for executing an audio parameter adjustment method.

[0164] This specification also provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions from the storage medium, the computer executes the audio parameter adjustment method as described in any of the above embodiments.

[0165] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0166] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0167] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0168] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0169] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0170] For each patent, patent application, patent application publication, and other material such as articles, books, specifications, publications, and documents referenced in this specification, the entire contents of which are incorporated herein by reference. This excludes historical application documents that are inconsistent with or conflict with the content of this specification, as well as documents that limit the broadest scope of the claims in this specification (currently or subsequently appended to this specification). It should be noted that in the event of any inconsistency or conflict between the descriptions, definitions, and / or terminology used in the supplementary materials to this specification and the content of this specification, the descriptions, definitions, and / or terminology used in this specification shall prevail.

[0171] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A method for controlling audio playback, characterized in that, include: Acquire environmental information of the monitored environment in real time or at intervals; Based on the environmental information, at least one object in the monitoring environment and its motion information are determined; In response to the motion information indicating that movement has occurred and the movement distance is greater than a threshold, the following operation is performed on at least one speaker: Based on the at least one object and its motion information, at least one set of candidate adjustment values ​​for the audio parameters of each of the at least one speakers are determined; Based on the at least one set of candidate adjustment values, at least one candidate solution is determined; The at least one candidate solution is iterated for at least one round until the preset update condition is met. The target solution is determined based on the results of at least one round of iterations; Based on the target scheme, determine the target adjustment amount of the audio parameters of the audio system; Each iteration in the at least one iteration includes: The candidate solutions in this round are subjected to a preset change process to obtain a first alternative solution; in the first round of iteration, the candidate solutions in this round are determined based on the at least one set of candidate adjustment amounts; the preset change process includes at least one of crossover processing, mutation processing and replication processing; Based on a preset evaluation algorithm, the evaluation value of the first alternative solution and the evaluation value of the candidate solutions in this round are determined. Based on the evaluation values ​​of the first alternative and the evaluation values ​​of the candidate solutions in this round, candidate solutions to enter the next iteration are determined.

2. The method according to claim 1, characterized in that, The step of determining at least one set of candidate adjustment values ​​for the audio parameters of each of the at least one speakers based on the at least one object and its motion information includes: Based on the at least one object and its motion information, a main speaker list is determined; Based on the main speaker list, at least one set of candidate adjustment values ​​for the audio parameters of each speaker in the at least one speaker is determined.

3. The method according to claim 2, characterized in that, The main speaker list includes information on multiple main speakers corresponding to an object; the level of each main speaker is determined based on the degree of influence of the corresponding speaker on the auditory experience of the corresponding object, and the greater the degree of influence, the higher the level.

4. The method according to claim 2, characterized in that, Determining the main audio list based on the at least one object and its motion information includes: Based on the motion information of each of the at least one objects, determine the distance influence and / or effect influence of each sound on the object; Based on the distance influence and / or the effect influence, determine the main speaker corresponding to each object; The list of main speakers is determined based on the main speaker corresponding to each of the objects.

5. The method according to claim 4, characterized in that, The process of determining the main speaker corresponding to each object based on the distance influence and / or the effect influence includes: The overall influence of each speaker is obtained by weighted summation of its distance and effect influence; wherein the weights of the distance and effect influence are related to the sensitivity of the object's experience to change, and the sensitivity to change is determined based on the difference between the experiences under different speaker parameters; the experience is determined based on an experience determination model, which is a machine learning model. The obtained comprehensive influence scores are sorted, and the speaker with the highest comprehensive influence score is determined as the main speaker.

6. The method according to claim 1, characterized in that, The probability of the candidate scheme undergoing the preset change processing is determined based on the main audio list.

7. The method according to claim 6, characterized in that, When performing the preset change processing on the candidate schemes, for multiple speakers in the candidate schemes, the volume parameter adjustment of the main speaker with the greater effect influence is more likely to be increased, and the volume parameter adjustment of the main speaker with the smaller effect influence is more likely to be decreased.

8. An audio playback control system, characterized in that, include: The acquisition module is configured to acquire environmental information of the monitored environment at certain time intervals or in real time. The first determining module is configured to determine at least one object in the monitoring environment and its motion information based on the environmental information. The execution module is configured to perform the following operation on at least one speaker in response to the motion information satisfying a first preset condition: Based on the at least one object and its motion information, at least one set of candidate adjustment values ​​for the audio parameters of each of the at least one speakers are determined; Based on the at least one set of candidate adjustment values, at least one candidate solution is determined; The at least one candidate solution is iterated for at least one round until the preset update condition is met. The target solution is determined based on the results of at least one round of iterations; Based on the target scheme, determine the target adjustment amount of the audio parameters of the audio system; Each iteration in the at least one iteration includes: The candidate solutions in this round are subjected to a preset change process to obtain a first alternative solution; in the first round of iteration, the candidate solutions in this round are determined based on the at least one set of candidate adjustment amounts; the preset change process includes at least one of crossover processing, mutation processing and replication processing; Based on a preset evaluation algorithm, the evaluation value of the first alternative solution and the evaluation value of the candidate solutions in this round are determined. Based on the evaluation values ​​of the first alternative and the evaluation values ​​of the candidate solutions in this round, candidate solutions to enter the next iteration are determined.

9. An audio playback control device, characterized in that, The device includes at least one processor and at least one memory; The at least one memory is used to store computer instructions; The at least one processor is used to execute at least some of the instructions in the computer instructions to implement the audio playback control method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions, which, when executed by a processor, implement the audio playback control method as described in any one of claims 1 to 7.

Images