Multi-device cooperative control method, device and medium

Abstract

The present disclosure provides a multi-device cooperative control method, device and medium, the method comprising: obtaining hinge posture data corresponding to a plurality of terminal devices respectively; wherein the hinge posture data comprises a hinge angle of an adjustable hinge structure of the terminal device; determining a cooperative model for representing a spatial acoustic distribution relationship of the plurality of terminal devices according to the hinge posture data corresponding to each terminal device respectively; determining target values of audio output parameters corresponding to each terminal device respectively according to the cooperative model; and sending each target value to the corresponding terminal device, and adjusting a current value of the audio output parameter to the corresponding target value through the corresponding terminal device.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent device collaborative control technology, and more specifically, to a multi-device collaborative control method, device, and medium. Background Technology

[0002] Currently, multi-device collaboration has become an important means of enhancing user experience. For example, in an in-vehicle environment, multi-speaker collaboration can create an immersive sound field; in a home environment, multi-speaker systems can achieve whole-house audio coverage. However, existing multi-device collaboration systems mainly rely on timestamps, network latency, or simple location information for collaboration, lacking the ability to perceive changes in the structural form of the devices themselves.

[0003] In related technologies, although some devices have adjustable structures (such as speaker cabinets or vehicle speaker units connected to mounting bases via rotatable brackets), their collaborative modeling is usually based on fixed installation positions, ignoring the impact of changes in device posture. Once the device posture changes, the system struggles to update the model in a timely manner, leading to sound field shifts, positioning errors, or degraded sound quality. Summary of the Invention

[0004] One objective of this disclosure is to provide a new technical solution for multi-device collaborative control.

[0005] According to a first aspect of the present disclosure, a multi-device collaborative control method is provided, the method comprising: Acquire hinge attitude data corresponding to multiple terminal devices respectively; wherein, the hinge attitude data includes the hinge angle of the adjustable hinge structure of the terminal device; Based on the hinge posture data corresponding to each of the terminal devices, a collaborative model is determined to characterize the spatial acoustic distribution relationship of the multiple terminal devices. Based on the collaborative model, the target values ​​of the audio output parameters corresponding to each of the terminal devices are determined. Each of the target values ​​is sent to the corresponding terminal device, and the current value of the audio output parameter is adjusted to the corresponding target value by the corresponding terminal device.

[0006] Optionally, determining a cooperative model to characterize the spatial acoustic distribution relationship of the multiple terminal devices based on the hinge posture data corresponding to each of the terminal devices includes: Time synchronization is performed on the hinge attitude data from each of the terminal devices to construct a global state vector; Based on the global state vector, determine the spatial direction vector corresponding to each terminal device and the relative positional relationship between each terminal device; The collaborative model is determined based on the spatial direction vectors corresponding to each terminal device and the relative positional relationships between the terminal devices.

[0007] Optionally, determining the target values ​​of the audio output parameters corresponding to each of the terminal devices based on the collaborative model includes: Obtain the target user's location information; Based on the location information and the collaborative model, target values ​​for the audio output parameters corresponding to each of the terminal devices are determined.

[0008] Optionally, determining the target values ​​of the audio output parameters corresponding to each of the terminal devices based on the location information and the collaboration model includes: Based on the location information of the target user and the collaborative model, the propagation distance from each terminal device to the target user, and the angle between the acoustic radiation direction of each terminal device and the target direction are determined. Based on the propagation distance from each terminal device to the target user, and the angle between the acoustic radiation direction of each terminal device and the target direction, the target values ​​of the audio output parameters corresponding to each terminal device are determined.

[0009] Optionally, determining the target values ​​of the audio output parameters corresponding to each of the terminal devices based on the location information and the collaboration model includes: If one of the target users is detected, enter single-user mode; When entering the single-user mode, the target values ​​of the audio output parameters corresponding to each terminal device are determined based on the location information of the target user and the collaboration model. If at least two of the target users are detected, enter multi-user mode; When entering the multi-user mode, the target values ​​of the audio output parameters corresponding to each of the terminal devices are determined based on the location information of at least two target users and the collaboration model.

[0010] Optionally, adjusting the current value of the audio output parameter to the target value via any terminal device includes: The current value of the audio output parameter is adjusted to the target value using any of the terminal devices in a gradual manner; The gradation method includes linearly interpolating the current value and target value of the audio output parameter according to the interpolation coefficient within a preset time window, so that the audio output parameter transitions from the current value to the target value.

[0011] Optionally, the method further includes: Obtain the change in hinge angle of each terminal device between two adjacent sampling times; If the change in hinge angle of any terminal device exceeds a preset attitude change threshold, the step of determining a cooperative model to characterize the spatial acoustic distribution relationship of multiple terminal devices based on the hinge attitude data of each terminal device is re-executed.

[0012] Optionally, the method further includes: Based on the hinge angles of each terminal device and the set mapping data, the acoustic coverage area of ​​each terminal device is determined; wherein, the mapping data reflects the correspondence between the hinge angles of each terminal device and their corresponding acoustic coverage areas. The overall coverage range of the multiple terminal devices is determined based on the acoustic coverage area corresponding to each terminal device. When the target user is outside the overall coverage area, the target values ​​of the audio output parameters corresponding to each of the terminal devices are adjusted to adjust the acoustic coverage area of ​​each terminal device so that the adjusted overall coverage area covers the target user.

[0013] According to a second aspect of the present disclosure, an electronic device is provided, including a memory and a processor, the memory being configured to store a computer program, and the processor being configured to execute the method described in accordance with the first aspect above under the control of the computer program.

[0014] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, the computer program implementing the method according to the first aspect above when executed by a processor.

[0015] One beneficial effect of this disclosure is that by acquiring hinge attitude data corresponding to multiple terminal devices, and determining a collaborative model to characterize the spatial acoustic distribution relationship of multiple terminal devices based on the hinge attitude data of each terminal device, and then determining and issuing target values ​​of audio output parameters corresponding to each terminal device based on the collaborative model, the collaborative model and audio output parameters can be automatically updated when the device hinge attitude changes without relying on fixed installation position modeling or manual recalibration. This achieves dynamic collaboration of the sound field of multiple devices as the structural attitude changes, improving the sound field positioning accuracy and user experience.

[0016] Other features and advantages of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of this specification.

[0018] Figure 1 This is a schematic diagram of the structure of a multi-device collaborative control system to which the multi-device collaborative control method of the present disclosure can be applied; Figure 2 This is a flowchart illustrating a multi-device collaborative control method according to some embodiments; Figure 3 This is a flowchart illustrating a multi-device collaborative control method according to other embodiments; Figure 4 This is a flowchart illustrating a multi-device collaborative control method according to other embodiments; Figure 5 This is a flowchart illustrating a multi-device collaborative control method according to other embodiments; Figure 6 This is a schematic diagram of the structure of an electronic device according to some embodiments. Detailed Implementation

[0019] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0020] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0021] Technologies and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such technologies and equipment should be considered part of the specification.

[0022] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0023] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0024] Figure 1 This is a schematic diagram of the structure of a multi-device collaborative control system to which the multi-device collaborative control method of the embodiments of this disclosure can be applied. For example... Figure 1 As shown, the multi-device collaborative control system 1 includes a processing device 21 and multiple terminal devices. Each terminal device is connected to the processing device 21 via a communication network.

[0025] The terminal device is a device with an adjustable hinge structure, such as a speaker or car loudspeaker. The adjustable hinge structure allows for a controllable angular change of the speaker acoustic components relative to the device body, and enables the acquisition of the current hinge angle. An angle sensor is integrated into the adjustable hinge structure to detect the hinge angle.

[0026] Each terminal device is equipped with at least an adjustable hinge structure, a processor, and a speaker acoustic component, which includes a speaker unit.

[0027] The processing device 21 includes at least a processor and a memory. The memory stores a computer program, and when the processor executes the computer program, it performs the following steps: receiving hinge posture data collected by each terminal device; determining a cooperative model to characterize the spatial acoustic distribution relationship of multiple terminal devices based on the hinge posture data corresponding to each terminal device; determining the target value of the audio output parameter corresponding to each terminal device based on the cooperative model; and sending each target value to the corresponding terminal device, thereby adjusting the current value of the audio output parameter to the corresponding target value through the corresponding terminal device.

[0028] In some embodiments, the processing device 21 may be a device other than the terminal devices, such as a vehicle host, home gateway, mobile phone and tablet computer, or it may be one of the multiple terminal devices. This disclosure does not limit it in this regard.

[0029] The following combination Figure 1 The system described herein illustrates various embodiments.

[0030] <First Embodiment> Figure 2 This is a flowchart illustrating a multi-device collaborative control method according to some embodiments. This multi-device collaborative control method can be executed by a processing device, which can be... Figure 1 The processing device 21 shown. For example... Figure 2 As shown, the execution of this embodiment may include the following steps S210 to S240.

[0031] Step S210: Obtain hinge attitude data corresponding to multiple terminal devices respectively; wherein, the hinge attitude data includes the hinge angle of the adjustable hinge structure of the terminal device.

[0032] An adjustable hinge structure refers to a mechanical structure that enables the speaker acoustic components of a terminal device to produce a controllable angular change relative to the device body and can acquire the current hinge angle. Taking a vehicle speaker as an example, a fixed mounting base is installed near the door, dashboard, or seat. The speaker acoustic components are rotatably connected to the fixed mounting base via a pivot, forming an adjustable hinge structure. The adjustable hinge structure integrates an angle sensor, such as a magnetic encoder or rotary encoder, which can acquire the current hinge angle of the adjustable hinge structure.

[0033] The hinge attitude data may include the hinge angle. Hinge angle It reflects the current rotation angle of the speaker acoustic component of the i-th terminal device relative to its mounting reference state.

[0034] Hinge attitude data can also include hinge angular velocity. hinge angular velocity Let represent the rate of change of the hinge angle of the i-th terminal device over time, and . .

[0035] In this embodiment, the hinge attitude data is collected by each terminal device and uploaded to the processing device in real time or periodically.

[0036] Step S220: Based on the hinge attitude data corresponding to each terminal device, determine a collaborative model to characterize the spatial acoustic distribution relationship of multiple terminal devices.

[0037] The cooperative model is a mathematical model that describes the acoustic propagation relationship between the spatial location of each terminal device, the acoustic radiation direction, and the location of the target user. The cooperative model can be at least one of the following: sound pressure level distribution model, sound energy distribution model, acoustic transfer function model, and beam coverage area model, or a combination of the above models. This disclosure does not limit the specific model used.

[0038] In one example, step S220, which determines the cooperative model for characterizing the spatial acoustic distribution relationship of multiple terminal devices based on the hinge attitude data corresponding to each terminal device, may further include the following steps S221 to S223: Step S221: Time synchronization of hinge attitude data from each terminal device is performed to construct a global state vector.

[0039] In step S221, since the starting time and sampling frequency of the hinge attitude data sampling of each terminal device may differ, the processing device first performs time alignment of the data of different terminal devices according to the timestamp carried by each hinge attitude data, and then obtains the global state vector S(t) under the same time reference. The global state vector can be determined according to the following formula (1): (1) in, This represents the hinge angle of the i-th terminal device at time t. Let represent the hinge angular velocity of the i-th terminal device at time t, N be the total number of terminal devices, and t represent the current timestamp.

[0040] For example, in an in-vehicle scenario, the left speaker collects the hinge angle at time t1. The hinge angle was collected by the right speaker at time t2. The processing device aligns the two to a unified time base based on the timestamp, obtaining the global state vector at the same moment, thereby ensuring the temporal consistency of subsequent spatial modeling.

[0041] Step S222: Based on the global state vector, determine the spatial direction vector corresponding to each terminal device and the relative positional relationship between each terminal device.

[0042] The spatial orientation vector characterizes the principal radiation direction of the loudspeaker acoustic components of each terminal device within the hinge rotation plane. For the i-th terminal device, its spatial orientation vector... Based on the corresponding hinge angle The spatial direction vector is composed of the cosine and sine values. When the hinge angle When changes occur, the spatial direction vector It is updated accordingly, thus reflecting the main radiation direction of the speaker acoustic components of the terminal device in real time.

[0043] The relative positional relationship refers to the spatial positional relationship and spatial orientation relationship between each terminal device in a unified coordinate system (such as a vehicle coordinate system or an indoor space coordinate system). The spatial positional relationship includes the relative position vector and / or relative distance between any two terminal devices, and the spatial orientation relationship includes the relative orientation between any two terminal devices.

[0044] In step S222, during device installation or initialization, the processing device will pre-record the installation reference position of each terminal device. and installation reference attitude After the hinge angle changes, the processing equipment will adjust according to the installation reference position of each terminal device. Installation reference posture And the hinge angle of each terminal device at present. Determine the current spatial location of each terminal device. and current spatial attitude And based on the current spatial location and current spatial attitude Determine the relative positional relationships.

[0045] Specifically, current spatial location It can be determined according to the following formula (2): (2) in, This indicates the change in position caused by the rotation of the hinge.

[0046] Specifically, current spatial attitude It can be determined according to the following formula (3): (3) in, Indicates the hinge angle A defined rotation transformation matrix.

[0047] Obtain the current spatial location of each terminal device Given the current spatial pose Ri, the relative position vector between any two terminal devices i and j relative distance relative orientation according to and The rotational relationship between them is determined. The relative position vector, relative distance, and relative orientation together constitute the relative positional relationship.

[0048] Step S223: Determine the collaborative model based on the spatial direction vectors corresponding to each terminal device and the relative positional relationships between each terminal device.

[0049] In step S223, the processing device uses the spatial direction vectors corresponding to each terminal device. In addition to the relative positional relationships between each terminal device, a collaborative model is established to describe the spatial acoustic distribution relationship of each terminal device. This collaborative model takes the spatial direction vector and relative positional relationship of each terminal device as known input conditions, and then takes the target user's position as external input. The collaborative model can then solve for the acoustic propagation parameters corresponding to each terminal device, and then determine the audio output parameters based on the acoustic propagation parameters.

[0050] For example, the collaborative model can be an acoustic transfer function model. For the i-th terminal device, its trajectory from its current spatial location P can be established. i Acoustic response function H pointing to spatial location X i (X), this function characterizes the acoustic contribution of terminal device i to spatial location X. Multiple terminal devices work together to form the overall sound field, and the overall sound field distribution can be expressed as... Furthermore, sound pressure level, sound energy, or transfer function distributions can be established for different frequencies to form a three-dimensional sound field distribution model.

[0051] Through the above steps S221 to S223, the spatial acoustic distribution relationship of each terminal device can still be accurately reflected even when the device structure changes, providing an accurate model basis for determining subsequent audio output parameters and improving the sound field positioning accuracy and user experience of multi-device collaboration.

[0052] Step S230: Based on the collaborative model, determine the target values ​​of the audio output parameters corresponding to each terminal device.

[0053] The audio output parameters refer to the parameters used to control the audio output state of each terminal device so that multiple terminal devices can collaboratively form a target sound field. The audio output parameters may include at least one of audio output gain, delay compensation, phase compensation, and beamforming weight. The target value refers to the audio output parameter value that each terminal device should achieve, obtained based on the current collaborative model and the location information of the target user.

[0054] In some examples, step S230, which determines the target values ​​of the audio output parameters corresponding to each terminal device based on the collaborative model, can be implemented in the following way: obtain the location information of the target user; and determine the target values ​​of the audio output parameters corresponding to each terminal device based on the location information and the collaborative model.

[0055] Among them, the target user's location information This refers to the target user's position coordinates in a unified spatial coordinate system. This location information... The location can be acquired through a location sensing device used to sense the user's position. This location sensing device may include at least one of an image acquisition device, a pressure sensing device, a wireless sensing device, or a positioning device. This location sensing device can be located on a corresponding terminal device or a processing device. The location information of the target user in a unified spatial coordinate system can be determined using this location sensing device. .

[0056] The above-mentioned determination of the target values ​​of the audio output parameters corresponding to each terminal device based on location information and collaborative model can further include the following steps S231 to S232: Step S231: Based on the target user's location information and the collaborative model, determine the propagation distance from each terminal device to the target user, as well as the angle between the acoustic radiation direction of each terminal device and the target direction.

[0057] Specifically, the target user location Input the collaborative model, which can be based on the target user's location information. Determine the propagation distance from each terminal device to the target user. and the angle between the acoustic radiation direction of each terminal device and the target direction. , as acoustic propagation parameters; where the target direction is the location information from the current spatial location of the terminal device to the target user. The direction.

[0058] Step S232: Based on the propagation distance from each terminal device to the target user and the angle between the acoustic radiation direction of each terminal device and the target direction, determine the target value of the audio output parameter corresponding to each terminal device.

[0059] Specifically, after obtaining the propagation distance corresponding to each terminal device and included angle Then, it can be determined based on the transmission distance. and included angle Determine the audio output parameters of each terminal device.

[0060] Using audio output parameters as audio output gain For example, the audio output gain of the i-th terminal device It can be determined according to the following formula (4): (4) in, The normalization coefficient is... Let be the angle between the acoustic radiation direction of the i-th terminal device and the target direction. This is the directional gain function related to the acoustic radiation direction of the terminal device. Let be the propagation distance between the i-th terminal device and the target user. A very small positive number set to prevent the denominator from being zero.

[0061] Taking the audio output parameters as the delay compensation amount as an example, the delay compensation amount of the i-th terminal device It can be determined according to the following formula (5): (5) Where c is the speed of sound. Let be the propagation distance between the i-th terminal device and the target user.

[0062] In other words, when the hinge angle of any terminal device changes, its current spatial position or orientation changes accordingly, the collaborative model is updated, and the coordinate system is redefined based on the updated collaborative model. and This updates the audio output gain. The delay compensation amount τi enables the audio output of each terminal device to be dynamically adjusted according to the changes in the device's structural posture, thereby forming a stable enhanced sound field at the target user's location, improving the sound field positioning accuracy and user experience.

[0063] Step S240: Send each target value to the corresponding terminal device, and adjust the current value of the audio output parameter to the corresponding target value through the corresponding terminal device.

[0064] Specifically, after determining the target values ​​for the audio output parameters corresponding to each terminal device, the processing device sends each target value to the corresponding terminal device. Each terminal device receives the target value and adjusts the current value of its audio output parameter to the corresponding target value.

[0065] After each terminal device completes its adjustment, it processes its own input audio signal according to the adjusted audio output parameters. For example, the i-th terminal device processes its input audio signal according to the adjusted audio output gain. The input audio signal is adjusted for gain and delayed by a delay compensation amount τi to obtain the output signal of the terminal device. The output signals of multiple terminal devices are superimposed in space to form a multi-device collaborative output signal.

[0066] In some examples, step S240, which adjusts the current value of the audio output parameter to the target value through any terminal device, can be implemented in the following way: the current value of the audio output parameter is adjusted to the target value through any terminal device in a gradual manner.

[0067] The gradation method involves linearly interpolating the current and target values ​​of the audio output parameters within a preset time window T using an interpolation coefficient λ, thus transitioning the audio output parameters from the current value to the target value. Typically, the interpolation coefficient λ characterizes the transition from the current value to the target value, and λ = t / T, where t is the time elapsed since the start of parameter adjustment, T is the preset time window, and 0 ≤ λ ≤ 1. When λ = 0, the audio output parameter is the current value; when λ = 1, the audio output parameter is the target value; and when 0 < λ < 1, the audio output parameter is the interpolation result between the current and target values.

[0068] By performing linear interpolation on the audio output parameters within a preset time window, the audio output parameters can smoothly transition from old values ​​to new values, effectively suppressing the impact of parameter mutations on the listening experience.

[0069] Through the embodiments of this disclosure, hinge attitude data corresponding to multiple terminal devices are obtained, and a collaborative model for characterizing the spatial acoustic distribution relationship of multiple terminal devices is determined based on the hinge attitude data of each terminal device. Then, the target values ​​of the audio output parameters corresponding to each terminal device are determined and issued based on the collaborative model. Without relying on fixed installation position modeling or manual recalibration, the collaborative model and audio output parameters can be automatically updated when the device hinge attitude changes, realizing dynamic collaboration of the sound field of multiple devices as the structural attitude changes, improving the sound field positioning accuracy and user experience.

[0070] <Second Embodiment> Figure 3 This is a flowchart illustrating a multi-device collaborative control method according to some embodiments. This multi-device collaborative control method can be executed by a processing device, which can be... Figure 1 The processing device 21 shown. For example... Figure 3 As shown, the execution of this embodiment may include the following steps S310 to S350.

[0071] Step S310: Obtain hinge attitude data corresponding to multiple terminal devices.

[0072] Step S320: Based on the hinge attitude data corresponding to each terminal device, determine a collaborative model to characterize the spatial acoustic distribution relationship of multiple terminal devices.

[0073] Step S330: Obtain the location information of the target user.

[0074] Steps S310 to S320 above correspond to steps S210 to S220 in the first embodiment. Step S330 corresponds to the sub-step of obtaining the target user's location information in step S230 of the first embodiment. The specific implementation method has been described in detail in the first embodiment and will not be repeated here.

[0075] Step S340: Based on the location information and the collaborative model, determine the target values ​​of the audio output parameters corresponding to each terminal device.

[0076] Based on step S330 in the first embodiment, this step further introduces a dynamic switching strategy between single-user mode and multi-user mode to adapt to the collaborative control needs of different numbers of target users.

[0077] Specifically, the processing device obtains the current number of valid users M through the user detection module in the processing device, and selects the corresponding coordination strategy according to the value of the current number of valid users M: Upon detecting a target user (M=1), enter single-user mode. Single-user mode refers to storing the location information of this unique target user. The location is used as the sole target location and input into the collaborative model. The collaborative model is based on the location information of this sole target user. Determine the propagation distance from each terminal device to the target user. and included angle This allows for the determination of target values ​​for the audio output parameters of each terminal device, enabling all terminal devices to collectively form a directional sound field directed towards the target user. For example, in an in-vehicle scenario, when only the driver is detected using the voice interaction function, a single-user mode is entered, and multiple speakers work together to form a directional sound field towards the driver.

[0078] If at least two target users are detected (M≥2), the system enters multi-user mode. Multi-user mode refers to acquiring the location information of each target user separately, inputting multiple location information into a collaborative model, and ultimately forming multiple sound field target regions.

[0079] Step S350: Send each target value to the corresponding terminal device, and adjust the current value of the audio output parameter to the corresponding target value through the corresponding terminal device.

[0080] Through the embodiments of this disclosure, by dynamically switching between single-user mode and multi-user mode according to the number of effective users, the collaborative control strategy can adapt to different usage scenarios, achieving directional sound field focusing in single-user scenarios and sound field coverage of multi-target areas in multi-user scenarios.

[0081] <Third Embodiment> Figure 4 This is a flowchart illustrating a multi-device collaborative control method according to some embodiments. This multi-device collaborative control method can be executed by a processing device, which can be... Figure 1 The processing device 21 shown. For example... Figure 4 As shown, the execution of this embodiment may include the following steps S410 to S460.

[0082] Step S410: Obtain hinge attitude data corresponding to multiple terminal devices.

[0083] Step S420: Based on the hinge attitude data corresponding to each terminal device, determine a collaborative model to characterize the spatial acoustic distribution relationship of multiple terminal devices.

[0084] Step S430: Based on the collaborative model, determine the target values ​​of the audio output parameters corresponding to each terminal device.

[0085] Step S440: Send each target value to the corresponding terminal device, and adjust the current value of the audio output parameter to the corresponding target value through the corresponding terminal device.

[0086] The steps S410 to S440 described above correspond to steps S210 to S240 in the first embodiment. The specific implementation method has been described in detail in the first embodiment and will not be repeated here.

[0087] Step S450: Obtain the change in hinge angle of each terminal device between two adjacent sampling times.

[0088] Step S460: If the change in hinge angle of any terminal device exceeds the preset attitude change threshold, step S420 is executed again.

[0089] The preset attitude change threshold can be set according to the type of terminal device, installation method, usage scenario, and user requirements for sound field stability.

[0090] Specifically, the processing device compares the hinge angle change of each terminal device between two adjacent sampling times with a preset attitude change threshold. When the hinge angle change of any terminal device exceeds the preset attitude change threshold, it is determined that the terminal device has undergone an attitude change. The processing device then re-executes step S420, that is, redetermines the cooperative model based on the updated hinge attitude data, and further updates the target values ​​of the audio output parameters corresponding to each terminal device.

[0091] In some examples, it can also be based on the hinge angular velocity. It helps determine the severity of posture changes.

[0092] Through the embodiments of this disclosure, real-time response to changes in device posture can be achieved, enabling audio output parameters to be updated promptly after substantial changes in the device's spatial state, thereby ensuring the continuous accuracy and stability of the multi-device collaborative sound field.

[0093] <Fourth Embodiment> Figure 5 This is a flowchart illustrating a multi-device collaborative control method according to some embodiments. This multi-device collaborative control method can be executed by a processing device, which can be... Figure 1 The processing device 21 shown. For example... Figure 5 As shown, the execution of this embodiment may include the following steps S510 to S570.

[0094] Step S510: Obtain hinge attitude data corresponding to multiple terminal devices.

[0095] Step S520: Based on the hinge attitude data corresponding to each terminal device, determine a collaborative model to characterize the spatial acoustic distribution relationship of multiple terminal devices.

[0096] Step S530: Based on the collaborative model, determine the target values ​​of the audio output parameters corresponding to each terminal device.

[0097] The steps S510 to S530 described above correspond to steps S210 to S230 in the first embodiment. The specific implementation method has been described in detail in the first embodiment and will not be repeated here.

[0098] Step S540: Determine the acoustic coverage area of ​​each terminal device based on the hinge angle corresponding to each terminal device and the set mapping data.

[0099] The mapping data reflects the correspondence between the hinge angles of each terminal device and their corresponding acoustic coverage areas.

[0100] Specifically, the deployed state of the equipment is determined by the hinge angle. The processing equipment pre-establishes the correspondence between the hinge angle and the speaker radiation direction and coverage area; that is, the mapping data reflects the correspondence between the hinge angle of each terminal device and its corresponding acoustic coverage area. Based on the current hinge angle of each terminal device, the current deployed state of each terminal device can be determined.

[0101] Here, in response to a change in the hinge angle of any terminal device, the acoustic coverage area of ​​each terminal device can be determined based on the corresponding hinge angle of each terminal device and the set mapping data.

[0102] Step S550: Determine the overall coverage range of multiple terminal devices based on the acoustic coverage area corresponding to each terminal device.

[0103] Step S560: When the target user is outside the overall coverage area, adjust the target values ​​of the audio output parameters corresponding to each terminal device to adjust the acoustic coverage area of ​​each terminal device so that the adjusted overall coverage area covers the target user.

[0104] For example, in a home multi-speaker scenario, when a target user is detected to be outside the overall coverage area, the audio output gain of the terminal device corresponding to the target direction can be increased to adjust the acoustic coverage area of ​​that terminal device; and / or, the delay compensation of other terminal devices can be adjusted so that the acoustic coverage area of ​​other terminal devices covers the target user.

[0105] Step S570: The adjusted target values ​​are sent to the corresponding terminal devices, and the current values ​​of the audio output parameters are adjusted to the corresponding target values ​​by the corresponding terminal devices.

[0106] Step S570 corresponds to step S240 in the first embodiment. The specific implementation method has been described in detail in the first embodiment and will not be repeated here.

[0107] Through the embodiments disclosed herein, the sound field coverage range can be automatically adjusted according to the deployed state of the device without manual intervention, ensuring that the target user is always within the effective sound field coverage, thereby improving the coverage flexibility and user experience of the multi-device collaborative sound field.

[0108] <Fifth Embodiment> Figure 6 These are schematic diagrams of the structure of an electronic device according to some embodiments. For example... Figure 6As shown, the electronic device 600 may include: a memory 620 for storing executable computer instructions; and a processor 610 for executing the method of any of the above embodiments under the control of the executable computer instructions. The processor may be one or more processors, which may execute the instructions individually or jointly. Similarly, the memory may be one or more memories, which may store the computer program individually or jointly. The electronic device 600 may be... Figure 1 The processing device 21 shown.

[0109] <Sixth Embodiment> This disclosure also provides a computer-readable storage medium storing computer instructions thereon, which are executed by a processor to perform the methods provided in this disclosure.

[0110] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0111] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0112] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0113] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0114] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0115] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0116] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a portion of an apparatus, segment, or instruction, which includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are equivalent.

[0118] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A multi-device collaborative control method, characterized in that, The method includes: Acquire hinge attitude data corresponding to multiple terminal devices respectively; wherein, the hinge attitude data includes the hinge angle of the adjustable hinge structure of the terminal device; Based on the hinge posture data corresponding to each of the terminal devices, a collaborative model is determined to characterize the spatial acoustic distribution relationship of the multiple terminal devices. Based on the collaborative model, the target values ​​of the audio output parameters corresponding to each of the terminal devices are determined. Each of the target values ​​is sent to the corresponding terminal device, and the current value of the audio output parameter is adjusted to the corresponding target value by the corresponding terminal device.

2. The method according to claim 1, characterized in that, The step of determining a collaborative model to characterize the spatial acoustic distribution relationship of multiple terminal devices based on the hinge posture data corresponding to each of the terminal devices includes: Time synchronization is performed on the hinge attitude data from each of the terminal devices to construct a global state vector; Based on the global state vector, determine the spatial direction vector corresponding to each terminal device and the relative positional relationship between each terminal device; The collaborative model is determined based on the spatial direction vectors corresponding to each terminal device and the relative positional relationships between the terminal devices.

3. The method according to claim 1, characterized in that, The step of determining the target values ​​of the audio output parameters corresponding to each of the terminal devices according to the collaborative model includes: Obtain the target user's location information; Based on the location information and the collaborative model, target values ​​for the audio output parameters corresponding to each of the terminal devices are determined.

4. The method according to claim 3, characterized in that, The step of determining the target values ​​of the audio output parameters corresponding to each of the terminal devices based on the location information and the collaborative model includes: Based on the location information of the target user and the collaborative model, the propagation distance from each terminal device to the target user, and the angle between the acoustic radiation direction of each terminal device and the target direction are determined. Based on the propagation distance from each terminal device to the target user, and the angle between the acoustic radiation direction of each terminal device and the target direction, the target values ​​of the audio output parameters corresponding to each terminal device are determined.

5. The method according to claim 3, characterized in that, The step of determining the target values ​​of the audio output parameters corresponding to each of the terminal devices based on the location information and the collaborative model includes: If one of the target users is detected, enter single-user mode; When entering the single-user mode, the target values ​​of the audio output parameters corresponding to each terminal device are determined based on the location information of the target user and the collaboration model. If at least two of the target users are detected, enter multi-user mode; When entering the multi-user mode, the target values ​​of the audio output parameters corresponding to each of the terminal devices are determined based on the location information of at least two target users and the collaboration model.

6. The method according to claim 1, characterized in that, Adjusting the current value of the audio output parameter to the target value via any terminal device includes: The current value of the audio output parameter is adjusted to the target value using any of the terminal devices in a gradual manner; The gradation method includes linearly interpolating the current value and target value of the audio output parameter according to the interpolation coefficient within a preset time window, so that the audio output parameter transitions from the current value to the target value.

7. The method according to claim 1, characterized in that, The method further includes: Obtain the change in hinge angle of each terminal device between two adjacent sampling times; If the change in hinge angle of any terminal device exceeds a preset attitude change threshold, the step of determining a cooperative model to characterize the spatial acoustic distribution relationship of multiple terminal devices based on the hinge attitude data of each terminal device is re-executed.

8. The method according to claim 1, characterized in that, The method further includes: Based on the hinge angles of each terminal device and the set mapping data, the acoustic coverage area of ​​each terminal device is determined; wherein, the mapping data reflects the correspondence between the hinge angles of each terminal device and their corresponding acoustic coverage areas. The overall coverage range of the multiple terminal devices is determined based on the acoustic coverage area corresponding to each terminal device. When the target user is outside the overall coverage area, the target values ​​of the audio output parameters corresponding to each of the terminal devices are adjusted to adjust the acoustic coverage area of ​​each terminal device so that the adjusted overall coverage area covers the target user.

9. An electronic device, characterized in that, It includes a memory and a processor, the memory being used to store a computer program, and the processor being used, under the control of the computer program, to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images