A method for directional audio playback for embodied agents and embodied agents

Abstract

The application relates to the technical field of human-computer interaction, and discloses a directional audio playing method of a body mechanism and the body mechanism. The method comprises the following steps: acquiring spatial orientation data of a target object; determining beamforming parameters of an acoustic transducer array based on the spatial orientation data, wherein the beamforming parameters comprise amplitude data and phase data corresponding to a plurality of transducing units in the acoustic transducer array; modulating a to-be-replayed audio signal to an ultrasonic carrier wave to generate an ultrasonic modulated signal; and driving the plurality of transducing units to emit the ultrasonic modulated signal based on the beamforming parameters. Through the technical scheme, the problem that related technologies are difficult to realize directional audio playing with high directivity and dynamic following of an interactive object in a limited space is solved, directional audio playing with directivity and dynamic following of an interactive object under the condition of a limited space is realized, and the human-computer interaction experience is improved.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction technology, specifically to a directional audio playback method and an embodied mechanism based on an embodied mechanism. Background Technology

[0002] With the development of embodied intelligence technology, robots are gradually entering public places such as shopping malls and factories to perform interactive tasks such as guiding, explaining, and providing services. In noisy environments, robots need to deliver clear voice information to specific interactive objects while avoiding noise interference with others around them, and have the ability to dynamically adjust the direction of sound projection as the interactive object moves.

[0003] In related technologies, robots commonly use dynamic loudspeakers as directional audio playback devices. However, this approach has several drawbacks: Firstly, the size of the robot's head limits the ability to stack multiple acoustic transducer arrays to achieve sufficient sound pressure levels. Secondly, while placing loudspeakers on the body can increase sound pressure, the sound direction remains fixed and cannot change with the robot's head rotation. Furthermore, it cannot actively follow the positional changes of the interactive object, making it impossible to achieve sufficient sound pressure levels within a limited space. Additionally, the large directional angle can cause significant sound pollution. Therefore, achieving highly directional audio playback that dynamically follows interactive objects within a limited space is a pressing technical problem that needs to be solved in this field. Summary of the Invention

[0004] This application provides a method for directional audio playback using an embodied mechanism and an embodied mechanism to solve the problem that related technologies are unable to achieve highly directional audio playback that can dynamically follow interactive objects within a limited space.

[0005] In a first aspect, this application provides a method for directional audio playback of an embodied mechanism, the method comprising: Obtain the spatial orientation data of the target object; Based on the spatial orientation data, the beamforming parameters of the acoustic transducer array are determined. The beamforming parameters include amplitude data and phase data corresponding to multiple transducer units in the acoustic transducer array. The audio signal to be reproduced is modulated onto an ultrasonic carrier wave to generate an ultrasonic modulated signal; and Based on the beamforming parameters, the plurality of transducer units are driven to emit the ultrasonic modulation signal to form an ultrasonic beam toward the target object. The ultrasonic beam is configured to self-demodulate an audible sound signal corresponding to the audio signal to be reproduced during propagation toward the target object.

[0006] According to the directional audio playback method for the embodied mechanism provided in this application, by acquiring the spatial orientation data of the target object and determining the amplitude and phase data of multiple transducer units in the acoustic transducer array based on the spatial orientation data, electronic beamforming control can be performed on the acoustic transducer array. This allows the ultrasonic modulation signals emitted by the multiple transducer units to form an ultrasonic beam directed towards the target object. Simultaneously, by modulating the audio signal to be reproduced onto an ultrasonic carrier, the ultrasonic modulation signal self-demodulates through the nonlinear effect of air as it propagates in the air, forming an audible sound signal corresponding to the audio signal to be reproduced. Therefore, a highly directional directional audio playback sound field can be formed within the limited installation space of the embodied mechanism, reducing reliance on multiple traditional loudspeakers or multiple independent acoustic arrays, which helps to reduce system size, power consumption, and structural load. Furthermore, by dynamically updating the beamforming parameters according to the spatial orientation data of the target object, the main lobe direction, focusing area, or audible coverage area of ​​the ultrasonic beam can be adjusted according to the position of the target object, thereby improving the problem that traditional directional sound fields are difficult to follow the movement of interactive objects. Furthermore, due to the strong directivity of ultrasonic beams, the resulting audible sound signal primarily covers the area where the target object is located, which helps reduce sound energy leakage in non-target areas, minimizes ambient noise interference, and enhances the privacy of audio interaction. Therefore, the method provided in this application can achieve highly directive, dynamically tracking, and low-leakage directional audio playback within a limited space, thereby improving the human-computer interaction experience of embodied mechanisms.

[0007] In one optional implementation, obtaining the spatial orientation data of the target object includes: Acquire the sound signal of the target object using an acoustic sensor, and determine the first orientation information of the target object based on the sound signal; and / or Image information of the target object is acquired through a visual sensor, and the second location information of the target object is determined based on the image information; The spatial orientation data is determined based on the first orientation information, the second orientation information, or the fusion information of the first orientation information and the second orientation information.

[0008] In one alternative implementation, forming an ultrasonic beam toward the target object includes: controlling the main lobe direction of the ultrasonic beam emitted by the plurality of transducers to point toward the target object, and / or positioning the focusing area of ​​the ultrasonic beam or the coverage area of ​​the audible sound signal within or maintaining a preset range based on the position of the target object.

[0009] In one alternative implementation, the method further includes: Continuously or periodically acquire the spatial orientation data of the target object; In response to changes in the spatial orientation data over time, pointing control information for maintaining the ultrasonic beam toward the target object is determined; and Based on the pointing control information, the physical orientation of the acoustic transducer array is adjusted and the beamforming parameters are updated so that the ultrasonic beam remains oriented toward the target object as the orientation of the target object changes.

[0010] In one alternative implementation, determining the pointing control information for maintaining the ultrasonic beam toward the target object includes: The target object is identified as the locked object; Based on the continuously or periodically acquired spatial orientation data, the movement status or movement trajectory of the locked object is determined. Based on the movement state or the movement trajectory, a driving command is generated to drive the actuator of the embodied mechanism to move, thereby adjusting the physical orientation of the acoustic transducer array.

[0011] In one alternative implementation, modulating the audio signal to be reproduced onto an ultrasonic carrier includes: The audio signal to be reproduced is loaded onto the ultrasonic carrier with a frequency greater than 20 kHz using at least one modulation method selected from double-sideband modulation, single-sideband modulation, amplitude modulation, or square root predistortion modulation. This embodiment allows for flexible selection of different modulation strategies. Double-sideband modulation is simple to implement, single-sideband modulation has high bandwidth efficiency, amplitude modulation is a mature implementation method, and square root predistortion processing can improve the audio quality after demodulation. A balance can be achieved between power consumption, bandwidth, and audio quality according to application requirements, improving the system's applicability and flexibility.

[0012] In one optional implementation, the highest frequency of the audio signal to be reproduced is less than or equal to a preset proportion of the frequency of the ultrasonic carrier, and when amplitude modulation is used, the modulation depth used when the audio signal to be reproduced is modulated onto the ultrasonic carrier is less than or equal to a preset threshold. The preset ratio and the preset threshold are respectively preset according to the frequency of the ultrasonic carrier, the modulation method and the target distortion index.

[0013] In one optional implementation, driving the plurality of transducer units to transmit the ultrasonic modulation signal based on the beamforming parameters includes: Based on the beamforming parameters and the ultrasonic modulation signal, multi-channel drive data corresponding one-to-one with the plurality of transducer units is generated; and The multi-channel drive data is time-synchronized and / or phase-synchronized and output to drive the multiple transducer units to transmit the ultrasonic modulation signal according to the corresponding amplitude data and phase data.

[0014] Secondly, this application provides an embodied mechanism, including: Acoustic transducer array, comprising multiple transducer units; At least one sensor; and A controller coupled to the acoustic transducer array and the at least one sensor, the controller being configured to: Spatial orientation data of the target object are acquired through at least one sensor; Based on the spatial orientation data, the beamforming parameters of the acoustic transducer array are determined, and the beamforming parameters include amplitude data and phase data corresponding to the plurality of transducer units; The audio signal to be reproduced is modulated onto an ultrasonic carrier wave to generate an ultrasonic modulation signal; and Based on the beamforming parameters, the plurality of transducer units are driven to emit the ultrasonic modulation signal to form an ultrasonic beam toward the target object. The ultrasonic beam is configured to self-demodulate an audible sound signal corresponding to the audio signal to be reproduced during propagation toward the target object.

[0015] In one optional implementation, the arrangement structure of the plurality of transducer units in the acoustic transducer array includes at least one of a rectangular array, a honeycomb array, a concentric ring array, an arc surface array, and a curved conformal array.

[0016] In one alternative embodiment, the acoustic transducer array has a highest ultrasonic operating frequency, and the center-to-center spacing between two adjacent transducer units is configured to be no greater than half the wavelength of the sound wave corresponding to the highest ultrasonic operating frequency.

[0017] In one alternative implementation, the highest ultrasonic operating frequency is greater than 20 kHz; In one alternative implementation, the number of multiple transducer units is not less than 64.

[0018] In one alternative embodiment, the embodied mechanism is a robot, and the acoustic transducer array is disposed on the head or torso of the robot and is symmetrically distributed about the central axis of the head or torso.

[0019] In one alternative embodiment, the acoustic transducer array is disposed on the outer surface of the housing of the head or torso or inside the housing.

[0020] In one optional implementation, the controller includes a first processing unit and a second processing unit coupled to the first processing unit; wherein, The first processing unit is configured to perform the generation of the ultrasonic modulation signal and the determination of the beamforming parameters; The second processing unit is coupled to the first processing unit via a data bus or communication interface and is configured to perform multi-channel parallel calculations based on the beamforming parameters and the ultrasonic modulation signal to generate multi-channel drive data corresponding to the plurality of transducer units.

[0021] In one alternative implementation, the controller further includes a multi-channel drive circuit electrically connected between the second processing unit and the acoustic transducer array, configured to receive the multi-channel drive data transmitted by the second processing unit and drive the plurality of transducer units.

[0022] In one alternative implementation, at least one sensor includes at least one of the following: An acoustic sensor is configured to acquire sound signals from the target object for sound source localization to determine first location information; A visual sensor is configured to acquire image information of the target object for target recognition, used to determine second positional information; or An attitude sensor is configured to acquire attitude data of the embodied mechanism to determine the relative orientation of the target object with respect to the acoustic transducer array.

[0023] In an alternative implementation, the embodied mechanism further includes an actuator coupled to the controller, which is further configured to perform the following tracking and locking operations: Continuously or periodically acquire the spatial orientation data of the target object; The movement state or trajectory of the target object is dynamically updated based on changes in the acquired spatial orientation data, and driving commands are generated based on the movement state or trajectory to drive the actuator to move, thereby adjusting the physical orientation of the acoustic transducer array; and The beamforming parameters are updated according to the relative orientation of the target object with respect to the adjusted acoustic transducer array, and the plurality of transducer units are driven to emit the ultrasonic modulation signal based on the updated beamforming parameters, so that the main lobe direction of the ultrasonic beam points to the target object, and / or the focusing area of ​​the ultrasonic beam or the coverage area of ​​the audible sound signal is located or maintained within a preset range based on the position of the target object.

[0024] In one alternative implementation, the actuator includes a multi-degree-of-freedom gimbal, a neck joint module, or a waist joint module. The acoustic transducer array is disposed on the head or torso of the animate, and the actuator is configured to drive the head or torso to perform horizontal rotation and / or pitching movements to adjust the physical orientation of the acoustic transducer array. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a directional audio playback method for an embodied mechanism according to an embodiment of this application; Figure 2 This is a schematic diagram illustrating the structure of an embodied mechanism according to an embodiment of this application; Figure 3 This is a schematic diagram comparing the sound pressure levels of a piezoelectric matrix and a planar piston according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an embodied mechanism according to an embodiment of this application; Figure 5 This is a layered schematic diagram of the architecture of a directional audio playback system with an embodied mechanism according to an embodiment of this application; Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0029] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0030] In this application, the term "embodied mechanism" can refer to an intelligent device or mechanical apparatus that has a physical body and is capable of interacting with the external environment or a target object. The embodied mechanism may include a body structure, sensors, controllers, actuators, and / or human-machine interaction components, and can acquire information about the target object or the external environment through sensors, and perform directional audio playback, posture adjustment, movement, turning, display, voice interaction, or other interactive operations based on the processing results of the controller.

[0031] In some embodiments, the embodied mechanism can be a robot, service robot, companion robot, educational robot, tour guide robot, welcoming robot, cleaning robot, delivery robot, nursing robot, bionic robot, humanoid robot, robot dog, mobile audio interactive device, or other intelligent device with a physical body and interactive capabilities. The embodied mechanism may also include at least one of a head, torso, base, mobile chassis, robotic arm, joints, multi-degree-of-freedom gimbal, or other actuators.

[0032] In related technologies, loudspeakers are commonly used as directional audio playback devices in embodied mechanisms. However, in noisy environments such as shopping malls and factories, loudspeaker solutions have significant shortcomings: Firstly, due to the limited space in the head of the embodied mechanism, it is difficult to arrange large-diameter loudspeakers or directional acoustic arrays composed of multiple traditional loudspeakers to obtain sufficient sound pressure levels, making it difficult for the interactive object to clearly receive voice information; secondly, even if loudspeakers are placed on the body to increase sound pressure, their sound emission direction is usually related to the loudspeaker's installation direction, making it difficult to actively change the sound projection direction without adjusting the overall or partial structure's posture, let alone actively follow the interactive object's positional changes; furthermore, loudspeakers have a large directivity angle, causing sound energy to radiate widely in the surrounding area, resulting in sound energy leakage and environmental noise interference. These shortcomings indicate that how to achieve highly directive directional audio playback that can dynamically follow the interactive object within a limited space has become a technical problem that urgently needs to be solved in this field.

[0033] To address the aforementioned technical problems, embodiments of this application provide a directional audio playback method, a embodied mechanism, a computer-readable storage medium, and a computer program product. This method acquires spatial orientation data of a target object and determines or updates the amplitude and phase data of each transducer unit in an acoustic transducer array based on the spatial orientation data to form an ultrasonic beam directed towards the target object. Simultaneously, the audio signal to be reproduced is modulated onto an ultrasonic carrier to generate an ultrasonic modulation signal, and multiple transducer units in the acoustic transducer array are driven to emit drive signals generated based on the ultrasonic modulation signal according to their corresponding amplitude and phase data. This allows the ultrasonic modulation signal to self-demodulate through the nonlinear effect of air as it propagates in the air, forming an audible sound signal corresponding to the audio signal to be reproduced. The technical solutions of this application embodiment are described in detail below with reference to the accompanying drawings.

[0034] According to an embodiment of this application, a method for directional audio playback of a embodied mechanism is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] This embodiment provides a method for directional audio playback of embodied mechanisms, which can be used in embodied mechanisms such as robots, robotic arms, and unmanned vehicles. Figure 1 This is a flowchart of a directional audio playback method for a embodied mechanism according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain the spatial orientation data of the target object.

[0036] Specifically, an embodied mechanism can refer to an intelligent device or mechanical apparatus that has a physical body, is capable of interacting with the external environment or a target object, and performs corresponding operations, such as a robot, robotic arm, unmanned vehicle, mobile interactive terminal, embodied intelligent agent, etc. Directional audio playback can refer to the process of generating and outputting an audible sound signal based on the audio signal to be reproduced; in this embodiment, directional audio playback may include modulating the audio signal to be reproduced onto an ultrasonic carrier to generate an ultrasonic modulation signal, and then demodulating the ultrasonic modulation signal through the nonlinear effect of air as it propagates in the air to form an audible sound signal corresponding to the audio signal to be reproduced. A target object can refer to an object that interacts with the embodied mechanism or needs to receive directional audio, such as a user, customer, worker, operator, equipment, or other object covered by directional audio. Spatial orientation data can refer to data used to characterize the spatial positional relationship of the target object relative to the embodied mechanism or acoustic transducer array, including azimuth angle, pitch angle, distance, three-dimensional coordinates, or combinations thereof.

[0037] In this step, the embodied mechanism can collect environmental data through at least one sensor and process the environmental data to obtain the spatial orientation data of the target object, providing a basis for the subsequent determination of beamforming parameters and dynamic tracking. Acquiring this spatial orientation data helps improve the accuracy of the ultrasonic beam's orientation towards the target object and enhances the stability of directional audio playback.

[0038] In one possible implementation, the embodied mechanism can acquire sound signals emitted by the target object through a microphone array positioned on the head, torso, or other locations, and determine the target object's first location information using a sound source localization algorithm. This sound source localization algorithm may include time difference of arrival (TDOA), angle of arrival (AHA) estimation, beam scanning, or a combination thereof. Simultaneously, the embodied mechanism can acquire image information of the target object through a camera, depth camera, or other visual sensors, and determine the target object's second location information using a target recognition algorithm. This target recognition algorithm may include face detection, human detection, human pose estimation, target tracking, or a combination thereof. Further, the first and second location information can be converted to the same coordinate system and fused based on timestamps, confidence levels, weighted fusion, Kalman filtering, or other filtering algorithms to obtain the spatial location data. When only one sensor is in effective working condition, or when the data output by only one sensor meets a preset confidence level, the spatial location data can also be determined based on the data output by that sensor. When the embodied mechanism includes an attitude sensor, the first orientation information, the second orientation information, or their fusion result can be used to perform coordinate transformation or attitude compensation based on the posture data of the embodied mechanism obtained by the attitude sensor, so as to obtain the relative orientation of the target object with respect to the acoustic transducer array.

[0039] Step S102: Based on the spatial orientation data, determine the beamforming parameters of the acoustic transducer array. The beamforming parameters include amplitude data and phase data corresponding to multiple transducer units in the acoustic transducer array. Specifically, an acoustic transducer array can refer to a sound source array formed by arranging multiple transducer units according to a preset geometry, such as a piezoelectric ultrasonic transducer array, a MEMS ultrasonic transducer array, a capacitive micromechanical ultrasonic transducer array, or a combination thereof. A transducer unit can refer to a basic acoustic emission unit constituting the acoustic transducer array. Beamforming parameters can refer to a set of parameters used to control the emission state of each transducer unit in the acoustic transducer array, so that the ultrasonic waves emitted by multiple transducer units are coherently superimposed in the target direction to form a directional ultrasonic beam. The beamforming parameters may include amplitude data and phase data corresponding to each transducer unit; wherein, the amplitude data can be used to control the amplitude or energy weight of the emitted signal of the corresponding transducer unit, and the phase data can be used to control the phase shift, phase delay, or equivalent time delay of the emitted signal of the corresponding transducer unit.

[0040] In this step, the amplitude and phase data corresponding to each transducer unit can be determined based on the spatial orientation data of the target object relative to the acoustic transducer array, so that the ultrasonic modulation signal emitted by the acoustic transducer array forms an ultrasonic beam directed towards the target object. In this way, the main lobe direction, focusing area, or acoustic energy coverage area of ​​the ultrasonic beam can be directed towards the area where the target object is located, thereby improving the acoustic energy concentration in the target direction and the directivity of directional audio playback.

[0041] In one possible implementation, taking a rectangular planar ultrasonic transducer array as an example, the array includes M×N transducer units, and the center-to-center distance between adjacent transducer units is d. The geometric center, acoustic center, phase center, or a preset reference transducer unit of the acoustic transducer array can be used as a reference point. Based on the spatial orientation data of the target object obtained in step S101, the path difference between the propagation path of each transducer unit to the target object and the reference path is calculated. The reference path can be the propagation path from the reference point to the target object. Further, the phase data of each transducer unit can be determined based on the path difference. For example, the phase compensation amount of the (i,j)th transducer unit can be determined according to φ(i,j)=K·2π·Δd(i,j) / λ, where Δd(i,j) represents the path difference corresponding to the (i,j)th transducer unit, λ represents the working wavelength or center carrier wavelength corresponding to the ultrasonic modulation signal, and K can be positive or negative depending on the adopted phase reference or compensation direction. When the target object is far from the acoustic transducer array and a far-field plane wave approximation can be used, the acoustic path difference can also be determined based on the target direction vector and the position vector of each transducer unit relative to a reference point. For example, when the target direction is characterized by azimuth and elevation angles, the acoustic path difference can be determined based on the projection of the position vector of each transducer unit onto the target direction. Accordingly, the acoustic path difference is converted into a phase compensation amount so that the ultrasonic waves emitted by multiple transducer units coherently superimpose in the target direction.

[0042] Furthermore, the amplitude data of each transducer unit can be determined based on the target object's location, desired main lobe width, side lobe suppression requirements, or target sound pressure coverage area. For example, uniform weighting, Hamming window, Hanning window, Chebyshev window, Taylor window, or other amplitude weighting methods can be used to determine the amplitude weight of each transducer unit, balancing main lobe width and side lobe suppression. Ultimately, the amplitude and phase data corresponding to each transducer unit together constitute the beamforming parameters of that transducer unit. If the target object's location changes, the beamforming parameters can be re-determined based on the updated spatial orientation data.

[0043] Step S103: Modulate the audio signal to be reproduced onto the ultrasonic carrier wave to generate an ultrasonic modulation signal.

[0044] Specifically, the audio signal to be reproduced can refer to the sound signal that needs to be reproduced to the target object, such as voice prompts, navigation instructions, interactive feedback tones, alarm tones, or personalized audio content from an embodied mechanism. The ultrasonic carrier can refer to a carrier signal with a frequency in the ultrasonic band, such as a sine wave, pulse wave, or other periodic carrier signal with a frequency greater than 20kHz. The ultrasonic modulated signal can refer to the signal formed after the audio signal to be reproduced is modulated onto the ultrasonic carrier. When the ultrasonic modulated signal propagates in the air, it can self-demodulate through the nonlinear effect of the air to form an audible sound signal corresponding to the audio signal to be reproduced.

[0045] In this step, by modulating the audio signal to be reproduced onto an ultrasonic carrier, the audio information can be carried in the ultrasonic frequency band, and the directional nature of the ultrasonic beam enables directional propagation towards the target object. Since the acoustic energy of the ultrasonic beam mainly propagates along the target direction, it helps to reduce acoustic energy leakage in non-target directions and provides a signal basis for subsequent self-demodulation in the air to form an audible sound signal.

[0046] In one possible implementation, the ultrasonic modulation signal can be generated using a double-sideband modulation method. Let the normalized audio signal to be reproduced be represented as x(t), and the ultrasonic carrier as cos(ωct), where ωc = 2πfc, and fc is the ultrasonic carrier frequency, for example, fc can be 40kHz. The modulation signal y(t) = [1 + m·x(t)]·cos(ωct) can be generated according to a preset modulation depth m, where the modulation depth m can be preset according to the ultrasonic carrier frequency, modulation method, and target distortion index to reduce overmodulation distortion or nonlinear distortion. In some embodiments, the highest frequency of the audio signal to be reproduced can be less than or equal to a preset proportion of the ultrasonic carrier frequency to reduce distortion or interference caused by unreasonable sideband distribution.

[0047] In one example, when the ultrasonic carrier frequency is 40 kHz, the bandwidth of the audio signal to be reproduced can be limited to the range of 100 Hz to 6 kHz, and / or the modulation depth can be controlled within a preset threshold. The above bandwidth range and modulation depth threshold are merely examples; actual values ​​can be determined based on the ultrasonic carrier frequency, the modulation method used, the operating bandwidth of the acoustic transducer array, and the target distortion index. Besides double-sideband modulation, single-sideband modulation, amplitude modulation, or other modulation methods suitable for ultrasonic parametric sound generation can also be used, and the ultrasonic modulated signal can be generated by combining square root predistortion processing or other predistortion processing.

[0048] Step S104: Based on the beamforming parameters, drive the plurality of transducer units to emit the ultrasonic modulation signal to form an ultrasonic beam toward the target object. The ultrasonic beam is configured to self-demodulate an audible sound signal corresponding to the audio signal to be reproduced during propagation toward the target object.

[0049] Specifically, the nonlinear effect of air refers to the nonlinear interaction between the ultrasonic carrier and its sideband components as the ultrasonic modulated signal propagates through the air medium. This interaction, caused by the nonlinear acoustic properties of the air medium, generates a low-frequency acoustic component corresponding to the audio signal to be reproduced. This low-frequency acoustic component falls within the audible frequency range, such as 20Hz to 20kHz, thus forming an audible sound signal corresponding to the audio signal to be reproduced. In some cases, the low-frequency acoustic component may include an acoustic component generated by the difference frequency interaction between the ultrasonic carrier component and the sideband component. For example, when the frequency of the ultrasonic carrier component is fc and the frequency of the sideband component is fc+Δf, the nonlinear interaction of air can generate a low-frequency acoustic component with a frequency of Δf.

[0050] In this step, multiple transducers, under the control of the beamforming parameters, emit ultrasonic waves corresponding to the driving signals generated based on the ultrasonic modulation signals, causing the ultrasonic modulation signals to form an ultrasonic beam directed towards the target object. Because the ultrasonic beam has strong directivity, the acoustic energy of the ultrasonic modulation signal mainly propagates along the main lobe direction of the ultrasonic beam, and during propagation in the air, it self-demodulates to form an audible sound signal corresponding to the audio signal to be reproduced. Therefore, the area where the target object is located can achieve relatively high audible sound coverage, and sound energy leakage in non-target areas can be reduced, thereby achieving directional audio playback with high directivity.

[0051] In one possible implementation, a 40kHz ultrasonic carrier wave is used, and the audio signal to be reproduced is modulated onto the ultrasonic carrier wave to generate an ultrasonic modulated signal. The audio signal to be reproduced may include audio components below 6kHz. The ultrasonic modulated signal can be transmitted into the air by a piezoelectric ultrasonic transducer array, a MEMS ultrasonic transducer array, or other acoustic transducer array according to the amplitude and phase data determined in step S102. During propagation, due to the nonlinear effect of air, a difference frequency component can be generated between the ultrasonic carrier component and its sideband components. For example, the difference frequency component generated between the 40kHz carrier component and the 40kHz+Δf sideband component is Δf; when Δf falls into the audible frequency band, an audible sound signal corresponding to the audio signal to be reproduced can be formed. By controlling the beamforming parameters, the main lobe direction, focusing area, or audible coverage area of ​​the ultrasonic beam can be directed towards the area where the target object is located, thereby improving the clarity and directionality of the audible sound signal received by the target object.

[0052] In some implementations, the driving amplitude, modulation depth, or output power of multiple transducer units can be controlled according to the location of the target object, the working state of the acoustic transducer array, the ambient acoustic conditions, or a preset sound pressure threshold, so as to ensure that the audible sound signal in the area where the target object is located meets the preset volume or sound pressure requirements, and to reduce the discomfort or safety risks caused by excessively high sound pressure output.

[0053] The directional audio playback method for the embodied mechanism provided in this embodiment acquires the spatial orientation data of the target object and determines the amplitude and phase data of each transducer unit in the acoustic transducer array based on this data. This enables electronic beamforming control of the acoustic transducer array, allowing the ultrasonic modulation signals emitted by multiple transducer units to form an ultrasonic beam directed towards the target object. Simultaneously, by modulating the audio signal to be reproduced onto an ultrasonic carrier, the ultrasonic modulation signal can self-demodulate through the nonlinear effect of air as it propagates in the air, forming an audible sound signal corresponding to the audio signal to be reproduced. Therefore, a highly directional directional audio playback sound field can be formed within the limited installation space of the embodied mechanism, reducing reliance on multiple traditional loudspeakers or multiple independent acoustic arrays, thus reducing system size, structural load, and integration complexity. Furthermore, by dynamically updating the beamforming parameters based on the spatial orientation data of the target object, the main lobe direction, focusing area, or audible coverage area of ​​the ultrasonic beam can be adjusted according to the position of the target object, thereby improving the problem that traditional directional sound fields cannot promptly follow the movement of interactive objects. Furthermore, since the generated audible sound signal primarily covers the area where the target object is located, it helps reduce sound energy leakage in non-target areas, minimizes ambient sound interference, and enhances the privacy of audio interaction. Therefore, the method provided in this embodiment can achieve highly directional, dynamically tracking, and low-leakage directional audio playback within a limited space.

[0054] In some optional implementations, step S101 above includes the following steps: Step S201: Acquire the sound signal of the target object through an acoustic sensor, and determine the first orientation information of the target object based on the sound signal.

[0055] Specifically, acoustic sensors are devices used to collect ambient sound signals, such as microphone arrays (composed of multiple microphones arranged in a specific geometric layout). Sound source localization refers to the process of using sound signals received by acoustic sensors to calculate the direction and / or distance of a sound source relative to the sensor array through algorithms, such as using Time Difference of Arrival (TDOA) or beamforming algorithms. First-order information refers to the direction estimate of a target object obtained solely through acoustic means, such as the horizontal and pitch angles of the target relative to the robot.

[0056] This step involves acquiring the voice or other sound signals emitted by the target object through a microphone array, and using a multi-channel signal processing algorithm to determine the direction of the target. This allows the system to obtain the location information of the interactive object without relying on visual conditions (such as insufficient light or occlusion), thereby improving the robustness of the system.

[0057] In one possible implementation, four microphones are symmetrically arranged on both sides of the robot's head, forming a linear or circular array. When the target object emits a sound, there is a time difference in the arrival time of the signal received by each microphone. By calculating the arrival time difference between every two microphones, and combining this with the known microphone spacing and sound speed, the orientation angle of the target relative to the center of the robot's head can be calculated. To improve accuracy, a generalized cross-correlation algorithm can be used to estimate the time delay, and the results from multiple microphone pairs can be fused and optimized to finally output the first orientation information.

[0058] Step S202: Acquire image information of the target object through a visual sensor, and perform target recognition based on the image information to determine the second location information.

[0059] Specifically, a vision sensor refers to a device used to acquire environmental image information, such as a red-green-blue (RGB) camera, a red-green-blue depth (RGB-D) camera, and a time-of-flight (ToF) sensor. Image information refers to the two-dimensional or three-dimensional image data output by the vision sensor. Target recognition refers to the process of detecting and identifying specific targets from images using image processing or deep learning algorithms, such as human detection, face detection, human pose estimation, or target tracking. Secondary location information refers to the spatial position information of the target object relative to the robot obtained through visual means, including orientation angle, distance, and three-dimensional coordinates.

[0060] This step involves acquiring visual information of the target object using a camera, employing computer vision algorithms to detect and locate the target's position in the image, and then converting this information into spatial orientation data. Visual positioning provides high angular resolution and distance estimation, is particularly suitable for stationary or slowly moving targets, and complements sound source localization.

[0061] In one possible implementation, a vision system is composed of an RGB camera and a depth camera or a ToF sensor. The RGB camera detects faces or bodies and outputs the bounding box of the target in the image. The controller can calculate the offset of its center point relative to the image center and convert it into a horizontal offset angle by combining the camera's horizontal field of view. The depth camera outputs the distance value from the target to the camera. Simultaneously, using multi-view geometry or PnP algorithms, the three-dimensional coordinates of the target in the robot coordinate system are calculated from the two-dimensional image coordinates and the depth value, thereby determining the second orientation information.

[0062] Step S203: Determine the spatial orientation data based on the first orientation information, the second orientation information, or the fusion information of the first orientation information and the second orientation information.

[0063] Specifically, spatial orientation data refers to the comprehensive spatial position information of a target object relative to its embodied mechanism, including orientation angles, distances, and three-dimensional coordinates. Compared with the first or second orientation information alone, the fused data usually has higher accuracy and robustness.

[0064] This step fuses the sound source localization results and visual recognition results, utilizing the complementary characteristics of the two sensors to output a target orientation estimate through methods such as weighted averaging, Kalman filtering, or Bayesian estimation. The fused spatial orientation data provides a more reliable input for subsequent beamforming and dynamic tracking, maintaining localization capability even when a single sensor fails or confidence levels decrease.

[0065] In one possible implementation, first azimuth information θ1 and α1 are obtained from step S201, and second azimuth information θ2 and α2, along with distance d, are obtained from step S202. First, the timestamps of both are aligned, and then adaptive weighted fusion is employed: weights are dynamically allocated based on the environmental noise level. For example, the weight of sound source localization is reduced when the environmental signal-to-noise ratio is low, and the weight of visual localization is reduced when the light is dim or the target is obscured. The final azimuth angle can be expressed as θ = (w1·θ1 + w2·θ2) / (w1 + w2), and the pitch angle can be expressed as α = (w1·α1 + w2·α2) / (w1 + w2), where the weights w1 and w2 can be dynamically adjusted based on confidence levels. Distance information can preferentially use depth values ​​provided by visual sensors; if no effective visual distance is available, coarse distance estimated acoustically, historical distance, or predicted distance can be used. The fused spatial azimuth data can be represented as (θ, α, d) or the corresponding three-dimensional coordinates.

[0066] This implementation method integrates acoustic positioning and visual recognition to achieve robust acquisition of spatial orientation data for a target object. When the ambient noise is high or the target is silent, visual information can compensate for the shortcomings of acoustic positioning; when the light is dim or the target is obscured, acoustic information can maintain the orientation estimation. The two sensors mutually verify and complement each other, significantly improving the reliability and accuracy of target positioning, and providing higher quality spatial orientation input for subsequent beamforming and dynamic tracking.

[0067] In some alternative implementations, forming an ultrasonic beam toward the target object includes the following steps: Step S301: Control the main lobe direction of the ultrasonic beam emitted by the plurality of transducer units to point towards the target object, and / or make the focusing area of ​​the ultrasonic beam or the coverage area of ​​the audible sound signal located or maintained within a preset range based on the position of the target object.

[0068] Specifically, an ultrasonic beam can refer to a directional acoustic energy coverage area formed by the coherent superposition of ultrasonic modulated signals emitted by multiple transducer units in an acoustic transducer array in space. The main lobe direction of the ultrasonic beam can refer to the main propagation direction in which the acoustic energy is relatively concentrated. The focusing region can refer to the area where the ultrasonic beam forms a high sound pressure or high sound energy density near the target object. The coverage area of ​​the audible sound signal can refer to the spatial region where the audible sound signal formed after the ultrasonic modulated signal is demodulated in the air can be received by the target object.

[0069] In this step, the controller can determine the beamforming parameters of the acoustic transducer array based on the spatial orientation data of the target object, and control multiple transducer units to transmit ultrasonic modulated signals according to the corresponding amplitude and phase data based on the beamforming parameters. By adjusting the phase relationship, amplitude relationship, or equivalent time delay relationship between each transducer unit, the main lobe direction of the ultrasonic beam can be made to point towards the target object, and / or the focusing area of ​​the ultrasonic beam or the coverage area of ​​the audible sound signal can be located within or maintained within a preset range based on the position of the target object.

[0070] The preset range can be determined based on the target object's head region, ear position, body size, positioning error range, system response error range, target sound pressure coverage range, or a combination thereof. For example, the preset range can be a spatial region centered on the target object's head or ear region, or it can be a sound field coverage area covering the target object's current position and its surrounding tolerance range. This allows audible sound signals to primarily cover the area where the target object is located, thereby improving the directivity of directional audio playback and reducing sound energy leakage from non-target areas.

[0071] In one possible implementation, when the target object is directly in front of the embodied mechanism, the controller can determine phase data and amplitude data that orient the main lobe of the ultrasonic beam toward the front; when the target object moves to the side front, the controller can re-determine the phase data and amplitude data so that the main lobe direction, focusing area, or audible coverage area of ​​the ultrasonic beam changes with the position of the target object.

[0072] In some optional implementations, the method further includes the following steps: Step S401: Continuously or periodically acquire the spatial orientation data of the target object.

[0073] Step S402: In response to the change of the spatial orientation data over time, determine the pointing control information for maintaining the ultrasonic beam toward the target object.

[0074] Step S403: Based on the pointing control information, adjust the physical orientation of the acoustic transducer array and update the beamforming parameters so that the ultrasonic beam remains oriented toward the target object as the orientation of the target object changes.

[0075] Specifically, the spatial orientation data of the target object may include the target object's azimuth angle, pitch angle, distance, three-dimensional coordinates, or a combination thereof relative to the embodied mechanism or acoustic transducer array. The change of the spatial orientation data over time may refer to changes in the aforementioned azimuth angle, pitch angle, distance, or three-dimensional coordinates at different times. The pointing control information may refer to control information used to maintain the ultrasonic beam's orientation towards the target object, such as beamforming parameter update information, array physical orientation adjustment information, actuator drive information, target object relative orientation information, or a combination thereof.

[0076] In step S401, the controller can continuously or periodically acquire the spatial orientation data of the target object according to a preset sampling period or a preset refresh frequency.

[0077] In step S402, the controller can determine whether the spatial orientation data changes over time. When the position, orientation angle, pitch angle, or distance of the target object changes, the controller can determine pointing control information in response to the change.

[0078] In step S403, the pointing control information can be used to update the beamforming parameters of the acoustic transducer array, or to drive the actuator of the embodied mechanism to adjust the physical orientation of the acoustic transducer array, or to simultaneously update the beamforming parameters and adjust the physical orientation.

[0079] In one possible implementation, the controller can acquire spatial orientation data of the target object at time points t1, t2, ..., tn. For example, the spatial orientation data may include the azimuth angle θi, the pitch angle αi, and the distance di. The controller can calculate the changes between adjacent time points, such as the change in azimuth angle Δθ = θi. θi 1. Change in pitch angle Δα = αi αi 1 and the change in distance Δd=di di 1. When the change exceeds a preset threshold, the controller can determine that the target object has changed its orientation and generate pointing control information to keep the ultrasonic beam pointing towards the target object.

[0080] In some implementations, if the target object's orientation change is within the adjustable range of the electronic beamforming, the controller can update the beamforming parameters based on the pointing control information, for example, by re-determining the phase and amplitude data corresponding to each transducer unit, so that the main lobe direction of the ultrasonic beam is adjusted to the new orientation of the target object. If the target object's orientation change is large, or exceeds the preset electronic deflection range of the acoustic transducer array, the controller can adjust the physical orientation of the acoustic transducer array based on the pointing control information, for example, by driving the head, torso, gimbal, or other mounting components to rotate. If improved tracking stability is required, the controller can also simultaneously adjust the physical orientation and update the beamforming parameters.

[0081] This implementation method enables the ultrasonic beam to remain oriented towards the target object as the target object's orientation changes, thereby improving the continuity, coverage stability, and directionality of directional audio playback during the target object's movement.

[0082] In some alternative implementations, determining the pointing control information for maintaining the ultrasonic beam toward the target object includes the following steps: Step S501: The target object is identified as the locked object.

[0083] Step S502: Based on the continuously or periodically acquired spatial orientation data, determine the movement status or movement trajectory of the locked object.

[0084] Step S503: Generate a driving command based on the movement state or the movement trajectory to drive the actuator of the embodied mechanism to move, thereby adjusting the physical orientation of the acoustic transducer array.

[0085] Specifically, the locked object can refer to the object identified by the embodied mechanism as the current target for directional audio playback. The movement state can refer to the locked object's current position, direction of movement, speed, angular velocity, distance change trend, or a combination thereof. The movement trajectory can refer to the spatial position change information of the locked object over a period of time, such as a position sequence including timestamps, azimuth angles, pitch angles, distance, three-dimensional coordinates, or combinations thereof. Drive commands can include position commands, speed commands, acceleration commands, pulse width modulation signals, bus control commands, or other signals used to control the movement of the actuator.

[0086] In step S501, the controller can determine the target object as the locked object based on the target recognition result, interaction status, voice wake-up result, user selection result, or priority rules.

[0087] In step S502, the controller determines the movement state or trajectory of the locked object based on continuously or periodically acquired spatial orientation data. For example, the controller can maintain a buffer for storing historical spatial orientation data of the locked object, and after acquiring new spatial orientation data, filter, fit, or predict the position sequence in the buffer to obtain the movement direction, movement speed, angular velocity, or predicted position of the locked object.

[0088] In step S503, the controller generates drive commands for the actuators based on the movement state or movement trajectory to drive the multi-degree-of-freedom gimbal, neck joint module, waist joint module or other actuators to move, thereby adjusting the physical orientation of the acoustic transducer array.

[0089] For example, the embodied mechanism can be a robot, with an acoustic transducer array positioned at the robot's head. When the target object moves from directly in front of the robot to its right front, the controller can generate a target rotation angle or target angular velocity command for the neck joint module based on the target object's movement state or trajectory, causing the robot's head to rotate towards the target object. As the physical orientation of the acoustic transducer array is adjusted, the controller can also update the beamforming parameters based on the target object's relative position to the adjusted acoustic transducer array, ensuring the ultrasonic beam continues to be directed towards the target object.

[0090] This implementation method enables dynamic tracking of a locked object through a combination of mechanical orientation adjustment and electronic beamforming adjustment. Mechanical orientation adjustment expands the effective coverage of the acoustic transducer array, while electronic beamforming adjustment allows for precise pointing control based on the adjusted physical orientation, thereby improving the stability and continuity of directional audio playback in mobile interactive scenarios.

[0091] In some optional embodiments, the step S103 above, which modulates the audio signal to be reproduced onto an ultrasonic carrier, includes the following steps: Step S601: Using at least one modulation method selected from double-sideband modulation, single-sideband modulation, amplitude modulation, or square root predistortion modulation, the audio signal to be reproduced is loaded onto the ultrasonic carrier with a frequency greater than 20kHz.

[0092] Specifically, double-sideband modulation refers to modulating the audio signal to be reproduced onto an ultrasonic carrier, so that the spectral components corresponding to the audio signal to be reproduced are distributed on both sides of the ultrasonic carrier frequency to form an upper sideband and a lower sideband. Single-sideband modulation refers to retaining one sideband of the upper or lower sideband after modulation and suppressing the other sideband and / or carrier component to reduce the bandwidth occupied by the modulated signal or improve energy utilization efficiency. Amplitude modulation refers to adjusting the envelope of the ultrasonic carrier according to the amplitude variation of the audio signal to be reproduced, so that the ultrasonic carrier carries the information of the audio signal to be reproduced. Square root predistortion modulation refers to performing square root predistortion processing on the audio signal to be reproduced in the modulation link and loading the processed signal onto the ultrasonic carrier to compensate for the distortion in the air nonlinear self-demodulation process.

[0093] In this step, by loading the audio signal to be reproduced onto an ultrasonic carrier with a frequency greater than 20kHz, the information of the audio signal to be reproduced can be carried in the ultrasonic frequency band, and directional propagation can be achieved by utilizing the stronger directional propagation characteristics of ultrasound compared to conventional audible sound. Different modulation methods can be applied to different application requirements. For example, double-sideband modulation is relatively simple to implement; single-sideband modulation is beneficial for reducing bandwidth occupation; amplitude modulation facilitates carrier envelope control; and square root predistortion modulation is beneficial for reducing nonlinear distortion after air self-demodulation.

[0094] In one possible implementation, a 40kHz sinusoidal signal can be used as the ultrasonic carrier. When using double-sideband modulation or amplitude modulation, the normalized audio signal to be reproduced can be modulated with the ultrasonic carrier to generate an ultrasonic modulated signal containing both the ultrasonic carrier component and the sideband components. When using single-sideband modulation, the single-sideband ultrasonic modulated signal can be generated through Hilbert transform, quadrature modulation, filtering, or digital signal processing. When using square root predistortion modulation, the audio signal to be reproduced can be normalized, biased, preprocessed with the square root, or subjected to other predistortion processing before or during modulation to compensate for distortion during the nonlinear self-demodulation process of air.

[0095] In some optional implementations, modulating the audio signal to be reproduced onto an ultrasonic carrier wave further includes the following steps: Step S701: Make the highest frequency of the audio signal to be reproduced less than or equal to a preset ratio of the frequency of the ultrasonic carrier.

[0096] Step S702: When amplitude modulation is used, the modulation depth used when the audio signal to be reproduced is modulated onto the ultrasonic carrier is less than or equal to a preset threshold.

[0097] The preset ratio and the preset threshold are respectively preset according to the frequency of the ultrasonic carrier, the modulation method and the target distortion index.

[0098] Specifically, the highest frequency of the audio signal to be reproduced can refer to the highest effective audio frequency in the audio signal to be reproduced that needs to be modulated onto the ultrasonic carrier. Modulation depth can refer to the degree to which the audio signal to be reproduced modulates the amplitude or envelope of the ultrasonic carrier when amplitude modulation is used. Target distortion metrics can include total harmonic distortion, intermodulation distortion, envelope distortion, audible intelligibility metrics, or other metrics used to evaluate audio quality.

[0099] In step S701, the controller can set or control the highest frequency of the audio signal to be reproduced based on the frequency of the ultrasonic carrier, the modulation method used, and the target distortion index. For example, when using double-sideband modulation, the preset ratio can be determined based on the carrier frequency, the upper and lower sideband spectral distribution, and the effective operating bandwidth of the acoustic transducer array; when using single-sideband modulation, the preset ratio can be determined based on the frequency range of the retained sidebands and the effective operating bandwidth of the transducer; when using square root predistortion modulation, the preset ratio can be determined by combining the spectral distribution after predistortion processing.

[0100] In step S702, when amplitude modulation is used, the controller can determine a preset threshold based on the frequency of the ultrasonic carrier, the driving capability of the transducer unit, the target sound pressure requirement, and the target distortion index, and make the modulation depth less than or equal to the preset threshold. By limiting the modulation depth, overmodulation distortion, envelope distortion, or nonlinear distortion caused by excessive modulation depth can be reduced, and it is beneficial to maintain a relatively stable amplitude variation range for the ultrasonic modulation signal output by the acoustic transducer array.

[0101] In one possible implementation, the controller can first perform spectral analysis on the audio signal to be reproduced to determine its highest effective audio frequency. When the highest effective audio frequency is higher than the upper frequency limit corresponding to a preset proportion of the ultrasonic carrier frequency, the audio signal to be reproduced can be subjected to low-pass filtering, band-limiting, or resampling. Furthermore, in the case of amplitude modulation, the controller can perform amplitude normalization, gain control, or dynamic range compression on the audio signal to be reproduced, and determine the modulation depth according to a preset threshold, so that the modulated ultrasonic signal meets the target distortion index.

[0102] This embodiment constrains the modulation depth in the highest frequency and amplitude modulation scenario of the audio signal to be reproduced based on the frequency of the ultrasonic carrier, the modulation method, and the target distortion index. This enables the modulated ultrasonic signal to self-demodulate and form an audible sound signal corresponding to the audio signal to be reproduced when it propagates in the air, and reduces the distortion caused by unreasonable sideband distribution, excessive modulation depth, or nonlinear air action.

[0103] In some alternative implementations, driving the plurality of transducer units to transmit the ultrasonic modulated signal based on the beamforming parameters includes the following steps: Step S801: Based on the beamforming parameters and the ultrasonic modulation signal, generate multi-channel drive data corresponding one-to-one with the plurality of transducer units.

[0104] Step S802: Time synchronization and / or phase synchronization output of the multi-channel drive data are performed to drive the multiple transducer units to transmit the ultrasonic modulation signal according to the corresponding amplitude data and phase data.

[0105] Specifically, multi-channel drive data can refer to a set of drive data used to drive multiple transducer units in an acoustic transducer array. The multi-channel drive data may include drive waveforms, amplitude control data, phase control data, delay control data, pulse width modulation data, digital-to-analog conversion data, or combinations thereof corresponding to each transducer unit.

[0106] In step S801, the controller can combine beamforming parameters with the ultrasonic modulation signal to generate multi-channel drive data corresponding one-to-one with multiple transducer units. Specifically, the controller can scale the amplitude or adjust the gain of the ultrasonic modulation signal according to the amplitude data corresponding to each transducer unit; and perform phase shifting, delay processing, or phase compensation on the ultrasonic modulation signal according to the phase data corresponding to each transducer unit. Thus, the ultrasonic modulation signals output by different transducer units can be coherently superimposed in the direction of the target object or within a preset spatial range according to the desired phase and amplitude relationships.

[0107] In one possible implementation, the controller can generate independent channel data for each transducer unit. For example, for the k-th transducer unit, the controller can perform channelization processing on the same ultrasonic modulation signal based on the amplitude and phase data corresponding to that transducer unit to obtain the k-th channel drive data. Different channel drive data can have the same audio modulation content but have different amplitude, phase, or delay characteristics, thus collectively forming multi-channel drive data for beamforming.

[0108] In step S802, the controller can perform time synchronization and / or phase synchronization output on the multi-channel drive data. The time synchronization and / or phase synchronization output can refer to the output of drive data from multiple channels to the corresponding transducer unit or multi-channel drive circuit according to a unified clock, synchronization trigger signal, frame synchronization signal, or preset output timing sequence, so as to reduce the time deviation or phase deviation between channels.

[0109] In one possible implementation, the controller can send the multi-channel drive data to a multi-channel drive circuit, which then outputs corresponding electrical drive signals based on the drive data for each channel, thereby driving multiple transducer units to emit ultrasonic modulated signals. Each transducer unit, under the influence of the corresponding channel's electrical drive signal, emits an ultrasonic modulated signal according to the corresponding amplitude and phase data.

[0110] In some implementations, the controller can also perform channel calibration on the multi-channel drive data. This channel calibration may include compensating for gain errors, phase errors, delay errors, or transducer response differences between different channels. For example, the controller can perform amplitude or phase correction on the drive data of each channel based on pre-calibrated channel calibration parameters to improve output consistency among multiple transducers.

[0111] This embodiment generates multi-channel drive data corresponding to multiple transducer units based on beamforming parameters and ultrasonic modulation signals. The multi-channel drive data is then time-synchronized and / or phase-synchronized and output, enabling multiple transducer units to emit ultrasonic modulation signals according to corresponding amplitude and phase data. This helps maintain the emission timing and phase relationship between multiple transducer units, supports the acoustic transducer array in forming ultrasonic beams directed towards the target object, and improves the directivity and stability of directional audio playback.

[0112] This embodiment provides a embodied mechanism. Figure 2 This is a schematic diagram illustrating the structure of a physical mechanism according to an embodiment of this application, such as... Figure 2 As shown, the embodied mechanism includes: Acoustic transducer array, comprising multiple transducer units; At least one sensor; and A controller coupled to an acoustic transducer array and at least one sensor is configured to: Based on the spatial orientation data, the beamforming parameters of the acoustic transducer array are determined, and the beamforming parameters include amplitude data and phase data corresponding to the plurality of transducer units; The audio signal to be reproduced is modulated onto an ultrasonic carrier wave to generate an ultrasonic modulation signal; and Based on the beamforming parameters, the plurality of transducer units are driven to emit the ultrasonic modulation signal to form an ultrasonic beam toward the target object, and an audible sound signal corresponding to the audio signal to be reproduced is formed within a preset range based on the position of the target object.

[0113] Specifically, an acoustic transducer array can refer to a sound source array formed by arranging multiple transducer units according to a preset geometry, used to emit ultrasonic modulated signals. The preset geometry can include rectangular arrays, honeycomb arrays, concentric ring arrays, arc arrays, conformal curved surface arrays, or other array structures suitable for beamforming. The transducer units can include piezoelectric ultrasonic transducers, MEMS ultrasonic transducers, capacitive micromechanical ultrasonic transducers, or other transducer devices capable of emitting ultrasonic waves. Each transducer unit can be driven according to corresponding amplitude and phase data to participate in forming an ultrasonic beam directed towards the target object.

[0114] The controller can refer to a processing unit or system used to perform target orientation processing, signal modulation, beamforming parameter calculation, and drive control. The controller may include a microprocessor, digital signal processor, field-programmable gate array, application-specific integrated circuit, system-on-a-chip, microcontroller, drive control circuit, or a combination thereof. The target detection sensor can refer to a sensor used to sense the spatial orientation information of a target object, such as an acoustic sensor, microphone array, vision camera, depth sensor, infrared sensor, millimeter-wave radar, lidar, or a combination thereof. The spatial orientation data can refer to the positional relationship data of the target object relative to the embody mechanism or acoustic transducer array, including azimuth angle, pitch angle, distance, three-dimensional coordinates, or a combination thereof.

[0115] The audio signal to be reproduced can refer to the sound content that needs to be reproduced to the target object, such as voice prompts, navigation instructions, interactive feedback tones, alarm tones, or personalized audio content from a embodied mechanism. The ultrasonic carrier can refer to a carrier signal with a frequency in the ultrasonic band, such as a sine wave, pulse wave, or other periodic carrier signal with a frequency greater than 20kHz. The ultrasonic modulation signal can refer to the signal formed after the audio signal to be reproduced is loaded onto the ultrasonic carrier using a modulation method. The modulation method can include double-sideband modulation, single-sideband modulation, amplitude modulation, square root predistortion modulation, or other modulation methods suitable for ultrasonic parametric sound generation. The beamforming parameters can refer to parameters used to control the transmission state of each transducer unit in the acoustic transducer array, including amplitude data and phase data corresponding to each transducer unit. The amplitude data can be used to control the transmission amplitude or energy weight of the corresponding transducer unit, and the phase data can be used to control the phase shift, phase delay, or equivalent time delay of the corresponding transducer unit.

[0116] In the embodied mechanism provided in this embodiment, the controller can acquire the spatial orientation data of the target object through a target detection sensor, and determine the amplitude and phase data corresponding to each transducer unit based on the spatial orientation data. Simultaneously, the controller can modulate the audio signal to be reproduced onto an ultrasonic carrier wave to generate an ultrasonic modulation signal. Subsequently, the controller drives multiple transducer units to emit the ultrasonic modulation signal based on the beamforming parameters, causing the ultrasonic modulation signals emitted by the multiple transducer units to form an ultrasonic beam directed towards the target object in space. When the ultrasonic modulation signal propagates in the air, it can self-demodulate through the nonlinear effect of air to form an audible sound signal corresponding to the audio signal to be reproduced.

[0117] In this embodiment, the acoustic transducer array, at least one sensor, and controller work together to form a highly directional directional audio playback sound field within the limited installation space of the embodied mechanism. Furthermore, by determining or updating beamforming parameters based on the spatial orientation data of the target object, the main lobe direction, focusing area, or audible coverage area of ​​the ultrasonic beam can be directed towards the target object's location, thereby improving the directional and coherent nature of the directional audio playback during the interaction between the embodied mechanism and the target object. Since the formed audible signal primarily covers the target object's location, this embodied mechanism helps reduce sound energy leakage in non-target areas, minimizes sound interference to the surrounding environment, and improves the privacy of audio interaction. Simultaneously, compared to solutions relying on multiple traditional loudspeakers to form a directional sound field, the embodied mechanism in this embodiment helps reduce the installation space requirements and structural load of the acoustic components, improving the integration of the embodied mechanism.

[0118] Figure 3 This is a schematic diagram comparing the sound pressure levels of a piezoelectric matrix and a planar piston according to an embodiment of this application. Figure 3 As shown, the horizontal axis represents the half-space angle, with the unit being rad. Angle 0 corresponds to the front, and half-space represents 180° space. The vertical axis represents the sound pressure level, with the unit being dB. Figure 3 The diagram shows three directivity curves: one for an acoustic transducer array using piezoelectric transducers under 20kHz excitation, another for a planar piston (i.e., a moving-coil loudspeaker) under 20kHz excitation, and a third for a planar piston under 200Hz excitation. The excitation signal voltage for all three curves is 2Vp. From... Figure 3It can be seen that, whether in the low-frequency range of 200Hz or the high-frequency range of 20kHz, the sound pressure level of the planar piston is lower than that of the piezoelectric transducer. In the low-frequency range of 200Hz, the directivity of the planar piston is weaker than that of the piezoelectric transducer, meaning the sound pressure level difference of the planar piston at different spatial angles is smaller. In the high-frequency range of 20kHz, the sound pressure level of the piezoelectric transducer near the front is significantly higher than that of the planar piston. Therefore, it can be concluded that using piezoelectric transducers to construct an acoustic transducer array, compared to a dynamic loudspeaker solution, is advantageous for achieving higher directivity in the low-frequency range and higher sound pressure levels in the high-frequency range.

[0119] In some optional embodiments, the arrangement structure of the plurality of transducer units in the acoustic transducer array includes at least one of a rectangular array, a honeycomb array, a concentric ring array, an arc surface array, and a curved conformal array.

[0120] Specifically, a rectangular array can refer to an array structure formed by arranging multiple transducer units in a row and column direction. A honeycomb array can refer to an array structure formed by arranging multiple transducer units in a hexagonal tessellation pattern. A concentric ring array can refer to an array structure formed by arranging multiple transducer units around the array center in one or more concentric rings. An arc surface array can refer to an array structure formed by arranging multiple transducer units along an arc-shaped surface with a certain curvature. A curved conformal array can refer to an array structure formed by arranging multiple transducer units to conform to the non-planar shape or shell surface of a robot mechanism, such as conforming to the convex curved surface of a robot's head or torso.

[0121] In this embodiment, different arrangement structures can adapt to different installation spaces, appearance structures, and beam control requirements. For example, rectangular arrays facilitate coordinate modeling and beamforming parameter calculation; honeycomb arrays are beneficial for increasing the transducer unit filling density per unit area; concentric ring arrays are beneficial for achieving axisymmetric beam control; and curved or conformal curved surface arrays are beneficial for fitting the head, torso, or other curved structures of the embodied mechanism and expanding the controllable beam angle range. Thus, a balance can be achieved between array directivity, space utilization, mechanical adaptability, and manufacturing complexity.

[0122] In some alternative implementations, the acoustic transducer array has a highest ultrasonic operating frequency, and the center-to-center spacing between two adjacent transducer units is configured to be no greater than half the wavelength of the sound wave corresponding to the highest ultrasonic operating frequency.

[0123] Specifically, the highest ultrasonic operating frequency can refer to the highest effective ultrasonic operating frequency used by the acoustic transducer array when transmitting ultrasonic modulated signals. Since the wavelength of sound waves is related to the operating frequency, a higher operating frequency corresponds to a shorter wavelength. Therefore, the upper limit of the center-to-center distance between adjacent transducer units can be determined based on the highest ultrasonic operating frequency. The center-to-center distance can refer to the distance between the acoustic center, geometric center, or effective radiation center of two adjacent transducer units.

[0124] In this embodiment, the center-to-center spacing between two adjacent transducer units is configured to be no greater than half the wavelength of the sound wave corresponding to the highest ultrasonic operating frequency. This is beneficial for improving the spatial sampling density of the acoustic transducer array at the highest ultrasonic operating frequency and suppressing grating lobes that may be generated during beamforming. As a result, the ultrasonic modulation signals emitted by multiple transducer units can be coherently superimposed in the target direction according to the corresponding beamforming parameters, thereby forming an ultrasonic beam with a more defined main lobe direction and reducing acoustic energy leakage in non-target directions.

[0125] In one possible implementation, when the highest ultrasonic operating frequency is 40 kHz, the center-to-center spacing between adjacent transducer units can be determined based on the wavelength of the sound wave corresponding to that frequency. For example, in room temperature air, the wavelength of a 40 kHz sound wave is approximately 8.5 mm, so the center-to-center spacing between two adjacent transducer units can be configured to be less than or equal to approximately 4.25 mm. The above values ​​are merely examples; the actual center-to-center spacing can be determined based on the air velocity, the highest ultrasonic operating frequency, the array arrangement, the transducer unit size, the target beamwidth, and the sidelobe suppression requirements.

[0126] In some alternative embodiments, the embodied mechanism is a robot, and the acoustic transducer array is disposed on the head or torso of the robot and is symmetrically distributed about the central axis of the head or torso.

[0127] Specifically, a symmetrical distribution along the central axis can mean that the geometric center, acoustic center, or array centerline of the acoustic transducer array is approximately aligned with the central axis of the robot's head or torso, and that multiple transducer units in the acoustic transducer array are arranged symmetrically with respect to the central axis, either horizontally, vertically, or in a mirror image. For example, when the acoustic transducer array adopts a rectangular array layout, the vertical centerline of the array can be aligned with the central axis of the robot's head or torso, and the number and layout of transducer units on the left and right sides of the array can be the same or substantially the same.

[0128] It should be noted that the "symmetrical distribution about the central axis" in this application is not limited to perfect geometric symmetry in a strictly mathematical sense. It can also include an approximately symmetrical distribution formed under the influence of factors such as manufacturing errors, assembly errors, shell surface deformation, wiring space limitations, sensor avoidance, or structural reinforcement avoidance. As long as the overall layout of the acoustic transducer array has a basically symmetrical arrangement trend relative to the central axis of the head or torso, or the acoustic center, geometric center, or main radiation direction of the array roughly corresponds to the central axis, the acoustic transducer array can be considered to be symmetrically distributed about the central axis.

[0129] In this embodiment, by placing the acoustic transducer array on the robot's head or torso and symmetrically distributing it about the central axis of the head or torso, it is beneficial to match the installation position of the acoustic transducer array with the robot's interaction direction facing the target object. When the robot's head or torso rotates under the drive of the actuator, the physical orientation of the acoustic transducer array can be adjusted according to the posture change of the head or torso, thereby facilitating the use of electronic beamforming to achieve directional audio playback in the direction of the target object.

[0130] Figure 4 This is a structural schematic diagram of an embodied mechanism according to an embodiment of this application. For example... Figure 4 As shown, the acoustic transducer array is positioned on the front of the robot's head and is symmetrically distributed about the central axis of the robot's head; microphone 1 and microphone 2 are respectively arranged on both sides of the robot's head. It should be noted that... Figure 4 The example uses two microphones, but in practical applications, the number of microphones can be more than two. For example, four or more microphones can be used, arranged appropriately in spatial positions, to improve sound pickup capabilities and achieve more accurate and smoother object localization. In addition, the robot's head can also be equipped with a camera to acquire spatial orientation information of the interactive object and a posture sensor to acquire robot posture data. Figure 4 (not shown in the image), wherein the camera may include at least one of an RGBD camera, a ToF sensor, or an RGB camera; the attitude sensor may include at least one of an inertial measurement unit or a joint encoder.

[0131] In some alternative embodiments, the acoustic transducer array is disposed on the outer surface of the housing of the head or torso or inside the housing.

[0132] Specifically, the outer surface of the shell can refer to the side of the robot's head or torso shell facing the external environment. When the acoustic transducer array is placed on the outer surface of the shell, multiple transducer units can directly emit ultrasonic modulated signals toward the external space, thereby reducing the shell's obstruction, attenuation, or reflection of the ultrasonic beam.

[0133] The term "within the housing" can refer to the inner space of the robot's head or torso housing. When the acoustic transducer array is placed within the housing, it can be positioned on the inner side of the housing, on an internal support, or on a mounting base, with the emitting surface of the array facing the openings, sound-transmitting areas, acoustic windows, or other areas that allow ultrasonic waves to pass through the housing. This improves both the overall aesthetics and structural protection while providing a propagation path for the ultrasonic modulation signal, reducing the impact of the housing on the ultrasonic beam's obstruction, attenuation, or reflection.

[0134] This embodiment provides two optional structures: mounting on the outer surface of the housing and mounting inside the housing. Mounting on the outer surface of the housing helps reduce propagation loss and facilitates maintenance and replacement; mounting inside the housing helps protect the transducer unit, reduces the impact of external collisions, dust or liquid intrusion on the transducer unit, and maintains the overall integrity of the robot's appearance.

[0135] In some alternative implementations, the controller includes a first processing unit and a second processing unit coupled to the first processing unit; wherein the first processing unit is configured to perform the generation of the ultrasonic modulation signal and the determination of the beamforming parameters; the second processing unit is coupled to the first processing unit via a data bus or communication interface and is configured to perform multi-channel parallel computation based on the beamforming parameters and the ultrasonic modulation signal to generate multi-channel drive data corresponding to the plurality of transducer units.

[0136] Specifically, the first processing unit can be used to perform relatively high-level signal processing and control operations. For example, the first processing unit can receive an audio signal to be reproduced and modulate the audio signal to be reproduced onto an ultrasonic carrier wave to generate an ultrasonic modulation signal; the first processing unit can also determine the beamforming parameters of the acoustic transducer array based on the spatial orientation data of the target object; the first processing unit can also perform tracking operations based on the change of the spatial orientation data of the target object over time to determine the target object's movement state, movement trajectory, relative orientation, or control information for maintaining the ultrasonic beam toward the target object. The first processing unit may include a microprocessor, digital signal processor, system-on-a-chip, microcontroller, graphics processor, neural network processor, or other processing devices capable of performing signal processing and control algorithms.

[0137] The second processing unit can be used to perform multi-channel parallel computing corresponding to multiple transducer units. Specifically, the second processing unit can receive beamforming parameters and ultrasonic modulation signals output by the first processing unit, and perform channelization processing on the ultrasonic modulation signals based on the amplitude data and phase data corresponding to each transducer unit to generate multi-channel drive data corresponding one-to-one with multiple transducer units. The multi-channel drive data may include drive waveforms, amplitude control data, phase control data, delay control data, pulse width modulation data, digital-to-analog conversion data, or combinations thereof for each channel. The second processing unit may include a field-programmable gate array, a digital signal processor, an application-specific integrated circuit, a system-on-a-chip, a microcontroller array, or other processing devices suitable for multi-channel parallel computing.

[0138] The first processing unit and the second processing unit can be coupled via a data bus or communication interface. The data bus or communication interface may include a parallel bus, serial bus, SPI interface, I2C interface, I2S interface, TDM interface, LVDS interface, SerDes interface, Ethernet interface, PCIe interface, or other communication links suitable for transmitting beamforming parameters, ultrasonic modulation signals, or multi-channel control data.

[0139] In one possible implementation, a first processing unit can calculate the amplitude and phase data of each transducer unit based on the spatial orientation data of the target object, and generate an ultrasonic modulation signal containing the audio information to be reproduced. Subsequently, the first processing unit sends the amplitude data, phase data, and ultrasonic modulation signal to a second processing unit. The second processing unit performs multi-channel expansion, phase shifting, delay compensation, amplitude scaling, or pulse width modulation processing on the ultrasonic modulation signal based on the amplitude and phase data to generate multi-channel drive data corresponding to multiple transducer units.

[0140] This implementation separates the generation of ultrasonic modulation signals, the determination of beamforming parameters, and the generation of multi-channel drive data, which helps reduce the computational burden of a single processing unit and improves the real-time performance and synchronization of multi-channel drive data generation. Especially when the acoustic transducer array includes many transducer units, the second processing unit generates drive data for multiple channels through parallel computation, improving the consistency and timing controllability of the data generation for each channel. This, in turn, helps the acoustic transducer array stably form an ultrasonic beam directed towards the target object.

[0141] In some alternative implementations, the controller further includes a multi-channel drive circuit electrically connected between the second processing unit and the acoustic transducer array, configured to receive the multi-channel drive data transmitted by the second processing unit and drive the plurality of transducer units.

[0142] Specifically, a multi-channel drive circuit can refer to a circuit module used to convert the multi-channel drive data into electrical drive signals required by multiple transducer units. The multi-channel drive circuit may include multiple drive channels, each of which may correspond to one transducer unit or a group of transducer units. The drive channel may include a digital-to-analog converter circuit, a pulse width modulation output circuit, a power amplifier circuit, a voltage amplifier circuit, a current drive circuit, an impedance matching circuit, a filter circuit, a protection circuit, or a combination thereof. The electrical drive signal may include an analog drive signal, a digital pulse drive signal, a pulse width modulation drive signal, a sinusoidal drive signal, a pulse sequence drive signal, or other signals suitable for driving the transducer units to emit ultrasonic modulated signals.

[0143] In one possible implementation, the second processing unit generates multi-channel drive data corresponding to multiple transducer units and transmits the multi-channel drive data to a multi-channel drive circuit. The multi-channel drive circuit outputs corresponding electrical drive signals based on the drive data for each channel, causing each transducer unit to transmit the ultrasonic modulation signal according to corresponding amplitude and phase data. For example, when the acoustic transducer array includes multiple transducer units, the multi-channel drive circuit may include independent drive channels corresponding to the number of transducer units, or it may include fewer drive channels than the number of transducer units, and drive the multiple transducer units through group driving, time-division driving, or multiplexing.

[0144] In this embodiment, a multi-channel drive circuit is electrically connected between the second processing unit and the acoustic transducer array, enabling the conversion from multi-channel drive data to actual electrical drive signals. By driving multiple transducer units separately according to the multi-channel drive data, it is beneficial to maintain the required output amplitude, phase, and transmission timing relationships among the multiple transducer units, thereby supporting the acoustic transducer array in forming an ultrasonic beam directed towards the target object. Furthermore, the multi-channel drive circuit can also reduce the impact of inter-channel differences on beamforming effects through channel calibration, gain compensation, phase compensation, delay compensation, or impedance matching.

[0145] In some alternative implementations, the at least one sensor includes at least one of the following: an acoustic sensor, a visual sensor, or an attitude sensor.

[0146] The acoustic sensor is configured to acquire sound signals from the target object for sound source localization to determine first azimuth information. The acoustic sensor may include a microphone, microphone array, or other device capable of acquiring sound signals. The first azimuth information may include the target object's orientation angle, pitch angle, sound source direction estimation result, or a combination thereof, relative to the acoustic sensor or the mounting mechanism.

[0147] The vision sensor is configured to acquire image information of the target object for target recognition, thereby determining second orientation information. The vision sensor may include an RGB camera, a depth camera, an RGB-D camera, a ToF sensor, or other devices capable of acquiring image information of the target object. The second orientation information may include the target object's orientation angle, distance, three-dimensional coordinates, target recognition result, or a combination thereof, relative to the vision sensor or the embodied mechanism.

[0148] The attitude sensor is configured to acquire attitude data of the android to determine the relative orientation of the target object with respect to the acoustic transducer array. The attitude sensor may include an inertial measurement unit, a joint encoder, a gyroscope, an accelerometer, or other devices capable of detecting the attitude and motion state of the android itself. The attitude data may include head posture, torso posture, joint angles, angular velocity, acceleration, or a combination thereof.

[0149] In this embodiment, acoustic and visual sensors can be used to acquire spatial orientation information related to the target object, while attitude sensors can be used to acquire the posture data of the embodied mechanism itself. The controller can determine the relative orientation of the target object with respect to the acoustic transducer array based on the first orientation information determined by the acoustic sensor, the second orientation information determined by the visual sensor, and / or the posture data acquired by the attitude sensor. For example, when the robot's head rotates, the controller can perform coordinate transformation on the target orientation output by the acoustic or visual sensor based on the head posture angle fed back by the attitude sensor, thereby obtaining the relative orientation of the target object with respect to the adjusted acoustic transducer array.

[0150] In one possible implementation, the robot is equipped with a microphone array, an RGB-D depth camera, and an inertial measurement unit. When the target object is in front of the robot and the lighting is sufficient, the visual sensor can be used first for target recognition and distance measurement, outputting second orientation information. When the target object is occluded or the visual confidence level is low, the acoustic sensor can be used for sound source localization, outputting first orientation information. When the robot's head or torso rotates, the attitude sensor provides real-time feedback of the current posture data to correct the relative orientation of the target object with respect to the acoustic transducer array and to compensate for beamforming parameters.

[0151] In some optional embodiments, the embodied mechanism further includes an actuator coupled to the controller, which is further configured to perform the following tracking and locking operations: continuously or periodically acquiring spatial orientation data of the target object; dynamically updating the movement state or trajectory of the target object based on changes in the acquired spatial orientation data, and generating drive commands based on the movement state or trajectory to drive the actuator to move, thereby adjusting the physical orientation of the acoustic transducer array; and updating the beamforming parameters based on the relative orientation of the target object with respect to the adjusted acoustic transducer array, and driving the plurality of transducer units to emit the ultrasonic modulation signal based on the updated beamforming parameters, so that the main lobe direction of the ultrasonic beam points to the target object, and / or so that the focusing area of ​​the ultrasonic beam or the coverage area of ​​the audible sound signal is located or maintained within a preset range based on the position of the target object.

[0152] Specifically, the actuator can refer to a mechanical motion device used to adjust the posture of the head, torso, gimbal, mounting bracket, or other components of the embodied mechanism. Tracking and locking operation can refer to the controller coordinating the actuator and acoustic transducer array based on the spatial orientation data and dynamic changes of the target object, so that the main lobe direction, focusing area, or audible coverage area of ​​the ultrasonic beam adjusts according to the position of the target object.

[0153] In this embodiment, the controller can continuously or periodically acquire spatial orientation data of the target object, and dynamically update the target object's movement status or trajectory based on the changes in the spatial orientation data over time. The movement status may include the target object's current position, movement direction, movement speed, angular velocity, distance change trend, or a combination thereof. The movement trajectory may include a sequence of the target object's position at multiple moments, including its orientation angle, pitch angle, distance, three-dimensional coordinates, or a combination thereof.

[0154] Furthermore, the controller can generate drive commands based on the movement state or trajectory to drive the actuators to move, thereby adjusting the physical orientation of the acoustic transducer array. For example, when the target object moves from the front of the robot to the side front, the controller can generate target angle commands or target angular velocity commands for the neck joint module, waist joint module, or multi-degree-of-freedom gimbal, so that the normal direction or main emission direction of the acoustic transducer array gradually approaches the direction of the target object.

[0155] After the physical orientation of the acoustic transducer array is adjusted, the controller can also update the beamforming parameters according to the relative orientation of the target object with respect to the adjusted acoustic transducer array, and drive multiple transducer units to emit ultrasonic modulated signals based on the updated beamforming parameters. This allows the main lobe of the ultrasonic beam to point towards the target object, and / or ensures that the focusing area of ​​the ultrasonic beam or the coverage area of ​​the audible sound signal is located within or maintained within a preset range based on the position of the target object.

[0156] In one possible implementation, the target object moves laterally in front of the robot. The controller continuously or periodically acquires the spatial orientation data of the target object and updates its movement state or trajectory based on the spatial orientation data at multiple times. The controller generates drive commands for the neck joint module, waist joint module, or multi-degree-of-freedom gimbal based on the movement state or trajectory, causing the robot's head or torso to rotate towards the target object. As the actuators adjust the physical orientation of the acoustic transducer array, the controller updates the beamforming parameters based on the target object's relative orientation to the adjusted acoustic transducer array, reducing the required deflection of the electronic beam and ensuring that the audible sound coverage area continues to cover the preset spatial range where the target object is located.

[0157] This implementation method achieves dynamic tracking of a target object through a combination of mechanical orientation adjustment and electronic beamforming adjustment. Mechanical orientation adjustment can be used to expand the effective coverage of the acoustic transducer array or compensate for significant positional changes in the target object; electronic beamforming adjustment can be used for fine-grained directional control over a smaller area or shorter timescale. The combination of these two methods improves the stability, continuity, and directionality of directional audio playback in mobile interactive scenarios.

[0158] In some alternative implementations, the actuator includes a multi-degree-of-freedom gimbal, a neck joint module, or a waist joint module; the acoustic transducer array is disposed on the head or torso of the automaton, and the actuator is configured to drive the head or torso to perform horizontal rotation and / or pitch movements to adjust the physical orientation of the acoustic transducer array.

[0159] Specifically, a multi-degree-of-freedom gimbal can refer to an electromechanical device with two or more degrees of freedom of motion, such as a gimbal structure capable of performing horizontal rotation and pitch movements. A neck joint module can refer to a joint assembly positioned between the robot's head and torso for driving head movement. A waist joint module can refer to a joint assembly positioned between the robot's torso or upper body and lower structure for driving torso or upper body movement.

[0160] In this embodiment, when the acoustic transducer array is disposed on the head of the android, the actuator can be a neck joint module or a multi-degree-of-freedom gimbal, used to drive the head to perform horizontal rotation and / or pitch movements, thereby adjusting the physical orientation of the acoustic transducer array. When the acoustic transducer array is disposed on the torso of the android, the actuator can be a waist joint module, a multi-degree-of-freedom gimbal, or other mechanisms that drive the torso to move, used to drive the torso to perform horizontal rotation and / or pitch movements, thereby adjusting the physical orientation of the acoustic transducer array.

[0161] In one possible implementation, the robot's head is connected to the torso via a three-degree-of-freedom neck joint module, and an acoustic transducer array is fixed to the head shell. When the target object moves from the front of the robot to its side front, the controller can drive the neck joint module to rotate the head horizontally and perform electronic beam compensation through beamforming parameter updates to direct the ultrasonic beam toward the target object. If the acoustic transducer array is located in the robot's torso, the controller can drive the waist joint module to rotate the torso and, combined with beamforming parameter updates, achieve tracking of the target object.

[0162] In this implementation, the actuator is responsible for adjusting the overall physical orientation of the acoustic transducer array, while electronic beamforming is responsible for fine-tuning the pointing based on the adjusted physical orientation. This reduces beam degradation at extreme electronic deflection angles and improves the directional audio playback tracking range and stability of the embodied mechanism in dynamic interactive scenarios.

[0163] Figure 5 This is a layered schematic diagram of the architecture of a directional audio playback system with an embodied mechanism according to an embodiment of this application. Figure 5 As shown, the system architecture includes a perception layer, a control layer, a power amplifier layer, and an execution layer. The perception layer includes a camera, a microphone array, and an attitude sensor, used to perceive the position of the target object and determine its orientation data through sound source localization via the microphone array or target recognition via the camera. The control layer includes a tracking algorithm, a beamforming algorithm, and a strategy generation module, used to track the real-time position of the target object based on the orientation data, adjust the amplitude and phase data of each transducer unit in the acoustic transducer array based on the robot's attitude relative to the target object, generate beamforming parameters for the directional ultrasonic beam, modulate the audio signal to be reproduced onto the ultrasonic carrier, and output amplitude and phase control data. The power amplifier layer includes multiple power amplifier channels, used to receive the amplitude and phase control data output from the control layer and generate corresponding drive signal data. The execution layer includes multiple transducer units, used to transmit ultrasonic modulated signals according to the drive signal data, sending the directional ultrasonic waves to the target object.

[0164] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0165] The following is a detailed reference. Figure 6 This diagram illustrates a suitable structural schematic for implementing the electronic device described in the embodiments of this application. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from memory 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the electronic device. The processor 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0166] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0167] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a memory 608, or installed from a ROM 602. When the computer program is executed by the processor 601, it performs the functions defined in the directional audio playback method of the embodied mechanism of this application.

[0168] Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0169] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A method for directional audio playback using a embodied mechanism, characterized in that, The method includes: Obtain the spatial orientation data of the target object; Based on the spatial orientation data, the beamforming parameters of the acoustic transducer array are determined. The beamforming parameters include amplitude data and phase data corresponding to multiple transducer units in the acoustic transducer array. The audio signal to be reproduced is modulated onto an ultrasonic carrier wave to generate an ultrasonic modulated signal; and Based on the beamforming parameters, the plurality of transducer units are driven to emit the ultrasonic modulation signal to form an ultrasonic beam toward the target object. The ultrasonic beam is configured to self-demodulate an audible sound signal corresponding to the audio signal to be reproduced during propagation toward the target object.

2. The method according to claim 1, characterized in that, The acquisition of the spatial orientation data of the target object includes: Acquire the sound signal of the target object using an acoustic sensor, and determine the first orientation information of the target object based on the sound signal; and / or Image information of the target object is acquired through a visual sensor, and the second location information of the target object is determined based on the image information; The spatial orientation data is determined based on the first orientation information, the second orientation information, or the fusion information of the first orientation information and the second orientation information.

3. The method according to any one of claims 1 to 2, characterized in that, The method further includes: Continuously or periodically acquire the spatial orientation data of the target object; In response to changes in the spatial orientation data over time, pointing control information for maintaining the ultrasonic beam toward the target object is determined; and Based on the pointing control information, the physical orientation of the acoustic transducer array is adjusted and the beamforming parameters are updated so that the ultrasonic beam remains oriented toward the target object as the orientation of the target object changes.

4. The method according to claim 3, characterized in that, The determination of the pointing control information for maintaining the ultrasonic beam toward the target object includes: The target object is identified as the locked object; Based on the continuously or periodically acquired spatial orientation data, the movement status or movement trajectory of the locked object is determined. Based on the movement state or the movement trajectory, a driving command is generated to drive the actuator of the embodied mechanism to move, thereby adjusting the physical orientation of the acoustic transducer array.

5. The method according to claim 1, characterized in that, The step of modulating the audio signal to be reproduced onto an ultrasonic carrier wave includes: The audio signal to be reproduced is loaded onto the ultrasonic carrier with a frequency greater than 20 kHz using at least one modulation method selected from double-sideband modulation, single-sideband modulation, amplitude modulation, or square root predistortion modulation.

6. The method according to claim 5, characterized in that, The highest frequency of the audio signal to be reproduced is less than or equal to a preset ratio of the frequency of the ultrasonic carrier, and when amplitude modulation is used, the modulation depth used when the audio signal to be reproduced is modulated to the ultrasonic carrier is less than or equal to a preset threshold. The preset ratio and the preset threshold are respectively preset according to the frequency of the ultrasonic carrier, the modulation method and the target distortion index.

7. The method according to claim 1, characterized in that, The step of driving the plurality of transducer units to transmit the ultrasonic modulated signal based on the beamforming parameters includes: Based on the beamforming parameters and the ultrasonic modulation signal, multi-channel drive data corresponding one-to-one with the plurality of transducer units is generated; and The multi-channel drive data is time-synchronized and / or phase-synchronized and output to drive the multiple transducer units to transmit the ultrasonic modulation signal according to the corresponding amplitude data and phase data.

8. An embodied mechanism, characterized in that, include: Acoustic transducer array, comprising multiple transducer units; At least one sensor; as well as A controller coupled to the acoustic transducer array and the at least one sensor, the controller being configured to: Spatial orientation data of the target object are acquired through at least one sensor; Based on the spatial orientation data, the beamforming parameters of the acoustic transducer array are determined, and the beamforming parameters include amplitude data and phase data corresponding to the plurality of transducer units; The audio signal to be reproduced is modulated onto an ultrasonic carrier wave to generate an ultrasonic modulation signal; as well as Based on the beamforming parameters, the plurality of transducer units are driven to emit the ultrasonic modulation signal to form an ultrasonic beam toward the target object. The ultrasonic beam is configured to self-demodulate an audible sound signal corresponding to the audio signal to be reproduced during propagation toward the target object.

9. The embodied mechanism according to claim 8, characterized in that, The arrangement structure of the multiple transducer units in the acoustic transducer array includes at least one of the following: rectangular array, honeycomb array, concentric ring array, arc surface array, and curved surface conformal array.

10. The embodied mechanism according to claim 8, characterized in that, The acoustic transducer array has a highest ultrasonic operating frequency, and the center-to-center distance between two adjacent transducer units is configured to be no greater than half the wavelength of the sound wave corresponding to the highest ultrasonic operating frequency.

11. The embodied mechanism according to claim 8, characterized in that, The embodied mechanism is a robot, and the acoustic transducer array is disposed on the head or torso of the robot and is symmetrically distributed about the central axis of the head or torso.

12. The embodied mechanism according to claim 8, characterized in that, The controller includes a first processing unit and a second processing unit coupled to the first processing unit; wherein... The first processing unit is configured to perform the generation of the ultrasonic modulation signal and the determination of the beamforming parameters; The second processing unit is coupled to the first processing unit via a data bus or communication interface and is configured to perform multi-channel parallel calculations based on the beamforming parameters and the ultrasonic modulation signal to generate multi-channel drive data corresponding to the plurality of transducer units.

13. The embodied mechanism according to claim 12, characterized in that, The controller further includes a multi-channel drive circuit electrically connected between the second processing unit and the acoustic transducer array, configured to receive the multi-channel drive data transmitted by the second processing unit and drive the plurality of transducer units.

14. The embodied mechanism according to claim 8, characterized in that, The at least one sensor includes at least one of the following: An acoustic sensor is configured to acquire sound signals from the target object for sound source localization to determine first location information; A visual sensor is configured to acquire image information of the target object for target recognition, in order to determine second location information; An attitude sensor is configured to acquire attitude data of the embodied mechanism to determine the relative orientation of the target object with respect to the acoustic transducer array.

15. The embodied mechanism according to claim 8, characterized in that, The embodied mechanism also includes an actuator coupled to the controller, which is further configured to perform the following tracking and locking operations: Continuously or periodically acquire the spatial orientation data of the target object; The movement state or trajectory of the target object is dynamically updated based on the changes in the acquired spatial orientation data, and driving commands are generated based on the movement state or trajectory to drive the actuator to move, thereby adjusting the physical orientation of the acoustic transducer array. as well as The beamforming parameters are updated according to the relative orientation of the target object with respect to the adjusted acoustic transducer array, and the plurality of transducer units are driven to emit the ultrasonic modulation signal based on the updated beamforming parameters, so that the main lobe direction of the ultrasonic beam points to the target object, and / or the focusing area of ​​the ultrasonic beam or the coverage area of ​​the audible sound signal is located or maintained within a preset range based on the position of the target object.

16. The embodied mechanism according to claim 15, characterized in that, The actuator includes a multi-degree-of-freedom gimbal, a neck joint module, or a waist joint module; The acoustic transducer array is disposed on the head or torso of the animate, and the actuator is configured to drive the head or torso to perform horizontal rotation and / or pitching movements to adjust the physical orientation of the acoustic transducer array.

Images