Method and apparatus for determining the location of a mobile terminal

Abstract

This specification provides one or more embodiments of a method and apparatus for determining the location of a mobile terminal. After the mobile terminal sends a sound wave signal to a desktop smart hardware product via its speaker, it acquires the sound wave signals received by various audio acquisition modules included in the desktop smart hardware product, and performs sound source localization based on the sound wave signals received by each audio acquisition module to determine the location information of the speaker. After determining the location information of the mobile terminal's speaker, it determines the location information of the mobile terminal based on the speaker's location information.

Description

TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of position determination of a mobile terminal, and in particular to a position determination method and device of a mobile terminal. BACKGROUND

[0002] The desktop intelligent hardware product is a robot product integrating voice interaction, remote control, communication exchange, motion interaction and other interaction modes. In the use process of the desktop intelligent hardware product, the mobile terminal generally needs to be fixed to the desktop intelligent hardware product to complete various tasks together with the desktop intelligent hardware product.

[0003] However, in the use process of the desktop intelligent hardware product, it is necessary to determine whether the mobile terminal is fixed at the specified position of the desktop intelligent hardware product. Meanwhile, after the mobile terminal is fixed to the desktop intelligent hardware product, the position of the mobile terminal needs to be accurately determined to prevent the mobile terminal from colliding in the movement process of the desktop intelligent hardware product. Therefore, how to determine the position of the mobile terminal in the use of the desktop intelligent hardware product is a technical problem to be solved at present. SUMMARY

[0004] Therefore, one or more embodiments of the present specification provide a position determination method and device of a mobile terminal.

[0005] To achieve the above object, one or more embodiments of the present specification provide technical solutions as follows: According to a first aspect of one or more embodiments of the present specification, a position determination method of a mobile terminal is provided, the method is applied to a desktop intelligent hardware product, the desktop intelligent hardware product includes a plurality of audio acquisition modules; the mobile terminal is detachably installed to the desktop intelligent hardware product to complete corresponding tasks together with the desktop intelligent hardware product, and the method includes: acquiring sound wave signals respectively received by each of the audio acquisition modules, the sound wave signals being emitted by a loudspeaker of the mobile terminal; performing sound source positioning based on the sound wave signals received by each of the audio acquisition modules to determine position information of the loudspeaker; determining position information of the mobile terminal based on the position information of the loudspeaker.

[0006] According to a second aspect of one or more embodiments of the present specification, a position determination method of a mobile terminal is provided, the method is applied to a mobile terminal, the mobile terminal is detachably installed to the desktop intelligent hardware product to complete corresponding tasks together with the desktop intelligent hardware product, the desktop intelligent hardware product includes a plurality of audio acquisition modules; and the method includes: determining a sound wave signal for positioning the mobile terminal; emitting, by a speaker of the mobile terminal, a sound wave signal for positioning the mobile terminal, so that the desktop intelligent hardware product determines position information of the speaker based on the sound wave signal emitted by the speaker and received by each of the audio acquisition modules, and determines position information of the mobile terminal based on the position information of the speaker.

[0007] According to a third aspect of one or more embodiments of the present specification, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor implements the method of the first aspect by running the executable instructions.

[0008] According to a fourth aspect of one or more embodiments of the present specification, a computer-readable storage medium is provided, having stored thereon computer instructions that, when executed by a processor, implement the steps of the method of the first aspect.

[0009] According to a fifth aspect of one or more embodiments of the present specification, a computer program product is provided, comprising: computer program / instructions that, when executed by a processor, implement the method of the first aspect.

[0010] As can be seen from the above embodiments, the mobile terminal position determination method and device provided by one or more embodiments of the present specification, after the mobile terminal's speaker sends a sound wave signal to the desktop intelligent hardware product, acquires the sound wave signal received by each audio acquisition module included in the desktop intelligent hardware product, and performs sound source positioning according to the sound wave signal received by each audio acquisition module to determine the position information of the speaker. After determining the position information of the speaker of the mobile terminal, the position information of the mobile terminal is determined according to the position information of the speaker. Since the speaker of the mobile terminal and the audio acquisition module of the desktop intelligent hardware product are inherent components of the equipment, the mobile terminal position determination method provided by the embodiments of the present specification can determine the position information of the mobile terminal without increasing any hardware cost, so as to provide position information for determining whether the mobile terminal is installed in place and obstacle avoidance of the mobile terminal during the use of the mobile terminal with the desktop intelligent hardware product. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a schematic diagram of a mobile terminal used in cooperation with a desktop intelligent hardware product provided in an example related technology.

[0012] Figure 2This is an exemplary embodiment of a diagram showing another mobile terminal used in conjunction with a desktop smart hardware product.

[0013] Figure 3 This is a flowchart illustrating a method for determining the location of a mobile terminal, provided in an exemplary embodiment.

[0014] Figure 4 This is a flowchart illustrating another method for determining the location of a mobile terminal, provided in an exemplary embodiment.

[0015] Figure 5 This is a schematic diagram of the structure of a device provided in an exemplary embodiment.

[0016] Figure 6 This is a block diagram of a location determination device for a mobile terminal provided in an exemplary embodiment.

[0017] Figure 7 This is a block diagram of a location determination device for a mobile terminal provided in an exemplary embodiment. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0019] The organizational information (including but not limited to organizational equipment information, organizational personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) involved in this manual are all information and data authorized by the organization or fully authorized by all parties. Furthermore, the collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation portals are provided for the organization to choose to authorize or refuse.

[0020] As described in the background section, desktop smart hardware products, with their compact size and flexible operation, are widely used in various fields such as office work, home services, education, and companionship. In actual use, considering cost, most of the computing power of desktop smart hardware products typically resides on mobile terminals. Therefore, it is necessary to use mobile terminals (such as mobile phones and tablets) in conjunction with desktop smart hardware products to control, interact with data, or expand functionality. In scenarios where mobile terminals and desktop smart hardware products are used together, it is necessary to ensure that the mobile terminal is fixed in the designated location on the desktop smart hardware product to assist in completing various tasks. However, in practical applications and related technologies, there is a lack of methods for desktop smart hardware products to actively determine whether the mobile terminal is installed in its designated location. Most solutions require manual confirmation of whether the mobile terminal is installed in the designated location on the desktop smart hardware product. Furthermore, if desktop smart hardware products are to be able to autonomously determine whether a mobile terminal is installed, additional components are usually required in the desktop smart hardware products and / or the mobile terminal. For example, a sensor for detecting the mobile terminal can be added to a designated location on the desktop smart hardware products. This would undoubtedly increase the hardware cost of the desktop smart hardware products and affect their overall structure.

[0021] Furthermore, accurately determining the mobile terminal's exact location is equally important after it has been successfully installed in the designated position on the desktop smart hardware product. When the desktop smart hardware product moves or turns within the desktop area, its surrounding environment often contains various obstacles such as books, stationery, and decorative items. Moreover, for some desktop smart hardware products that include multi-segment robotic arms, the mobile terminal may collide with the product itself. Therefore, without accurately obtaining the mobile terminal's location information, it is difficult to effectively predict and avoid its movement trajectory, easily leading to collisions with other obstacles, causing damage to the mobile terminal or the desktop smart hardware product, and even affecting the successful completion of the entire task. However, even with sensors installed at the designated location on the desktop smart hardware product to detect the mobile terminal, it is still difficult to accurately determine its position. For example, when the sensor is a pressure sensor, it can only detect whether the mobile terminal is installed in the designated location, but the specific detailed location information of the mobile terminal is still unavailable. Furthermore, related technologies, after considering the installation position error of the mobile terminal, usually limit the movement range of the desktop smart hardware product, reducing its flexibility. Therefore, determining the location of a mobile terminal when using desktop smart hardware products is a technical problem that urgently needs to be solved.

[0022] In summary, to help desktop smart hardware products determine the location information of mobile terminals without increasing hardware costs, this specification provides a method for determining the location of mobile terminals. This method, without adding any hardware costs, fully utilizes the inherent hardware of both the mobile terminal and the desktop smart hardware product. It innovatively proposes the combined use of the mobile terminal's speaker and the desktop smart hardware product's audio acquisition module, transforming the mobile terminal location determination problem into a sound source localization problem. Furthermore, it uses the location information of the mobile terminal's speaker to determine the mobile terminal's location, allowing the determination of whether the mobile terminal has been installed at a designated location on the desktop smart hardware product. Simultaneously, the location information can be used to prevent collisions during the movement of the desktop smart hardware product. This provides a beneficial solution for the intelligent enhancement and lightweighting of desktop smart hardware products.

[0023] Figure 1 This is a schematic diagram illustrating the use of a mobile terminal in conjunction with a desktop smart hardware product, as provided in an exemplary embodiment. Figure 1 As shown, the mobile terminal 10 is installed at a designated location on the desktop smart hardware product 20. The mobile terminal 10 is equipped with a speaker 101, and its display screen can serve as the facial expression display interface for the desktop smart hardware product. The desktop smart hardware product 20 is equipped with multiple audio acquisition modules 201. When confirming the location of the mobile terminal, the speaker 101 of the mobile terminal 10 emits sound wave signals. Each audio acquisition module 201 of the desktop smart hardware product receives the sound wave signals emitted by the speaker 101. The desktop smart hardware product then performs sound source localization based on the sound wave signals received by each audio acquisition module 201 to determine the location information of the speaker 101, and determines the location information of the mobile terminal based on the location information of the speaker 101. In some embodiments, the mobile terminal 10 may be a mobile phone, tablet computer, PDA (Personal Digital Assistant), or other mobile terminal. This specification does not limit this to one or more embodiments.

[0024] Figure 2 This is an exemplary embodiment illustrating the use of another mobile terminal in conjunction with a desktop smart hardware product. For example... Figure 2 As shown, a multi-segment connection structure is provided between the mobile terminal 10 and the base of the desktop smart hardware product 20, enabling the mobile terminal 10 to move relative to the base. In order to avoid collisions during the movement of the mobile terminal, the speaker 101 of the mobile terminal 10 needs to emit sound wave signals so that the desktop smart hardware product 20 can determine the position information of the mobile terminal 10 based on the sound wave signals received by the audio receiving module 201.

[0025] It should be noted that during the operation of desktop smart hardware products, in order for mobile terminals to cooperate with the desktop smart hardware products to complete various tasks, in some embodiments, the mobile terminal can run a program of an application associated with the desktop smart hardware product system, or switch the mobile terminal to a mode associated with the desktop smart hardware product system to realize the relevant functions of the application. For example, when the mobile terminal runs the program service of the aforementioned mobile terminal location determination method, it can realize a mobile terminal service for that program. The mobile terminal application of the aforementioned program service can be launched and run on the mobile terminal. The program on the mobile terminal side can be a native application installed on the mobile terminal, or it can be a mini-program, quick app, or other similar form. Of course, when using web technologies such as HTML5 or similar technologies, the relevant functions can be realized through a page displayed by a browser. Here, the browser can be a standalone browser application or a browser module embedded in some applications.

[0026] refer to Figure 3 This is a flowchart of a method for determining the location of a mobile terminal provided in this specification. This method is applied to a desktop smart hardware product, which includes multiple audio acquisition modules. The mobile terminal is detachably installed on the desktop smart hardware product to cooperate with it in completing corresponding tasks. The method includes the following steps: S302, acquire the sound wave signals received by each of the audio acquisition modules, the sound wave signals being emitted by the speaker of the mobile terminal.

[0027] To enable the mobile terminal to collaborate with desktop smart hardware products while maintaining independent operation, the mobile terminal can be installed on the desktop smart hardware product during operation to perform various tasks. When the desktop smart hardware product is not running, the mobile terminal can be detached from it for independent use as a regular mobile terminal. Therefore, in the embodiments of this specification, the mobile terminal is detachably installed on the desktop smart hardware product. It should be noted that the mounting components used to achieve detachable installation can be magnetic or snap-fit ​​structures, and the specific mounting components can be set as needed and are not limited thereto.

[0028] To confirm the location of the mobile terminal, the mobile terminal can emit a sound wave signal through a speaker. The frequency of this sound wave signal can be set as needed. In some embodiments of this specification, the frequency of the sound wave signal emitted by the speaker can be determined by the frequency of the noise in the environment where the desktop smart hardware product is located. That is, when determining the frequency of the sound wave signal to be transmitted, the mobile terminal can first obtain the frequency of the current environmental noise from the sound collection module on the mobile terminal, and then determine the frequency of the sound wave signal to be transmitted based on the frequency of the current environmental noise, so that the frequency of the sound wave signal emitted by the speaker is different from the frequency of the noise. In some embodiments, the frequency of the sound wave signal to be transmitted can be any frequency in the range of the noise frequency, and there is no limitation thereto.

[0029] After the mobile terminal's speaker emits a sound wave signal, each audio acquisition module of the desktop smart hardware product receives the sound wave signal emitted by the mobile terminal's speaker. The desktop smart hardware product can acquire the sound wave signals received by each audio acquisition module. In some embodiments, to ensure that the desktop smart hardware product can accurately determine the sound wave signal emitted by the mobile terminal's speaker from multiple sound wave signals, the desktop smart hardware product and the mobile terminal can agree on the frequency of the sound wave signal emitted by the speaker. That is, the mobile terminal's speaker can emit a sound wave signal of a specified frequency, and then the desktop smart hardware product acquires the sound wave signal of that specified frequency received by the audio acquisition modules.

[0030] S304, based on the sound wave signals received by each of the audio acquisition modules, sound source localization is performed to determine the location information of the speaker.

[0031] After acquiring the sound wave signals emitted by the speaker of the mobile terminal received by each audio acquisition module, the desktop smart hardware product can treat the speaker as a sound source and locate the sound source based on the sound wave signals received by each audio acquisition module to determine the speaker's location information. It should be noted that the speaker included in the mobile terminal is generally used for playing music or video audio. The audio acquisition modules included in the desktop smart hardware product are generally used to collect user voice commands. However, in this embodiment, a novel approach is proposed to locate the speaker by receiving sound wave signals emitted by the speaker through the audio acquisition module. This provides an implementable technical solution for confirming the location of the mobile terminal without adding any additional hardware.

[0032] To accurately determine the location of the speaker, in some embodiments of this specification, sound source localization is performed based on the sound wave signals received by each of the audio acquisition modules, including: The positioning assistance information sent by the mobile terminal is obtained, wherein the positioning assistance information represents information about the sound wave signal emitted by the speaker; Using the positioning assistance information as a unified signal processing basis, the sound wave signals received by each of the audio acquisition modules are processed with reference to the positioning assistance information. The processing results of the sound wave signals received by each of the audio acquisition modules are summarized to determine the location information of the speaker.

[0033] Considering that mobile terminals and desktop smart hardware products can interact with each other, in this embodiment, when locating the speaker, the mobile terminal can further send location assistance information to the desktop smart hardware product. This provides additional information for speaker location, reducing the computational difficulty of speaker location and providing greater assurance for accurately calculating the speaker's position. The location assistance information can be sent through the network used for interaction between the mobile terminal and the desktop smart hardware product. In some embodiments, the network used for interaction between the mobile terminal (such as a mobile phone) and the desktop smart hardware product can be either wired or wireless, depending on the communication methods supported by the mobile terminal. This specification does not impose any limitations on this. For example, if the mobile terminal supports both wired and wireless communication, then either wired or wireless network can be used as needed. The location assistance information sent by the mobile terminal can characterize the sound wave signal emitted by the speaker. After obtaining the positioning assistance information, the desktop smart hardware product can use the positioning assistance information as a unified signal processing basis to process the sound wave signals received by each of the audio acquisition modules respectively with reference to the positioning assistance information, and summarize the processing results corresponding to the sound wave signals received by each of the audio acquisition modules to determine the location information of the speaker.

[0034] In some embodiments of this specification, the sound wave signals received by each of the audio acquisition modules are processed with reference to the positioning assistance information, including: Based on the positioning assistance information and the sound wave signals received by each audio acquisition module, the target duration for the sound wave signal to propagate from the speaker to each audio acquisition module is determined respectively. The distance between each audio acquisition module and the speaker is determined based on the target duration and the propagation speed of the sound wave signal.

[0035] When processing the sound wave signals received by each of the audio acquisition modules based on the positioning assistance information, the target duration for the sound wave signal to travel from the speaker to each audio acquisition module can be determined first, based on the positioning assistance information and the sound wave signals received by each audio acquisition module. Since the speed of sound wave propagation in air is generally a fixed value, after obtaining the target duration for the sound wave signal to travel from the speaker to each audio acquisition module, the distance between each audio acquisition module and the speaker can be determined based on its corresponding target duration and the speed of sound wave propagation. In some embodiments, for ease of calculation, the speed of sound wave propagation can be set to a fixed value, for example, 340 m / s. Alternatively, the temperature of the environment in which the desktop smart hardware product is located can be obtained first, and then the aforementioned fixed value can be adjusted according to the ambient temperature.

[0036] To accurately determine the target duration of sound wave signal propagation from the speaker to each audio acquisition module, in some embodiments of this specification, based on the positioning assistance information and the sound wave signal received by each audio acquisition module, the target duration of sound wave signal propagation from the speaker to each audio acquisition module is determined, including: When the positioning assistance information includes the emission time of the sound wave signal emitted by the speaker, for each audio acquisition module, the receiving time corresponding to the sound wave signal received by the audio acquisition module is obtained, and the target duration required for the sound wave signal to propagate from the speaker to the audio acquisition module is determined based on the emission time and the receiving time corresponding to the audio acquisition module. or, When the sound wave signal emitted by the speaker is a linear frequency modulated continuous sound wave signal, and the positioning assistance information includes the sound wave signal emitted by the speaker, for each audio acquisition module, the sound wave signal received by the acquisition module is mixed with the sound wave signal emitted by the speaker included in the positioning assistance information. For the obtained mixed signal, the beat frequency corresponding to the low frequency part is determined, and the target duration required for the sound wave signal to propagate from the speaker to the audio acquisition module is determined according to the sweep bandwidth, period and beat frequency of the linear frequency modulated continuous sound wave signal.

[0037] In some embodiments, the positioning assistance information may include the emission time of the sound wave signal emitted by the speaker, and then, in conjunction with the reception time of the sound wave signal received by different audio acquisition modules, the target duration required for the sound wave signal to propagate from the speaker to each audio acquisition module can be determined by using the difference between the two.

[0038] Considering the relatively fast propagation speed of sound waves and the close proximity of the speaker to the audio acquisition module, determining the target duration by the time difference between the emission and reception of the sound wave signal requires high accuracy in the reception time of the sound wave signal received by the audio acquisition module. For the audio acquisition module to confirm that the received sound is a sound wave signal emitted by the speaker, the received sound wave signal generally needs to reach a certain threshold. Therefore, errors generally occur when determining the reception time of the sound wave signal. To further improve the accuracy of the target duration, in some embodiments, the sound wave signal emitted by the speaker can be a linear frequency modulated continuous sound wave signal, and the positioning assistance information can include the sound wave signal emitted by the speaker. It should be noted that since the sound wave signals received by the audio acquisition module have a certain propagation delay, the sound wave signal received by the audio acquisition module is not equal to the original sound wave signal emitted by the speaker. In this embodiment, the original sound wave information emitted by the speaker can be transmitted to the desktop smart hardware product through the positioning assistance information. In this way, desktop smart hardware products can simultaneously obtain the initial sound wave signal before the delay and the sound wave signal after the delay. After mixing the two, a low-frequency beat frequency signal that is linearly related to the delay will be generated. By extracting this beat frequency, the target duration (i.e., the sound wave propagation delay) can be deduced in reverse.

[0039] In some embodiments, the process of determining the target duration using a mixing signal may include: for each audio acquisition module, multiplying the sound wave signal received by the audio acquisition module with the sound wave signal emitted by the speaker included in the positioning assistance information to obtain a mixing signal; then obtaining the starting frequency of the linear frequency modulated continuous sound wave signal; determining the cutoff frequency based on the starting frequency; and performing low-pass filtering on the mixing signal based on the cutoff frequency to obtain a low-frequency signal, i.e., the low-frequency signal is a signal with a frequency lower than the cutoff frequency. In one example, the cutoff frequency can be set to twice the starting frequency. After obtaining the low-frequency signal, it can be converted into a frequency domain signal using a fast Fourier transform, and the beat frequency of the low-frequency signal can be determined based on the frequency corresponding to the peak with the largest amplitude in the frequency domain signal; then, obtaining the sweep bandwidth and period of the linear frequency modulated continuous sound wave signal, and determining the target duration required for the sound wave signal to propagate from the speaker to the audio acquisition module based on the sweep bandwidth, the period, and the beat frequency.

[0040] It should be noted that the target duration can be determined using the following formula: ; in, Indicates the target duration. f bThe frequency is represented by T, the period is represented by B, and the sweep bandwidth is represented by B.

[0041] To reduce the number of audio acquisition modules used, in some embodiments of this specification, the plurality of audio acquisition modules includes at least three non-collinear audio acquisition modules; the processing results corresponding to the sound wave signals received by each of the audio acquisition modules are summarized, including: Based on the distance between each of the three non-collinear audio acquisition modules and the speaker, two alternative position information for the speaker are determined; When the mobile terminal has been installed on the connection part of the desktop smart hardware product, the location information of the connection part is determined, and the candidate location information that is relatively closer to the connection part among the two candidate location information is determined as the location information of the speaker.

[0042] After obtaining the distances between each audio acquisition module and the speaker, based on the principle that four spheres can determine a point in space, generally at least four non-coplanar audio acquisition modules are needed to determine the speaker's position. However, in this embodiment, to reduce the number of audio acquisition modules used in some scenarios, two alternative speaker positions can be determined first based on the distances between the three non-collinear audio acquisition modules and the speaker; that is, three spheres in space can determine two points. Then, with the mobile terminal already installed on the connection part of the desktop smart hardware product, the position information of the connection part is determined, and the alternative position information that is closer to the connection part is determined as the speaker's position information. It should be noted that in some scenarios, even if it has been confirmed that the mobile terminal has been installed on the connection part of the desktop smart hardware product, it is still necessary to further determine the speaker's information to more accurately determine the mobile terminal's position information for obstacle avoidance. Considering that users generally install the mobile terminal on a designated location (i.e., the connection part) when using the desktop smart hardware product, in some embodiments, when determining the speaker's position information through the connection part, it can be directly assumed that the mobile terminal has been installed on the connection part. Alternatively, in some embodiments, confirmation can be obtained manually or through a connection protocol between the mobile terminal and the connector to confirm that the mobile terminal has been installed on the connector. In some embodiments, the specific structure of the connector can be customized as needed; for example, it can be configured as a suction cup or a snap-fit ​​structure, without limitation.

[0043] In some embodiments, the desktop smart hardware product further includes an intermediate component disposed between the connecting part and the base of the desktop smart hardware product. The intermediate component is used to control the displacement of the connecting part relative to the base. The initial position information of the connecting part can be calibrated at the factory and saved in the desktop smart hardware product. When the position of the connecting part changes, the displacement information of the intermediate component and the initial position information of the connecting part can be obtained, and then the position information of the connecting part can be determined based on the displacement information and the initial position information. In some embodiments, the position information of the connecting part may include the spatial coordinates and pose information of the connecting part. In some embodiments, the intermediate component may be an intermediate component capable of rotation, translation, etc., for example, a robotic arm, and is not limited thereto.

[0044] To accurately locate the sound source, in some embodiments of this specification, sound source localization is performed based on the sound wave signals received by each of the audio acquisition modules, including: Based on the differences in how sound waves are propagated from sound sources at different locations to the multiple audio acquisition modules, the sound waves received by the multiple audio acquisition modules are comprehensively calculated to obtain the location information of the loudspeaker.

[0045] Considering that the positioning assistance signal may be affected by network signal interference, this embodiment makes full use of the differences in how sound wave signals are propagated to the multiple audio acquisition modules from sound sources at different locations, and performs comprehensive calculations on the sound wave signals received by the multiple audio acquisition modules respectively to obtain the speaker's position information.

[0046] To ensure accurate comprehensive calculations, in some embodiments of this specification, the plurality of audio acquisition modules includes at least three non-collinear audio acquisition modules; the comprehensive calculation of the sound wave signals received by the plurality of audio acquisition modules includes: Based on the spatial coordinate information of the three non-collinear audio acquisition modules and the signal reception time difference between each pair, a set of constraint equations for the position of the speaker is established. The position information of the loudspeaker is determined by the set of constraint equations.

[0047] In practice, the spatial coordinates of the three non-collinear audio acquisition modules in a spatial coordinate system can be obtained first. This spatial coordinate system can be a spatial coordinate system with a preset reference point as its origin. In some embodiments, for ease of calculation, this preset reference point can be set in the desktop smart hardware product. Then, based on the reception time corresponding to the sound wave signals received by each of the three non-collinear audio acquisition modules, the signal reception time difference between any two of the three non-collinear audio acquisition modules is calculated. Based on the spatial coordinates of the three non-collinear audio acquisition modules, the signal reception time difference between any two audio acquisition modules, and the propagation speed of the sound wave signal, a set of constraint equations constraining the position of the speaker is established. Finally, the position information of the speaker is determined through the set of constraint equations.

[0048] In some embodiments, the set of constraint equations includes: ; in,( x, y, z ) represents the coordinates of the loudspeaker in the spatial coordinate system, ( x i ，y i ，z i ) indicates the first i The spatial coordinates of each audio acquisition module, ( x j ，y j ，z j ) indicates the first j The spatial coordinates of each audio acquisition module, and i and j They are not equal. c represents the speed of sound wave propagation. τ ij Indicates the first i The audio acquisition module and the first j The signal reception time difference of each audio acquisition module.

[0049] In some embodiments of this specification, the plurality of audio acquisition modules includes at least three non-collinear audio acquisition modules; the comprehensive calculation of the sound wave signals received by the plurality of audio acquisition modules includes: When the mobile terminal has been installed on the connection part of the desktop smart hardware product, the spatial direction range of the speaker's position relative to a preset reference point is determined based on the position information of the connection part. Obtain the spatial coordinate information of each of the three non-collinear audio acquisition modules with the preset reference point as the origin; For each spatial direction within the spatial direction range, the signal strength of the three non-collinear audio acquisition modules in that spatial direction is determined by performing time delay compensation on the sound wave signal. The target spatial direction with the highest signal strength is determined from each spatial direction within the spatial direction range, and the position information of the loudspeaker is determined based on the target spatial direction.

[0050] In this embodiment, the location information of the speaker can be determined by beamforming sound source localization technology. However, considering that beamforming requires determining the spatial direction range of the speaker's position relative to a preset reference point, current related technologies generally use the preset reference point as the center and determine all spatial directions corresponding to the entire sphere or half of the sphere as the spatial direction range. This undoubtedly increases the computational load of beamforming. In order to improve the computational efficiency of beamforming, this embodiment considers determining the spatial direction range of the speaker's position relative to the preset reference point based on the location information of the connection part when the mobile terminal has been installed on the connection part of the desktop smart hardware product. That is, only the spatial direction of the speaker's position relative to the preset reference point corresponding to the four sides of the connection part (preset spatial range) is used as the spatial direction range that needs to be calculated.

[0051] After acquiring the spatial coordinate information of each of the three non-collinear audio acquisition modules in a spatial coordinate system with the preset reference point as the origin, for each spatial direction within the spatial direction range, the source direction of the sound wave signal received by each of the three non-collinear audio acquisition modules is determined as the spatial direction. Time delay compensation is then performed on the sound wave signal received by each of the three non-collinear audio acquisition modules from the spatial direction to obtain three compensated sound wave signals corresponding to the spatial direction. The three compensated sound wave signals are then weighted and summed, and the signal strength corresponding to the spatial direction is determined based on the weighted summation result. After determining the signal strength of the three non-collinear audio acquisition modules in each spatial direction, the target spatial direction with the highest signal strength is determined from each spatial direction within the spatial direction range, and the position information of the speaker is determined based on the target spatial direction. In some embodiments, after determining the target spatial direction corresponding to the speaker, the target distance between the speaker and the preset reference point can also be determined first, and then the speaker's position information can be determined based on the target distance and the target spatial direction. Alternatively, in some embodiments, the distance between the speaker and any one of the three non-collinear audio acquisition modules can be determined first, and then the speaker's position information can be determined based on the distance and the target spatial direction.

[0052] To improve the efficiency of calculating the speaker's position information, in some embodiments of this specification, the speaker's position information is determined based on the target spatial direction, including: The position information of the plane where the speaker is located is determined based on the pose information of the connecting part; Based on the position information of the plane where the speaker is located and the target spatial direction, determine the position information of the target intersection point of the straight line corresponding to the plane where the speaker is located and the target spatial direction, and determine the position information of the target intersection point as the position information of the speaker.

[0053] When the mobile terminal is installed on the connection part of the desktop smart hardware product, the mobile terminal is generally parallel to the connection part and attached to it. Therefore, the position information of the plane where the speaker is located can be determined first based on the pose information of the connection part. Then, the position information of the target intersection point of the straight line corresponding to the plane where the speaker is located and the target spatial direction can be determined based on the position information of the plane where the speaker is located and the target spatial direction. The position information of the target intersection point is then determined as the position information of the speaker.

[0054] S306, Based on the location information of the speaker, determine the location information of the mobile terminal.

[0055] After obtaining the speaker's location information, since the speaker is mounted on the mobile terminal, the location information of the mobile terminal can be determined using the speaker's location information. In some embodiments, when determining the location information of the mobile terminal based on the speaker's location information, the speaker's location information can be directly used as the location information of the mobile terminal. Although this has a certain degree of error, it can help desktop smart hardware products determine whether the mobile terminal has been installed in a specified location.

[0056] To determine the precise location of a mobile terminal, in some embodiments, determining the location information of the mobile terminal based on the location information of the speaker may include: Obtain the dimensions and specifications of the mobile terminal, as well as the mounting position information of the speaker on the mobile terminal; The location information of the mobile terminal is determined based on the size specifications, the assembly position information, and the speaker position information.

[0057] It should be noted that the dimensions of the mobile terminal and the mounting position of the speaker on the mobile terminal can be determined based on the specific model of the mobile terminal. Alternatively, in some embodiments, the dimensions and mounting position information can be set as default information based on experience, and this is not limited. Having obtained the dimensions of the mobile terminal and the mounting position information of the speaker on the mobile terminal, the position information of the mobile terminal can be determined more accurately based on this information and the speaker's position information.

[0058] In some embodiments of this specification, when the mobile terminal includes at least three non-collinear speakers, determining the position information of the mobile terminal based on the size specifications, the assembly position information, and the position information of the speakers includes: Based on the size specifications, the position information of the three non-collinear speakers, and their respective assembly position information, the position information of the mobile terminal is determined.

[0059] When the mobile terminal includes at least three non-collinear speakers, based on the principle that three points determine a plane, the position information of the mobile terminal can be determined directly based on the position information of the three non-collinear speakers, the size specifications, and the assembly position information corresponding to the three non-collinear speakers.

[0060] In some embodiments of this specification, when the number of speakers in the mobile terminal is less than or equal to two, determining the position information of the mobile terminal based on the size specifications, the assembly position information, and the position information of the speakers includes: When the mobile terminal has been installed on the connection part of the desktop smart hardware product, the position and pose information of the connection part is determined; The position information of the plane where the mobile terminal is located is determined based on the pose information; Based on the size specifications, the speaker's location information, and the assembly location information, the location information of the mobile terminal is determined from the plane where the mobile terminal is located.

[0061] Considering that in some scenarios, the mobile terminal may not be equipped with three or more speakers, in order to accurately determine the position of the mobile terminal, it is advisable to fully utilize the pose information of the connector when the mobile terminal is already installed on the connection part of the desktop smart hardware product. That is, the position information of the plane on which the mobile terminal is located can be determined first through the pose information of the connector. Then, based on the size specifications, the position information of the speakers, and the assembly position information, the position information of the mobile terminal can be determined from the plane on which the mobile terminal is located. For example, when the mobile terminal includes two speakers, the position information of the two speakers can be used to fix the mobile terminal in its plane.

[0062] In some embodiments of this specification, when the mobile terminal includes a speaker, determining the position information of the mobile terminal from the plane where the mobile terminal is located based on the size specifications, the speaker's position information, and the assembly position information includes: Based on the pose information, determine the position information of the connection point where the mobile terminal contacts the connection part; Based on the size specifications, the speaker's position information, the assembly position information, and the connection point's position information, the position information of the mobile terminal is determined from the plane where the mobile terminal is located.

[0063] When a mobile terminal includes a speaker, the mobile terminal can rotate around the speaker's position on its own plane. Therefore, in order to accurately determine the position of the mobile terminal, in this embodiment, the position information of the connection point where the mobile terminal contacts the connection part is first determined based on the pose information of the connection part. This connection point can generally be assumed to be the center point of the plane where the mobile terminal is located, or, as needed, a connection point at another position on the plane can be set; there is no limitation on this. After determining the position information of the connection point, the position information of the mobile terminal can be determined from the plane where the mobile terminal is located based on the size specifications, the speaker's position information, the assembly position information, and the connection point's position information.

[0064] In some embodiments of this specification, the method further includes: If, based on the location information of the mobile terminal, it is determined that the distance between the mobile terminal and the connection part of the desktop smart hardware product is less than a preset distance, it is determined that the mobile terminal has been installed on the connection part; And / or, The anti-collision zone of the mobile terminal is determined based on the location information of the mobile terminal in order to avoid collisions during the movement of the mobile terminal in the desktop smart hardware product.

[0065] After determining the location information of the mobile terminal, the desktop smart hardware product can determine whether the distance between the mobile terminal and the connection point of the desktop smart hardware product is less than a preset distance. If it is less, the desktop smart hardware product can determine that the mobile terminal has been installed on the connection point. Simultaneously, the desktop smart hardware product can also determine the anti-collision zone of the mobile terminal based on its location information to prevent the mobile terminal from colliding with the desktop smart hardware product during its movement.

[0066] In some embodiments of this specification, the sound wave signal emitted by the speaker is two adjacent linear frequency modulated (LFM) continuous sound wave signals. The method further includes: acquiring two adjacent LFM continuous sound wave signals emitted by the speaker of the mobile terminal and received by the target audio acquisition module, and calculating the phase difference between the two adjacent LFM continuous sound wave signals at a target distance gate; the target audio acquisition module is any one of the plurality of audio acquisition modules; acquiring the center frequency of the LFM continuous sound wave signal, and determining the wavelength of the sound wave signal based on the center frequency and the propagation speed of the sound wave signal; acquiring the transmission time interval of the two adjacent LFM continuous sound wave signals, and determining the moving speed of the mobile terminal relative to the target audio acquisition module based on the relationship between the transmission time interval, the wavelength, the phase difference, and the speed of the speaker. It should be noted that after determining the moving speed of the mobile terminal relative to the target audio acquisition module, the anti-collision zone of the mobile terminal can be determined more accurately based on the moving speed and the position information of the mobile terminal.

[0067] In some embodiments, the moving speed of the mobile terminal relative to the target audio acquisition module can be calculated using the following formula: ; ; in: v T represents the moving speed of the mobile terminal relative to the target audio acquisition module. f The time interval between the transmission of two adjacent linear frequency modulated continuous acoustic signals is represented by ΔΦ, where ΔΦ represents the phase difference, λ represents the wavelength of the acoustic signal, and c represents the propagation speed of the acoustic signal. f c It represents the center frequency of a linear frequency modulated continuous acoustic signal.

[0068] The mobile terminal location determination method provided in this specification involves sending sound wave signals from the mobile terminal's speaker to a desktop smart hardware product, acquiring the sound wave signals received by each audio acquisition module of the desktop smart hardware product, and locating the sound source based on the sound wave signals received by each audio acquisition module to determine the speaker's location information. After determining the location information of the mobile terminal's speaker, the location information of the mobile terminal is determined based on the speaker's location information. Since the mobile terminal's speaker and the desktop smart hardware product's audio acquisition modules are inherent components of the device, the mobile terminal location determination method provided in this specification can determine the mobile terminal's location information without increasing any hardware costs. This provides location information for determining whether the mobile terminal is properly installed and for obstacle avoidance during the use of the mobile terminal in conjunction with the desktop smart hardware product.

[0069] refer to Figure 4 This is a flowchart of a method for determining the location of a mobile terminal provided in this specification. The method is applied to a mobile terminal that is detachably installed on a desktop smart hardware product to cooperate with the desktop smart hardware product in completing corresponding tasks. The desktop smart hardware product includes multiple audio acquisition modules. The method includes the following steps: S402, determine the acoustic signal used to locate the mobile terminal.

[0070] To help desktop smart hardware products determine the location of mobile terminals, the mobile terminals need to first determine a sound wave signal to be transmitted, which is used to locate the mobile terminals. In some embodiments, the sound wave signal to be transmitted can be preset as needed; for example, the frequency of the sound wave signal to be transmitted can be set to any frequency. To further reduce the interference of environmental noise on the sound wave signal, in some embodiments, determining the sound wave signal used to locate the mobile terminals includes: first acquiring the frequency of the noise in the environment where the desktop smart hardware product is located, and then determining the frequency of the sound wave signal used to locate the mobile terminals based on the frequency of the noise in the environment where the desktop smart hardware product is located, so that the frequency of the sound wave signal emitted by the speaker is different from the frequency of the noise.

[0071] S404, the mobile terminal emits a sound wave signal for locating the mobile terminal through its speaker, so that the desktop smart hardware product determines the location information of the speaker based on the sound wave signal emitted by the speaker received by each of the audio acquisition modules, and determines the location information of the mobile terminal based on the location information of the speaker.

[0072] After determining the sound wave signal to be transmitted, the mobile terminal can emit a sound wave signal through a speaker to locate itself. This allows the desktop smart hardware product to determine the speaker's location information based on the sound wave signals received by each of the audio acquisition modules, and then determine the mobile terminal's location information based on the speaker's location information. To further reduce the complexity of speaker positioning calculations, in some embodiments, the mobile terminal's location determination method further includes sending positioning assistance information to the desktop smart hardware product; wherein the positioning assistance information represents information about the sound wave signal emitted by the speaker.

[0073] In some embodiments of this specification, after receiving positioning assistance information sent by a mobile terminal, the desktop smart hardware product uses the positioning assistance information as a unified signal processing basis, and processes the sound wave signals received by each of the audio acquisition modules by referring to the positioning assistance information; and determines the location information of the speaker by summarizing the processing results corresponding to the sound wave signals received by each of the audio acquisition modules.

[0074] In some embodiments of this specification, the desktop smart hardware product determines the target duration for the sound wave signal to propagate from the speaker to each audio acquisition module based on the positioning assistance information and the sound wave signal received by each audio acquisition module, and determines the distance between each audio acquisition module and the speaker based on the target duration and the propagation speed of the sound wave signal.

[0075] In some embodiments of this specification, when the positioning assistance information includes the emission time of the sound wave signal emitted by the speaker, the desktop smart hardware product, for each audio acquisition module, obtains the reception time corresponding to the sound wave signal received by the audio acquisition module, and determines the target duration required for the sound wave signal to propagate from the speaker to the audio acquisition module based on the emission time and the reception time corresponding to the audio acquisition module.

[0076] In some embodiments of this specification, when the sound wave signal emitted by the speaker is a linear frequency modulated continuous sound wave signal and the positioning assistance information includes the sound wave signal emitted by the speaker, for each audio acquisition module, the sound wave signal received by the acquisition module is mixed with the sound wave signal emitted by the speaker included in the positioning assistance information. For the obtained mixed signal, the beat frequency corresponding to the low frequency part is determined, and the target duration required for the sound wave signal to propagate from the speaker to the audio acquisition module is determined according to the sweep bandwidth, period, and beat frequency of the linear frequency modulated continuous sound wave signal.

[0077] In some embodiments of this specification, the plurality of audio acquisition modules includes at least three non-collinear audio acquisition modules. The desktop smart hardware product determines two alternative location information of the speaker based on the distance between each of the three non-collinear audio acquisition modules and the speaker. When the mobile terminal has been installed on the connection part of the desktop smart hardware product, the location information of the connection part is determined, and the alternative location information that is relatively closer to the connection part among the two alternative location information is determined as the location information of the speaker.

[0078] In some embodiments of this specification, the desktop smart hardware product uses the differences in how sound wave signals are propagated from sound sources at different locations to the multiple audio acquisition modules as a basis to perform comprehensive calculations on the sound wave signals received by the multiple audio acquisition modules respectively, thereby obtaining the speaker's location information.

[0079] In some embodiments of this specification, the plurality of audio acquisition modules includes at least three non-collinear audio acquisition modules. The desktop smart hardware product establishes a set of constraint equations for the position of the speaker based on the spatial coordinate information of each of the three non-collinear audio acquisition modules and the signal reception time difference between each pair; and determines the position information of the speaker through the set of constraint equations.

[0080] In some embodiments of this specification, the plurality of audio acquisition modules includes at least three non-collinear audio acquisition modules; when the mobile terminal is installed on the connection part of the desktop smart hardware product, the desktop smart hardware product determines the spatial direction range of the speaker's position relative to a preset reference point based on the position information of the connection part; acquires the spatial coordinate information of each of the three non-collinear audio acquisition modules with the preset reference point as the origin; for each spatial direction within the spatial direction range, the signal strength of the three non-collinear audio acquisition modules in that spatial direction is determined by performing time delay compensation on the sound wave signal; the target spatial direction with the highest signal strength is determined from each spatial direction within the spatial direction range, and the position information of the speaker is determined based on the target spatial direction.

[0081] In some embodiments of this specification, the desktop smart hardware product determines the position information of the plane where the speaker is located based on the pose information of the connecting part; determines the position information of the target intersection point of the straight line corresponding to the plane where the speaker is located and the target spatial direction based on the position information of the plane where the speaker is located and the target spatial direction, and determines the position information of the target intersection point as the position information of the speaker.

[0082] In some embodiments of this specification, the desktop smart hardware product determines the location information of the speaker as the location information of the mobile terminal.

[0083] In some embodiments of this specification, the desktop smart hardware product acquires the size specifications of the mobile terminal and the assembly position information of the speaker on the mobile terminal; and determines the position information of the mobile terminal based on the size specifications, the assembly position information and the position information of the speaker.

[0084] In some embodiments of this specification, when the number of speakers in the mobile terminal is less than or equal to two, and the mobile terminal is installed on the connection part of the desktop smart hardware product, the desktop smart hardware product determines the pose information of the connection part; determines the position information of the plane where the mobile terminal is located based on the pose information; and determines the position information of the mobile terminal from the plane where the mobile terminal is located based on the size specifications, the position information of the speakers, and the assembly position information.

[0085] In some embodiments of this specification, when the mobile terminal includes a speaker and the mobile terminal is installed to the connection part of the desktop smart hardware product, the desktop smart hardware product determines the position information of the connection point where the mobile terminal contacts the connection part based on the pose information; and determines the position information of the mobile terminal from the plane where the mobile terminal is located based on the size specifications, the position information of the speaker, the assembly position information, and the position information of the connection point.

[0086] In some embodiments of this specification, when the mobile terminal includes at least three non-collinear speakers, the desktop smart hardware product determines the position information of the mobile terminal based on the size specifications, the position information of each of the three non-collinear speakers, and their respective assembly position information.

[0087] In some embodiments of this specification, if the desktop smart hardware product determines that the distance between the mobile terminal and the connection part of the desktop smart hardware product is less than a preset distance based on the location information of the mobile terminal, it determines that the mobile terminal has been installed on the connection part.

[0088] In some embodiments of this specification, the desktop smart hardware product determines the anti-collision zone of the mobile terminal based on the location information of the mobile terminal, so as to avoid the mobile terminal from colliding with the desktop smart hardware product during its movement.

[0089] It should be noted that, in the above embodiments, the specific process by which the desktop smart hardware product determines the location information of the speaker based on the sound wave signal emitted by the speaker received by each of the audio acquisition modules, and determines the location information of the mobile terminal based on the location information of the speaker, can be referred to the specific process of steps S302 to S306 above, and will not be repeated here.

[0090] Figure 5 This is a schematic structural diagram of a device provided in an exemplary embodiment. For example... Figure 5 As shown, device 500 mainly consists of a communication interface 502, a mechanism interface 504, a processor 506, and a data storage 508. These components are interconnected and communicate with each other via a method bus, network, or other connection mechanism 510. The communication interface 502 enables device 500 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 502 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 502 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 502 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 502 may also include multiple physical communication interfaces, such as Wi-Fi, Bluetooth, and wide-area wireless interfaces.

[0091] Mechanism interface 504 includes receiving mechanism input and providing output to the mechanism. Therefore, mechanism interface 504 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. Mechanism interface 504 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, mechanism interface 504 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external mechanism input / output devices. Additionally or alternatively, device 500 may support remote access from other devices via communication interface 502 or another physical interface (not shown). Mechanism interface 504 may be configured to receive mechanism input, the position and movement of which may be indicated by an indicator or cursor described herein. Mechanism interface 504 may also be configured as a display device for rendering or displaying text fragments.

[0092] Processor 506 may contain one or more general-purpose processors and / or special-purpose processors.

[0093] Data storage 508 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 506. Data storage 508 may include removable and non-removable components.

[0094] Processor 506 is capable of executing program instructions 518 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 508 to perform the various functions described herein. Data storage 508 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 500, enable device 500 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 518 by processor 506 may result in processor 506 using data 512.

[0095] For example, program instructions 518 may include operating methods 522 (e.g., operating method kernels, device drivers, and / or other modules) installed on device 500 and one or more applications 520 (e.g., browsers, social applications, or game applications). Similarly, data 512 may include operating method data 516 and application data 514. Operating method data 516 is primarily accessible to operating method 522, while application data 514 is primarily accessible to one or more applications 520. Application data 514 may reside in file methods visible or hidden from the device 500.

[0096] Application 520 can communicate with operation method 522 through one or more application programming interfaces (APIs). These APIs help application 520 read and / or write application data 514, transmit or receive information via communication interface 502, receive or display information on mechanism interface 504, etc.

[0097] In some terminology, application 520 may be simply referred to as "app". Furthermore, application 520 can be downloaded to device 500 through one or more online app stores or app markets. However, applications can also be installed on device 500 in other ways, such as through a web browser or a physical interface on device 500 (e.g., a USB port).

[0098] Please refer to Figure 6 In some embodiments, the location determination device for a mobile terminal can be applied to, for example... Figure 5The device shown implements the technical solution described in this specification. The electronic device may be a desktop smart hardware product, which includes multiple audio acquisition modules; the mobile terminal is detachably installed on the desktop smart hardware product to cooperate with it in completing corresponding tasks; the location determination device of the mobile terminal may include: The signal acquisition unit 602 acquires the sound wave signals received by each of the audio acquisition modules, and the sound wave signals are emitted by the speaker of the mobile terminal. The first position determination unit 604 performs sound source localization based on the sound wave signals received by each of the audio acquisition modules to determine the position information of the loudspeaker; The second position determination unit 606 determines the position information of the mobile terminal based on the position information of the speaker.

[0099] Please refer to Figure 7 In some embodiments, the location determination device for a mobile terminal can be applied to, for example... Figure 5 The device shown implements the technical solution of this specification. The electronic device can be a mobile terminal, which is detachably installed on the desktop smart hardware product to cooperate with the desktop smart hardware product in completing corresponding tasks. The desktop smart hardware product includes multiple audio acquisition modules. The location determination device of the mobile terminal may include: The determining unit 702 determines the acoustic signal used to locate the mobile terminal; The sound wave transmitting unit 704 emits a sound wave signal for locating the mobile terminal through the speaker of the mobile terminal, so that the desktop smart hardware product determines the location information of the speaker based on the sound wave signal emitted by the speaker received by each of the audio acquisition modules, and determines the location information of the mobile terminal based on the location information of the speaker.

[0100] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another method, or some features may be ignored or not executed.

[0101] Based on the same concept as the above method, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor executes the executable instructions to implement the steps of the location determination method for a mobile terminal as described in any of the above embodiments.

[0102] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the location determination method for a mobile terminal as described in any of the above embodiments.

[0103] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the location determination method for a mobile terminal as described in any of the above embodiments.

[0104] What those skilled in the art will understand is: In this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded.

[0105] In this specification, “a,” “an,” and “the” do not specifically refer to the singular, but may also include the plural.

[0106] In this specification, ordinal numbers such as "first," "second," etc., do not necessarily indicate order; they are often used to distinguish between objects. For example, "first server" and "second server" usually refer to two servers. To differentiate between these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0107] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving the data sent by B, or it can be understood as A indirectly receiving the data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending the data directly to A, or it can be understood as B indirectly sending the data to A through other entities such as C. Here, C can be one entity, or it can be two or more entities.

[0108] In this specification, unless explicitly stated otherwise, the relationships between structures can be direct or indirect. For example, when describing "A is connected to B," unless it is explicitly stated that A and B are directly connected, it should be understood that A can be directly connected to B or indirectly connected to B. Similarly, when describing "A is on top of B," unless it is explicitly stated that A is directly above B (AB is adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements, and A is above B). And so on.

[0109] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0110] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

Claims

1. A method for determining the location of a mobile terminal, the method being applied to a desktop smart hardware product, the desktop smart hardware product including multiple audio acquisition modules; the mobile terminal being detachably installed to the desktop smart hardware product to cooperate with the desktop smart hardware product in completing corresponding tasks, the method comprising: The sound wave signals received by each of the audio acquisition modules are acquired, and the sound wave signals are emitted by the speaker of the mobile terminal. The sound source is located based on the sound wave signals received by each of the audio acquisition modules to determine the location information of the speaker; The location information of the mobile terminal is determined based on the location information of the speaker.

2. The method according to claim 1, wherein sound source localization is performed based on the sound wave signals received by each of the audio acquisition modules, comprising: The positioning assistance information sent by the mobile terminal is obtained, wherein the positioning assistance information represents information about the sound wave signal emitted by the speaker; Using the positioning assistance information as a unified signal processing basis, the sound wave signals received by each of the audio acquisition modules are processed with reference to the positioning assistance information. The processing results of the sound wave signals received by each of the audio acquisition modules are summarized to determine the location information of the speaker.

3. The method according to claim 2, wherein the acoustic signals received by each of the audio acquisition modules are processed with reference to the positioning assistance information, comprising: Based on the positioning assistance information and the sound wave signals received by each audio acquisition module, the target duration for the sound wave signal to propagate from the speaker to each audio acquisition module is determined respectively. The distance between each audio acquisition module and the speaker is determined based on the target duration and the propagation speed of the sound wave signal.

4. The method according to claim 3, wherein, based on the positioning assistance information and the sound wave signals received by each audio acquisition module, the target duration for the sound wave signal to propagate from the speaker to each audio acquisition module is determined, including: When the positioning assistance information includes the emission time of the sound wave signal emitted by the speaker, for each audio acquisition module, the receiving time corresponding to the sound wave signal received by the audio acquisition module is obtained, and the target duration required for the sound wave signal to propagate from the speaker to the audio acquisition module is determined based on the emission time and the receiving time corresponding to the audio acquisition module. or, When the sound wave signal emitted by the speaker is a linear frequency modulated continuous sound wave signal, and the positioning assistance information includes the sound wave signal emitted by the speaker, for each audio acquisition module, the sound wave signal received by the acquisition module is mixed with the sound wave signal emitted by the speaker included in the positioning assistance information. For the obtained mixed signal, the beat frequency corresponding to the low frequency part is determined, and the target duration required for the sound wave signal to propagate from the speaker to the audio acquisition module is determined according to the sweep bandwidth, period and beat frequency of the linear frequency modulated continuous sound wave signal.

5. The method according to claim 2, wherein the plurality of audio acquisition modules comprises at least three non-collinear audio acquisition modules; The processing results of the sound wave signals received by each of the aforementioned audio acquisition modules are summarized, including: Based on the distance between each of the three non-collinear audio acquisition modules and the speaker, two alternative position information for the speaker are determined; When the mobile terminal has been installed on the connection part of the desktop smart hardware product, the location information of the connection part is determined, and the candidate location information that is relatively closer to the connection part among the two candidate location information is determined as the location information of the speaker.

6. The method according to claim 1, wherein sound source localization is performed based on the sound wave signals received by each of the audio acquisition modules, comprising: Based on the differences in how sound waves are propagated from sound sources at different locations to the multiple audio acquisition modules, the sound waves received by the multiple audio acquisition modules are comprehensively calculated to obtain the location information of the loudspeaker.

7. The method according to claim 6, wherein the plurality of audio acquisition modules comprises at least three non-collinear audio acquisition modules; The sound wave signals received by the multiple audio acquisition modules are comprehensively calculated, including: Based on the spatial coordinate information of the three non-collinear audio acquisition modules and the signal reception time difference between each pair, a set of constraint equations for the position of the speaker is established. The position information of the loudspeaker is determined by the set of constraint equations.

8. The method according to claim 6, wherein the plurality of audio acquisition modules comprises at least three non-collinear audio acquisition modules; The sound wave signals received by the multiple audio acquisition modules are comprehensively calculated, including: When the mobile terminal has been installed on the connection part of the desktop smart hardware product, the spatial direction range of the speaker's position relative to a preset reference point is determined based on the position information of the connection part. Obtain the spatial coordinate information of each of the three non-collinear audio acquisition modules with the preset reference point as the origin; For each spatial direction within the spatial direction range, the signal strength of the three non-collinear audio acquisition modules in that spatial direction is determined by performing time delay compensation on the sound wave signal. The target spatial direction with the highest signal strength is determined from each spatial direction within the spatial direction range, and the position information of the loudspeaker is determined based on the target spatial direction.

9. The method according to claim 8, wherein determining the position information of the loudspeaker based on the target spatial direction includes: The position information of the plane where the speaker is located is determined based on the pose information of the connecting part; Based on the position information of the plane where the speaker is located and the target spatial direction, determine the position information of the target intersection point of the straight line corresponding to the plane where the speaker is located and the target spatial direction, and determine the position information of the target intersection point as the position information of the speaker.

10. The method according to claim 1, wherein determining the location information of the mobile terminal based on the location information of the speaker comprises: The location information of the speaker is determined as the location information of the mobile terminal; or, Obtain the dimensions and specifications of the mobile terminal, as well as the mounting position information of the speaker on the mobile terminal; The location information of the mobile terminal is determined based on the size specifications, the assembly position information, and the speaker position information.

11. The method according to claim 10, wherein when the number of speakers in the mobile terminal is less than or equal to two, determining the position information of the mobile terminal based on the size specifications, the assembly position information, and the position information of the speakers, includes: When the mobile terminal has been installed on the connection part of the desktop smart hardware product, the position and pose information of the connection part is determined; The position information of the plane where the mobile terminal is located is determined based on the pose information; Based on the size specifications, the speaker's location information, and the assembly location information, the location information of the mobile terminal is determined from the plane where the mobile terminal is located.

12. The method according to claim 11, wherein, when the mobile terminal includes a speaker, determining the position information of the mobile terminal from the plane where the mobile terminal is located based on the size specifications, the position information of the speaker, and the assembly position information, includes: Based on the pose information, determine the position information of the connection point where the mobile terminal contacts the connection part; Based on the size specifications, the speaker's position information, the assembly position information, and the connection point's position information, the position information of the mobile terminal is determined from the plane where the mobile terminal is located.

13. The method according to claim 10, wherein when the mobile terminal includes at least three non-collinear speakers, determining the position information of the mobile terminal based on the size specifications, the assembly position information, and the position information of the speakers, includes: Based on the size specifications, the position information of the three non-collinear speakers, and their respective assembly position information, the position information of the mobile terminal is determined.

14. The method according to claim 1, further comprising: If, based on the location information of the mobile terminal, it is determined that the distance between the mobile terminal and the connection part of the desktop smart hardware product is less than a preset distance, it is determined that the mobile terminal has been installed on the connection part; And / or, The anti-collision zone of the mobile terminal is determined based on the location information of the mobile terminal in order to avoid collisions during the movement of the mobile terminal in the desktop smart hardware product.

15. The method according to claim 1, wherein the frequency of the sound wave signal emitted by the speaker is determined by the frequency of the noise in the environment where the desktop smart hardware product is located, so that the frequency of the sound wave signal emitted by the speaker is different from the frequency of the noise.

16. A method for determining the location of a mobile terminal, the method being applied to the mobile terminal, the mobile terminal being detachably installed on a desktop smart hardware product to cooperate with the desktop smart hardware product in completing corresponding tasks, the desktop smart hardware product including multiple audio acquisition modules, the method comprising: Determine the acoustic signal used to locate the mobile terminal; The mobile terminal emits sound wave signals through its speaker to locate the mobile terminal, so that the desktop smart hardware product determines the location information of the speaker based on the sound wave signals emitted by the speaker received by each of the audio acquisition modules, and determines the location information of the mobile terminal based on the location information of the speaker.

17. The method according to claim 16, further comprising: Send positioning assistance information to the desktop smart hardware product; wherein the positioning assistance information represents information about the sound wave signal emitted by the speaker.

18. The method of claim 16, wherein determining the acoustic signal for locating the mobile terminal comprises: Obtain the frequency of the noise in the environment where the desktop smart hardware product is located; The frequency of the acoustic signal used to locate the mobile terminal is determined based on the frequency of the noise in the environment where the desktop smart hardware product is located, so that the frequency of the acoustic signal emitted by the speaker is different from the frequency of the noise.

19. An electronic device comprising: processor; Memory used to store processor-executable instructions; The processor executes the executable instructions to implement the method as described in any one of claims 1-16.

20. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1-18.

21. A computer program product, comprising: A computer program / instruction that, when executed by a processor, implements the method as described in any one of claims 1-18.

Images