Equipment collaboration method and device, electronic equipment and medium

Abstract

The invention relates to an equipment collaboration method and device, electronic equipment and a medium. The device cooperation method comprises the steps of receiving target audio data sent by a first electronic device; the target audio data is audio data acquired when the first electronic equipment plays the preset ultrasonic data; based on the target audio data, target collaborative operation is determined, and the target collaborative operation is used for assisting the first electronic equipment in processing related to the audio data; and executing the target collaborative operation. Cooperative configuration of the second electronic equipment and the first electronic equipment can be realized, and then the second electronic equipment is adopted to assist the first electronic equipment in processing related to the audio data, so that the configuration requirement, the working difficulty and the manufacturing cost of the first electronic equipment are reduced.

Description

Technical Field

[0001] This disclosure relates to the field of device collaboration technology, and in particular to a device collaboration method, apparatus, electronic device and medium. Background Technology

[0002] With the development of mobile internet and the Internet of Things, people's demand for a more convenient and intelligent lifestyle is constantly increasing, leading to the emergence of electronic devices that can sense users' usage status and behavioral patterns. These devices can effectively sense users' usage status and behavioral patterns, thereby responding to user needs, performing corresponding operations, and ultimately improving the user experience.

[0003] Current electronic devices typically operate in a standalone mode, meaning they complete the sensing process independently. However, this standalone mode requires high-performance electronic devices. Consequently, current electronic devices have high configuration requirements, are complex to operate, and are costly to manufacture. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a device coordination method, apparatus, electronic device, and medium.

[0005] According to a first aspect of the present disclosure, a device collaboration method is provided, comprising:

[0006] Receive target audio data sent by a first electronic device; the target audio data is audio data collected when the first electronic device plays preset ultrasound data;

[0007] Based on the target audio data, a target cooperative operation is determined, which is used to assist the first electronic device in performing audio data-related processing.

[0008] Perform the target cooperative operation.

[0009] In some embodiments, determining the target cooperative operation based on the target audio data includes:

[0010] Determine the evaluation parameters of the target audio data; the evaluation parameters are used to characterize the sound acquisition capability of the first audio acquisition module, which is the audio acquisition module of the first electronic device;

[0011] Based on the evaluation parameters, the target cooperative operation is determined.

[0012] In some embodiments, determining the target cooperative operation based on the evaluation parameters includes:

[0013] When the evaluation parameter is less than the first preset threshold, the target collaborative operation is determined to be the first collaborative operation; the first collaborative operation represents the use of the second audio acquisition module to acquire audio data, and the second audio acquisition module is the audio acquisition module of the second electronic device.

[0014] In some embodiments, the method further includes:

[0015] When the evaluation parameter is greater than or equal to the first preset threshold and less than the second preset threshold, a prompt message is sent to the first electronic device; the prompt message is used to indicate the sound reception capability of the first audio acquisition module.

[0016] In some embodiments, determining the target cooperative operation based on the target audio data includes:

[0017] The target audio data is identified to obtain a first identification result of the target audio data; the first identification result is used to characterize whether the target audio data includes the posture data of the target object, and the posture data is used to characterize the posture of the target object.

[0018] Based on the first identification result, the target cooperative operation is determined.

[0019] In some embodiments, determining the target cooperative operation based on the first identification result includes:

[0020] When the first recognition result indicates that the target audio data includes the posture data of the target object, the target cooperative operation is determined to be a second cooperative operation, which is used to characterize the operation indicated by the posture data.

[0021] In some embodiments, the method further includes:

[0022] Determine the posture type corresponding to the posture data;

[0023] Perform the operation corresponding to the gesture type.

[0024] According to a second aspect of the present disclosure, a device collaboration method is provided, comprising:

[0025] Collect target audio data; the target audio data is the audio data collected by the first electronic device when playing preset ultrasound data;

[0026] The target audio data is sent to the second electronic device. The target audio data is used to determine a target cooperative operation. The target cooperative operation is a cooperative operation to be executed by the second electronic device. The target cooperative operation is used to assist the first electronic device in performing processing related to the audio data.

[0027] In some embodiments, sending the target audio data to the second electronic device includes:

[0028] When the first electronic device activates the first preset function, it sends the target audio data to the second electronic device; the first preset function is a function for detecting the sound reception effect of the first audio acquisition module, and the first audio acquisition module is the audio acquisition module of the first electronic device.

[0029] In some embodiments, the method further includes:

[0030] The system receives a prompt message sent by the second electronic device; the prompt message is used to indicate the sound reception capability of the first audio acquisition module.

[0031] Based on the prompt information, the sound reception capability of the first audio acquisition module is adjusted.

[0032] In some embodiments, sending the target audio data to the second electronic device includes:

[0033] When the first electronic device activates the second preset function, it identifies the target audio data and obtains a second identification result of the target audio data; the second preset function is a function for identifying the posture of the target object, and the second identification result is used to characterize whether the target audio data includes the posture data of the target object, and the posture data is used to characterize the posture of the target object.

[0034] When the second recognition result indicates that the target audio data includes the posture data of the target object, the target audio data is sent to the second electronic device.

[0035] According to a third aspect of the present disclosure, a device collaboration apparatus is provided, comprising:

[0036] The receiving module is configured to receive target audio data sent by a first electronic device; the target audio data is audio data collected when the first electronic device plays preset ultrasound data;

[0037] The determination module is configured to determine a target cooperative operation based on the target audio data, the target cooperative operation being used to assist the first electronic device in performing audio data-related processing;

[0038] The processing module is configured to perform the target cooperative operation.

[0039] According to a fourth aspect of the present disclosure, a device collaboration apparatus is provided, comprising:

[0040] The acquisition module is configured to acquire target audio data; the target audio data is the audio data acquired by the first electronic device when playing preset ultrasound data;

[0041] The sending module is configured to send the target audio data to a second electronic device. The target audio data is used to determine a target collaborative operation, which is a collaborative operation to be executed by the second electronic device. The target collaborative operation is used to assist the first electronic device in performing processing related to the audio data.

[0042] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising:

[0043] processor;

[0044] Memory used to store processor-executable instructions;

[0045] The processor is configured to perform the device coordination method as described in the first or second aspect of this disclosure.

[0046] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform a device coordination method as described in the first or second aspect of the present disclosure.

[0047] The above-described method of this disclosure has the following beneficial effects: This disclosure can realize the collaborative configuration of the second electronic device and the first electronic device, and then use the second electronic device to assist the first electronic device in processing related to audio data, thereby reducing the configuration requirements, working difficulty and manufacturing cost of the first electronic device.

[0048] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0050] Figure 1 This is a flowchart illustrating a device collaboration method according to an exemplary embodiment.

[0051] Figure 2 This is a flowchart illustrating a device collaboration method according to an exemplary embodiment.

[0052] Figure 3 This is a flowchart illustrating a device collaboration method according to an exemplary embodiment.

[0053] Figure 4 This is a flowchart illustrating a device collaboration method according to an exemplary embodiment.

[0054] Figure 5 This is a flowchart illustrating a device collaboration method according to an exemplary embodiment.

[0055] Figure 6 This is a flowchart illustrating a device collaboration method according to an exemplary embodiment.

[0056] Figure 7 This is a block diagram illustrating a device collaboration device according to an exemplary embodiment.

[0057] Figure 8 This is a block diagram illustrating a device collaboration device according to an exemplary embodiment.

[0058] Figure 9 This is a block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0059] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0060] With the development of mobile internet and the Internet of Things (IoT), people's demand for a more convenient and intelligent lifestyle is constantly increasing, leading to the emergence of electronic devices that can sense users' usage status and behavioral patterns. These devices can effectively sense users' usage status and behavioral patterns, thereby responding to user needs and performing corresponding operations, ultimately improving the user experience. Among these devices are smart wearable devices, such as audio glasses, which transmit audio information to the wearer's ears, allowing for easier and more convenient access to audio services.

[0061] Current electronic devices typically operate in a standalone mode, meaning they complete the sensing process independently. However, this standalone mode requires high-performance electronic devices. Consequently, current electronic devices have high configuration requirements, are complex to operate, and are costly to manufacture.

[0062] To address the aforementioned problems, this disclosure provides a device collaboration method, comprising: receiving target audio data sent by a first electronic device; the target audio data being audio data collected by the first electronic device while playing preset ultrasound data; determining a target collaborative operation based on the target audio data; the target collaborative operation being used to assist the first electronic device in performing audio data-related processing; and executing the target collaborative operation. This disclosure enables collaborative configuration between a second electronic device and a first electronic device, thereby allowing the second electronic device to assist the first electronic device in performing audio data-related processing, thus reducing the configuration requirements, operational complexity, and manufacturing cost of the first electronic device.

[0063] The device collaboration method provided in this embodiment is executed by a second electronic device, which may specifically be a mobile phone, tablet computer, laptop, smart robot, smart wearable device, or other smart device. Furthermore, the second electronic device is also equipped with various hardware resources and an energy storage device that provides power for the operation of these hardware resources.

[0064] Figure 1 This is a flowchart illustrating a device collaboration method according to an exemplary embodiment, executed by a second electronic device, see [link to flowchart]. Figure 1 The method includes the following steps:

[0065] Step S101: Receive target audio data sent by the first electronic device; the target audio data is the audio data collected when the first electronic device plays preset ultrasound data.

[0066] The target audio data is represented as a digital signal, and the preset ultrasound data parameters can be set and selected based on actual needs. For example, the preset ultrasound data can be ultrasound waves with preset frequency and preset duration.

[0067] In some embodiments, after the first electronic device and the second electronic device are started, a communication connection can be established between the first electronic device and the second electronic device to facilitate subsequent data transmission between them. This embodiment does not limit the specific form of the communication connection; for example, the communication connection can be a wireless communication connection or a wired communication connection.

[0068] In some embodiments, the ADSP (Audio Digital Signal Processor) in the second electronic device can open an uplink path and receive target audio data sent by the first electronic device. It should be noted that the uplink path of the ADSP can be used to process and transmit audio signals; for example, the uplink path of the ADSP can receive and process audio signals from external devices.

[0069] Optionally, after receiving the target audio data, the ADSP can perform a series of processes on the received target audio data to improve the data quality and make the target audio data easier to transmit, store, and process. For example, the ADSP can perform filtering, amplification, compression, or encoding on the target audio data.

[0070] Step S102: Based on the target audio data, determine the target cooperative operation, which is used to assist the first electronic device in performing audio data-related processing.

[0071] In some embodiments, the target cooperative operation can be a processing related to audio data. For example, the target cooperative operation can be an audio data acquisition operation, an audio data switching operation, or an audio data start playback, pause playback, or other operations.

[0072] In some embodiments, the target audio data can be analyzed and processed to determine the target cooperative operation. For example, after receiving the target audio data, the ADSP can send the target audio data to the analysis and processing module, which can then analyze and process the target audio data to obtain the target cooperative operation.

[0073] Step S103: Perform the target collaborative operation.

[0074] In some embodiments, the second electronic device may generate instructions corresponding to the target cooperative operation based on the target cooperative operation, and execute the instructions corresponding to the target cooperative operation. For example, the processor of the second electronic device may generate instructions corresponding to the target cooperative operation.

[0075] In this embodiment, the second electronic device and the first electronic device can be configured collaboratively. The second electronic device assists the first electronic device in processing audio data, thereby reducing the configuration requirements, operational complexity, and manufacturing cost of the first electronic device. Furthermore, using ultrasonic data allows for seamless sensing without affecting normal user operation, improving the user experience.

[0076] In some embodiments, the target cooperative operation includes a first cooperative operation and a second cooperative operation, wherein the first cooperative operation represents the acquisition of audio data using a second audio acquisition module, which is the audio acquisition module of a second electronic device; the second cooperative operation represents the execution of the operation indicated by the gesture data. The following describes... Figure 2 The illustrated embodiment explains the process of determining the first cooperative operation, and through Figure 3 The illustrated embodiment explains the process of determining the second cooperative operation.

[0077] Figure 2This is a flowchart illustrating a device collaboration method according to an exemplary embodiment, executed by a second electronic device, see [link to flowchart]. Figure 2 The method includes the following steps:

[0078] Step S201: Determine the evaluation parameters of the target audio data; the evaluation parameters are used to characterize the sound acquisition capability of the first audio acquisition module, which is the audio acquisition module of the first electronic device.

[0079] In some embodiments, the target audio data can be processed and calculated to obtain evaluation parameters for the target audio data.

[0080] Optionally, the target audio data can be processed and calculated using an algorithm module to obtain evaluation parameters for the target audio data. This algorithm module includes a pre-trained evaluation algorithm, which is trained using multiple audio data sets and the corresponding evaluation parameters for each audio data set. Thus, by inputting the target audio data into the pre-trained evaluation algorithm, the evaluation parameters corresponding to the target audio data can be obtained.

[0081] Optionally, the target audio data can be compared with preset ultrasound data to obtain evaluation parameters for the target audio data. For example, by comparing the sound waveforms of the target audio data and the preset ultrasound data, and analyzing the similarity between the sound waveforms of the target audio data and the preset ultrasound data, evaluation parameters for the target audio data can be obtained.

[0082] In some embodiments, the first audio acquisition module may be a sound sensor such as a microphone, which may be a built-in microphone or a microphone extended from an external device.

[0083] Step S202: Based on the evaluation parameters, determine the target cooperative operation.

[0084] In some embodiments, the sound pickup capability of the first audio acquisition module can be determined based on evaluation parameters and multiple preset thresholds. The sound pickup capability of the first audio acquisition module is determined below under three conditions based on a first preset threshold, a second preset threshold, and evaluation parameters:

[0085] In the first scenario, when the evaluation parameter is less than the first preset threshold, it indicates that the first audio acquisition module has very poor sound pickup capability. Poor sound pickup capability may be caused by severe microphone blockage or significant functional problems, such as serious damage to the physical components of the first audio acquisition module. It should be noted that if the audio data acquisition process is in a call or recording state, the audio data acquired by the first audio acquisition module may not effectively represent the actual audio data. Similarly, the audio data fed back to the call or recording device may be problematic; for example, the call device may not be able to clearly hear the target's audio data, or the recording device may not have recorded the target's audio data. Therefore, when the evaluation parameter is less than the first preset threshold, the poor sound pickup capability of the first audio acquisition module will lead to problems such as choppy calls or recordings, resulting in a poor user experience.

[0086] In some embodiments, when the evaluation parameter is less than a first preset threshold, the target cooperative operation can be determined as a first cooperative operation. The first cooperative operation represents the use of a second audio acquisition module to acquire audio data; the second audio acquisition module is the audio acquisition module of a second electronic device. That is, when the evaluation parameter is less than the first preset threshold, the second audio acquisition module of the second electronic device can be used to assist the first electronic device in acquiring audio data, thereby solving the problem that the audio data acquired by the first audio acquisition module cannot effectively represent the true audio data, or that the audio data fed back by the first audio acquisition module to the call device or recording device is inaccurate.

[0087] Optionally, in the first scenario, after determining that the evaluation parameter is less than a first preset threshold, the second electronic device can send feedback information to the first electronic device (the feedback information can be displayed on the display interface of the first electronic device; or, if the first electronic device does not have a display interface, the feedback information can be displayed on the display interface of another electronic device connected to the first electronic device, such as the display interface of the second electronic device). In response to the operation on the feedback information, the target collaborative operation is determined as the first collaborative operation. The feedback information can be displayed on the display device in the form of a pop-up window, a prompt box, etc. The feedback information can be used to indicate the sound reception capability of the first audio acquisition module, and whether to use the second audio acquisition module of the second electronic device to collect audio data based on the sound reception capability of the first audio acquisition module. When it is determined that the second audio acquisition module of the second electronic device will be used to collect audio data, the target collaborative operation is determined as the first collaborative operation.

[0088] The following is a specific example to illustrate the feedback information:

[0089] For example, feedback information can be displayed as a pop-up on the display interface of the first electronic device. The pop-up includes "The sound reception capability of the first audio acquisition module of the current electronic device is very poor (prompting on the sound reception capability of the first audio acquisition module), please determine whether to use the second audio acquisition module of the second electronic device to collect audio data", as well as "Yes" and "No" buttons (regarding the sound reception capability of the first audio acquisition module, whether to use the second audio acquisition module of the second electronic device to collect audio data). If the user clicks "Yes", the second audio acquisition module of the second electronic device will be used to assist the first electronic device in collecting audio data, and the target collaborative operation will be determined as the first collaborative operation; otherwise, if the user clicks "No", the second audio acquisition module of the second electronic device will not be used to assist the first electronic device in collecting audio data, and the target collaborative operation will be determined as a non-first collaborative operation.

[0090] Optionally, not using the second audio acquisition module of the second electronic device to assist the first electronic device in acquiring audio data includes: using the audio acquisition module of another electronic device that is communicatively connected to the first electronic device to acquire audio data. For example, if the first electronic device has a communication connection with a third electronic device in addition to the second electronic device, then the audio acquisition module of the third electronic device can be used to assist the first electronic device in acquiring audio data, or the first audio acquisition module of the first electronic device can continue to be used to acquire audio data.

[0091] In this step, when the sound pickup capability of the first audio acquisition module is poor, the second audio acquisition module of the second electronic device is used to assist the first electronic device in acquiring audio data. This can effectively acquire real audio data and feed it back to the call device and the recording device, thereby improving the authenticity and accuracy of the acquired audio data and thus enhancing the user experience.

[0092] In the second scenario, when the evaluation parameter is greater than or equal to the first preset threshold and less than the second preset threshold, it indicates that the first audio acquisition module has poor sound pickup capability. It should be noted that the reasons for poor sound pickup capability of the first audio acquisition module may include: the user posture of the first electronic device, slight microphone blockage caused by the wearing posture of the first electronic device, or minor functional problems with the first audio acquisition module. In this case, the second electronic device can send a prompt message to the first electronic device. This prompt message can be used to indicate the sound pickup capability of the first audio acquisition module, so that the first electronic device can adjust its sound pickup capability accordingly. The prompt message can be displayed on the display interface of the first electronic device; or, if the first electronic device does not have a display interface, the prompt message can be displayed on the display interface of another electronic device connected to the first electronic device, such as the display interface of the second electronic device. The prompt message in this scenario is similar to the feedback message in the first scenario; the relevant content of the prompt message can be referred to the feedback message in the previous scenario, and will not be repeated here.

[0093] In some embodiments, after the first audio acquisition module has completed the adjustment of its sound reception capability, the second electronic device can receive the target audio data again and determine the evaluation parameters of the target audio data again. If the evaluation parameters are greater than or equal to the first preset threshold and less than the second preset threshold, or if the evaluation parameters are less than the first preset threshold, it can be determined that the adjustment of the sound reception capability of the first audio acquisition module has failed. In this case, the second audio acquisition module and the first audio acquisition module can be used to acquire audio data simultaneously, or the second audio acquisition module can be used to acquire audio data.

[0094] In the third scenario, when the evaluation parameter exceeds the first preset threshold, it indicates that the first audio acquisition module has good sound reception capability, and the first audio acquisition module is sufficient to meet the sound reception requirements. Accordingly, in this case, no processing is required on the audio acquisition modules of the first and second electronic devices; the first audio acquisition module can continue to collect audio data. Optionally, when the evaluation parameter exceeds the first preset threshold, the second electronic device will feed back the evaluation parameter to the first electronic device.

[0095] This embodiment of the disclosure can determine the sound reception capability of the first audio acquisition module of the first electronic device through ultrasonic data, thereby determining the working status of the first audio acquisition module during the recording or call process. When the sound reception capability of the first audio acquisition module is very poor, the second audio acquisition module is used to acquire audio data. In this way, it can always ensure that real audio data is acquired during the recording or call process, so that the user on the call device end and the recording device end receive effective audio data, thereby improving the user experience.

[0096] Figure 3This is a flowchart illustrating a device collaboration method according to an exemplary embodiment, executed by a second electronic device, see [link to flowchart]. Figure 3 The method includes the following steps:

[0097] Step S301: Identify the target audio data to obtain a first identification result of the target audio data; the first identification result is used to characterize whether the target audio data includes the pose data of the target object, and the pose data is used to characterize the pose of the target object.

[0098] Optionally, the target audio data can be identified through an algorithm module to obtain a first recognition result. This algorithm module includes a preset recognition algorithm, which is trained using multiple audio data sets and the recognition results corresponding to each audio data set. For example, the training data includes audio data A and its corresponding recognition result (including posture data), and audio data B and its corresponding recognition result (excluding posture data).

[0099] When the target audio data includes the posture data of the target object, due to the diversity of the target object's posture data, the audio features of the target audio data may have multiple forms. For example, when the target object's posture data is posture data 'a', the audio feature of the target audio data is 'a1'; when the target object's posture data is posture data 'b', the audio feature of the target audio data is 'b1', and 'a1' is different from 'b1'. Optionally, the target audio data can be compared with preset audio data. When the audio features of the target audio data and the preset audio data are different, the first recognition result of the target audio data is determined to be that the target audio data includes the posture data of the target object; and when the audio features of the target audio data and the preset audio data are the same, the first recognition result of the target audio data is determined to be that the target audio data does not include the posture data of the target object. The preset audio data is at least one audio data that is pre-set and does not include the posture data of the target object.

[0100] Optionally, frequency variations in the ultrasonic data within the target audio data can be analyzed to determine the target object's posture data.

[0101] Step S302: Based on the first identification result, determine the target cooperative operation.

[0102] In some embodiments, when the first recognition result characterizes the target audio data to include the pose data of the target object, the target cooperative operation is determined to be a second cooperative operation, which is used to characterize the operation indicated by the pose data.

[0103] In some embodiments, when the first recognition result characterizes the target audio data as including the pose data of the target object, the pose type of the pose data can also be determined, and an operation corresponding to the pose type can be performed. Performing the operation corresponding to the pose type can include determining an instruction for the operation corresponding to the pose type and executing that instruction. For example, the ADSP module sends the pose type corresponding to the pose data to a processor in a second electronic device, and the processor executes the instruction for the operation corresponding to the pose type.

[0104] In some embodiments, the gesture type can be either the target object's gesture type or the target object's body posture type. For example, the gesture type can be a thumbs-up, thumbs-down, fist, open palm, thumb to the left, thumb to the right, etc., and the body posture type can be a raised hand, head up, etc. Optionally, the second electronic device has a pre-defined correspondence between gesture types and operations to be performed. That is, the second electronic device includes gesture types and corresponding operations. For example, the operation corresponding to a thumbs-up gesture is to increase the volume, the operation corresponding to a thumbs-down gesture is to decrease the volume, the operation corresponding to a fist gesture is to pause playback, the operation corresponding to an open palm gesture is to start playback, the operation corresponding to a thumb to the left gesture is to play the previous audio data, and the operation corresponding to a thumb to the right gesture is to play the next audio data. In one example, when the target object's gesture data has a thumbs-up gesture, the second electronic device can perform a volume increase operation or execute the instruction corresponding to the volume increase operation.

[0105] Currently, when users want to adjust the volume or switch audio data, they typically use methods like rotating or clicking buttons. Due to the small size of electronic devices, the buttons on them are also small, making these operations difficult and time-consuming, resulting in a poor user experience. Furthermore, existing electronic device sensing solutions usually require additional physical components to perceive user posture data, such as image sensors or radar. This not only occupies space but also increases the weight of the device, making it less portable and comfortable. However, the embodiments of this disclosure can identify the posture data of a target object using ultrasonic data and then execute the operations indicated by the posture data. This eliminates the need for touch buttons, simplifying the operation process, saving users time and effort, and also eliminates the need for additional physical components, optimizing the user experience of the first electronic device. Simultaneously, the coordinated configuration of the first and second electronic devices can reduce the configuration requirements of the first electronic device.

[0106] The device collaboration method provided in this embodiment is executed by a first electronic device, which may be a mobile phone, tablet computer, laptop, smart robot, smart wearable device, or other smart device. Furthermore, the first electronic device is also equipped with various hardware resources and an energy storage device that provides power for the operation of these hardware resources.

[0107] Figure 4 This is a flowchart illustrating a device collaboration method according to an exemplary embodiment, executed by a first electronic device, see [link to flowchart]. Figure 4 The method includes the following steps:

[0108] Step S401: Collect target audio data; the target audio data is the audio data collected by the first electronic device when playing preset ultrasound data.

[0109] The preset ultrasound data can be played through the audio playback device of the first electronic device, which can refer to the speaker of the first electronic device. Furthermore, the data parameters of the preset ultrasound data can be set based on actual needs, which will not be elaborated here.

[0110] In some embodiments, when the first electronic device plays preset ultrasound data, the first audio acquisition module in the first electronic device acquires target audio data. It should be noted that the duration of the target audio data can be the same as the duration of the preset ultrasound data.

[0111] Step S402: Send target audio data to the second electronic device. The target audio data is used to determine the target cooperative operation. The target cooperative operation is the cooperative operation to be executed by the second electronic device. The target cooperative operation is used to assist the first electronic device in performing processing related to the audio data.

[0112] In some embodiments, the first electronic device sends target audio data to the ADSP in the second electronic device, and correspondingly, the ADSP in the second electronic device opens the uplink path and receives the target audio data sent by the first electronic device.

[0113] It should be noted that the target audio data can be converted in format before being sent to the second electronic device. In one example, the target audio data can be converted from an analog signal to a digital signal and then sent to the second electronic device as a digital signal. This embodiment does not elaborate on the process of converting analog signals to digital signals. Additionally, noise reduction processing can be performed on the target audio data before sending it to the second electronic device to reduce the impact of noise on the audio data analysis and processing, resulting in more accurate analysis and processing results.

[0114] In some embodiments, sending target audio data to a second electronic device falls into two categories:

[0115] The first scenario: When the first electronic device activates the first preset function, it sends target audio data to the second electronic device. The first preset function is used to detect the sound reception effect of the first audio acquisition module, which is the audio acquisition module of the first electronic device.

[0116] In some embodiments, when the first electronic device activates the first preset function and the first preset function is in a triggered state, the first electronic device collects target audio data and directly sends the target audio data to the second electronic device. Optionally, the first preset function can be triggered synchronously each time the first electronic device's radio function is activated. For example, the first preset function is in a triggered state when the first electronic device has just entered a radio reception scenario.

[0117] Optionally, when the first electronic device is in a recording scenario, the first preset function is triggered every second preset time interval. For example, the first electronic device enters a recording scenario and triggers the first preset function every 10 seconds. The recording scenario can include telephone calls, voice calls, video calls, using a recorder, using a camera to record audio, etc.

[0118] The above method can be used to determine whether the first audio acquisition module of the first electronic device is in a normal state. Optionally, if it is determined that the first audio acquisition module of the first electronic device has good sound reception capability and is in a normal state, the first preset function can be turned off until the sound reception function is turned on or the first electronic device is in a sound reception scenario.

[0119] The second scenario: When the first electronic device activates the second preset function, it identifies the target audio data and obtains a second identification result. If the second identification result indicates that the target audio data includes the posture data of the target object, the target audio data is sent to the second electronic device. The second preset function is used to identify the posture of the target object, and the second identification result indicates whether the target audio data includes the posture data of the target object. The posture data represents the posture of the target object. In other words, the first electronic device first identifies the target audio data, and only when the second identification result indicates that the target audio data includes the posture data of the target object does it send the target audio data to the second electronic device. Through the initial identification by the first electronic device, target audio data that may include the posture data of the target object can be sent to the second electronic device, thereby reducing unnecessary data transmission, improving the identification effect of the target audio data, and reducing the power consumption of both the first and second electronic devices.

[0120] In some embodiments, the process of the first electronic device recognizing the target audio data and obtaining the second recognition result of the target audio data can refer to the first recognition result of the target audio data in step S302 above, and will not be repeated here.

[0121] In some embodiments, the first electronic device can receive a prompt message sent by the second electronic device and adjust the sound reception capability of the first audio acquisition module based on the prompt message. The prompt message is used to indicate the sound reception capability of the first audio acquisition module.

[0122] Optionally, adjusting the sound pickup capability of the first audio acquisition module may include displaying the sound pickup capability of the first audio acquisition module on a display interface for easy viewing by the target user. In this way, the target user can adjust the sound pickup capability of the first audio acquisition module. For example, the target user can adjust the posture in which the first electronic device is used (when the first electronic device is headphones, the target user can adjust the wearing posture of the first electronic device).

[0123] Optionally, adjusting the sound pickup capability of the first audio acquisition module may include detecting potential problems with the first audio acquisition module and automatically repairing it. If the automatic repair fails, the target object is notified to repair the first audio acquisition module.

[0124] The embodiments disclosed herein enable the first electronic device and the second electronic device to be configured collaboratively, thereby allowing the second electronic device to assist the first electronic device in performing data processing, reducing the configuration requirements and manufacturing costs of the first electronic device, making the first electronic device lighter and more compact, and improving the user experience.

[0125] The interaction process between the first electronic device and the second electronic device will be explained below, based on the cases where the first electronic device activates the first preset function and the cases where the first electronic device activates the second preset function.

[0126] Figure 5 This is a flowchart illustrating a device collaboration method according to an exemplary embodiment, executed by a first electronic device and a second electronic device, see [link to flowchart]. Figure 5 The method includes the following steps:

[0127] In step S501, the first electronic device determines whether the first preset function is in a triggered state. If yes, then step S502 is executed; otherwise, step S501 is executed.

[0128] In step S502, the first electronic device plays preset ultrasound data.

[0129] Step S503: The first electronic device acquires the target audio data.

[0130] In step S504, the first electronic device sends the target audio data to the second electronic device.

[0131] In step S505, the second electronic device receives the target audio data and determines the evaluation parameters of the target audio data.

[0132] Based on the evaluation parameters and preset thresholds, there are three possible scenarios: the evaluation parameter is less than the first threshold; the evaluation parameter is greater than or equal to the first preset threshold and less than the second preset threshold; and the evaluation parameter is greater than the second preset threshold. When the evaluation parameter is less than the first threshold, proceed to step S506; when the evaluation parameter is greater than or equal to the first preset threshold and less than the second preset threshold, proceed to step S508; and when the evaluation parameter is greater than the second preset threshold, proceed to step S509.

[0133] In step S506, the second electronic device determines the target cooperative operation as the first cooperative operation.

[0134] In step S507, the second electronic device performs the first cooperative operation.

[0135] In step S508, the second electronic device sends a prompt message to the first electronic device.

[0136] In step S509, the second electronic device feeds back the evaluation parameters to the first electronic device.

[0137] In step S510, the first electronic device receives the information sent by the second electronic device and performs corresponding processing.

[0138] In step S511, the first electronic device determines whether the process has ended. If not, step S501 is executed; if yes, the interaction process between the first and second electronic devices ends.

[0139] In step S510, corresponding to step S508, the first electronic device can receive a prompt message and adjust the sound reception capability of the first audio acquisition module; corresponding to step S509, the first electronic device can receive evaluation parameters and use the first audio acquisition module to acquire audio data.

[0140] In addition, after step S507, the second electronic device can send feedback information to the first electronic device. The feedback information is used to indicate that the first electronic device is currently using the second audio acquisition module to acquire audio data.

[0141] Figure 6 This is a flowchart illustrating a device collaboration method according to an exemplary embodiment, executed by a first electronic device and a second electronic device, see [link to flowchart]. Figure 6The method includes the following steps:

[0142] In step S601, the first electronic device determines whether the second preset function is enabled. If yes, proceed to step S602; otherwise, proceed to step S601.

[0143] In step S602, the first electronic device plays preset ultrasound data.

[0144] Step S603: The first electronic device acquires the target audio data.

[0145] In step S604, the first electronic device identifies the target audio data and obtains a second identification result of the target audio data.

[0146] In step S605, the first electronic device determines, based on the second recognition result, whether the target audio data includes the pose data of the target object. If yes, proceed to step S606; otherwise, proceed to step S612.

[0147] In step S606, the first electronic device sends the target audio data to the second electronic device.

[0148] In step S607, the second electronic device receives the target audio data sent by the first electronic device.

[0149] In step S608, the second electronic device identifies the target audio data and obtains the first identification result of the target audio data.

[0150] In step S609, the second electronic device determines, based on the first recognition result, whether the target audio data includes the pose data of the target object. If yes, proceed to step S610; otherwise, proceed to step S612.

[0151] In step S610, the second electronic device determines the target cooperative operation as the second cooperative operation.

[0152] In step S611, the second electronic device determines the posture type corresponding to the posture data and performs the operation corresponding to the posture type.

[0153] In step S612, the first electronic device determines whether the process has ended. If not, step S601 is executed; if yes, the interaction process between the first and second electronic devices ends.

[0154] Figure 7 This is a block diagram illustrating a device collaboration device configured in a first electronic device, according to an exemplary embodiment. See also... Figure 7 The device includes:

[0155] The receiving module 701 is configured to receive target audio data sent by the first electronic device; the target audio data is the audio data collected when the first electronic device plays preset ultrasound data;

[0156] The determination module 702 is configured to determine a target cooperative operation based on the target audio data. The target cooperative operation is used to assist the first electronic device in performing audio data-related processing.

[0157] Processing module 703 is configured to perform target cooperative operations.

[0158] In some embodiments, the determining module 702 is configured to:

[0159] Determine the evaluation parameters for the target audio data; the evaluation parameters are used to characterize the sound acquisition capability of the first audio acquisition module, which is the audio acquisition module of the first electronic device;

[0160] Based on the evaluation parameters, the target collaborative operations are determined.

[0161] In some embodiments, the determining module 702 is configured to:

[0162] When the evaluation parameter is less than the first preset threshold, the target collaborative operation is determined to be the first collaborative operation; the first collaborative operation is characterized by the use of the second audio acquisition module to acquire audio data, and the second audio acquisition module is the audio acquisition module of the second electronic device.

[0163] In some embodiments, the determining module 702 is configured to:

[0164] When the evaluation parameter is greater than or equal to the first preset threshold and less than the second preset threshold, a prompt message is sent to the first electronic device; the prompt message is used to indicate the sound reception capability of the first audio acquisition module.

[0165] In some embodiments, the determining module 702 is configured to:

[0166] The target audio data is identified to obtain a first identification result of the target audio data; the first identification result is used to characterize whether the target audio data includes the pose data of the target object, and the pose data is used to characterize the pose of the target object.

[0167] Based on the first identification result, the target collaborative operation is determined.

[0168] In some embodiments, the determining module 702 is configured to:

[0169] When the first recognition result indicates that the target audio data includes the pose data of the target object, the target cooperative operation is determined to be the second cooperative operation, which is used to characterize the operation indicated by the pose data.

[0170] In some embodiments, the processing module 703 is configured to:

[0171] Determine the pose type corresponding to the pose data;

[0172] Perform the operation corresponding to the gesture type.

[0173] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0174] Figure 8 This is a block diagram illustrating a device collaboration device according to an exemplary embodiment, configured in a second electronic device, see [link to example]. Figure 8 The device includes:

[0175] The acquisition module 801 is configured to acquire target audio data; the target audio data is the audio data acquired by the first electronic device when playing preset ultrasound data;

[0176] The sending module 802 is configured to send target audio data to the second electronic device. The target audio data is used to determine the target cooperative operation, which is the cooperative operation to be executed by the second electronic device. The target cooperative operation is used to assist the first electronic device in performing processing related to the audio data.

[0177] In some embodiments, the sending module 802 is configured to:

[0178] When the first electronic device activates the first preset function, it sends target audio data to the second electronic device; the first preset function is a function used to detect the sound reception effect of the first audio acquisition module, and the first audio acquisition module is the audio acquisition module of the first electronic device.

[0179] In some embodiments, the sending module 802 is configured to:

[0180] Receive prompt information sent by the second electronic device; the prompt information is used to indicate the sound reception capability of the first audio acquisition module;

[0181] Based on the prompt information, the sound reception capability of the first audio acquisition module is adjusted.

[0182] In some embodiments, the sending module 802 is configured to:

[0183] When the first electronic device activates the second preset function, it identifies the target audio data and obtains the second identification result of the target audio data. The second preset function is a function for identifying the posture of the target object. The second identification result is used to characterize whether the target audio data includes the posture data of the target object. The posture data is used to characterize the posture of the target object.

[0184] When the second recognition result indicates that the target audio data includes the posture data of the target object, the target audio data is sent to the second electronic device.

[0185] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0186] This disclosure also provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the device coordination method described above.

[0187] Figure 9 This is a block diagram of an electronic device 900 according to an exemplary embodiment.

[0188] Reference Figure 9 The electronic device 900 may include one or more of the following components: a processing component 902, a memory 904, a power supply component 906, a multimedia component 908, an audio component 910, an input / output (I / O) interface 912, a sensor component 914, and a communication component 916.

[0189] Processing component 902 typically controls the overall operation of electronic device 900, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 902 may include one or more processors 920 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 902 may include one or more modules to facilitate interaction between processing component 902 and other components. For example, processing component 902 may include a multimedia module to facilitate interaction between multimedia component 908 and processing component 902.

[0190] Memory 904 is configured to store various types of data to support the operation of electronic device 900. Examples of this data include instructions for any application or method operating on electronic device 900, contact data, phonebook data, messages, pictures, videos, etc. Memory 904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0191] Power supply component 906 provides power to various components of electronic device 900. Power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 900.

[0192] Multimedia component 908 includes a screen that provides an output interface between the electronic device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 908 includes a front-facing camera and / or a rear-facing camera. When the electronic device 900 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0193] Audio component 910 is configured to output and / or input audio signals. For example, audio component 910 includes a microphone (MIC) configured to receive external audio signals when electronic device 900 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 904 or transmitted via communication component 916. In some embodiments, audio component 910 also includes a speaker for outputting audio signals.

[0194] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0195] Sensor assembly 914 includes one or more sensors for providing state assessments of various aspects of electronic device 900. For example, sensor assembly 914 can detect the on / off state of electronic device 900, the relative positioning of components such as the display and keypad of electronic device 900, changes in position of electronic device 900 or a component of electronic device 900, the presence or absence of user contact with electronic device 900, orientation or acceleration / deceleration of electronic device 900, and temperature changes of electronic device 900. Sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 914 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0196] Communication component 916 is configured to facilitate wired or wireless communication between electronic device 900 and other devices. Electronic device 900 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 916 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 916 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0197] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0198] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, which can be executed by a processor 920 of an electronic device 900 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0199] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the device coordination method provided by exemplary embodiments of this disclosure.

[0200] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0201] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A device coordination method, characterized in that, include: Receive target audio data sent by a first electronic device; the target audio data is audio data collected when the first electronic device plays preset ultrasound data; Based on the target audio data, a target cooperative operation is determined, which is used to assist the first electronic device in performing audio data-related processing. Perform the target cooperative operation.

2. The device coordination method according to claim 1, characterized in that, The step of determining the target cooperative operation based on the target audio data includes: Determine the evaluation parameters of the target audio data; the evaluation parameters are used to characterize the sound acquisition capability of the first audio acquisition module, which is the audio acquisition module of the first electronic device; Based on the evaluation parameters, the target cooperative operation is determined.

3. The device coordination method according to claim 2, characterized in that, Determining the target cooperative operation based on the evaluation parameters includes: When the evaluation parameter is less than the first preset threshold, the target collaborative operation is determined to be the first collaborative operation; the first collaborative operation represents the use of the second audio acquisition module to acquire audio data, and the second audio acquisition module is the audio acquisition module of the second electronic device.

4. The device coordination method according to claim 2, characterized in that, The method further includes: When the evaluation parameter is greater than or equal to the first preset threshold and less than the second preset threshold, a prompt message is sent to the first electronic device; the prompt message is used to indicate the sound reception capability of the first audio acquisition module.

5. The device coordination method according to claim 1, characterized in that, The step of determining the target cooperative operation based on the target audio data includes: The target audio data is identified to obtain a first identification result of the target audio data; the first identification result is used to characterize whether the target audio data includes the posture data of the target object, and the posture data is used to characterize the posture of the target object. Based on the first identification result, the target cooperative operation is determined.

6. The device coordination method according to claim 5, characterized in that, The step of determining the target cooperative operation based on the first identification result includes: When the first recognition result indicates that the target audio data includes the posture data of the target object, the target cooperative operation is determined to be a second cooperative operation, which is used to characterize the operation indicated by the posture data.

7. The device coordination method according to claim 6, characterized in that, The method further includes: Determine the posture type corresponding to the posture data; Perform the operation corresponding to the gesture type.

8. A device coordination method, characterized in that, include: Collect target audio data; the target audio data is the audio data collected by the first electronic device when playing preset ultrasound data; The target audio data is sent to the second electronic device. The target audio data is used to determine a target cooperative operation. The target cooperative operation is a cooperative operation to be executed by the second electronic device. The target cooperative operation is used to assist the first electronic device in performing processing related to the audio data.

9. The device coordination method according to claim 8, characterized in that, Sending the target audio data to the second electronic device includes: When the first electronic device activates the first preset function, it sends the target audio data to the second electronic device; the first preset function is a function for detecting the sound reception effect of the first audio acquisition module, and the first audio acquisition module is the audio acquisition module of the first electronic device.

10. The device coordination method according to claim 9, characterized in that, The method further includes: The system receives a prompt message sent by the second electronic device; the prompt message is used to indicate the sound reception capability of the first audio acquisition module. Based on the prompt information, the sound reception capability of the first audio acquisition module is adjusted.

11. The device coordination method according to claim 8, characterized in that, Sending the target audio data to the second electronic device includes: When the first electronic device activates the second preset function, it identifies the target audio data and obtains a second identification result of the target audio data; the second preset function is a function for identifying the posture of the target object, and the second identification result is used to characterize whether the target audio data includes the posture data of the target object, and the posture data is used to characterize the posture of the target object. When the second recognition result indicates that the target audio data includes the posture data of the target object, the target audio data is sent to the second electronic device.

12. A device for coordinating equipment, characterized in that, include: The receiving module is configured to receive target audio data sent by the first electronic device; The target audio data is the audio data collected when the first electronic device plays preset ultrasound data; The determination module is configured to determine a target cooperative operation based on the target audio data, the target cooperative operation being used to assist the first electronic device in performing audio data-related processing. The processing module is configured to perform the target cooperative operation.

13. A device for coordinating equipment, characterized in that, include: The acquisition module is configured to acquire target audio data; the target audio data is the audio data acquired by the first electronic device when playing preset ultrasound data; The sending module is configured to send the target audio data to a second electronic device. The target audio data is used to determine a target collaborative operation, which is a collaborative operation to be executed by the second electronic device. The target collaborative operation is used to assist the first electronic device in performing processing related to the audio data.

14. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the device coordination method as described in claims 1-7 or as described in any one of claims 8-11.

15. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the device coordination method as described in claims 1-7 or as described in any one of claims 8-11.

Images