Voice control method, device, apparatus, medium and program product

Abstract

The application provides a voice control method and device, equipment, medium and program product. Sound collection information sent by multiple to-be-controlled devices is received, the sound collection information corresponds to a voice instruction issued by a user, a preset analysis model is used to determine a control instruction, a target control device, a response device and a response mode of the voice instruction according to the sound collection information, the response device corresponds to the response mode, finally, the target control device is controlled to execute a target task according to the control instruction, and the response device is controlled to feed back execution of the target task to the user. The technical problem of how to avoid the need for the user to be close to the target control device for accurate wake-up and control in the nearby wake-up mechanism is solved, and the technical effect of improving the user experience of the distributed multi-device voice control system is achieved.

Description

Technical Field

[0001] This application relates to the field of voice control technology, specifically to a voice control method, device, equipment, medium, and program product. Background Technology

[0002] In a distributed multi-device voice control system, if multiple devices use the same wake-up mechanism, a voice control strategy of waking up or responding to the nearest device is generally adopted to avoid confusion caused by multiple devices responding at the same time.

[0003] However, this also forces users to walk close to a device in order to accurately control it. For some users with limited mobility, or for devices placed in corners, it is difficult to get close to the target control device, which causes a lot of inconvenience to the user in actual use.

[0004] Therefore, how to avoid the need for users to be close to the target control device for accurate wake-up and control in the proximity wake-up mechanism has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a voice control method, device, equipment, medium, and program product, which solves the technical problem of how to avoid the need for the user to be close to the target control device in the proximity wake-up mechanism for accurate wake-up and control.

[0006] Firstly, this application provides a voice control method, including:

[0007] It receives sound acquisition information from multiple devices to be controlled, and the sound acquisition information corresponds to the voice commands issued by the user.

[0008] Using a pre-defined analytical model, based on the sound acquisition information, the control command, target control device, response device, and response method to the voice command are determined, with the response device corresponding to the response method.

[0009] The system controls the target device to execute the target task according to the control command, and controls the response device to provide feedback to the user on the execution status of the target task.

[0010] In one possible design, the response methods include: self-response mode and proximity response mode, and the response devices include: a first response device and a second response device;

[0011] The first response device corresponds to the self-response mode, and the first response device includes: target control device;

[0012] The second response device corresponds to the nearest response mode. The second response device is configured such that the distance from the user meets a preset distance requirement and includes at least one device to be controlled.

[0013] In one possible design, a pre-defined analytical model is used to determine the control commands, target control devices, response devices, and response methods to voice commands based on the acquired sound information. This includes:

[0014] Using a semantic parsing model, the sound acquisition information is semantically parsed to determine the control commands and the target control device;

[0015] Using a pose analysis model, based on the sound acquisition information, it is determined whether the relative position and posture of the target control device and the user meet the non-proximity response requirements;

[0016] If so, then the response mode is determined to be self-response mode;

[0017] If not, then the response method is determined to be the nearest response mode.

[0018] In one possible design, a pose analysis model is used to determine, based on the sound acquisition information, whether the relative position and posture of the target control device and the user meet the non-proximity response requirements, including:

[0019] Using a pose analysis model, position information and orientation information are determined based on the sound acquisition information. Position information is used to represent the user's position relative to the target control device, and orientation information is used to represent the direction the user is facing.

[0020] Based on location and orientation information, determine whether the target control device and the user meet the non-nearest response requirements.

[0021] In one possible design, based on location and orientation information, it is determined whether the target control device and the user meet the non-proximity response requirement, including:

[0022] Based on the location information, determine whether the distance between the user and the target control device is less than or equal to a first preset distance;

[0023] If so, then based on the orientation information and the preset angle threshold, determine whether the user is facing the target control device;

[0024] If so, then the non-nearest response requirement is met.

[0025] In one possible design, after determining whether the user is facing the target control device based on orientation information and a preset angle threshold, the following is also included:

[0026] If not, then based on the location information, determine whether the distance between the user and the target control device is less than or equal to the second preset distance;

[0027] If so, then the non-nearest response requirement is met;

[0028] The second preset distance is equal to the product of the first preset distance and the preset weight value.

[0029] In one possible design, based on location and orientation information, it is determined whether the target control device and the user meet the non-proximity response requirement, including:

[0030] Based on orientation information and preset angle thresholds, determine the target control device that the user is facing;

[0031] If the distance between the user and the target control device is determined to be less than or equal to the first preset distance based on the location information, then the preset requirement is met.

[0032] In one possible design, based on location and orientation information, it is determined whether the target control device and the user meet the non-proximity response requirement, including:

[0033] Based on orientation information and preset angle thresholds, it is determined that the user is not facing the target control device;

[0034] If the distance between the user and the target control device is determined to be less than or equal to the second preset distance based on the location information, then the preset requirement is met.

[0035] The second preset distance is equal to the product of the first preset distance and the preset weight value.

[0036] In one possible design, the preset distance requirement includes: being as close as possible to the user.

[0037] In one possible design, the sound acquisition information includes sound wave signals received by the device to be controlled, including direct wave signals and reflected wave signals. Using a pose analysis model, position and orientation information are determined based on the sound acquisition information, including:

[0038] Calculate the ratio of the direct wave signal to the reflected wave signal;

[0039] If the ratio is greater than or equal to the preset ratio, then the user is determined to be facing the device to be controlled.

[0040] In one possible design, the sound acquisition information includes sound wave signals received by the device to be controlled, including direct wave signals and reflected wave signals. Using a pose analysis model, position and orientation information are determined based on the sound acquisition information, including:

[0041] Using DRR digital reconstruction of radiographic imaging models, the user's acoustic 3D image is determined based on acoustic signals;

[0042] Using an image analysis model, orientation information is determined based on acoustic 3D images.

[0043] Secondly, this application provides a voice control device, comprising:

[0044] The receiving module is used to receive sound acquisition information sent by multiple devices to be controlled, and the sound acquisition information corresponds to the voice commands issued by the user.

[0045] The processing module is used to determine, based on the sound acquisition information and a preset analytical model, a control command, a target control device, a response device, and a response method to the voice command, wherein the response device corresponds to the response method.

[0046] The processing module is also configured to control the target control device to execute the target task according to the control command, and to control the response device to provide feedback to the user on the execution status of the target task.

[0047] In one possible design, the response methods include: self-response mode and proximity response mode, and the response devices include: a first response device and a second response device;

[0048] The first response device corresponds to the self-response mode, and the first response device includes: target control device;

[0049] The second response device corresponds to the nearest response mode. The second response device is configured such that the distance from the user meets a preset distance requirement and includes at least one device to be controlled.

[0050] In one possible design, the processing module is specifically used for:

[0051] Using a semantic parsing model, the sound acquisition information is semantically parsed to determine the control commands and the target control device;

[0052] Using a pose analysis model, based on the sound acquisition information, it is determined whether the relative position and posture of the target control device and the user meet the non-proximity response requirements;

[0053] If so, then the response mode is determined to be self-response mode;

[0054] If not, then the response method is determined to be the nearest response mode.

[0055] In one possible design, the processing module is specifically used for:

[0056] Using a pose analysis model, position information and orientation information are determined based on the sound acquisition information. Position information is used to represent the user's position relative to the target control device, and orientation information is used to represent the direction the user is facing.

[0057] Based on location and orientation information, determine whether the target control device and the user meet the non-nearest response requirements.

[0058] In one possible design, the processing module is specifically used for:

[0059] Based on the location information, determine whether the distance between the user and the target control device is less than or equal to a first preset distance;

[0060] If so, then based on the orientation information and the preset angle threshold, determine whether the user is facing the target control device;

[0061] If so, then the non-nearest response requirement is met.

[0062] In one possible design, the processing module is specifically used for:

[0063] If not, then based on the location information, determine whether the distance between the user and the target control device is less than or equal to the second preset distance;

[0064] If so, then the non-nearest response requirement is met;

[0065] The second preset distance is equal to the product of the first preset distance and the preset weight value.

[0066] In one possible design, the processing module is specifically used for:

[0067] Based on orientation information and preset angle thresholds, determine the target control device that the user is facing;

[0068] If the distance between the user and the target control device is determined to be less than or equal to the first preset distance based on the location information, then the preset requirement is met.

[0069] In one possible design, the processing module is specifically used for:

[0070] Based on orientation information and preset angle thresholds, it is determined that the user is not facing the target control device;

[0071] If the distance between the user and the target control device is determined to be less than or equal to the second preset distance based on the location information, then the preset requirement is met.

[0072] The second preset distance is equal to the product of the first preset distance and the preset weight value.

[0073] In one possible design, the preset distance requirement includes: being as close as possible to the user.

[0074] In one possible design, the sound acquisition information includes the sound wave signals received by the device to be controlled, which include: direct wave signals and reflected wave signals;

[0075] The processing module is specifically used for:

[0076] Calculate the ratio of the direct wave signal to the reflected wave signal;

[0077] If the ratio is greater than or equal to the preset ratio, then the user is determined to be facing the device to be controlled.

[0078] In one possible design, the sound acquisition information includes the sound wave signals received by the device to be controlled, which include: direct wave signals and reflected wave signals;

[0079] The processing module is specifically used for:

[0080] Using DRR digital reconstruction of radiographic imaging models, the user's acoustic 3D image is determined based on acoustic signals;

[0081] Using an image analysis model, orientation information is determined based on acoustic 3D images.

[0082] Thirdly, this application provides an electronic device, comprising:

[0083] Processor; and,

[0084] Memory for storing the executable instructions of the processor;

[0085] The processor is configured to execute any of the possible voice control methods provided in the first aspect by executing the executable instructions.

[0086] Fourthly, this application also provides a storage medium storing a computer program for executing any of the possible voice control methods provided in the first aspect.

[0087] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the possible voice control methods provided in the first aspect.

[0088] This application provides a voice control method, apparatus, device, medium, and program product. It receives sound acquisition information from multiple devices to be controlled, with the sound acquisition information corresponding to user-issued voice commands. Then, using a preset parsing model, it determines the control command, target control device, response device, and response method to the voice command based on the sound acquisition information. The response device corresponds to the response method. Finally, it controls the target control device to execute the target task according to the control command and controls the response device to provide feedback to the user on the execution status of the target task. This solves the technical problem of how to avoid requiring the user to be close to the target control device for accurate wake-up and control in proximity wake-up mechanisms, thus improving the user experience of the distributed multi-device voice control system. Attached Figure Description

[0089] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0090] Figure 1 A schematic diagram illustrating an application scenario of the voice control method provided in this application;

[0091] Figure 2 A flowchart illustrating a voice control method provided in an embodiment of this application;

[0092] Figures 3a-3b This is a schematic diagram of an audio acquisition array provided in this embodiment;

[0093] Figure 4 A flowchart illustrating another voice control method provided in an embodiment of this application;

[0094] Figure 5 A schematic diagram of the structure of a voice control device provided in this application;

[0095] Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0096] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort, including but not limited to combinations of multiple embodiments, are within the scope of protection of this application.

[0097] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0098] Currently, distributed multi-device voice control systems use a proximity wake-up mechanism to prevent situations where one call results in multiple responses. However, this also presents a problem: users must approach the target control device to accurately wake it up and control it. This can cause significant inconvenience for users when multiple devices to be controlled coexist and are located close together.

[0099] Alternatively, when the target control device is in a hard-to-reach location, such as a corner or hanging on a corner of the roof, it is difficult for the user to approach it.

[0100] Alternatively, if a user has difficulty moving around, the proximity wake-up or proximity response mechanism can cause a lot of inconvenience for the user in order to accurately wake up and control the target device among multiple devices to be controlled.

[0101] Regarding the aforementioned technical problem of how to avoid requiring the user to be close to the target control device for accurate wake-up and control in the proximity wake-up mechanism, the inventive concept of this application is:

[0102] Multiple devices to be controlled simultaneously receive voice commands from the user. By analyzing the time and / or angle difference when the sound wave signal corresponding to the voice command is received by different devices, the user's position and the direction the user is facing at that moment are determined. Combined with the task corresponding to the voice command, the target control device is automatically identified. If the distance between the target control device and the user meets preset requirements, the target control device responds and provides feedback on the execution of the voice command. If not, nearby devices provide feedback on the execution of the voice command, and the server remotely issues control commands to control the target control device. This effectively solves the problem of proximity-based wake-up or proximity-based response mechanisms, which require the user to be close to the target control device for accurate control.

[0103] The voice control method provided in this application will be described in detail below with reference to the accompanying drawings.

[0104] Figure 1 This is a schematic diagram illustrating an application scenario of the voice control method provided in this application. For example... Figure 1 As shown, the distributed multi-device voice control system includes multiple devices to be controlled: devices to be controlled 121 to 126. The user 11 issues a voice command to one of the devices to be controlled from any position in the space where each device is located. The dashed circle indicates the propagation range of the sound wave.

[0105] The voice control method provided in this application does not require the user to be near the target control device, such as... Figure 1 In the process, all devices 121 to 126 under control receive the user's voice commands. They analyze the received sound wave signals through a preset analytical model or algorithm to determine the target control device. Based on the distance between the target control device and the user, they decide whether the target control device should respond or provide feedback on the execution status of the task.

[0106] For example, user 11 issues a voice command "Start the washing machine for a quick wash." Each device to be controlled receives the corresponding sound wave signal and sends it to the server. The server then analyzes the data and determines that the target device is the washing machine. Figure 1 Assuming the controlled device 125 represents the washing machine, a quick wash start control command is sent to the washing machine. The server then determines whether the washing machine should respond or provide feedback, based on the user's and washing machine's locations, or whether the washing machine has voice response or feedback capabilities. If the washing machine is outside the user's preset 5-meter range, a proximity response mechanism is used, with the nearest controlled device 121 providing feedback on the washing machine's quick wash task execution status, such as a voice announcement: "Received, washing machine quick wash mode is activated."

[0107] The specific steps of the voice control method provided in this application will be described in detail below with reference to several embodiments.

[0108] Figure 2 This is a flowchart illustrating a voice control method provided in an embodiment of this application. Figure 2 As shown, the specific steps of this voice control method include:

[0109] S201. Multiple devices to be controlled simultaneously collect voice commands issued by the user to determine the sound collection information.

[0110] In this step, at least one audio acquisition array, such as a microphone array group, is set in the device to be controlled. Each audio acquisition array includes at least two independent acquisition modules, such as two independent microphones.

[0111] When a user speaks a voice command from any location, the corresponding sound wave signal is transmitted to the user's environment and received by the audio acquisition array on each device to be controlled.

[0112] Figures 3a-3b This is a schematic diagram of an audio acquisition array provided in this embodiment, as shown below. Figure 3a As shown, the audio acquisition array in the device to be controlled 300 includes microphones 301 and 302. The positions of the two microphones are represented by A and B, respectively, and the length of line segment AB is the distance between the two microphones. After the user 301 speaks a voice command, the time it takes for the received direct sound waves to arrive differs due to the different positions of the two microphones. Thus, a preset algorithm can be used to calculate the sound wave source, i.e., the user's position.

[0113] Furthermore, such as Figure 3b As shown, the audio acquisition array includes microphones 301, 302, and 303, with their positions denoted by A, B, and C, respectively. The location of the sound source, i.e., the user's location, can be calculated by combining any two microphones. Figure 3b Three combinations can be configured: microphone 301 and microphone 302, microphone 301 and microphone 303, and microphone 302 and microphone 303.

[0114] In one possible design, only one set is used for sound source location identification, while the other two sets are used as backups.

[0115] In another possible design, two sets of receivers are selected as the sound source location identification receivers. The locations of the two sound sources are obtained from the two sets respectively. Then, based on the positional relationship between the two sets, a correction algorithm is used to further improve the localization accuracy of the sound source location. The third set can serve as a backup, automatically replacing the first set if either set malfunctions.

[0116] In another possible design, all three microphones can be used as receivers for sound source location identification, which can identify the user's specific location in three-dimensional space. For example, when a user issues a voice control command from the first floor to a device on the second floor, three-dimensional positioning can more accurately identify the user's specific location.

[0117] It should be noted that the position of the three groups, i.e., triangle ABC, can be any shape of triangle. Depending on how the device to be controlled is placed, it can be set as an equilateral triangle or a non-equilateral triangle. For example, if the device is placed near the center of the room, triangle ABC can be set as an equilateral triangle; when the device is placed in a corner against a wall, triangle ABC can be set as a triangle with three sides of unequal length.

[0118] Of course, through the above Figure 3a and Figure 3b As can be seen from the examples, audio acquisition arrays can also use a larger number of microphones to form polygons of arbitrary shapes, including: spatial polygons (i.e., the positions of each microphone are the vertices of a spatial polyhedron), planar regular polygons, planar non-regular polygons, and so on.

[0119] S202, Send sound acquisition information to the central control device.

[0120] In this step, each device to be controlled sends the sound acquisition information received by the audio acquisition array to the central control device.

[0121] It should be noted that the implementation methods of the central control device include at least one of the following: cloud server, local server, independent central control unit, and a preset device to be controlled.

[0122] S203: Receive sound acquisition information sent by multiple devices to be controlled.

[0123] In this step, the server or central control device receives sound acquisition information reported by multiple devices to be controlled. In addition to the voice sound wave signal, the sound acquisition information also includes the positioning parameters attached by the audio acquisition array during acquisition, including: reception time, reception angle, direct sound wave signal, reflected sound wave signal, etc.

[0124] S204. Using a preset analytical model, determine the control command, target control device, response device, and response method to the voice command based on the sound acquisition information.

[0125] In this step, the response device corresponds to the response method.

[0126] Specifically, a semantic parsing model is used to perform semantic parsing on the sound acquisition information in order to determine the target control device and control commands;

[0127] For example, the audio content in the audio acquisition information is identified and converted into corresponding text. Then, semantic recognition is performed on the text to extract the target control device.

[0128] For example, if the text recognized from the sound content is "wash jeans in standard mode", then the semantic recognition indicates that the target control device is a washing machine, and the control command is to turn on standard mode.

[0129] It should be noted that, in one possible implementation, voice commands may not include a wake word, which provides users with a more intelligent user experience. This also avoids the confusion caused by multiple devices responding simultaneously when the same wake word can wake up multiple devices, or avoids activating the nearest-neighbor wake-up mechanism.

[0130] Using a pose analysis model, the user's position and orientation information are determined based on the sound acquisition information. This analysis determines whether a response needs to be given to the user by the target control device, or whether the target control device should provide feedback, thus determining the response method. The position information represents the user's position relative to the target control device, and the orientation information represents the direction the user is facing.

[0131] In one possible design, the response methods include: self-response mode and proximity response mode, and the response devices include: a first response device and a second response device.

[0132] The first response device corresponds to the self-response mode, and the first response device includes: target control device.

[0133] The second response device corresponds to the nearest response mode. The second response device is configured such that the distance from the user meets a preset distance requirement and includes at least one device to be controlled.

[0134] In one possible design, when the distance between the user and the target control device is within a preset range, and the user is facing the target control device, such as within 5 meters, the self-response mode is selected to respond.

[0135] When the distance between the user and the target control device is within a preset range, but the user is not facing the target control device, the distance between the user and the target control device is multiplied by a weighting coefficient. If the resulting product is still within the preset range, the self-response mode is selected to respond.

[0136] When the distance between the user and the target control device is outside the preset range, select the nearest response mode to respond.

[0137] When the target control device does not have a voice feedback device (such as a speaker), the nearest response mode is selected to respond, and the second response device selects the control device that has a voice feedback device.

[0138] In one possible design, based on location and orientation information, it is determined whether the target control device and the user meet the non-proximity response requirement, specifically including:

[0139] Based on orientation information and preset angle thresholds, determine the target control device that the user is facing;

[0140] If the distance between the user and the target control device is determined to be less than or equal to the first preset distance based on the location information, then the preset requirement is met.

[0141] Furthermore, based on location and orientation information, determining whether the target control device and the user meet the non-proximity response requirements also includes:

[0142] Based on orientation information and preset angle thresholds, it is determined that the user is not facing the target control device;

[0143] If the distance between the user and the target control device is determined to be less than or equal to the second preset distance based on the location information, then the preset requirement is met.

[0144] The second preset distance is equal to the product of the first preset distance and the preset weight value.

[0145] S205. Control the target control device to execute the target task according to the control command, and control the response device to provide feedback to the user on the execution status of the target task.

[0146] In this step, the central control device sends a control command to the target control device so that the target control device can start executing the corresponding target task. If the previous step determined to respond in self-response mode, the target control device will directly provide feedback to the user on the execution status of the target task. If the previous step determined to respond in proximity mode, the target control device will provide feedback on the execution status to the central control device, which will then send the feedback information to the second response device, which will then provide feedback to the user.

[0147] Specifically, in one possible implementation, the self-response mode can be understood as using the target control device to respond to the user's voice commands, such as issuing a voice broadcast "Received, now starting the quick wash mode for you", and issuing a voice broadcast "Quick wash task completed" after the task is completed.

[0148] The proximity response mode can be understood as using the device closest to the user to respond to the user's voice commands. For example, if the user is closest to the refrigerator but issues a washing machine command, the refrigerator will announce, "The washing machine has started the quick wash mode for you."

[0149] This embodiment provides a voice control method. It receives sound acquisition information from multiple devices to be controlled, which corresponds to voice commands issued by the user. Then, using a preset parsing model, it determines the control command, target control device, response device, and response method to the voice command based on the sound acquisition information. The response device corresponds to the response method. Finally, it controls the target control device to execute the target task according to the control command and controls the response device to provide feedback to the user on the execution status of the target task. This solves the technical problem of avoiding the need for the user to be close to the target control device for accurate wake-up and control in proximity wake-up mechanisms, thus improving the user experience of the distributed multi-device voice control system.

[0150] The following further explains the possible specific implementation methods of steps S204 to S205.

[0151] Figure 4 This is a flowchart illustrating another voice control method provided in an embodiment of this application. Figure 4 As shown, the specific steps of this voice control method include:

[0152] S401: Multiple devices to be controlled simultaneously collect voice commands issued by the user to determine the sound collection information.

[0153] S402, Send sound acquisition information to the central control device.

[0154] For a detailed explanation of steps S401 to S402, please refer to [link / reference]. Figure 2 The steps S201 to S202 of the illustrated embodiment will not be described again here.

[0155] S403. Using a semantic parsing model, perform semantic parsing on the sound acquisition information to determine the control commands and target control devices.

[0156] In this step, the audio content in the audio acquisition information is identified and converted into corresponding text. Then, semantic recognition is performed on the text to extract the target control device and control commands.

[0157] For example, if the text recognized from the sound content is "wash jeans in standard mode", then the semantic recognition indicates that the target control device is a washing machine, and the control command is to turn on standard mode.

[0158] S404. Using the pose analysis model, based on the sound acquisition information, determine whether the relative position and posture of the target control device and the user meet the non-nearest response requirements.

[0159] In this embodiment, the response methods include: self-response mode and proximity response mode, and the response devices include: a first response device and a second response device.

[0160] The first response device corresponds to the self-response mode, and the first response device includes: target control device;

[0161] The second response device corresponds to the nearest response mode. The second response device is configured such that the distance from the user meets a preset distance requirement and includes at least one device to be controlled.

[0162] In this step, if yes, the response mode is determined to be self-response mode, and steps S405 and S406 are executed; if no, the response mode is determined to be the nearest response mode, and steps S405 and S407 are executed.

[0163] In one possible design, a pose analysis model is used to determine, based on the sound acquisition information, whether the relative position and posture of the target control device and the user meet the non-proximity response requirement. Specifically, this includes:

[0164] S4041. Using the pose analysis model, determine the position and orientation information based on the sound acquisition information.

[0165] In this step, the location information is used to indicate the user's position relative to the target control device, the orientation information is used to indicate the direction the user is facing, and the sound acquisition information includes the sound wave signals received by the device to be controlled, which include: direct wave signals and reflected wave signals.

[0166] Specifically, calculate the ratio of the direct wave signal to the reflected wave signal;

[0167] If the ratio is greater than or equal to the preset ratio, then the user is determined to be facing the device to be controlled.

[0168] It should be noted that when the application scenario or usage environment is indoors, the voice commands issued by the user have two parts. The corresponding sound wave signals are directly transmitted to the receiving device of the device to be controlled, which is the so-called direct wave signal. The other part reaches the device to be controlled through reflection from surrounding hard surfaces such as walls, which is the reflected wave signal. By calculating the ratio of the direct wave signal to the reflected wave signal through a preset algorithm, it can be determined whether the user is facing the target device to control.

[0169] In another possible design, DRR (Digitally Reconstructed Radiograph) is used to reconstruct the radiographic image model and determine the user's acoustic 3D image based on the acoustic signal.

[0170] Using an image analysis model, orientation information is determined based on acoustic 3D images.

[0171] It should be noted that DRR digital reconstruction radiographic imaging models are generally used to process medical examination imaging, such as the construction of medical CT (Computed Tomography) images. This embodiment, however, breaks with the inertia or limitations of this technology by using it to identify the user's posture and determine whether the user is facing the target control device.

[0172] This involves using DRR technology to construct an acoustic 3D image, and then using a trained image analysis model to identify this acoustic 3D image, thereby determining whether the user is currently facing the target control device.

[0173] It should also be noted that the sound wave signal corresponds to the voice command (that is, the carrier of the voice command is the sound wave signal), and the sound wave signal is received by the audio acquisition module in the device to be controlled.

[0174] S4042. Based on the location information and orientation information, determine whether the target control device and the user meet the non-nearest response requirements.

[0175] Specifically, based on location information, it is determined whether the distance between the user and the target control device is less than or equal to a first preset distance;

[0176] If so, then based on the orientation information and the preset angle threshold, determine whether the user is facing the target control device; if so, determine that the non-nearby response requirement is met.

[0177] If not, then based on the location information, determine whether the distance between the user and the target control device is less than or equal to the second preset distance; if yes, then determine that the non-nearest response requirement is met.

[0178] The second preset distance is equal to the product of the first preset distance and the preset weight value.

[0179] It should be noted that when the user is facing the target control device, the preset weight value ranges from [0,1], that is, greater than or equal to 0 and less than or equal to 1; when the user is not facing the target control device, the preset weight value ranges from greater than 1.

[0180] It should also be noted that for appliances that users are not allowed to approach during operation, or that may generate harmful light or heat radiation during operation, such as microwave ovens and electric stoves, the range of the aforementioned weight values ​​can be reversed. That is, when the user is facing the target control device, the preset weight value range is greater than 1; when the user is not facing the target control device, the preset weight value range is [0,1]. This is to prevent users from being harmed.

[0181] In one possible design, the user's identity can be further identified based on the frequency of the sound wave signal in the sound acquisition information. Different weight values ​​can be set according to different user identities. For example, the weight value can be smaller for users with mobility impairments, so as to expand the range of voice control that users can perform.

[0182] S405. Control the target device to perform the target task according to the control command.

[0183] S406. The target control device provides feedback to the user on the execution status of the target task.

[0184] S407. Control the second response device selected from the devices to be controlled according to the preset filtering rules, and provide feedback to the user on the execution status of the target task.

[0185] In this embodiment, the preset filtering rule is: based on the distance between each device to be controlled and the user, the device to be controlled that is closest to the user is selected as the second response device.

[0186] This embodiment provides a voice control method. It receives sound acquisition information from multiple devices to be controlled, which corresponds to voice commands issued by the user. Then, using a preset parsing model, it determines the control command, target control device, response device, and response method to the voice command based on the sound acquisition information. The response device corresponds to the response method. Finally, it controls the target control device to execute the target task according to the control command and controls the response device to provide feedback to the user on the execution status of the target task. This solves the technical problem of avoiding the need for the user to be close to the target control device for accurate wake-up and control in proximity wake-up mechanisms, thus improving the user experience of the distributed multi-device voice control system.

[0187] Figure 5 This is a schematic diagram of a voice control device provided in this application. The voice control device can be implemented through software, hardware, or a combination of both.

[0188] like Figure 5 As shown, the voice control device 500 provided in this embodiment includes:

[0189] The receiving module 501 is used to receive sound acquisition information sent by multiple devices to be controlled, wherein the sound acquisition information corresponds to the voice commands issued by the user;

[0190] The processing module 502 is used to determine, based on the sound acquisition information and a preset analytical model, a control command, a target control device, a response device, and a response method to the voice command, wherein the response device corresponds to the response method.

[0191] The processing module 502 is further configured to control the target control device to execute the target task according to the control command, and to control the response device to provide feedback to the user on the execution status of the target task.

[0192] In one possible design, the response methods include: self-response mode and proximity response mode, and the response devices include: a first response device and a second response device;

[0193] The first response device corresponds to the self-response mode, and the first response device includes: target control device;

[0194] The second response device corresponds to the nearest response mode. The second response device is configured such that the distance from the user meets a preset distance requirement and includes at least one device to be controlled.

[0195] In one possible design, the processing module 502 is specifically used for:

[0196] Using a semantic parsing model, the sound acquisition information is semantically parsed to determine the control commands and the target control device;

[0197] Using a pose analysis model, based on the sound acquisition information, it is determined whether the relative position and posture of the target control device and the user meet the non-proximity response requirements;

[0198] If so, then the response mode is determined to be self-response mode;

[0199] If not, then the response method is determined to be the nearest response mode.

[0200] In one possible design, the processing module 502 is specifically used for:

[0201] Using a pose analysis model, position information and orientation information are determined based on the sound acquisition information. Position information is used to represent the user's position relative to the target control device, and orientation information is used to represent the direction the user is facing.

[0202] Based on location and orientation information, determine whether the target control device and the user meet the non-nearest response requirements.

[0203] In one possible design, the processing module 502 is specifically used for:

[0204] Based on the location information, determine whether the distance between the user and the target control device is less than or equal to a first preset distance;

[0205] If so, then based on the orientation information and the preset angle threshold, determine whether the user is facing the target control device;

[0206] If so, then the non-nearest response requirement is met.

[0207] In one possible design, the processing module 502 is specifically used for:

[0208] If not, then based on the location information, determine whether the distance between the user and the target control device is less than or equal to the second preset distance;

[0209] If so, then the non-nearest response requirement is met;

[0210] The second preset distance is equal to the product of the first preset distance and the preset weight value.

[0211] In one possible design, processing module 502 is specifically used for:

[0212] Based on orientation information and preset angle thresholds, determine the target control device that the user is facing;

[0213] If the distance between the user and the target control device is determined to be less than or equal to the first preset distance based on the location information, then the preset requirement is met.

[0214] In one possible design, the processing module 502 is specifically used for:

[0215] Based on orientation information and preset angle thresholds, it is determined that the user is not facing the target control device;

[0216] If the distance between the user and the target control device is determined to be less than or equal to the second preset distance based on the location information, then the preset requirement is met.

[0217] The second preset distance is equal to the product of the first preset distance and the preset weight value.

[0218] In one possible design, the preset distance requirement includes: being as close as possible to the user.

[0219] In one possible design, the sound acquisition information includes the sound wave signals received by the device to be controlled, which include: direct wave signals and reflected wave signals;

[0220] The processing module 502 is specifically used for:

[0221] Calculate the ratio of the direct wave signal to the reflected wave signal;

[0222] If the ratio is greater than or equal to the preset ratio, then the user is determined to be facing the device to be controlled.

[0223] In one possible design, the sound acquisition information includes the sound wave signals received by the device to be controlled, which include: direct wave signals and reflected wave signals;

[0224] The processing module 502 is specifically used for:

[0225] Using DRR digital reconstruction of radiographic imaging models, the user's acoustic 3D image is determined based on acoustic signals;

[0226] Using an image analysis model, orientation information is determined based on acoustic 3D images.

[0227] It is worth noting that, Figure 5 The voice control device provided in the illustrated embodiment can execute the method provided in any of the above method embodiments. Its specific implementation principle, technical features, explanation of technical terms and technical effects are similar, and will not be repeated here.

[0228] Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 6 As shown, the electronic device 600 may include at least one processor 601 and a memory 602. Figure 6 The example shown is an electronic device using a processor.

[0229] The memory 602 is used to store programs. Specifically, the program may include program code, which includes computer operation instructions.

[0230] The memory 602 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0231] The processor 601 is used to execute computer execution instructions stored in the memory 602 to implement the methods described in the above embodiments.

[0232] The processor 601 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0233] Optionally, the memory 602 can be either standalone or integrated with the processor 601. When the memory 602 is a device independent of the processor 601, the electronic device 600 may further include:

[0234] Bus 603 is used to connect the processor 601 and the memory 602. The bus can be an industry standard architecture (ISA) bus, a peripheral component (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not mean there is only one bus or one type of bus.

[0235] Optionally, in a specific implementation, if the memory 602 and the processor 601 are integrated on a single chip, the memory 602 and the processor 601 can communicate through an internal interface.

[0236] This application also provides a computer-readable storage medium, which may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. Specifically, the computer-readable storage medium stores program instructions, which are used in the methods described in the above embodiments.

[0237] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in the above embodiments.

[0238] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A voice control method, characterized by, The method comprises the following steps: receiving sound collection information sent by a plurality of to-be-controlled devices, the sound collection information corresponding to a voice instruction issued by a user; determining, by using a preset analysis model, a control instruction, a target control device, a response device, and a response mode to the voice instruction according to the sound collection information, the response device corresponding to the response mode; controlling the target control device to execute a target task according to the control instruction, and controlling the response device to feed back execution of the target task to the user; the response mode comprises a self-response mode and a nearby-response mode; the method comprises the following steps: performing semantic analysis on the sound collection information by using a semantic analysis model to determine the control instruction and the target control device; judging whether a relative position and posture of the target control device and the user meet a non-nearby-response requirement according to the sound collection information by using a pose analysis model; if yes, determining that the response mode is the self-response mode; if no, determining that the response mode is the nearby-response mode.

2. The voice control method of claim 1, wherein, the response device comprises a first response device and a second response device; the first response device corresponds to the self-response mode, and the first response device comprises the target control device; the second response device corresponds to the nearby-response mode, and the second response device is configured to satisfy a preset distance requirement and comprises at least one to-be-controlled device.

3. The voice control method of claim 2, wherein, the method comprises the following steps: determining position information and orientation information according to the sound collection information by using the pose analysis model, the position information being used to represent a position of the user relative to the target control device, and the orientation information being used to represent a direction faced by the user; judging whether the non-nearby-response requirement is met between the target control device and the user according to the position information and the orientation information.

4. The voice control method of claim 3, wherein, the method comprises the following steps: judging whether a distance between the user and the target control device is less than or equal to a first preset distance according to the position information; if yes, judging whether the user faces the target control device according to the orientation information and a preset angle threshold; if yes, determining that the non-nearby-response requirement is met.

5. The voice control method of claim 4, wherein, after the step of judging whether the user faces the target control device according to the orientation information and a preset angle threshold, the method further comprises the following steps: if no, judging whether a distance between the user and the target control device is less than or equal to a second preset distance according to the position information; if yes, determining that the non-nearby-response requirement is met. The second preset distance is equal to a product of the first preset distance and a preset weight value.

6. The voice control method according to any one of claims 2-5, characterized in that, The preset distance requirement comprises being closest to the user.

7. The voice control method of any of claims 3-5, wherein, The sound collection information comprises sound wave signals received by the to-be-controlled device, and the sound wave signals comprise direct wave signals and reflected wave signals. A ratio of the direct wave signals and the reflected wave signals is calculated. If the ratio is greater than or equal to a preset ratio, it is determined that the user faces the to-be-controlled device.

8. The voice control method of any one of claims 3-5, wherein, The sound collection information comprises sound wave signals received by the to-be-controlled device, and the sound wave signals comprise direct wave signals and reflected wave signals. An acoustic three-dimensional image of the user is determined according to the sound wave signals by using a DRR digital reconstruction radiographic image model. The orientation information is determined according to the acoustic three-dimensional image by using an image analysis model.

9. A voice control device, characterized in that The method comprises the following steps: The receiving module is configured to receive sound collection information sent by a plurality of to-be-controlled devices, the sound collection information corresponding to a voice instruction issued by a user. The processing module is configured to determine a control instruction, a target control device, a response device and a response mode of the voice instruction according to the sound collection information by using a preset analysis model, the response device corresponding to the response mode. The processing module is further configured to control the target control device to perform a target task according to the control instruction, and control the response device to feed back execution of the target task to the user. The response mode comprises a self-response mode and a nearby-response mode. The processing module is specifically configured to perform semantic analysis on the sound collection information by using a semantic analysis model, to determine the control instruction and the target control device. The relative position and attitude of the target control device and the user are determined according to the sound collection information by using a pose analysis model, to determine whether the non-nearby-response requirement is met. If yes, the response mode is determined to be the self-response mode. If no, the response mode is determined to be the nearby-response mode.

10. An electronic device, comprising: The method comprises the following steps: A processor; and A memory for storing an executable computer program of the processor; The processor is configured to execute the voice control method of any one of claims 1 to 8 by executing the executable computer program.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the voice control method of any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the voice control method of any one of claims 1 to 8.

Images