Voice processing method, robot, and readable storage medium

Abstract

Embodiments of the present application provide a voice processing method, a robot and a readable storage medium, and relate to the technical field of robots. In the method, in response to receiving first voice information input by a first user, the first user is announced first response information at a first announcement volume. The first voice information is input at a first distance. In response to receiving second voice information input by the first user, the first user is announced second response information at a second announcement volume. The second voice information is input at a second distance, the second distance is greater than the first distance, and the second announcement volume is greater than the first announcement volume. In this way, based on the distance between the user and the robot, the announcement volume used is dynamically adjusted. In the case where the user is closer to the robot, a smaller announcement volume is used. In the case where the user is farther away from the robot, a larger announcement volume is used. Thus, the problem that the user cannot hear clearly or feels uncomfortable due to an unsuitable volume can be avoided.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a speech processing method, a robot, and a readable storage medium. Background Technology

[0002] With the continuous development of embodied intelligence technology, robots are being used in more and more scenarios. Among these scenarios, voice interaction, as one of the core interaction methods between humans and robots, is widely used.

[0003] In related technologies, robots often use a fixed volume when broadcasting voice messages. This method has the problem of inappropriate volume causing users to not hear clearly or feel uncomfortable. Summary of the Invention

[0004] This application provides a voice processing method, a robot, and a readable storage medium to solve the problem that inappropriate volume causes users to be unable to hear clearly or feel uncomfortable.

[0005] In a first aspect, embodiments of this application disclose a speech processing method applied to a robot, the method comprising: In response to receiving first voice information input by a first user, first response information is played to the first user at a first playback volume; the first voice information is input at a first distance. In response to receiving the second voice information input by the first user, the system plays the second response information to the first user at a second playback volume; the second voice information is input at a second distance, the second distance being greater than the first distance, and the second playback volume being greater than the first playback volume. This application also discloses a robot, including: a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface communicate with each other through the communication bus. The memory is used to store executable instructions, which cause the processor to execute the method described above.

[0006] This application also discloses a readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described above.

[0007] This application also discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0008] The embodiments of this application include the following advantages: In response to receiving first voice information input by a first user, first response information is played to the first user at a first playback volume; the first voice information is input at a first distance. In response to receiving second voice information input by the first user, second response information is played to the first user at a second playback volume; the second voice information is input at a second distance, where the second distance is greater than the first distance, and the second playback volume is greater than the first playback volume. In this way, based on the distance between the user and the robot, the playback volume is dynamically adjusted, using a smaller playback volume when the user is closer to the robot and a larger playback volume when the user is farther from the robot, thereby avoiding the problem of the user not being able to hear clearly or feeling uncomfortable due to inappropriate volume. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart of the steps of a speech processing method provided in an embodiment of this application; Figure 2 This is a schematic diagram of a processing flow provided in an embodiment of this application; Figure 3 This is a block diagram of a voice processing device provided in an embodiment of this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] First, an exemplary application scenario related to the embodiments of this application will be described. The voice processing method provided in the embodiments of this application can be applied to voice interaction scenarios between users and robots. In various scenarios (e.g., home, hotel, exhibition hall, shopping mall, etc.), the robot uses voice to converse with users, broadcast information, and execute instructions. The user experience is affected by the volume of the robot's voice broadcast. If the broadcast volume is too loud, it will cause user discomfort. If the broadcast volume is too low, it will cause user to be unable to hear clearly. In related technologies, the user manually sets a fixed broadcast volume, and the robot always broadcasts at this fixed volume. The fixed volume broadcast method has poor flexibility and may result in the user not being able to hear clearly or feeling uncomfortable due to an inappropriate volume.

[0013] Therefore, this application provides a speech processing method. The speech processing method will be described in detail below.

[0014] Figure 1 This is a flowchart of the steps of a speech processing method provided in an embodiment of this application, as follows: Figure 1 As shown, the method may include: Step 101: In response to receiving the first voice information input by the first user, broadcast the first response information to the first user at the first broadcast volume; the first voice information is input at the first distance.

[0015] Step 103: In response to receiving the second voice information input by the first user, broadcast the second response information to the first user at a second broadcast volume; the second voice information is input at a second distance, the second distance is greater than the first distance, and the second broadcast volume is greater than the first broadcast volume.

[0016] The voice processing method provided in this application can be applied to a robot. The first user can be any user using the robot, the first voice information can be any voice information input by the first user to the robot, and the first distance can be the distance between the first user and the robot when the first voice information is input. The second voice information can be any voice information input by the first user to the robot, and the second distance can be the distance between the first user and the robot when the second voice information is input. The first voice information and the second voice information are voice information input by the first user at positions at different distances from the robot. The first voice information and the second voice information can be the same or different, and this application does not impose any restrictions on this.

[0017] The first response information is the response information generated by the robot in response to the first voice information. The first response information can be generated based on the semantics and intent of the first voice information. For example, assuming the first voice information is "Xiao X, please help me organize the wardrobe," the first response information could be "Okay, I'm about to go to the wardrobe area to organize." The second response information is the response information generated by the robot in response to the second voice information. The second response information can be generated based on the semantics and intent of the second voice information. For example, assuming the second voice information is "Xiao X, what's the temperature today? What clothes should I wear?", the second response information could be "The high temperature today is 38℃, I suggest wearing short sleeves and shorts."

[0018] When the robot outputs the first and second response information, it does so via voice broadcast. The first broadcast volume is the output volume used by the robot when broadcasting the first response information, and the second broadcast volume is the output volume used by the robot when broadcasting the second response information. In practical applications, the user's perception of volume is affected by the distance between them and the robot. When the robot uses the same broadcast volume, the closer the user is to the robot, the louder the sound will seem; the farther the user is from the robot, the quieter the sound will seem. In this embodiment, when the second distance is greater than the first distance, the robot uses a second broadcast volume that is greater than the first broadcast volume.

[0019] In related technologies, using a fixed volume cannot adapt to changes in the distance between the user and the robot. When the user is far from the robot, the announcement volume may be too low, making it difficult for the user to hear. When the user is close to the robot, the announcement volume may be too high, causing discomfort. This new technology dynamically adjusts the announcement volume based on the distance between the user and the robot. The closer the user is to the robot when inputting voice information, the lower the announcement volume; the farther the user is from the robot, the higher the announcement volume. This allows the announcement volume to dynamically adapt to the distance between the user and the robot.

[0020] In summary, the voice processing method provided in this application embodiment, in response to receiving first voice information input by a first user, plays first response information to the first user at a first playback volume; the first voice information is input at a first distance. In response to receiving second voice information input by the first user, plays second response information to the first user at a second playback volume; the second voice information is input at a second distance, where the second distance is greater than the first distance, and the second playback volume is greater than the first playback volume. Thus, based on the distance between the user and the robot, the playback volume is dynamically adjusted, using a smaller volume when the user is closer to the robot and a larger volume when the user is farther away, thereby avoiding the problem of the user not being able to hear clearly or feeling uncomfortable due to inappropriate volume.

[0021] Optionally, the embodiments of this application may further include: in response to receiving third voice information input by a second user at a third distance, broadcasting third response information to the second user at a third broadcast volume; in response to receiving fourth voice information input by a third user at the third distance, broadcasting fourth response information to the third user at a fourth broadcast volume; wherein the fourth broadcast volume is different from the third broadcast volume, or the fourth broadcast volume is the same as the third broadcast volume.

[0022] The second user can be any user using the robot. The second user can be the same as or different from the first user. The third user can be any user using the robot. The third user can be the same as or different from the first user. The third user is different from the second user.

[0023] The third voice information can be voice information input by the second user at a position a third distance away from the robot, and the fourth voice information can be voice information input by the third user at a position a third distance away from the robot. That is, the third voice information and the fourth voice information are voice information input by different users at positions equidistant from the robot. The third voice information and the fourth voice information can be input simultaneously or sequentially, and this application embodiment does not impose any restrictions on this.

[0024] The third response information is the response information generated by the robot in response to the third voice information. The third response information can be generated based on the semantics and intent of the third voice information. The fourth response information is the response information generated by the robot in response to the fourth voice information. The fourth response information can also be generated based on the semantics and intent of the fourth voice information. When outputting the third and fourth response information, the robot outputs them in the form of voice broadcasts. The third broadcast volume is the output volume used by the robot when broadcasting the third response information, and the fourth broadcast volume is the output volume used by the robot when broadcasting the fourth response information.

[0025] In this embodiment, when responding to voice information input by a second user and a third user at the same distance from the robot, the same or different broadcast volumes can be used, making the voice broadcast more flexible.

[0026] Optionally, in this embodiment of the application, when the input volume of the third voice information is different from the input volume of the fourth voice information, and the second user and the third user adopt the same distance and volume mapping relationship, the fourth broadcast volume is the same as the third broadcast volume; when the second user and the third user adopt different distance and volume mapping relationships, the fourth broadcast volume is different from the third broadcast volume.

[0027] In this embodiment, the volume used by the robot to output response information in response to user-input voice information is determined based on the distance between the user and the robot at the time of input. For example, the corresponding volume can be found from a distance-volume mapping relationship based on the distance between the user and the robot, and used as the broadcast volume.

[0028] In real-world applications, different users often speak at varying volumes. Therefore, the input volume of the third voice information from the second user will differ from the input volume of the fourth voice information from the fourth user. One implementation uses the same distance-volume mapping relationship for different users, ensuring that the third and fourth broadcast volumes are identical for the second and third users. This allows the robot to accurately and dynamically adjust the broadcast volume according to distance, even when different users input voice information at varying volumes, thus avoiding the impact of individual differences on the broadcast volume.

[0029] In real-world applications, different users have varying abilities to perceive volume levels. In another approach, different distance-volume mapping relationships can be used for different users. By maintaining personalized distance-volume mapping relationships for different users, it can be ensured to some extent that the final broadcast volume can better suit the user's individual situation, thereby improving the adaptability of the broadcast volume to the user.

[0030] Optionally, embodiments of this application may further include: Step S21: In response to the first feedback event from the fourth user, increase the volume of the fifth broadcast of the fifth response information to the fifth voice information input by the fourth user at the same distance. The first feedback event indicates that the robot's broadcast volume is lower than the fourth user's expected acceptable volume. Alternatively, Step S22: In response to the second feedback event of the fourth user, reduce the fifth broadcast volume, whereby the second feedback event indicates that the broadcast volume is higher than the expected acceptable volume.

[0031] In this embodiment, the fourth user can be any user using the robot, the fifth voice information can be any voice information input by the fourth user to the robot, and the fifth response information is the response information generated by the robot in response to the fifth voice information. The fifth response information can be generated based on the semantics and intent of the fifth voice information. When the robot outputs the fifth response information, it outputs it in the form of voice broadcast, and the fifth broadcast volume is the output volume used by the robot when broadcasting the fifth response information.

[0032] The expected acceptable volume for the fourth user refers to the volume level that the fourth user finds acceptable. Influenced by individual differences, the acceptable volume level can vary between users. The expected acceptable volume for the fourth user can be a volume level that allows the fourth user to hear clearly and feel comfortable. The first feedback event can be an event triggered when the fourth user perceives the robot's broadcast volume (e.g., the robot's current broadcast volume and / or the historical broadcast volume used when previously outputting response information) to be lower than the fourth user's expected acceptable volume. In other words, the first feedback event can be an event triggered when the fourth user perceives the robot's broadcast volume to be too low. The second feedback event can be an event triggered when the fourth user perceives the robot's broadcast volume to be higher than the fourth user's expected acceptable volume. In other words, the second feedback event can be an event triggered when the fourth user perceives the robot's broadcast volume to be too high.

[0033] Upon receiving a first feedback event, the robot can increase the volume of its voice messages input by the user at the same distance. For example, the robot can adjust the distance-volume mapping so that it responds with a higher volume to subsequent users inputting voice messages at the same distance. Upon receiving a second feedback event, the robot can decrease the volume of its voice messages input by the user at the same distance. For example, the robot can adjust the distance-volume mapping so that it responds with a lower volume to subsequent users inputting voice messages at the same distance.

[0034] In this embodiment, in response to a first feedback event from a fourth user, the volume of the fifth playback of the fifth response information to the fifth voice information input by the fourth user at the same distance is increased. In response to a second feedback event from the fourth user, the volume of the fifth playback is decreased. Thus, by dynamically adjusting the volume based on user feedback, the final playback volume is better suited to user needs, thereby improving the voice interaction effect.

[0035] Optionally, the first feedback event is any one of the robot detecting the fourth user's approach, the fourth user triggering a first input event, and a volume increase event; the second feedback event is any one of the robot detecting the fourth user's departure, the fourth user triggering a second input event, and a volume decrease event. In this embodiment, when the robot broadcasts response information, if the robot detects that the fourth user is approaching, the detected approach event of the fourth user can be determined. For example, a lidar or vision camera can be used to identify whether the fourth user is approaching. The first input event can be an event that asks for the broadcast content or instructs the robot to rebroadcast. The information corresponding to the first input event can be text or voice. For example, if the user inputs "What did you say?" or "Say it again" to the robot via a mobile terminal or directly, the first input event can be considered detected. When the robot broadcasts response information, if the robot detects that the fourth user is increasing the device volume, for example, receiving a volume increase command and triggering the volume increase button, the volume increase event can be considered detected.

[0036] When the robot broadcasts response information, if it detects that a fourth user has moved away from the robot, a detected event of the fourth user moving away can be identified. For example, LiDAR or a vision camera can be used to identify whether the fourth user has moved away. The second input event can be an event that inputs information to instruct the robot to lower its volume. The information corresponding to the second input event can be text or voice. For example, if the user inputs "You're too noisy" or "Turn down" directly to the robot via a mobile terminal, it can be considered that a second input event has been detected. When the robot broadcasts response information, if it detects that the fourth user has lowered the device volume, for example, by receiving a volume-down command and triggering the volume-down button, it can be considered that a volume-down event has been detected. Optionally, the robot can also perform emotion recognition on the user using a vision camera. If it detects that the fourth user is exhibiting an unpleasant emotion (e.g., detecting the user frowning), it can be considered that a second input event has been detected.

[0037] In practical applications, when users cannot clearly understand the broadcast content based on the current volume, they often actively move closer to the robot, increase their device volume, actively tell the robot they cannot hear clearly, or request the robot to rebroadcast. In this embodiment, actively increasing the fifth broadcast volume based on the proximity event, the first input event, and the volume increase event can, to some extent, ensure that the broadcast volume can be increased in a timely manner. When users feel uncomfortable at the current broadcast volume, they often actively move away from the robot, actively decrease their device volume, or actively tell the robot the sound is too loud. In this embodiment, actively decreasing the fifth broadcast volume based on the distance event, the second input event, and the volume decrease event can, to some extent, ensure that the broadcast volume can be decreased in a timely manner.

[0038] Optionally, this application embodiment may further include: if the number of times the first feedback event is triggered within the current time window exceeds a first preset threshold, then the step of increasing the fifth broadcast volume of the fifth response information to the fifth voice information input by the fourth user at the same distance is executed; if the number of times the second feedback event is triggered within the current time window exceeds a second preset threshold, then the step of decreasing the fifth broadcast volume is executed.

[0039] In this embodiment, the window length of the current time window can be preset. For example, the window length of the current time window can be a week, a day, etc., and this embodiment does not limit this. The first preset number of times threshold and the second preset number of times threshold can be the same or different. The specific values ​​of the first preset number of times threshold and the second preset number of times threshold can be preset as needed, and this embodiment does not limit this.

[0040] The number of times the first feedback event is triggered within the current time window refers to the number of first feedback events detected within the current time window. If the number of times the first feedback event is triggered within the current time window exceeds a first preset threshold, the robot can determine that the broadcast volume is too low relative to the expected volume of the fourth user, and therefore, can execute the step of increasing the fifth broadcast volume. The number of times the second feedback event is triggered within the current time window refers to the number of second feedback events detected within the current time window. If the number of times the second feedback event is triggered within the current time window exceeds a second preset threshold, the robot can determine that the broadcast volume is too high relative to the expected volume of the fourth user, and therefore, can execute the step of decreasing the fifth broadcast volume. The first and second preset thresholds can be preset, and they can be the same or different; this embodiment does not impose any restrictions on this.

[0041] In this embodiment of the invention, the number of times the first feedback event is triggered and the number of times the second feedback event is triggered within the current time window are used as the basis for adjustment, which can avoid the problem of incorrect adjustment caused by a single erroneous triggering of the feedback event.

[0042] Optionally, embodiments of this application may further include: Step S31: In response to receiving the first instruction information from the fifth user, directly increase the sixth broadcast volume of the sixth response information to the sixth voice information input by the fifth user at the fifth distance; the first instruction information is used to instruct the robot to directly increase the subsequent broadcast volume.

[0043] Step S32: In response to receiving the second instruction information from the fifth user, directly reduce the volume of the sixth broadcast; the second instruction information is used to instruct the robot to directly reduce the volume of subsequent broadcasts.

[0044] In this embodiment, the fifth user can be any user using the robot, the sixth voice information can be any voice information input by the fifth user to the robot, and the sixth response information is the response information generated by the robot in response to the sixth voice information. The sixth response information can be generated based on the semantics and intent of the sixth voice information. When the robot outputs the sixth response information, it outputs it in the form of voice broadcast, and the sixth broadcast volume is the output volume used by the robot when broadcasting the sixth response information.

[0045] The fifth user can input the first or second instruction to the robot via voice, or via a mobile terminal to input the first or second instruction in the form of a command. For example, the fifth user can input "Speak louder next time" to the robot via a mobile terminal or directly to send the first instruction. The fifth user can input "Speak softer next time" to the robot via a mobile terminal or directly to send the second instruction.

[0046] Upon receiving the first instruction, the robot can directly increase the volume of its voice message output to the fifth user at the same distance. For example, the robot can adjust the distance-volume mapping so that it responds with a higher volume when subsequent users input voice messages at the same distance. Upon receiving the second instruction, the robot can decrease the volume of its voice message output to the same user at the same distance. For example, the robot can adjust the distance-volume mapping so that it responds with a lower volume when subsequent users input voice messages at the same distance.

[0047] In this embodiment, in response to receiving the first instruction from the fifth user, the sixth playback volume of the sixth response information to the sixth voice information input by the fifth user at the fifth distance is directly increased. In response to receiving the second instruction from the fifth user, the sixth playback volume is directly decreased. Thus, by directly increasing or decreasing the playback volume of the voice information input by the fifth user according to the first and second instruction information, the timeliness of volume adjustment can be ensured.

[0048] Optionally, embodiments of this application may further include: Step S41: In response to receiving voice information from the target user, determine the target distance between the target user and the robot.

[0049] Step S42: Based on the distance-volume mapping relationship, determine the volume corresponding to the target distance, and obtain the broadcast volume corresponding to the voice information.

[0050] In this embodiment, the target user can be any one of the first, second, third, fourth, and fifth users mentioned above; that is, the target user can be any user using the robot. The target distance refers to the distance between the target user and the robot. Upon receiving voice information input from the target user, the target distance can be determined first. In one implementation, the target distance can be measured using a lidar. For example, upon receiving voice information input from the target user, a lidar can be used to emit a laser pulse, and the product of the time it takes for the reflected light from the emitted laser pulse to the received laser pulse and the propagation speed of the laser pulse can be calculated. This product divided by 2 is taken as the target distance. In another implementation, the target distance can be measured using a binocular vision camera (e.g., an RGBD camera). For example, upon receiving voice information input from the target user, a binocular vision camera can be used to capture an image of the user, the parallax can be determined based on the user image, and the target distance can be determined based on the parallax. The larger the parallax, the larger the target distance.

[0051] The system can find the volume corresponding to the target distance from the distance-volume mapping relationship and use it as the playback volume for the voice information input by the target user. The robot's audio playback control module can adjust the speaker output gain according to the playback volume, allowing the robot to determine the appropriate playback volume output response information. After receiving voice information from the user each time, the system automatically performs distance estimation, determines the volume based on the distance, and adaptively adjusts parameters when the conditions for adjusting the distance-volume mapping relationship are met, preparing for the next playback.

[0052] In this embodiment, in response to receiving voice information from a target user, the target distance between the target user and the robot is determined. Based on the distance-volume mapping relationship, the volume corresponding to the target distance is determined, resulting in the corresponding broadcast volume. Thus, based on the target distance between the target user and the robot, the broadcast volume is dynamically determined, providing an appropriate broadcast volume for the target user and avoiding problems such as the user not being able to hear clearly or experiencing discomfort due to inappropriate volume.

[0053] Optionally, in another implementation, the step of determining the target distance between the target user and the robot in response to receiving voice information from the target user may specifically include: Step S411: In response to receiving the voice information of the target user, extract the voice features of the voice information, wherein the voice features are different when the distance between the target user and the robot is different.

[0054] Step S412: Determine the target distance based on the mapping relationship between speech features and distance, and the speech features.

[0055] In this embodiment, upon receiving voice information input by a target user, voice features of the input voice information can be extracted. The types of extracted voice features can be preset. Specifically, the extracted voice features can be distance-related features; the extracted voice features will differ depending on the distance between the target user and the robot.

[0056] The distance corresponding to the voice features extracted from the target user's input voice information can be found from the mapping relationship between voice features and distance, thereby determining the target distance. In this embodiment, voice features of the target user's voice information are extracted. The voice features differ depending on the distance between the target user and the robot, meaning the extracted voice features can reflect the distance between the target user and the robot. Therefore, based on the mapping relationship between voice features and distance, and the extracted voice features, the target distance can be accurately determined, thereby ensuring the accuracy of the broadcast volume used.

[0057] Optionally, the speech features include at least one of sound pressure level, direct reverberation ratio, and inter-channel phase difference variance. Since sound pressure level, direct reverberation ratio, and inter-channel phase difference variance are all distance-related, extracting at least one of these parameters in this embodiment can provide a valid basis for subsequent distance determination, thereby ensuring the reliability of the determined distance.

[0058] In this embodiment, the robot can extract valid speech segments from the voice information input by the target user. For example, valid speech segments can be extracted through voice activity detection. After extracting the valid speech segments, the root mean square (RMS) value of the valid speech signal can be calculated. It can be calculated in the following way:

[0059] in, s ( t ) represents the time-domain signal of the effective speech segment. N This indicates the number of sampling points corresponding to a valid speech segment.

[0060] Next, the root mean square value can be converted to obtain the sound pressure level extracted from the speech information. The higher the sound pressure level, the closer the target user is to the robot. Among these, sound pressure level... It can be calculated in the following way:

[0061] in, Indicates a reference sound pressure level, for example, The value can be taken as Pa.

[0062] When extracting the direct reverberation ratio, a short-time Fourier transform can be performed on the effective speech segment to obtain the frequency domain signal. Taking the logarithm of the spectrum and then performing an inverse Fourier transform yields the cepstral coefficients. :

[0063] in, k For frequency domain indexing, j The imaginary unit is used to indicate direction. For time domain / cepter domain indexes.

[0064] In the cepstral domain, the lower time-domain region corresponds to the overall spectral envelope of the signal (including direct sound features), while the higher time-domain region corresponds to the fine structure of the spectrum (including reverberation features). This can be achieved by setting a time-domain threshold. and The energy in the two regions is summed separately, and the direct reverberation ratio is calculated:

[0065] in, , As the time-domain threshold, at a sampling rate of 16kHz, Corresponding to 2ms, Corresponding to 10ms, This represents the energy reaching the direct sound region. It is the sum of the energy in the reverberation region. The higher the direct reverberation ratio, the closer the target user is to the robot.

[0066] When extracting the phase difference variance between channels, the speech information extracted from the left and right audio channels can be divided into multiple audio frames. A Fourier transform is performed on each audio frame to obtain the phase angle at each frequency point. The difference between the phase angles at each frequency point is calculated to obtain the phase difference corresponding to each frequency point. For any audio frame, the variance of the phase difference corresponding to the frequency points included in that audio frame is calculated. The average or median of the variances corresponding to all audio frames is determined to obtain the phase difference variance between channels. The smaller the phase difference variance between channels, the smaller the distance between the target user and the robot.

[0067] Optionally, in one implementation, the speech features include multiple features, and the step of determining the target distance based on the mapping relationship between speech features and distance, and the speech features, specifically includes: Step S412a: For any one of the multiple speech features, determine the distance corresponding to the speech feature based on the mapping relationship between the speech feature and the distance.

[0068] Step S41b: Determine the target distance based on the distances corresponding to the multiple speech features.

[0069] In this embodiment of the application, a mapping relationship between the speech feature and the distance can be set for each speech feature. For example, a mapping relationship between sound pressure level and distance, a mapping relationship between direct reverberation ratio and distance, and a mapping relationship between inter-channel phase difference variance and distance can be set.

[0070] For each extracted speech feature, the distance corresponding to that speech feature can be found from the mapping relationship between the corresponding type of speech feature and distance. Finally, by combining the multiple distances corresponding to the extracted speech features, the distance between the target user and the robot is determined, thus obtaining the target distance.

[0071] In this embodiment, multiple voice features are extracted from the voice information input by the target user. Based on these multiple voice features, multiple distances corresponding to the multiple voice features are first determined. The target distance is then determined based on these multiple distances, which can improve the reliability and accuracy of the target distance to a certain extent.

[0072] Optionally, in this embodiment of the application, determining the distance corresponding to the speech feature based on the mapping relationship between the speech feature and the distance includes: Step S412a1: Determine the feature information of the target user, wherein the feature information includes at least one of the following: image feature information and voiceprint information.

[0073] Step S412a2: Determine the mapping relationship between the speech features associated with the feature information and the distance, as the first mapping relationship; the first mapping relationship is constructed based on the speech features of the target user when inputting speech at at least two specified distances.

[0074] Step S412a3: Based on the first mapping relationship, determine the distance corresponding to the speech feature.

[0075] The feature information can be used to identify different users. For example, for any user, the speech features of the user inputting speech at at least two specified distances can be obtained, and a speech feature-distance mapping relationship can be constructed. The feature information of the user inputting speech at at least two specified distances can be extracted. For example, an image including the user can be captured, and the facial region in the image can be used as the user's image feature information. And / or, voiceprint extraction can be performed on the user's input speech, and the user's image feature information and / or voiceprint information can be associated with the speech feature-distance mapping relationship established for that user, thereby maintaining a personalized speech feature-distance mapping relationship for each user.

[0076] Taking sound pressure level (SPL) as an example, assuming users a and b are included, the SPL of the speech input by user a at at least two specified distances can be obtained. A mapping relationship between SPL and distance can be constructed, and this mapping relationship can be associated with and stored along with user a's feature information (image feature information and / or voiceprint information). Similarly, the SPL of the speech input by user b at at least two specified distances can be obtained, a mapping relationship between SPL and distance can be constructed, and this mapping relationship can be associated with and stored along with user b's feature information (image feature information and / or voiceprint information).

[0077] The system can extract the target user's feature information and compare it with feature information associated with multiple pre-created sound pressure level (SPL) and distance mapping relationships. If a matching feature is found, the identity recognition is successful, and the SPL and distance mapping relationship associated with the matching feature information can be used as the target user's SPL and distance mapping relationship (i.e., the first mapping relationship determined when determining the distance corresponding to the SPL). The distance corresponding to the SPL extracted from the target user's input voice information is then found from this first mapping relationship to obtain the distance corresponding to the SPL. If recognition fails, it indicates that the current target user is a new, unregistered user. Accordingly, the system's default general SPL and distance mapping relationship can be used as the first mapping relationship initially, and the user will be prompted to create a personalized SPL and distance mapping relationship later.

[0078] Taking direct reverberation ratio as an example, assuming users a and b are included, the direct reverberation ratio of the speech input by user a at at least two specified distances can be obtained. A mapping relationship between direct reverberation ratio and distance can be constructed, and this mapping relationship can be associated with and stored with user a's feature information. Similarly, the direct reverberation ratio of the speech input by user b at at least two specified distances can be obtained, and a mapping relationship between direct reverberation ratio and distance can be constructed, and this mapping relationship can be associated with and stored with user b's feature information.

[0079] The system can extract feature information from the target user and compare it with feature information associated with multiple pre-created direct reverberation ratio (RPR) and distance mapping relationships. The direct RPR and distance mapping relationship associated with matching feature information is used as the target user's direct RPR and distance mapping relationship (i.e., the first mapping relationship determined when determining the distance corresponding to the direct RPR). The distance corresponding to the direct RPR extracted from the target user's input speech information is then retrieved from this first mapping relationship to obtain the distance corresponding to the direct RPR.

[0080] The method for determining the distance corresponding to the phase difference variance between channels can refer to the methods for determining the distance corresponding to the sound pressure level and the distance corresponding to the direct reverberation ratio, as described above, and will not be repeated here.

[0081] In this embodiment, user feature information is used to distinguish the mapping relationship between voice features and distance for different users, and to maintain a personalized mapping relationship between voice features and distance for each user. This can, to a certain extent, more accurately determine the distance for different users and improve the accuracy of distance determination.

[0082] Optionally, the multiple speech features include sound pressure level (SPL) and direct reverberation ratio (DRR). Using SPL to determine distance is affected by the user's emotional state and habitual volume fluctuations, but it is highly accurate in close-range scenarios. Using DRR to determine distance is unaffected by speech volume and remains effective in long-range scenarios, but the difference in DRR measured at different distances in close-range scenarios is small, resulting in limited accuracy. In this embodiment, the SPL and DRR of the speech information input by the target user are extracted, and the distance determined based on SPL and DRR is fused to obtain the final target distance. SPL is highly accurate in close-range scenarios but is affected by user volume fluctuations, while DRR is unaffected by user volume and remains effective in long-range scenarios. The two are complementary; therefore, the target distance obtained after fusion is more accurate and robust than the distance determined based on a single speech feature.

[0083] When different users speak to a robot at the same distance, the sound pressure level may differ. If the same sound pressure level distance mapping is used for different users, the robot may determine different distances based on the extracted sound pressure levels. For example, suppose user A is accustomed to speaking loudly, while user B is accustomed to speaking softly. When users A and B speak to the robot at the same distance, the sound pressure level extracted by the robot from user A's speech input will be greater than the sound pressure level extracted from user B's speech input. Therefore, the distance determined for user A will be less than the distance determined for user B. In this embodiment, a sound pressure level (SPL) and distance mapping relationship is constructed based on the SPL when the target user inputs speech at at least two specified distances. This ensures that the SPL and distance mapping relationship associated with the target user can adapt to the target user's speaking habits. It also personalizes the SPL and distance mapping relationship for different users, eliminating the interference of differences in speaking habits that could lead to deviations in the distance determined based on SPL. This ensures that when different users input speech information at the same distance with different volumes, the distance determined based on SPL is the same, thus accurately estimating the distance for different users in the same environment.

[0084] Optionally, the embodiments of this application may further include the following steps: Step S51: For each of at least two specified distances, if the distance between the target user and the robot reaches the specified distance, obtain the calibration voice information input by the target user at the specified distance.

[0085] Step S52: Based on the at least two specified distances and the calibrated speech information corresponding to the at least two specified distances, determine the mapping relationship between the speech features and the distances.

[0086] The specified distances can be preset according to actual needs. For example, at least two specified distances may include 1m, 3m, and 5m, and this application embodiment does not impose any limitations on this. When the target user reaches a specified distance, the calibration voice information input by the target user at that specified distance can be obtained. The calibration voice information can be voice information containing specified content; for example, the calibration voice information can be "Hello, Xiao X".

[0087] In this embodiment, the user inputs the same calibration voice information at each specified distance location, which can eliminate the impact of differences in voice content and further ensure the accuracy of the established mapping relationship.

[0088] Since the specified distance is known, the speech features of the speech input by the target user at that specified distance can be obtained based on the calibrated speech information input by the user at that specified distance. Accordingly, based on at least two specified distances and the corresponding calibrated speech information, an accurate mapping relationship between speech features and distances can be established for the target user, thereby ensuring the accuracy of the distance determined using this mapping relationship.

[0089] Optionally, this embodiment may further include: responding to receiving a trigger operation on the calibration option or receiving a voice command indicating the start of calibration, entering the process of constructing the mapping relationship between the voice features and the distance. The calibration option can be provided by the robot's corresponding control software. For example, the calibration option can be a button provided on the control software, and the trigger operation on the calibration option can be a single click, a long press, etc., on the button; this embodiment does not limit this. The voice command indicating the start of calibration can be a voice command indicating the start of establishing a mapping relationship between voice features and distance. For example, taking sound pressure level as an example, the user can input "Please establish a sound pressure level and distance mapping relationship for me," and the robot can confirm receipt of the voice command indicating the start of calibration. Then, step S51 is executed to enter the process of constructing the mapping relationship between voice features and distance. In this embodiment, the user can conveniently control the robot to establish a personalized mapping relationship between voice features and distance based on the trigger operation on the calibration option or by inputting a voice command indicating the start of calibration, thus ensuring user convenience to a certain extent.

[0090] It should be noted that, in one implementation, a default direct-to-reverberation ratio (DRR) and distance mapping relationship can be associated with each user. When a user speaks, the sound received by the robot microphone consists of two parts: direct sound and reverberation. Direct sound refers to the sound that travels directly from the user's mouth to the microphone, with a short propagation path. Reverberation refers to the sound that travels after being reflected by walls, ceilings, furniture, etc., before reaching the microphone, with a long propagation path. When the user moves away from the robot, the direct sound attenuates significantly, while the reverberation remains relatively unchanged. Therefore, the ratio of direct sound energy to reverberation energy, i.e., the direct-to-reverberation ratio, decreases monotonically with increasing distance. The direct-to-reverberation ratio is only related to distance and is not affected by the volume of the user's speech. Therefore, in this embodiment, a default direct-to-reverberation ratio and distance mapping relationship can be associated with each user. For example, the direct reverberation ratio of audio data at known distances can be collected offline. Based on multiple known distances and the direct reverberation ratio of audio data at known distances, a mapping relationship between the direct reverberation ratio and distance is fitted and established. This mapping relationship is applied to each user. For example, the mapping relationship between the direct reverberation ratio and distance can be expressed as: Calibration can be completed for typical indoor scenarios before leaving the factory. Specifically, based on multiple known distances and the direct reverberation ratio of audio data at those known distances, the coefficients in the direct reverberation ratio-distance mapping relationship can be calculated before shipment. , ,Will , As a fixed system parameter, it does not require calibration on the user side.

[0091] In one implementation, a default mapping relationship between inter-channel phase difference variance and distance can be associated with each user. Correspondingly, the sound pressure level and distance mapping relationship can be constructed simply by following the aforementioned process for constructing the mapping relationship between speech features and distance. Alternatively, the aforementioned process for constructing the mapping relationship between speech features and distance can be used to construct the direct reverberation ratio and distance mapping relationship, and the inter-channel phase difference variance and distance mapping relationship; this application does not impose any limitations on this approach.

[0092] Optionally, in this embodiment of the application, the step of determining the mapping relationship between the speech features and the distances based on the at least two specified distances and the calibrated speech information corresponding to the at least two specified distances specifically includes: Step S521: Extract the speech features of the calibration speech information as sample speech features; the sample speech features and the specified distance form a calibration data point.

[0093] Step S522: Based on multiple calibration data points, determine the mapping relationship between the voice features and distance, and associate it with the feature information of the target user.

[0094] For any given specified distance among multiple specified distances, the speech features of the calibration speech information input by the target user at that specified distance can be extracted to obtain the sample speech features corresponding to that specified distance. These sample speech features, along with the specified distance itself, are used as a calibration data point. Multiple calibration data points can be obtained for multiple specified distances.

[0095] Optionally, this embodiment may further include: outputting a first prompt message; the first prompt message is used to prompt the target user to proceed to the specified distance and input the calibration voice information to the robot. In response to detecting that the target user has reached the specified distance, outputting a second prompt message; the second prompt message is used to prompt the target user that they have now reached the specified distance.

[0096] After establishing the mapping relationship between voice features and distance, the system can sequentially output first and second prompts for each specified distance. The first prompt can specifically guide the target user to the specified distance and input calibration voice information multiple times for the robot, thus avoiding the need to provide sufficient voice information for the calibration process. For example, the prompts can be given sequentially from near to far or from far to near. Taking the near-to-far order as an example, the system begins in response to a trigger operation on the calibration option or a voice command indicating the start of calibration. The robot can first announce the first prompt: "Please say 'Hello, Xiao X' to me at 1m, and repeat 5 times." When the user reaches 1m from the robot, facing the robot, they can repeat the calibration voice information "Hello, Xiao X" 5 times. After receiving the calibration voice information input by the user at 1m, the system can continue to announce the first prompt for the next specified distance: "Please say 'Hello, Xiao X' to me at 3m, and repeat 5 times." The user can then reach 3m from the robot, facing the robot, and repeat the calibration voice information "Hello, Xiao X" 5 times. After receiving the calibration voice information input by the user at 3m, the system can continue to broadcast the first prompt message for the next specified distance: "Please say 'Hello, Xiao X' to me at 5m, and repeat 5 times." The user can repeat the calibration voice information "Hello, Xiao X" 5 times while facing the robot at a distance of 5m.

[0097] In one implementation, the user can independently determine the distance. In another implementation, the user can measure the current distance between the user and the robot using LiDAR or a vision camera. If the current distance equals a specified distance, and the specified distance corresponds to the first prompt message output, a second prompt message is output to indicate that the target user has reached that specified distance. In this embodiment, by outputting the first prompt message and then outputting the second prompt message when the target user is detected to have reached the specified distance, clear operational guidance is provided to the user, thus improving the convenience of user operation.

[0098] Taking the construction of a sound pressure level-distance mapping relationship as an example, after each received calibration speech information, speech activity detection, effective speech segment extraction, and root mean square value calculation can be performed to obtain the sound pressure level corresponding to each specified distance.

[0099] During sound wave propagation, the sound pressure level decreases with increasing propagation distance. Under free field conditions, the sound pressure level has a linear relationship with the logarithm of distance:

[0100] in, Sound pressure level, measured in dB. For reference distance, the unit is meters, which can be measured using a sound pressure meter. Sound pressure is obtained The distance can be calculated using the above formula. sound pressure value .

[0101] In actual indoor environments, due to the presence of reflective surfaces such as walls, floors, and furniture, sound wave propagation is not an ideal free field. In this case, the sound pressure level and the logarithmic distance still maintain an approximately linear relationship, with only the attenuation coefficient depending on the actual environment. Therefore, in this embodiment, the following semi-logarithmic linear regression model can be used to represent the mapping relationship between sound pressure level and distance:

[0102] in, The distance between the user and the robot, in meters. For model intercept, The model slope, i.e., the sound attenuation coefficient in the actual environment, is used to reflect the sound propagation characteristics in the real environment. It can be solved using multiple calibration data points. , Therefore, when using the sound pressure level and distance mapping relationship later, the sound pressure level of the target user's input speech information will be used as... Substituting this sound pressure level into the distance mapping relationship, due to the fact that... , Given this, we can deduce the distance between the user and the robot. That is, the distance corresponding to the sound pressure level.

[0103] In this embodiment of the application, the solution can be obtained using the least squares method. , .make ， This transforms the sound pressure level-distance mapping relationship into a linear regression equation: .in, This represents the independent variable in the linear regression equation. This represents the dependent variable in a linear regression equation.

[0104] Assumptions include One calibration data point: ,correspond One calibration data point can be obtained Group of known data points: .this A set of known data points can be represented as: ,in, , , ∈[1, ], Indicates the first A specified distance among calibration data points, Indicates the first Sample speech features from each calibration data point.

[0105] Define the loss function as the sum of squared residuals: .right and Taking the partial derivatives of each and setting them to zero, we obtain the least squares solution: , .

[0106] in: , , express The mean, express The mean.

[0107] In this embodiment, the target user's feature information can also be extracted from the received calibration voice information. The established sound pressure level-distance mapping relationship is then associated and stored with the target user's feature information to achieve binding. When it is subsequently necessary to determine the distance corresponding to the target user's sound pressure level, the established sound pressure level-distance mapping relationship can be invoked based on the target user's feature information.

[0108] In this embodiment, speech features of the calibrated speech information are extracted as sample speech features, and the sample speech features and a specified distance constitute a calibration data point. Based on multiple calibration data points, the mapping relationship between speech features and distance can be determined, and the constructed mapping relationship between speech features and distance is associated with the feature information of the target user. This ensures that in multi-user scenarios, each user has an independent mapping relationship between speech features and distance, providing a personalized and accurate basis for subsequent distance recognition.

[0109] Compared to using fixed coefficients that rely on free field theory formulas, in this embodiment, the calibration voice information input by the target user at a specified distance from the robot in the environment is obtained. Multiple calibration data points are generated based on the calibration voice information. Based on the multiple calibration data points, the coefficients in the sound pressure level-distance mapping relationship of the target user are calculated, so that the constructed sound pressure level-distance mapping relationship can automatically adapt to the characteristics of the indoor environment and the user's personal speaking volume.

[0110] In one implementation, the step of determining the target distance based on the distances corresponding to the multiple speech features may specifically include: using the distances corresponding to the multiple speech features as the input of a preset Kalman filter, and using the output value of the Kalman filter as the target distance.

[0111] In this embodiment, the distances corresponding to multiple speech features can be combined into an observation vector. For example, taking the distances corresponding to multiple speech features as including the distance corresponding to the sound pressure level extracted from the speech information input by the target user, and the distance corresponding to the direct-to-reverberation ratio extracted from the speech information input by the target user, after the robot enters the normal interaction phase, it can continuously collect the speech signal of the target user. Whenever a valid speech segment is received and detected, the distances corresponding to these two speech features are first determined in parallel based on the sound pressure level and direct-to-reverberation ratio of the valid speech segment, and finally, the target distance is determined by Kalman filtering fusion.

[0112] by Representing the current moment, the observation vector can be... ,in for Distance determined by sound pressure level at all times. for The distance value is determined based on the direct reverberation ratio at any given moment. Both directly reflect the current distance between the target user and the robot, but not the rate of change of distance. The observation vector is used as the input to a Kalman filter, which determines the target distance using the Kalman filtering algorithm. The specific implementation of the Kalman filtering algorithm can refer to related technologies, and this application does not limit it.

[0113] For example, in a Kalman filter, the state vector is defined as consisting of the current distance and the rate of change of distance:

[0114] in, Indicates the current distance. This represents the rate of change of distance.

[0115] The Kalman filter takes distances determined by different speech features as input, combines them with a motion model of distance changing over time, and outputs the fused target distance. After the Kalman filter update is complete, the final calculated result can be... The first dimension This serves as the target distance obtained after fusion.

[0116] Assuming the user's motion is uniform, the state transition equation can be expressed as:

[0117] in, , The time interval between two adjacent distance estimation operations.

[0118] Kalman filtering is divided into a prediction phase and an update phase. In the prediction phase, the state vector from the previous time step (…) can be used as a reference. ) and motion model ( ), predict the current state vector ( In the Kalman filter algorithm, each calculation uses the state vector and motion model from the previous moment to predict the state vector at the current moment, which can output a smooth and continuous target distance through multiple estimations.

[0119] Meanwhile, the prediction is based on the error covariance matrix of the previous time step ( ),Sure Predicted values ​​of the error covariance matrix at time: .

[0120] in, This is the process noise covariance matrix, which is calibrated during algorithm training.

[0121] During the update phase, the Kalman gain can be calculated: .

[0122] Next, the error covariance matrix can be updated: .

[0123] in, The observation matrix, used to map the state to the observation space, can be a constant matrix vector. , It is an identity matrix.

[0124] Among them, the Kalman gain can reflect which is more reliable, the observed value or the predicted value. The noise covariance matrix was calibrated during algorithm training to observe it.

[0125] The Kalman filter can correct the predicted value (the state vector at the current moment) based on the actual observations (i.e., the observation vector), obtaining the optimal state estimate at the current moment and thus the final output. For example, the output can be represented as: .

[0126] Figure 2 This is a schematic diagram of a processing flow provided in an embodiment of this application, such as... Figure 2As shown, the robot first uses a microphone to pick up the voice information input by the target user, and performs front-end acoustic processing to obtain the sound pressure level (SPL) and direct reverberation ratio (DRR) of the target user's voice information. Next, a distance calculation module determines the distance corresponding to each SPL and DRR in parallel. For example, the distance calculation module determines the distance corresponding to the SPL based on the mapping relationship between SPL and distance associated with the target user's voiceprint information, and the SPL extracted from the target user's voice information. Based on the mapping relationship between DRR and distance, and the DRR extracted from the target user's voice information, the module determines the distance corresponding to the DRR. Kalman fusion is performed on the distance corresponding to the SPL and the distance corresponding to the DRR to obtain the target distance. Based on the target distance, volume is calculated, and the speaker is controlled to output volume according to the target distance.

[0127] In this implementation, the distances corresponding to multiple speech features are fused into a target distance using Kalman filtering. Because the Kalman gain in the Kalman filtering algorithm is dynamically updated, the weights of the distances corresponding to each speech feature are dynamically adjusted during the fusion process, thus ensuring the accuracy of the target distance obtained through fusion.

[0128] In this embodiment, distance estimation does not require additional hardware such as ultrasonic, infrared, or visual sensors, thus reducing hardware implementation costs.

[0129] Alternatively, in another implementation, the step of determining the target distance based on the distances corresponding to the multiple speech features may specifically include: using the distances corresponding to the multiple speech features as input to a preset distance fusion model, and using the output value of the distance fusion model as the target distance.

[0130] In this implementation, the distance fusion model can be pre-trained. For example, input speech at a preset distance can be used as sample speech. The distances corresponding to each speech feature of the sample speech are determined and used as sample distances. Multiple sample distances corresponding to multiple speech features of the sample speech are used as input to the fusion model to be trained. Based on the output of the fusion model to be trained and the preset distances, the model loss is calculated. The options for the fusion model to be trained can be selected as needed. For example, a neural network model can be selected. The larger the model loss, the greater the deviation between the current output of the fusion model to be trained and the preset distance. The fusion model to be trained is adjusted based on the model loss, and training continues until a preset termination condition is reached. The model loss can be calculated using cross-entropy loss function, mean squared error function, etc., and this embodiment does not limit this. The preset termination condition can be that the number of training epochs reaches a preset epoch threshold, or that the model loss is less than a preset loss threshold, etc., and this embodiment does not limit this.

[0131] In this implementation, the distances corresponding to multiple voice features extracted from the voice information input by the target user are input into a preset distance fusion model to obtain the target distance, which has a high efficiency in obtaining the target distance.

[0132] Alternatively, in another implementation, the step of determining the target distance based on the distances corresponding to multiple speech features may specifically include: performing a weighted summation based on the weights corresponding to each speech feature and the distances corresponding to each speech feature to obtain the target distance.

[0133] The weights for each speech feature can be pre-set, and these weights can be fixed or dynamic. For example, taking multiple speech features including sound pressure level (SPL) and direct reverberation ratio (DRR), the weight corresponding to the SPL can be adaptively determined based on its magnitude. For example, since SPL-based distance estimation is more accurate in close-range scenarios, a negatively correlated linear function can be used, where a larger output value is assigned as the weight for a smaller SPL. Similarly, the weight corresponding to the DRR can be adaptively determined based on its magnitude. For example, since DRR-based distance estimation has limited accuracy in close-range scenarios, a positively correlated linear function can be used, where a larger output value is assigned as the weight for a larger DRR.

[0134] The target distance can be obtained by calculating the product of the weight corresponding to each speech feature and the distance corresponding to that speech feature, and then summing the products. In this implementation, the target distance is obtained by weighted summation based on the weights and distances corresponding to each speech feature. The computational complexity is low, and the implementation cost is low.

[0135] Optionally, in this embodiment of the application, the step of determining the volume corresponding to the target distance based on the distance-volume mapping relationship may specifically include: Step S42a: Determine the feature information of the target user, wherein the feature information includes at least one of the following: image feature information and voiceprint information.

[0136] Step S42b: Determine the distance-volume mapping relationship associated with the feature information.

[0137] Step S42c: Based on the distance-volume mapping relationship associated with the feature information, determine the volume corresponding to the target distance.

[0138] In this embodiment, initially, a default distance-volume mapping relationship can be associated with each user. The default distance-volume mapping relationship can be set as needed, and this embodiment does not impose any restrictions on it. For example, the default distance-volume mapping relationship can adopt a piecewise linear mapping strategy, dividing the distance to the robot into three intervals, each interval using a different volume mapping rule. The default distance-volume mapping relationship can be expressed as:

[0139] in, The output value represents the distance-volume mapping relationship, i.e., the volume corresponding to the target distance, with a value range of 0 to 100. This is the lower limit threshold for distance. This is the upper limit threshold for distance; when the target distance is within... and During this period, the volume increases linearly with distance, and the volume change is smooth and continuous, ensuring that the user will not feel a sudden change in volume when moving. This achieves automatic and continuous adaptive adjustment of the broadcast volume according to the change in distance between the user and the robot. The minimum volume threshold is set so that when the target distance is too close (the target distance is below the minimum distance threshold), the user is in the robot's close range. The output is maintained at the minimum volume threshold, which can ensure that the user can hear clearly while avoiding discomfort such as ear-piercing caused by excessive volume at close range. The maximum volume threshold is set so that when the target distance is too far (exceeding the upper limit of the distance threshold), the user is too far from the robot. The output can be maintained at the maximum volume threshold to ensure the user can still hear the content clearly, while avoiding excessive volume. Of course, in one implementation, the target distance and... , Based on the size relationship, directly substitute the target distance into the equation. The volume corresponding to the target distance is obtained, but this application embodiment does not limit this. , , , The initial value can be set by the factory configuration, for example, , , , The initial values ​​can be: 0.5m, 3m, 20, 80.

[0140] In this embodiment, the target distance is mapped to the volume by the distance-volume mapping relationship, so that the volume is adaptively adjusted as the distance between the user and the robot changes. The volume is reduced when the user is close to avoid causing discomfort to the user, and increased when the user is far away to ensure that the user can hear clearly.

[0141] In this embodiment, the parameters in the distance-volume mapping relationship can be adjusted based on the first feedback event and the second feedback event of the user, so that the distance-volume mapping relationship associated with the user is personalized and adapted to the user.

[0142] In this embodiment, the feature information of the target user can be compared with the feature information associated with various distance-volume mapping relationships. The target distance is obtained by substituting the target distance into the distance-volume mapping relationship associated with the matched feature information.

[0143] In this embodiment, the characteristic information of the target user is determined. A distance-volume mapping relationship associated with the characteristic information is determined. The distance-volume mapping relationship is maintained individually for the target user. Using the distance-volume mapping relationship associated with the target user's characteristic information, the volume corresponding to the target distance is determined, which to a certain extent ensures the adaptability of the final broadcast volume to the target user.

[0144] Optionally, in this embodiment of the application, increasing the volume of the fifth response information to the fifth voice information input by the fourth user at the same distance may specifically include: Step S21a: Increase the first parameter in the distance-volume mapping relationship associated with the feature information of the fourth user, and / or decrease the second parameter in the distance-volume mapping relationship to increase the volume of the fifth broadcast; the output value of the distance-volume mapping relationship is positively correlated with the first parameter and negatively correlated with the second parameter.

[0145] Decreasing the volume of the fifth announcement may specifically include: step S22a, decreasing the first parameter, and / or increasing the second parameter to decrease the volume of the fifth announcement.

[0146] In this embodiment, the distance-volume mapping relationship, which is consistent with the feature information of the fourth user, can be used as the adjustment object. If it is determined that the broadcast volume is too low relative to the expected volume of the fourth user, it can be considered that the volume determined by the current distance-volume mapping relationship is too low. Therefore, a parameter adjustment operation can be performed: increasing the first parameter and / or decreasing the second parameter, so that the volume determined by the distance-volume mapping relationship is increased in the future, thereby improving the broadcast volume when responding to the input voice information at the same distance.

[0147] If the broadcast volume is determined to be too high relative to the expected volume of the fourth user, it can be assumed that the volume determined by the current distance-volume mapping relationship is too high. Therefore, parameter adjustment operations can be performed: decrease the first parameter and / or increase the second parameter, so that the volume determined by the distance-volume mapping relationship is reduced in the future, thereby reducing the broadcast volume when responding to the input voice information at the same distance.

[0148] By adapting and adjusting the parameters in the distance-volume mapping relationship, the distance-volume mapping relationship can be dynamically adjusted according to the user's auditory preferences and the actual usage environment. This enables automatic, continuous, and personalized adaptive adjustment of the broadcast volume as the user's distance changes, significantly improving the user's voice interaction experience in long-distance scenarios while avoiding the discomfort caused by close-range broadcasts.

[0149] In this embodiment of the application, by increasing the first parameter and / or decreasing the second parameter in the distance-volume mapping relationship associated with the fourth user's feature information, or by decreasing the first parameter and / or increasing the second parameter, the playback volume when responding to the input voice information at the same distance can be dynamically adjusted according to the user's interactive feedback.

[0150] Optionally, increasing the first parameter in the distance-volume mapping relationship associated with the feature information of the fourth user, and / or decreasing the second parameter in the distance-volume mapping relationship, may specifically include: increasing the maximum volume threshold, decreasing the upper distance threshold, and / or decreasing the lower distance threshold in the distance-volume mapping relationship associated with the feature information of the fourth user; the output value changes linearly within the range formed by the upper distance threshold and the lower distance threshold, and the maximum volume threshold is positively correlated with the output value.

[0151] The aforementioned reduction of the first parameter and / or increase of the second parameter may specifically include: reducing the maximum volume threshold, increasing the upper distance threshold, and / or increasing the lower distance threshold.

[0152] In this embodiment, the parameters in the distance-volume mapping relationship are dynamically adjusted based on user interaction feedback during robot use, thereby adapting to different users' auditory preferences and usage scenarios, and achieving personalized adaptation of the distance-volume mapping relationship. Specifically, the first parameter may include the above-mentioned... The second parameter may include the above. as well as This avoids adjusting the minimum volume threshold, thus preventing the robot from broadcasting at too low a volume.

[0153] By increasing , reduce This can increase the slope of the distance-volume mapping, making the volume change steeper and improving the playback volume when responding to input voice information at the same distance. By reducing , making Increasing the difference can increase the final calculated value, thereby increasing the playback volume when responding to input speech information at the same distance. By reducing Increase This can reduce the slope of the distance-volume mapping, making volume changes smoother and lowering the playback volume when responding to input voice information at the same distance. By increasing... , making The difference decreases, which in turn reduces the final calculated value, thereby reducing the broadcast volume when responding to input voice information at the same distance.

[0154] When adjusting parameters, the adjustment range can be preset, and adjustments can be made in steps. A preset step size is added to the parameter, resulting in the adjusted parameter. For example, .

[0155] in, For the current , For the adjusted , For example, to preset the step size, It can be 5.

[0156] Figure 3 This is a block diagram of a voice processing device provided in an embodiment of this application. The device includes: Memory 201 and processor 202; Memory 201 is used to store program code; Processor 202 calls program code, which, when executed, performs the following operations: In response to receiving first voice information input by a first user, first response information is played to the first user at a first playback volume; the first voice information is input at a first distance. In response to receiving the second voice information input by the first user, the system plays the second response information to the first user at a second playback volume; the second voice information is input at a second distance, the second distance being greater than the first distance, and the second playback volume being greater than the first playback volume.

[0157] In summary, in the voice processing device provided in this application embodiment, in response to receiving first voice information input by a first user, first response information is played to the first user at a first playback volume; the first voice information is input at a first distance. In response to receiving second voice information input by the first user, second response information is played to the first user at a second playback volume; the second voice information is input at a second distance, where the second distance is greater than the first distance, and the second playback volume is greater than the first playback volume. Thus, based on the distance between the user and the robot, the playback volume is dynamically adjusted, using a smaller playback volume when the user is closer to the robot and a larger playback volume when the user is farther from the robot, thereby avoiding the problem of the user not being able to hear clearly or feeling uncomfortable due to inappropriate volume.

[0158] Optionally, the processor 202 is further configured to perform: In response to receiving a third voice message input by a second user at a third distance, a third response message is played to the second user at a third playback volume; in response to receiving a fourth voice message input by a third user at the third distance, a fourth response message is played to the third user at a fourth playback volume; the fourth playback volume is different from the third playback volume, or the fourth playback volume is the same as the third playback volume.

[0159] Optionally, if the input volume of the third voice information is different from the input volume of the fourth voice information, and the second user and the third user adopt the same distance and volume mapping relationship, the fourth broadcast volume is the same as the third broadcast volume; When the second user and the third user have different distance and volume mapping relationships, the fourth broadcast volume is different from the third broadcast volume.

[0160] Optionally, the processor 202 is further configured to perform: In response to a first feedback event from a fourth user, the robot increases the volume of its fifth response to the fifth voice information input by the fourth user at the same distance, wherein the first feedback event indicates that the robot's volume is lower than the fourth user's expected volume; or in response to a second feedback event from a fourth user, the robot decreases the volume of its fifth response, wherein the second feedback event indicates that the volume is higher than the expected volume.

[0161] Optionally, the processor 202 is further configured to perform: In response to receiving voice information from a target user, the target distance between the target user and the robot is determined; Based on the distance-volume mapping relationship, the volume corresponding to the target distance is determined, and the broadcast volume corresponding to the voice information is obtained.

[0162] Optionally, the processor 202 is further configured to perform: In response to receiving voice information from the target user, the voice features of the voice information are extracted, wherein the voice features differ depending on the distance between the target user and the robot. The target distance is determined based on the mapping relationship between speech features and distance, as well as the speech features themselves.

[0163] Optionally, the speech features include at least one of sound pressure level, direct reverberation ratio, and inter-channel phase difference variance.

[0164] Optionally, the voice features include multiple features; the processor 202 is further configured to perform: For any one of the multiple speech features, the distance corresponding to the speech feature is determined based on the mapping relationship between the speech feature and the distance; The target distance is determined based on the distances corresponding to multiple of the aforementioned speech features.

[0165] Optionally, the processor 202 is further configured to perform: Determine the feature information of the target user, wherein the feature information includes at least one of the following: image feature information, voiceprint information; A mapping relationship between the speech features associated with the feature information and the distance is determined as a first mapping relationship; the first mapping relationship is constructed based on the speech features of the target user when the user inputs speech at at least two specified distances. Based on the first mapping relationship, the distance corresponding to the speech feature is determined.

[0166] Optionally, the processor 202 is further configured to perform: The distances corresponding to multiple speech features are used as inputs to a preset Kalman filter, and the output value of the Kalman filter is used as the target distance; or, The distances corresponding to multiple of the stated speech features are used as input to a preset distance fusion model, and the output value of the distance fusion model is used as the target distance; or, The target distance is obtained by weighted summation based on the weights and distances corresponding to each of the aforementioned speech features.

[0167] Optionally, the speech features include sound pressure level and direct reverberation ratio.

[0168] Optionally, the processor 202 is further configured to perform: Determine the feature information of the target user, wherein the feature information includes at least one of the following: image feature information, voiceprint information; Determine the distance-volume mapping relationship associated with the feature information; Based on the distance-volume mapping relationship associated with the feature information, the volume corresponding to the target distance is determined.

[0169] Optionally, the processor 202 is further configured to perform: For each of at least two specified distances, if the distance between the target user and the robot reaches that specified distance, obtain the calibration voice information input by the target user at that specified distance; Based on the at least two specified distances and the calibrated speech information corresponding to the at least two specified distances, the mapping relationship between the speech features and the distances is determined.

[0170] Optionally, the processor 202 is further configured to perform: The speech features of the calibration speech information are extracted as sample speech features; the sample speech features and the specified distance form a calibration data point; Based on multiple calibration data points, the mapping relationship between the voice features and distance is determined and associated with the feature information of the target user.

[0171] Optionally, the processor 202 is further configured to perform: In response to receiving a trigger operation on the calibration option or receiving a voice command indicating the start of calibration, the process of constructing the mapping relationship between the voice features and the distance is initiated.

[0172] Optionally, the processor 202 is further configured to perform: Output a first prompt message; the first prompt message is used to prompt the target user to go to the specified distance and to input the calibration voice information for the robot; In response to detecting that the target user has reached the specified distance, a second prompt message is output; the second prompt message is used to indicate to the target user that they have now reached the specified distance.

[0173] Optionally, the processor 202 is further configured to perform: If the number of times the first feedback event is triggered within the current time window exceeds the first preset threshold, then the step of increasing the fifth broadcast volume of the fifth response information to the fifth voice information input by the fourth user at the same distance is executed. If the number of times the second feedback event is triggered within the current time window exceeds the second preset threshold, then the step of reducing the volume of the fifth broadcast is executed.

[0174] Optionally, the processor 202 is further configured to perform: In response to receiving the first instruction information from the fifth user, the robot directly increases the volume of the sixth broadcast of the sixth response information to the sixth voice information input by the fifth user at the fifth distance; the first instruction information is used to instruct the robot to directly increase the volume of subsequent broadcasts. In response to receiving the second instruction information from the fifth user, the robot directly reduces the volume of the sixth broadcast; the second instruction information is used to instruct the robot to directly reduce the volume of subsequent broadcasts.

[0175] Optionally, the first feedback event is any one of the following: the robot detecting the fourth user's proximity event, the fourth user triggering a first input event, and the volume increase event; the second feedback event is any one of the robot detecting the fourth user's distance event, the fourth user triggering a second input event, and the volume decrease event.

[0176] Optionally, the processor 202 is further configured to perform: Increase the first parameter in the distance-volume mapping relationship associated with the feature information of the fourth user, and / or decrease the second parameter in the distance-volume mapping relationship to increase the volume of the fifth broadcast; the output value of the distance-volume mapping relationship is positively correlated with the first parameter and negatively correlated with the second parameter; Reducing the volume of the fifth broadcast includes: decreasing the first parameter and / or increasing the second parameter to reduce the volume of the fifth broadcast.

[0177] Optionally, the processor 202 is further configured to perform: Increase the maximum volume threshold, decrease the upper distance threshold, and / or decrease the lower distance threshold in the distance-volume mapping relationship associated with the feature information of the fourth user; the output value changes linearly within the range formed by the upper distance threshold and the lower distance threshold, and the maximum volume threshold is positively correlated with the output value; Decreasing the first parameter and / or increasing the second parameter includes: decreasing the maximum volume threshold, increasing the upper distance threshold, and / or increasing the lower distance threshold.

[0178] This application provides a robot, including a processor, a memory, a communication interface, and a communication bus. The processor, memory, and communication interface communicate with each other via the communication bus; the memory stores at least one executable instruction, which causes the processor to execute the speech processing method described in the preceding embodiments. The executable instructions can form a program.

[0179] This application provides a readable storage medium storing instructions that, when executed by one or more processors, enable the processors to perform the speech processing method described in the foregoing embodiments. The readable storage medium can be one or more.

[0180] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the speech processing method described in the foregoing embodiments.

[0181] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0182] Those skilled in the art will understand that embodiments of this application may provide a voice processing method, apparatus, or computer program product. Therefore, embodiments of this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0183] It should be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0184] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0185] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors can be used in practice to implement some or all of the functions of some or all of the components in the control terminal according to the embodiments of this application. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0186] This disclosure provides an embodiment, including the following examples.

[0187] Example 1: A speech processing method applied to a robot, the method comprising: In response to receiving first voice information input by a first user, first response information is played to the first user at a first playback volume; the first voice information is input at a first distance. In response to receiving the second voice information input by the first user, the system plays the second response information to the first user at a second playback volume; the second voice information is input at a second distance, the second distance being greater than the first distance, and the second playback volume being greater than the first playback volume.

[0188] Example 2: According to the method described in Example 1, the method further includes: In response to receiving a third voice message input by a second user at a third distance, a third response message is played to the second user at a third playback volume; in response to receiving a fourth voice message input by a third user at the third distance, a fourth response message is played to the third user at a fourth playback volume; the fourth playback volume is different from the third playback volume, or the fourth playback volume is the same as the third playback volume.

[0189] Example 3: According to the method described in Example 2, when the input volume of the third voice information is different from the input volume of the fourth voice information, and the second user and the third user adopt the same distance and volume mapping relationship, the fourth broadcast volume is the same as the third broadcast volume. When the second user and the third user have different distance and volume mapping relationships, the fourth broadcast volume is different from the third broadcast volume.

[0190] Example 4: According to the method described in Example 1, the method further includes: In response to a first feedback event from a fourth user, the robot increases the volume of its fifth response to the fifth voice information input by the fourth user at the same distance, wherein the first feedback event indicates that the robot's volume is lower than the fourth user's expected volume; or in response to a second feedback event from a fourth user, the robot decreases the volume of its fifth response, wherein the second feedback event indicates that the volume is higher than the expected volume.

[0191] Example 5: The method described according to any one of Examples 1-4, the method further comprising: In response to receiving voice information from a target user, the target distance between the target user and the robot is determined; Based on the distance-volume mapping relationship, the volume corresponding to the target distance is determined, and the broadcast volume corresponding to the voice information is obtained.

[0192] Example 6. According to the method described in Example 5, the step of determining the target distance between the target user and the robot in response to receiving voice information from the target user includes: In response to receiving voice information from the target user, the voice features of the voice information are extracted, wherein the voice features differ depending on the distance between the target user and the robot. The target distance is determined based on the mapping relationship between speech features and distance, as well as the speech features themselves.

[0193] Example 7: According to the method described in Example 6, the speech features include at least one of sound pressure level, direct reverberation ratio, and inter-channel phase difference variance.

[0194] Example 8: According to the method described in Example 6, the speech features include multiple components; determining the target distance based on the mapping relationship between the speech features and the distance, and the speech features, includes: For any one of the multiple speech features, the distance corresponding to the speech feature is determined based on the mapping relationship between the speech feature and the distance; The target distance is determined based on the distances corresponding to multiple of the aforementioned speech features.

[0195] Example 9. According to the method described in Example 8, determining the distance corresponding to the speech feature based on the mapping relationship between the speech feature and the distance includes: Determine the feature information of the target user, wherein the feature information includes at least one of the following: image feature information, voiceprint information; A mapping relationship between the speech features associated with the feature information and the distance is determined as a first mapping relationship; the first mapping relationship is constructed based on the speech features of the target user when the user inputs speech at at least two specified distances. Based on the first mapping relationship, the distance corresponding to the speech feature is determined.

[0196] Example 10: According to the method described in Example 8, determining the target distance based on the distances corresponding to multiple speech features includes: The distances corresponding to multiple speech features are used as inputs to a preset Kalman filter, and the output value of the Kalman filter is used as the target distance; or, The distances corresponding to multiple of the stated speech features are used as input to a preset distance fusion model, and the output value of the distance fusion model is used as the target distance; or, The target distance is obtained by weighted summation based on the weights and distances corresponding to each of the aforementioned speech features.

[0197] Example 11: According to the method described in Example 8, the plurality of speech features include sound pressure level and direct reverberation ratio.

[0198] Example 12: According to the method described in Example 5, determining the volume corresponding to the target distance based on the distance-volume mapping relationship includes: Determine the feature information of the target user, wherein the feature information includes at least one of the following: image feature information, voiceprint information; Determine the distance-volume mapping relationship associated with the feature information; Based on the distance-volume mapping relationship associated with the feature information, the volume corresponding to the target distance is determined.

[0199] Example 13. According to the method described in Example 6, the method further includes: For each of at least two specified distances, if the distance between the target user and the robot reaches that specified distance, obtain the calibration voice information input by the target user at that specified distance; Based on the at least two specified distances and the calibrated speech information corresponding to the at least two specified distances, the mapping relationship between the speech features and the distances is determined.

[0200] Example 14. According to the method described in Example 13, determining the mapping relationship between the speech features and the distances based on the at least two specified distances and the calibrated speech information corresponding to the at least two specified distances includes: The speech features of the calibration speech information are extracted as sample speech features; the sample speech features and the specified distance form a calibration data point; Based on multiple calibration data points, the mapping relationship between the voice features and distance is determined and associated with the feature information of the target user.

[0201] Example 15: According to the method described in Example 13, the method further includes: In response to receiving a trigger operation on the calibration option or receiving a voice command indicating the start of calibration, the process of constructing the mapping relationship between the voice features and the distance is initiated.

[0202] Example 16. The method according to Example 13, further comprising: Output a first prompt message; the first prompt message is used to prompt the target user to go to the specified distance and to input the calibration voice information for the robot; In response to detecting that the target user has reached the specified distance, a second prompt message is output; the second prompt message is used to indicate to the target user that they have now reached the specified distance.

[0203] Example 17. According to the method described in Example 4, the method further includes: If the number of times the first feedback event is triggered within the current time window exceeds the first preset threshold, then the step of increasing the fifth broadcast volume of the fifth response information to the fifth voice information input by the fourth user at the same distance is executed. If the number of times the second feedback event is triggered within the current time window exceeds the second preset threshold, then the step of reducing the volume of the fifth broadcast is executed.

[0204] Example 18. According to the method described in Example 1, the method further includes: In response to receiving the first instruction information from the fifth user, the robot directly increases the volume of the sixth broadcast of the sixth response information to the sixth voice information input by the fifth user at the fifth distance; the first instruction information is used to instruct the robot to directly increase the volume of subsequent broadcasts. In response to receiving the second instruction information from the fifth user, the robot directly reduces the volume of the sixth broadcast; the second instruction information is used to instruct the robot to directly reduce the volume of subsequent broadcasts.

[0205] Example 19: According to the method described in Example 4, the first feedback event is any one of the robot detecting the proximity event of the fourth user, the first input event triggered by the fourth user, and the volume increase event; the second feedback event is any one of the robot detecting the distance event of the fourth user, the second input event triggered by the fourth user, and the volume decrease event.

[0206] Example 20: According to the method described in Example 4, increasing the volume of the fifth broadcast of the fifth response information to the fifth voice information input by the fourth user at the same distance includes: Increase the first parameter in the distance-volume mapping relationship associated with the feature information of the fourth user, and / or decrease the second parameter in the distance-volume mapping relationship to increase the volume of the fifth broadcast; the output value of the distance-volume mapping relationship is positively correlated with the first parameter and negatively correlated with the second parameter; Reducing the volume of the fifth broadcast includes: decreasing the first parameter and / or increasing the second parameter to reduce the volume of the fifth broadcast.

[0207] Example 21: According to the method described in Example 20, increasing the first parameter in the distance-volume mapping relationship associated with the feature information of the fourth user, and / or decreasing the second parameter in the distance-volume mapping relationship, includes: Increase the maximum volume threshold, decrease the upper distance threshold, and / or decrease the lower distance threshold in the distance-volume mapping relationship associated with the feature information of the fourth user; the output value changes linearly within the range formed by the upper distance threshold and the lower distance threshold, and the maximum volume threshold is positively correlated with the output value; Decreasing the first parameter and / or increasing the second parameter includes: decreasing the maximum volume threshold, increasing the upper distance threshold, and / or increasing the lower distance threshold.

[0208] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The terms "an embodiment," "embodiment," or "one or more embodiments" as used herein mean that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of this application. Furthermore, please note that the phrase "in one embodiment" does not necessarily refer to the same embodiment in all instances.

[0209] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0210] In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A speech processing method, characterized in that, Applied to robots, the method includes: In response to receiving first voice information input by a first user, first response information is played to the first user at a first playback volume; the first voice information is input at a first distance. In response to receiving the second voice information input by the first user, the system plays the second response information to the first user at a second playback volume; the second voice information is input at a second distance, the second distance being greater than the first distance, and the second playback volume being greater than the first playback volume.

2. The method according to claim 1, characterized in that, The method further includes: In response to receiving a third voice message input by a second user at a third distance, a third response message is played to the second user at a third playback volume; in response to receiving a fourth voice message input by a third user at the third distance, a fourth response message is played to the third user at a fourth playback volume; the fourth playback volume is different from the third playback volume, or the fourth playback volume is the same as the third playback volume.

3. The method according to claim 2, characterized in that, When the input volume of the third voice information is different from the input volume of the fourth voice information, and the second user and the third user adopt the same distance and volume mapping relationship, the fourth broadcast volume is the same as the third broadcast volume; When the second user and the third user have different distance and volume mapping relationships, the fourth broadcast volume is different from the third broadcast volume.

4. The method according to claim 1, characterized in that, The method further includes: In response to a first feedback event from a fourth user, the robot increases the volume of its fifth response to the fifth voice information input by the fourth user at the same distance, wherein the first feedback event indicates that the robot's volume is lower than the fourth user's expected volume; or in response to a second feedback event from a fourth user, the robot decreases the volume of its fifth response, wherein the second feedback event indicates that the volume is higher than the expected volume.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: In response to receiving voice information from a target user, the target distance between the target user and the robot is determined; Based on the distance-volume mapping relationship, the volume corresponding to the target distance is determined, and the broadcast volume corresponding to the voice information is obtained.

6. The method according to claim 5, characterized in that, The step of determining the target distance between the target user and the robot in response to receiving voice information from the target user includes: In response to receiving voice information from the target user, the voice features of the voice information are extracted, wherein the voice features differ depending on the distance between the target user and the robot. The target distance is determined based on the mapping relationship between speech features and distance, as well as the speech features themselves.

7. The method according to claim 6, characterized in that, The speech features include at least one of sound pressure level, direct reverberation ratio, and inter-channel phase difference variance.

8. The method according to claim 6, characterized in that, The speech features include multiple components; determining the target distance based on the mapping relationship between speech features and distance, and the speech features, includes: For any one of the multiple speech features, the distance corresponding to the speech feature is determined based on the mapping relationship between the speech feature and the distance; The target distance is determined based on the distances corresponding to multiple of the aforementioned speech features.

9. The method according to claim 8, characterized in that, The step of determining the distance corresponding to the speech feature based on the mapping relationship between the speech feature and the distance includes: Determine the feature information of the target user, wherein the feature information includes at least one of the following: image feature information, voiceprint information; A mapping relationship between the speech features associated with the feature information and the distance is determined as a first mapping relationship; the first mapping relationship is constructed based on the speech features of the target user when the user inputs speech at at least two specified distances. Based on the first mapping relationship, the distance corresponding to the speech feature is determined.

10. The method according to claim 8, characterized in that, Determining the target distance based on the distances corresponding to multiple of the aforementioned speech features includes: The distances corresponding to multiple speech features are used as inputs to a preset Kalman filter, and the output value of the Kalman filter is used as the target distance; or, The distances corresponding to multiple speech features are used as input to a preset distance fusion model, and the output value of the distance fusion model is used as the target distance; or, The target distance is obtained by weighted summation based on the weights and distances corresponding to each of the aforementioned speech features.

11. The method according to claim 8, characterized in that, The speech features include sound pressure level and direct reverberation ratio.

12. The method according to claim 5, characterized in that, The step of determining the volume corresponding to the target distance based on the distance-volume mapping relationship includes: Determine the feature information of the target user, wherein the feature information includes at least one of the following: image feature information, voiceprint information; Determine the distance-volume mapping relationship associated with the feature information; Based on the distance-volume mapping relationship associated with the feature information, the volume corresponding to the target distance is determined.

13. The method according to claim 6, characterized in that, The method further includes: For each of at least two specified distances, if the distance between the target user and the robot reaches that specified distance, obtain the calibration voice information input by the target user at that specified distance; Based on the at least two specified distances and the calibrated speech information corresponding to the at least two specified distances, the mapping relationship between the speech features and the distances is determined.

14. The method according to claim 13, characterized in that, Determining the mapping relationship between the speech features and the distances based on the at least two specified distances and the calibrated speech information corresponding to the at least two specified distances includes: The speech features of the calibration speech information are extracted as sample speech features; the sample speech features and the specified distance form a calibration data point; Based on multiple calibration data points, the mapping relationship between the voice features and distance is determined and associated with the feature information of the target user.

15. The method according to claim 13, characterized in that, The method further includes: In response to receiving a trigger operation on the calibration option or receiving a voice command indicating the start of calibration, the process of constructing the mapping relationship between the voice features and the distance is initiated.

16. The method according to claim 13, characterized in that, The method further includes: Output a first prompt message; the first prompt message is used to prompt the target user to go to the specified distance and to input the calibration voice information for the robot; In response to detecting that the target user has reached the specified distance, a second prompt message is output; the second prompt message is used to indicate to the target user that they have now reached the specified distance.

17. The method according to claim 4, characterized in that, The method further includes: If the number of times the first feedback event is triggered within the current time window exceeds the first preset threshold, then the step of increasing the fifth broadcast volume of the fifth response information to the fifth voice information input by the fourth user at the same distance is executed. If the number of times the second feedback event is triggered within the current time window exceeds the second preset threshold, then the step of reducing the volume of the fifth broadcast is executed.

18. The method according to claim 1, characterized in that, The method further includes: In response to receiving the first instruction information from the fifth user, the robot directly increases the volume of the sixth broadcast of the sixth response information to the sixth voice information input by the fifth user at the fifth distance; the first instruction information is used to instruct the robot to directly increase the volume of subsequent broadcasts. In response to receiving the second instruction information from the fifth user, the robot directly reduces the volume of the sixth broadcast; the second instruction information is used to instruct the robot to directly reduce the volume of subsequent broadcasts.

19. The method according to claim 4, characterized in that, The first feedback event is any one of the following: the robot detecting the fourth user's proximity event, the fourth user triggering a first input event, and the volume increase event; the second feedback event is any one of the robot detecting the fourth user's distance event, the fourth user triggering a second input event, and the volume decrease event.

20. The method according to claim 4, characterized in that, Increasing the volume of the fifth response information to the fifth voice information input by the fourth user at the same distance includes: Increase the first parameter in the distance-volume mapping relationship associated with the feature information of the fourth user, and / or decrease the second parameter in the distance-volume mapping relationship to increase the volume of the fifth broadcast; the output value of the distance-volume mapping relationship is positively correlated with the first parameter and negatively correlated with the second parameter; Reducing the volume of the fifth broadcast includes: decreasing the first parameter and / or increasing the second parameter to reduce the volume of the fifth broadcast.

21. The method according to claim 20, characterized in that, The process of increasing the first parameter in the distance-volume mapping relationship associated with the feature information of the fourth user, and / or decreasing the second parameter in the distance-volume mapping relationship, includes: Increase the maximum volume threshold, decrease the upper distance threshold, and / or decrease the lower distance threshold in the distance-volume mapping relationship associated with the feature information of the fourth user; the output value changes linearly within the range formed by the upper distance threshold and the lower distance threshold, and the maximum volume threshold is positively correlated with the output value; Decreasing the first parameter and / or increasing the second parameter includes: decreasing the maximum volume threshold, increasing the upper distance threshold, and / or increasing the lower distance threshold.

22. A robot, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store executable instructions that cause the processor to perform the method as described in any one of claims 1 to 21.

23. A readable storage medium, characterized in that, It stores instructions that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1 to 21.

Images