Somatosensory-based pronunciation interaction method, intelligent mattress and storage medium

Abstract

This application relates to the field of motion-sensing interaction technology, disclosing a motion-sensing-based pronunciation interaction method, a smart mattress, and a storage medium. The method includes: the smart mattress entering a teaching mode in response to a first gesture; if any teaching sub-area within the teaching area of ​​the smart mattress is touched by a first user in the teaching mode, then playing the pronunciation corresponding to the musical character configured in that teaching sub-area; and the smart mattress switching from the teaching mode to a follow-up learning mode in response to a second gesture; if any follow-up learning sub-area within the follow-up learning area of ​​the smart mattress is touched by a second user in the follow-up learning mode, then playing the pronunciation corresponding to the musical character configured in that follow-up learning sub-area. Implementing this application embodiment enables motion-sensing-based pronunciation interaction between users on a smart mattress, enhancing children's enthusiasm for learning the pronunciation of musical characters on the smart mattress.

Description

Technical Field

[0001] This application relates to the field of motion-sensing interaction technology, specifically to a motion-sensing-based voice interaction method, a smart mattress, and a storage medium. Background Technology

[0002] With the rapid development of information technology, more and more smart home devices are appearing in people's field of vision, and smart mattresses are one of them.

[0003] Currently, some smart mattresses on the market can play the pronunciation of musical characters through speakers to children on the mattress, in order to help them with their intellectual development. However, when these smart mattresses play the pronunciation of musical characters through speakers, children can only passively listen to the pronunciation of musical characters, which lacks interactivity and is not conducive to improving children's enthusiasm for learning the pronunciation of musical characters on the smart mattress. Summary of the Invention

[0004] This application discloses a motion-sensing-based pronunciation interaction method, a smart mattress, and a storage medium, which enables motion-sensing-based pronunciation interaction between users on the smart mattress, thereby enhancing children's enthusiasm for learning the pronunciation of musical characters on the smart mattress.

[0005] The first aspect of this application discloses a motion-sensing-based voice interaction method, applied to a smart mattress, the method comprising:

[0006] The smart mattress enters the teaching mode in response to a first gesture. If, in the teaching mode, it is detected that any teaching sub-area of ​​the teaching area of ​​the smart mattress is touched by the first user, the sound corresponding to the music character configured in the teaching sub-area is played.

[0007] The smart mattress responds to a second gesture to switch from the teaching mode to the follow-up learning mode. If, in the follow-up learning mode, it is detected that any follow-up learning sub-area of ​​the smart mattress is touched by a second user, the sound corresponding to the music character configured in the follow-up learning sub-area is played.

[0008] As an optional implementation, in a first aspect of the embodiments of this application, before the smart mattress enters the teaching mode in response to a first gesture, the method further includes:

[0009] The smart mattress records the teaching area and follow-up learning area marked on its mattress surface; the teaching area contains several teaching sub-areas, and the follow-up learning area contains several follow-up learning sub-areas.

[0010] The smart mattress records the music characters configured in each teaching sub-area; wherein, the music characters configured in any two teaching sub-areas are different from each other;

[0011] The smart mattress records the music characters configured for each student area; wherein any two music characters configured for student areas are different from each other.

[0012] As another optional implementation, in the first aspect of the embodiments of this application, after the smart mattress plays the pronunciation corresponding to the music characters configured for the student area, the method further includes:

[0013] The smart mattress determines whether the pronunciation of the music characters configured in the learning area is the same as the pronunciation of the music characters configured in the teaching area; if they are the same, it outputs a praise voice; if they are different, it outputs an encouragement voice.

[0014] As an optional implementation, in the first aspect of the embodiments of this application, after the smart mattress outputs praise voice or encouragement voice, the method further includes:

[0015] The smart mattress responds to a third gesture by switching from the follow-up mode to the comparison mode. In the comparison mode, according to the recorded touch order of each teaching sub-area touched by the first user in the teaching mode, the mattress sequentially splices together the pronunciations corresponding to the music characters configured in each teaching sub-area touched by the first user to obtain a teaching audio segment.

[0016] In addition, in the comparison mode, the smart mattress sequentially splices together the pronunciations corresponding to the music characters configured in each learning area touched by the second user in the learning mode according to the touch order recorded in the learning mode, to obtain the learning audio segment.

[0017] Furthermore, the smart mattress calculates the similarity between the follow-up audio segment and the teaching audio segment, and outputs the similarity.

[0018] As another optional implementation, in a first aspect of the embodiments of this application, the smart mattress calculates the similarity between the follow-up audio segment and the teaching audio segment, including:

[0019] The smart mattress determines the first frame length L1 based on the number s1 of each teaching sub-area touched by the first user in the teaching mode; the first frame length L1 is inversely proportional to the number s1.

[0020] The smart mattress divides the teaching audio segment into frames according to the first frame length L1 to obtain each teaching audio frame; wherein each teaching audio frame is associated with a frame number, and any two teaching audio frames are associated with different frame numbers; and there is an overlap portion with a preset frame shift between any two adjacent teaching audio frames; and the spectrum corresponding to each teaching audio frame is calculated.

[0021] The smart mattress determines the second frame length L2 based on the number s2 of each learning area touched by the second user in the learning mode; the second frame length L2 is inversely proportional to the number s2.

[0022] The smart mattress divides the learning audio segment into frames according to the second frame length L2 to obtain each learning audio frame; wherein each learning audio frame is associated with a frame number, and any two learning audio frames are associated with different frame numbers; and there is an overlap of the preset frame shift between any two adjacent learning audio frames; and the spectrum corresponding to each learning audio frame is calculated.

[0023] For each target teaching audio frame in each teaching audio frame, the smart mattress determines whether a target follow-up audio frame exists in each follow-up audio frame, wherein the frame number associated with the target follow-up audio frame is the same as the frame number associated with the target teaching audio frame; if it exists, it calculates the similarity between the spectrum corresponding to the target teaching audio frame and the spectrum corresponding to the target teaching audio frame to obtain a spectrum similarity.

[0024] The smart mattress determines the similarity between the follow-up audio segment and the teaching audio segment based on the calculated similarity of various spectra.

[0025] The second aspect of this application discloses a smart mattress, comprising:

[0026] A mode control unit is used to enter the teaching mode in response to a first gesture;

[0027] The pronunciation playback unit is used to play the pronunciation corresponding to the music characters configured in the teaching sub-area when the teaching mode detects that any teaching sub-area in the teaching area of ​​the smart mattress is touched by the first user;

[0028] The mode control unit is also configured to switch from the teaching mode to the follow-up learning mode in response to a second gesture;

[0029] The pronunciation playback unit is also used to play the pronunciation corresponding to the music character configured in the learning area when it is detected that any learning sub-area in the learning area of ​​the smart mattress is touched by a second user in the learning mode.

[0030] As an optional implementation, in a second aspect of the embodiments of this application, the smart mattress further includes:

[0031] A recording unit is used to record the teaching area and follow-up learning area marked on the mattress surface of the mode control unit before the mode control unit enters the teaching mode in response to the first gesture; the teaching area includes several teaching sub-areas, and the follow-up learning area includes several follow-up learning sub-areas.

[0032] The recording unit is also used to record the music characters configured in each teaching sub-region; wherein the music characters configured in any two teaching sub-regions are different from each other;

[0033] The recording unit is also used to record the music characters configured in each student area; wherein any two music characters configured in the student areas are different from each other.

[0034] As another optional implementation, in a second aspect of the embodiments of this application, the smart mattress further includes:

[0035] The judgment unit is used to determine whether the pronunciation corresponding to the music character configured in the learning sub-area is the same as the pronunciation corresponding to the music character configured in the teaching sub-area after the pronunciation playback unit plays the pronunciation corresponding to the music character configured in the learning sub-area.

[0036] The pronunciation playback unit is further configured to output praise speech when the judgment results of the judgment unit are the same; or, it is further configured to output encouragement speech when the judgment results of the judgment unit are different.

[0037] As another optional implementation, in the second aspect of the embodiments of this application, the mode control unit is further configured to switch from the follow-up mode to the comparison mode in response to a third gesture after the pronunciation playback unit outputs praise speech or outputs encouragement speech;

[0038] The smart mattress also includes a splicing unit, a computing unit, and an output unit, wherein:

[0039] The splicing unit is used to sequentially splice the pronunciations corresponding to the music characters configured in each teaching sub-region touched by the first user in the teaching mode according to the recorded touch order of each teaching sub-region touched by the first user in the teaching mode, in the comparison mode, to obtain a teaching audio segment.

[0040] The splicing unit is also used in the comparison mode to sequentially splice the pronunciations corresponding to the music characters configured in each of the follow-up sub-areas touched by the second user in the following mode according to the touch order recorded in the following mode, so as to obtain the following audio segment.

[0041] The calculation unit is used to calculate the similarity between the learning audio segment and the teaching audio segment;

[0042] The output unit is used to output the similarity score.

[0043] The third aspect of this application discloses a smart mattress, comprising:

[0044] Memory containing executable program code;

[0045] A processor coupled to the memory;

[0046] The processor calls the executable program code stored in the memory and performs the following steps:

[0047] In response to the first gesture to enter the teaching mode, if the first user touches any teaching sub-area in the teaching area of ​​the smart mattress in the teaching mode, the pronunciation corresponding to the music character configured in the teaching sub-area is played.

[0048] In response to the second gesture switching from the teaching mode to the follow-up learning mode, if the follow-up learning mode detects that any follow-up learning sub-area in the follow-up learning area of ​​the smart mattress is touched by the second user, the pronunciation corresponding to the music character configured in the follow-up learning sub-area is played.

[0049] As an optional implementation, in a third aspect of the embodiments of this application, before entering the teaching mode in response to the first gesture, the processor further performs the following steps:

[0050] Record the teaching area and follow-up learning area marked on the surface of the smart mattress; the teaching area includes several teaching sub-areas, and the follow-up learning area includes several follow-up learning sub-areas;

[0051] Record the music characters configured in each teaching sub-region; wherein, the music characters configured in any two teaching sub-regions are different from each other;

[0052] Record the music characters configured for each student area; among them, any two music characters configured for each student area are different.

[0053] As another optional implementation, in a third aspect of the embodiments of this application, after playing the pronunciation corresponding to the musical characters configured for the student area, the processor further performs the following steps:

[0054] Determine whether the pronunciation corresponding to the music character configured in the learning sub-area is the same as the pronunciation corresponding to the music character configured in the teaching sub-area; if they are the same, output a praise voice; if they are different, output an encouragement voice.

[0055] As another optional implementation, in a third aspect of the embodiments of this application, after outputting the praise voice or the encouragement voice, the processor further performs the following steps:

[0056] In response to the third gesture, the system switches from the follow-up mode to the comparison mode. In the comparison mode, the system sequentially splices together the pronunciations corresponding to the music characters configured in the teaching sub-areas touched by the first user in the teaching mode according to the recorded touch order of each teaching sub-area touched by the first user in the teaching mode, to obtain a teaching audio segment.

[0057] In addition, in the comparison mode, according to the recorded touch order of each follow-up sub-area touched by the second user in the follow-up mode, the pronunciations corresponding to the music characters configured in each follow-up sub-area touched by the second user are sequentially spliced ​​together to obtain the follow-up audio segment.

[0058] In addition, the similarity between the learning audio segment and the teaching audio segment is calculated and the similarity is output.

[0059] As another optional implementation, in a third aspect of the embodiments of this application, the processor calculates the similarity between the follow-up audio segment and the teaching audio segment in the following manner:

[0060] The first frame length L1 is determined based on the number s1 of each teaching sub-region touched by the first user in the teaching mode; the first frame length L1 is inversely proportional to the number s1.

[0061] The teaching audio segment is divided into frames according to the first frame length L1 to obtain each teaching audio frame; wherein each teaching audio frame is associated with a frame number, and any two teaching audio frames are associated with different frame numbers; and any two adjacent teaching audio frames have an overlapping portion with a preset frame shift; and the spectrum corresponding to each teaching audio frame is calculated.

[0062] The second frame length L2 is determined based on the number s2 of each follow-up sub-region touched by the second user in the follow-up mode; the second frame length L2 is inversely proportional to the number s2.

[0063] The learning audio segment is divided into frames according to the second frame length L2 to obtain each learning audio frame; wherein each learning audio frame is associated with a frame number, and any two learning audio frames are associated with different frame numbers; and any two adjacent learning audio frames have an overlapping portion of the preset frame shift; and the spectrum corresponding to each learning audio frame is calculated.

[0064] For each target teaching audio frame in each teaching audio frame, determine whether there is a target follow-up audio frame in each follow-up audio frame, wherein the frame number associated with the target follow-up audio frame is the same as the frame number associated with the target teaching audio frame; if so, calculate the similarity between the spectrum corresponding to the target teaching audio frame and the spectrum corresponding to the target teaching audio frame to obtain a spectrum similarity.

[0065] Based on the calculated spectral similarity, the similarity between the learning audio segment and the teaching audio segment is determined.

[0066] The fourth aspect of this application discloses a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the motion-based voice interaction method in the first aspect of this application or any optional implementation of the first aspect of this application.

[0067] Compared with the prior art, the embodiments of this application have the following beneficial effects:

[0068] In this embodiment, the smart mattress can enter a teaching mode in response to a first gesture. If, in the teaching mode, any teaching sub-area within the teaching area of ​​the smart mattress is touched by a first user, the sound corresponding to the musical character configured in that sub-area can be played. Furthermore, the smart mattress can also switch from the teaching mode to a follow-up learning mode in response to a second gesture. If, in the follow-up learning mode, any follow-up learning sub-area within the follow-up learning area of ​​the smart mattress is touched by a second user, the sound corresponding to the musical character configured in that follow-up learning sub-area can be played. Therefore, implementing this embodiment enables motion-based sound interaction between users on the smart mattress, thereby increasing the interactivity for children learning the sound of musical characters on the smart mattress and enhancing their enthusiasm for learning. In addition, implementing this embodiment can expand the functionality of the smart mattress, improve its intelligence level, and facilitate its widespread application. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 This is a flowchart illustrating the first embodiment of the motion-based voice interaction method disclosed in this application.

[0071] Figure 2This is a sub-region distribution map of the mattress surface of a smart mattress disclosed in an embodiment of this application;

[0072] Figure 3 This is a flowchart illustrating a second embodiment of the motion-based voice interaction method disclosed in this application.

[0073] Figure 4 This is a flowchart illustrating the third embodiment of the motion-based voice interaction method disclosed in this application.

[0074] Figure 5 This is a schematic diagram of the structure of the first embodiment of the smart mattress disclosed in this application;

[0075] Figure 6 This is a schematic diagram of the structure of the second embodiment of the smart mattress disclosed in this application;

[0076] Figure 7 This is a schematic diagram of the structure of the third embodiment of the smart mattress disclosed in this application;

[0077] Figure 8 This is a structural schematic diagram of the fourth embodiment of the smart mattress disclosed in this application. Detailed Implementation

[0078] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0079] It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0080] This application discloses a motion-sensing-based voice interaction method, a smart mattress, and a storage medium. This method enables motion-sensing-based voice interaction between users on a smart mattress, thereby expanding the functionality of the smart mattress, enhancing its intelligence, and facilitating its widespread application. A detailed description is provided below with reference to the accompanying drawings.

[0081] Please see Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the motion-sensing-based voice interaction method disclosed in this application. Figure 1 As shown, this motion-based speech interaction method includes the following steps:

[0082] 101. The smart mattress responds to the first gesture and enters the teaching mode. If the first user touches any teaching sub-area in the teaching area of ​​the smart mattress in the teaching mode, the sound corresponding to the music character configured in that teaching sub-area will be played.

[0083] In this embodiment, the teaching area of ​​the smart mattress may include multiple teaching sub-areas, and a thin-film sensor array is laid beneath each teaching sub-area. Based on this, the smart mattress can capture a first gesture made by a first user via a camera and enter a teaching mode in response to the gesture. If, in the teaching mode, the thin-film sensor array beneath any teaching sub-area of ​​the smart mattress detects that the teaching sub-area has been touched by the first user (e.g., hammered or pressed), the sound corresponding to the musical character configured for that teaching sub-area is played.

[0084] For example, the first user can be a parent, guardian, or teacher, but this application embodiment does not limit this.

[0085] In this embodiment, the musical characters configured in the teaching sub-region can also be referred to as the musical notes configured in the teaching sub-region. For example, the musical characters configured in the teaching sub-region can be any one of the musical notes "1", "2", "3", "4", "5", "6" or "7".

[0086] For example, if the music character configured in the teaching sub-area is "3", the smart mattress can control the speaker to play the pronunciation "mi" of the music character "3"; or, if the music character configured in the teaching sub-area is "4", the smart mattress can control the speaker to play the pronunciation "fa" of the music character "4".

[0087] 102. The smart mattress responds to a second gesture to switch from teaching mode to follow-up learning mode. If any follow-up learning area of ​​the smart mattress is touched by the second user in follow-up learning mode, the sound corresponding to the music character configured in that follow-up learning area will be played.

[0088] In this embodiment, the learning area of ​​the smart mattress may include multiple learning sub-areas, and each learning sub-area is also covered with a thin-film sensor array (such as a thin-film pressure sensor array). Based on this, the smart mattress can capture a second gesture made by a second user via a camera, and in response to the second gesture, switch from teaching mode to learning mode. If, in learning mode, the thin-film sensor array covering any learning sub-area in the smart mattress's learning area detects that the learning sub-area has been touched (e.g., hammered or pressed) by the second user, then the sound corresponding to the musical character configured for that learning sub-area is played.

[0089] For example, the second user may include, but is not limited to, a child.

[0090] In this embodiment of the application, the musical character configured in the student area can also be referred to as the musical note configured in the student area. For example, the musical character configured in the student area can be any one of the musical notes "1", "2", "3", "4", "5", "6" or "7".

[0091] For example, if the music character configured for the learning area is "3", the smart mattress can control the speaker to play the pronunciation "mi" of the music character "3"; or, if the music character configured for the learning area is "4", the smart mattress can control the speaker to play the pronunciation "fa" of the music character "4".

[0092] In some implementations, the second gesture made by the second user differs from the first gesture made by the first user. It is understood that the pronunciation of the musical character in the teaching sub-area touched by the first user may be the same as the pronunciation of the musical character in the teaching sub-area touched by the second user; or, the pronunciation of the musical character in the teaching sub-area touched by the first user may be different from the pronunciation of the musical character in the teaching sub-area touched by the second user.

[0093] Please refer to the following: Figure 2 , Figure 2 This is a sub-region distribution map of the mattress surface of a smart mattress disclosed in an embodiment of this application. For example... Figure 2 As shown, the right side of the sub-region distribution map of the smart mattress surface, composed of the various sub-regions on the right, can be marked as the teaching area, while the left side of the sub-region distribution map of the smart mattress surface, composed of the various sub-regions on the left, can be marked as the learning area. For example, in... Figure 2The teaching area can include seven sub-areas, arranged along the length of the smart mattress. Along the length of the smart mattress, the first sub-area is configured with the music character "1", the second with "2", the third with "3", the fourth with "4", the fifth with "5", the sixth with "6", and the seventh with "7". For example, in... Figure 2 The learning area can include seven learning zones, arranged along the length of the smart mattress. Along the length of the mattress, the first learning zone is configured with the music character "1", the second with "2", the third with "3", the fourth with "4", the fifth with "5", the sixth with "6", and the seventh with "7". In practice... Figure 2 The musical characters configured in the seven teaching sub-areas are not visible on the surface of the smart mattress. Figure 2 The musical characters configured in the seven instructional sub-areas are displayed on the surface of the smart mattress solely for ease of understanding. Similarly, in practice... Figure 2 The seven musical characters configured for the student area are not visible on the surface of the smart mattress. Figure 2 The seven musical characters associated with the student area are displayed on the surface of the smart mattress solely for ease of understanding.

[0094] by Figure 2Taking the diagram showing the surface division of a smart mattress as an example, the smart mattress can enter a teaching mode in response to the first user's first gesture. If, in the teaching mode, the first user touches the teaching sub-area containing the musical character "3" (e.g., detected by a thin-film sensor array beneath the teaching sub-area containing the musical character "3"), then the smart mattress can control the speaker to play the sound of the musical character "3". The smart mattress can switch from teaching mode to follow-up mode in response to a second gesture made by a second user. If the follow-up learning mode detects that the follow-up learning area with the musical character "3" is touched by the second user (e.g., the follow-up learning area with the musical character "3" is touched by the second user through a thin-film sensor array laid under the follow-up learning area with the musical character "3"), then the smart mattress can control the speaker to play the pronunciation of the musical character "3" "mi".

[0095] It is evident that implementation Figure 1 The motion-based pronunciation interaction method shown enables motion-based pronunciation interaction between users on a smart mattress, thereby increasing the interactivity for children learning to pronounce musical characters on the smart mattress and enhancing their enthusiasm for learning to pronounce musical characters. Furthermore, implementation... Figure 1 The motion-based voice interaction method shown can also expand the functions of smart mattresses, enhance their intelligence, and promote their widespread application.

[0096] Please see Figure 3 , Figure 3 This is a flowchart illustrating a second embodiment of the motion-based voice interaction method disclosed in this application. Figure 3 As shown, this motion-based speech interaction method includes the following steps:

[0097] 301. The smart mattress records the teaching area and follow-up learning area marked on its mattress surface; wherein the teaching area contains several teaching sub-areas and the follow-up learning area contains several follow-up learning sub-areas.

[0098] In some implementations, the smart mattress can be connected to an interactive screen in its indoor environment. Accordingly, the method by which the smart mattress records the teaching and learning areas marked on its surface in step 301 above can be:

[0099] The smart mattress pushes a sub-region distribution map of its surface to the interactive screen, allowing the first user to mark the teaching area and the learning area on the sub-region distribution map; wherein the teaching area contains several teaching sub-areas, and the learning area contains several learning sub-areas.

[0100] Furthermore, after detecting the teaching and learning areas marked by the first user on the sub-regional distribution map of the smart mattress surface, the smart mattress can record the teaching and learning areas marked on its mattress surface.

[0101] like Figure 2 As shown, the first user can mark the right side of the sub-regions on the right side of the sub-region distribution map of the mattress surface of the first smart mattress as the teaching area, and the left side of the sub-regions on the left side of the sub-region distribution map of the mattress surface of the smart mattress as the follow-up learning area; correspondingly, the smart mattress can record the teaching area and follow-up learning area marked by the first user on the mattress surface of the smart mattress; wherein, the teaching area contains seven teaching sub-regions, and the follow-up learning area contains seven follow-up learning sub-regions.

[0102] By implementing the above-described method, the first user can flexibly mark the teaching area and the learning area on the surface of the smart mattress according to the positions of the first user and the second user on the smart mattress. This allows the first user and the second user to engage in motion-based pronunciation interaction without being confined to fixed positions on the smart mattress, thereby enhancing the experience of motion-based pronunciation interaction for both users on the smart mattress.

[0103] 302. The smart mattress records the music characters configured in each teaching sub-area; among them, the music characters configured in any two teaching sub-areas are different from each other.

[0104] 303. The smart mattress records the music characters configured for each student area; among them, any two music characters configured for student areas are different from each other.

[0105] In some implementations, the smart mattress can also connect to the user terminal (such as a mobile phone or watch) of the first user in its indoor environment. Accordingly, in step 302 above, the smart mattress can record the music characters configured for each teaching sub-area in the following way:

[0106] When a user's terminal plays a song, it can detect a starting time point selected by the user from the song's playback progress bar, and another ending time point selected by the user from the song's playback progress bar; wherein the ending time point is later than the starting time point; and send the target audio segment contained in the song, located between the starting time point and the ending time point, to the smart mattress.

[0107] Accordingly, the smart mattress receives the target audio segment and identifies the individual musical characters contained in the target audio segment; and pushes the individual musical characters contained in the target audio segment to the interactive screen;

[0108] In addition, the smart mattress detects one of the music characters in the target audio segment that is dragged into each teaching sub-area by the first user, and records one of the music characters in the target audio segment that is dragged into each teaching sub-area by the first user as the music character configured for that teaching sub-area.

[0109] Accordingly, in step 302 above, the smart mattress can record the music characters configured for each student area in the following way:

[0110] The smart mattress detects one of the music characters in the target audio segment that was dragged into each student area by the first user, and records one of the music characters in the target audio segment that was dragged into each student area by the first user as the music character configured for that student area.

[0111] By implementing the above-described method, the first user can flexibly decide which musical character is configured in each teaching sub-area. This allows the musical characters configured in each teaching sub-area to better adapt to the first user's hand gestures, thereby enhancing the first user's experience when engaging in motion-based pronunciation interaction.

[0112] Furthermore, by implementing the above-described method, the first user can flexibly decide which musical character is configured in each student area, thereby enhancing the first user's experience when engaging in motion-based pronunciation interaction.

[0113] 304. The smart mattress responds to the first gesture and enters the teaching mode. If the first user touches any teaching sub-area in the teaching area of ​​the smart mattress in the teaching mode, the sound corresponding to the music character configured in that teaching sub-area will be played.

[0114] 305. The smart mattress responds to a second gesture to switch from teaching mode to follow-up learning mode. If, in follow-up learning mode, it is detected that any follow-up learning area of ​​the smart mattress is touched by the second user, the pronunciation corresponding to the music character configured in that follow-up learning area will be played.

[0115] 306. The smart mattress determines whether the pronunciation of the music character configured in the learning area is the same as the pronunciation of the music character configured in the teaching sub-area; if they are the same, proceed to step 307; if they are not the same, proceed to step 308.

[0116] 307. The smart mattress outputs praise voice messages.

[0117] 308. The smart mattress outputs encouraging voice prompts.

[0118] It is evident that implementation Figure 3 The motion-based pronunciation interaction method shown enables motion-based pronunciation interaction between users on a smart mattress, thereby increasing the interactivity for children learning to pronounce musical characters on the smart mattress and enhancing their enthusiasm for learning to pronounce musical characters. Furthermore, implementation... Figure 3 The motion-based voice interaction method shown can also expand the functions of smart mattresses, enhance their intelligence, and promote their widespread application.

[0119] Please see Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the motion-sensing-based voice interaction method disclosed in this application. Figure 4 As shown, this motion-based speech interaction method includes the following steps:

[0120] Steps 401 to 405 are the same as steps 401 to 405 in the previous embodiments, and will not be repeated here.

[0121] 406. The smart mattress determines whether the pronunciation of the music character configured in the learning area is the same as the pronunciation of the music character configured in the teaching sub-area; if they are the same, proceed to step 407; if they are not the same, proceed to step 408.

[0122] 407. The smart mattress outputs a praising voice and executes steps 409 to 411.

[0123] 408. The smart mattress outputs a stimulating voice and executes steps 409 to 411.

[0124] 409. The smart mattress responds to a third gesture by switching from the follow-up mode to the comparison mode. In the comparison mode, it sequentially splices together the pronunciations corresponding to the music characters configured in the teaching sub-areas touched by the first user in the recorded touch order in the teaching mode to obtain the teaching audio segment.

[0125] In some implementations, the smart mattress can capture a third gesture made by a first or second user via a camera. This third gesture is different from both the first and second gestures.

[0126] 410. In comparison mode, the smart mattress sequentially splices together the pronunciations corresponding to the music characters configured in the respective learning areas touched by the second user in the recorded learning mode, to obtain the learning audio segment.

[0127] The execution order of steps 409 and 410 can also be interchanged, and this application embodiment does not limit it.

[0128] 411. The smart mattress calculates the similarity between the follow-up audio segment and the teaching audio segment, and outputs the similarity.

[0129] In some implementations, the smart mattress may calculate the similarity between the learning audio segment and the teaching audio segment in the following ways:

[0130] The smart mattress determines the first frame length L1 based on the number s1 of each teaching sub-area touched by the first user in the teaching mode. The first frame length L1 is inversely proportional to the number s1. That is, the larger the number s1 is, the shorter the first frame length L1 is; conversely, the smaller the number s1 is, the longer the first frame length L1 is.

[0131] Furthermore, the smart mattress divides the teaching audio segment into frames according to the first frame length L1 to obtain each teaching audio frame; wherein each teaching audio frame is associated with a frame number, and any two teaching audio frames are associated with different frame numbers; furthermore, there is an overlap of a preset frame shift between any two adjacent teaching audio frames; and the spectrum corresponding to each teaching audio frame is calculated.

[0132] Furthermore, the smart mattress determines the second frame length L2 based on the number s2 of each learning area touched by the second user in the learning mode; wherein, the first frame length L2 is inversely proportional to the number s2, that is, when the number s2 is larger, the second frame length L2 is shorter; conversely, when the number s2 is smaller, the second frame length L2 is longer.

[0133] In addition, the smart mattress divides the learning audio segment into frames according to the second frame length L2 to obtain each learning audio frame; wherein each learning audio frame is associated with a frame number, and any two learning audio frames are associated with different frame numbers; and there is an overlap of the preset frame shift between any two adjacent learning audio frames; and the spectrum corresponding to each learning audio frame is calculated.

[0134] Furthermore, for each target teaching audio frame in the aforementioned teaching audio frames, the smart mattress determines whether a target follow-up audio frame exists in the aforementioned follow-up audio frames, wherein the frame number associated with the target follow-up audio frame is the same as the frame number associated with the target teaching audio frame; if it exists, the similarity between the spectrum corresponding to the target teaching audio frame and the spectrum corresponding to the target teaching audio frame is calculated to obtain a spectrum similarity.

[0135] Furthermore, the smart mattress determines the similarity between the learning audio segment and the teaching audio segment based on the calculated similarity of various spectra.

[0136] In this embodiment, the smart mattress divides the teaching audio segment into frames based on the first frame length L1, which differs from segmenting the teaching audio segment. Segmenting the teaching audio segment usually involves dividing it into larger-grained segments at specified time points. However, the smart mattress divides the teaching audio segment into frames based on the first frame length L1, which is inversely proportional to the quantity s1, to obtain individual teaching audio frames. Furthermore, there is an overlap between any two adjacent teaching audio frames with a preset frame shift, allowing each teaching audio frame to be a smaller-grained segment. Similarly, each follow-up audio frame can also be a smaller-grained segment. When both the teaching audio frames and the follow-up audio frames are smaller-grained segments, the similarity between the follow-up audio segments and the teaching audio segments can be determined more accurately.

[0137] In some implementations, the smart mattress determines the similarity between the learning audio segment and the teaching audio segment based on the calculated spectral similarity as follows:

[0138] The smart mattress determines the spectral similarity exceeding a specified threshold from the calculated spectral similarity, and calculates a first ratio of the number of spectral similarities exceeding the specified threshold to the number of calculated spectral similarities, and calculates a second ratio of the first ratio to the aforementioned teaching audio frames, as the similarity between the follow-up audio segment and the teaching audio segment.

[0139] When each teaching audio frame is a small-granular segment and each follow-up audio frame can also be a small-granular segment, implementing this method can more accurately determine the similarity between the follow-up audio segment and the teaching audio segment.

[0140] It is evident that implementation Figure 4 The motion-based pronunciation interaction method shown enables motion-based pronunciation interaction between users on a smart mattress, thereby increasing the interactivity for children learning to pronounce musical characters on the smart mattress and enhancing their enthusiasm for learning to pronounce musical characters. Furthermore, implementation... Figure 4 The motion-based voice interaction method shown can also expand the functions of smart mattresses, enhance their intelligence, and promote their widespread application.

[0141] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of the first embodiment of the smart mattress disclosed in this application. Figure 5 As shown, the smart mattress includes:

[0142] Mode control unit 501 is used to enter teaching mode in response to a first gesture;

[0143] The pronunciation playback unit 502 is used to play the pronunciation corresponding to the music character configured in the teaching sub-area when it is detected that any teaching sub-area in the teaching area of ​​the smart mattress is touched by the first user in the teaching mode.

[0144] The mode control unit 501 is also used to switch from teaching mode to follow-up mode in response to a second gesture;

[0145] The pronunciation playback unit 502 is also used to play the pronunciation corresponding to the music character configured in the learning area when it is detected that any learning sub-area in the learning area of ​​the smart mattress is touched by a second user in the learning mode.

[0146] In this embodiment, the teaching area of ​​the smart mattress may include multiple teaching sub-areas, and a thin-film sensor array is laid beneath each teaching sub-area. Based on this, the mode control unit 501 can capture a first gesture made by the first user via a camera and enter the teaching mode in response to the first gesture; correspondingly, in the teaching mode, the sound playback unit 502 detects, through the thin-film sensor array laid beneath any teaching sub-area in the teaching area of ​​the smart mattress, that the teaching sub-area has been touched by the first user (e.g., hammered or pressed), and plays the pronunciation corresponding to the musical character configured in that teaching sub-area.

[0147] In this embodiment, the learning area of ​​the smart mattress may include multiple learning sub-areas, and each learning sub-area is also covered with a thin-film sensor array (such as a thin-film pressure sensor array). Based on this, the mode control unit 501 can capture a second gesture made by the second user through a camera, and switch from teaching mode to learning mode in response to the second gesture; correspondingly, in learning mode, the sound playback unit 502 detects, through the thin-film sensor array under any learning sub-area in the learning area of ​​the smart mattress, that the learning sub-area is touched by the second user (such as hammered or pressed), and plays the sound corresponding to the musical character configured in that learning sub-area.

[0148] In some implementations, the second gesture made by the second user is different from the first gesture made by the first user.

[0149] It is evident that implementation Figure 5 The smart mattress shown enables motion-based sound interaction between users, increasing the interactivity for children learning musical notes on the mattress and thus enhancing their enthusiasm for this activity. Furthermore, [the text abruptly ends here, likely due to an incomplete sentence or missing information]. Figure 5 The smart mattress shown can also expand its functions and enhance its intelligence, which is conducive to the promotion and application of smart mattresses.

[0150] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of the second embodiment of the smart mattress disclosed in this application. Figure 6 The smart mattress shown is made by Figure 5 The smart mattress shown is optimized to be similar to... Figure 5 Compared to the smart mattress shown, Figure 6 The smart mattress shown also includes:

[0151] The recording unit 503 is used to record the teaching area and the follow-up learning area marked on the mattress surface before the mode control unit 501 enters the teaching mode in response to the first gesture; wherein the teaching area includes several teaching sub-areas and the follow-up learning area includes several follow-up learning sub-areas.

[0152] The recording unit 503 is also used to record the music characters configured in each teaching sub-area; wherein the music characters configured in any two teaching sub-areas are different from each other;

[0153] The recording unit 503 is also used to record the music characters configured for each student area; wherein any two music characters configured for the student areas are different from each other.

[0154] In some implementations... Figure 6The smart mattress shown also includes:

[0155] The judgment unit 504 is used to determine whether the pronunciation corresponding to the music character configured in the learning area is the same as the pronunciation corresponding to the music character configured in the teaching area after the pronunciation playback unit 502 plays the pronunciation corresponding to the music character configured in the learning area.

[0156] The pronunciation playback unit 502 is also used to output praise speech when the judgment result of the judgment unit 504 is the same; or, it is also used to output encouragement speech when the judgment result of the judgment unit 504 is different.

[0157] In some implementations... Figure 6 The smart mattress shown can be connected to an interactive screen in its indoor environment. Correspondingly, the recording unit 503 can record the teaching and learning areas marked on the surface of the smart mattress in the following ways:

[0158] The recording unit 503 pushes a sub-region distribution map of the smart mattress surface to the interactive screen, so that the first user can mark the teaching area and the learning area on the sub-region distribution map of the smart mattress surface; wherein, the teaching area contains several teaching sub-areas, and the learning area contains several learning sub-areas.

[0159] Furthermore, after detecting the teaching area and follow-up learning area marked by the first user on the sub-region distribution map of the mattress surface of the smart mattress, the recording unit 503 can record the teaching area and follow-up learning area marked on the mattress surface.

[0160] like Figure 2 As shown, the first user can mark the right side of the sub-regions on the right side of the sub-region distribution map of the mattress surface of the first smart mattress as the teaching area, and the left side of the sub-regions on the left side of the sub-region distribution map of the mattress surface of the smart mattress as the follow-up learning area; correspondingly, the recording unit 503 can record the teaching area and follow-up learning area marked by the first user on the mattress surface of the smart mattress; wherein, the teaching area includes seven teaching sub-regions, and the follow-up learning area includes seven follow-up learning sub-regions.

[0161] By implementing the above-described method, the first user can flexibly mark the teaching area and the learning area on the surface of the smart mattress according to the positions of the first user and the second user on the smart mattress. This allows the first user and the second user to engage in motion-based pronunciation interaction without being confined to fixed positions on the smart mattress, thereby enhancing the experience of motion-based pronunciation interaction for both users on the smart mattress.

[0162] In some implementations... Figure 6The smart mattress shown can also connect to the user terminal (such as a mobile phone or watch) of the first user in its indoor environment. Correspondingly, the recording unit 503 can record the music characters configured for each teaching sub-area in the following way:

[0163] When a user's terminal plays a song, it can detect a starting time point selected by the user from the song's playback progress bar, and another ending time point selected by the user from the song's playback progress bar; wherein the ending time point is later than the starting time point; and send the target audio segment contained in the song, located between the starting time point and the ending time point, to the smart mattress.

[0164] Accordingly, the recording unit 503 receives the target audio segment and identifies each musical character contained in the target audio segment; and pushes each musical character contained in the target audio segment to the interactive screen;

[0165] In addition, the recording unit 503 detects one of the music characters in the target audio segment that is dragged into each teaching sub-region by the first user, and records one of the music characters in the target audio segment that is dragged into each teaching sub-region by the first user as the music character configured for that teaching sub-region.

[0166] Accordingly, the recording unit 503 can record each musical character configured in the student area in the following ways:

[0167] The recording unit 503 detects one of the music characters in the target audio segment that was dragged into each student area by the first user, and records one of the music characters in the target audio segment that was dragged into each student area by the first user as the music character configured for that student area.

[0168] By implementing the above-described method, the first user can flexibly decide which musical character is configured in each teaching sub-area. This allows the musical characters configured in each teaching sub-area to better adapt to the first user's hand gestures, thereby enhancing the first user's experience when engaging in motion-based pronunciation interaction.

[0169] Furthermore, by implementing the above-described method, the first user can flexibly decide which musical character is configured in each student area, thereby enhancing the first user's experience when engaging in motion-based pronunciation interaction.

[0170] Please see Figure 7 , Figure 7 This is a structural schematic diagram of the third embodiment of the smart mattress disclosed in this application. Wherein, Figure 7 The smart mattress shown is made by Figure 6 The smart mattress shown is optimized to be similar to... Figure 6 Compared to the smart mattress shown, Figure 7 The smart mattress shown also includes:

[0171] The assembly unit 505, the calculation unit 506, and the output unit 507, wherein:

[0172] The mode control unit 401 is also used to switch from the follow-up mode to the comparison mode in response to a third gesture after the pronunciation playback unit 502 outputs a praise voice or an incentive voice.

[0173] The splicing unit 505 is used to splice the pronunciations corresponding to the music characters configured in each teaching sub-area touched by the first user in the teaching mode according to the recorded touch order of each teaching sub-area touched by the first user in the teaching mode, in the comparison mode, to obtain the teaching audio segment.

[0174] The splicing unit 505 is also used in comparison mode to sequentially splice the pronunciations corresponding to the music characters configured in each follower area touched by the second user in the recorded touch order of each follower area touched by the second user in follower mode to obtain the follower audio segment.

[0175] The calculation unit 506 is used to calculate the similarity between the learning audio segment and the teaching audio segment;

[0176] Output unit 507 is used to output the similarity score.

[0177] In some embodiments, the computing unit 506 includes:

[0178] The first determining subunit 5061 is used to determine the first frame length L1 based on the number s1 of each teaching sub-region touched by the first user in the teaching mode; the first frame length L1 is inversely proportional to the number s1.

[0179] Framing subunit 5062 is used to framing the teaching audio segment according to the first frame length L1 to obtain each teaching audio frame; wherein each teaching audio frame is associated with a frame number, and any two teaching audio frames are associated with different frame numbers; and any two adjacent teaching audio frames have an overlapping part with a preset frame shift; and the spectrum corresponding to each teaching audio frame is calculated.

[0180] The first determining subunit 5061 is also used to determine the second frame length L2 based on the number s2 of each learning sub-region touched by the second user in the learning mode; the first frame length L2 is inversely proportional to the number s2.

[0181] The framing subunit 5062 is further configured to framing the learning audio segment according to the second frame length L2 to obtain each learning audio frame; wherein each learning audio frame is associated with a frame number, and any two learning audio frames are associated with different frame numbers; and any two adjacent learning audio frames have an overlapping portion of the preset frame shift; and calculates the spectrum corresponding to each learning audio frame.

[0182] The calculation subunit 5063 is used to determine whether there is a target follow-up audio frame in each follow-up audio frame for each target teaching audio frame in each teaching audio frame. The frame number associated with the target follow-up audio frame is the same as the frame number associated with the target teaching audio frame. If it exists, the similarity between the spectrum corresponding to the target teaching audio frame and the spectrum corresponding to the target teaching audio frame is calculated to obtain a spectrum similarity.

[0183] The second determining subunit 5064 is used to determine the similarity between the learning audio segment and the teaching audio segment based on the calculated spectral similarity.

[0184] In this embodiment, the framing subunit 5062 framing the teaching audio segment according to the first frame length L1 differs from segmenting the teaching audio segment. Segmenting the teaching audio segment usually involves dividing the segment into larger-granularity fragments at specified time points. However, the framing subunit 5062 framing the teaching audio segment according to the first frame length L1, which is inversely proportional to the quantity s1, yields individual teaching audio frames. Furthermore, there is an overlap between any two adjacent teaching audio frames with a preset frame shift, allowing each teaching audio frame to be a smaller-granularity fragment. Similarly, each follow-up audio frame can also be a smaller-granularity fragment. When both the teaching audio frames and the follow-up audio frames are smaller-granularity fragments, the similarity between the follow-up audio segments and the teaching audio segments can be determined more accurately.

[0185] In some implementations, the second determining subunit 5064 determines the similarity between the learning audio segment and the teaching audio segment based on the calculated spectral similarities as follows:

[0186] The second determining subunit 5064 determines the spectral similarity exceeding a specified threshold from the calculated spectral similarity, and calculates a first ratio of the number of spectral similarities exceeding the specified threshold to the number of calculated spectral similarities, and calculates a second ratio of the first ratio to the aforementioned teaching audio frames, as the similarity between the follow-up audio segment and the teaching audio segment.

[0187] When each teaching audio frame is a small-granular segment and each follow-up audio frame can also be a small-granular segment, implementing this method can more accurately determine the similarity between the follow-up audio segment and the teaching audio segment.

[0188] Please see Figure 8 , Figure 8 This is a structural schematic diagram of the fourth embodiment of the smart mattress disclosed in this application. Figure 8 As shown, the smart mattress also includes:

[0189] Memory 801 storing executable program code;

[0190] Processor 802 coupled to the memory;

[0191] The processor 802 calls the executable program code stored in memory 801 and performs the following steps:

[0192] In response to the first gesture to enter the teaching mode, if the first user touches any teaching sub-area in the teaching area of ​​the smart mattress in the teaching mode, the pronunciation corresponding to the music character configured in the teaching sub-area will be played.

[0193] In response to the second gesture switching from teaching mode to follow-up learning mode, if any follow-up learning area of ​​the smart mattress is touched by the second user in follow-up learning mode, the pronunciation corresponding to the music character configured in that follow-up learning area will be played.

[0194] In some implementations, before entering the teaching mode in response to the first gesture, the processor 802 also performs the following steps:

[0195] Record the teaching areas and follow-up learning areas marked on the mattress surface; the teaching area contains several teaching sub-areas, and the follow-up learning area contains several follow-up learning sub-areas;

[0196] Record the music characters configured in each teaching sub-region; wherein, the music characters configured in any two teaching sub-regions are different from each other;

[0197] Record the music characters configured for each student area; among them, any two music characters configured for each student area are different.

[0198] In some implementations, after playing the pronunciation corresponding to the musical character configured in the student area, the processor 802 further performs the following steps:

[0199] Determine whether the pronunciation of the music character configured in the student area is the same as the pronunciation of the music character configured in the teaching area; if they are the same, output a praise voice; if they are different, output an encouragement voice.

[0200] In some implementations, after outputting the praise speech or the encouragement speech, the processor 802 further performs the following steps:

[0201] In response to the third gesture, the system switches from the follow-up mode to the comparison mode. In the comparison mode, the system sequentially splices together the pronunciations corresponding to the music characters configured in the teaching sub-areas touched by the first user in the teaching mode, according to the recorded touch order of each teaching sub-area touched by the first user in the teaching mode, to obtain the teaching audio segment.

[0202] In addition, in comparison mode, according to the recorded touch order of each follower area touched by the second user in follower mode, the pronunciations corresponding to the music characters configured in each follower area touched by the second user are sequentially spliced ​​together to obtain the follower audio segment.

[0203] Additionally, the similarity between the follow-up audio segment and the teaching audio segment is calculated and output.

[0204] In some implementations, the processor 802 calculates the similarity between the follow-up audio segment and the instructional audio segment in the following way:

[0205] The first frame length L1 is determined based on the number s1 of each teaching sub-region touched by the first user in the teaching mode; the first frame length L1 is inversely proportional to the number s1.

[0206] The teaching audio segment is divided into frames according to the first frame length L1 to obtain each teaching audio frame; wherein each teaching audio frame is associated with a frame number, and any two teaching audio frames are associated with different frame numbers; and any two adjacent teaching audio frames have an overlapping portion with a preset frame shift; and the spectrum corresponding to each teaching audio frame is calculated.

[0207] The second frame length L2 is determined based on the number s2 of each follower area touched by the second user in the follower mode; the second frame length L2 is inversely proportional to the number s2.

[0208] The learning audio segment is divided into frames according to the second frame length L2 to obtain each learning audio frame; wherein each learning audio frame is associated with a frame number, and any two learning audio frames are associated with different frame numbers; and there is an overlap of the preset frame shift between any two adjacent learning audio frames; and the spectrum corresponding to each learning audio frame is calculated.

[0209] For each target teaching audio frame in each teaching audio frame, determine whether there is a target follow-up audio frame in each follow-up audio frame, and the frame number associated with the target follow-up audio frame is the same as the frame number associated with the target teaching audio frame; if so, calculate the similarity between the spectrum corresponding to the target teaching audio frame and the spectrum corresponding to the target teaching audio frame to obtain a spectrum similarity.

[0210] Based on the calculated spectral similarity, the similarity between the follow-up audio segment and the teaching audio segment is determined.

[0211] In some implementations, the processor 802 determines the similarity between the learning audio segment and the teaching audio segment based on the calculated spectral similarities as follows:

[0212] The calculated spectral similarities are used to determine the spectral similarities that exceed a specified threshold. The first ratio of the number of spectral similarities exceeding the specified threshold to the number of calculated spectral similarities is calculated. The first ratio is then compared with the second ratio of the aforementioned teaching audio frames, which is used as the similarity between the follow-up audio segment and the teaching audio segment.

[0213] When each teaching audio frame is a small-granular segment and each follow-up audio frame can also be a small-granular segment, implementing this method can more accurately determine the similarity between the follow-up audio segment and the teaching audio segment.

[0214] It is evident that implementation Figure 8 The smart mattress shown enables motion-based sound interaction between users, increasing the interactivity for children learning to pronounce musical characters on the mattress and thus enhancing their enthusiasm for this activity. Furthermore, implementation... Figure 8 The smart mattress shown can also expand its functions and enhance its intelligence, which is conducive to the promotion and application of smart mattresses.

[0215] This application further discloses a storage medium storing a computer program thereon, which, when executed by a processor, implements all or part of the steps in the preceding method embodiments.

[0216] The foregoing has provided a detailed description of a motion-sensing-based voice interaction method and related products disclosed in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of the application.

Claims

1. A somatosensory-based pronunciation interaction method, characterized in that, The method is applied to a smart mattress, and the method comprises: The smart mattress records teaching areas and following-up areas marked on the surface of the mattress; the teaching areas comprise a plurality of teaching sub-areas, and the following-up areas comprise a plurality of following-up sub-areas; the smart mattress records music characters configured for each teaching sub-area; the music characters configured for any two teaching sub-areas are different from each other; the smart mattress records music characters configured for each following-up sub-area; the music characters configured for any two following-up sub-areas are different from each other; the plurality of teaching sub-areas and the plurality of following-up sub-areas correspond to each other, and the music characters configured for each teaching sub-area are the same as those configured for the corresponding following-up sub-area; The smart mattress enters a teaching mode in response to a first gesture, and if any teaching sub-area in the teaching area of the smart mattress is touched by a first user in the teaching mode, the smart mattress plays the pronunciation corresponding to the music characters configured for the teaching sub-area; The smart mattress switches from the teaching mode to a following-up mode in response to a second gesture, and if any following-up sub-area in the following-up area of the smart mattress is touched by a second user in the following-up mode, the smart mattress plays the pronunciation corresponding to the music characters configured for the following-up sub-area; The smart mattress connects a user terminal of the first user, and the smart mattress records the music characters configured for each teaching sub-area in the following manner: The user terminal of the first user selects a starting time point from the playing time progress bar of a song as a starting time point and selects another time point from the playing time progress bar of the song as an ending time point when playing the song, wherein the ending time point is later than the starting time point; and the smart mattress sends a target audio segment included in the song and located between the starting time point and the ending time point to the smart mattress; The smart mattress receives the target audio segment, identifies each music character included in the target audio segment, pushes each music character included in the target audio segment to an interactive screen, and records one music character included in the target audio segment and dragged into each teaching sub-area by the first user through the interactive screen as the music character configured for the teaching sub-area; The smart mattress records the teaching areas and the following-up areas marked on the surface of the mattress in the following manner: The smart mattress pushes a sub-area distribution diagram of the surface of the mattress to the interactive screen, so that the first user marks the teaching areas and the following-up areas on the sub-area distribution diagram of the surface of the mattress; and After detecting the teaching areas and the following-up areas marked by the first user on the sub-area distribution diagram of the surface of the mattress, the smart mattress records the teaching areas and the following-up areas marked on the surface of the mattress.

2. The somatosensory-based vocalization interaction method according to claim 1, wherein, After the smart mattress plays the pronunciation corresponding to the music characters configured for the following-up sub-area, the method further comprises: The smart mattress judges whether the pronunciation corresponding to the music character configured in the follow-up sub-area is the same as the pronunciation corresponding to the music character configured in the teaching sub-area; if the same, output praise voice; if not the same, output encouragement voice.

3. The somatosensory-based vocalization interaction method according to claim 2, characterized in that, After the smart mattress outputs the praise voice or the encouragement voice, the method further comprises: The smart mattress switches from the follow-up mode to the comparison mode in response to a third gesture, and in the comparison mode, the pronunciation corresponding to the music character configured in each teaching sub-area touched by the first user in the teaching mode is sequentially spliced according to the recorded touch order to obtain a teaching audio segment; And the smart mattress in the comparison mode, according to the recorded touch order of each follow-up sub-area touched by the second user in the follow-up mode, the pronunciation corresponding to the music character configured in each follow-up sub-area touched by the second user is sequentially spliced to obtain a follow-up audio segment; And the smart mattress calculates the similarity between the follow-up audio segment and the teaching audio segment, and outputs the similarity.

4. The somatosensory-based vocalization interaction method according to claim 3, characterized in that, The smart mattress calculates the similarity between the follow-up audio segment and the teaching audio segment, comprising: The smart mattress determines a first frame length L1 according to the number s1 of each teaching sub-area touched by the first user in the teaching mode; the first frame length L1 is inversely proportional to the number s1; The smart mattress frames the teaching audio segment according to the first frame length L1 to obtain each teaching audio frame; wherein each teaching audio frame is associated with a frame number, and the frame numbers associated with any two teaching audio frames are different; and there is an overlap part of a preset frame shift amount between any two adjacent teaching audio frames; and the spectrum corresponding to each teaching audio frame is calculated; The smart mattress determines a second frame length L2 according to the number s2 of each follow-up sub-area touched by the second user in the follow-up mode; the first frame length L2 is inversely proportional to the number s2; The smart mattress frames the follow-up audio segment according to the second frame length L2 to obtain each follow-up audio frame; wherein each follow-up audio frame is associated with a frame number, and the frame numbers associated with any two follow-up audio frames are different; and there is an overlap part of the preset frame shift amount between any two adjacent follow-up audio frames; and the spectrum corresponding to each follow-up audio frame is calculated; The smart mattress judges whether there is a target follow-up audio frame in the follow-up audio frame for each target teaching audio frame in the teaching audio frame, the frame number associated with the target follow-up audio frame is the same as the frame number associated with the target teaching audio frame; if there is, the similarity between the spectrum corresponding to the target teaching audio frame and the spectrum corresponding to the target teaching audio frame is calculated to obtain a spectrum similarity; The smart mattress determines the similarity between the follow-up audio segment and the teaching audio segment according to the calculated each spectrum similarity.

5. A smart mattress, characterized in that, Comprise: A mode control unit is configured to enter a teaching mode in response to a first gesture; A pronunciation playing unit is configured to play a pronunciation corresponding to a music character configured for a teaching sub-area in the teaching mode when detecting that any of the teaching sub-areas in a teaching area of the smart mattress is touched by a first user; The mode control unit is further configured to switch from the teaching mode to a following mode in response to a second gesture; The pronunciation playing unit is further configured to play a pronunciation corresponding to a music character configured for a following sub-area in the following mode when detecting that any of the following sub-areas in a following area of the smart mattress is touched by a second user; The smart mattress further comprises: A recording unit is configured to record a teaching area and a following area marked on a surface of the smart mattress before the mode control unit enters the teaching mode in response to the first gesture; the teaching area comprises a plurality of teaching sub-areas, and the following area comprises a plurality of following sub-areas; The recording unit is further configured to record a music character configured for each teaching sub-area; wherein the music characters configured for any two teaching sub-areas are different from each other; The recording unit is further configured to record a music character configured for each following sub-area; wherein the music characters configured for any two following sub-areas are different from each other; the plurality of teaching sub-areas correspond to the plurality of following sub-areas one by one, and the music characters configured for each teaching sub-area and a corresponding following sub-area are the same; The smart mattress is connected to a user terminal of the first user, and the recording unit records the music character configured for each teaching sub-area in the following manner: When playing a song, the user terminal of the first user detects a starting time point selected by the first user from a play time progress bar of the song and detects an ending time point selected by the first user from the play time progress bar of the song; the ending time point is later than the starting time point; and the recording unit sends a target audio segment included in the song and located between the starting time point and the ending time point to the smart mattress; The recording unit receives the target audio segment and identifies each music character included in the target audio segment; pushes each music character included in the target audio segment to an interactive screen, the interactive screen being any screen in an indoor environment where the smart mattress is located; and records a music character included in the target audio segment and dragged into each teaching sub-area by the first user through the interactive screen as a music character configured for the teaching sub-area; The recording unit is further configured to push a sub-area distribution diagram of a surface of the smart mattress to the interactive screen to enable the first user to mark the teaching area and the following area on the sub-area distribution diagram of the surface of the smart mattress; and record the teaching area and the following area marked on the surface of the smart mattress after detecting that the first user marks the teaching area and the following area on the sub-area distribution diagram of the surface of the smart mattress.

6. A smart mattress, characterized in that, Comprise: a memory storing executable program code; a processor coupled with the memory; the processor invokes the executable program code stored in the memory to perform the following steps: record the teaching area and the follow-up area marked on the surface of the smart mattress; the teaching area contains a plurality of teaching sub-areas, and the follow-up area contains a plurality of follow-up sub-areas; record the musical characters configured in each teaching sub-area; wherein the musical characters configured in any two teaching sub-areas are different from each other; record the musical characters configured in each follow-up sub-area; wherein the musical characters configured in any two follow-up sub-areas are different from each other, the plurality of teaching sub-areas correspond to the plurality of follow-up sub-areas one by one, and the musical characters configured in each teaching sub-area are the same as those in the corresponding follow-up sub-area; in response to a first gesture entering a teaching mode, if any teaching sub-area in the teaching area of the smart mattress is touched by a first user in the teaching mode, play the pronunciation corresponding to the musical characters configured in the teaching sub-area; in response to a second gesture switching from the teaching mode to a follow-up mode, if any follow-up sub-area in the follow-up area of the smart mattress is touched by a second user in the follow-up mode, play the pronunciation corresponding to the musical characters configured in the follow-up sub-area; the smart mattress is connected to a user terminal of the first user, and the processor records the musical characters configured in each teaching sub-area in the following manner: the user terminal of the first user selects a starting time point from the playing time progress bar of a song as a starting time point and selects another time point from the playing time progress bar of the song as an ending time point when playing the song, wherein the ending time point is later than the starting time point; sends a target audio segment contained in the song and located between the starting time point and the ending time point to the smart mattress; the processor receives the target audio segment, identifies each musical character contained in the target audio segment, pushes each musical character contained in the target audio segment to an interactive screen, and records one of the musical characters contained in the target audio segment as the musical characters configured in each teaching sub-area when the teaching sub-area is dragged into the interactive screen by the first user; the smart mattress pushes a sub-area distribution map of the mattress surface of the smart mattress to the interactive screen, so that the first user marks the teaching area and the follow-up area on the sub-area distribution map of the mattress surface of the smart mattress; and the smart mattress records the teaching area and the follow-up area marked on the surface of the smart mattress after detecting the teaching area and the follow-up area marked by the first user on the sub-area distribution map of the mattress surface of the smart mattress.

7. The smart mattress of claim 6, wherein, after playing the pronunciation corresponding to the musical characters configured in the follow-up sub-area, the processor further performs the following steps: determine whether the pronunciation corresponding to the music character configured in the teaching sub-region is the same as the pronunciation corresponding to the music character configured in the following sub-region; if the pronunciation is the same, output a praise voice; if the pronunciation is not the same, output an encouragement voice.

8. The smart mattress of claim 7, wherein, After outputting the praise voice or the encouragement voice, the processor further performs the following steps: switch from the following mode to a comparison mode in response to a third gesture, and in the comparison mode, sequentially splice the pronunciation corresponding to the music character configured in each teaching sub-region touched by the first user according to the recorded touch order of each teaching sub-region touched by the first user in the teaching mode, to obtain a teaching audio segment; and in the comparison mode, sequentially splice the pronunciation corresponding to the music character configured in each following sub-region touched by the second user according to the recorded touch order of each following sub-region touched by the second user in the following mode, to obtain a following audio segment; and calculate the similarity between the following audio segment and the teaching audio segment, and output the similarity.

9. The smart mattress of claim 8, wherein, The processor calculates the similarity between the following audio segment and the teaching audio segment in the following manner: determine a first frame length L1 according to the number s1 of each teaching sub-region touched by the first user in the teaching mode; the first frame length L1 is in inverse proportional relationship with the number s1; frame the teaching audio segment according to the first frame length L1 to obtain each teaching audio frame; each teaching audio frame is associated with a frame number, and the frame numbers of any two teaching audio frames are different; there is a preset frame shift amount overlap between any two adjacent teaching audio frames; and the spectrum corresponding to each teaching audio frame is calculated; determine a second frame length L2 according to the number s2 of each following sub-region touched by the second user in the following mode; the second frame length L2 is in inverse proportional relationship with the number s2; frame the following audio segment according to the second frame length L2 to obtain each following audio frame; each following audio frame is associated with a frame number, and the frame numbers of any two following audio frames are different; there is a preset frame shift amount overlap between any two adjacent following audio frames; and the spectrum corresponding to each following audio frame is calculated; for each target teaching audio frame in the teaching audio frames, determine whether there is a target following audio frame in the following audio frames, the frame number of the target following audio frame being the same as the frame number of the target teaching audio frame; if there is, calculate the similarity between the spectrum corresponding to the target teaching audio frame and the spectrum corresponding to the target following audio frame, to obtain a spectrum similarity; determine the similarity between the following audio segment and the teaching audio segment according to the calculated spectrum similarities.

10. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps in the somatosensory-based pronunciation interaction method of any one of claims 1 to 4. The computer program is executed by the processor to implement the steps in the somatosensory-based pronunciation interaction method of any one of claims 1 to 4.

Images