Method and device for identifying user operation through music rhythm, electronic device and medium

Abstract

The application relates to a user operation method and device by music rhythm recognition, electronic equipment and medium, and belongs to the technical field of information recognition. The method comprises the following steps: generating a demonstration melody by combination according to at least one first non-rhythm melody, a rhythm melody and at least one second non-rhythm melody arranged in sequence; generating an operation melody by combination according to the at least one first non-rhythm melody, a blank melody and the at least one second non-rhythm melody arranged in sequence, wherein the time length of the blank melody is equal to that of the rhythm melody; playing the demonstration melody and the operation melody in sequence, obtaining operation information of a blind user who imitates the rhythm melody when the blank melody is played; judging whether the operation information matches the rhythm melody, if yes, changing a parameter value of a target object; otherwise, determining that the operation information is wrong. The application has the effect that a blind user can accurately and quickly operate a target object.

Description

Technical Field

[0001] This application relates to the technical field of information recognition, and in particular to a method, device, electronic device, and medium for recognizing user operations through music rhythm. Background Technology

[0002] With the development of the internet, more and more people are using computers to access the internet. However, blind people face difficulties accessing the internet because they cannot see images. For example, when they need to perform actions on a target object on the interface, such as opening, closing, playing, stopping, or scrolling, blind people cannot use their eyes to operate the mouse to locate the target object and thus cannot accurately control it. Summary of the Invention

[0003] In order to enable blind users to operate target objects accurately and quickly, this application provides a method, device, electronic device and medium for recognizing user operation through music rhythm.

[0004] Firstly, this application provides a method for recognizing user operations through music rhythm, employing the following technical solution:

[0005] A demonstration melody is generated by combining at least one first non-rhythmic melody, one rhythmic melody, and at least one second non-rhythmic melody arranged in sequence.

[0006] An operational melody is generated by combining the at least one first non-rhythmic melody, the blank melody, and the at least one second non-rhythmic melody arranged in sequence, wherein the duration of the blank melody is equal to the duration of the rhythmic melody.

[0007] Play the demonstration melody and the operation melody in sequence, and obtain the operation information of the user imitating the rhythmic melody when playing the blank melody;

[0008] Determine whether the operation information matches the rhythmic melody; if so, change the parameter value of the target object.

[0009] Otherwise, the operation information is determined to be incorrect.

[0010] By adopting the above technical solution, the electronic device generates a demonstration melody by combining a first non-rhythmic melody, a rhythmic melody, and a second non-rhythmic melody. It then determines a blank melody with the same duration as the rhythmic melody. An operation melody is generated based on the first non-rhythmic melody, the blank melody, and the second non-rhythmic melody. The demonstration melody and the operation melody are played sequentially, and the user's operation information during the playback of the blank melody is obtained. The operation melody is then compared with the rhythmic melody to determine if they match. If the music rhythm matches, the user's operation is correct, and the parameter value of the target object is changed. If they do not match, the user's operation information is determined to be incorrect. This system can recognize the blind user's operation information related to music rhythm, enabling the blind to operate the target object quickly and accurately, thus allowing blind users to access the internet smoothly.

[0011] Further, determining whether the operation information matches the rhythmic melody includes:

[0012] The rhythmic melody is divided into multiple first sub-melodies, each of which includes one note;

[0013] The operation information is divided into multiple second sub-melodies based on the duration of the first sub-melodies;

[0014] Determine the first start time and the first end time of each note in the rhythmic melody, and the second start time and the second end time of each note in the operation information;

[0015] Compare the second start time in each second sub-melody with the first start time in the corresponding first sub-melody, and / or compare the second end time in each second sub-melody with the first end time in the corresponding first sub-melody to determine the matching degree between the second sub-melody and the corresponding first sub-melody;

[0016] Based on the matching degree between each second sub-melody and the corresponding first sub-melody in the operation information, it is determined whether the operation information matches the rhythmic melody.

[0017] By adopting the above technical solution, the electronic device divides the rhythmic melody into multiple first sub-melodies and divides the operation information into multiple second sub-melodies. It further determines the start and end times of each note, allowing the electronic device to compare the second start time of the corresponding note in the second sub-melody with the first start time of the corresponding first sub-melody, and the second end time of the second sub-melody with the second end time of the corresponding first sub-melody. This comparison determines the matching degree between the second sub-melody and the corresponding first sub-melody. Based on the matching degree of each second sub-melody in the operation information, the electronic device determines whether the operation information matches the rhythmic melody. Therefore, through precise division and judgment of whether each note matches, the electronic device can quickly and accurately determine the user's operation.

[0018] Further, the step of comparing the second start time in each second sub-melody with the first start time in the corresponding first sub-melody, and / or comparing the second end time in each second sub-melody with the first end time in the corresponding first sub-melody to determine the matching degree includes:

[0019] Determine whether there is a second start moment for the corresponding note in the second sub-melody;

[0020] If so, then

[0021] Determine the first difference between the second start time in the second sub-melody and the first start time in the corresponding first sub-melody;

[0022] Determine the second difference between the second end time of the corresponding note in the second sub-melody and the first end time of the corresponding first sub-melody;

[0023] The absolute values ​​of the first difference and the second difference are added together to obtain the median value. The matching degree is obtained by calculating the ratio of the median value to the duration of the first sub-melody.

[0024] If not, then the match rate is determined to be zero.

[0025] By adopting the above technical solution, the electronic device first determines the matching degree based on whether a second start time exists in the second sub-melody. If no second start time exists, the matching degree is determined to be zero. If it exists, the device further determines the first difference between the second start time and the first start time, and the second difference between the second end time and the first end time, and calculates the matching degree based on the first and second differences. Therefore, the device can quickly and accurately judge the user's operation information.

[0026] In another possible implementation, after determining that the operation information is incorrect, the method further includes:

[0027] Determine whether the number of errors in the same operation information within a continuous time period reaches a preset number;

[0028] If not, then play the demonstration melody and the operation melody again;

[0029] If so, the difficulty of the next demonstration melody is determined based on the difficulty of the current demonstration melody, wherein the difficulty of the next demonstration melody is less than the difficulty of the current demonstration melody;

[0030] A new rhythmic melody is determined based on the difficulty of the next demonstrated melody;

[0031] Based on the new rhythmic melody, generate a new demonstration melody and a new operational melody.

[0032] By adopting the above technical solution, the electronic device determines whether the user has repeatedly made operational errors in a short period of time. If the number of errors has not reached the preset number, the demonstration melody and operation melody are played again so that the user can perform the operation again. When the number of errors reaches the preset number, the electronic device reduces the difficulty of the demonstration melody, that is, generates a new rhythmic melody, so that the user can perform the operation according to the new demonstration melody and the new operation melody. This allows the user to quickly and skillfully master the operation method and automatically adjusts according to the user's needs, thereby improving intelligence.

[0033] In another possible implementation, after determining that the operation information is incorrect, the method further includes:

[0034] Obtain the mismatched segments from the operation information;

[0035] Based on the operation melody and the operation information, a playback melody is generated by combining them.

[0036] When playing the revisited melody, increase the volume of the mismatched segments.

[0037] By adopting the above technical solution, after the electronic device determines that the operation information is incorrect, it identifies the mismatched segment and increases the volume of the mismatched segment when playing the playback melody. This provides a simple and direct way to alert the user to the incorrect segment so that they can pay attention to it in the next operation.

[0038] Furthermore, the notes of the rhythmic melody are percussion instruments, and the notes of the non-rhythmic melody are non-percussion instruments.

[0039] By adopting the above technical solutions, the rhythmic melodies of percussion instruments can be better distinguished, and the difference between rhythmic and non-rhythmic melodies can be better reflected, making the rhythmic melodies more noticeable to the ear.

[0040] Furthermore, the method also includes: determining the type of rhythmic melody according to the user's needs, the type including a type without strong or weak beats and a type with strong or weak beats.

[0041] By adopting the above technical solutions, the layers of musical rhythm are enriched, increasing the fun.

[0042] Secondly, this application provides a device for recognizing user operations through music rhythm, employing the following technical solution:

[0043] The demonstration melody generation module is used to generate a demonstration melody by combining at least one first non-rhythmic melody, a rhythmic melody, and at least one second non-rhythmic melody arranged in sequence.

[0044] The operation melody generation module is used to generate an operation melody by combining the at least one first non-rhythmic melody, the blank melody and the at least one second non-rhythmic melody arranged in sequence, wherein the duration of the blank melody is equal to the duration of the rhythmic melody.

[0045] The operation information acquisition module is used to play the demonstration melody and the operation melody in sequence, and acquire the operation information of the user imitating the rhythmic melody when playing the blank melody.

[0046] The judgment module is used to determine whether the operation information matches the rhythmic melody;

[0047] An execution module is used to change the parameter value of the target object when the judgment module determines that it is true;

[0048] An error determination module is used to determine that the operation information is incorrect when the judgment module determines that it is not.

[0049] By adopting the above technical solution, the demonstration melody generation module generates a demonstration melody by combining a first non-rhythmic melody, a rhythmic melody, and a second non-rhythmic melody. Then, the operation melody generation module determines a blank melody with the same duration as the rhythmic melody and generates an operation melody based on the first non-rhythmic melody, the blank melody, and the second non-rhythmic melody. The operation information acquisition module sequentially plays the demonstration melody and the operation melody, acquiring the user's operation information while the blank melody is played. The judgment module then compares the operation melody with the rhythmic melody to determine if they match. If the music rhythm matches, the user's operation is correct, and the execution module changes the parameter value of the target object. If they do not match, the error determination module determines that the user's operation information is incorrect. This system can recognize the blind user's operation information related to music rhythm, enabling the blind to operate on the target object quickly and accurately, thus allowing blind users to access the internet smoothly.

[0050] Thirdly, this application provides an electronic device that adopts the following technical solution:

[0051] An electronic device, comprising:

[0052] At least one processor;

[0053] Memory;

[0054] At least one application, wherein the at least one application is stored in the memory and configured to be executed by the at least one processor, the at least one application being configured to: perform the user operation method for recognizing music rhythm as described in any one of the first aspects.

[0055] By adopting the above technical solution, the processor loads and executes the application program in the memory, generates a demonstration melody by combining the first non-rhythmic melody, the rhythmic melody, and the second non-rhythmic melody, and then determines a blank melody with the same duration as the rhythmic melody. An operation melody is generated based on the first non-rhythmic melody, the blank melody, and the second non-rhythmic melody. The demonstration melody and the operation melody are played sequentially, and the user's operation information during the playback of the blank melody is obtained. The operation melody is then compared with the rhythmic melody to determine if they match. If the music rhythm matches, the user's operation is correct, and the parameter value of the target object is changed. If they do not match, the user's operation information is determined to be incorrect. This system can recognize the blind user's operation information related to music rhythm, enabling the blind user to operate the target object quickly and accurately, thus allowing the blind user to access the internet smoothly.

[0056] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution:

[0057] A computer-readable storage medium storing a computer program that can be loaded by a processor and executed as described in any one of the first aspects, which recognizes a user operation method by means of music rhythm.

[0058] By adopting the above technical solution, the processor loads and executes the application program in the computer-readable storage medium. Based on the first non-rhythmic melody, the rhythmic melody, and the second non-rhythmic melody, it generates a demonstration melody and then determines a blank melody with the same duration as the rhythmic melody. Based on the first non-rhythmic melody, the blank melody, and the second non-rhythmic melody, it generates an operation melody. The demonstration melody and the operation melody are played sequentially, and the user's operation information when the blank melody is played is obtained. Then, the operation melody is compared with the rhythmic melody to determine whether they match. If the music rhythm matches, the user's operation is correct, and the parameter value of the target object is changed. If they do not match, the user's operation information is determined to be incorrect. This system can recognize the blind user's operation information related to music rhythm, enabling the blind user to operate the target object quickly and accurately, and enabling the blind user to access the Internet smoothly.

[0059] In summary, this application includes at least one of the following beneficial technical effects:

[0060] 1. Play the demonstration melody and the operation melody in sequence, obtain the user's operation information when playing the blank melody, and then compare the operation melody with the rhythmic melody to determine if they match. If the music rhythm matches, the user's operation is correct, and the parameter value of the target object is changed. If they do not match, the user's operation information is determined to be incorrect. This can identify the blind user's operation information related to music rhythm, so that the blind can operate the target object, quickly and accurately achieve the operation goal, and enable the blind user to access the Internet smoothly.

[0061] 2. The electronic device divides the rhythmic melody into multiple first sub-melodies, divides the operation information into multiple second sub-melodies, determines the matching degree between the second sub-melodies and the corresponding first sub-melodies, and then determines whether the operation information matches the rhythmic melody based on the matching degree of each second sub-melody in the operation information. Therefore, through fine division, it can judge whether each note matches, and quickly and accurately judge the user's operation.

[0062] 3. When the number of errors reaches a preset number, the electronic device reduces the difficulty of the demonstration melody, allowing the user to operate according to the new demonstration melody and the new operation melody. This enables the user to quickly and skillfully master the operation method and automatically adjusts according to the user's needs, thus improving intelligence. Attached Figure Description

[0063] Figure 1 This is a flowchart illustrating the method for recognizing user operations through music rhythm in an embodiment of this application.

[0064] Figure 2 This is a schematic diagram of the exemplary melody and guiding melody in the embodiments of this application.

[0065] Figure 3 This is a schematic diagram of multiple second sub-melodies in the embodiments of this application.

[0066] Figure 4 This is a schematic diagram of the first start time and the first end time in the first sub-melody, and the second start time and the second end time in the second sub-melody in the embodiments of this application.

[0067] Figure 5 This is a structural block diagram of a user operation device that recognizes music rhythm in an embodiment of this application.

[0068] Figure 6 This is a structural block diagram of the electronic device in the embodiments of this application. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0071] Blind users cannot see the screen, but they can receive feedback information through hearing. In order to enable blind users to operate, this application enables users to operate by using the rhythm of music, thereby achieving control over the target object.

[0072] This application discloses a method for recognizing user operations through music rhythm. (See also...) Figure 1 Performed by an electronic device, including (steps S101 to S106):

[0073] Step S101: Generate a demonstration melody by combining at least one first non-rhythmic melody, a rhythmic melody, and at least one second non-rhythmic melody arranged in sequence.

[0074] Specifically, electronic devices are preset with a variety of non-rhythmic and rhythmic melodies. Among them, rhythms that have typical characteristics and recur are called rhythmic melodies, while other rhythms are non-rhythmic melodies.

[0075] For example, the basic rhythmic patterns commonly used in music include several types of rhythms such as the evenly divided note value pattern, the alternation of long and short notes pattern, the dotted rhythmic pattern, the syncopated rhythmic pattern, and the rhythmic pattern containing rests. Each type includes multiple rhythms.

[0076] Taking a quarter note as one beat as an example, the evenly divided rhythm types include quarter rhythm, eighth rhythm, sixteenth rhythm, etc., while the alternating long and short rhythm types include sixteenth-eighth rhythm and eighth-sixteenth rhythm, etc.

[0077] Electronic devices can set the duration of a rhythmic melody to one beat, with a quarter note as one beat, thus allowing for the selection of various rhythmic melodies.

[0078] To make it easier to distinguish, electronic devices generate rhythmic melodies through percussion and non-rhythmic melodies through non-percussion instruments. For example, a piano melody or a guitar melody can be used as a non-rhythmic melody.

[0079] The first and second arrhythmic melodies can be the same or different. To distinguish the demonstration melody from the operational melody below, the first and second arrhythmic melodies are different.

[0080] Furthermore, referring to Figure 2 When generating a demonstration melody, the non-rhythmic melody can include multiple segments, with at least one non-rhythmic melody playing before and at least one non-rhythmic melody playing after the rhythmic melody. Therefore, the non-rhythmic melody serves as a cue. For example, in a demonstration melody composed of a first non-rhythmic melody + a rhythmic melody + a second non-rhythmic melody, the user can focus on the rhythmic melody after hearing the end of the first non-rhythmic melody, and confirm the end of the rhythmic melody after hearing the second rhythmic melody.

[0081] Step S102: Based on at least one first non-rhythmic melody, a blank melody, and at least one second non-rhythmic melody arranged in sequence, generate an operational melody by combining them, wherein the duration of the blank melody is equal to the duration of the rhythmic melody.

[0082] Specifically, refer to Figure 2 The electronic device replaces the rhythmic melody in the demonstration melody with a blank melody of the same duration, thus obtaining the operating melody. The blank melody is a melody without percussion.

[0083] Step S103: Play the demonstration melody and the operation melody in sequence to obtain the operation information of the user imitating the rhythmic melody when playing the blank melody.

[0084] Specifically, the electronic device first plays a demonstration melody, which serves as a demonstration, and then plays the operation melody. While the operation melody is playing, the user performs operations that imitate the rhythmic pattern of the blank melody.

[0085] The user can operate by pressing the keyboard, such as the up, down, left, and right arrow keys. If a quarter note is one beat and the rhythmic melody is a quarter rhythm, pressing the left key on the keyboard will generate one beat of percussion. The user needs to press the left key when the blank melody starts playing and release the left key when the blank melody finishes playing to complete the operation.

[0086] Reference Figure 2 If a quarter note is one beat, and the rhythmic melody includes one beat and is an eighth-note rhythm, then the user needs to click the left mouse button twice at the same time interval during the blank melody period to generate two notes, thus completing the operation.

[0087] The electronic device acquires operation information, which is the key presses made by the user when playing a blank melody.

[0088] Step S104: Determine whether the operation information matches the rhythmic melody; if yes, proceed to step S105: Change the parameter value of the target object; otherwise, proceed to step S106: Generate error information.

[0089] Specifically, step S104 performed by the electronic device includes (steps S11 to S15):

[0090] Step S11: Divide the rhythmic melody into multiple first sub-melodies, each of which contains one note.

[0091] Specifically, the electronic device categorizes rhythmic melodies based on their type. If the rhythmic melody is a rhythm with equal note values, the electronic device divides it equally according to the number of notes per beat. For example, if a quarter note is one beat and the rhythmic melody is an eighth-beat rhythm, then each beat contains two eighth notes. The electronic device halves each beat of the rhythmic melody to obtain two first sub-melodies. If the rhythmic melody is a rhythm with alternating long and short notes, such as a sixteenth-eighth beat, then each beat includes two sixteenth notes and one eight-beat note. The electronic device then divides the rhythmic melody according to the number of notes per beat. : : The proportion is used to divide each beat in the rhythmic melody, resulting in three sub-melodies per beat.

[0092] Step S12: Divide the operation information into multiple second sub-melodies according to the duration of the first sub-melody.

[0093] Specifically, the electronic device maps operation information to rhythmic melodies one-to-one. For example, if each beat of a rhythmic melody is divided into two first sub-melodies, the duration of each first sub-melody can be obtained. Based on these durations, the electronic device then divides the operation information into two second sub-melodies, with each second sub-melody having the same duration as its corresponding first sub-melody.

[0094] Step S13: Determine the first start time and the first end time of each note in the rhythmic melody, as well as the second start time and the second end time of each note in the operation information.

[0095] Specifically, the moment a note begins to sound is called the start moment, and the moment it stops sounding is called the end moment. Therefore, electronic devices can determine the first start moment and the first end moment of a note in a rhythmic melody.

[0096] The electronic device determines the second start moment when the user presses the button to start the note, and the second end moment when the user releases the button to stop the note from playing.

[0097] Step S14: Compare the second start time in each second sub-melody with the first start time in the corresponding first sub-melody, and / or compare the second end time in each second sub-melody with the first end time in the corresponding first sub-melody to determine the matching degree.

[0098] Further, step S14 includes (steps S141 to S145):

[0099] Step S141: Determine whether there is a second start moment corresponding to the note in the second sub-melody. If so, proceed to steps S142 to S144; otherwise, proceed to step S145: Determine that the matching degree is zero.

[0100] Specifically, the second submelody may contain corresponding notes as well as corresponding notes in the previous or next submelody. The electronic device only needs to analyze the second start time and / or second end time of the corresponding note in the current second submelody.

[0101] Since the first non-rhythmic melody plays before the blank melody, users typically begin pressing buttons after the first non-rhythmic melody. Therefore, if a user presses a button before the first non-rhythmic melody has finished playing, the user's operation is incorrect. Similarly, if there is no corresponding second start moment for the note in each second sub-melody, it indicates that the user's operation is incorrect, and the electronic device determines the matching degree to be zero.

[0102] However, since a second non-rhythmic melody is played after the blank melody, the user may stop pressing the keys only after hearing the second non-rhythmic melody. For blind people, the time to stop pressing the keys may be earlier or later. In turn, the electronic device allows the second end time of the corresponding note in the second sub-melody to be later than the first end time. In order to further determine the matching degree, the electronic device executes steps S142 to S144.

[0103] Reference Figure 3 The second sub-melody A is a standard note with a matching degree of 100%. If it is B, C, or D, the matching degree is zero. If it is E, F, or G, further matching degree needs to be determined.

[0104] Step S142: Determine the first difference between the second start time in the second sub-melody and the first start time in the corresponding first sub-melody.

[0105] Specifically, the electronic device subtracts the first start time from the second start time to obtain the first difference.

[0106] Step S143: Determine the second difference between the second end time of the corresponding note in the second sub-melody and the first end time of the corresponding first sub-melody.

[0107] Specifically, the electronic device subtracts the corresponding first end time from the second end time to obtain the second difference.

[0108] Reference Figure 4 Let A be the first submelody, a be the first start time, and b be the first end time; let G be the second submelody, a1 be the second start time, and b1 be the second end time. The electronic device calculates a1-a to obtain the first difference and calculates b1-b to obtain the second difference.

[0109] Step S144: Add the absolute values ​​of the first difference and the second difference to obtain the intermediate value. Based on the ratio of the calculated intermediate value to the duration of the first sub-melody, determine the matching degree between the second sub-melody and the corresponding first sub-melody.

[0110] Specifically, the larger the ratio calculated by the electronic device, the smaller the matching degree between the second sub-melody and the corresponding first sub-melody. The electronic device subtracts 1 from the ratio to calculate the matching degree.

[0111] Step S15: Determine whether the operation information matches the rhythmic melody based on the matching degree between each second sub-melody and the corresponding first sub-melody in the operation information.

[0112] Specifically, the electronic device stores preset values ​​that are used to determine whether the matching degree of the second sub-melody meets the standard. When the matching degree reaches the preset value, it is determined that the second sub-melody matches the corresponding first sub-melody. The preset value can be set between 80% and 90%.

[0113] Furthermore, if all the second sub-melodies in the operation information match, the electronic device determines whether the operation information matches the rhythmic melody.

[0114] Step S105: Change the parameter value of the target object.

[0115] Specifically, the target object is the object that the user interacts with, which can be a webpage, video, a character in a game, etc. If the purpose of the user's interaction is to make the target object perform a corresponding action, such as opening a webpage, then after the user's interaction information matches the rhythmic melody, the electronic device changes the parameter values ​​of the target object, and the target object then performs the corresponding action.

[0116] Electronic devices can preset different parameter values ​​for a target object, with each parameter value corresponding to an action. Therefore, users can control the target object to perform the corresponding action by changing the parameter values ​​of the target object.

[0117] Step S106: Determine that the operation information is incorrect.

[0118] Specifically, when the user's operation information does not match the rhythmic melody, the electronic device determines that the operation information is incorrect and prompts the user, which can be a voice prompt.

[0119] To enable the user to perform the operation again, the user can repeat steps S101 to S104 until the user's operation information matches the rhythmic melody and the operation is completed.

[0120] Furthermore, in order to enable the user to automatically perform the operation again, after the electronic device determines that the operation information is incorrect, the above method further includes (steps S21 to S25):

[0121] Step S21: Determine whether the number of times the same operation information is incorrect within a continuous time period has reached the preset number; if not, proceed to step S22; if yes, proceed to step S23.

[0122] Specifically, the continuous time includes the time elapsed since the user performed the current operation, plus the sum of two preset time intervals before and after the current operation. In other words, the electronic device can determine whether the user has made consecutive operational errors based on the number of incorrect operation messages.

[0123] If the number of incorrect operation information reaches the preset number within a short period of time, the current music rhythm will be too difficult for the user, making it difficult for the user to complete the operation quickly.

[0124] The preset number of attempts can be two or three.

[0125] Step S22: Play the demonstration melody and the operation melody again.

[0126] Specifically, the electronic device plays the demonstration melody and the operation melody again, and obtains the user's operation information during the playback of the blank melody again, so as to determine whether the operation information matches the rhythmic melody again.

[0127] Step S23: Determine the difficulty of the next demonstration melody based on the difficulty of the current demonstration melody. The difficulty of the next demonstration melody should be less than the difficulty of the current demonstration melody.

[0128] Specifically, the difficulty of the demonstrated melody lies primarily in the difficulty of its rhythmic pattern. Given the same beat and duration, the more complex the rhythmic type of the melody, the higher the difficulty. For example, eighth-note rhythms are more difficult than quarter-note rhythms. The difficulty gradually increases with the following rhythmic patterns: evenly divided note values, alternating long and short notes, dotted rhythms, syncopated rhythms, and rhythms containing rests.

[0129] On the other hand, electronic devices can adjust the difficulty based on the duration of each beat; the longer the duration, the lower the difficulty. The electronic devices store the difficulty of each type of rhythm for each beat at different durations.

[0130] If the difficulty of the current demonstration melody is 3, then the difficulty of the next demonstration melody is set to 2.

[0131] Step S24: Determine a new rhythmic melody based on the difficulty of the next demonstration melody.

[0132] Specifically, the electronic device first changes the duration and then the rhythm type to reduce the difficulty of the demonstrated melody. For example, if the current rhythm type is an eighth tempo, the electronic device first increases the duration of each beat to determine a new rhythm type. If the user still cannot operate accurately after the duration of each beat has increased to the upper limit, the electronic device then determines the new rhythm type to be a quarter tempo.

[0133] Step S25: Generate a new demonstration melody and a new operational melody based on the new rhythmic pattern melody.

[0134] Specifically, the electronic device replaces the old rhythmic melody with the new rhythmic melody to generate a new demonstration melody. Furthermore, the electronic device generates a new blank melody of the same duration as the new rhythmic melody and replaces the old blank melody with the new blank melody, thereby generating a new operational melody.

[0135] Furthermore, in order to indicate the location of the operation error to the user, after confirming that the operation information is incorrect, the above method also includes (steps S31 to S33):

[0136] Step S31: Obtain the mismatched segments from the operation information.

[0137] Specifically, when comparing operational information with rhythmic melodies, electronic devices can identify mismatched segments. Each segment where a note is ahead or behind is considered a mismatched segment.

[0138] Step S32: Generate a playback melody by combining the operation melody and operation information.

[0139] Specifically, the electronic device inserts the operation information into the operation melody based on the operation information and the timeline of the operation melody playback, and combines them to generate the playback melody.

[0140] Step S33: When playing the revisit melody, increase the volume of the mismatched segments.

[0141] Specifically, increasing the volume of mismatched segments can draw the user's attention, and the user's operation information may overlap with the first or second non-rhythmic melody. The overlapping sound may reduce the user's attention to the mismatched segments, so the volume is increased.

[0142] Furthermore, to enrich the fun of musical rhythm, the above method also includes: determining the type of rhythmic melody according to the user's needs, including types without strong and weak beats and types with strong and weak beats.

[0143] A rhythmic pattern without strong or weak beats refers to a single beat where all notes have the same dynamic range, meaning the user can control the rhythm by pressing the same key repeatedly. A rhythmic pattern with strong or weak beats refers to a beat where multiple notes have varying dynamic ranges. Generally, the first note is the strongest, and the remaining notes are categorized into three types: secondary strongest, secondary weakest, and weakest. Each type of strong or weak beat corresponds to a single key press. Therefore, rhythmic melodies with strong or weak beats are generally more difficult than those without.

[0144] To better implement the above method, this application also provides a device for recognizing user operations through music rhythm, referring to... Figure 5 The user operation device 200, which recognizes user operation through music rhythm, includes:

[0145] The demonstration melody generation module 201 is used to generate a demonstration melody by combining at least one first non-rhythmic melody, a rhythmic melody and at least one second non-rhythmic melody arranged in sequence.

[0146] The operation melody generation module 202 is used to generate an operation melody by combining at least one first non-rhythmic melody, a blank melody and at least one second non-rhythmic melody arranged in sequence, wherein the duration of the blank melody is equal to the duration of the rhythmic melody.

[0147] The operation information acquisition module 203 is used to play the demonstration melody and the operation melody in sequence, and to acquire the operation information of the user imitating the rhythmic melody when playing the blank melody.

[0148] The judgment module 204 is used to determine whether the operation information matches the rhythmic melody.

[0149] Execution module 205 is used to change the parameter value of the target object when the judgment module determines that it is true;

[0150] Error determination module 206 is used to determine that the operation information is incorrect when the judgment module determines that it is not.

[0151] Furthermore, the judgment module 204 is specifically used for:

[0152] The rhythmic melody is divided into multiple first sub-melodies, each of which contains one note;

[0153] The operation information is divided into multiple second sub-melodies based on the duration of the first sub-melodies;

[0154] Determine the first start and first end times of each note in the rhythmic melody, as well as the second start and second end times of each note in the operation information;

[0155] Compare the second start time in each second submelody with the first start time in the corresponding first submelody, and / or compare the second end time in each second submelody with the first end time in the corresponding first submelody to determine the degree of matching between the second submelody and the corresponding first submelody;

[0156] Based on the matching degree between each second sub-melody and its corresponding first sub-melody in the operation information, determine whether the operation information matches the rhythmic melody.

[0157] Furthermore, when determining the matching degree by comparing the second start time in each second sub-melody with the first start time in the corresponding first sub-melody, and / or comparing the second end time in each second sub-melody with the first end time in the corresponding first sub-melody, the judgment module 204 is specifically used for:

[0158] Determine whether there is a second start moment for the corresponding note in the second sub-melody;

[0159] If so, then

[0160] Determine the first difference between the second start time in the second sub-melody and the first start time in the corresponding first sub-melody;

[0161] Determine the second difference between the second end time of the corresponding note in the second sub-melody and the first end time of the corresponding first sub-melody;

[0162] Add the absolute values ​​of the first and second differences to get the median value. Calculate the matching degree by the ratio of the median value to the duration of the first sub-melody.

[0163] If not, then the match rate is determined to be zero.

[0164] In another possible implementation, the user operation device 200, which recognizes music rhythm, also includes:

[0165] The count determination module is used to determine whether the number of times the same operation information is erroneous within a continuous time period has reached a preset number;

[0166] The replay module is used to replay the demonstration melody and the operation melody when the count judgment module determines that it is not true.

[0167] The difficulty determination module is used to determine the difficulty of the next demonstration melody based on the difficulty of the current demonstration melody when the count judgment module determines it to be true. The difficulty of the next demonstration melody is less than the difficulty of the current demonstration melody.

[0168] A new rhythmic melody determination module is used to determine a new rhythmic melody based on the difficulty of the next demonstration melody;

[0169] A new operational melody determination module is used to generate new demonstration melodies and new operational melodies based on new rhythmic patterns.

[0170] In another possible implementation, the user operation device 200, which recognizes music rhythm, also includes:

[0171] The mismatched segment acquisition module is used to acquire mismatched segments from the operation information;

[0172] The playback melody generation module is used to combine and generate playback melodies based on the operation melody and operation information.

[0173] The volume boost module is used to increase the volume of mismatched segments when playing a replay melody.

[0174] In another possible implementation, the user operation device 200, which recognizes music rhythm, also includes:

[0175] The type determination module is used to determine the type of rhythmic melody according to the user's needs. The types include no dynamic type and dynamic type.

[0176] The various variations and specific examples of the methods in the foregoing embodiments are also applicable to the user operation recognition device based on music rhythm in this embodiment. Through the foregoing detailed description of the user operation recognition method based on music rhythm, those skilled in the art can clearly understand the implementation method of the user operation recognition device based on music rhythm in this embodiment. Therefore, for the sake of brevity, it will not be described in detail here.

[0177] To better implement the above methods, embodiments of this application provide an electronic device, referring to... Figure 6 The electronic device 300 includes a processor 301 and a memory 303. The memory 303 is connected to the processor 301, such as via a bus 302. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that in practical applications, the transceiver 304 is not limited to one type, and the structure of this electronic device 300 does not constitute a limitation on the embodiments of this application.

[0178] Processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0179] Bus 302 may include a pathway for transmitting information between the aforementioned components. Bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 302 may be divided into address bus, data bus, control bus, etc.

[0180] The memory 303 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0181] The memory 303 is used to store application code that executes the solution of this application, and its execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the content shown in the foregoing method embodiments.

[0182] Figure 6 The electronic device 300 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0183] This application embodiment also provides a computer-readable storage medium storing a computer program. When executed by a processor, the program implements the user operation recognition method based on music rhythm provided in the above embodiment. The processor loads and executes the application program in the computer-readable storage medium, generates a demonstration melody by combining a first non-rhythmic melody, a rhythmic melody, and a second non-rhythmic melody, and then determines a blank melody with the same duration as the rhythmic melody. An operation melody is generated based on the first non-rhythmic melody, the blank melody, and the second non-rhythmic melody. The demonstration melody and the operation melody are played sequentially, and the user's operation information when the blank melody is played is obtained. The operation melody is then compared with the rhythmic melody to determine whether they match. If the music rhythm matches, the user's operation is correct, and the parameter value of the target object is changed. If they do not match, the user's operation information is determined to be incorrect. This method can recognize the operation information of blind users regarding music rhythm, enabling blind users to operate on the target object quickly and accurately, and enabling blind users to access the Internet smoothly.

[0184] In this embodiment, the computer-readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. The computer-readable storage medium can be, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof. Specifically, the computer-readable storage medium can be a portable computer disk, a hard disk, a USB flash drive, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), staging random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory stick, floppy disk, optical disk, magnetic disk, mechanical encoding device, or any combination thereof.

[0185] The computer program in this embodiment includes program code for performing all the aforementioned methods. The program code may include instructions corresponding to the method steps provided in the above embodiments. The computer program can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The computer program can be executed entirely on the user's computer as a standalone software package.

[0186] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

[0187] Additionally, it should be understood that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

Claims

1. A method for recognizing user operations through music rhythm, characterized in that, include: A demonstration melody is generated by combining at least one first non-rhythmic melody, one rhythmic melody, and at least one second non-rhythmic melody arranged in sequence. An operational melody is generated by combining the at least one first non-rhythmic melody, the blank melody, and the at least one second non-rhythmic melody arranged in sequence, wherein the duration of the blank melody is equal to the duration of the rhythmic melody. Play the demonstration melody and the operation melody in sequence, and obtain the operation information of the user imitating the rhythmic melody when playing the blank melody; Determine whether the operation information matches the rhythmic melody. If so, change the parameter value of the target object, and the target object will perform the corresponding action. The target object is the object operated by the user. Otherwise, the operation information is determined to be incorrect.

2. The method according to claim 1, characterized in that, The step of determining whether the operation information matches the rhythmic melody includes: The rhythmic melody is divided into multiple first sub-melodies, each of which includes one note; The operation information is divided into multiple second sub-melodies based on the duration of the first sub-melodies; Determine the first start time and the first end time of each note in the rhythmic melody, and the second start time and the second end time of each note in the operation information; Compare the second start time in each second sub-melody with the first start time in the corresponding first sub-melody, and / or compare the second end time in each second sub-melody with the first end time in the corresponding first sub-melody to determine the matching degree between the second sub-melody and the corresponding first sub-melody; Based on the matching degree between each second sub-melody and the corresponding first sub-melody in the operation information, it is determined whether the operation information matches the rhythmic melody.

3. The method according to claim 2, characterized in that, The step of comparing the second start time in each second sub-melody with the first start time in the corresponding first sub-melody, and / or comparing the second end time in each second sub-melody with the first end time in the corresponding first sub-melody to determine the matching degree includes: Determine whether there is a second start moment for the corresponding note in the second sub-melody; If so, then Determine the first difference between the second start time in the second sub-melody and the first start time in the corresponding first sub-melody; Determine the second difference between the second end time of the corresponding note in the second sub-melody and the first end time of the corresponding first sub-melody; The absolute values ​​of the first difference and the second difference are added together to obtain the median value. The matching degree is obtained by calculating the ratio of the median value to the duration of the first sub-melody. If not, then the match rate is determined to be zero.

4. The method according to claim 1, characterized in that, After determining that the operation information is incorrect, the method further includes: Determine whether the number of errors in the same operation information within a continuous time period reaches a preset number; If not, then play the demonstration melody and the operation melody again; If so, the difficulty of the next demonstration melody is determined based on the difficulty of the current demonstration melody, wherein the difficulty of the next demonstration melody is less than the difficulty of the current demonstration melody; A new rhythmic melody is determined based on the difficulty of the next demonstrated melody; Based on the new rhythmic melody, generate a new demonstration melody and a new operational melody.

5. The method according to claim 1, characterized in that, After determining that the operation information is incorrect, the method further includes: Obtain the mismatched segments from the operation information; Based on the operation melody and the operation information, a playback melody is generated by combining them. While playing the replay melody, increase the volume of the mismatched segments.

6. The method according to claim 1, characterized in that, The notes of the rhythmic melody are percussion instruments, and the notes of the non-rhythmic melody are non-percussion instruments.

7. The method according to claim 1, characterized in that, The method further includes: determining the type of rhythmic melody according to the user's needs, wherein the type includes a type without strong or weak beats and a type with strong or weak beats.

8. A device for recognizing user operations through music rhythm, characterized in that, include: The demonstration melody generation module is used to generate a demonstration melody by combining at least one first non-rhythmic melody, a rhythmic melody, and at least one second non-rhythmic melody arranged in sequence. The operation melody generation module is used to generate an operation melody by combining the at least one first non-rhythmic melody, the blank melody and the at least one second non-rhythmic melody arranged in sequence, wherein the duration of the blank melody is equal to the duration of the rhythmic melody. The operation information acquisition module is used to play the demonstration melody and the operation melody in sequence, and acquire the operation information of the user imitating the rhythmic melody when playing the blank melody. The judgment module is used to determine whether the operation information matches the rhythmic melody; An execution module is used to change the parameter value of the target object when the judgment module determines that it is true, and the target object then performs the corresponding action, wherein the target object is the object operated by the user; An error determination module is used to determine that the operation information is incorrect when the judgment module determines that it is not.

9. An electronic device, characterized in that, At least one processor; Memory; At least one application, wherein the at least one application is stored in the memory and configured to be executed by the at least one processor, the at least one application being configured to: perform the method for recognizing user actions by music rhythm as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored and can be loaded by a processor and executed as described in any one of claims 1 to 7, which identifies a user's operation method by means of music rhythm.

Images