Method, device, robot and medium for constructing robot hand motion sequence

Abstract

This application discloses a method, apparatus, robot, and medium for constructing a robot's hand motion sequence. The method includes assigning candidate finger indices to text characters based on preset fingering rules and key values; obtaining the movement cost of the candidate finger indices and selecting a finger index for the text character from the candidate finger indices based on the movement cost; and determining the hand motion sequence corresponding to the preset text based on the finger index and key values ​​of the text character. This application first assigns one or more candidate finger indices to the text character according to preset fingering rules, and then selects the candidate finger index with the lowest movement cost as the finger index of the text character, so as to reduce the movement time between notes. This can improve the smoothness of the preset text operation and shorten the operation time of the preset text.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a method, apparatus, robot, and medium for constructing a robot's hand motion sequence. Background Technology

[0002] With the rapid development of artificial intelligence, a large number of robots have emerged in various practical applications, such as intelligent educational robots, emotional communication robots, cooking robots, and performance robots. As the precision of robots continues to improve, the degree of anthropomorphism they can achieve is becoming increasingly higher, leading to the emergence of robots with even greater anthropomorphic requirements, such as piano-playing robots.

[0003] Existing piano-playing robots consist of a robotic arm and a robotic hand. The robotic arm moves to the playing position, and the robotic hand presses the keys to play music. Typically, the correspondence between the robotic hand's fingers and notes is pre-defined. When the robot needs to play a piece, it directly determines the corresponding finger for each note based on this fixed correspondence, enabling accurate playing. However, this method increases the movement time between notes, thus affecting the overall fluency of the playing and ultimately impacting the overall performance quality.

[0004] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0005] The technical problem to be solved by this application is to provide a method, device, robot and medium for constructing a robot's hand motion sequence, in order to address the shortcomings of the prior art.

[0006] To address the aforementioned technical problems, the first aspect of this application provides a method for constructing a robot's hand motion sequence, wherein the method specifically includes:

[0007] Candidate finger indices are assigned to text characters in a preset text according to preset finger placement rules and the key values ​​of text characters, wherein the text characters carry key values;

[0008] For a text character corresponding to a candidate finger index, the candidate finger index is used as the finger index of the text character;

[0009] For a text character corresponding to at least two candidate finger indices, obtain the movement cost of the candidate finger indices, and select a finger index for the text character from the at least two candidate finger indices according to the movement cost;

[0010] The hand movement sequence corresponding to the preset text is determined based on the finger index and key value of the text characters.

[0011] The method for constructing the hand movement sequence of the robot, wherein the preset fingering rules include a fingering allocation table and a correspondence between key types and finger indices; the step of allocating candidate finger indices to text characters in a preset text based on the preset fingering rules and the key position values ​​of text characters specifically includes:

[0012] Match the key values ​​of the text characters with the finger assignment table;

[0013] For a successfully matched key value, the matched finger index is used as the candidate finger index for the text character;

[0014] For key values ​​that fail to match, candidate finger indices are assigned to the text characters according to the correspondence between the key type and the finger index.

[0015] The method for constructing the hand motion sequence of the robot, wherein the movement cost for obtaining candidate finger indices specifically includes:

[0016] Obtain the finger position sequence of the preceding text character;

[0017] The candidate finger index is used as the finger index of the text character, and the finger position sequence of the text character is obtained;

[0018] The current movement cost of the candidate finger index is calculated based on the finger position sequence of the previous text character and the finger position sequence of the text character.

[0019] Obtain the key values ​​of several subsequent text characters of the text character, and calculate the future movement cost of the candidate finger index based on the key values ​​of the subsequent text characters and the finger position sequence of the text character;

[0020] The movement cost of the candidate finger index is calculated based on the current movement cost and the future movement cost.

[0021] The method for constructing the hand movement sequence of the robot, wherein determining the hand movement sequence corresponding to the preset text based on the finger index and key value of the text character specifically includes:

[0022] Obtain the finger position sequence of the text characters, and select a preset finger position from the finger position sequence of the text characters;

[0023] Associating the finger index of the text character with the selected finger position to obtain the finger movement of the text character;

[0024] The preset text is converted into a hand movement sequence based on the finger movements of the text characters.

[0025] The method for constructing the hand motion sequence of the robot, wherein the process of obtaining the finger position sequence of the text characters specifically includes:

[0026] The key value of the text character is used as the finger position corresponding to the finger index of the text character;

[0027] For each adjacent finger index other than the finger index of the text character, calculate the index difference between the adjacent finger index and the finger index of the text character, and determine the adjacent finger position of the adjacent finger index based on the preset finger position relationship, the index difference and the key value of the text character;

[0028] A finger position sequence is constructed based on the finger position and the adjacent finger positions.

[0029] The method for constructing the hand movement sequence of the robot, wherein after determining the hand movement sequence corresponding to the preset text based on the finger index of the text character, the method further includes:

[0030] Control the robotic arm to execute the first finger movement in the hand motion sequence, and calculate the movement time between the first finger movement and the second finger movement in the hand motion sequence;

[0031] Once the first finger action is completed, the robotic arm is controlled to move by the movement time and execute the second finger action;

[0032] This process continues until the last finger movement of the hand movement sequence is completed, thus completing the execution of the preset text.

[0033] The method for constructing a robot's hand motion sequence, wherein before assigning candidate finger indices to text characters in a preset text based on preset finger placement rules and key values ​​of text characters, the method further includes:

[0034] Obtain the key value corresponding to the text character in the preset text, and associate the key value with the text character.

[0035] A second aspect of this application provides an apparatus for constructing a robot's hand motion sequence, wherein the apparatus specifically includes:

[0036] The allocation module is used to allocate candidate finger indices to text characters in a preset text according to preset fingering rules and the key values ​​of text characters, wherein the text characters carry key values;

[0037] The first selection module is used to select the candidate finger index of the text character corresponding to a candidate finger index as the finger index of the text character.

[0038] The second selection module is used to obtain the movement cost of each candidate finger index of a text character corresponding to at least two candidate finger indices, and select a finger index for the text character from the at least two candidate finger indices according to the movement cost;

[0039] The determination module is used to determine the hand movement sequence corresponding to the preset text based on the finger index and key value of the text characters.

[0040] A third aspect of this application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the method for constructing a robot hand motion sequence as described above.

[0041] A fourth aspect of this application provides a robot comprising: a processor and a memory;

[0042] The memory stores a computer-readable program that can be executed by the processor;

[0043] When the processor executes the computer-readable program, it implements the steps in the method for constructing the hand motion sequence of a robot as described above.

[0044] Beneficial Effects: Compared with the prior art, this application provides a method, apparatus, robot, and medium for constructing a robot's hand motion sequence. The method includes assigning candidate finger indices to text characters in a preset text according to preset fingering rules and the key values ​​of text characters; for a text character corresponding to one candidate finger index, using that candidate finger index as the finger index of the text character; for a text character corresponding to at least two candidate finger indices, obtaining the movement cost of the candidate finger indices, and selecting a finger index for the text character from the at least two candidate finger indices according to the movement cost; and determining the hand motion sequence corresponding to the preset text based on the finger index and key values ​​of the text character. This application first assigns one or more candidate finger indices to the text character according to preset fingering rules, and then selects the candidate finger index with the lowest movement cost as the finger index of the text character, thereby reducing the movement time between notes. This improves the smoothness of the preset text operation and shortens the operation time of the preset text. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating a method for constructing a robot's hand motion sequence as provided in an embodiment of this application.

[0047] Figure 2 This is a flowchart illustrating the process of acquiring the preset text.

[0048] Figure 3 This is a flowchart illustrating the process of obtaining sheet music.

[0049] Figure 4 This is a flowchart illustrating the process of obtaining the movement cost.

[0050] Figure 5 This is a flowchart illustrating the process of constructing hand motion sequences.

[0051] Figure 6 A schematic diagram of a device for constructing a robot's hand motion sequence provided in an embodiment of this application.

[0052] Figure 7 A schematic diagram of the robot provided in the embodiments of this application. Detailed Implementation

[0053] This application provides a method, apparatus, robot, and medium for constructing a robot's hand motion sequence. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit this application.

[0054] Those skilled in the art will understand that, unless explicitly stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0055] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0056] It should be understood that the sequence number and size of each step in this embodiment do not imply the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0057] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0058] This embodiment provides a method for constructing a robot's hand movement sequence. This method is applicable to scenarios where the robot performs text operations by executing operation keys located on the same row in the operating device. For example, the robot can play a piano piece, input a number sequence by using the number keys set in the row direction on the keyboard, or input an English article by using a row of letter keys.

[0059] Specifically, if Figure 1 As shown in the figure, this embodiment provides a method for constructing a robot's hand motion sequence, which specifically includes:

[0060] S10. Based on the preset finger placement rules and the key position values ​​of the text characters, assign candidate finger indices to the text characters in the preset text.

[0061] Specifically, the preset text can be pre-loaded within the robot, interacted with by the user, or sent to the robot by an external device; it is content that requires robot operation, such as a piano piece, a number sequence, or a Chinese article. It should be noted that when the preset text is interacted with by the user or sent to the robot by an external device, the user or external device can send only a text description corresponding to the preset text (e.g., the name of the piece, the storage location of the number sequence, the title of the Chinese article, etc.). The robot can download the text content based on the text description and use that text content as the preset text.

[0062] It should be noted that when the preset text needs to be operated synchronously by the robot's left and right robotic arms, the preset text can be divided into left-hand text and right-hand text based on the range of motion of the left and right robotic arms. Then, the left-hand text and right-hand text are used as preset text to construct left-hand motion sequences for the left robotic arm and right-hand motion sequences for the right robotic arm, respectively. For ease of explanation, this example illustrates constructing a hand motion sequence for a robotic arm based on preset text.

[0063] In one embodiment, the preset text is the name of a piano piece. Before assigning candidate finger indices to the text characters in the preset text based on preset fingering rules and the key values ​​of the text characters, such as... Figure 2 As shown, the method further includes:

[0064] S11. Receive the track to be played from user interaction and obtain the MIDI file of the track to be played;

[0065] S12. Convert the MIDI file into preset text.

[0066] In step S11, user interaction can be through text or semantic means to interact with the robot about the piece to be played. For example, the user can input the piece to be played through the keyboard, the microphone can pick up the user's voice and convert the user's voice into text information to obtain the piece to be played, or the user can input the piece to be played by triggering the piece selection button, etc.

[0067] The MIDI file for the piece to be performed can be obtained from the internet or found in a local database. MIDI (Musical Instrument Digital Interface) is a digital music interface standard used to transmit music data between electronic musical instruments, computers, and other musical devices. A MIDI file consists of two parts: the Header Chunk (MThd) and the TrackChunk (MTrk). The MThd is the file header of the MIDI format file, containing basic information such as the format and attributes of the MIDI file, including the format type, number of tracks, and time division. The MTrk is the track information in the MIDI format file, which includes note information for each note, such as the note start time, note end time, pitch, and velocity.

[0068] In step S12, the sheet music is generated from a MIDI file, which includes performance information for each note in the piece to be played. That is, after obtaining the MIDI file, the track information of the sheet music is extracted according to the MIDI format's sheet music storage rules, and the performance information of the notes is determined based on the extracted track information. Then, the performance information of each note is sorted to obtain the sheet music. Based on this, in one implementation, such as... Figure 3 As shown, converting the audio tracks in the MIDI file into sheet music specifically includes:

[0069] S121. Extract the note start time, note end time, pitch value, and velocity value of notes from a MIDI file;

[0070] S122. Decompose the pitch value of each note into basic notes and octave numbers, determine the note name of the note based on the basic notes, and combine the note name and the octave number to obtain the standard note symbol of the note.

[0071] S123. Calculate the duration of the note based on the start time and end time of the note;

[0072] S124. Combine the standard note symbol, the note start time, and the note duration into performance information;

[0073] S125. Sort the performance information according to the start time of the notes to obtain the score.

[0074] In step S121, the piece to be played may correspond to multiple tracks, thus the MIDI file may include multiple track blocks, each track block corresponding to one track, used to store the track information of that track. Therefore, when extracting the note start time, note end time, pitch value, and velocity value from the MIDI file, the note start time, note end time, pitch value, and velocity value are extracted from each track separately, so as to construct a corresponding score for each track. This allows for the reading of the score corresponding to each track from the score when determining the melody and accompaniment scores based on the tracks, improving the speed of dividing the melody and accompaniment scores.

[0075] In step S122, the basic note is a pre-set basic pitch value used to determine the note name, and the octave number is used to represent the offset of the note relative to the basic note, determined based on the pitch value of the note and the pitch range of the basic pitch value. That is, a pre-set note sequence containing multiple consecutive basic notes is obtained, the basic note corresponding to that note is selected from the note sequence, and then the octave number of the note is calculated based on the pitch value and the pitch range corresponding to the note sequence. The calculation formulas for the basic note and the octave number can be:

[0076] ,

[0077] ,

[0078] in, The basic pitch value represents the basic note. This indicates the pitch range of the preset basic pitch value sequence. Indicates pitch value, Indicates octave numbering. This represents the modulo operation. This represents the floor function.

[0079] Furthermore, the standard note symbols include the note names and octave numbers of the basic notes. The standard note symbols can be used to convert notes in MIDI files into notes in sheet music, so that each note in the converted sheet music uses the standard note symbols. The note names of the basic notes are selected according to the preset correspondence between basic notes and note names.

[0080] For example: The preset basic note sequence is {0, 1, ..., 11}, the pitch range is 12, and the correspondence between basic notes and note names is as follows: The pitch value of the note is 60, the basic note corresponding to the note = 60 mod 12 = 0, and the octave number = The name of the note corresponding to the basic note is The standard note symbol corresponding to this note is .

[0081] In steps S123 and S124, the duration of a note refers to the duration of the note, which is the time difference between the end time of the note and the start time of the note, i.e., the duration of the note. , Indicates the end time of the note. This indicates the start time of the note. After obtaining the note duration and standard note symbol, the data group consisting of the standard note symbol, the note start time, and the note duration is used as performance information.

[0082] In step S125, the note start time is used to indicate when the note begins to be played. That is, the playing order of each note can be determined according to the note start time, i.e., the note with an earlier start time is played before the note with a later start time. Therefore, after obtaining the playing information corresponding to each note, the playing information of each note can be sorted in ascending order of note start time to obtain the score, and the score is used as a preset text.

[0083] Furthermore, the key value is the key number in the operating device, and the operating key in the operating device can be determined based on the key value. For example, if the operating device is a piano, the key value is the number of the piano keys. The key value corresponding to the text character in the preset text can be determined after obtaining the preset text. Accordingly, before allocating candidate finger indices according to the preset fingering rules and the key value of the text character in the preset text, the method further includes:

[0084] Obtain the key value corresponding to the text character in the preset text, and associate the key value with the text character.

[0085] Specifically, the key values ​​can be obtained based on the correspondence between text characters and operation keys. For example, when the text characters are musical notes and the operation keys are piano keys, the correspondence between the text characters and operation keys can be: When the text characters are numbers and the operation keys are arranged in a row of number keys, the correspondence between the text characters and the operation keys can be: After obtaining the key value corresponding to the text character, the key value is associated with the text character so that each text character in the preset text carries a key value.

[0086] Preset fingering rules are the basic principles for determining the finger indices corresponding to text characters. These rules can vary depending on the text type of the preset text. That is, after obtaining the preset text, the corresponding preset fingering rule can be selected based on its text type, and then finger indices can be assigned to the preset text according to that rule. Text types can include sheet music and text types, among others.

[0087] A finger index is a unique identifier pre-configured for each finger of the robotic hand. Each finger in the robotic hand corresponds to a finger index, and the finger indices of each finger are distinct. For example, the robotic hand includes an index finger, middle finger, ring finger, and little finger. The index finger has a finger index of 0, the middle finger has a finger index of 1, the ring finger has a finger index of 2, the little finger has a finger index of 3, and so on. Candidate finger indices are alternative finger indices assigned to text characters according to preset fingering rules, satisfying the basic principle. One of the candidate finger indices is used to determine the finger that executes the text character. The candidate finger index assigned to each character according to the preset fingering rules can be one or more (including two or more).

[0088] In one embodiment, assigning candidate finger indices to text characters in a preset text based on preset finger placement rules and key values ​​of text characters specifically includes:

[0089] Match the key values ​​of the text characters with the finger assignment table;

[0090] For a successfully matched key value, the matched finger index is used as the candidate finger index for the text character;

[0091] For key values ​​that fail to match, candidate finger indices are assigned to the text characters according to the correspondence between the key type and the finger index.

[0092] Specifically, the preset finger placement rules include a finger placement allocation table and a key type-finger index correspondence. The finger placement allocation table includes finger indices corresponding to key values, allowing selection of finger indices based on key values. The key type-finger index correspondence serves as a supplementary condition for the finger placement allocation table, used to assign finger indices to text characters for which no finger index is matched according to the preset finger placement rules. When assigning candidate finger indices to text characters according to the preset finger placement rules, the finger index corresponding to the text character is first matched in the finger placement allocation table, and then the key type-finger index correspondence is used to assign finger indices to text characters for which no finger index is matched according to the preset finger placement rules.

[0093] For example: Assume the preset text type is music score type, and the fingering assignment table is shown in Table 1. The correspondence between key type and finger index can be as follows: If the key position value corresponds to a white key, then the finger index of one of the fingers (index, middle, ring, and little fingers) is used as the finger index corresponding to the note. The corresponding key value is 0. The preset fingering rule table finds the finger index corresponding to key value 0 to be 3. Therefore, the note... It can only be played with the ring finger, the notes The corresponding key value is 18. No finger index for key value 18 was found in the preset fingering rule table. The key corresponding to key value 18 is a white key. Therefore, the finger index of one of the following fingers—index, middle, ring, or pinky—can be selected as the finger index corresponding to key value 18, i.e., the note. You can use any one of the following fingers: index, middle, ring, or little finger.

[0094] Table 1 Finger Assignment Table

[0095]

[0096] S20. For a text character corresponding to a candidate finger index, the candidate finger index is used as the finger index of the text character; for a text character corresponding to at least two candidate finger indices, the movement cost of the candidate finger indices is obtained, and a finger index is selected for the text character from the at least two candidate finger indices according to the movement cost.

[0097] Specifically, each text character in the preset text corresponds to at least one candidate finger index; that is, a text character can have one, two, or more candidate finger indices. When a text character has one candidate finger index, that candidate finger index can be directly used as the finger index of the text character. When a text character has two or more candidate finger indices, one candidate finger index needs to be selected from the two or more candidate finger indices as the finger index of the text character.

[0098] Movement cost is the basis for finger index selection, representing the time cost of the robotic arm manipulating text characters. A higher movement cost indicates a higher time cost to move from the previous text character to the current one, and vice versa. For robot manipulation of preset text, a lower time cost for text character manipulation results in higher execution continuity and efficiency. Therefore, when selecting a finger index for a text character from at least two candidate finger indices based on movement cost, the candidate finger index with the lowest movement cost can be selected as the finger index for the text character.

[0099] In one embodiment, such as Figure 4 As shown, the movement cost for obtaining candidate finger indices specifically includes:

[0100] S21. Obtain the finger position sequence of the preceding text character;

[0101] S22. Use the candidate finger index as the finger index of the text character, and obtain the finger position sequence of the text character;

[0102] S23. Calculate the current movement cost of the candidate finger index based on the finger position sequence of the previous text character and the finger position sequence of the text character;

[0103] S24. Obtain the key values ​​of several subsequent text characters of the text character, and calculate the future movement cost of the candidate finger index based on the key values ​​of the subsequent text characters and the finger position sequence of the text character.

[0104] S25. Calculate the movement cost of the candidate finger index based on the current movement cost and the future movement cost.

[0105] In step S21, the preceding text character refers to the text character that is closest to the text character in the operation sequence. For example, if the text character is the third text character in the preset text, then the preceding text character is the second text character. The finger position sequence includes the finger positions of each finger in the robotic hand. For example, when the robotic hand includes the index finger, middle finger, ring finger, and little finger, the finger position sequence is {index finger position, middle finger position, ring finger position, little finger position}. Here, finger position represents the key value where the finger is located when the robotic hand operates on the text character, and the number of interval operation keys between two adjacent finger positions in the finger position sequence is the same, such as 1, 2, 3, etc.

[0106] The finger position sequence of the previous text character is calculated and stored by the robot when determining the finger index of the previous text character. When storing the finger position sequence corresponding to the previous text character, the robot can directly store the finger position sequence as a storage object; or it can store the finger positions of preset fingers and preset finger position relationships in the finger position sequence, and restore the finger position sequence based on the finger positions of preset fingers and preset finger position relationships when using the finger position sequence.

[0107] In this embodiment, the robot stores preset finger positions and preset finger position relationships. The preset finger position relationships include the number of key intervals between adjacent finger positions and the direction of finger position expansion, thereby reducing the robot's memory usage. For example, if the preset index finger is the index finger, the number of key intervals is 1, and the expansion direction is from left to right, then the finger position sequence of text characters is stored as the index finger position. When a finger position sequence is needed, it can be obtained based on the number of interval operation keys and the direction of expansion. The finger position sequence is as follows: .

[0108] In step S22, the finger position sequence of the text character is determined based on the key value and finger index corresponding to the text character. That is, after using the candidate finger index as the finger index, the finger position sequence corresponding to the text character when the robotic finger corresponding to the finger index is moved to the key value is calculated based on the key value of the text character, the finger index of the text character, and the preset finger position relationship.

[0109] In one embodiment, the process of obtaining the finger position sequence of the text characters specifically includes:

[0110] The key value of the text character is used as the finger position corresponding to the finger index of the text character;

[0111] For each adjacent finger index other than the finger index of the text character, calculate the index difference between the adjacent finger index and the finger index of the text character, and determine the adjacent finger position of the adjacent finger index based on the preset finger position relationship, the index difference and the key value of the text character;

[0112] A finger position sequence is constructed based on the finger position and the adjacent finger positions.

[0113] Specifically, adjacent finger indices refer to the finger indices in the robotic hand other than the finger index of the text character. For example, the finger indices of the robotic hand include the index finger index, middle finger index, ring finger index, and little finger index. The finger index corresponding to the text character is the middle finger index. Then, the adjacent finger indices include the index finger index, ring finger index, and little finger index.

[0114] Furthermore, to avoid the finger positions exceeding the operation key range of the device due to excessively large finger index intervals in the robotic arm, the finger index interval can be set to 1. Alternatively, after obtaining the index difference between adjacent finger indices and the finger index of the text character, all index differences can be reduced by an equal factor so that the smallest index difference is 1. For example, if the robotic arm includes an index finger, middle finger, ring finger, and little finger, and the index of the index finger is 0, the index of the middle finger is 1, the index of the ring finger is 2, and the index of the little finger is 3, then the calculated index difference can be used directly to determine the finger position sequence. If the index of the index finger is 0, the index of the middle finger is 2, the index of the ring finger is 4, and the index of the little finger is 6, then after obtaining the index difference, all index differences can be reduced by a factor of 2.

[0115] As described above, the preset finger position relationship includes the number of key intervals between two adjacent finger positions and the direction of finger position expansion. After obtaining the index difference, the key value distance between the adjacent finger position of the adjacent finger index and the key value of the text character can be calculated based on the index difference and the number of key intervals. Then, the positive or negative direction of this key value distance can be determined based on the expansion direction. Therefore, the adjacent finger position of the adjacent finger index can be calculated based on the key value of the text character, the key value distance, and the positive or negative direction. Finally, the sequence formed by the adjacent finger positions of the adjacent finger index and the finger position of the text character is taken as the finger position sequence of the text character.

[0116] In one embodiment, the number of operation keys between two adjacent finger positions is 1, and the expansion direction is from left to right. Therefore, the formula for calculating the adjacent finger position of the adjacent finger index can be:

[0117] ,

[0118] in, The key value representing a text character. Indicates the index of adjacent fingers. The finger index representing a text character.

[0119] For example: when the finger index of the text character is the index of the right ring finger, that is... If the key value for a text character is 18, then the index finger index (i.e. The finger position of the index finger is 18 + 2 * (0 - 2) = 14. The finger position of the little finger is 18 + 2*(1-2) = 16, and the index of the little finger (i.e. The finger position of the text character is 18 + 2*(3-2) = 20, thus the finger position sequence of the text character is... .

[0120] Further, in step S23, after obtaining the finger position sequence of the previous text character and the finger position sequence of the text character, the movement distance of the robotic arm from the finger position sequence of the previous text character to the finger position sequence of the text character can be calculated, and then this movement distance is used as the current movement cost of the candidate finger index. The formula for calculating the movement distance can be:

[0121] ,

[0122] in, Indicates the current movement cost. Finger index representing text character The position of the fingers, Finger index representing the previous text character The position of the fingers.

[0123] Further, in step S24, the "several subsequent text characters" are the text characters that are located after and closest to the current text character in the operation sequence. For example, if the "several subsequent text characters" include three subsequent text characters, and the current text character is the third text character, then the key values ​​of the fourth, fifth, and sixth text characters will be obtained. Then, based on the key values ​​of each subsequent text character and the finger position sequence of the text character, the future movement cost of the candidate finger index is calculated. The future movement cost is used to predict the movement distance of the robotic arm to move the operation key position of the subsequent text character. The formula for calculating the future movement cost is as follows:

[0124] ,

[0125] in, Indicates the cost of future mobility. Finger index representing text character The position of the fingers, Indicates the key value of the subsequent text character.

[0126] Furthermore, in step S25, the movement cost can be the sum of the current movement cost and the future movement cost, or it can be a weighted average of the current movement cost and the future movement cost, or it can be the average of the current movement cost and the future movement cost, etc. In this embodiment of the application, the movement cost is the sum of the current movement cost and the future movement cost, so that the movement cost includes the movement cost of moving the current text character from the previous text character and the movement cost of several future text characters. By comprehensively considering the movement cost, the accuracy of subsequent finger index selection based on the movement cost can be improved.

[0127] S30. Determine the hand movement sequence corresponding to the preset text based on the finger index and key value of the text character.

[0128] Specifically, the hand movement sequence includes finger movements for each text character in the preset text, with each finger movement including finger index and finger position. Of course, when the text characters in the preset text have operation time and key press duration requirements, the finger movements can also include operation start time and operation duration. The operation start time indicates the start time when the text character is operated on, and the operation duration indicates the duration of the text character's operation. For example, if the text character is a musical note, then the finger movements include finger index, finger position, note start time, and note duration. When the text character is a number, the finger movements can include finger index and finger position. The operation start time and operation duration can be acquired simultaneously when the preset text is obtained.

[0129] In one embodiment, such as Figure 5 As shown, determining the hand movement sequence corresponding to the preset text based on the finger index and key value of the text characters specifically includes:

[0130] Step S31: Obtain the finger position sequence of the text characters, and select the finger position of a preset finger from the finger position sequence of the text characters;

[0131] Step S32: Associate the finger index of the text character with the selected finger position to obtain the finger movement of the text character;

[0132] Step S33: Convert the preset text into a hand movement sequence based on the finger movements of the text characters.

[0133] Specifically, the preset finger is a pre-defined finger, which can be the index finger, middle finger, ring finger, or little finger, etc. The finger position of the preset finger is used to replace the finger position sequence to generate finger movements, thereby reducing the amount of data involved in the finger movements. The preset finger can be any one of all the fingers included in the robotic hand, for example, the preset finger is the index finger, etc.

[0134] Once the finger index and finger position of the text character are obtained, the finger index, finger position, and text character can be used as the finger action of the text character, and the finger actions can be sorted according to the operation order of the text character to convert the preset text into a hand movement sequence.

[0135] In one embodiment, after acquiring a hand motion sequence, a robot can be controlled to execute the hand motion sequence to manipulate preset text. Based on this, after determining the hand motion sequence corresponding to the preset text according to the finger index of the text characters, the method further includes:

[0136] Control the robotic arm to execute the first finger movement in the hand motion sequence, and calculate the movement time between the first finger movement and the second finger movement in the hand motion sequence;

[0137] Once the first finger action is completed, the robotic arm is controlled to move by the movement time and execute the second finger action;

[0138] This process continues until the last finger movement of the hand movement sequence is completed, thus completing the execution of the preset text.

[0139] Specifically, the movement time is the time required for the robotic arm to move from the first finger to the second finger. This movement time can be calculated based on the distance between the keystrokes for the first and second finger movements and the robotic arm's movement speed. For example, if the finger position for the first finger movement is 'a', the finger position for the second finger movement is 'b', and the robotic arm's movement speed is 'c', then the movement time = c.

[0140] It should be noted that when the operating device is a piano, because pianos have white and black keys, the movement time between the first and second finger movements is determined by the key type corresponding to each finger movement. If both the first and second finger movements correspond to either white or black keys, the robotic arm's movement time... If the first finger movement and the second finger movement correspond to different key types, one is a white key and the other is a black key (i.e., a black / white key switch is required), the robotic arm movement time... This is to ensure that the robotic arm can move to the next key.

[0141] Furthermore, after obtaining the movement time, the robotic arm can be controlled to move according to the movement time. After the robotic arm moves for the movement time, it will detect whether the second finger action includes the operation start time and operation duration. If it includes the operation start time, it can be based on the waiting time between the time the operation key corresponding to the second finger action is reached and the operation start time. If the waiting time is less than or equal to zero, the second finger action is executed immediately. If the waiting time is greater than zero, the second finger action is executed after waiting for the waiting time, so as to control the robot to execute the preset text.

[0142] It should be noted that when executing preset text via a robot-operated device, the positional relationship between the robot and the device can be fixed in advance. Based on this relationship, the coordinates of each operation key on the device relative to the robot can be determined. These coordinates can be represented as (x, y, z), where x represents the x-axis coordinate, y represents the y-axis coordinate, and z represents the z-axis coordinate. The y-axis coordinate is determined based on the key spacing, while the x and z axes represent the robot's x-axis and z-axis coordinates when placed on the target key. Therefore, the key coordinate acquisition process involves controlling the robot to move above the target key, acquiring its x and z-axis coordinates, and using these coordinates as the target key's x and z-axis coordinates. Then, the key information for the target key and the initial key is acquired, including the key span from the initial key to the target key. Based on the key spacing pattern and the key information, the y-axis coordinate of the target key is determined to obtain the final key coordinates. In addition, to improve the smoothness of operation, after obtaining the coordinates of the operation keys, the two robotic arms can be controlled to move to the position of each operation key one by one according to the coordinates. If a collision occurs during the movement of the robotic arms, the coordinates of the operation keys are adjusted so that the robotic arm fingers can press the designated operation keys and no collision occurs during the movement.

[0143] In summary, this embodiment provides a method, apparatus, robot, and medium for constructing a robot's hand motion sequence. The method includes assigning candidate finger indices to text characters in a preset text according to preset fingering rules and the key values ​​of text characters; for a text character corresponding to one candidate finger index, using that candidate finger index as the finger index of the text character; for a text character corresponding to at least two candidate finger indices, obtaining the movement cost of the candidate finger indices, and selecting a finger index for the text character from the at least two candidate finger indices according to the movement cost; and determining the hand motion sequence corresponding to the preset text based on the finger index and key values ​​of the text character. This application first assigns one or more candidate finger indices to the text character according to preset fingering rules, and then selects the candidate finger index with the lowest movement cost as the finger index of the text character, thereby reducing the movement time between notes. This improves the smoothness of operation of the preset text and shortens the operation time of the preset text.

[0144] Based on the above-described method for constructing robot hand motion sequences, this embodiment provides a device for constructing robot hand motion sequences, such as... Figure 6 As shown, the device for constructing the robot's hand movement sequence specifically includes:

[0145] The allocation module 100 is used to allocate candidate finger indices to text characters in a preset text according to preset fingering rules and the key values ​​of text characters, wherein the text characters carry key values;

[0146] The first selection module 200 is used to select the candidate finger index of the text character corresponding to a candidate finger index as the finger index of the text character.

[0147] The second selection module 300 is used to obtain the movement cost of each candidate finger index of a text character corresponding to at least two candidate finger indices, and select a finger index for the text character from the at least two candidate finger indices according to the movement cost;

[0148] The determining module 400 is used to determine the hand movement sequence corresponding to the preset text based on the finger index and key value of the text characters.

[0149] Based on the above-described method for constructing the robot's hand motion sequence, this embodiment provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the robot's hand motion sequence construction method as described in the above embodiment.

[0150] Based on the aforementioned method for playing the piano using a robot, this application also provides a robot, such as... Figure 7As shown, it includes at least one processor 21 and memory 22, and may also include a communications interface 23 and a bus 24. The processor 21, memory 22, and communications interface 23 can communicate with each other via the bus 24. The communications interface 23 can transmit information. The processor 21 can invoke logical instructions stored in memory 22 to execute the methods described in the above embodiments.

[0151] Furthermore, the logical instructions in the aforementioned memory 22 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0152] The memory 22, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 21 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, thereby implementing the methods in the above embodiments.

[0153] The memory 22 may include a program storage area and a data storage area. The program storage area may store the operating system and an application program required for at least one function; the data storage area may store data created based on the robot's use. Furthermore, the memory 22 may include high-speed random access memory (RAM) and non-volatile memory. Examples include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, as well as transient storage media.

[0154] Furthermore, the specific process of loading and executing the aforementioned storage medium and multiple instruction processors in the robot has been described in detail in the above method, and will not be repeated here.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for constructing a robot's hand motion sequence, characterized in that, The method for constructing the robot's hand motion sequence specifically includes: Candidate finger indices are assigned to text characters in a preset text according to preset finger placement rules, wherein the text characters carry key values; For a text character corresponding to a candidate finger index, the candidate finger index is used as the finger index of the text character; For a text character corresponding to at least two candidate finger indices, obtain the movement cost of the candidate finger indices, and select a finger index for the text character from the at least two candidate finger indices according to the movement cost; The hand movement sequence corresponding to the preset text is determined based on the finger index and key value of the text character; The preset finger placement rules include a finger placement table and a correspondence between key types and finger indices; the process of assigning candidate finger indices to text characters in a preset text according to the preset finger placement rules specifically includes: Match the key values ​​of the text characters with the finger assignment table; For a successfully matched key value, the matched finger index is used as the candidate finger index for the text character; For key values ​​that fail to match, candidate finger indices are assigned to the text characters according to the correspondence between the key type and the finger index.

2. The method for constructing the hand motion sequence of a robot according to claim 1, characterized in that, The movement cost for obtaining the candidate finger index specifically includes: Obtain the finger position sequence of the preceding text character; The candidate finger index is used as the finger index of the text character, and the finger position sequence of the text character is obtained; The current movement cost of the candidate finger index is calculated based on the finger position sequence of the previous text character and the finger position sequence of the text character. Obtain the key values ​​of several subsequent text characters of the text character, and calculate the future movement cost of the candidate finger index based on the key values ​​of the subsequent text characters and the finger position sequence of the text character; The movement cost of the candidate finger index is calculated based on the current movement cost and the future movement cost.

3. The method for constructing the hand motion sequence of a robot according to claim 1, characterized in that, The step of determining the hand movement sequence corresponding to the preset text based on the finger index and key value of the text characters specifically includes: Obtain the finger position sequence of the text characters, and select a preset finger position from the finger position sequence of the text characters; The finger index of the text character is associated with the finger position of a preset finger to obtain the finger movement of the text character; The preset text is converted into a hand movement sequence based on the finger movements of the text characters.

4. The method for constructing the hand motion sequence of a robot according to claim 2 or 3, characterized in that, The process of obtaining the finger position sequence of the text characters specifically includes: The key value of the text character is used as the finger position corresponding to the finger index of the text character; For each adjacent finger index other than the finger index of the text character, calculate the index difference between the adjacent finger index and the finger index of the text character, and determine the adjacent finger position of the adjacent finger index based on the preset finger position relationship, the index difference and the key value of the text character; A finger position sequence is constructed based on the finger position and the adjacent finger positions.

5. The method for constructing the hand motion sequence of the robot according to claim 1, characterized in that, After determining the hand movement sequence corresponding to the preset text based on the finger index of the text characters, the method further includes: Control the robotic arm to execute the first finger movement in the hand motion sequence, and calculate the movement time between the first finger movement and the second finger movement in the hand motion sequence; Once the first finger action is completed, the robotic arm is controlled to move by the movement time and execute the second finger action; This process continues until the last finger movement of the hand movement sequence is completed, thus completing the execution of the preset text.

6. The method for constructing a robot's hand motion sequence according to claim 1, characterized in that, Before assigning candidate finger indices to text characters in the preset text according to preset fingering rules, the method further includes: Obtain the key value corresponding to the text character in the preset text, and associate the key value with the text character.

7. A device for constructing a robot's hand motion sequence, characterized in that, The device for constructing the robot's hand movement sequence specifically includes: The allocation module is used to allocate candidate finger indices to text characters in a preset text according to preset fingering rules, wherein the text characters carry key values; The first selection module is used to select the candidate finger index of the text character corresponding to a candidate finger index as the finger index of the text character. The second selection module is used to obtain the movement cost of each candidate finger index of a text character corresponding to at least two candidate finger indices, and select a finger index for the text character from the at least two candidate finger indices according to the movement cost; The determination module is used to determine the hand movement sequence corresponding to the preset text based on the finger index and key value of the text characters; The preset finger placement rules include a finger placement table and a correspondence between key types and finger indices; the process of assigning candidate finger indices to text characters in a preset text according to the preset finger placement rules specifically includes: Match the key values ​​of the text characters with the finger assignment table; For a successfully matched key value, the matched finger index is used as the candidate finger index for the text character; For key values ​​that fail to match, candidate finger indices are assigned to the text characters according to the correspondence between the key type and the finger index.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the method for constructing a robot hand motion sequence as described in any one of claims 1-6.

9. A robot, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps in the method for constructing a robot hand motion sequence as described in any one of claims 1-6.

Images