Dynamic path planning method, device and equipment for robot performance and medium

Abstract

The present application relates to the technical field of robot motion planning, and discloses a dynamic path planning method, device, equipment and medium for robot performance. First, based on the key mapping information and feasibility constraint conditions of the to-be-performed music, feasibility analysis is performed to determine the feasible state corresponding to each beat, and the candidate state not meeting the actual performance condition is eliminated, thereby significantly improving the overall planning efficiency. Then, each feasible state of the previous beat and each feasible state of the next beat in adjacent beats are combined two by two, the transition cost corresponding to each combination is calculated, the state pair transition cost set corresponding to adjacent beats is obtained, and the connection relationship between adjacent beats is quantitatively expressed. On this basis, global optimization is performed based on the state pair transition cost set corresponding to adjacent beats, the total transition cost of the to-be-performed music is minimized as the target, the path is traced back, and the target performance state sequence corresponding to the to-be-performed music is obtained.

Description

Technical Field

[0001] This invention relates to the field of robot motion planning technology, and in particular to a dynamic path planning method, apparatus, device, and medium for robot performance. Background Technology

[0002] With the rapid development of service robots, performance robots and humanoid robots, robots automatically playing the piano has become a typical application scenario due to its significant commercial and demonstration value.

[0003] In related technologies, musicians or robotics engineers annotate each note of a piece of music, specifying whether it is handled by the left or right hand, which finger presses it, and the duration of the press. The robot then executes the annotations note by note. However, this approach is labor-intensive and not scalable. Each new piece of music requires annotation of the left and right hands, fingerings, and pressing duration for each note. When the robot's configuration changes, the same piece of music needs to be re-annotated. The annotation cost increases linearly with the size of the music library and the robot itself, making it difficult to support the expansion of a large-scale music library. Summary of the Invention

[0004] The embodiments described in this specification aim to at least partially solve one of the technical problems in the related art. To this end, the embodiments of this specification propose a dynamic path planning method, apparatus, device, and medium for robot performance.

[0005] This specification provides a dynamic path planning method for robot performance, the method comprising: Feasibility analysis is conducted based on the key mapping information and feasibility constraints of the piece to be played to determine the feasible state corresponding to each beat; For each feasible state in the previous cycle and each feasible state in the next cycle, pairwise combinations are made, and the transition cost corresponding to each combination is calculated to obtain the set of state pair transition costs corresponding to the adjacent cycle. Based on the states corresponding to the adjacent beats, a global optimization is performed on the set of transition costs. With the goal of minimizing the total transition cost of the entire piece of music to be played, the path is backtracked to obtain the target performance state sequence corresponding to the piece of music to be played.

[0006] In one implementation, the key mapping information includes the key number of the beat to be played, and the feasibility constraints include: The robotic arm's operation window in the feasible state covers the key number of the beat to be played; and The operation window of the robotic arm corresponding to the feasible state is within the reachable range of the robotic arm; and The left-hand operation window of the left robotic arm and the right-hand operation window of the right robotic arm corresponding to the feasible state do not overlap.

[0007] In one implementation, the step of pairwise combining each feasible state of the previous cycle with each feasible state of the next cycle in adjacent cycles, and calculating the transition cost corresponding to each combination, yields the set of state pair transition costs for the adjacent cycles, including: Based on the preset beat cycle, determine the available movement time between the previous beat and the next beat; The number of lateral movement steps and the indicator function are calculated based on the feasible state of the previous beat and the feasible state of the next beat. Based on the available movement time, the number of lateral movement steps, and the indicator function, the state pair transition cost corresponding to adjacent beats is obtained.

[0008] In one implementation, the step of globally optimizing the transition cost set based on the states corresponding to adjacent beats, with the goal of minimizing the total transition cost of the entire piece of music to be played, involves backtracking the path to obtain the target performance state sequence corresponding to the piece of music to be played, including: Iterate through each feasible state of the next cycle in adjacent cycles. For each feasible state, select the state that minimizes the cumulative transfer cost from all feasible states of the previous cycle as the predecessor state of the next cycle, and record the predecessor state and the corresponding minimum cumulative cost. At the final beat of the piece to be played, the feasible state with the minimum cumulative cost is selected as the termination state. Starting from the termination state, the target performance state sequence corresponding to the piece of music to be played is obtained by backtracking beat by beat according to the recorded preceding states.

[0009] In one embodiment, the method further includes: Based on the arm affiliation identifier of each beat in the piece of music to be played, the piece of music to be played is decomposed to obtain the first note subsequence corresponding to the first robotic arm and the second note subsequence corresponding to the second robotic arm. Based on the first note subsequence corresponding to the first robotic arm, the transfer cost is calculated and global optimization is performed to obtain the first performance state sequence corresponding to the first note subsequence. Based on the second note subsequence corresponding to the second robotic arm, the transfer cost is calculated and global optimization is performed to obtain the second performance state sequence corresponding to the second note subsequence. Conflict detection is performed based on the first performance state sequence and the second performance state sequence to obtain conflict detection results.

[0010] In one embodiment, the method further includes: In the event of a conflict in the conflict detection result, a first forbidden zone is set in the first note subsequence and a second forbidden zone is set in the second note subsequence based on the key area corresponding to the conflicting beat. The transfer cost is calculated and global optimization is performed based on the first restricted area occupancy interval to update the first performance state sequence; The transfer cost is calculated and global optimization is performed based on the second restricted area to update the second performance state sequence.

[0011] In one embodiment, the method further includes: If a conflict exists in the conflict detection result, the attribution identifiers of the first robotic arm and the second robotic arm at the conflict beat are swapped, and the first performance state sequence and the second performance state sequence are updated according to the swapped attribution identifiers.

[0012] This specification provides a dynamic path planning device for robot performance, the device comprising: The feasibility state analysis module is used to perform feasibility analysis based on the key mapping information and feasibility constraints of the music to be played, and to determine the feasibility state corresponding to each beat. The transition cost calculation module is used to combine each feasible state of the previous cycle with each feasible state of the next cycle in adjacent cycles, calculate the transition cost corresponding to each combination, and obtain the state pair transition cost set corresponding to the adjacent cycles. The performance path determination module is used to perform global optimization of the transition cost set based on the states corresponding to the adjacent beats, with the goal of minimizing the total transition cost of the entire piece of music to be performed, and backtracks the path to obtain the target performance state sequence corresponding to the piece of music to be performed.

[0013] This specification provides a computer device comprising: a memory, and one or more processors communicatively connected to the memory; the memory stores instructions executable by the one or more processors, the instructions being executed by the one or more processors to cause the one or more processors to perform the steps of the method described in any of the above embodiments.

[0014] This specification provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0015] This specification provides a computer program product that includes instructions that, when executed by a processor of a computer device, enable the computer device to perform the steps of the method described in any of the above embodiments.

[0016] In the above-described implementation method, firstly, a feasibility analysis is performed based on the key mapping information and feasibility constraints of the piece to be played. Feasible states for each beat are determined, and candidate states that do not meet actual performance conditions are eliminated. The set of states with operational possibilities in each beat is retained, thus avoiding subsequent searches on a large number of invalid states and significantly improving overall planning efficiency. Next, each feasible state of the preceding beat is paired with each feasible state of the following beat in adjacent beats, and the transition cost corresponding to each pair is calculated. This yields a set of state-to-state transition costs for adjacent beats, quantifying the connection between adjacent beats and providing precise local cost basis for overall piece optimization. Based on this, global optimization is performed on the set of state-to-state transition costs for adjacent beats, aiming to minimize the total transition cost throughout the piece. The path is backtracked to obtain the target performance state sequence for the piece. This mechanism selects the globally optimal solution from all feasible trajectories, thus avoiding the constraint of fingering choices in the preceding section on subsequent performance or causing significant hand position jumps. As a result, the transitions between states in the entire piece are smoother and more natural, effectively reducing hand fatigue and technical burden, while improving the stability, accuracy, and coherence of continuous performance. This solution eliminates unfavorable phenomena such as back-and-forth jumping and excessive empty-walking distances at the mechanism level, ensuring the economy and fluency of the overall performance state sequence. Attached Figure Description

[0017] Figure 1 A flowchart illustrating the dynamic path planning method for robot performance provided in the embodiments of this specification; Figure 2 A flowchart illustrating the process of obtaining the state pair transition cost corresponding to adjacent beats, provided for the implementation of this specification; Figure 3a A flowchart illustrating the process of obtaining the target performance state sequence corresponding to the piece of music to be played, provided for the implementation of this specification; Figure 3b A partial schematic diagram of path planning provided for the implementation of this specification; Figure 4 A schematic diagram of the process for obtaining collision detection results provided for the implementation of this specification; Figure 5 A flowchart illustrating the update of the performance state sequence provided for the embodiments of this specification; Figure 6 A flowchart for resolving four-hand playing conflicts provided in the embodiments of this specification; Figure 7 A schematic diagram of a dynamic path planning device for robot performance provided in the embodiments of this specification; Figure 8 An internal structural diagram of a computer device provided for embodiments of this specification. Detailed Implementation

[0018] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0019] With the rapid development of service robots, performance robots, and humanoid robots, robotic piano playing has become a typical application scenario due to its significant commercial and demonstration value. In this scenario, the task of robotic piano playing requires, given a piece of music (given as a sequence of notes), pre-determining which robotic arm and which finger will press each note, when the arm will move, how far it will move, and how the finger will lift and press down before reaching the target key. In human performance, this decision-making process is done by the performer based on experience. However, in a robotic system, an automatic orchestration algorithm must generate an executable sequence of instructions at once. This sequence must simultaneously meet three requirements: performance feasibility (the finger can reach the target key, and there is no physical collision when switching between black and white keys), global optimality (minimizing the total displacement, idle time, and number of accelerations and decelerations throughout the piece), and format compatibility with the downstream robot instruction compiler.

[0020] Among related technologies, there is a manual note-by-note pre-arrangement scheme. A typical approach involves musicians or robotic engineers marking each note in a piece of music with information such as whether it's played by the left or right hand, which finger (index, middle, ring, or little finger) is used, and the duration of the press. The robot then executes the arrangement note by note according to the manual annotations, for example, by manually dragging and dropping fingerings into a graphical music editing tool. However, this scheme is labor-intensive and not scalable. Each new piece requires marking each note with the left / right hand, fingering, and press duration. When the robot's configuration changes (e.g., from two hands to four, or changes in arm length leading to changes in reach), the same piece must be re-marked. The annotation cost increases linearly with the size of the music library and the robot itself, making it difficult to support the expansion of a large-scale music library.

[0021] In related technologies, the note-by-note local decision-making scheme based on greedy search is characterized by the algorithm making independent local decisions for each note in the music. For example, it selects the hand that is currently closest to the note key, selects the finger with the smallest distance from the target key among the reachable fingers, and triggers the arm lateral movement only when the reachable window of the hand does not cover the target key. The decision is made forward according to the original time sequence of the note sequence without backtracking. A typical example is a greedy algorithm that scans the note sequence from left to right using heuristic rules and selects the hand and finger for each note. However, this solution suffers from local optima traps and excessive global displacement costs. The greedy strategy only makes decisions based on the local distance between the current hand position and the note, without considering the reverse constraints of subsequent notes on the current decision. This often results in the current note being greedily pressed by one hand, causing several subsequent notes to be outside the reachable window of the other hand and forcing a large horizontal movement. If the hand slightly farther away is selected on the current note, all subsequent notes can fall within the same window without horizontal movement. This leads to problems such as the robotic arm "jumping back and forth", excessive empty walking distance, and too many acceleration and deceleration times. Furthermore, it degrades significantly in four-hand mode, often resulting in two pairs of arms competing for adjacent keys at the same time and obstructing each other.

[0022] In related technologies, there is also a unified processing scheme that does not distinguish between black and white key planes. This scheme models all piano keys as discrete coordinate points on the same plane according to their physical horizontal positions. It does not establish a separate reachability model for black keys, but uses the reachability and pressing action model of the white keys. The physical offset of black keys along the length of the key (y-direction) and perpendicular to the keyboard surface (z-direction) is not explicitly introduced into the arrangement algorithm. This is common in the arrangement of pieces with dense white keys. Because this scheme lacks modeling of the physical offset of the black key plane, it leads to problems such as black keys going out of bounds and colliding with other keys. For example, a black key might be offset about half a key depth in the y-direction relative to the white keys, and higher than the white keys in the z-direction. When a robotic hand presses a black key, the wrist tilt angle and finger extension differ from when pressing a white key. The reachability window width of the black key plane is generally smaller than that of the white key plane. In unified processing, errors such as the algorithm determining that a key is reachable but it is physically unreachable, the finger landing point crossing adjacent black keys, and the hand colliding with the edge of a white key occur in musical passages with many black keys.

[0023] In related technologies, for four-hand performance scenarios, a global search scheme in a four-dimensional state space is directly extended. This involves modeling the left and right hand positions of the two pairs of robotic arms as quadruples, using the condition that all four hand positions can simultaneously reach all notes on the current beat, and at the cost of the sum of the movement distances of the four hand positions, to perform a global search in the four-dimensional state space. However, the state space of this scheme increases exponentially with the number of arms (from O(R²) for one or two hands to O(R²) for four hands). 4 (R is the sum of the distribution span of a single-beat note on the one-dimensional key coordinate axis and the number of digits that the arm can reach). When the number of beats in a piece of music is large, the calculation cost is too high and it is difficult to apply in practice.

[0024] In related technologies, there is also a scheme that directly outputs the arrangement result as an intermediate expression from "note to finger" without directly connecting to the robot instruction compiler. Its output format is a list of key-value pairs from "note number to (hand, finger, press duration)," excluding the arm's offset field on the one-dimensional key coordinate axis. The downstream process then converts the hand and finger into robot joint-level instructions. Because this scheme lacks the arm position offset for each beat, it requires extra calculation of the actual arm movement from "hand plus finger" and "press position in the preceding and following beats." This can easily introduce ambiguity in scenarios where the arm moves without pressing down on multiple consecutive beats, and it lengthens the engineering chain from "musical score to robot executable instructions," increasing the risk of errors in the format conversion stage.

[0025] Based on the above analysis, this specification provides a dynamic path planning method for robot performance. First, a feasibility analysis is performed based on the key mapping information and feasibility constraints of the piece to be played. Feasible states for each beat are determined, and candidate states that do not meet actual performance conditions are eliminated. The set of states with operational possibilities in each beat is retained, thus avoiding subsequent searches on a large number of invalid states and significantly improving overall planning efficiency. Next, each feasible state of the preceding beat is paired with each feasible state of the following beat in adjacent beats, and the transition cost corresponding to each pair is calculated. This yields a set of state-to-state transition costs for adjacent beats, quantifying the connection between adjacent beats and providing precise local cost basis for overall piece optimization. Based on this, global optimization is performed on the set of state-to-state transition costs for adjacent beats, aiming to minimize the total transition cost throughout the piece. The path is backtracked to obtain the target performance state sequence for the piece. This mechanism selects the globally optimal solution from all feasible trajectories, thus avoiding the constraint of fingering choices in the preceding section on subsequent performance or causing significant hand position jumps. As a result, the transitions between states in the entire piece are smoother and more natural, effectively reducing hand fatigue and technical burden, while improving the stability, accuracy, and coherence of continuous performance. This solution eliminates unfavorable phenomena such as back-and-forth jumping and excessive empty-walking distances at the mechanism level, ensuring the economy and fluency of the overall performance state sequence.

[0026] This specification provides a dynamic path planning method for robot performance. Please refer to [link / reference]. Figure 1 The dynamic path planning method for the robot's performance may include the following steps: S110. Based on the key mapping information and feasibility constraints of the piece to be played, perform a feasibility analysis to determine the feasible state corresponding to each beat.

[0027] In some cases, the white keys on a piano can be numbered sequentially as integers according to their physical arrangement from lowest to highest note. Key mapping information can include the beat number of the piece to be played, the piano key number, and an indication that the key is black or white. For example, (beat_index) can be used. i key_order i indicator i ) indicates that, among which, beat_index i The key_order is the beat number. i The indicator represents the piano key positions. i It serves as an indicator for either the black or white keys.

[0028] It should be noted that black and white keys are inherently highly different in physical structure, which is reflected in both playing feel and acoustic characteristics. However, the mapping scheme adopted in this implementation distinguishes key colors by adding one-dimensional state information to the original key position number, without increasing the structural complexity of the dynamic programming solver. As a result, the same solver can seamlessly adapt to three different performance scenarios: pure white key music, pure black key music, and mixed music with alternating black and white keys, without the need to design separate solution logic for each scenario.

[0029] Specifically, based on the beat number, piano key number, and black or white key indicator carried in the key mapping information, and combined with the feasibility constraints that the performance executor needs to satisfy during the performance, such as the number of fingers, the range of movement of each finger, the accessibility of fingers moving from the current key to the target key between adjacent beats, and the mutual exclusion constraint of the allocation relationship between different fingers and different keys within the same beat, it is determined beat by beat whether there is at least one allocation scheme of fingers and piano keys within that beat, such that all piano keys that need to produce sound within that beat are effectively pressed and the movement of each finger does not violate the above feasibility constraints. Each finger and piano key allocation scheme that meets the conditions is determined as a feasible state corresponding to that beat, thereby obtaining the set of feasible states corresponding to each beat, so as to determine the overall performance scheme based on the state transition relationship between adjacent beats.

[0030] For example, for the i-th beat of the piece of music to be played, its feasible state can be represented by a triple state[i] = (xl (i) xr (i) p (i) It is represented by ) . Where, xl (i) The key_order indicates the key number of the leftmost white key in the left-hand operation window corresponding to the left robotic arm; that is, the smallest white key number among the four consecutive white keys covered by the four fingers of the left robotic arm.(i) This refers to the key order of the leftmost white key in the right-hand operation window of the right robotic arm under the same robotic arm. In other words, it's the smallest white key number among the four consecutive white keys covered by the four fingers of the right robotic arm. (i) This indicates the key plane identifier where the robotic arm's finger is located in the i-th beat. The value is 0 or 1, where 0 indicates that the finger is in the white key plane and 1 indicates that the finger is in the black key plane.

[0031] With the robotic arm configured with four fingers, for the i-th beat, the white key sequence range covered by the operation window of the left robotic arm corresponding to that state is [xl (i) ,xl (i) +3], the white key sequence range covered by the operation window of the right robotic arm is [xr (i) xr (i) +3] means that the left and right mechanical hands each cover four consecutive keys with four fingers.

[0032] S120. For each feasible state in the previous cycle and each feasible state in the next cycle, perform pairwise combinations, calculate the transition cost corresponding to each combination, and obtain the set of state pair transition costs corresponding to adjacent cycles.

[0033] Specifically, for the preceding and following beats in adjacent beats, each feasible state in the feasible state set corresponding to the preceding beat can be paired with each feasible state in the feasible state set corresponding to the following beat to form multiple state pairs consisting of the states of the preceding and following beats. For each state pair, based on the leftmost white key number of the left robotic arm finger window, the leftmost white key number of the right robotic arm finger window, and the indicator of the plane where the fingers are located, as well as the corresponding information contained in the following beat state, the translation cost of the finger windows of the left and right robotic arms due to position changes between adjacent beats, the plane switching cost of the fingers switching between the black and white key planes, and the motion cost of each finger moving from the currently covered key to the target covered key are calculated. These cost components are then weighted and summed to obtain the transition cost corresponding to the state pair. By summarizing all state pairs and their corresponding transition costs, we can obtain the set of state pair transition costs corresponding to the adjacent beat. This set can then be used to search for state transition paths between adjacent beats and optimize the overall performance scheme.

[0034] S130. Based on the states corresponding to adjacent beats, perform global optimization on the set of transition costs. With the goal of minimizing the total transition cost of the entire piece of music to be played, backtrack the path to obtain the target performance state sequence corresponding to the piece of music to be played.

[0035] Specifically, the entire score of the piece to be played is divided into multiple adjacent consecutive stages. The feasible state corresponding to each stage is used as the candidate state node for that stage. The state pairs between adjacent stages and their corresponding transition costs are used as directed edges and their weights connecting the state nodes of the previous and next stages, thus constructing a weighted directed acyclic state transition graph from the first stage to the last stage. Based on this state transition graph, with the optimization objective of minimizing the sum of the transition costs of all adjacent state pairs along the entire path from the first stage to the last stage, dynamic programming is used to calculate the minimum cumulative transition cost to reach each feasible state, starting from the first stage and proceeding stage by stage. Simultaneously, the state of the previous stage corresponding to this minimum cumulative transition cost is recorded as the predecessor state of that state. After calculating the minimum cumulative transition cost for all feasible states in the last beat, starting from the feasible state with the minimum cumulative transition cost in the last beat, the system backtracks backward beat by beat based on the recorded predecessor states, and determines the state corresponding to each beat in turn. Finally, a target performance state sequence consisting of one feasible state for each beat is obtained. This target performance state sequence is the performance state planning result that minimizes the total transition cost of the entire piece of music under the condition of satisfying the feasibility constraints.

[0036] In the above implementation, firstly, a feasibility analysis is performed based on the key mapping information and feasibility constraints of the piece to be played. Feasible states for each beat are determined, and candidate states that do not meet actual performance conditions are eliminated. The set of states with operational possibilities in each beat is retained, thus avoiding subsequent searches on a large number of invalid states and significantly improving overall planning efficiency. Next, each feasible state of the preceding beat is paired with each feasible state of the following beat in adjacent beats, and the transition cost corresponding to each pair is calculated. This yields a set of state-to-state transition costs for adjacent beats, quantifying the connection between adjacent beats and providing precise local cost basis for overall piece optimization. Based on this, global optimization is performed on the set of state-to-state transition costs for adjacent beats, aiming to minimize the total transition cost throughout the piece. The path is backtracked to obtain the target performance state sequence for the piece. This mechanism selects the globally optimal solution from all feasible trajectories, thus avoiding constraints or significant hand position jumps caused by the fingering selection in the preceding section. As a result, the transitions between states in the entire piece are smoother and more natural, effectively reducing hand fatigue and technical burden, while improving the stability, accuracy, and coherence of continuous performance. This solution eliminates unfavorable phenomena such as back-and-forth jumping and excessive empty-walking distances at the mechanism level, ensuring the economy and fluency of the overall performance state sequence.

[0037] In some implementations, the key mapping information includes the key number of the beat to be played, and the feasibility constraints include: The robotic arm's operation window in the feasible state covers the key number of the beat to be played; and The feasible state corresponds to an operation window for the robotic arm that is within the reachable range of the robotic arm; and The left-hand operation window of the left robot and the right-hand operation window of the right robot do not overlap in the feasible state.

[0038] Specifically, the key mapping information includes the key number of the beat to be played, which indicates the specific key position to be played in the current beat. When determining the feasibility of a candidate state, constraint checks need to be performed on the operation windows corresponding to the left and right robotic arms separately. For either robotic arm, the key range corresponding to its operation window should be able to cover the key number of the beat to be played. In other words, the key represented by the key number should fall within the key range of the operation window, and cannot be outside the coverage area of ​​the operation window. In other words, the key number of the beat to be played only needs to fall within the operation window of one robotic arm; it is not required that the key number fall within both the left and right robotic arm operation windows simultaneously. Simultaneously, the operating window of either robotic arm must be within its own reachable range. Specifically, the left-hand operating window of the left robotic arm should fall within its reachable range, and the right-hand operating window of the right robotic arm should fall within its reachable range. This ensures that the robotic arms do not exceed their range of motion when performing playing actions within their operating windows. Furthermore, there should be no overlapping areas between the left-hand operating window of the left robotic arm and the right-hand operating window of the right robotic arm. In other words, the key positions controlled by the left and right hands should avoid overlap in the key layout space to prevent interference or collisions. All of the above feasibility constraints must be met simultaneously for the candidate state to be determined as a feasible state.

[0039] For example, the note to be pressed on the i-th beat (beat_index) i key_order i indicator i When determining whether the state corresponding to a given timeframe is feasible, the following constraint must be satisfied simultaneously: key_order i ∈[xl (i) ,xl (i) +3]∪[xr (i) xr (i) +3], while xl (i) It should fall within the reachable range corresponding to the left robotic arm, xr (i) It should fall within the reachable range corresponding to the right robotic arm, and xr (i) ≥xl (i) +W win Among them, W win The number of fingers for the robotic arm.

[0040] In the above embodiments, the feasibility constraints ensure that the robot can perform in a safe, accurate, efficient and smooth manner.

[0041] In some implementations, please refer to Figure 2 For each feasible state in the preceding cycle and each feasible state in the following cycle, pairwise combinations are performed, and the transition cost corresponding to each combination is calculated to obtain the set of state pair transition costs for adjacent cycles. This may include the following steps: S210. Based on the preset beat cycle, determine the available movement time between the previous beat and the next beat.

[0042] S220. Calculate the number of lateral movement steps and the indicator function based on the feasible state of the previous cycle and the feasible state of the next cycle.

[0043] S230. Based on the available movement time, lateral movement steps, and indicator function, obtain the state pair transition cost corresponding to adjacent beats.

[0044] Specifically, firstly, the time interval between the previous and next beats is determined according to a preset beat cycle, and this time interval is used as the available movement time when switching from the state corresponding to the previous beat to the state corresponding to the next beat. Then, the feasible state sets corresponding to the previous beat and the next beat are obtained, and the feasible states in the two sets are combined pairwise for calculation. For each state pair, based on the key number corresponding to the leftmost white key in the operation window recorded in the state, the number of keys that need to be traversed to move from the position of the previous state to the position of the next state is calculated, and this number of traversals is used as the horizontal movement step. Simultaneously, the color attributes of the keys corresponding to the previous beat and the keys corresponding to the next beat are compared. If both are white keys or both are black keys, the value of the indicator function is set to 0; if one is a white key and the other is a black key, the value of the indicator function is set to 1, thus obtaining the indicator function value used to characterize whether the key colors of adjacent beats are consistent. Based on this, the available movement time, the number of lateral movement steps, and the aforementioned indicator function value are used as inputs to comprehensively evaluate the transition cost between state pairs corresponding to adjacent beats according to a preset cost calculation method.

[0045] For example, when transitioning from a feasible state state[i-1] in the previous cycle to a feasible state state[i] in the next cycle, the single-step transition cost is measured by a cost function composed of the squared term of the lateral shift normalized by the cycle duration and the plane switching penalty term. The specific expression is as follows:

[0046] in, , , representing the number of horizontal steps the left and right robotic arms move along the one-dimensional white key coordinate axis, respectively, in "white keys". The actual physical distance needs to be multiplied by the key width key_width for conversion. It represents the available movement time between the previous and next beats in an adjacent beat, expressed in the smallest unit of beat, and its value is equal to the duration of the previous pressed beat in the musical score structure.

[0047] The quadratic squared term in the cost function is physically equivalent to the sum of squares of the lateral velocity normalized by time. Compared to the linear distance cost, this form imposes a stronger penalty on abrupt, large, and rapid lateral movements, thus causing the global optimal solution to tend to distribute the required lateral movement evenly across multiple frames, avoiding excessively large instantaneous movements within a single frame.

[0048] Indicator function item Weights are applied to the black-and-white key plane switching that occurs between two consecutive beats. Additional penalties, among which The indicator function takes a value of 1 when the planes (i.e. black and white keys) of the previous and next beats in adjacent beats are different, and 0 otherwise. Introducing this penalty term can effectively suppress the ill-conditioned trajectory of "switching planes on every beat" that may occur in the dynamic programming solution process in passages containing alternating black and white keys.

[0049] In a practical implementation, the plane switching penalty weight It can be set to 0, meaning that the cost of plane switching is entirely reflected by the actual lateral movement distance, or it can be set to a small positive value to appropriately suppress frequent switching according to specific performance needs.

[0050] The above single-step transfer cost model is based on Bellman optimal substructure. The recursive process of the main loop is therefore guaranteed to be correct. That is, the optimal solution for the entire piece of music must be composed of the optimal solutions of each sub-segment. Therefore, the global optimal trajectory can be constructed from bottom to top according to the rhythm progression, so as to obtain a robotic arm motion planning scheme that takes into account both the smoothness of lateral movement and the rationality of planar switching within a feasible time complexity.

[0051] In the above implementation, based on a preset beat cycle, the available movement time between the previous beat and the next beat is determined. Based on the feasible states of the previous beat and the feasible states of the next beat, the number of lateral movement steps and the indicator function are calculated. Based on the available movement time, the number of lateral movement steps and the indicator function, the state pair transition cost corresponding to adjacent beats is obtained. This can provide a basis for selecting the performance or control path with the minimum total cost from multiple candidate beat state sequences.

[0052] In some implementations, please refer to Figure 3aThe process involves globally optimizing the set of transition costs based on the states corresponding to adjacent beats, with the goal of minimizing the total transition cost of the entire piece of music to be played. This process backtracks the path to obtain the target performance state sequence corresponding to the piece of music to be played, and may include the following steps: S310. Traverse each feasible state of the next feasible state in the adjacent beats. For each feasible state, select the state that minimizes the cumulative transition cost from all feasible states of the previous beat as the predecessor state of the next beat, and record the predecessor state and the corresponding minimum cumulative cost.

[0053] S320. At the end beat of the piece to be played, select the feasible state with the minimum cumulative cost as the termination state.

[0054] S330. Starting from the termination state, backtrack beat by beat according to the recorded preceding state to obtain the target performance state sequence corresponding to the piece to be played.

[0055] In some cases, the robotic arm is pre-set with initial key positions. These initial key positions are determined by adjusting the robotic arm to a neutral, stable, and far-from-joint-limits reset position, and using this reset position as a reference, recording the position of the leftmost white key on the keyboard for each finger window of the robotic arm in this position. Here, a finger window can be understood as the corresponding controllable continuous key area for each finger on the keyboard, with the position of the leftmost white key in this area serving as the positioning reference for that finger window. The recorded initial key positions serve two purposes: firstly, to indicate the robotic arm's ready state before the performance begins; and secondly, to serve as a reference for calculating the initial movement amount for the first beat, ensuring that the state search and cost accumulation for subsequent beats all start from a unified starting position.

[0056] After determining the initial key positions, dynamic programming is performed beat by beat, starting from these initial key positions and proceeding sequentially along the rhythm of the music to be played. Specifically, with the initial key positions determined, state search and recursive calculation are performed for each beat of the music, following the beat order. For the second beat in two adjacent beats, all playable feasible states under that second beat are traversed. For each feasible state, the preceding beat state that minimizes the cumulative transfer cost required to reach the feasible state of the second beat from the initial key positions is selected from all feasible states of the preceding beat. This selected preceding beat state is recorded as the predecessor state of the current second beat state, and the value of the minimum cumulative transfer cost is saved. This recursive processing is performed on each pair of adjacent beats in this manner until the final beat of the music to be played is reached. At the final beat, the cumulative transfer costs corresponding to all feasible playable states under that beat are compared, and the feasible playable state with the minimum cumulative transfer cost is selected as the final state. Subsequently, starting from the terminating state, and following the preceding relationships between the recorded states, the process is repeated beat by beat in the opposite direction of the beat sequence to determine the performance states corresponding to each beat before the terminating beat, ultimately obtaining the target performance state sequence for the piece to be performed. For example, please refer to [link to example]. Figure 3b , Figure 3b A partial schematic diagram of the path planning is shown. Figure 3b Not all possible states and routes are shown in the diagram; this illustration is for illustrative purposes only and is not intended to be limiting.

[0057] In some implementations, a two-dimensional dynamic programming table dp[i][s] is first constructed. i ], where index i represents the beat number, ranging from 0 to the total number of beats N, and s represents a feasible state under that beat, specifically represented as a triplet (xl) of the positions of the left and right robotic arms and the playing hand shape. (i) xr (i) p (i) In the initialization phase, let dp[0][s0] equal to 0, where s0 is the initial state of the robotic arm at the initial key position. Then, proceed to the forward recursion phase, advancing sequentially according to the cycle index i from 1 to N. For the i-th cycle, enumerate all feasible states that satisfy the preset feasibility constraints for that cycle, and for each feasible state i, further enumerate all feasible states of the previous cycle (i.e., the (i-1)-th cycle), based on the state transition cost. According to dp[i][s i ]=mins_i-1(dp[i-1][s i-1 ]+cost(s i-1 →s iThe recursive formula is used to calculate the minimum cumulative transition cost from the initial state to the current cycle. Simultaneously, the feasible state at the (i-1)th cycle that achieves this minimum value is recorded as the predecessor state and stored in `prev[i][s]`. i This is stored in [ ] for later backtracking. It is important to note that during the forward recursion, all feasible states in the (i-1)th cycle remain valid and independently alive. Each state retains the minimum cumulative cost dp[i-1][·] from the starting point to that state, and multiple states are not merged at the (i-1)th cycle. Therefore, when advancing to the i-th cycle and calculating a certain state s... i When calculating the cumulative cost, it is necessary to traverse all feasible states s from the previous iteration. i-1 Select the minimum value and record the corresponding predecessor relationship.

[0058] Regarding the global optimization objective, this method uses minimizing the total transfer cost of the entire piece of music as the optimality criterion, that is, minimizing the sum of the transfer costs between all adjacent beats from beat 1 to beat N. The initial state state[0] of beat 0 is fixed as the initial key positions of the left and right robotic arms, while the final state state[N] is the feasible state with the minimum cumulative cost among all feasible states of beat N. After completing all forward recursion, at the last beat i=N, the minimum value in dp[N][·] is found, and its corresponding state is recorded as sN. Next, starting from the terminated state, based on the stored predecessor relationship prev[·][·], backtrack beat by beat in the direction of decreasing beat index, and restore the optimal performance state of each beat before the terminated beat in turn, thus obtaining the complete optimal state sequence.

[0059] To illustrate the above recursion and backtracking process more clearly, an example is provided: Let the initial state of the 0th step be S0, the feasible states of the 1st step be S11 and S12, the feasible states of the 2nd step be S21 and S22, and the feasible states of the 3rd step be S31 and S32. If the transfer costs are cost(S0, S11)=1, cost(S0, S12)=10, cost(S11, S21)=100, cost(S12, S21)=101, cost(S11, S22)=10, cost(S12, S22)=11, cost(S21, S31)=100, cost(S22, S31)=101, cost(S21, S32)=10, cost(S22, S32)=3, then by forward recursion we can get dp[2][S21]=101, dp[2][S22]=11, and further dp[3][S31]=112, dp[3][S32]=14. After comparing the last beat, S32 is selected as the termination state, and backtracking is performed along the predecessor relationship. The final optimal path is S0→S11→S22→S32.

[0060] Regarding computational complexity and pruning strategy, when enumerating feasible states for the i-th beat, only states satisfying the reachability constraint of the notes in that beat are enumerated, and the key position range of the notes in that beat is limited by [min key_orderi, max key_orderi], thus determining the effective candidate intervals for xl and xr. Since the span of note key position distribution within a single beat in actual music is usually finite, the number of candidate states is much smaller than the total number of key position combinations. |² is equivalent to 52². The overall time complexity is O(N · R² · R²) = O(N · R²). 4 ), where R represents the one-dimensional span of the note key distribution in this beat, and its typical value is within 15 after the look-ahead window is expanded. In a preferred embodiment, the look-ahead window is 40 beats and the look-back window is 5 beats, so the solution time for the entire piece of music is within a controllable range.

[0061] In the above implementation, each feasible state of the next beat in adjacent beats is traversed. For each feasible state, the state that minimizes the cumulative transition cost is selected from all feasible states of the previous beat as the preceding state for the next beat, and the preceding state and its corresponding minimum cumulative cost are recorded. At the final beat of the piece to be played, the feasible state with the minimum cumulative cost is selected as the final state. Starting from the final state, the process is reversed beat by beat based on the recorded preceding state to obtain the target performance state sequence for the piece to be played. This mechanism selects the globally optimal solution from all feasible trajectories, thereby avoiding the constraint of fingering selection in the preceding section on subsequent performance or causing large hand position jumps. As a result, the state transitions of the entire piece are smoother and more natural, effectively reducing hand fatigue and technical burden, while improving the stability, accuracy, and coherence of continuous performance and musical expression. This scheme eliminates adverse phenomena such as back-and-forth jumping and excessive empty walking distance at the mechanism level, ensuring the economy and smoothness of the overall performance state sequence.

[0062] In some implementations, please refer to Figure 4 The method may also include the following steps: S410. Based on the arm affiliation identifier of each beat in the piece of music to be played, decompose the piece of music to be played to obtain the first note subsequence corresponding to the first robotic arm and the second note subsequence corresponding to the second robotic arm.

[0063] Specifically, piano playing can be a collaborative effort between two people. Therefore, each beat or note in the piece can be pre-labeled with an arm affixation marker to distinguish whether the corresponding beat or note is played by the first or second robotic arm. It should be noted that both the first and second robotic arms each contain a left and a right robotic hand, meaning each arm has the ability to operate independently with both hands. When decomposing the music, the original note sequence can be traversed according to the beat order of the piece, reading the arm affixation marker for each note. If the arm affixation marker for the current note points to the first robotic arm, the note, along with its beat index in the original music, is extracted into the first note subsequence corresponding to the first robotic arm. If the arm affixation marker for the current note points to the second robotic arm, the note and its original beat index are extracted into the second note subsequence corresponding to the second robotic arm. For example, the notes belonging to the arm of performer A are extracted as the subsequence "notes". A The notes belonging to the arm of performer B are extracted into a subsequence of notes. B During the decomposition process, the original beat indices are preserved, ensuring that both the first and second note subsequences use the beat positions of the original music as their time references. For the same beat in the original music, if the first and second robotic arms each have a corresponding note, the two subsequences record their respective notes at that beat index. If a robotic arm does not have a corresponding note at a beat, the corresponding subsequence can be recorded as an empty note or have its position reserved at that beat index without deleting the beat index. This ensures that the two subsequences maintain consistency with the original music in terms of beat timing, facilitating subsequent alignment and synthesis based on the beat indices.

[0064] S420. Based on the first note subsequence corresponding to the first robotic arm, perform transfer cost calculation and global optimization to obtain the first performance state sequence corresponding to the first note subsequence.

[0065] Specifically, after determining the first note subsequence corresponding to the first robotic arm, a feasibility analysis is performed based on the key mapping information and feasibility constraints of the first note subsequence to determine the feasible states corresponding to each beat. The transition cost is calculated for the changes in feasible states between adjacent beats in the first note subsequence, and global optimization is performed using the total transition cost of the candidate performance state paths corresponding to the entire first note subsequence as the optimization objective. From the feasible candidate performance states for each beat, the state combination that optimizes the cumulative transition cost is selected to form the first performance state sequence.

[0066] S430. Based on the second note subsequence corresponding to the second robotic arm, perform transfer cost calculation and global optimization to obtain the second performance state sequence corresponding to the second note subsequence.

[0067] Specifically, after determining the second note subsequence corresponding to the second robotic arm, a feasibility analysis is performed based on the key mapping information and feasibility constraints of the second note subsequence to determine the feasible state corresponding to each beat. For the second note subsequence corresponding to the second robotic arm, the same solver as the first robotic arm is called, and the same transition cost calculation method and global optimization strategy are used to process it, thereby obtaining the second performance state sequence. This ensures that the state sequence generation process of the two robotic arms maintains a consistent solution standard and optimization scale.

[0068] S440. Detect conflict based on the first performance state sequence and the second performance state sequence to obtain the conflict detection result.

[0069] Specifically, after obtaining the first performance state sequence and the second performance state sequence, the first performance state sequence and the second performance state sequence are aligned according to the beat. For each identical beat, the performance state corresponding to the first robotic arm and the performance state corresponding to the second robotic arm are obtained, and their spatial positions are compared to determine whether there is spatial overlap. If spatial overlap occurs, it is determined that there is a conflict for that beat; otherwise, it is determined that there is no conflict for that beat. In this way, conflict detection results are obtained beat by beat.

[0070] For example, for the same beat, the first robotic arm sol is obtained from the first performance state sequence and the second performance state sequence respectively. A Second robotic arm sol B The finger windows are combined at this beat. Among them, the first robotic arm sol A The union of the finger windows is defined by the operating range [xl1] corresponding to its left robotic arm. (i) ,xl1 (i) +3] The operating range corresponding to the right robotic arm [xr1] (i) xr1 (i) +3] Finding the union of the sets yields the second robotic arm sol. B The union of the finger windows is also composed of the operating range [xl2] corresponding to its left robotic arm. (i) ,xl2 (i) +3] The operating range corresponding to the right robotic arm [xr2] (i) xr2 (i)+3] The union of the two finger windows is obtained. The union of the two finger windows is mapped onto the same one-dimensional white key coordinate axis for comparison to determine whether there is an intersection. If the intersection of the two finger window unions on the coordinate axis is empty, it means that the finger coverage areas of the first and second robotic arms do not overlap in this beat, and there is no mutual interference caused by the superposition of finger coverage areas, so it is not judged as a conflict. If the intersection of the two finger window unions on the coordinate axis is not empty, the set of keys that the first and second robotic arms actually need to press in this beat is further obtained, and it is determined whether the key corresponding to the intersection falls within the set of keys that either robotic arm needs to press in this beat. If the intersection falls within the set of keys that either robotic arm needs to press in this beat, it means that the finger windows of the two robotic arms not only overlap, but the overlapping area contains at least one key that actually needs to be triggered, and it is judged as a conflict. If the intersection does not fall within the set of keys that either robotic arm needs to press in this beat, it means that the window overlap only occurs in the key area that does not need to be pressed, and will not cause interference between the actual playing actions, so it is not judged as a conflict.

[0071] In the above implementation, based on the arm affiliation identifiers for each beat in the music to be played, the music is decomposed to obtain the first note subsequence corresponding to the first robotic arm and the second note subsequence corresponding to the second robotic arm. Based on the first note subsequence corresponding to the first robotic arm, transition costs are calculated and global optimization is performed to obtain the first performance state sequence corresponding to the first note subsequence. Similarly, based on the second note subsequence corresponding to the second robotic arm, transition costs are calculated and global optimization is performed to obtain the second performance state sequence corresponding to the second note subsequence. Conflict detection is then performed based on the first and second performance state sequences to obtain the conflict detection results, forming a closed-loop processing mechanism of "decomposition—independent optimization—conflict verification." This method effectively avoids mutual interference between the two arms in the spatiotemporal domain while ensuring efficient, continuous, and smooth single-arm movements, significantly improving the reliability, safety, and execution efficiency of collaborative performance, while reducing the complexity of manual teaching and parameter adjustment.

[0072] In some implementations, please refer to Figure 5 The method may also include the following steps: S510. In the event of a conflict in the conflict detection result, a first restricted area is set in the first note subsequence and a second restricted area is set in the second note subsequence based on the key area corresponding to the conflicting beat.

[0073] Specifically, when conflict detection indicates a conflict at a certain beat, a first operation window for the first robotic arm and a second operation window for the second robotic arm can be determined for that conflicting beat. It should be noted that the first operation window is the union of the left and right robotic arm operation windows under the first robotic arm, and the second operation window is the union of the left and right robotic arm operation windows under the second robotic arm. Based on this, for the position corresponding to the conflicting beat in the first note subsequence, the second operation window corresponding to the second robotic arm is designated as the first restricted area occupied by the first robotic arm under that conflicting beat, thus ensuring that the first robotic arm avoids the spatial range corresponding to the operation window of the second robotic arm when performing the operation corresponding to that beat in the first note subsequence. Correspondingly, for the position corresponding to the conflicting beat in the second note subsequence, the first operation window corresponding to the first robotic arm is designated as the second restricted area occupied by the second robotic arm under that conflicting beat, thus ensuring that the second robotic arm avoids the spatial range corresponding to the operation window of the first robotic arm when performing the operation corresponding to that beat in the second note subsequence. By using the above method, corresponding restricted areas are defined for the two robotic arms at the detected conflict beats, avoiding motion interference between the two robotic arms due to the intersection or overlap of their operation windows within that beat, and ensuring that the first note subsequence and the second note subsequence can be executed in coordination under the same conflict beat.

[0074] Furthermore, if a safety margin for the mechanical structure is required, several additional keys can be added to both sides of the aforementioned union as safety boundaries. It should be noted that restricted areas should at least be set at conflicting beats. To avoid abrupt changes in motion between two beats simply by correcting the conflicting beat, it is usually necessary to re-plan the local time interval consisting of the conflicting beat and several beats before and after it, using the original states at both ends of this local time interval as boundary conditions. If no feasible solution is found under these conditions, the local planning scope can be further expanded, or a feasible solution can be obtained by adjusting note affixes.

[0075] S520. Calculate the transfer cost and perform global optimization based on the first restricted area occupied interval to update the first performance state sequence.

[0076] Specifically, based on the key mapping information of the first note subsequence, combined with the preset performance feasibility constraints and the first restricted area occupancy interval, the feasibility of candidate performance states corresponding to each beat in the first note subsequence is re-evaluated. For any candidate performance state, if its corresponding key falls into the first restricted area occupancy interval, or if the candidate performance state does not meet the preset performance feasibility constraints, then the candidate performance state is determined as infeasible and removed from the candidate states of that beat; otherwise, the candidate performance state is retained as the re-determined feasible state of that beat. After obtaining the feasible states corresponding to each beat in the first note subsequence, the transition cost is calculated for the changes in the re-determined feasible states between adjacent beats. After calculating the transition cost between adjacent beats, global optimization is performed with the total transition cost of the candidate performance state paths corresponding to the entire first note subsequence as the optimization objective. That is, a complete path is formed by selecting a candidate performance state for each beat according to the beat order, and the path that minimizes the cumulative transition cost of the first note subsequence from beginning to end is searched among all candidate paths that meet the performance feasibility constraints and do not occupy the first restricted area occupancy interval. Based on the global optimization results, the combination of states that maximizes the cumulative transition cost is selected from the feasible candidate performance states for each beat, and the first performance state sequence is updated accordingly. This results in the final first performance state sequence having the best overall performance coherence while satisfying performance feasibility and forbidden zone constraints.

[0077] S530. Based on the second restricted area occupied interval, perform transfer cost calculation and global optimization to update the second performance state sequence.

[0078] Specifically, based on the key mapping information of the second note subsequence, combined with the preset performance feasibility constraints and the second forbidden zone occupancy interval, the feasibility of candidate performance states corresponding to each beat in the second note subsequence is re-evaluated. For any candidate performance state, if its corresponding key falls into the second forbidden zone occupancy interval, or if the candidate performance state does not meet the preset performance feasibility constraints, then the candidate performance state is determined as infeasible and removed from the candidate states of that beat; otherwise, the candidate performance state is retained as the re-determined feasible state of that beat. After obtaining the feasible states corresponding to each beat in the second note subsequence, the transition cost is calculated for the changes in the re-determined feasible states between adjacent beats. After calculating the transition cost between adjacent beats, global optimization is performed with the total transition cost of the candidate performance state paths corresponding to the entire second note subsequence as the optimization objective. That is, a complete path is formed by selecting a candidate performance state for each beat according to the beat order, and the path that minimizes the cumulative transition cost of the second note subsequence from beginning to end is searched among all candidate paths that meet the performance feasibility constraints and do not occupy the second forbidden zone occupancy interval. Based on the global optimization results, the combination of states that maximizes the cumulative transition cost is selected from the feasible candidate performance states for each beat, and the second performance state sequence is updated accordingly. This results in the final second performance state sequence having the best overall performance coherence while satisfying performance feasibility and forbidden zone constraints.

[0079] In the above implementation, when there is a conflict in the conflict detection result, a first forbidden zone occupancy interval is set in the first note subsequence based on the key area corresponding to the conflicting beat, and a second forbidden zone occupancy interval is set in the second note subsequence. The transfer cost is calculated and global optimization is performed based on the first forbidden zone occupancy interval to update the first performance state sequence, and the transfer cost is calculated and global optimization is performed based on the second forbidden zone occupancy interval to update the second performance state sequence. This can effectively eliminate the performance key conflict while ensuring that the updated first and second performance state sequences are generally better and have a natural connection, thereby improving the rationality, feasibility and hand coordination of the performance allocation.

[0080] In some implementations, the method may further include: if there is a conflict in the conflict detection results, swapping the attribution identifiers of the first robotic arm and the second robotic arm at the conflict beat, and updating the first performance state sequence and the second performance state sequence according to the swapped attribution identifiers.

[0081] Specifically, when the conflict detection result indicates that a conflict exists, the ownership identifiers of the first and second robotic arms at the conflict beats are swapped. Based on the swapped arm ownership identifiers of each beat in the piece to be played, the piece is decomposed again to obtain a new first note subsequence corresponding to the first robotic arm and a new second note subsequence corresponding to the second robotic arm.

[0082] Based on the key mapping information of the new first note subsequence and combined with the preset performance feasibility constraints, the feasibility of each candidate performance state corresponding to each beat in the new first note subsequence is re-evaluated. For any candidate performance state, if it does not meet the preset performance feasibility constraints, it is determined as infeasible and removed from the candidate states for that beat; otherwise, it is retained as a newly determined feasible state for that beat. After obtaining the feasible states corresponding to each beat in the new first note subsequence, the transition cost is calculated based on the changes in feasible states between adjacent beats. Global optimization is performed with the total transition cost of the candidate performance state paths corresponding to the entire new first note subsequence as the optimization objective. That is, for each beat, a candidate performance state is selected and a complete path is formed according to the beat order. Among all candidate paths that meet the performance feasibility constraints, the path that minimizes the cumulative transition cost of the new first note subsequence from beginning to end is searched. Based on the global optimization results, the combination of states that maximizes the cumulative transition cost is selected from the feasible candidate performance states for each beat, and the first performance state sequence is updated accordingly, so that the final first performance state sequence has the best overall performance coherence while satisfying the performance feasibility constraints.

[0083] Based on the key mapping information of the new second note subsequence and combined with the preset performance feasibility constraints, the feasibility of each candidate performance state corresponding to each beat in the new second note subsequence is re-evaluated. For any candidate performance state, if it does not meet the preset performance feasibility constraints, it is determined as infeasible and removed from the candidate states for that beat; otherwise, it is retained as a newly determined feasible state for that beat. After obtaining the feasible states corresponding to each beat in the new first note subsequence, the transition cost is calculated based on the changes in feasible states between adjacent beats. Global optimization is performed with the total transition cost of the candidate performance state paths corresponding to the entire new second note subsequence as the optimization objective. That is, for each beat, a candidate performance state is selected and a complete path is formed according to the beat order. Among all candidate paths that meet the performance feasibility constraints, the path that minimizes the cumulative transition cost of the new second note subsequence from beginning to end is searched. Based on the global optimization results, the combination of states that maximizes the cumulative transition cost is selected from the feasible candidate performance states for each beat, and the second performance state sequence is updated accordingly, so that the final second performance state sequence has the best overall performance coherence while satisfying the performance feasibility constraints.

[0084] In the above embodiments, when there is a conflict in the conflict detection results, the ownership identifiers of the first and second robotic arms at the conflict beat are swapped, and the first and second performance state sequences are updated according to the swapped ownership identifiers. This can quickly eliminate the conflict of robotic arm movements or resource contention at the same beat without replanning the overall performance movements or changing the original performance content. This makes the task allocation of the two robotic arms coordinated, avoids motion interference, and ensures the continuity and safety of the performance process.

[0085] This specification provides a dynamic path planning method for robot performance. Please refer to [link / reference]. Figure 6 The notes belonging to the arm of performer A are extracted into a subsequence called notes. A The notes belonging to the arm of performer B are extracted into a subsequence of notes. B Using the solver, the transition cost is calculated and global optimization is performed based on the first note subsequence corresponding to the first robotic arm to obtain the first performance state sequence sol corresponding to the first note subsequence. A Using the same solver, the transition cost is calculated and global optimization is performed based on the second note subsequence corresponding to the second robotic arm to obtain the second performance state sequence sol corresponding to the second note subsequence. B Based on the first performance state sequence sol A Second performance state sequence sol B Conflict detection is performed to obtain the conflict detection results. If a conflict exists in the conflict detection results, a penalty for restricted areas is introduced, and the first performance state sequence is updated by locally resolving the conflicting beats for several beats before and after the conflict. Alternatively, if a conflict exists in the conflict detection results, the robotic arm affiliation of a note in the conflicting beat is swapped to update the second performance state sequence. The final output is the performance state sequence under conflict-free conditions.

[0086] This specification provides a dynamic path planning device 700 for robot performance. Please refer to [link / reference]. Figure 7 The dynamic path planning device 700 for robot performance includes: a feasible state analysis module 710, a transfer cost calculation module 720, and a performance path determination module 730.

[0087] The feasibility state analysis module 710 is used to perform feasibility analysis based on the key mapping information and feasibility constraints of the music to be played, and to determine the feasibility state corresponding to each beat. The transition cost calculation module 720 is used to combine each feasible state of the previous cycle with each feasible state of the next cycle in adjacent cycles, calculate the transition cost corresponding to each combination, and obtain the state pair transition cost set corresponding to the adjacent cycles. The performance path determination module 730 is used to perform global optimization on the set of transition costs based on the states corresponding to the adjacent beats, with the goal of minimizing the total transition cost of the entire piece of music to be performed, and to backtrack the path to obtain the target performance state sequence corresponding to the piece of music to be performed.

[0088] For a detailed description of the dynamic path planning device for robot performance, please refer to the description of the dynamic path planning method for robot performance above, which will not be repeated here.

[0089] In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a dynamic path planning method for robot performance. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0090] Those skilled in the art will understand that Figure 8 The structures shown are merely block diagrams of some structures related to the solutions disclosed in this specification, and do not constitute a limitation on the computer device to which the solutions disclosed in this specification are applied. Specifically, the computer device may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements.

[0091] In some embodiments, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method steps described above.

[0092] This specification provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method in any of the above embodiments.

[0093] One embodiment of this specification provides a computer program product including instructions that, when executed by a processor of a computer device, enable the computer device to perform the steps of the method described in any of the above embodiments.

[0094] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

Claims

1. A dynamic path planning method for robot performance, characterized in that, The method includes: Feasibility analysis is conducted based on the key mapping information and feasibility constraints of the piece to be played to determine the feasible state corresponding to each beat; Based on the preset beat cycle, determine the available movement time between the previous beat and the next beat in adjacent beats; The number of lateral movement steps and the indicator function are calculated based on the feasible state of the previous beat and the feasible state of the next beat. Based on the available movement time, the number of lateral movement steps, and the indicator function, obtain the set of state pair transition costs corresponding to adjacent beats; Based on the states corresponding to the adjacent beats, a global optimization is performed on the set of transition costs. With the goal of minimizing the total transition cost of the entire piece of music to be played, the path is backtracked to obtain the target performance state sequence corresponding to the piece of music to be played.

2. The method according to claim 1, characterized in that, The key mapping information includes the key number of the beat to be played, and the feasibility constraints include: The robotic arm's operation window in the feasible state covers the key number of the beat to be played; and The operation window of the robotic arm corresponding to the feasible state is within the reachable range of the robotic arm; and The left-hand operation window of the left robotic arm and the right-hand operation window of the right robotic arm corresponding to the feasible state do not overlap.

3. The method according to claim 1, characterized in that, The step involves globally optimizing the transition cost set based on the states corresponding to adjacent beats, with the goal of minimizing the total transition cost throughout the entire piece of music. This is achieved by backtracking the path to obtain the target performance state sequence corresponding to the piece of music, including: Iterate through each feasible state of the next cycle in adjacent cycles. For each feasible state, select the state that minimizes the cumulative transfer cost from all feasible states of the previous cycle as the predecessor state of the next cycle, and record the predecessor state and the corresponding minimum cumulative cost. At the final beat of the piece to be played, the feasible state with the minimum cumulative cost is selected as the termination state. Starting from the termination state, the target performance state sequence corresponding to the piece of music to be played is obtained by backtracking beat by beat according to the recorded preceding states.

4. The method according to claim 1, characterized in that, The method further includes: Based on the arm affiliation identifier of each beat in the piece of music to be played, the piece of music to be played is decomposed to obtain the first note subsequence corresponding to the first robotic arm and the second note subsequence corresponding to the second robotic arm. Based on the first note subsequence corresponding to the first robotic arm, the transfer cost is calculated and global optimization is performed to obtain the first performance state sequence corresponding to the first note subsequence. Based on the second note subsequence corresponding to the second robotic arm, the transfer cost is calculated and global optimization is performed to obtain the second performance state sequence corresponding to the second note subsequence. Conflict detection is performed based on the first performance state sequence and the second performance state sequence to obtain conflict detection results.

5. The method according to claim 4, characterized in that, The method further includes: In the event of a conflict in the conflict detection result, a first forbidden zone is set in the first note subsequence and a second forbidden zone is set in the second note subsequence based on the key area corresponding to the conflicting beat. The transfer cost is calculated and global optimization is performed based on the first restricted area occupancy interval to update the first performance state sequence; The transfer cost is calculated and global optimization is performed based on the second restricted area to update the second performance state sequence.

6. The method according to claim 4, characterized in that, The method further includes: If a conflict exists in the conflict detection result, the attribution identifiers of the first robotic arm and the second robotic arm at the conflict beat are swapped, and the first performance state sequence and the second performance state sequence are updated according to the swapped attribution identifiers.

7. A dynamic path planning device for robot performance, characterized in that, The device includes: The feasibility state analysis module is used to perform feasibility analysis based on the key mapping information and feasibility constraints of the music to be played, and to determine the feasibility state corresponding to each beat. The transfer cost calculation module is used to determine the available movement time between the previous and next beats based on a preset beat cycle; calculate the number of lateral movement steps and the indicator function based on the feasible state of the previous beat and the feasible state of the next beat; and obtain the set of state pair transfer costs corresponding to adjacent beats based on the available movement time, the number of lateral movement steps and the indicator function. The performance path determination module is used to perform global optimization of the transition cost set based on the states corresponding to the adjacent beats, with the goal of minimizing the total transition cost of the entire piece of music to be performed, and backtracks the path to obtain the target performance state sequence corresponding to the piece of music to be performed.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images