Abnormal gait intervention and correction method, music generation method, device, system, equipment and medium

Abstract

The present application discloses an abnormal gait intervention and correction method, a music generation method, an apparatus, a system, a device and a medium, and relates to the field of smart medical technology. The method comprises: in the process of abnormal gait intervention treatment of a target subject according to rhythmic music, real-time acquisition of multimodal physiological data of the target subject, determining the multidimensional emotional characteristics and physical physiological characteristics of the target subject according to the multimodal physiological data; based on the emotional characteristics and the physical physiological characteristics, determining the music parameters at each moment according to the multimodal physiological data and gait parameters; dynamically generating rhythmic music adapted to the current physiological state of the target subject according to the music parameters at each moment. This solution realizes the multi-dimensional collaborative optimization of rhythmic music, improves the personalization and emotional expression of rhythmic music, and improves the synchronization rate of gait movement and hearing.

Description

Technical Field

[0001] The present application relates to the field of computer technology, specifically to the field of smart medical technology, and in particular to an abnormal gait intervention and correction method, a music generation method, apparatus, system, equipment and medium. Background Art

[0002] Rhythmic Auditory Stimulation (RAS) is a neurorehabilitation technique based on music therapy. By providing rhythmic stimulation (such as music and beats) to the motor center, RAS encourages patients with impaired neurological function to align their movement patterns with an external rhythm, thereby improving motor function. RAS is widely used in neurorehabilitation to improve gait and motor function, particularly in patients with Parkinson's disease, stroke sequelae, and cerebral palsy. The core principle of RAS lies in its ability to activate the auditory and motor centers of the brain. Through rhythmic stimulation, RAS controls lower limb muscle movement, adjusts gait patterns, and thus improves gait ability. This technique utilizes the brain's natural response to rhythm, known as the rhythmic entrainment mechanism, to synchronize the patient's motor system with the externally supplied rhythm, thereby improving movement coordination and efficiency.

[0003] Disadvantages of existing gait intervention treatment options:

[0004] Rigidity of music rhythm intervention: Existing gait rhythmic music cues do not match the patient's real-time gait phase, resulting in low motor-auditory synchronization.

[0005] Isolated adjustment of music parameters: Only the rhythm of the music parameters can be adjusted independently, the parameter dimension is single, and there is a lack of multi-dimensional collaborative optimization. Summary of the Invention

[0006] Aiming at the technical problems of low synchronization rate and single rhythmic music in existing gait intervention, an abnormal gait intervention correction method, music generation method, device, system, equipment and medium are provided.

[0007] According to a first aspect, a method for generating music based on physiological data in abnormal gait intervention and correction is provided, comprising:

[0008] During the process of the target subject undergoing abnormal gait intervention treatment according to rhythmic music, multimodal physiological data of the target subject is collected in real time, and multi-dimensional emotional characteristics and physical physiological characteristics of the target subject are determined based on the multimodal physiological data;

[0009] Based on the emotional characteristics and the body physiological characteristics, determining the music parameters at each moment according to the multimodal physiological data and gait parameters;

[0010] Rhythmic music adapted to the current physiological state of the target object is dynamically generated according to the music parameters at each moment.

[0011] According to a second aspect, a method for intervening and correcting abnormal gait is provided, comprising:

[0012] Dynamically generate rhythmic music required for abnormal gait intervention treatment for the target subject by any of the above-mentioned music generation methods based on physiological data in abnormal gait intervention correction;

[0013] The rhythmic music is played to the target subject for auditory stimulation, so as to guide the target subject to perform intervention treatment for abnormal gait based on the rhythmic music.

[0014] According to a third aspect, a music generation device based on physiological data in abnormal gait intervention and correction is provided, comprising:

[0015] a data acquisition unit, configured to acquire multimodal physiological data of the target subject in real time during the process of the target subject undergoing abnormal gait intervention treatment according to rhythmic music, and determine the multidimensional emotional characteristics and physical physiological characteristics of the target subject based on the multimodal physiological data;

[0016] a music parameter determination unit, configured to determine the music parameter at each moment based on the emotional characteristics and the body physiological characteristics and according to the multimodal physiological data and gait parameters;

[0017] The music generation unit is used to dynamically generate rhythmic music adapted to the current physiological state of the target object according to the music parameters at each moment.

[0018] According to a fourth aspect, a system for intervening and correcting abnormal gait is provided, comprising:

[0019] A music generation device based on physiological data during abnormal gait intervention and correction, for generating rhythmic music required for the target subject to undergo abnormal gait intervention treatment in real time;

[0020] A playing unit is used to play the rhythmic music to the target subject for auditory stimulation, so as to guide the target subject to perform intervention treatment for abnormal gait based on the rhythmic music.

[0021] According to the fifth aspect, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a method such as any embodiment of the method for generating music based on physiological data in abnormal gait intervention and correction or an abnormal gait intervention and correction method.

[0022] According to the sixth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, it implements the method of any embodiment of the music generation method based on physiological data in abnormal gait intervention correction or the abnormal gait intervention correction method.

[0023] According to the solution of the present application, during the process of abnormal gait intervention treatment for the target subject based on rhythmic music, multimodal physiological data of the target subject is collected in real time, and the multidimensional emotional characteristics and physical physiological characteristics of the target subject are determined based on the multimodal physiological data. Then, based on the emotional characteristics and the physical physiological characteristics, multiple music parameters at each moment are determined according to the multimodal physiological data and gait parameters, so that the determined multiple music parameters are matched with the physiological state, emotional state and gait of the target subject in real time and dynamically, and then rhythmic music adapted to the current physiological state of the target subject is dynamically generated based on the music parameters at each moment, ensuring that the generated rhythmic music is adapted and matched with the current physiological state and gait of the target subject, which is conducive to improving the gait movement-hearing synchronization rate; at the same time, rhythmic music adapted to the current physiological state of the target subject is dynamically generated based on the multiple music parameters at each moment, realizing multi-dimensional collaborative optimization of rhythmic music, improving the personalization and emotional expression of rhythmic music, and can stimulate and motivate the target subject in multiple dimensions such as emotion, physiology, and hearing, which is conducive to improving the rehabilitation treatment effect of gait intervention correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0025] Figure 1 is a flowchart of an embodiment of a method for generating music based on physiological data in abnormal gait intervention and correction according to the present application;

[0026] Figure 2 is a schematic diagram of an application scenario of the method for generating music based on physiological data in abnormal gait intervention and correction according to the present application;

[0027] Figure 3 is a flow chart of an embodiment of an abnormal gait intervention and correction method according to the present application;

[0028] Figure 4 1 is a schematic structural diagram of an embodiment of a music generation device based on physiological data in abnormal gait intervention and correction according to the present application;

[0029] Figure 5 is a structural schematic diagram of an embodiment of an abnormal gait intervention and correction system according to the present application;

[0030] Figure 6 It is a block diagram of an electronic device used to implement the music generation method based on physiological data in abnormal gait intervention and correction or the abnormal gait intervention and correction method in the embodiment of the present application. DETAILED DESCRIPTION

[0031] The following description of exemplary embodiments of the present application is made in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0032] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0033] An exemplary system architecture to which embodiments of the present application's music generation method (device) or abnormal gait intervention and correction method (system) based on physiological data can be applied may include multiple terminal devices, a network, and a server. The network is a medium that provides communication links between multiple terminal devices and the server. The network can include various connection types, such as wired or wireless communication links or fiber optic cables.

[0034] Multiple terminal devices can be used to collect multimodal physiological data and interact with the server through the network. The multiple terminal devices can be brain wave acquisition devices, heart rate acquisition devices, myoelectric signal acquisition devices, etc.

[0035] The server can be a server that provides various services, such as a backend server that supports the terminal device. The backend server can analyze and process the received multimodal physiological data and other data, and feed the processing results (such as generated rhythmic music) back to the terminal device for playback.

[0036] refer to Figure 1 , shows a process 100 of an embodiment of a method for generating music based on physiological data in abnormal gait intervention and correction according to the present application. The method for generating music based on physiological data in abnormal gait intervention and correction includes the following steps:

[0037] Step 101 : During the process of abnormal gait intervention treatment of a target subject according to rhythmic music, multimodal physiological data of the target subject is collected in real time, and multi-dimensional emotional characteristics and physical physiological characteristics of the target subject are determined based on the multimodal physiological data.

[0038] Step 102: Based on the emotional characteristics and the body physiological characteristics, determine the music parameters at each moment according to the multimodal physiological data and gait parameters.

[0039] Step 103: Dynamically generate rhythmic music adapted to the current physiological state of the target object according to the music parameters at each moment.

[0040] In some optional implementations of this embodiment, in order to accurately determine the emotional characteristics and physical physiological characteristics of the current step in real time, so as to generate comprehensive, personalized, and accurately adapted rhythmic music, it is proposed to determine the multi-dimensional emotional characteristics and physical physiological characteristics of the target object based on the multimodal physiological data, and then determine the music parameters at each moment based on the multimodal physiological data and gait parameters. The inventors of this application have found that the multi-dimensional emotional characteristics and physical physiological characteristics of the target object can be determined by multimodal physiological data such as brain waves, heart rate, and electromyography. For example, the multi-dimensional emotional characteristics and physical physiological characteristics of the target object determined based on the multimodal physiological data include:

[0041] determining the relaxation state of the target subject based on different frequency bands of brain waves;

[0042] determining the target subject's emotional state based on the heart rate;

[0043] The muscle state of the target object is determined based on the myoelectric activity.

[0044] In some optional implementations of this embodiment, during the process of abnormal gait intervention treatment for the target subject according to rhythmic music, the current alpha wave power (Pα) and the minimum value of the alpha wave power (Pα) of the brain electrical signal can be collected by the existing brain electrical signal collection terminal device. ), the maximum value of α wave power ( ), β wave power (Pβ), minimum value of β wave power ( ), the maximum value of β wave power ( ), theta wave power (Pθ), the minimum value of theta wave power ( ), the maximum value of theta wave power ( ).

[0045] The heart rate sensor can be used to collect the average heart rate of the human body during exercise ( ), minimum heart rate ( ), the maximum value of heart rate ( ), heart rate variance (VarHR), minimum heart rate variance ( ), the maximum value of heart rate variance ( )wait.

[0046] The intensity of myoelectric activity during human movement can be collected by myoelectric sensors ( ), the minimum value of myoelectric activity intensity ( ), the maximum value of myoelectric activity intensity ( ), the frequency of myoelectric activity ( ), the minimum value of the myoelectric activity frequency ( ), the maximum value of the myoelectric activity frequency ( )wait.

[0047] In some optional implementations of this embodiment, multimodal physiological data of the target object during the gait intervention and correction process, such as EEG, heart rate, and electromyography, can be collected. The emotions, physiological reactions, and characteristics of the target object during the gait intervention and correction process can be analyzed through the multimodal physiological data. These emotions, physiological reactions, and characteristics can reflect and understand the body's adaptation and needs for gait intervention treatment, so as to generate rhythmic music that adapts to and meets the adaptation and needs.

[0048] In some optional implementations of this embodiment, in order to reflect and understand the body's adaptation and needs to gait intervention therapy based on emotions, physiological reactions, and characteristics, and then generate rhythmic music that adapts to and meets the adaptation and needs, for example, based on the emotional characteristics and the body's physiological characteristics, the music parameters at each moment are determined according to the multimodal physiological data and gait parameters, including:

[0049] determining a first rhythm speed according to the alpha wave power of the brainwave, determining a second rhythm speed according to the gait cycle, and determining a final rhythm speed according to the first rhythm speed and the second rhythm speed, wherein the alpha wave power reflects the relaxation level of the target subject. For example, a higher alpha wave power generally indicates that the individual is in a relaxed state. In this case, the higher alpha wave power can be mapped to a faster first rhythm speed to generate more energetic music;

[0050] determining a first pitch range based on the beta wave power of the brainwave, determining a second pitch range based on the step length, and determining a final pitch range based on the first pitch range and the second pitch range, wherein the beta wave power reflects the alertness of the target subject. For example, a higher beta wave power indicates that the individual is in an alert state, which can be mapped to a wider first pitch range to generate a melody with more variation and tension;

[0051] determining a first harmonic complexity based on the theta wave power of the brainwaves, determining a second harmonic complexity based on an upper limb (e.g., arm) swing angle, and determining a final harmonic complexity based on the first harmonic complexity and the second harmonic complexity, wherein the theta wave power reflects the deep relaxation and creativity of the target subject. For example, a higher theta wave power can be mapped to a more complex harmony, generating a richer musical texture;

[0052] Determining a musical key according to the mean of the heart rate, wherein the mean of the heart rate reflects the excitement level of the target subject. For example, a higher mean of the heart rate indicates that the individual is in an excited state, which can be mapped to a higher musical key to generate more energetic and passionate music;

[0053] Determining a volume variation range based on the heart rate variance, wherein the heart rate variance reflects the degree of emotional fluctuation of the target subject. For example, a larger heart rate variance indicates a larger emotional fluctuation, which can be mapped to a larger volume variation range to generate more dynamic music;

[0054] Determining the sound intensity based on the intensity of the myoelectric activity, wherein the intensity of the myoelectric activity reflects the strength of the target subject's muscle contraction. For example, a higher intensity of the myoelectric activity indicates a more intense muscle contraction, which can be mapped to a higher sound intensity to generate more powerful music.

[0055] The rhythm intensity is determined based on the frequency of the electromyographic activity, wherein the frequency of the electromyographic activity reflects the rate of muscle contraction and relaxation of the target object. For example, a higher frequency of electromyographic activity indicates more frequent muscle contraction and relaxation, which can be mapped to a stronger rhythm intensity to generate more dynamic music.

[0056] In some optional implementations of this embodiment, in order to accurately determine the music parameters at each moment based on the multimodal physiological data and gait parameters, it is proposed to determine the first rhythm speed based on the alpha wave power of the brainwave, including:

[0057] + )

[0058] Where BPM1 is the first rhythm speed, Pα is the current α wave power, unit: μV²; is the minimum value of α wave power, unit: μV²; is the maximum value of α wave power, unit: μV²; The minimum value allowed for the tempo. is the maximum value among the permissible values of the rhythm speed (i.e., the permissible and allowable values of the rhythm speed in the prior art);

[0059] The second cadence speed is determined from the gait cycle using the following formula:

[0060]

[0061] in, is the second rhythm speed, The gait cycle factor is used to convert the gait cycle into a reasonable musical tempo (e.g., the time from the left foot landing to the right foot landing after taking a step, or the time from the right foot landing to the left foot landing after taking a step). The gait cycle factor can be a series of parameters that quantify and describe the gait cycle, which can comprehensively reflect gait characteristics and movement state. For example, the gait cycle factor can include any one or any combination of many gait parameters during walking, such as step length, cadence, step width, upper limb swing angle, phase difference, etc.

[0062] After obtaining the first rhythm speed and the second rhythm speed, the average or weighted sum of the first rhythm speed and the second rhythm speed may be determined as the final rhythm speed.

[0063] Determining a first pitch range according to the beta wave power of the brain wave includes:

[0064] + )

[0065] in, is the first pitch range, Pβ is the current β wave power, is the minimum value of β wave power, is the maximum value of β wave power, is the minimum value in the allowed value range of the pitch (i.e., the value allowed in the pitch range in the prior art), The maximum value allowed in the pitch range;

[0066] The second pitch range is determined according to the step length by the following formula, including:

[0067] PR2= ×(PRmax PRmin)+PRmin

[0068] Among them, PR2 is the second pitch range, is the current step size, is the minimum step size, is the maximum value of the step size (for example, the step size can range from 0.5 to 1.0 meters). Assume =3, =7, is 0.8 meters, then:

[0069] PR2= = =0.6×4+3=2.4+3=5.4.

[0070] After obtaining the first pitch range and the second pitch range, the average or weighted sum of the first pitch range and the second pitch range may be determined as the final pitch range.

[0071] Determining a first harmonic complexity according to the theta wave power of the brain wave includes:

[0072]

[0073] in, is the first harmonic complexity, Pθ is the current θ wave power, is the minimum value of theta wave power, is the maximum value of theta wave power;

[0074] The second harmonic complexity is determined based on the upper limb swing angle using the following formula:

[0075] HC2= ×( )+

[0076] Among them, HC2 is the second harmonic complexity, is the upper limb swing angle, is the minimum value of the upper limb swing angle, is the maximum value of the upper limb swing angle, is the maximum value allowed for harmony complexity, The minimum value allowed for harmony complexity.

[0077] After obtaining the first harmony complexity and the second harmony complexity, an average or weighted sum of the first harmony complexity and the second harmony complexity may be determined as a final harmony complexity.

[0078] Determining a music key according to the mean of the heart rates, comprising:

[0079] + )

[0080] in, For the music tone, is the mean heart rate, is the minimum heart rate, is the maximum heart rate, is the minimum value allowed for the music key. The maximum value allowed for the music key;

[0081] Determining a volume variation range according to the heart rate variance includes:

[0082] + )

[0083] in, is the volume variation range, VarHR is the variance of heart rate, is the minimum value of the heart rate variance, is the maximum value of the heart rate variance, The minimum value allowed in the volume change range. The maximum value allowed in the volume variation range;

[0084] Determining the sound intensity according to the intensity of the myoelectric activity includes:

[0085] + )

[0086] in, is the sound intensity, EMGactivity is the intensity of myoelectric activity, is the minimum value of the intensity of myoelectric activity, is the maximum value of the intensity of myoelectric activity, is the minimum value allowed for the sound intensity. The maximum value allowed for the sound intensity;

[0087] Determining the rhythm intensity according to the frequency of the myoelectric activity includes:

[0088] RI

[0089] Where RI is the rhythm intensity (for example, the value range of the rhythm intensity can be [0,1], 0 represents a weak rhythm intensity, and 1 represents a strong rhythm intensity), EMGfrequency is the frequency of electromyographic activity, is the minimum value of the frequency of myoelectric activity, is the maximum value of the frequency of myoelectric activity.

[0090] In some optional implementations of this embodiment, after obtaining multiple music parameters at each moment, rhythmic music can be generated in real time and dynamically based on the music parameters at each moment. For example, rhythmic music adapted to the current physiological state of the target subject can be dynamically generated based on the music parameters at each moment, including:

[0091] Determine the pitch at each moment based on the final pitch range and the musical key;

[0092] Determine the note duration at each moment based on the final rhythm speed and rhythm stability at that moment, wherein the rhythm stability is determined based on the note intensity and rhythm dynamics;

[0093] Determine the volume at each moment according to the volume variation range at that moment;

[0094] Determine the chord for each moment based on the final harmonic complexity of that moment;

[0095] Determine the rhythm of each moment based on the rhythm stability of that moment;

[0096] The pitch, note duration, volume, chord and rhythm at that moment are combined to form a note at that moment, and the notes at each moment form rhythmic music adapted to the current physiological state of the target object in chronological order.

[0097] In some optional implementations of this embodiment, determining the pitch at each moment based on the final pitch range and the musical key at each moment includes:

[0098] For each pitch in the scale template, the weight of the pitch is calculated based on the final pitch range at that moment, the musical key, and the value of the pitch; the probability of the pitch is determined based on the ratio of the pitch weight to the sum of the weights of all pitches, and the probabilities of each two adjacent pitches are used as the upper and lower limits of a probability interval in time sequence; a random number is randomly generated, and the pitch corresponding to the lower limit of the probability interval in which the random number falls is determined as the pitch at that moment;

[0099] For example, for the gait moment t, according to the music style selected by the target object, the corresponding known scale template sequence S={ , ,…, For each pitch in the scale template, the weight of the pitch is calculated using the following formula based on the final pitch range at the gait moment, the musical key, and the value of the pitch: ,in, is the weight of the i-th pitch, is the value of the i-th pitch, is the final pitch range, is the musical pitch, and the probability of this pitch is calculated using the following formula: = , is the probability of the i-th pitch, is the weight of the jth pitch, and n is the total number of pitches in the scale template.

[0100] For example, let's assume that the scale S is the C major scale. The pitch sequence contained in its scale template is [60, 62, 64, 65, 67, 69, 71] (corresponding to C4, D4, E4, F4, G4, A4, B4). The musical key K is 60 (C4). The final pitch range PR is 5. The current gait time t is 1. The process of calculating the pitch at the current gait time t includes the following steps:

[0101] Step 1: Calculate the weight wi for each pitch si:

[0102]

[0103] For each pitch si the weight is:

[0104] s1=60, = exp(0)=1;

[0105] s2=62, = exp( )=≈exp( 0.08)≈0.923;

[0106] s3=64, = exp( )=≈exp( 0.32)≈0.726;

[0107] s4=65, = exp( )=≈exp( 0.5)≈0.607;

[0108] s5=67, = exp( )=≈exp( 0.98)≈0.375;

[0109] s6=69, = exp( )=≈exp( 1.62)≈0.198;

[0110] s7=71, = exp( )=≈exp( 2.42)≈0.090;

[0111] Step 2: Normalize the weights to get the probability distribution pi of each pitch si and calculate the sum of all weights:

[0112] =1+0.923+0.726+0.607+0.375+0.198+0.090≈3.919.

[0113] Then calculate the probability of each pitch, and use the probability of each two adjacent pitches as the upper and lower limits of a probability interval:

[0114] p1= ≈0.255;

[0115] p2= ≈0.235;

[0116] p3= ≈0.185;

[0117] p4= ≈0.155;

[0118] p5= ≈0.096;

[0119] p6= ≈0.050;

[0120] p7= ≈0.023;

[0121] Step 3: Randomly select a pitch based on the probability distribution and determine it as the pitch at that moment:

[0122] Use a random number generator to generate a random number between 0 and 1, and select the corresponding pitch based on the probability interval that the random number falls in. For example, if the random number is 0.24, it falls between the probability intervals of p1 and p2, so s2 = 62 (D4) is selected as the pitch at that moment.

[0123] In some optional implementations of this embodiment, determining the note duration at each moment based on the final rhythm speed and rhythm stability at that moment includes:

[0124] For rhythm templates (such as rhythm template R={ , ,…, }), for each note duration in the rhythm, the weight of the note duration is calculated according to the final rhythm speed, rhythm stability and value of the note duration at that moment; the probability of the note duration is determined according to the ratio of the weight of the note duration to the total weight of all note durations, and the probabilities of every two adjacent note durations are used as the upper and lower limits of a probability interval in time sequence; a random number is randomly generated, and the note duration corresponding to the lower limit of the probability interval in which the random number falls is determined as the first note duration at that moment, the second note duration at that moment is determined according to the step size, and the final note duration at that moment is determined according to the first note duration and the second note duration at that moment;

[0125] The process of calculating the first note duration is similar to the process of calculating the pitch at each moment. Specifically, the weight of each note duration is calculated using the following formula:

[0126] =exp( )

[0127] in, is the weight of the duration of the i-th note, is the value of the i-th note duration, is the final rhythm speed, is the rhythm stability (e.g., ranging from 0 to 1),

[0128]

[0129] in, is the minimum value allowed for rhythm stability (e.g., 0), is the maximum value allowed for rhythm stability (e.g., 1), is the minimum value allowed for the sound intensity. is the maximum value allowed for the sound intensity. The minimum value (for example, 0) allowed for the rhythm intensity (for example, the range of the rhythm intensity value can be 0 to 1), The maximum value allowed for the rhythmic velocity (for example, 1).

[0130] The probability of each note value is calculated using the following formula:

[0131] = , is the probability of the i-th pitch, r is the weight of the jth pitch, and p is the total number of note durations in the rhythm template R.

[0132] The duration of the second note at that moment is determined according to the step length using the following formula:

[0133] NDR= ×(NDRmax NDRmin)+NDRmin

[0134] Among them, NDR is the duration of the second note at that moment, NDRmax is the maximum value allowed among the values of the note duration, and NDRmin is the minimum value allowed among the values of the note duration.

[0135] After obtaining the first note duration and the second note duration at the moment, the average or weighted sum of the first note duration and the second note duration can be determined as the final note duration at the moment.

[0136] In some optional implementations of this embodiment, determining the volume at each moment according to the volume variation range at that moment includes:

[0137] The volume at that moment is determined by the following formula:

[0138]

[0139] in, For volume, rand() is a random function, is the allowable value range of the volume change range;

[0140] In some optional implementations of this embodiment, determining the chord at each moment according to the final harmonic complexity at that moment includes:

[0141] For chord pattern templates (such as chord pattern C={ ,c2,…, }), calculate the weight of each chord according to the final harmonic complexity at that moment and the value of the chord; determine the probability of the chord according to the ratio of the weight of the chord to the total weight of all chords, and use the probabilities of each two adjacent chords as the upper and lower limits of a probability interval in time sequence; randomly generate a random number, and determine the chord corresponding to the lower limit of the probability interval in which the random number falls as the chord at that moment , will determine the chord at that moment Added to the music sequence M, corresponding to the current note at that moment Play simultaneously;

[0142] The process of calculating chords is similar to the process of calculating the pitch at each moment. Specifically, the weight of each chord is calculated using the following formula:

[0143] =exp( )

[0144] in, is the weight of the i-th chord, is the i-th chord The note of the current moment The pitch distance between for the final harmonic complexity.

[0145] The probability of each note value is calculated using the following formula:

[0146] = , is the probability of the i-th chord, c is the weight of the jth chord, and m is the total number of chords in the chord pattern C.

[0147] In some optional implementations of this embodiment, determining the rhythm at each moment according to the rhythm stability at that moment includes:

[0148] The rhythm of that moment is determined by the following formula:

[0149] =R RS

[0150] in, is the rhythm at that moment, R is the rhythm type in the rhythm template, and RS is the rhythm stability.

[0151] Continue to see Figure 2 , Figure 2 This is a schematic diagram of an application scenario of the method for generating music based on physiological data in abnormal gait intervention and correction according to this embodiment. Figure 2 In an application scenario, while a target subject is undergoing abnormal gait intervention treatment based on rhythmic music, the execution entity 201 collects multimodal physiological data 202 of the target subject in real time and determines the target subject's multi-dimensional emotional characteristics and physical physiological characteristics 203 based on the multimodal physiological data. Based on the emotional characteristics and physical physiological characteristics, the execution entity 201 determines music parameters 204 at each moment according to the multimodal physiological data and gait parameters. Based on the music parameters at each moment, the execution entity 201 dynamically generates rhythmic music 205 that is adapted to the target subject's current physiological state.

[0152] Continue to see Figure 3 , Figure 3 FIG. 1 is a flow chart of an embodiment of the abnormal gait intervention and correction method according to this embodiment, comprising the following steps:

[0153] Step 301: Dynamically generate rhythmic music required for abnormal gait intervention treatment for the target subject by any of the aforementioned music generation methods based on physiological data in abnormal gait intervention correction;

[0154] Step 302: Play the rhythmic music to the target subject for auditory stimulation, so as to guide the target subject to perform intervention treatment for abnormal gait based on the rhythmic music.

[0155] Further references Figure 4 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a music generation device based on physiological data in abnormal gait intervention correction. Figure 1 Corresponding to the method embodiment shown, in addition to the features described below, the device embodiment may also include Figure 1 The device can be applied to various electronic devices.

[0156] like Figure 4As shown, the music generation device 400 based on physiological data in abnormal gait intervention and correction of this embodiment includes: a data acquisition unit 401, a music parameter determination unit 402, and a music generation unit 403. The data acquisition unit 401 is configured to collect multimodal physiological data of the target subject in real time during the process of the target subject undergoing abnormal gait intervention treatment based on rhythmic music, and determine the multi-dimensional emotional characteristics and physical physiological characteristics of the target subject based on the multimodal physiological data; the music parameter determination unit 402 is configured to determine the music parameters at each moment based on the emotional characteristics and physical physiological characteristics, the multimodal physiological data, and the gait parameters; and the music generation unit 403 is configured to dynamically generate rhythmic music adapted to the current physiological state of the target subject based on the music parameters at each moment.

[0157] In this embodiment, the specific processing of the data acquisition unit 401, the music parameter determination unit 402 and the music generation unit 403 of the music generation device 400 based on physiological data in abnormal gait intervention correction and the technical effects thereof can be referred to respectively. Figure 1 The relevant descriptions of step 101, step 102 and step 103 in the corresponding embodiment are not repeated here.

[0158] Further references Figure 5 As an implementation of the methods shown in the above figures, the present application provides an embodiment of an abnormal gait intervention and correction system 500. Figure 3 Corresponding to the method embodiment shown, in addition to the features described below, the system embodiment may also include Figure 3 The system can be applied to various electronic devices.

[0159] like Figure 5 As shown, the abnormal gait intervention and correction system 500 of this embodiment includes: a music generation device 400 based on physiological data during abnormal gait intervention and correction, and a playback unit 501. The music generation device 400 based on physiological data during abnormal gait intervention and correction is configured to generate rhythmic music required for the target subject to undergo abnormal gait intervention treatment in real time; and the playback unit 501 is configured to play the rhythmic music to the target subject for auditory stimulation, thereby guiding the target subject to undergo abnormal gait intervention treatment based on the rhythmic music.

[0160] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0161] like Figure 6, is a block diagram of an electronic device for a method for generating music based on physiological data in abnormal gait intervention correction or an abnormal gait intervention correction method according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0162] like Figure 6 As shown, the electronic device includes: one or more processors 601, a memory 602, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 601 is taken as an example.

[0163] Memory 602 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor, causing the at least one processor to execute the music generation method based on physiological data for abnormal gait intervention and correction or the abnormal gait intervention and correction method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to execute the music generation method based on physiological data for abnormal gait intervention and correction or the abnormal gait intervention and correction method provided in this application.

[0164] The memory 602 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the music generation method based on physiological data in abnormal gait intervention and correction in the embodiment of the present application or the program instructions / modules corresponding to the abnormal gait intervention and correction method (for example, the attached Figure 4The processor 601 executes the non-transient software programs, instructions, and modules stored in the memory 602 to execute various functional applications and data processing of the server, thereby implementing the music generation method based on physiological data in the abnormal gait intervention and correction method or the abnormal gait intervention and correction method in the above-mentioned method embodiment.

[0165] The memory 602 may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the method for generating music based on physiological data in abnormal gait intervention and correction or the use of an electronic device for the abnormal gait intervention and correction method. Furthermore, the memory 602 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory 602 may optionally include a memory remotely located relative to the processor 601. These remote memories may be connected to the method for generating music based on physiological data in abnormal gait intervention and correction or the electronic device for the abnormal gait intervention and correction method via a network. Examples of the aforementioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0166] The electronic device of the method for generating music based on physiological data in abnormal gait intervention and correction or the method for abnormal gait intervention and correction may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603 and the output device 604 may be connected via a bus or other means. Figure 6 The bus connection is taken as an example.

[0167] The input device 603 can receive input digital or character information and generate key signal input related to user settings and function control of the method for generating music based on physiological data in abnormal gait intervention and correction, or an electronic device for the abnormal gait intervention and correction method. Input devices such as a touch screen, keypad, mouse, trackpad, touchpad, indicator stick, one or more mouse buttons, trackball, joystick, and the like can be used. The output device 604 can include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibration motors). The display device may include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0168] Various implementations of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuitry, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0169] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0170] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0171] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0172] Computer systems may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other.

[0173] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0174] The units involved in the embodiments described in this application can be implemented by software or hardware. The units described can also be set in a processor. For example, it can be described as: a processor includes a data acquisition unit, a music parameter determination unit, and a music generation unit. In some cases, the names of these units do not constitute limitations on the units themselves. For example, the data acquisition unit can also be described as a "unit for collecting data."

[0175] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the device, the device enables the device to: collect multimodal physiological data of the target object in real time during the process of abnormal gait intervention treatment of the target object according to rhythmic music, and determine the multi-dimensional emotional characteristics and physical physiological characteristics of the target object based on the multimodal physiological data; based on the emotional characteristics and the physical physiological characteristics, determine the music parameters at each moment according to the multimodal physiological data and gait parameters; and dynamically generate rhythmic music adapted to the current physiological state of the target object according to the music parameters at each moment.

[0176] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention herein is not limited to technical solutions formed by a specific combination of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the aforementioned inventive concept. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for generating music based on physiological data during abnormal gait intervention and correction, the method comprising: During the process of the target subject undergoing abnormal gait intervention treatment according to rhythmic music, multimodal physiological data of the target subject is collected in real time, and multi-dimensional emotional characteristics and physical physiological characteristics of the target subject are determined based on the multimodal physiological data; Based on the emotional characteristics and the body physiological characteristics, determining the music parameters at each moment according to the multimodal physiological data and gait parameters; Dynamically generate rhythmic music adapted to the current physiological state of the target subject according to the music parameters at each moment; Based on the emotional characteristics and the body physiological characteristics, determining the music parameters at each moment according to the multimodal physiological data and gait parameters, including: determining a first rhythm speed according to alpha wave power of brain waves, determining a second rhythm speed according to a gait cycle, and determining a final rhythm speed according to the first rhythm speed and the second rhythm speed, wherein the alpha wave power reflects the relaxation level of the target subject; determining a first pitch range according to the beta wave power of the brainwave, determining a second pitch range according to the step length, and determining a final pitch range according to the first pitch range and the second pitch range, wherein the beta wave power reflects the alertness of the target subject; determining a first harmonic complexity according to the theta wave power of the brainwave, determining a second harmonic complexity according to the upper limb swing angle, and determining a final harmonic complexity according to the first harmonic complexity and the second harmonic complexity, wherein the theta wave power reflects the deep relaxation and creativity of the target subject; determining a music key according to a mean value of the heart rate, wherein the mean value of the heart rate reflects the excitement level of the target subject; determining a volume variation range according to the variance of the heart rate, wherein the variance of the heart rate reflects the degree of emotional fluctuation of the target subject; Determining the sound intensity according to the intensity of the electromyographic activity, wherein the intensity of the electromyographic activity reflects the strength of the muscle contraction of the target object; The rhythm strength is determined according to the frequency of the myoelectric activity, wherein the frequency of the myoelectric activity reflects the rate of muscle contraction and relaxation of the target object.

2. The method according to claim 1, wherein Determining the multi-dimensional emotional characteristics and physical physiological characteristics of the target object based on the multimodal physiological data includes: determining the relaxation state of the target subject based on different frequency bands of brain waves; determining the target subject's emotional state based on the heart rate; The muscle state of the target object is determined based on the myoelectric activity.

3. The method according to claim 1, wherein Determining a first rhythm speed according to the alpha wave power of the brain wave includes: + ） Among them, BPM1 is the first rhythm speed, Pα is the current α wave power, is the minimum value of α wave power, is the maximum value of α wave power, The minimum value allowed for the tempo. The maximum value allowed for the tempo; Determining a first pitch range according to the beta wave power of the brain wave includes: + ） in, is the first pitch range, Pβ is the current β wave power, is the minimum value of β wave power, is the maximum value of β wave power, is the minimum value allowed in the pitch range, The maximum value allowed in the pitch range; Determining a first harmonic complexity according to the theta wave power of the brain wave includes: in, is the first harmonic complexity, Pθ is the current θ wave power, is the minimum value of theta wave power, is the maximum value of theta wave power; Determining a music key according to the mean of the heart rates includes: + ） in, For the music tone, is the mean heart rate, is the minimum heart rate, is the maximum heart rate, is the minimum value allowed for the music key. The maximum value allowed for the music key; Determining a volume variation range according to the heart rate variance includes: + ） in, is the volume variation range, VarHR is the variance of heart rate, is the minimum value of the heart rate variance, is the maximum value of the heart rate variance, The minimum value allowed in the volume change range. The maximum value allowed in the volume variation range; Determining the sound intensity according to the intensity of the myoelectric activity includes: + ） in, is the sound intensity, EMGactivity is the intensity of myoelectric activity, is the minimum value of the intensity of myoelectric activity, is the maximum value of the intensity of myoelectric activity, is the minimum value allowed for the sound intensity. The maximum value allowed for the sound intensity; Determining the rhythm intensity according to the frequency of the myoelectric activity includes: RI Among them, RI is the rhythm intensity, EMGfrequency is the frequency of electromyographic activity, is the minimum frequency of myoelectric activity, is the maximum value of the frequency of myoelectric activity.

4. The method according to claim 1, wherein Dynamically generating rhythmic music adapted to the current physiological state of the target subject according to the music parameters at each moment, including: Determine the pitch at each moment based on the final pitch range and the musical key; Determine the note duration at each moment based on the final rhythm speed and rhythm stability at that moment, wherein the rhythm stability is determined based on the note intensity and rhythm dynamics; Determine the volume at each moment according to the volume variation range at that moment; Determine the chord for each moment based on the final harmonic complexity of that moment; Determine the rhythm of each moment based on the rhythm stability of that moment; The pitch, note duration, volume, chord and rhythm at that moment are combined to form a note at that moment, and the notes at each moment form rhythmic music adapted to the current physiological state of the target object in chronological order.

5. The method according to claim 4, wherein Determine the pitch at each moment based on the final pitch range and the musical key, including: For each pitch in the scale template, the weight of the pitch is calculated based on the final pitch range at that moment, the musical key, and the value of the pitch; the probability of the pitch is determined based on the ratio of the pitch weight to the sum of the weights of all pitches, and the probabilities of each two adjacent pitches are used as the upper and lower limits of a probability interval in time sequence; a random number is randomly generated, and the pitch corresponding to the lower limit of the probability interval in which the random number falls is determined as the pitch at that moment; Determine the note value at each moment based on the final rhythmic speed and rhythmic stability, including: For each note duration in the rhythm template, the weight of the note duration is calculated based on the final rhythm speed, rhythm stability, and the value of the note duration at that moment; the probability of the note duration is determined based on the ratio of the weight of the note duration to the total weight of all note durations, and the probabilities of every two adjacent note durations are used as the upper and lower limits of a probability interval in time sequence; a random number is randomly generated, and the note duration corresponding to the lower limit of the probability interval in which the random number falls is determined as the first note duration at that moment; the second note duration at that moment is determined based on the step size; and the final note duration at that moment is determined based on the first and second note values at that moment; Determine the volume at each moment based on the volume variation range at that moment, including: The volume at that moment is determined by the following formula: in, For volume, rand() is a random function. is the allowable value range of the volume change range; Determine the chord at each moment based on the final harmonic complexity of that moment, including: For each chord in the chord pattern template, the weight of the chord is calculated based on the final harmonic complexity at that moment and the value of the chord; the probability of the chord is determined based on the ratio of the weight of the chord to the total weight of all chords, and the probabilities of each two adjacent chords are used as the upper and lower limits of a probability interval in time sequence; a random number is randomly generated, and the chord corresponding to the lower limit of the probability interval in which the random number falls is determined as the chord at that moment; Determine the rhythm of each moment based on the rhythm stability of that moment, including: The rhythm of that moment is determined by the following formula: =R RS in, is the rhythm at that moment, R is the rhythm type in the rhythm template, and RS is the rhythm stability.

6. A music generation device based on physiological data during abnormal gait intervention and correction, the device comprising: a data acquisition unit, configured to acquire multimodal physiological data of the target subject in real time during the process of the target subject undergoing abnormal gait intervention treatment according to rhythmic music, and determine the multidimensional emotional characteristics and physical physiological characteristics of the target subject based on the multimodal physiological data; a music parameter determination unit, configured to determine the music parameter at each moment based on the emotional characteristics and the body physiological characteristics and according to the multimodal physiological data and gait parameters; a music generation unit, configured to dynamically generate rhythmic music adapted to the current physiological state of the target subject according to the music parameters at each moment; The music parameter determination unit is configured to determine a first rhythm speed according to alpha wave power of brain waves, determine a second rhythm speed according to a gait cycle, and determine a final rhythm speed according to the first rhythm speed and the second rhythm speed, wherein the alpha wave power reflects the relaxation level of the target subject; determining a first pitch range according to the beta wave power of the brainwave, determining a second pitch range according to the step length, and determining a final pitch range according to the first pitch range and the second pitch range, wherein the beta wave power reflects the alertness of the target subject; determining a first harmonic complexity according to the theta wave power of the brainwave, determining a second harmonic complexity according to the upper limb swing angle, and determining a final harmonic complexity according to the first harmonic complexity and the second harmonic complexity, wherein the theta wave power reflects the deep relaxation and creativity of the target subject; determining a music key according to a mean value of the heart rate, wherein the mean value of the heart rate reflects the excitement level of the target subject; determining a volume variation range according to the variance of the heart rate, wherein the variance of the heart rate reflects the degree of emotional fluctuation of the target subject; Determining the sound intensity according to the intensity of the electromyographic activity, wherein the intensity of the electromyographic activity reflects the strength of the muscle contraction of the target object; The rhythm strength is determined according to the frequency of the myoelectric activity, wherein the frequency of the myoelectric activity reflects the rate of muscle contraction and relaxation of the target object.

7. A system for intervention and correction of abnormal gait, comprising: The device of claim 6 is used to generate rhythmic music required for abnormal gait intervention treatment of the target subject in real time; A playing unit is used to play the rhythmic music to the target subject for auditory stimulation, so as to guide the target subject to perform intervention treatment for abnormal gait based on the rhythmic music.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images