Speech rehabilitation training control method based on voice initiation state evolution

Abstract

The present application relates to the technical field of language rehabilitation, and particularly relates to a speech rehabilitation training adaptive control method and device based on vocalization starting state evolution and a storage medium. The speech rehabilitation training control method based on vocalization starting state evolution comprises the following steps: in the speech rehabilitation training process, a vocalization feature signal is acquired, the vocalization feature signal is used to represent state changes of at least two action links in a vocalization starting action chain; based on the vocalization feature signal, phase states of the at least two action links in the vocalization starting action chain are determined, the phase states are used to represent the advancing degree of the action links in the vocalization starting process; according to the phase states of the action links, a cooperative advancing state of the vocalization starting action chain is determined; and according to the cooperative advancing state, a speech rehabilitation training mode is adjusted, the speech rehabilitation training mode is defined by at least one non-speech output guide channel and / or participation weight configuration of at least one speech output guide channel.

Description

Technical Field

[0001] This invention relates to the field of language rehabilitation technology, and in particular to an adaptive control method, device and storage medium for speech rehabilitation training based on the evolution of vocal activation state. Background Technology

[0002] Existing speech rehabilitation training techniques mainly include therapist-led artificial speech training, observation-imitation training based on mirror neuron mechanisms, and human-computer interactive intelligent speech training combined with multimodal feedback. These training methods typically use the training subject's effective vocalization or clear vocalization attempts as the trigger condition for the training process, and adjust the difficulty of the training content, prompting methods, and training pace based on pronunciation accuracy, vocabulary / sentence completion, or comprehension feedback results during the training process.

[0003] However, in clinical rehabilitation practice, the phenomenon of "difficulty initiating vocalization" is common among people with non-fluent aphasia or severe articulation disorders. That is, even if the trainees have a certain level of comprehension and basic articulation-related motor skills, they still find it difficult to complete the vocalization initiation action or maintain a stable vocalization attempt, resulting in frequent interruptions in the training process and insufficient effective stimulation. Summary of the Invention

[0004] In a first aspect, the present invention provides a speech rehabilitation training control method based on the evolution of vocal priming states, comprising the following steps: During speech rehabilitation training, vocalization feature signals are acquired, which are used to characterize the state changes of at least two action links in the vocalization initiation action chain. Based on the vocal characteristic signal, the stage states of at least two action links in the vocal initiation action chain are determined, and the stage states are used to characterize the degree of advancement of the action links in the vocal initiation process. Based on the stage status of each action link, determine the coordinated advancement status of the sound initiation action chain; Based on the collaborative advancement status, the speech rehabilitation training mode is adjusted, and the speech rehabilitation training mode is defined by the participation weight configuration of at least one non-speech output guidance channel and / or at least one speech output guidance channel.

[0005] In some embodiments, the stage states of each action link include at least an inactive state, an advancing state, and an blocked state.

[0006] In some embodiments, determining the stage state of an action step includes the following steps: Acquire the vocal characteristic signal corresponding to the action segment; Based on the time-domain distribution of the vocal characteristic signals, the stage state of the action segment is determined according to the following rules: When the vocal characteristic signal exists continuously within a preset time range and its variation amplitude remains within a preset variation range, the action segment is determined to be in the advancement stage. When the vocal feature signal is not detected within the preset time range, or the duration of the vocal feature signal within the preset time range is shorter than the preset duration threshold, or the change amplitude of the vocal feature signal is lower than the lower limit of the preset change range, the action segment is determined to be in the non-start stage. Once the action segment has been determined to be in the advancement phase, if the sound characteristic signal is interrupted within the preset time range, or if its change amplitude is lower than the lower limit of the preset change range, the action segment will be determined to be in the blocking phase.

[0007] In some embodiments, the cooperative propulsion state includes at least a normal propulsion state and an abnormal retreat state.

[0008] In some embodiments, the coordinated advancement state of the sound initiation action chain is determined based on the following rules: When at least two action steps are determined to be in the advancement phase and no action steps are determined to be in the blocking phase, the coordinated advancement state is determined to be a normal advancement state; otherwise, the coordinated advancement state is determined to be an abnormal retreat state.

[0009] In some embodiments, when the collaborative advancement state is determined to be a normal advancement state, the adjustment of the speech rehabilitation training mode includes the following rules: If all action links in the vocalization initiation action chain are determined to be in the advancing state, then the speech rehabilitation training mode is configured to increase the participation weight of at least one speech output guidance channel, and to ensure that the participation weight of the speech output guidance channel is not lower than the participation weight of the non-speech output guidance channel. If at least one action link in the speech initiation action chain is determined to be in an inactive state, then the inactive action link is determined as the target action link, and the participation weight of at least one non-voice output guidance channel corresponding to the target action link is increased, while the participation weight of at least one voice output guidance channel is maintained or decreased.

[0010] In some embodiments, when the collaborative advancement state is determined to be an abnormal regression state, the adjustment of the speech rehabilitation training mode includes the following rules: If at least one action link in the speech initiation action chain is determined to be in a blocked state, then the blocked action link is determined to be the blocking target link, the participation weight of at least one voice output guidance channel is reduced or suspended, and the participation weight of at least one non-voice output guidance channel corresponding to the blocking target link is increased, so as to perform a backtracking re-induction on the blocking target link. If there are no action links in the speech initiation action chain that are blocked, and the number of action links in the promotion state is less than two, then at least one action link in the non-initiated state is determined as the compensation target link, the participation weight of at least one non-voice output guidance channel corresponding to the compensation target link is increased, and the participation weight of at least one voice output guidance channel is maintained or decreased, so that the speech rehabilitation training mode is maintained as a non-voice output guidance-dominated initiation induction mode. If the number of actions deemed to be in a blocked state reaches a preset threshold, the speech rehabilitation training mode is configured as a low-channel guidance mode to ensure that the number of guidance channels participating simultaneously does not exceed the preset channel limit, and the non-voice output guidance channel corresponding to the blocked action is designated as the dominant guidance channel.

[0011] In some embodiments, the vocalization initiation action chain includes at least an articulation preparation stage, a breathing coordination stage, and a laryngeal vocalization stage; the non-vocal output guidance channel includes at least an articulation guidance channel, a breathing guidance channel, and a laryngeal vocalization guidance channel.

[0012] Secondly, based on the speech rehabilitation training control method based on the evolution of vocal priming state provided in the first aspect, the present invention also provides a speech rehabilitation training control device based on the evolution of vocal priming state, comprising: The signal acquisition module is used to acquire vocal feature signals during speech rehabilitation training. The vocal feature signals are used to characterize the state changes of at least two action links in the vocal initiation action chain. A stage state determination module is used to determine the stage state of at least two action links in the sound initiation action chain based on the sound characteristic signal. The stage state is used to characterize the degree of advancement of the action link in the sound initiation process. The collaborative advancement state determination module is used to determine the collaborative advancement state of the sound initiation action chain based on the stage state of each action link. The training mode adjustment module is used to adjust the speech rehabilitation training mode according to the collaborative advancement state, wherein the speech rehabilitation training mode is defined by the participation weight configuration of at least one non-speech output guidance channel and / or at least one speech output guidance channel.

[0013] Thirdly, based on the speech rehabilitation training control method based on the evolution of vocal priming state provided in the first aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, is used for the speech rehabilitation training control method based on the evolution of vocal priming state.

[0014] The speech rehabilitation training control method, device, and storage medium based on the evolution of vocal priming state provided by this invention have gains including at least: By acquiring vocal characteristic signals that characterize the state changes of each action link in the vocal initiation chain, the stage states of action links such as articulation preparation, respiratory coordination, and laryngeal vocalization preparation are identified. Based on the coordinated advancement state between multiple links, the participation weight configuration of the speech output guidance channel and the non-speech output guidance channel is adjusted to dynamically adapt the training strategy. When the coordinated advancement state is abnormally regressed, a low-channel guidance mode and a re-induction mechanism of the blocking link are further adopted to improve the intervention effectiveness of rehabilitation training in the early stage of vocal initiation. Attached Figure Description

[0015] From the following description of embodiments in conjunction with the accompanying drawings, aspects, features, and advantages of the present invention will become clearer and more readily understood, in which: Figure 1 A flowchart of a speech rehabilitation training control method based on the evolution of vocal priming state provided by the present invention; Figure 2 This is a schematic diagram of the components of a speech rehabilitation training control device based on the evolution of vocal activation state provided by the present invention. Detailed Implementation

[0016] To facilitate understanding of the present invention by those skilled in the art, several embodiments are now described in detail with reference to the accompanying drawings. It should be understood that the embodiments are for illustrative purposes only and not for limiting the scope of protection of the present invention; the scope of protection of the present invention is defined by the claims, and includes equivalent schemes and equivalent transformations of the claims.

[0017] As mentioned above, in the current speech rehabilitation training process, when patients' vocal initiation is blocked or they have not yet formed effective speech output, the training process relies too much on the vocal results and it is difficult to effectively control different motor stages in the vocal initiation process.

[0018] This invention provides a speech rehabilitation training control method based on the evolution of vocal initiation state. By identifying the stage states of multiple action links in the vocal initiation action chain, constructing their collaborative advancement state, and adaptively adjusting the participation weight configuration of non-speech output guidance channel and speech output guidance channel based on the collaborative advancement state, dynamic control of the vocal initiation process is achieved. This maintains the continuity of training and avoids premature interruption before the patient has formed a stable voice, and enhances the rehabilitation adaptability and intervention accuracy for people with vocal initiation difficulties.

[0019] In one embodiment, see Figure 1 , Figure 1 The flowchart of a speech rehabilitation training control method based on the evolution of vocal priming state provided by the present invention is shown.

[0020] In this embodiment, the speech rehabilitation training control method based on the evolution of vocal priming states specifically includes, as follows: Figure 1 The steps shown are as follows: S01. During speech rehabilitation training, vocalization feature signals are acquired, which are used to characterize the state changes of at least two action links in the vocalization initiation action chain.

[0021] In this embodiment, the vocal initiation action chain refers to a series of measurable action preparations and physiological regulation processes related to vocal initiation that the patient completes sequentially or in synergy during the process of attempting to enter a vocal state from a resting state.

[0022] It should be noted that the vocalization initiation action chain consists of multiple action links with sequential or coordinated relationships, and is not a single action or a single physiological event.

[0023] For example, the vocalization initiation action chain includes at least an articulation preparation step, a breathing coordination step, and a laryngeal vocalization step.

[0024] Specifically, the articulation preparation stage is used to characterize the process of articulation-related muscle groups such as the mouth, lips, tongue, or face changing from a resting state to a vocal preparation state; the breathing coordination stage is used to characterize the preparation process for the formation of a synergistic relationship between breathing rhythm, breathing drive, and articulation movements; and the laryngeal vocalization stage is used to characterize the process of the glottis and laryngeal structures changing from a non-vocal state to a vocal state.

[0025] Furthermore, the articulation preparation stage, the breathing coordination stage, and the laryngeal vocalization stage can be sequential in time, or they can be partially overlapping or coordinated in time, together forming the vocalization initiation action chain.

[0026] In this embodiment, the vocal characteristic signal refers to a signal that reflects the state changes of at least two action links in the vocal initiation action chain, so as to perceive the patient's vocal initiation process before the patient has formed effective speech.

[0027] Based on the articulation preparation stage, breathing coordination stage, and laryngeal vocalization stage in the above examples; furthermore, the vocalization feature signal includes at least the articulation preparation stage feature signal, the breathing coordination stage feature signal, and the laryngeal vocalization stage feature signal.

[0028] In some embodiments, the characteristic signals of the articulation preparation stage are obtained by acquiring information such as changes in mouth shape, lip opening and closing amplitude, and jaw movement trend through a camera device, or by acquiring changes in electromyographic activity of articulation-related muscle groups through a surface electromyography sensor, in order to characterize the state changes of the articulation preparation stage.

[0029] In some embodiments, respiratory coordination characteristic signals are acquired through chest and abdominal respiratory sensors, airflow sensors, or pressure sensors to obtain respiratory rhythm, inspiratory-exhalation transitions, or changes in airflow intensity, in order to characterize the state changes of the respiratory coordination process.

[0030] In some embodiments, the characteristic signals of the laryngeal vocalization process are obtained by acquiring information on changes in laryngeal micro-vibration or glottal preparation through a vibration sensor, a contact pickup device, or an accelerometer located in the neck, in order to characterize the state changes of the laryngeal vocalization process.

[0031] To ensure that various types of vocal feature signals can accurately reflect the state changes of the action links in the vocal initiation action chain, in some other embodiments, step S01 further performs validity judgment on the collected original vocal feature signals.

[0032] For example, the corresponding vocal feature signal segment is marked as a valid signal segment only if the amplitude change of the vocal feature signal exceeds a preset amplitude threshold and the change remains continuous within a preset duration; or abnormal vocal feature signals, such as vocal feature signal segments with obvious noise interference, abnormal amplitude abrupt changes, or abnormally short or long durations, are removed.

[0033] S02. Based on the vocal characteristic signal, determine the stage state of at least two action links in the vocal initiation action chain, wherein the stage state is used to characterize the degree of advancement of the action link in the vocal initiation process.

[0034] Furthermore, the stage state of any action link is used to characterize at least whether the action link has begun to enter the start-up preparation, whether it is in the continuous process of advancement, or whether it has been blocked in the process of advancement and failed to complete the advancement.

[0035] In this embodiment, the stage states of each action link include at least an unstarted stage, a progressive stage, and an interrupted stage. Specifically: when the action link is in the unstarted stage, it indicates that the action link has not yet entered the sound production preparation process or has not yet formed an identifiable start trend; when the action link is in the progressive stage, it indicates that the action link has been activated and continues to evolve towards the sound production preparation state within a preset time window; when the action link is in the interrupted stage, it indicates that the action link is interrupted or regressed after the progressive trend has appeared, resulting in its failure to complete the effective progression from the resting state to the sound production preparation state.

[0036] Furthermore, the stage state of each action can be determined based on the change characteristics of the vocal characteristic signal corresponding to that action in the time dimension.

[0037] In some embodiments, determining the stage state of an action step includes the following steps: S021. Obtain the vocal characteristic signal corresponding to the action segment.

[0038] S022. Based on the time-domain distribution of the vocal characteristic signal, determine the stage state of the action segment according to the following rules: When the vocal characteristic signal exists continuously within a preset time range and its variation amplitude remains within a preset variation interval, the action segment is determined to be in the advancement phase; when the vocal characteristic signal is not detected within the preset time range, or the duration of the vocal characteristic signal within the preset time range is shorter than a preset duration threshold, or the variation amplitude of the vocal characteristic signal is lower than the lower limit of the preset variation interval, the action segment is determined to be in the non-start phase; when the action segment has been determined to be in the advancement phase, if the vocal characteristic signal is interrupted within the preset time range, or its variation amplitude is lower than the lower limit of the preset variation interval, the action segment is determined to be in the blocking phase.

[0039] For example, when the action phase is the articulation preparation phase, the stage state can be determined based on oral movement trajectory signals (such as lip distance, mandibular angle, tongue displacement trajectory, etc.) and / or electromyographic signals related to oral movements. For example, when a continuous movement trajectory of lip closing-opening actions is detected within a preset time range, and the corresponding signal amplitude and its rate of change meet the preset change range constraints, it can be determined as the advancement phase; when only brief, discontinuous micro-movements or noise-level fluctuations are detected, or their duration is shorter than a preset duration threshold, it can be determined as the non-starting phase; when a continuous advancement trend has appeared, and the signal trajectory suddenly interrupts or the amplitude falls back to below the lower limit of the preset change range, it can be determined as the blocking phase.

[0040] For example, when the action segment is a respiratory coordination segment, the stage state can be determined based on the chest and abdominal volume change signal, expiratory airflow signal, and / or respiratory muscle electromyographic signal. For instance, when continuous expiratory airflow is detected within a preset time range, and the airflow intensity and duration meet the preset change range and preset duration threshold, it can be determined as the advancement stage; when the respiratory signal is weak, short, intermittent, or does not meet the continuity requirement, it can be determined as the non-initiated stage; when the expiratory signal is interrupted, the airflow intensity drops sharply, or the duration is significantly shortened after the advancement stage has begun, causing it to no longer meet the preset change range constraint, it can be determined as the blocking stage.

[0041] For example, when the action phase is the laryngeal vocalization preparation phase, the stage state can be determined based on the electromyographic signals of the glottic muscles, the laryngeal vibration detection signal, and / or the fundamental frequency signal related to vocalization preparation. For instance, when the activation level of the glottic muscles is continuously enhanced within a preset time range, or when the fundamental frequency-related features corresponding to vocalization preparation are continuously formed and meet the preset change range constraints, it can be determined as the advancement phase; when the signal does not show effective changes or only shows brief discontinuous fluctuations, it can be determined as the non-starting phase; when the signal is interrupted, falls back, or remains at a low activation level for a long time after an advancement trend has been formed, causing its change amplitude to fall below the lower limit of the preset change range, it can be determined as the blocking phase.

[0042] It should be noted that the key parameters used to determine the state of each action stage, such as the preset time range, preset change interval, and preset duration threshold, can be set according to the rehabilitation stage, signal type, and training task content of the target population, and individualized initialization and dynamic adjustment are supported during the training initialization stage.

[0043] Specifically, the preset time range is used to define the effective response time window of the action signal in order to determine whether the signal change has sufficient continuity and stability; for example, the articulation preparation stage can be set to 500ms to 2s, the breathing coordination stage can be set to 1s to 4s, and the laryngeal vocalization preparation stage can be set to 300ms to 1s. The preset change interval is used to define the effective change amplitude of the vocal characteristic signal, and can be in the form of an absolute amplitude threshold, a change rate threshold, or a relative amplitude threshold, to identify whether the action is in a real progression process. The preset duration threshold is used to exclude interference from short-term occasional fluctuations, ensuring that only signal changes whose duration meets the minimum execution requirement are judged as effective changes, for example, it can be set to no less than 200ms.

[0044] S03. Determine the coordinated advancement state of the sound initiation action chain based on the stage state of each action link.

[0045] In this embodiment, the collaborative advancement state refers to the link-level state formed within a preset time window based on the stage states of multiple action links in the sound-initiating action link.

[0046] The preset time window refers to the time range used to uniformly align and jointly determine the stage states of multiple action links.

[0047] In some embodiments, the preset time window is set as a sliding time window of the most recent 1s to 5s, or as a time window corresponding to the most recent N sampling periods, where N is a positive integer.

[0048] In other embodiments, the moment when any action step enters the advancement stage from the non-starting stage is taken as the starting point of the window, and a judgment window is formed after the starting point for a preset duration, so as to include the state changes of multiple steps in a single sound initiation attempt into the same judgment range.

[0049] Furthermore, the coordinated advancement state is used to reflect whether each action link has established a coordinated advancement relationship that meets the requirements for sound initiation in the time dimension, and it includes at least one of the following: the action links advance sequentially in time order, multiple action links advance overlapping in time, or multiple action links enter the advancement stage synchronously within the same time window.

[0050] In this embodiment, the coordinated advancement state includes a normal advancement state; further, the normal advancement state represents that within a preset time window, the stage states of multiple action links meet the preset requirement of the number of advancement stages and the advancement process is continuous, that is, at least two action links are in the advancement stage, and no action link changes from the advancement stage to the blocking stage; it can be understood that in the normal advancement state, the stage states of each action link show an evolution characteristic from the non-starting stage to the advancement stage in terms of time sequence or time overlap, and there is no interruption or fallback that causes the link advancement to fail.

[0051] In this embodiment, the collaborative advancement state also includes an abnormal rollback state; further, the abnormal rollback state indicates that within a preset time window, there is an action segment that is determined to be in the blocking phase; or the number of action segments in the advancement phase is less than a preset number requirement; or the phase state of the action segment is discontinuous in the time dimension, causing the link to be unable to maintain advancement. By merging the above situations into an abnormal rollback state, rollback-type re-induction and guidance channel configuration adjustment can be triggered at the link level.

[0052] It should be noted that the normal advancement state and abnormal retreat state in this embodiment are link-level states for the entire sound initiation action chain, used to characterize the coordinated advancement of multiple action links within a preset time window, rather than the local stage state of a single action link; in other words, even if individual action links are in the advancement stage, they may be judged as abnormal retreat states due to the blockage of other action links or insufficient advancement.

[0053] In some embodiments, the coordinated advancement state of the sound-initiating action chain can be determined based on the following rules: when at least two action links are determined to be in the advancement stage and there are no action links determined to be in the blocking stage, the coordinated advancement state is determined to be a normal advancement state; otherwise, the coordinated advancement state is determined to be an abnormal retreat state.

[0054] For example, within a preset time window, if both action segment A and action segment B are determined to be in the advancement phase, and other action segments are at most in the non-starting phase without any blocking phase, then the link can be determined to be in a normal advancement state. At this time, it can be considered that at least two action segments have established a time overlap or sequential advancement relationship, thus possessing the link basis for evolving to the sound emission state. Conversely, if action segment A has been determined to be in the advancement phase but subsequently turns into the blocking phase, or if only one action segment is in the advancement phase within the same preset time window, then the link advancement condition is not met, and it should be determined to be an abnormal regression state to trigger the subsequent regression re-induction or low-channel guidance mode.

[0055] S04. Adjust the speech rehabilitation training mode according to the collaborative advancement state. The speech rehabilitation training mode is defined by the participation weight configuration of at least one non-speech output guidance channel and / or at least one speech output guidance channel.

[0056] In this embodiment, the speech rehabilitation training mode refers to the organization and presentation structure of the guidance information output by the trainee. It is defined by at least the type of guidance channel participating in the training, the output strategy of each guidance channel, and the participation weight configuration of each guidance channel, so as to guide the trainee to complete the training process from vocalization initiation preparation to speech output.

[0057] Among them, the non-voice output guidance channel refers to the guidance channel that guides the trainee to advance the vocalization initiation action chain through non-voice prompts without requiring the trainee to produce effective voice output.

[0058] For example, the non-speech output guidance channel may include an articulation guidance channel, a breathing guidance channel, and a laryngeal vocalization guidance channel; wherein, the articulation guidance channel can output visual prompts or step-by-step action sequence prompts of mouth shape / tongue position / jaw opening and closing through the display terminal to induce the articulation preparation stage to enter the advancement state; the breathing guidance channel can guide the trainee to complete the breathing coordination through rhythm strips, breathing progress prompts, or inhalation and exhalation rhythm prompts; the laryngeal vocalization guidance channel can guide the trainee to complete the vocalization preparation related actions without producing speech content through vocalization initiation timing prompts, continuous sound prompts, or non-semantic prompts.

[0059] Among them, the speech output guidance channel refers to a guidance channel that aims to guide the training object to output the target speech through speech-related task prompts, follow-up prompts, speech target prompts, or speech content generation prompts.

[0060] For example, the speech output guidance channel may include a target syllable / word prompting channel, a follow-up prompting channel, a speech feedback prompting channel, etc., to guide the training subject to try to pronounce and output the target syllable, word, or short sentence.

[0061] Furthermore, the participation weight configuration is used to characterize the degree of participation of each guidance channel in the training mode. For example, the participation weight configuration can be reflected in at least one of the following ways: the proportion of output duration of each guidance channel in the training cycle, the proportion of cue frequency, the cue intensity / priority, or the dominant channel selection strategy when multiple channels exist simultaneously.

[0062] Accordingly, adjusting the speech rehabilitation training mode refers to updating the participation weight configuration according to the collaborative advancement status, thereby changing the dominant relationship, output frequency, duration or channel combination of the non-speech output guidance channel and the speech output guidance channel, so that the training process can match the current state of the vocalization initiation action chain.

[0063] In some specific embodiments, when the coordinated advancement state is determined to be a normal advancement state, the adjustment of the speech rehabilitation training mode includes the following rules: If all action links in the vocalization initiation action chain are determined to be in the advancing state, then the speech rehabilitation training mode is configured to increase the participation weight of at least one speech output guidance channel, and to ensure that the participation weight of the speech output guidance channel is not lower than the participation weight of the non-speech output guidance channel. If at least one action link in the speech initiation action chain is determined to be in an inactive state, then the inactive action link is determined as the target action link, and the participation weight of at least one non-voice output guidance channel corresponding to the target action link is increased, while the participation weight of at least one voice output guidance channel is maintained or decreased.

[0064] For example, when it is detected that both the articulation preparation stage and the breathing coordination stage are in progress, and the laryngeal vocalization preparation stage is also in progress, the frequency or duration of prompts in the speech output guidance channel can be increased to guide the trainee from the preparation stage to the attempt to vocalize. When it is detected that the articulation preparation stage is not yet started while other action stages have progressed, the output frequency or dominant priority of the articulation guidance channel can be increased, while the participation weight of the speech output guidance channel can be reduced to avoid the trainee being asked to output speech prematurely when articulation preparation is insufficient, which could lead to failure and interruption.

[0065] When the collaborative advancement state is determined to be an abnormal regression state, the adjustment of the speech rehabilitation training mode includes the following rules: If at least one action link in the speech initiation action chain is determined to be in a blocked state, then the blocked action link is determined to be the blocking target link, the participation weight of at least one voice output guidance channel is reduced or suspended, and the participation weight of at least one non-voice output guidance channel corresponding to the blocking target link is increased, so as to perform a backtracking re-induction on the blocking target link. If there are no action links in the speech initiation action chain that are blocked, and the number of action links in the promotion state is less than two, then at least one action link in the non-initiated state is determined as the compensation target link, the participation weight of at least one non-voice output guidance channel corresponding to the compensation target link is increased, and the participation weight of at least one voice output guidance channel is maintained or decreased, so that the speech rehabilitation training mode is maintained as a non-voice output guidance-dominated initiation induction mode. If the number of actions deemed to be in a blocked state reaches a preset threshold, the speech rehabilitation training mode is configured as a low-channel guidance mode to ensure that the number of guidance channels participating simultaneously does not exceed the preset channel limit, and the non-voice output guidance channel corresponding to the blocked action is designated as the dominant guidance channel.

[0066] For example, when the laryngeal vocalization preparation stage is detected to change from an advancing state to an obstructed state, the prompting for the follow-up reading voice task can be paused, and the frequency or output duration of the prompting in the laryngeal vocalization guidance channel can be increased to guide the trainee to re-enter vocalization preparation. When it is detected that a sufficient number of advancing states have not been formed in each action stage, the initiation induction mode dominated by non-vocal output guidance can be maintained, and the advancing trend can be gradually established through a combination of articulation guidance and breathing guidance. When multiple action stages are blocked simultaneously, causing instability in the link, a low-channel guidance mode can be switched, for example, only the non-vocal output guidance channel corresponding to the main blocking target stage can be retained as the dominant channel to reduce prompt source conflicts and increase the probability of successful re-induction.

[0067] In one embodiment, the present invention provides a speech rehabilitation training control device based on the evolution of vocal activation state, for executing the aforementioned speech rehabilitation training control method, which can be deployed in a rehabilitation training terminal, a computing device, or a control unit that communicates with external sensing devices.

[0068] See Figure 2 , Figure 2 This is a schematic diagram of the components of the speech rehabilitation training control device based on the evolution of vocal activation state provided by the present invention. Figure 2 As shown, the speech rehabilitation training control device based on the evolution of vocal activation state includes the following functional modules: signal acquisition module, stage state determination module, collaborative advancement state determination module, and training mode adjustment module.

[0069] Furthermore, the signal acquisition module is used to collect vocal characteristic signals during speech rehabilitation training. In some specific embodiments, the signal acquisition module can be implemented by communicating with external sensors, acquisition terminals, or data interfaces, or it can be integrated into the training terminal.

[0070] Furthermore, the stage state determination module is connected to the signal acquisition module and is used to analyze and process the acquired vocal characteristic signals, and determine the stage state of at least two action links in the vocal initiation action chain based on preset judgment rules.

[0071] Furthermore, the collaborative advancement state determination module is connected to the stage state determination module and is used to determine the collaborative advancement state of the sound initiation action link within a preset time window based on the stage states of multiple action links.

[0072] Furthermore, the training mode adjustment module is connected to the collaborative advancement state determination module, and is used to adjust the speech rehabilitation training mode according to the determined collaborative advancement state. The speech rehabilitation training mode is defined by the participation weight configuration of at least one non-speech output guidance channel and / or at least one speech output guidance channel.

[0073] For example, the training mode adjustment module can increase the participation weight of the voice output guidance channel when the collaborative advancement state is in the normal advancement state; and decrease the participation weight of the voice output guidance channel and increase the participation weight of the non-voice output guidance channel corresponding to the target action segment when the collaborative advancement state is in the abnormal retreat state, or configure it to a low channel guidance mode.

[0074] In the above embodiments, each functional module can be implemented as a software functional unit, a hardware functional module, or a combination of software and hardware; data interaction between modules can be accomplished through a bus, interface, or internal communication mechanism. Through this modular structure, the device can continuously monitor and determine the state of the vocalization initiation action chain even before the training subject has developed stable speech output, and dynamically adjust the training mode based on the chain-level collaborative advancement state, thereby achieving adaptive control of the speech rehabilitation training process.

[0075] In other embodiments, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is used to implement the speech rehabilitation training control method based on vocal priming state evolution as described in any embodiment of this specification.

[0076] Furthermore, the computer-readable storage medium is used to store a set of program instructions for executing the speech rehabilitation training control method. When the set of program instructions is loaded and executed by a computing device, it can sequentially perform steps including: acquiring vocalization feature signals, determining the stage state of the action link, determining the coordinated advancement state of the vocalization initiation action link, and adjusting the speech rehabilitation training mode according to the coordinated advancement state.

[0077] For example, the computer-readable storage medium may include, but is not limited to: disk, optical disk, flash memory, read-only memory (ROM), random access memory (RAM), solid-state drive, USB flash drive, portable hard drive, or other storage media capable of storing program instructions and readable by a computing device.

[0078] Furthermore, the computer program may exist in the form of source code, object code, executable file, or other arbitrary program instructions, and may be loaded and executed by computing devices including general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other computing devices with data processing capabilities.

[0079] In a specific implementation, by writing each step of the speech rehabilitation training control method based on the evolution of vocal priming state into the computer-readable storage medium in the form of program instructions, when the computer-readable storage medium is loaded onto the target computing device and executed, the computing device can realize the state monitoring of the vocal priming action link, the determination of the collaborative advancement state, and the adaptive adjustment of the training mode without changing the hardware structure, thereby completing the vocal priming monitoring and control process provided by the present invention.

[0080] In the above embodiments, the descriptions of different embodiments have different emphases; technical features not detailed or recorded in a certain embodiment can be understood and implemented by referring to the corresponding records of other embodiments. Unless otherwise expressly stated to the contrary: technical features in each embodiment can be substituted or combined with each other without technical conflict; the order of method steps can be adjusted without affecting the function; the device / module / unit can be implemented by hardware, software or a combination thereof, and can be centralized or distributed; parameters, values ​​or ranges include reasonable errors and equivalent values, and the terms "about", "greater than / less than", "between", and range endpoints are all covered without affecting the technical effect; ordinal numbers such as "first / second" are only used for distinction and do not limit the quantity, priority or structural relationship; the reference numerals and names in the specification and drawings are only illustrative and do not limit the structural form, size ratio or installation position; improvements, substitutions or equivalent solutions that are not explicitly stated but can be obtained by those skilled in the art without creative effort should all be included in the protection scope of this invention.

Claims

1. A speech rehabilitation training control method based on the evolution of vocal priming states, characterized in that, Includes the following steps: During speech rehabilitation training, vocalization feature signals are acquired, which are used to characterize the state changes of at least two action links in the vocalization initiation action chain. Based on the vocal characteristic signal, the stage states of at least two action links in the vocal initiation action chain are determined, and the stage states are used to characterize the degree of advancement of the action links in the vocal initiation process. Based on the stage status of each action link, determine the coordinated advancement status of the sound initiation action chain; Based on the collaborative advancement status, the speech rehabilitation training mode is adjusted, and the speech rehabilitation training mode is defined by the participation weight configuration of at least one non-speech output guidance channel and / or at least one speech output guidance channel.

2. The speech rehabilitation training control method based on the evolution of vocal priming state according to claim 1, characterized in that, Each action phase includes at least three stages: inactive, advancing, and blocked.

3. The speech rehabilitation training control method based on the evolution of vocal priming state according to claim 2, characterized in that, Determining the stage state of an action involves the following steps: Acquire the vocal characteristic signal corresponding to the action segment; Based on the time-domain distribution of the vocal characteristic signals, the stage state of the action segment is determined according to the following rules: When the vocal characteristic signal exists continuously within a preset time range and its variation amplitude remains within a preset variation range, the action segment is determined to be in the advancement stage. When the vocal feature signal is not detected within the preset time range, or the duration of the vocal feature signal within the preset time range is shorter than the preset duration threshold, or the change amplitude of the vocal feature signal is lower than the lower limit of the preset change range, the action segment is determined to be in the non-start stage. Once the action segment has been determined to be in the advancement phase, if the sound characteristic signal is interrupted within the preset time range, or if its change amplitude is lower than the lower limit of the preset change range, the action segment will be determined to be in the blocking phase.

4. The speech rehabilitation training control method based on the evolution of vocal priming state according to claim 2, characterized in that, The coordinated advancement state includes at least a normal advancement state and an abnormal retreat state.

5. The speech rehabilitation training control method based on the evolution of vocal priming state according to claim 4, characterized in that, The coordinated advancement state of the sound-initiating action chain is determined based on the following rules: When at least two action steps are determined to be in the advancement phase and no action steps are determined to be in the blocking phase, the coordinated advancement state is determined to be a normal advancement state; otherwise, the coordinated advancement state is determined to be an abnormal retreat state.

6. The speech rehabilitation training control method based on the evolution of vocal priming state according to claim 5, characterized in that, When the coordinated advancement state is determined to be a normal advancement state, the adjustment of the speech rehabilitation training mode includes the following rules: If all action links in the vocalization initiation action chain are determined to be in the advancing state, then the speech rehabilitation training mode is configured to increase the participation weight of at least one speech output guidance channel, and to ensure that the participation weight of the speech output guidance channel is not lower than the participation weight of the non-speech output guidance channel. If at least one action link in the speech initiation action chain is determined to be in an inactive state, then the inactive action link is determined as the target action link, and the participation weight of at least one non-voice output guidance channel corresponding to the target action link is increased, while the participation weight of at least one voice output guidance channel is maintained or decreased.

7. The speech rehabilitation training control method based on the evolution of vocal priming state according to claim 5, characterized in that, When the collaborative advancement state is determined to be an abnormal regression state, the adjustment of the speech rehabilitation training mode includes the following rules: If at least one action link in the speech initiation action chain is determined to be in a blocked state, then the blocked action link is determined to be the blocking target link, the participation weight of at least one voice output guidance channel is reduced or suspended, and the participation weight of at least one non-voice output guidance channel corresponding to the blocking target link is increased, so as to perform a backtracking re-induction on the blocking target link. If there are no action links in the speech initiation action chain that are blocked, and the number of action links in the promotion state is less than two, then at least one action link in the non-initiated state is determined as the compensation target link, the participation weight of at least one non-voice output guidance channel corresponding to the compensation target link is increased, and the participation weight of at least one voice output guidance channel is maintained or decreased, so that the speech rehabilitation training mode is maintained as a non-voice output guidance-dominated initiation induction mode. If the number of actions deemed to be in a blocked state reaches a preset threshold, the speech rehabilitation training mode is configured as a low-channel guidance mode to ensure that the number of guidance channels participating simultaneously does not exceed the preset channel limit, and the non-voice output guidance channel corresponding to the blocked action is designated as the dominant guidance channel.

8. The speech rehabilitation training control method based on the evolution of vocal priming state according to claims 1-7, characterized in that, The vocalization initiation action chain includes at least the articulation preparation stage, the breathing coordination stage, and the laryngeal vocalization stage; The non-voice output guidance channel includes at least an articulation guidance channel, a breathing guidance channel, and a laryngeal vocalization guidance channel.

9. A speech rehabilitation training control device based on the evolution of vocal priming states, characterized in that, include: The signal acquisition module is used to acquire vocal feature signals during speech rehabilitation training. The vocal feature signals are used to characterize the state changes of at least two action links in the vocal initiation action chain. A stage state determination module is used to determine the stage state of at least two action links in the sound initiation action chain based on the sound characteristic signal. The stage state is used to characterize the degree of advancement of the action link in the sound initiation process. The collaborative advancement state determination module is used to determine the collaborative advancement state of the sound initiation action chain based on the stage state of each action link. The training mode adjustment module is used to adjust the speech rehabilitation training mode according to the collaborative advancement state, wherein the speech rehabilitation training mode is defined by the participation weight configuration of at least one non-speech output guidance channel and / or at least one speech output guidance channel.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, is used to implement the speech rehabilitation training control method based on the evolution of vocal priming state as described in any one of claims 1 to 8.

Images