A data processing method and system based on a language function detection system

Abstract

This invention relates to the field of speech data processing technology, and discloses a data processing method and system based on a language function detection system. The method includes: acquiring raw detection audio streams and task event logs through the language function detection system; constructing a task-level unified time index sequence; constructing a dynamic silence threshold; performing task-level coarse segmentation on the raw detection audio stream; constructing effective response scores; identifying effective response segments and secondary correction segments; reconstructing task-level structured results; and outputting aphasia language ability assessment data. Compared to existing technologies that primarily process entire recordings uniformly, especially in aphasia rehabilitation assessment scenarios with long pauses, repetitive pronunciations, and intrusive prompts, this invention addresses the technical problem of accurately extracting effective response segments. By using task event-driven temporal alignment and dynamic silence threshold segmentation, this method improves the reliability of aphasia language function assessment data.

Description

Technical Field

[0001] This invention relates to the field of speech data processing technology, and in particular to a data processing method and system based on a language function detection system. Background Technology

[0002] Currently, in aphasia rehabilitation assessment and language function testing applications, language function testing systems are typically used to collect speech data from subjects during tasks such as repetition, naming, following instructions, and free expression. The results of these tasks are then combined to assess language ability. However, existing data processing methods often involve uniform processing of the entire recording or speech segmentation based on fixed thresholds, lacking detailed modeling of the testing process and speech structure features, leading to significant shortcomings in practical applications.

[0003] For example, in the testing scenario of post-stroke aphasia patients, subjects often exhibit characteristics such as prolonged pauses, repetitive pronunciation, speech interruptions, self-correction, and reliance on prompts to complete answers. Simultaneously, the testing process may be marred by therapist prompts, environmental noise, and system notifications. Current technologies typically segment speech segments based solely on fixed energy thresholds or simple speech activity detection methods, making it difficult to accurately distinguish between valid responses and distracting prompts. This can easily lead to the inclusion of invalid speech in the analysis or the accidental deletion of corrective speech containing valid information, resulting in inaccurate correspondence between speech segments and the testing task. Furthermore, current technologies lack a mechanism for processing the temporal relationship between task issuance and speech response, failing to achieve precise alignment between the testing task and speech data. In multi-task continuous testing, this can easily lead to misclassification of speech segments across tasks.

[0004] Therefore, there is an urgent need for a data processing method that can still achieve accurate alignment between the detection task and speech data, accurate extraction of effective response segments, and complete preservation of corrected speech even under complex conditions such as long pauses, repetitive pronunciations, self-correction, and cue interference, so as to improve the accuracy, completeness, and reliability of aphasia language function assessment data. Summary of the Invention

[0005] To address the aforementioned technical shortcomings, the present invention aims to propose a data processing method based on a language function detection system. This method addresses the technical problem that existing technologies primarily rely on processing entire recordings uniformly, particularly in aphasia rehabilitation assessment scenarios where long pauses, repetitive pronunciations, and intrusive prompts prevent the accurate extraction of effective response segments.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: The present invention provides a data processing method based on a language function detection system.

[0007] The data processing method based on the language function detection system includes: Step S10: Collect the original detection audio stream and task event log through the language function detection system, and perform audio timing alignment task based on the original detection audio stream and task event log using a unified timing index construction mechanism, and output a task-level unified timing index sequence; Step S20: Based on the task-level unified time index sequence, a dynamic silence threshold segmentation mechanism is used to perform a task-level coarse segmentation task, and outputs a set of candidate active speech segments and a set of prompting interference speech segments; Step S30: Based on the candidate active speech segment set, a cue interference isolation mechanism is used to perform a preprocessing task, and output the patient's candidate response speech segment set; Step S40: Based on the patient candidate response speech segment set, the effective response extraction and correction chain retention mechanism is used to perform the effective response recognition task, and output the effective response segment set and the secondary correction segment tag set; Step S50: Reconstruct the task-level structured results based on the set of valid response segments, the set of prompting interference speech segments, and the set of secondary correction segment markers, and output the aphasia language ability assessment data.

[0008] Preferably, step S10, which involves collecting the original detection audio stream and task event log through the language function detection system, performing an audio timing alignment task based on the original detection audio stream and task event log using a unified timing index construction mechanism, and outputting a task-level unified timing index sequence, specifically includes: Step S101: Obtain the raw detection audio stream and task event logs collected by the language function detection system during the detection process; Step S102: Obtain the task event anchor point in the task event log and the corresponding acoustic response position in the original detection audio stream. Calculate the time drift between the event timestamp and the audio timestamp based on the task event anchor point and the corresponding acoustic response position. Then, calibrate the task event log according to the time drift to obtain the corrected event time set. Step S103: Establish a mapping relationship between task event status and audio frame index based on the set of corrected event times and the frame-level time series of the original detected audio stream, and output a task-level unified time index sequence.

[0009] Preferably, step S20, which involves performing a task-level coarse segmentation task based on a dynamic silence threshold segmentation mechanism using a task-level unified time index sequence, and outputting a set of candidate active speech segments and a set of cue interference speech segments, specifically includes: Step S201: Obtain the start reference time and end reference time corresponding to each detection task from the task-level unified time index sequence, and construct the candidate segmentation window corresponding to each detection task by combining the preset pre-task extension time and post-task extension time. Step S202: In each candidate segmentation window, the average noise energy, standard deviation of noise energy, and mean of noise zero-crossing rate of the preceding silent segment are extracted using a short-time frame-by-frame statistical method. Based on the average noise energy, standard deviation of noise energy, and mean of noise zero-crossing rate of the preceding silent segment, a task-specific dynamic silent threshold is constructed using a weighted threshold function mapping method. Step S203: Detect speech activity in audio frames within each candidate segmentation window based on the task-specific dynamic silence threshold, and merge audio frames that meet the requirements of short-time frame energy being greater than the task-specific dynamic silence threshold and duration being greater than the preset minimum activity duration threshold into candidate active speech segments, and output a set of candidate active speech segments corresponding to each detection task. Step S204: Merge audio frames that meet the following conditions: short-time frame energy is higher than the average noise energy of the preceding silent section, duration is less than the preset minimum activity duration threshold, and is located in the neighborhood of the start reference time and end reference time corresponding to the detection task into a prompt interference speech segment, and output a set of prompt interference speech segments.

[0010] Preferably, in step S202, the first Each detection task corresponds to a task-specific dynamic mute threshold. Determine according to the following expression: ; in, Indicates the first The average noise energy of the pre-silent segment of the candidate segmentation window corresponding to each detection task. Indicates the first The standard deviation of noise energy for each candidate segmentation window corresponding to a detection task. Indicates the first The mean noise zero-crossing rate of the candidate segmentation window corresponding to each detection task. , as well as This represents the threshold adjustment coefficient.

[0011] Preferably, step S30, which involves performing a preprocessing task based on the candidate active speech segment set using a cue interference isolation mechanism to output the patient's candidate response speech segment set, specifically includes: Step S301: Extract speech feature parameters, prosodic feature parameters, and temporal neighborhood parameters from each speech segment in the candidate activity speech segment set, and establish a segment feature vector corresponding to each speech segment based on the speech feature parameters, prosodic feature parameters, and temporal neighborhood parameters; wherein, the speech feature parameters are used to characterize the acoustic identity characteristics of the speaker in the corresponding speech segment; the prosodic feature parameters are used to characterize the intensity changes, pitch changes, speech rate changes, and pause distribution characteristics of the corresponding speech segment; the temporal neighborhood parameters are used to characterize the temporal positional relationship of the corresponding speech segment relative to the task start reference time, the task end reference time, and the prompt event trigger time; Step S302: Calculate speaker deviation similarity using cosine similarity calculation method based on segment feature vector and preset patient baseline voiceprint template, and construct prompt interference score corresponding to each speech segment based on speaker deviation similarity using exponential scoring function mapping method; Step S303: Divide the speech segments with a prompt interference score less than the preset prompt interference threshold into patient candidate response speech segments, and output the patient candidate response speech segment set.

[0012] Preferably, step S40, which involves performing a valid response recognition task based on the patient's candidate response speech segment set using a valid response extraction and correction chain retention mechanism, and outputting a set of valid response segments and a set of secondary correction segment tags, specifically includes: Step S401: Extract task matching degree, duration reasonableness, internal pause structure reasonableness, repetition suppression degree, and response effectiveness for each speech segment in the patient's candidate response speech segment set, and calculate the effective response score corresponding to each speech segment; Step S402: Determine the audio segments in the patient's candidate audio segment set whose valid answer scores reach the preset correction judgment range as audio segments to be judged and corrected; Step S403: Obtain the time interval and content continuity relationship between adjacent speech segments and the speech segment to be corrected, as well as the syllable continuity, word completion, semantic continuity, and task target matching improvement relationship between the end content of the speech segment to be corrected and the beginning content of the subsequent adjacent speech segments; Step S404: When the time interval between the speech segment to be corrected and its subsequent adjacent speech segments is less than the preset correction interval threshold, and the continuity relationship between the preceding and following content meets the preset correction chain establishment condition, the speech segment to be corrected and its corresponding subsequent adjacent speech segments are merged into a valid response segment, and a corresponding secondary correction segment marker is generated. When the valid answer score of the speech segment to be corrected is greater than or equal to the preset valid answer threshold, and the time interval is greater than or equal to the preset correction interval threshold, the speech segment to be corrected will be directly determined as a valid answer segment. Step S405: The final output is the set of valid response segments and the set of secondary correction segment tags.

[0013] Preferably, in step S401, the score for a valid answer is calculated as follows: For the first... Based on the candidate patient response voice fragments, and the task matching degree Reasonable duration Internal pause structure rationality Repetitive speech inhibition and response effectiveness Constructing effective response scores The score for the valid answer Determine according to the following expression: ; in, , , , as well as Let these be the weight coefficients for each evaluation item, and satisfy the following: ; Among them, task matching degree Used to characterize the The degree of matching between the candidate patient's voice segments and the target content of the current testing task; the reasonableness of the duration. Used to characterize the The degree of conformity between the duration of each patient's candidate voice segment and the preset duration range corresponding to the current detection task; the rationality of the internal pause structure. Used to characterize the The degree of conformity between the pause distribution within each patient's candidate speech segment and the preset pause distribution conditions; repetition suppression degree. Used to characterize the The degree of inhibition of repetitive syllables, repetitive words, or repetitive initial pronunciation segments in the candidate patient's speech fragments; response validity. Used to characterize the The timeliness of the response of each patient's candidate voice segment relative to the reference time at the start of the current detection task.

[0014] The present invention also provides a data processing system based on a language function detection system, comprising: The timing alignment module is used to collect the original detection audio stream and task event log through the language function detection system, and to perform audio timing alignment tasks based on the original detection audio stream and task event log using a unified timing index construction mechanism, and output a task-level unified time index sequence. The coarse segmentation module is used to perform task-level coarse segmentation based on a task-level unified time index sequence and a dynamic silence threshold segmentation mechanism, and outputs a set of candidate active speech segments and a set of prompting interference speech segments. The preprocessing module is used to perform preprocessing tasks based on the candidate active speech segment set using a cue interference isolation mechanism, and outputs a set of candidate patient response speech segments; The effective extraction module is used to perform the effective response recognition task based on the patient candidate response speech segment set using the effective response extraction and correction chain retention mechanism, and outputs the effective response segment set and the secondary correction segment tag set; The results reconstruction module is used to reconstruct task-level structured results based on the set of valid response segments, the set of prompting interference speech segments, and the set of secondary correction segment markers, and output aphasia language ability assessment data.

[0015] The present invention also provides a data processing device based on a language function detection system, comprising: a memory, a processor, and a data processing program based on the language function detection system stored in the memory and executable on the processor. When the data processing program based on the language function detection system is executed by the processor, it implements a data processing method based on the language function detection system.

[0016] The present invention also provides a computer program product, including a data processing program based on a language function detection system, wherein the data processing program based on the language function detection system implements the data processing method based on the language function detection system when executed by a processor.

[0017] The beneficial effects of this invention are as follows: By introducing a unified temporal index construction mechanism driven by task events and a dynamic silence threshold segmentation method based on the noise parameters of the preceding silence segment, this invention achieves accurate alignment between the original detection audio stream and the detection task, and can adaptively adjust the speech activity detection threshold according to the noise environment corresponding to different tasks. This effectively avoids the misjudgment of background noise and non-task speech by the traditional fixed threshold segmentation method in complex environments, and improves the accuracy and stability of task-level speech segmentation.

[0018] This invention constructs a cue interference isolation mechanism based on speech feature parameters, prosodic feature parameters, and temporal neighborhood parameters, and combines it with an effective response score calculation and correction chain retention processing method. This enables the precise identification and retention of repetitive pronunciation, self-correction, and cue-dependent behaviors in aphasic patients, thereby avoiding the problem of erroneous deletion of effective response information by traditional whole-segment processing or simple threshold screening methods, and improving the expressive integrity and judgment reliability of language function assessment data. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the first embodiment of a data processing method based on a language function detection system according to the present invention.

[0021] Figure 2 This is a schematic diagram of a data processing method based on a language function detection system according to the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1: As Figure 1 The diagram shown is a flowchart of the first embodiment of the data processing method based on the language function detection system of the present invention, which presents the first embodiment of the data processing method based on the language function detection system of the present invention.

[0024] In the first embodiment, the data processing method based on the language function detection system includes: Step S10: Collect the original detection audio stream and task event log through the language function detection system, and perform audio timing alignment task based on the original detection audio stream and task event log using a unified timing index construction mechanism, and output a task-level unified timing index sequence; It should be noted that the raw audio stream refers to the speech data continuously collected by the language function detection system during the aphasia rehabilitation assessment process. This includes not only the patient's vocalizations in each detection task, but also background noise in the detection environment, system prompts, therapist assistance prompts, and accompanying acoustic information such as inhalation sounds, cough sounds, and oral preparation sounds before and after the patient's formal responses. The task event log refers to the event information recorded by the detection system during operation related to task progress, including the task issuance time, task display time, prompt trigger time, recording start / stop time, interface switching time, manual intervention time, and task end time. The unified temporal index construction mechanism refers to using the event time nodes in the task event log as a guide, combined with the acoustic response positions corresponding to the event time nodes in the raw audio stream, to uniformly map the audio timeline to the task event timeline, thereby establishing a correspondence between task event states and audio frame positions. The task-level unified temporal index sequence can be understood as a time reference sequence for subsequent task-level processing. Its function is to unify the originally independent "detection task progress process" and "continuous speech acquisition process" into a single temporal framework.

[0025] Understandably, in aphasia rehabilitation assessment scenarios, patients' responses often do not strictly occur immediately after the task is assigned, but may exhibit significant delays in initiation, slow entry into the response state, or continued expression across tasks. Simultaneously, the interface refresh, recording module startup, and prompt tone playback within the detection system may also have device-level or program-level time deviations. Without establishing a unified task-level time index beforehand, subsequent candidate speech segmentation can easily lead to mismatches between speech segments and tasks, thus affecting the accuracy of valid response recognition results. This step ensures that each detection task has a corresponding start reference time, end reference time, and event status marker, thus providing a clear basis for establishing subsequent candidate segmentation windows.

[0026] It should be understood that, compared to the traditional method of roughly segmenting speech based solely on the start and end times of the recording file, page click times, or task start button trigger times, this step does not simply use a specific event time as the task boundary. Instead, it calibrates the task event time using the corresponding acoustic response positions in the original detection audio stream, and further establishes a unified mapping relationship between the task event state and the audio frame index. In other words, traditional processing methods typically assume that the "recorded task start time" is the "starting point of the patient entering the response state." However, in actual detection environments, these two often do not coincide, especially under conditions of device response delays, buffered prompt playback, and therapists needing to wait for the patient's attention to shift. This difference becomes more pronounced. This embodiment, by introducing a unified temporal index construction mechanism, can establish a more accurate correspondence between the task event log and continuous speech data, thereby reducing subsequent misjudgments caused by task boundary offsets. This not only improves the accuracy of task-level speech segmentation but also provides a more stable foundation for subsequent prompt interference recognition, correction chain recognition, and result reconstruction.

[0027] Step S20: Based on the task-level unified time index sequence, a dynamic silence threshold segmentation mechanism is used to perform a task-level coarse segmentation task, and outputs a set of candidate active speech segments and a set of prompting interference speech segments; It should be noted that the dynamic silence threshold segmentation mechanism refers to the use of a fixed, single silence threshold instead of a fixed one. Instead, it adaptively constructs a task-specific threshold for distinguishing active speech from silent backgrounds, based on the noise characteristics of the preceding silent segment within each detection task's candidate segmentation window and the corresponding candidate segmentation window. The candidate active speech segment set refers to the set of speech segments with potential response value extracted from short-term speech activity detection results within the time window corresponding to each detection task. This set may include the patient's actual response speech, as well as prompts, short environmental disturbances, and unstable vocal segments adjacent to the response. The cue interference speech segment set refers to suspected cue speech or short-term interference speech segments identified during the candidate segmentation process, located in the neighborhood of the task start reference time, the neighborhood of the cue event trigger, or short-term non-continuous speech activity segments. Here, "cue interference" is not limited to direct verbal prompts from the therapist, but also includes short voices broadcast at the start of the task, non-patient vocalizations, and discontinuous target vocalizations appearing near the task boundary.

[0028] It should be understood that, compared to traditional methods that rely on fixed energy thresholds or fixed zero-crossing rate thresholds for speech activity detection, this step emphasizes "a dynamic silence judgment benchmark corresponding to each task window." Traditional approaches assume that environmental noise is stable throughout the detection process, thus typically setting a uniform threshold at the start of recording and continuously using that threshold in subsequent detections. However, in actual rehabilitation assessment environments, background noise usually exhibits significant fluctuations. For example, the periodic starting of an air conditioner compressor, movement at the doorway, friction from equipment placement, therapists flipping through pens and papers, and patients touching the table with their limbs can all cause changes in the noise baseline within different task windows. If a fixed threshold is still used, it is easy to misjudge background noise as active speech when noise levels rise, and it is also easy to miss valid segments when the patient answers in a low voice due to an excessively high threshold. This embodiment dynamically constructs thresholds based on the average noise energy, noise fluctuation level, and zero-crossing rate characteristics of the pre-task silence segment, ensuring that different detection tasks obtain segmentation criteria that match the current environmental state, thereby enhancing the robustness of the method in complex acoustic environments. Furthermore, this step extracts short, non-continuous activity segments that are immediately adjacent to the cueing event. This eliminates the need for blindly searching the entire candidate activity segment during subsequent cue interference isolation, allowing a candidate set for cue interference to be established at the initial segmentation stage. For example, during a bedside speech assessment in a ward, the patient's surroundings were relatively quiet when completing the first naming task, but before the second repetition task began, a monitor alarm briefly sounded and a nursing cart passed by. If a fixed threshold is used for segmentation, the short-term noise before the second task might be misidentified as the patient's voice and incorrectly included in the response segment in subsequent processing. By re-estimating the noise energy and fluctuation level of the pre-task silence segment in this step, the silence threshold for that task window can be appropriately increased, thus excluding the alarm tail sound and the cart friction sound from the candidate activity speech segment. As another example, when a therapist says "Look at this picture, tell me what it is" at the moment the task begins, this cueing voice is usually located near the task start reference time, with a short duration but high energy. Through the coarse segmentation process in this step, these short-term activity segments can be separately marked from general candidate speech and used as part of the set of cue interference speech segments, thus creating conditions for subsequent fine isolation.

[0029] Step S30: Based on the candidate active speech segment set, a cue interference isolation mechanism is used to perform a preprocessing task, and output the patient's candidate response speech segment set; It should be noted that the cue interference isolation mechanism refers to extracting multiple parameters from each speech segment in the candidate active speech segment set that characterize the speaker's features, vocalization style, and temporal location. Based on these parameters, the segments are classified and judged to identify the patient's candidate response speech segments and eliminate cue interference speech. Speech feature parameters are mainly used to characterize the acoustic identity attributes of the speaker corresponding to the speech segment, including but not limited to spectral envelope differences, formant distribution, fundamental frequency stability, and voiceprint embedding representation. Prosodic feature parameters are mainly used to characterize the pitch fluctuations, intensity changes, tempo, pauses, and intonation patterns of the speech segment. Temporal neighborhood parameters are mainly used to characterize the temporal location relationship of the speech segment relative to the task start reference time, task end reference time, and cue event trigger time. The patient candidate response speech segment set refers to the set of speech segments that, after cue interference isolation processing, are determined to be more likely to originate from the patient's actual response process. This set will serve as the direct input for subsequent effective response recognition tasks.

[0030] Understandably, the main purpose of this step is to further refine the "potentially useful active speech" obtained in step S20. Since step S20 aims to "avoid missing potential response segments as much as possible," its output candidate active speech segments inevitably contain a certain proportion of non-patient responses, such as therapist prompts, announcements, short interjections near boundaries, and segments whose duration meets the activity criteria but does not belong to the patient's actual expression. Without further isolation of these segments, subsequent effective response recognition results are prone to deviating from the patient's true language ability. This step, by jointly utilizing speech feature parameters, prosodic feature parameters, and temporal neighborhood parameters, can not only determine whether a speech segment "sounds like what the patient said," but also whether it "occurs at a reasonable time and position where the patient should respond" and "whether it has the patient's typical response rhythm and acoustic pattern," thus forming a more reliable set of candidate response speech. For example, in a "follow instructions" task, after showing the patient the operating instructions, the therapist, to confirm the patient's attention, adds, "Raise your hand." The prompt and the patient's subsequent imitative repetition of "lift up" may be similar in literal meaning. If only content relevance is considered, the therapist's prompt could easily be mistakenly included in the patient's response speech set. This isolation mechanism identifies the therapist's prompt as significantly different from the patient's template in acoustics, and its timing is close to the trigger point of the prompt event, with a more complete and fluent rhythm. Therefore, it can be identified as a prompt interference speech. The patient's subsequent intermittent, weak, and slow "lift...up" is more consistent with the patient's response characteristics and is thus retained in the patient's candidate response speech segment set. Similarly, in a naming task, a patient might first utter a meaningless short "uh" before attempting to name the target object. This step does not directly consider this short sound as a response simply because it resembles the patient's template. Instead, it comprehensively judges based on its rhythmic form, duration, and timing to avoid mistakenly including purely preparatory sounds in the subsequent response scoring.

[0031] Step S40: Based on the patient candidate response speech segment set, the effective response extraction and correction chain retention mechanism is used to perform the effective response recognition task, and output the effective response segment set and the secondary correction segment tag set; It should be noted that the effective response extraction and correction chain retention mechanism refers to the multi-dimensional evaluation of each speech segment in the patient's candidate response speech segment set. While identifying effective response content, it establishes sequential relationships between speech segments exhibiting characteristics such as self-correction, repeated attempts, and discontinuous completion, thus preserving the complete response trajectory. The effective response segment set refers to the set of speech segments that, after screening, are determined to effectively reflect the patient's response to the current testing task. The secondary correction segment tagging set refers to the data set formed by tagging the segment chain of "first probing vocalization—then correcting and supplementing—finally completing the expression" that occurs during the patient's response process. This data is used to record the corrective behaviors exhibited by the patient when completing the response. These corrections can be syllable-level corrections, word-level completions, sentence-level restarts, or reorganization of output after a short pause.

[0032] Understandably, the main purpose of this step is to further filter out "responses with real assessment value" from "what sounds like what the patient is saying," without losing the correction and recovery information demonstrated by the patient during the process of completing the response. In aphasia assessment, a patient's responses are often not completed in one go, but may involve pauses, probing, incorrect initiation, repetition of syllables, and self-correction before gradually approaching the task goal. If the traditional method is still used, only the longest continuous segment, the segment with the highest energy, or the segment that is finally formed in one go, although a certain "final result" can be obtained, a large amount of process information highly related to the state of language function will be lost. Through this step, on the one hand, the response value of each speech segment can be comprehensively evaluated based on parameters such as task matching degree, duration reasonableness, internal pause structure reasonableness, repetition suppression degree, and response effectiveness; on the other hand, the speech segments that fall into the correction judgment range can be further analyzed to identify the temporal continuity and content succession relationship with subsequent segments, thereby identifying and retaining the correction chain. The effective set of response segments obtained in this way not only reflects "what was said" but also "how it was said". For example, in a naming task, a picture of a "key" is shown. The patient first utters "ya...", pauses, then utters "yao...", and finally, after a short pause, says "key" completely. If the traditional method of retaining the longest continuous segment is used, only the final "key" might be retained, and the first two tentative utterances might be considered useless. If a uniform scoring method is used, the first two incomplete utterances might significantly lower the overall score, masking the fact that the patient ultimately succeeded in naming. This step identifies the first two utterances as segments to be corrected within the correction judgment range. Then, combining the short time interval between these segments and the subsequent complete utterance, as well as the obvious content completion relationship, they are associated with the final complete naming segment as a correction chain response event. Thus, the output not only shows that the patient ultimately completed the naming task but also indicates that the completion process relied on multiple initiations and corrections. As another example, in a repetition task, the patient first says the beginning of a sentence, then pauses for a long time, and then continues to complete the end of the sentence. This step can improve the relationship and the continuity of the content by improving the task matching degree, and recognize two separate speech segments as part of the same response chain, thereby avoiding the misinterpretation of the subsequent supplementary part as irrelevant speech simply because the pause in the middle is too long.

[0033] Step S50: Reconstruct the task-level structured results based on the set of valid response segments, the set of prompting interference speech segments, and the set of secondary correction segment markers, and output the aphasia language ability assessment data.

[0034] It should be noted that task-level structured result reconstruction refers to merging, associating, and formatting the valid response segments, cue interference speech fragments, and correction behavior markers obtained in the preceding steps, using the detection task as the organizational unit, to form structured result data that can directly reflect the response status of each task. Aphasia language ability assessment data refers to a dataset that can be accessed by subsequent assessment and analysis modules, doctor's image reading interface, or rehabilitation training tracking module. Its content includes, but is not limited to, whether the task was completed, initial response delay, number of valid responses, number of long pauses, number of repeated pronunciations, number of correction chains, recovery status after prompts, and fluency representation of responses. In other words, this step is not simply outputting several speech fragments, but rather further transforming these speech fragments into interpretable, comparable, and traceable assessment information.

[0035] It should be understood that, compared to traditional methods that only output entire audio files, single recognized text, or simple scores, the improvement in this step lies in expanding the language function assessment results from "single-valued results" to "process-oriented structured results." Traditional technologies often only tell users whether a task was "done well" or "what the score is," but cannot clearly explain what pauses, corrections, repetitions, and cue-dependent processes the patient experienced during the response. This makes traditional results, while convenient for quick display, lack sufficient depth of interpretation in medical assessment scenarios, especially unfavorable for detailed observation of recovery trajectories in rehabilitation treatment. The aphasia language ability assessment data obtained through reconstruction in this embodiment includes both the completion status at the result level and behavioral characteristics at the process level, making it more suitable for judging the true changes in language function. For example, two test scores may be similar, but one was completed with the help of multiple cue-dependent responses, while the other was completed voluntarily. Such differences may be completely invisible in traditional score outputs, but in this embodiment, they can be intuitively reflected through structured fields.

[0036] For example, when a patient completed the three tasks of "naming," "repetition," and "following instructions" in the first rehabilitation assessment, the structured output showed that: in the naming task, there were two correction chains and recovery after one prompt; in the repetition task, the initial response delay was relatively long and there were many long pauses; in the follow-up instruction task, although the patient eventually completed it, it relied on a second prompt from the therapist. After two weeks of training, a reassessment revealed a new set of structured results: the number of correction chains in the naming task decreased, the initial response delay in the repetition task shortened, and the reliance on prompts in the follow-up instruction task decreased. By comparing the aphasia language ability assessment data from these two assessments, doctors can more intuitively judge the patient's recovery trend in different language areas. If traditional methods of only retaining the final speech text or a single score are used, it is difficult to identify such rehabilitation details where "the completion method has improved but the final result has not changed much."

[0037] Example 2: Furthermore, the data processing system based on a language function detection system provided by the present invention employs a data processing method based on a language function detection system as described in the above embodiments, and can solve a technical problem related to data processing based on a language function detection system. The beneficial effects of the data processing system based on a language function detection system provided by the present invention are the same as those of the data processing method based on a language function detection system provided in the above embodiments, and other technical features of the data processing system based on a language function detection system are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0038] Example 3: This invention provides a data processing device based on a language function detection system. Please refer to... Figure 2A data processing device based on a language function detection system includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the data processing method based on the language function detection system described in Embodiment 1 above. The data processing device based on a language function detection system in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. This data processing device based on a language function detection system is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this invention. A data processing device based on a language function detection system may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. Random access memory 1004 also stores various programs and data required for the operation of a data processing device based on a language function detection system. Processing device 1001, read-only memory 1002, and random access memory 1004 are interconnected via bus 1005. I / O interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows a data processing device based on a language function detection system to communicate wirelessly or wiredly with other devices to exchange data. Although a data processing device based on a language function detection system with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0039] Example 4: This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data processing method based on a language function detection system as described above. The computer program product provided by this invention can solve a technical problem related to data processing based on a language function detection system. Compared with the prior art, the beneficial effects of the computer program product provided by this invention are the same as those of the data processing method based on a language function detection system provided in the above embodiments, and will not be repeated here.

[0040] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this invention.

[0041] It should be understood that the various parts disclosed in this invention can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0042] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A data processing method based on a language function detection system, characterized in that, The methods include: Step S10: Collect the original detection audio stream and task event log through the language function detection system, and perform audio timing alignment task based on the original detection audio stream and task event log using a unified timing index construction mechanism, and output a task-level unified timing index sequence; Step S20: Based on the task-level unified time index sequence, a dynamic silence threshold segmentation mechanism is used to perform a task-level coarse segmentation task, and outputs a set of candidate active speech segments and a set of prompting interference speech segments; Step S30: Based on the candidate active speech segment set, a cue interference isolation mechanism is used to perform a preprocessing task, and output the patient's candidate response speech segment set; Step S40: Based on the patient candidate response speech segment set, the effective response extraction and correction chain retention mechanism is used to perform the effective response recognition task, and output the effective response segment set and the secondary correction segment tag set; Step S50: Reconstruct the task-level structured results based on the set of valid response segments, the set of prompting interference speech segments, and the set of secondary correction segment markers, and output the aphasia language ability assessment data.

2. The data processing method based on a language function detection system as described in claim 1, characterized in that, In step S10, the steps of collecting the original detection audio stream and task event log through the language function detection system, performing audio timing alignment based on the original detection audio stream and task event log using a unified timing index construction mechanism, and outputting a task-level unified timing index sequence specifically include: Step S101: Obtain the raw detection audio stream and task event logs collected by the language function detection system during the detection process; Step S102: Obtain the task event anchor point in the task event log and the corresponding acoustic response position in the original detection audio stream. Calculate the time drift between the event timestamp and the audio timestamp based on the task event anchor point and the corresponding acoustic response position. Then, calibrate the task event log according to the time drift to obtain the corrected event time set. Step S103: Establish a mapping relationship between task event status and audio frame index based on the set of corrected event times and the frame-level time series of the original detected audio stream, and output a task-level unified time index sequence.

3. The data processing method based on a language function detection system as described in claim 1, characterized in that, Step S20, which involves performing a task-level coarse segmentation task based on a dynamic silence threshold segmentation mechanism using a task-level unified time index sequence, and outputting a set of candidate active speech segments and a set of cue interference speech segments, specifically includes: Step S201: Obtain the start reference time and end reference time corresponding to each detection task from the task-level unified time index sequence, and construct the candidate segmentation window corresponding to each detection task by combining the preset pre-task extension time and post-task extension time. Step S202: In each candidate segmentation window, the average noise energy, standard deviation of noise energy, and mean of noise zero-crossing rate of the preceding silent segment are extracted using a short-time frame-by-frame statistical method. Based on the average noise energy, standard deviation of noise energy, and mean of noise zero-crossing rate of the preceding silent segment, a task-specific dynamic silent threshold is constructed using a weighted threshold function mapping method. Step S203: Detect speech activity in audio frames within each candidate segmentation window based on the task-specific dynamic silence threshold, and merge audio frames that meet the requirements of short-time frame energy being greater than the task-specific dynamic silence threshold and duration being greater than the preset minimum activity duration threshold into candidate active speech segments, and output a set of candidate active speech segments corresponding to each detection task. Step S204: Merge audio frames that meet the following conditions: short-time frame energy is higher than the average noise energy of the preceding silent section, duration is less than the preset minimum activity duration threshold, and is located in the neighborhood of the start reference time and end reference time corresponding to the detection task into a prompt interference speech segment, and output a set of prompt interference speech segments.

4. The data processing method based on a language function detection system as described in claim 3, characterized in that, In step S202, the first Each detection task corresponds to a task-specific dynamic mute threshold. Determine according to the following expression: ； in, Indicates the first The average noise energy of the pre-silent segment of the candidate segmentation window corresponding to each detection task. Indicates the first The standard deviation of noise energy for each detection task corresponding to the candidate segmentation window. Indicates the first The mean noise zero-crossing rate of the candidate segmentation window corresponding to each detection task. , as well as This represents the threshold adjustment coefficient.

5. The data processing method based on a language function detection system as described in claim 1, characterized in that, Step S30, which involves performing a preprocessing task based on the candidate active speech segment set using a cue interference isolation mechanism and outputting a set of candidate patient response speech segments, specifically includes: Step S301: Extract speech feature parameters, prosodic feature parameters, and temporal neighborhood parameters from each speech segment in the candidate activity speech segment set, and establish a segment feature vector corresponding to each speech segment based on the speech feature parameters, prosodic feature parameters, and temporal neighborhood parameters; wherein, the speech feature parameters are used to characterize the acoustic identity characteristics of the speaker in the corresponding speech segment; the prosodic feature parameters are used to characterize the intensity changes, pitch changes, speech rate changes, and pause distribution characteristics of the corresponding speech segment; the temporal neighborhood parameters are used to characterize the temporal positional relationship of the corresponding speech segment relative to the task start reference time, the task end reference time, and the prompt event trigger time; Step S302: Calculate speaker deviation similarity using cosine similarity calculation method based on segment feature vector and preset patient baseline voiceprint template, and construct prompt interference score corresponding to each speech segment based on speaker deviation similarity using exponential scoring function mapping method; Step S303: Divide the speech segments with a prompt interference score less than the preset prompt interference threshold into patient candidate response speech segments, and output the patient candidate response speech segment set.

6. The data processing method based on a language function detection system as described in claim 1, characterized in that, Step S40, which involves performing a valid response recognition task based on the patient's candidate response speech segment set using a valid response extraction and correction chain retention mechanism, and outputting a set of valid response segments and a set of secondary correction segment tags, specifically includes: Step S401: Extract task matching degree, duration reasonableness, internal pause structure reasonableness, repetition suppression degree, and response effectiveness for each speech segment in the patient's candidate response speech segment set, and calculate the effective response score corresponding to each speech segment; Step S402: Determine the audio segments in the patient's candidate audio segment set whose valid answer scores reach the preset correction judgment range as audio segments to be judged and corrected; Step S403: Obtain the time interval and content continuity relationship between adjacent speech segments and the speech segment to be corrected, as well as the syllable continuity, word completion, semantic continuity, and task target matching improvement relationship between the end content of the speech segment to be corrected and the beginning content of the subsequent adjacent speech segments; Step S404: When the time interval between the speech segment to be corrected and its subsequent adjacent speech segments is less than the preset correction interval threshold, and the continuity relationship between the preceding and following content meets the preset correction chain establishment condition, the speech segment to be corrected and its corresponding subsequent adjacent speech segments are merged into a valid response segment, and a corresponding secondary correction segment marker is generated. When the valid answer score of the speech segment to be corrected is greater than or equal to the preset valid answer threshold, and the time interval is greater than or equal to the preset correction interval threshold, the speech segment to be corrected will be directly determined as a valid answer segment. Step S405: The final output is the set of valid response segments and the set of secondary correction segment tags.

7. The data processing method based on a language function detection system as described in claim 6, characterized in that, In step S401, the score for a valid answer is calculated as follows: For the first... Based on the candidate patient response voice fragments, and the task matching degree Reasonable duration Internal pause structure rationality Repetitive speech inhibition and response effectiveness Constructing effective response scores The score for the valid answer Determine according to the following expression: ； in, , , , as well as Let these be the weight coefficients for each evaluation item, and satisfy the following: ； Among them, task matching degree Used to characterize the The degree of matching between the candidate patient's voice segments and the target content of the current testing task; the reasonableness of the duration. Used to characterize the The degree of conformity between the duration of each patient's candidate voice segment and the preset duration range corresponding to the current detection task; the rationality of the internal pause structure. Used to characterize the The degree of conformity between the pause distribution within each patient's candidate speech segment and the preset pause distribution conditions; repetition suppression degree. Used to characterize the The degree of inhibition of repetitive syllables, repetitive words, or repetitive initial pronunciation segments in the candidate patient's speech fragments; response validity. Used to characterize the The timeliness of the response of each patient's candidate voice segment relative to the reference time at the start of the current detection task.

8. A data processing system based on a language function detection system, applied to the data processing method based on a language function detection system according to any one of claims 1 to 7, characterized in that, The data processing system based on the language function detection system includes: The timing alignment module is used to collect the original detection audio stream and task event log through the language function detection system, and to perform audio timing alignment tasks based on the original detection audio stream and task event log using a unified timing index construction mechanism, and output a task-level unified time index sequence. The coarse segmentation module is used to perform task-level coarse segmentation based on a task-level unified time index sequence and a dynamic silence threshold segmentation mechanism, and outputs a set of candidate active speech segments and a set of prompting interference speech segments. The preprocessing module is used to perform preprocessing tasks based on the candidate active speech segment set using a cue interference isolation mechanism, and outputs a set of candidate patient response speech segments; The effective extraction module is used to perform the effective response recognition task based on the patient candidate response speech segment set using the effective response extraction and correction chain retention mechanism, and outputs the effective response segment set and the secondary correction segment tag set; The results reconstruction module is used to reconstruct task-level structured results based on the set of valid response segments, the set of prompting interference speech segments, and the set of secondary correction segment markers, and output aphasia language ability assessment data.

9. A data processing device based on a language function detection system, characterized in that, The data processing device based on the language function detection system includes: a memory, a processor, and a data processing program based on the language function detection system stored in the memory and executable on the processor. When the data processing program based on the language function detection system is executed by the processor, it implements a data processing method based on the language function detection system according to any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a data processing program based on a language function detection system, which, when executed by a processor, implements a data processing method based on a language function detection system as described in any one of claims 1 to 7.

Images