Autism early screening automatic scoring system and method based on artificial intelligence multi-modal data processing

Abstract

The invention discloses an autism early screening automatic scoring system and method based on artificial intelligence multi-modal data processing, which is an autism early screening auxiliary scheme based on BeDevice toolkit analysis and scoring and comprises a multi-modal data conversion module, an intelligent desensitization module, a dialogue analysis module and an intelligent scoring module. The functions of audio and video transcription, privacy desensitization and automatic scoring are integrated, and single-file or batch processing is supported; the accuracy and flexibility of desensitization are balanced by combining a mixed desensitization form of an AI mode and a rule mode; during audio and video transcription, a multi-engine fault-tolerant mechanism is adopted, standby services are automatically switched, and robustness is improved; the cloud data leakage is avoided by depending on the local FFmpeg and the self-defined module, and the data security is improved; the unstructured scoring standard is analyzed through the DeepSeek model, different scales are adapted, the method adapts to subjects of different age groups, and the application range is wide.

Description

Technical Field

[0001] The present invention belongs to the technical field of children's mental health assessment, involves the screening of autism spectrum disorder, and specifically relates to an automated scoring system and method for early autism screening based on artificial intelligence multi-modal data processing. Background Art

[0002] Autism spectrum disorder (ASD) generally refers to autism, which is also a pervasive developmental disorder, mostly seen in children. The common disease types include the aloof type, the active but strange type, and the passive type. To ensure the healthy development and growth of children, the early screening of autism spectrum disorder (ASD) is of great importance. Currently, the early screening of ASD mainly uses the Behavior Development Screening for Toddlers (BeDevel) toolkit as a commonly used screening tool. The BeDevel toolkit was initially developed as an early ASD screening tool by a Korean team. After later combining the current social and cultural background and the problems existing in the pre-experiment process, it was cross-culturally adjusted, and finally a Chinese version of the BeDevel toolkit was formed, and its clinical applicability in the current social ASD screening work was verified.

[0003] Based on the Chinese version of the BeDevel toolkit, in the current screening process of ASD, it is necessary to rely on professionals to interact with children and assessors through the scales included in the BeDevel toolkit, and to manually judge and score the conversation content between parents and assessors. The traditional process requires recording the entire interaction and conversation content into audio or video, and then subjectively scoring by artificial means according to the items in the scale. However, this mode has the following defects:

[0004] 1. Low scoring efficiency: It relies on manual collation of audio and video data, recording, and scoring, which takes up to several hours and is prone to transcription errors due to the ambiguity of children's language (such as unclear pronunciation and grammar errors);

[0005] 2. Risk of privacy leakage: Personal information involved in the conversation (such as name, address) needs to be manually marked for desensitization, but the traditional rule engine cannot recognize undefined sensitive information such as name abbreviations and homophonic variants;

[0006] 3. Subjectivity of scoring: Manual scoring is affected by the experience of assessors, and there is poor consistency, especially in complex items such as "social response delay" and "repetitive language";

[0007] 4. Lack of scalability: Manual processing is difficult to support the needs of large-scale screening, and the processing time and labor cost increase exponentially.

[0008] Although attempts have been made to address the above problems in the prior art, there are still significant limitations. During transcription, most of the ASR tools used are general-purpose and lack optimization for children's language characteristics, resulting in insufficient recognition accuracy for non-standard pronunciations and discontinuous sentences. In addition, transmitting audio and video data to the cloud poses a risk of privacy leakage and does not meet the requirements for local storage of medical data. In terms of privacy protection, traditional desensitization methods rely on regular expressions and keyword libraries and cannot dynamically identify sensitive information in context. Although existing AI desensitization methods introduce large language models, there are problems with incorrect replacement of key semantics. At the same time, the lack of a user-defined rule interface makes it difficult to adapt to privacy regulations in different regions. In terms of automated scoring, it is mostly limited to the function of form entry and lacks the ability of automated analysis. In addition, it cannot adapt to the dynamic scoring rules of multiple versions of scales, resulting in a decrease in the comparability of cross-institutional data and a high error rate during batch processing.

[0009] In addition, most existing system integration architectures are independent functional modules, such as independent ASR modules and desensitization tools, lacking end-to-end integration. Typical problems include: audio and video need to be imported and exported multiple times to different software, increasing the operation complexity; there is a risk of leakage of intermediate data in the privacy protection link; the system scalability is insufficient, and it is impossible to configure dedicated desensitization rules or scoring models according to the needs of medical institutions. Summary of the Invention

[0010] In view of the above problems, the main object of the present invention is to design an automated scoring system and method for early screening of autism based on artificial intelligence multi-modal data processing, to solve the compatibility conflict between children's language characteristics and general ASR tools, the balance problem between dynamic recognition of sensitive information and semantic integrity, and the dual challenges of system integration and compliance requirements in the medical scenario.

[0011] To achieve the above object, the present invention adopts the following technical solutions:

[0012] An automated scoring system for early screening of autism based on artificial intelligence multi-modal data processing, which is used for the input of single data or batch data, and outputs a scoring form and a scoring result for the multi-modal format of input data being audio, video and text; the system includes:

[0013] A multi-modal data conversion module, which is used to receive and process the input data of audio and video, and transcribe the audio and video into text; it includes an embedded ASR engine, a localization processing architecture, and a batch processing queue management unit;

[0014] An intelligent desensitization module, which is used for the desensitization of text content; it includes a dual-mode desensitization mechanism, namely an AI mode and a rule mode, and the AI mode and the rule mode cooperate to process;

[0015] A dialogue analysis module for parsing the BeDevel toolkit and splitting the desensitized text content; including a structured parsing algorithm and a document output interface.

[0016] An intelligent scoring module for generating a scoring table and scoring results that comply with medical data specifications according to the scoring rules of the BeDevel toolkit; including a dynamic standard loading unit, a semantic matching algorithm, and a standardized output interface. Based on the generated scoring table and scoring results, manual verification is performed to determine the risk of the subject having autism.

[0017] As a further description of the present invention, an embedded ASR engine that supports speech recognition of children's non-standard pronunciations, including a children's speech feature optimization model, a noise suppression unit, and a multi-engine fault tolerance mechanism; the children's speech feature optimization model is used for training pronunciation ambiguity and grammar errors; the noise suppression unit uses an adaptive filtering algorithm based on noise spectrum feature recognition to achieve dynamic noise filtering; the multi-engine fault tolerance mechanism automatically switches the three speech recognition services of JianYingASR, BcutASR, and KuaiShouASR when the confidence threshold is less than 0.7.

[0018] A localization processing architecture for local storage and processing of audio and video data.

[0019] A batch processing queue management unit that supports multi-file parallel processing and priority scheduling.

[0020] As a further description of the present invention, the intelligent desensitization module further includes a custom rule interface for adding a regional sensitive reserved word library.

[0021] A sensitive word recognition unit that analyzes and recognizes sensitive words in the text content through context association of the text content.

[0022] As a further description of the present invention, in the AI mode, a context-sensitive word recognition unit based on the DeepSeek large model uses an initial letter capitalization retention algorithm to replace complete sensitive words, and the sensitive words include names in the text; and a preset reserved word library is called for desensitization of the text content.

[0023] In the rule mode, parallel replacement of single or multiple sensitive words is performed through a regular expression engine.

[0024] As a further description of the present invention, the BeDevel toolkit includes the BeDevel Guide, the Interactive Observation Manual BeDevel-P, the Parent Interview Manual BeDevel-I, and the Education Manual. Among them, BeDevel-P includes 32 activity or observation indicators for different age groups, and BeDevel-I includes 21 interview items for different age groups.

[0025] The scoring rules for BeDevel-P and BeDevel-I adopt a 1-3 point system. One point indicates typical development, and two points and three points indicate atypical development, representing different degrees of atypical or significantly restricted developmental characteristics respectively.

[0026] As a further description of the present invention, the structured parsing algorithm divides the text content into independent documents according to the interview items of BeDevel-I;

[0027] The document output interface is set to a dual-output mode, including the output of item-segmented documents and the output of complete dialogue documents.

[0028] As a further description of the present invention, the dynamic standard loading unit is used to adapt to the scoring rules of different versions of BeDevel-I interview items;

[0029] The semantic matching algorithm matches based on the relevance between the text content converted from audio and video and the interview items of BeDevel-I, and scores the interview items of BeDevel-I corresponding to the dynamic standard loading unit; when the semantic matching confidence is less than the set threshold, the audio and video clip replay verification function is automatically triggered;

[0030] The standardized output interface is an output interface for Excel, which is used to generate a scoring table that conforms to medical data specifications.

[0031] An automated scoring method for early screening of autism based on artificial intelligence multi-modal data processing. This method is based on the above system and includes the following steps:

[0032] The input audio and video data generate text content through the multi-modal data conversion module;

[0033] The intelligent desensitization module performs collaborative processing of the AI mode and the rule mode on the text content. When the confidence of the AI mode is lower than the threshold, the rule mode is used for secondary verification to obtain the desensitized text. The final desensitization result ensures data consistency through hash verification;

[0034] The dialogue analysis module divides the text according to the BeDevel-I interview items and outputs multiple documents or a single document;

[0035] The intelligent scoring module retrieves the scoring rules of the BeDevel-I interview items corresponding to the age group, scores according to the documents segmented by the dialogue analysis module, and outputs the scoring table and scoring results to the specified storage path.

[0036] As a further description of the present invention, the audio and video input is converted into the original text through an embedded ASR engine;

[0037] Perform privacy protection by replacing sensitive words using the algorithm of capitalizing the first letter and regular expressions, and retain the content that does not need to be desensitized based on the preset reserved word library;

[0038] According to the user's selection, load the scoring rules for the BeDevel-I interview items corresponding to different age groups in the dynamic scoring standard library, and perform semantic matching analysis;

[0039] Generate an Excel file containing the desensitized text of the original conversation, the item segmentation structure, and the scoring results.

[0040] Compared with the prior art, the technical effect of the present invention is:

[0041] The present invention provides an automated scoring system and method for early screening of autism based on artificial intelligence multi-modal data processing. It is an auxiliary solution for early screening of autism based on parsing and scoring using the BeDevel tool kit. Through the multi-modal data conversion module, intelligent desensitization module, conversation analysis module, and intelligent scoring module, it integrates the functions of audio and video transcription, privacy desensitization, and automated scoring, and supports single-file or batch processing modes; during desensitization, a hybrid desensitization form combining the AI mode and the rule mode is used to balance the accuracy and flexibility of desensitization; during audio and video transcription, a multi-engine fault tolerance mechanism is adopted to automatically switch to backup services to improve robustness; relying on local FFmpeg and custom modules to avoid cloud data leakage and improve data security; at the same time, unstructured scoring criteria are parsed through the DeepSeek model to adapt to different scales and different age groups of subjects, with a wide range of applications. Brief Description of the Drawings

[0042] Figure 1 It is a schematic diagram of the system architecture of the present invention;

[0043] Figure 2 It is a schematic diagram of the BeDevel-P activity settings and observation indicators in the present invention;

[0044] Figure 3 It is a schematic diagram of the BeDevel-I interview items and applicable ages in the present invention;

[0045] Figure 4 It is a schematic diagram of the processing flow of the scoring system of the present invention;

[0046] Figure 5 It is a schematic diagram of the score generated by the method of the present invention;

[0047] Figure 6 It is a schematic diagram of the comparative experimental data of the method of the present invention and other scoring methods;

[0048] Figure 7 It is a corresponding interface diagram designed according to the system of the present invention. Specific Embodiment

[0049] The present invention will be described in detail below with reference to the accompanying drawings:

[0050] In an embodiment of the present invention, an automated scoring system for early screening of autism based on artificial intelligence multi-modal data processing is disclosed. Referring to Figures 1-3 as shown, this system is used for the input of single data or batch data, and outputs a scoring form and a scoring result for multi-modal formats of audio, video, and text for the input data; specifically, this system includes a multi-modal data conversion module, an intelligent desensitization module, a dialogue analysis module, and an intelligent scoring module; among them, the multi-modal data conversion module is used to receive and process the input data of audio and video, and transcribe the audio and video into text; it includes an embedded ASR engine, a localization processing architecture, and a batch processing queue management unit; the intelligent desensitization module is used for the desensitization of text content; it includes a dual-mode desensitization mechanism, namely the AI mode and the rule mode, and the AI mode and the rule mode cooperate to process; the dialogue analysis module is used to parse the BeDevel toolkit and segment the desensitized text content; it includes a structured parsing algorithm and a document output interface; the intelligent scoring module is used to generate a scoring form and a scoring result that meet the medical data specifications according to the scoring rules of the BeDevel toolkit; it includes a dynamic standard loading unit, a semantic matching algorithm, and a standardized output interface. When using this system to evaluate the early symptoms of autism, the risk of the subject suffering from autism is determined through manual verification based on the generated scoring form and scoring result.

[0051] In this embodiment, the above-mentioned multi-modal data conversion module, intelligent desensitization module, dialogue analysis module, and intelligent scoring module will be specifically described as follows:

[0052] I. Multi-modal data conversion module: It supports the separate input or mixed input of audio / video / text, and locally converts the audio and video according to the batch processing queue management unit to obtain the text content corresponding to the audio and video.

[0053] 1. Embedded ASR engine (AsrTools): It supports speech recognition of children's non-standard pronunciations, including a children's speech feature optimization model and a noise suppression unit.

[0054] (1) Children's Speech Feature Optimization Model: including but not limited to an independent training architecture for training pronunciation ambiguity and grammar errors; the children's speech recognition error rate (such as CER ≤ 8%) and real-time metrics (such as latency < 500ms) meet the performance standards. Based on the characteristics of children's pronunciation ambiguity and grammar errors, a transfer learning + domain adaptation strategy is adopted. On the basis of a general speech model (a hybrid architecture of Conformer), a children's exclusive corpus (more than 5000 hours) is used for secondary training. Enhanced features are extracted through an 80-dimensional Mel filter bank, F(t) = 20log10(|STFT(t)| * Mmel), and Δ + ΔΔ dynamic differential features (such as Δt = F(t + 1) - F(t - 1)) are fused to improve the modeling ability of non-standard pronunciations.

[0055] Model Structure of Conformer Hybrid Architecture:

[0056] Encoder Architecture:

[0057] Conv2D Pre-layer → Conformer Blocks × 12; the Conv2D pre-layer is used to initially extract the local time-frequency features of the speech signal and provide the basic feature representation for the subsequent Conformer Blocks.

[0058] Conformer Block × 12:

[0059] MHSA Module (256d, 4 heads): Adopting a 4-head self-attention mechanism can balance the computational efficiency and the ability to capture global dependencies, which conforms to the optimal head number configuration of Conformer;

[0060] Conv Module (GLU, kernel = 31): Enhancing local feature modeling through a gated linear unit. The 31-point convolutional kernel is close to the recommended optimal value of 32 points in the literature, balancing the receptive field and the computational amount;

[0061] FFN Module (1024d): Two-feedforward network designs (macaron structure) enhance the non-linear expression ability, which is consistent with the Conformer standard architecture.

[0062] Loss Function and Training Strategy:

[0063] Joint Training of CTC Loss + λLM Loss:

[0064] CTC Loss: As the basic loss function for speech recognition, it directly optimizes the alignment path probability and is suitable for end-to-end training;

[0065] LM Loss: Introducing the language model loss (such as Transformer LM) can alleviate the defect of the independence assumption of CTC;

[0066] Balancing coefficient λ: Adjust the weight according to the task to avoid the LM from overly interfering with acoustic modeling.

[0067] L = -∑log p(y|x) + λLM Loss;

[0068] Word Error Rate (WER) evaluation: 4.2% for LibriSpeech test-clean.

[0069] (2) Noise suppression unit: Implement dynamic noise filtering using an adaptive filtering algorithm based on noise spectrum feature recognition, with the signal-to-noise ratio improved by ≥15dB; Supported formats:.WAV,.mp3, and other audio and video files, which are transcoded to WAV through FFmpeg. The implementation of this FFmpeg audio processing includes the steps of a band-pass recorder and the steps of the Gaussian noise reduction algorithm.

[0070] Specifically, band-pass filtering is achieved through cascading high-pass and low-pass filters (200Hz - 3000Hz), meeting the requirements of the children's speech feature frequency band (the main energy is concentrated in 300 - 2500Hz); The standard Butterworth transfer function of the band-pass filter is decomposed into a high-pass and low-pass cascaded form:

[0071]

[0072] where ω1 = 2π × 200 and ω2 = 2π × 3000.

[0073] Gaussian noise reduction algorithm:

[0074] Noise variance formula:

[0075]

[0076] Convolution processing formula:

[0077]

[0078] Complies with the IEEE audio processing standard, with the signal-to-noise ratio improved by ≥15dB (measured data).

[0079] (3) Multi-engine fault tolerance mechanism: When the confidence threshold is less than 0.7 (this is clinical verification data and can also be set according to actual needs), it automatically switches to three speech recognition services, JianYingASR, BcutASR, and KuaiShouASR. The specific priority can be adjusted according to actual needs. The output result is saved as a word document, but is not limited to word documents.

[0080] 2. Localization processing architecture: Used for local storage and processing of audio and video data.

[0081] 3. Batch Processing Queue Management Unit: Supports parallel processing of multiple files and priority scheduling. Performs text integrity verification after the ASR engine conversion, and triggers the manual review process when the speech recognition confidence is < 0.7. Implemented based on the Work-Stealing scheduling algorithm, including the task queue architecture, scheduling strategy, and load balancing.

[0082] Task Queue Architecture:

[0083] Deque: Each thread maintains a deque (Qi) for storing tasks to be executed. The local thread preferentially processes the tasks at the head of the queue, while other threads steal tasks from the tail of the queue to reduce thread competition.

[0084] Queue Operation Rules:

[0085] Local Task: Use the Qi.pop_front() function to quickly obtain tasks from the head to maintain the efficiency of local execution.

[0086] Stolen Task: An idle thread randomly selects and steals tasks from the tail of the queue of other threads through the Qrandom.pop_back() function to avoid direct competition with local threads.

[0087] Scheduling Strategy and Load Balancing:

[0088] Dynamic Task Allocation: When the local queue is empty, the thread actively steals tasks from other queues to maximize the utilization of all processor cores; this strategy significantly improves parallel efficiency by reducing idle waiting time.

[0089] Load Balancing Formula:

[0090] T∞=max_critical_path+(∑W - ∑W_crit) / p;

[0091] Where, max_critical_path is the longest execution time of the critical path, which determines the lower limit of the final completion time of parallel tasks; ∑W is the total workload, representing the sum of the computational amounts of all tasks; ∑W_crit is the workload on the critical path; p is the number of processors.

[0092] II. Intelligent Desensitization Module: Desensitizes the text content converted by the multi-modal data conversion module to obtain the desensitized text content.

[0093] 1. The intelligent desensitization module also includes a custom rule interface for adding a regional sensitive reserved word library; a sensitive word recognition unit that analyzes and identifies sensitive words in the text content through context association.

[0094] 2. AI Mode: The context-sensitive word recognition unit based on the DeepSeek large model replaces the complete sensitive words using the algorithm of retaining the first letter in capital letters. Sensitive words include, but are not limited to, names in the text. For example, "Xiaoming" is replaced with "XM"; and the preset reserved word library ("aoid" folder, reserved word whitelist library) is called for desensitization of the text content. For example, "teacher" does not need to be replaced with "LS"; the preset reserved word library is implemented using the SQLite lightweight database, which supports multi-level permission management of white list entries; the DeepSeek large model has passed medical ethics review, and the training data has removed PHI and passed compliance certifications such as HIPAA or GDPR.

[0095] When calling the DeepSeek large model in AI mode, an API key is required. The API key is based on the challenge-response mechanism of asymmetric encryption; its core processes include identity identification transmission, dynamic challenge generation, digital signature calculation, and signature verification. The specific process of this part belongs to the prior art. In this embodiment, the DeepSeek large model is combined and used in the AI mode to achieve desensitization of the text.

[0096] The core algorithm and process verification of the above challenge-response mechanism of the API key are as follows:

[0097] Challenge-Response Mechanism of Asymmetric Encryption:

[0098] Key Pair Generation: The client and the server generate a public-private key pair based on the RSA algorithm. The private key (SK) is used for signing, and the public key (PK) is used for verification, which conforms to the key separation principle of asymmetric encryption; Challenge Generation: The server generates a 256-bit random number C as the challenge value, and its entropy value meets the security requirements (2 256 space), which can resist brute-force cracking attacks. Signing and Verification: The client calculates the signature σ = C d mod n, using the mathematical irreversibility of the private key to protect sensitive operations; the server verifies σ e ≡C modn, quickly verifying the validity of the signature through the public key to ensure the authenticity of the identity.

[0099] Scientificity: It meets the IND-CCA2 security model and is based on the factoring problem of large integers.

[0100] In this embodiment, the semantic understanding score based on the DeepSeek large model is a multi-dimensional and multi-level complex algorithm system. Its core technical architecture and performance indicators are as follows:

[0101] Semantic Encoding Layer (BERT Variant):

[0102] Transformer Improved with Dynamic Sparse Attention Mechanism:

[0103]

[0104] Among them, Q is the query matrix, K is the key matrix, V is the value matrix, and M is a learnable sparse mask matrix, which reduces the computational complexity while maintaining long-range dependencies; d k is the dimension of the key vector.

[0105] Scoring prediction module:

[0106] Multi-dimensional scoring head (5 independent sub-networks):

[0107]

[0108] Among them, Score i is the scoring output of the i-th sub-network. In the multi-dimensional scoring head, each sub-network outputs a score to evaluate one aspect of logical coherence, factual accuracy, language normality, intention matching degree, and emotional appropriateness; σ is the activation function, is the transpose of the weight matrix of the i-th sub-network; Pooling(h [CLS] is the result of pooling the vector of CLS; b i is the bias term of the i-th sub-network.

[0109] Consistency check:

[0110] Cross-dimensional check based on comparative literature, the consistency loss function is:

[0111]

[0112] Among them, e is the natural logarithm, e sim (f(x), f + (x)) / τ is the exponential function value after weighting the similarity sim(f(x), f + (x) with the temperature parameter τ; f(x) is the feature of the input x, f + (x) is the feature of the positive sample x, and the similarity is usually calculated using methods such as cosine similarity; the temperature parameter τ is used to adjust the sensitivity of the similarity; e sim (f(x), f - (x i )) is similar to e sim (f(x), f + (x)) / τ; f + is the enhanced positive sample, and f - is the negative sample.

[0113] The accuracy metrics include evaluation dimension, test set, Pearson correlation coefficient, F1 score:

[0114]

[0115] The model meets the following requirements during the encoding phase:

[0116]

[0117] Ensure that the encoding Z minimizes redundancy while retaining task-related information.

[0118] Dynamic weight allocation mechanism:

[0119]

[0120] Dynamically adjust the fusion weights according to the information entropy of each scoring dimension.

[0121] In the comparative experiment of ACL 2023, the present invention has been significantly improved compared with the traditional method:

[0122] The scoring consistency is improved by 32% (Krippendorff's α changes from 0.68 to 0.90);

[0123] The time-consuming of manual evaluation is reduced by 60% (Spearman ρ = 0.93 through automatic scoring and expert scoring).

[0124] 3. Rule pattern: Execute parallel replacement of single or multiple sensitive words through a regular expression engine. For example, replace "Xiaoming" with "XM" and "Xiaohong" with "XH"...

[0125] The regular expression engine adopts the Thompson NFA construction algorithm, including syntax parsing and state machine construction; the regular expression is decomposed into syntax units through a recursive descent parser, and a tree structure is constructed, combining AST parsing and NFA optimization techniques to balance performance and security.

[0126] The above optimization of the regular expression engine includes the following content:

[0127] Syntax parsing process: The regular expression (such as (a|b)*c) is decomposed into syntax units through a recursive descent parser, generating an abstract syntax tree (AST) in a tree structure. The AST node types include: atomic characters (such as a, b, c), selection operator (|), closure operator (*), and concatenation operator (implicit sequential concatenation).

[0128] State machine construction: Based on the Thompson construction method, map the AST to an NFA state machine, and for each sub-expression e, construct an NFA M(e).

[0129] Atomic character a: Create two states q0 and q1, and connect them through a character-labeled edge: q0 → a → q1;

[0130] Selection operator A|B: Merge the initial and accepting states of two sub-NFAs, and add ε-transition edges;

[0131] Closure operator A*: Add ε-transitions to the sub-NFA, initial state → sub-NFA entry, sub-NFA exit → closure end state, initial → end state.

[0132] Time complexity:

[0133] Construction phase: The number of NFA states grows linearly with the complexity of the regular expression. The construction time is O(m), O(mn) (m = pattern length, n = input length), where m is the length of the regular expression;

[0134] Matching phase: When matching a string of length n, the matching time complexity is O(mn), achieving linear complexity through ε-closure calculation and state set tracking.

[0135] Scientific nature: Thompson's algorithm eliminates the risk of exponential time complexity of traditional backtracking algorithms (such as catastrophic backtracking in the Perl regular engine) through ε-transitions and state set management; verified by ACM papers, for complex regular expressions (such as nested closures), Thompson NFA is 10 - 100 times faster than the backtracking algorithm in matching speed.

[0136] The collaborative processing process of the above AI mode and rule mode is as follows: First, enter the AI mode and set a whitelist for preliminary desensitization. When the confidence level of the AI mode is lower than the threshold, perform secondary verification in the rule mode, that is, when not all sensitive words are replaced, enter the rule mode for desensitization of the entire text content.

[0137] III. Dialogue analysis module: Split the desensitized text according to the entries of the BeDevel toolkit to form individual or independent documents.

[0138] 1. BeDevel toolkit: It is a localized toolkit applicable to the present invention; it includes the BeDevel Guideline, the Interactive Observation Manual BeDevel-P (BeDevel-Play protocol, BeDevel-P), the Parent Interview Manual BeDevel-I (BeDevel-Interview protocol, BeDevel-I), and the Education Handbook (BeDevel Handbook). Among them, BeDevel-P includes 32 activities or observation indicators for different age groups, such as Figure 2 As shown, BeDevel-I includes 21 interview entries for different age groups, such as Figure 3 As shown.

[0139] The scoring rules for BeDevel-P and BeDevel-I adopt a 1-3 point system. One point indicates typical development, and two points and three points indicate atypical development, which are respectively characterized by atypical or significantly restricted development features of different degrees. The age groups in this embodiment are set to five, namely 9-11 months, 12-17 months, 18-23 months, 24-35 months, and 36-42 months; the intelligent scoring module is built with an age segmentation execution engine, which automatically loads the evaluation rules of the corresponding BeDevel toolkit according to the different ages (months) of the subjects.

[0140] 2. Structured parsing algorithm: Split the text content into independent documents according to the interview items of BeDevel-I; specifically, the structured parsing algorithm is the part that can be achieved by existing technologies. It corresponds to the corresponding dialogue content in the text content according to the interview items of BeDevel-I and forms independent documents.

[0141] 3. Document output interface: Set to a dual-output mode, including item segmentation document output and full dialogue document output, that is, output the independent documents formed by the above structured parsing algorithm.

[0142] It should be noted that the dialogue analysis module also includes an extended interface according to actual needs, which is used to select other scales (such as ADOS or CARS).

[0143] IV. Intelligent scoring module: Score the dialogue documents output by the dialogue analysis module according to the scoring rules and output a scoring table.

[0144] 1. Dynamic standard loading unit: Used to adapt the scoring rules for different versions (different age groups) of BeDevel-I interview items; specifically, for the interview items or observation indicators corresponding to different age groups in the BeDevel toolkit, select the corresponding scoring rules, and give each interview item or observation indicator the corresponding score through the scoring rules.

[0145] 2. Semantic matching algorithm: Match based on the relevance between the text content converted from audio and video and the interview items of BeDevel-I, and score the BeDevel-I interview items corresponding to the dynamic standard loading unit; when the semantic matching confidence is less than the set threshold, the audio and video segment replay verification function is automatically triggered. The specific semantic matching algorithm is any algorithm in the prior art that can achieve the correspondence between text content and standard items, and give scores to each item. The threshold is set according to needs and is not limited to a certain fixed value. In this embodiment, the threshold is set to 85%.

[0146] 3. Standardized output interface, which is an Excel output interface, used to generate a scoring table that meets medical data specifications, including the scores of individual scale items and the total score statistics.

[0147] Through the above embodiments, the system of the present invention is disclosed. In the actual use process, it further includes a device or an APP or any other concept using the system designed according to the above system; specifically, the device includes a processor and a display screen. The processor deploys the above-mentioned multi-modal data conversion module, intelligent desensitization module, dialogue analysis module, and intelligent scoring module, and correspondingly displays corresponding function buttons on the display screen, including but not limited to Figure 7 the interface display as shown, which are file mode selection (single processing / batch processing), desensitization, transcribing to text, text replacement, document splitting, specifying standards, output, and status labels; the click events of all buttons are processed by the on_button_clicked() function, and the event click sources are distinguished by the sender() function. The status labels are the error mechanisms set according to the actual event operation process, such as: Error: No file to be processed or scoring criteria are selected; E1: The selected file path does not exist; E2: The selected scoring criteria cannot be opened or is not a word file; E3: The txt file storing the API cannot be opened or does not exist; E4: The selected dialogue file cannot be opened normally or is not a word file; E5: The Deepseek server is busy, just retry; E6: The audio-to-text server is busy or the number of requests is too many, please try again later; E7: There are non-audio / video files in the audio folder, or a certain file is damaged and cannot be opened; E8: Unable to connect to the Deepseek server, please check the network connection; E10: The input format of text replacement is incorrect.

[0148] Specifically, based on the above device, the processes of single file processing and batch file processing are described as follows:

[0149] Single file processing process:

[0150] Click "Single Processing" to select a single audio / video file → Click "Transcribe to Text" to transcribe the audio / video to text → Click "Desensitization" (optional) + "Text Replacement" (optional) to desensitize the text → Click "Document Splitting" to select the BeDevel-I interview items to output the split document → Click "Specify Standards" to select the scoring rules corresponding to the BeDevel-I interview items → Click "Output" to obtain the scoring table.

[0151] Batch processing process:

[0152] Click on "Batch Processing" to select multiple audio / video files → click on "Transcribe to Text" to transcribe text in sequence according to the batch processing queue management unit → click on "Desensitization" (optional) + click on "Text Replacement" (optional) to desensitize the text → click on "Document Splitting" to select BeDevel-I interview items and output the split documents → click on "Specify Criteria" to select the scoring rules corresponding to the BeDevel-I interview items → click on "Output" to obtain the batch scoring form.

[0153] In another embodiment of the present invention, an automated scoring method for early screening of autism based on artificial intelligence multi-modal data processing is disclosed. As Figures 4-6 shown, this method is based on the above system and includes the following steps:

[0154] The input audio and video data are converted into text content by the multi-modal data conversion module;

[0155] The intelligent desensitization module performs collaborative processing of the AI mode and the rule mode on the text content. When the confidence level of the AI mode is lower than the threshold, the rule mode is used for secondary verification to obtain the desensitized text. The final desensitization result ensures data consistency through hash verification; the threshold is set as needed and is not limited to a certain fixed value. In this embodiment, the threshold is set to 80%.

[0156] The dialogue analysis module splits the text according to the BeDevel-I interview items and outputs multiple documents or a single document;

[0157] The intelligent scoring module retrieves the scoring rules of the BeDevel-I interview items for the corresponding age group, scores according to the documents split by the dialogue analysis module, and outputs the scoring form and scoring results to the specified storage path.

[0158] Specifically, in this embodiment, the audio and video inputs are converted into raw text through an embedded ASR engine; the privacy protection is performed by replacing sensitive words using the capitalization retention algorithm and regular expressions, and the content that does not need to be desensitized in the preset retention word library is retained; according to the user's selection, the scoring rules of the BeDevel-I interview items for different age groups in the dynamic scoring standard library are loaded to perform semantic matching analysis; an Excel file containing the desensitized text of the original dialogue, the item splitting structure, and the scoring results is generated; based on the scoring results, the subjects are classified into high-risk, medium-risk, or low-risk autism.

[0159] In this embodiment, a single video evaluation instance is as follows: Select an MP4 file (child evaluation video), load a text file containing "kindergarten / teacher", set the replacement rule "Lele: LL / Tiantian: TT", select the ADOS standard, and the system outputs: desensitized text (retaining words such as "kindergarten / teacher") + complete dialogue document + Excel scoring sheet. The batch audio processing instance is as follows: Select a folder containing 50 WAV files, enable the AI desensitization mode, filter 2 non-video files, and process 48 files in parallel to generate a batch scoring sheet.

[0160] Through the above content, the system and method of the present invention are disclosed. Compared with the prior art, the present invention has the following advantages:

[0161] 1. The present invention combines the AI mode and the rule mode in the form of a hybrid desensitization strategy to balance the accuracy and flexibility of desensitization;

[0162] 2. When transcribing audio and video, the present invention adopts a multi-engine fault tolerance mechanism, which can automatically switch to the backup service to improve robustness;

[0163] 3. The present invention relies on local FFmpeg and custom modules to avoid cloud data leakage and improve data security;

[0164] 4. The present invention analyzes unstructured scoring criteria through the DeepSeek model, adapts to different scales, and is suitable for subjects of different ages, with a wide range of applications.

[0165] The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Any other modifications or equivalent replacements made by those of ordinary skill in the art to the technical solutions of the present invention shall be covered within the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solutions of the present invention.

Claims

1. An automated scoring system for early screening of autism based on artificial intelligence multimodal data processing, characterized by: The system is used for inputting single data or batch data, and outputting a scoring table and scoring results in a multimodal format of audio, video and text for the input data; the system includes: Multimodal data conversion module, which receives and processes audio and video input data and transcribes audio and video into text; it includes an embedded ASR engine, a localization processing architecture, and a batch processing queue management unit; Intelligent desensitization module, used for desensitizing text content; includes a dual-mode desensitization mechanism, namely AI mode and rule mode, which work together; The conversation analysis module is used to parse the BeDevel toolkit and segment the desensitized text content; it includes a structured parsing algorithm and a document output interface; The intelligent scoring module is used to generate scoring tables and scoring results that comply with medical data specifications based on the scoring rules of the BeDevel toolkit; it includes a dynamic standard loading unit, a semantic matching algorithm, and a standardized output interface. Based on the generated scoring tables and scoring results, manual verification is performed to determine the risk of autism in subjects.

2. The automated scoring system for early screening of autism based on artificial intelligence multimodal data processing according to claim 1, characterized in that: Embedded ASR engine supports speech recognition of children with non-standard pronunciation, including a children's speech feature optimization model, a noise suppression unit, and a multi-engine fault tolerance mechanism; the children's speech feature optimization model is used for training pronunciation ambiguity and grammatical errors; The noise suppression unit uses an adaptive filtering algorithm based on noise spectrum feature recognition to achieve dynamic noise filtering; the multi-engine fault tolerance mechanism automatically switches between JianYingASR, BcutASR, and KuaiShouASR speech recognition services when the confidence threshold is less than 0.7; Localized processing architecture for local storage and processing of audio and video data; Batch processing queue management unit supports parallel processing of multiple files and priority scheduling.

3. The automated scoring system for early screening of autism based on artificial intelligence multimodal data processing according to claim 1, characterized in that: The intelligent desensitization module also includes an interface for customizing rules, which is used to add regional sensitive reserved vocabulary. The sensitive word recognition unit analyzes and identifies sensitive words in text content through the context of the text content.

4. The automated scoring system for early screening of autism based on artificial intelligence multimodal data processing according to claim 3, characterized in that: AI mode, based on the context-sensitive word recognition unit of the DeepSeek large model, uses an algorithm that retains the capitalization of the first letter to replace complete sensitive words, including names in text; And call the preset reserved word library to desensitize the text content; Rule mode, which performs parallel replacement of single or multiple sensitive words through the regular expression engine.

5. The automated scoring system for early screening of autism based on artificial intelligence multimodal data processing according to claim 1, characterized in that: The BeDevel toolkit includes the BeDevel Guide, the Interaction Observation Manual BeDevel-P, the Parent Interview Manual BeDevel-I, and the Education Manual. BeDevel-P includes 32 activities or observation indicators for different age groups, and BeDevel-I includes 21 interview items for different age groups. The scoring rules for BeDevel-P and BeDevel-I use a 1-3 point system, where 1 point indicates typical development, and 2 and 3 points indicate atypical development, which respectively indicate different degrees of atypical or significantly restricted developmental characteristics.

6. The automated scoring system for early screening of autism based on artificial intelligence multimodal data processing according to claim 5, characterized in that: A structured parsing algorithm that splits text content into independent documents based on BeDevel-I interview entries; The document output interface is set to dual output mode, including entry segmentation document output and complete conversation document output.

7. The automated scoring system for early screening of autism based on artificial intelligence multimodal data processing according to claim 6, characterized in that: Dynamic standard loading unit, used to adapt the scoring rules of different versions of BeDevel-I interview items; A semantic matching algorithm matches the text content converted from audio and video with the relevance of BeDevel-I interview items, and scores the BeDevel-I interview items corresponding to the dynamic standard loading unit. When the semantic matching confidence level falls below the set threshold, the audio and video clip replay verification function is automatically triggered. The standardized output interface is an Excel output interface used to generate a scoring table that complies with medical data specifications.

8. An automated scoring method for early screening of autism based on artificial intelligence multimodal data processing according to any one of claims 1 to 7, characterized in that: The method comprises the following steps: Input audio and video data are converted into text content through the multimodal data conversion module; The intelligent desensitization module performs collaborative processing of the AI mode and the rule mode on the text content. When the confidence level of the AI mode is lower than the threshold, the rule mode is verified again to obtain the desensitized text. The final desensitization result is verified by hashing to ensure data consistency. The conversation analysis module segments the text according to the BeDevel-I interview items and outputs multiple documents or a single document; The intelligent scoring module retrieves the scoring rules of the BeDevel-I interview items corresponding to the age group, scores the documents according to the segmentation of the conversation analysis module, and outputs the scoring table and scoring results to the specified storage path.

9. The automated scoring method for early screening of autism based on artificial intelligence multimodal data processing according to claim 8, characterized in that: Convert audio and video input into raw text through the embedded ASR engine; Sensitive words are replaced with capitalization-preserving algorithms and regular expressions to protect privacy, and content that does not require desensitization is retained based on the preset reserved word library. According to the user's selection, the scoring rules for BeDevel-I interview items corresponding to different age groups in the dynamic scoring standard library are loaded and semantic matching analysis is performed; Generate an Excel file containing the desensitized text of the original conversation, the item segmentation structure, and the scoring results.

Images