A system for processing code-switching transcription

Abstract

A system for processing code-switching transcription, comprising: a self-trained automatic speech recognition (ASR) module configured to transcribe an audio signal being represented in at least a first language into an informal transcript in at least the first language; a translation module configured to selectively generate a formal translation in a second language based on the informal transcript in at least the first language; and a user interface configured to receive the audio signal in at least the first language and present the formal translation generated by the translation module in the second language.

Description

1 A SYSTEM FOR PROCESSING CODE-SWITCHING TRANSCRIPTION TECHNICAL FIELD The present invention relates to a system for processing code-switching transcription, and particularly, although not exclusively, to a system for processing code-switching transcription between a first language and a second language. 5 BACKGROUND In professional environments such as legal, governmental, and corporate sectors in regions like Hong Kong, transcription of Cantonese audio is a critical yet burdensome task. However, the transcription of Cantonese audio can be difficult due to the language’s 10 own complexity and uniqueness in its usage by Chinese speakers. Although Cantonese is a commonly spoken type of Chinese, its speakers from all over China, including the various Yuet provinces and Hong Kong SAR China, Macau SAR China and even overseas Chinese communities such as in Malaysia, Singapore, etc, each have their own style, choice of words or tonal adjustments. In turn, the transcription of Cantonese may be 15 difficult to correctly transcribe Cantonese in its daily usage. SUMMARY OF THE INVENTION In accordance with a first aspect of the present invention, there is provided a system for processing code-switching transcription, comprising: - a self-trained automatic speech recognition (ASR) module configured to transcribe an 20 audio signal being represented in at least a first language into an informal transcript in at least the first language; - a translation module configured to selectively generate a formal translation in a second language based on the informal transcript in at least the first language; and - a user interface configured to receive the audio signal in at least the first language 25 and present the formal translation generated by the translation module in the second language. In accordance with the first aspect, the automatic speech recognition (ASR) module is configured to transcribe Cantonese audio signal into Cantonese transcript and the 30 translation module is configured to translate the Cantonese transcript into written Chinese text. HK 30135233 A 2 In accordance with the first aspect, the automatic speech recognition (ASR) module further comprises: - an acoustic model configured to convert Cantonese audio signal into possible subword or phoneme sequences; and - a language model configured to handle the word sequences based on their likelihood 5 of occurring in natural Cantonese language. In accordance with the first aspect, the translation module further comprises a translation model configured to convert informal Cantonese transcript into formal written Chinese text. 10 In accordance with the first aspect, the automatic speech recognition (ASR) module is configured to selectively transcribe Cantonese portion of the audio signal into informal Cantonese transcript and English portion of the audio signal into English transcript and the translation module is configured to translate the informal Cantonese transcript and 15 English transcript into formal written Chinese text. In accordance with the first aspect, the audio signal includes indigenous Cantonese and the translation module is configured to translate indigenous Cantonese text into written Chinese text. 20 In accordance with the first aspect, the automatic speech recognition (ASR) module is pretrained and finetuned by one or more training strategies so as to improve the transcription performance. 25 In accordance with the first aspect, the translation module is pretrained and finetuned on large language models (LLMs) so as to improve overall translation performance. In accordance with the first aspect, the automatic speech recognition (ASR) module is pretrained by Cantonese audio-transcript data pairs in a closed loop configuration. 30 In accordance with the first aspect, the translation module is pretrained by cleansed Cantonese-Chinese bilingual corpus data in a closed loop configuration. In accordance with the first aspect, the automatic speech recognition (ASR) module is 35 progressively trained with one or more complex tasks or data by a curriculum learning training strategy. HK 30135233 A 3 In accordance with the first aspect, the automatic speech recognition (ASR) module is trained with a data augmentation training strategy and configured to: - perform data augmentation with a diversity of dataset; and, - perform synthetic modifications of environmental acoustics. 5 In accordance with the first aspect, the automatic speech recognition (ASR) module is trained by a regularization training strategy and configured to add a penalty to the loss function so as to prevent overfitting. In accordance with the first aspect, the automatic speech recognition (ASR) module is 10 trained with further strategies configured to: - adopt label smoothing to achieve better calibration; - dropout random neurons to reduce overfitting; and - perform weight noise injection to improve robustness. 15 In accordance with the first aspect, the translation module is finetuned in a two-stage approach: - a first model finetuning process on Cantonese syntax and part of speech (POS); and - a second model finetuning process onMachine Translation (MT) with sentence pairs. 20 In accordance with the first aspect, the translation module further comprises one or more additional large language models (LLMs) configured to perform further correction to written Chinese translation outputs. In accordance with the first aspect, the first model finetuning process is further 25 configured to improve performance by: - predict single-word syntax and part of speech (POS); - reorder syntax and part of speech (POS); - random mask the predicted part of speech (POS). 30 In accordance with the first aspect, the additional large language model (LLM) is further configured to correct one or more of the following: - potential language errors; and - grammatical mistakes, lexical errors, and syntactic irregularities. 35 In accordance with the first aspect, the pretraining data are collected and preprocessed by a preprocessing module. HK 30135233 A 4 In accordance with the first aspect, the user interface is configured to split the Cantonese audio signal into a plurality of segments and display simultaneously the Cantonese transcript and the written Chinese text corresponding to each segment. BRIEF DESCRIPTION OF THE DRAWINGS 5 Embodiments of the present invention will now be described, by way of example, with reference to the accompanying drawings in which: FIG. 1 is a high-level architectural block diagram of the Indigenous Cantonese-led Transcription & Translation Engine in accordance with an embodiment of the present invention. 10 FIG. 2 is a flowchart of the Automatic Speech Recognition (ASR) Training and Deployment Lifecycle in accordance with an embodiment of the present invention. FIG. 3 is a comparative table of code-switching transcription performance in accordance with an embodiment of the present invention. FIG. 4 is a comparative table of professional terminology transcription in accordance 15 with an embodiment of the present invention. FIG. 5 is a diagram illustrating the multi-stage LLM fine-tuning process in accordance with an embodiment of the present invention. FIG. 6 is a diagram of the syntax and part-of-speech (POS) tagging flow in accordance with an embodiment of the present invention. 20 FIG. 7 is a diagram illustrating the iterative correction process for outputs in accordance with an embodiment of the present invention. FIG. 8 is a flowchart of the Translation Model Training and Deployment Lifecycle in accordance with an embodiment of the present invention. FIG. 9 is a comparative table of informal-to-formal translation performance in 25 accordance with an embodiment of the present invention. FIG. 10 is a comparative table of technical term translation performance in accordance with an embodiment of the present invention. FIG. 11 is a user interface diagram showing the end-to-end operational workflow in accordance with an embodiment of the present invention. 30 HK 30135233 A 5 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Without wishing to be bound by theory, the inventors have discovered several critical pain points in the field of speech processing, primarily that traditional manual transcription is tedious, time-consuming, and cost-prohibitive. Furthermore, existing automated services, such as cloud-based Automatic Speech Recognition (ASR), 5 present significant confidentiality risks by requiring the transmission of sensitive audio data to external servers. Beyond operational hurdles, Cantonese presents unique linguistic challenges, specifically "code-switching"—the interspersion of English terms within Cantonese syntax, such as "你 mind 唔 mind" (meaning "would you mind")—and the use of 10 localized colloquialisms like "睇返轉頭" (meaning "retrospect" or "review") that do not map directly to formal written Chinese. Consequently, existing ASR and translation engines often fail to accurately capture these nuances or convert them into professional, formal text. Referring to Figure 1, there is shown an embodiment of a system 100 for processing 15 code-switching transcription. The system 100 comprises a self-trained automatic speech recognition (ASR) module 120 configured to transcribe an audio signal being represented in at least a first language into an informal transcript in at least the first language. The system 100 further comprises a translation module 130 configured to selectively generate a formal translation in a second language based on the informal 20 transcript in at least the first language. The system further comprises a user interface 140 configured to receive the audio signal in at least the first language and present the formal translation generated by the translation module in the second language. For the purposes of this document, the term “language” may include any type and 25 variations of spoken or written languages, such as, but not limited to, spoken Chinese such as Mandarin, Cantonese and written Chinese such as Traditional Chinese and Simplified Chinese. The term “first language” may include spoken Cantonese while the term “second language” may include written Chinese. The “at least a first language” may include a combination of two or more languages such as indigenous Cantonese 30 and English. Figure 1 illustrates the overall architecture of the system 100 for processing code- switching transcription. The system 100 includes an Indigenous Cantonese-led Transcription & Translation Engine 110 and a User Interface 140. The Indigenous 35 Cantonese-led Transcription & Translation Engine 110 acts as a bridge between raw HK 30135233 A 6 audio input and polished documentation and includes a self-trained Cantonese-English code-switching ASR module 120 and an Informal Cantonese to Formal Written Chinese Translator module 130. The User Interface 140 manages the ingestion of audio files from a User 10, typically in formats like .mp3, .wav, or .mp4. These files are processed by the self-trained Cantonese-English code-switching ASR module 120, which uniquely 5 treats Cantonese and English inserts as a singular linguistic stream. Once transcribed, the text is passed to the Informal Cantonese to Formal Written Chinese Translator module 130, which employs a specialized Translation Model 132. This multi-layered approach ensures the Transcription 102 phase captures every verbatim detail, while the Translation 104 phase applies the necessary formalization logic. 10 The system architecture is centered around the Engine 110 and the User Interface 140, which mediates between the User 10 and core processing components. Within the ASR module 120, there is provided an Acoustic Model 122 which may convert raw Cantonese audio signal into possible word or phoneme sequences and a Language 15 Model 124 which may rank the sequences based on their likelihood of occurring in natural Cantonese language. The Acoustic Model 122 and the Language Model 124 work in parallel to decode speech signals. Preferably, the Acoustic Model 122 is trained on indigenous Cantonese audio to 20 recognize unique tonal qualities and naturally pronounced English loanwords (e.g., "bylaws," "proposal"), allowing it to differentiate English code-switched terms from homophonic Cantonese sounds. The Language Model 124 works in tandem to predict word sequences based on a specialized Cantonese-English corpus, ensuring the output follows natural code-switching grammar. 25 During the Transcription 102 phase, the ASR module 120 converts the raw audio signal into "informal" Cantonese text. This verbatim output includes indigenous particles (e.g., "呢" (ni1),"呀" (aa3), "嘅" (ge3)) and embedded English terms, forming the basis for the subsequent stage. The Translation 104 phase, governed by the Translator 30 module 130, performs a structural and stylistic transformation rather than simple word replacement. The Translation Model 132 maps informal spoken syntax to formal Standard Written Chinese structures, replacing Cantonese markers like "嘅" (’s) and " 話 " (said) with formal equivalents like "的 " (‘s) and "説 " (said) while correctly integrating English terms. 35 The synergy between the ASR module 120 and Translator module 130 defines the Engine 110. By separating localized transcription from specialized stylistic translation, HK 30135233 A 7 the Engine 110 achieves high accuracy in formal output. The final result is presented to the User 10 via the User Interface 140, typically in a side-by-side view. This allows the User 10 to verify the verbatim transcript against the formal translation, combining high-speed Large Language Model (LLM) processing with human oversight for mission- critical documents like meeting minutes or technical reports. 5 Preferably, the ASR module 120 serves as the primary engine for converting indigenous Cantonese speech into a high-fidelity verbatim transcript. To optimize transcription performance, the module applies advanced training strategies including curriculum learning, which trains the model on progressively complex data, and data 10 augmentation to increase dataset diversity through synthetic acoustic modifications like reverberation and background interference. Regularization techniques, such as label smoothing and dropout, are further utilized to prevent overfitting by adding penalties to the loss function. 15 Figure 2 details the ASR system 120 lifecycle, featuring a closed-loop architecture designed to handle the variability of indigenous Cantonese and English code-switching. The process is divided into three stages: Data Preparation, Model Training, and Operational Deployment. It begins with Data Collection 212, where Cantonese audio- transcript pairs 216 are gathered from local professional environments like legal 20 proceedings and seminars. These pairs are specifically curated to include informal Cantonese particles and technical English terms, ensuring the models are sensitive to mixed-mode speech. Before training, raw data enters the Pre-processing engine 214 for audio normalization, 25 text tokenization, and phonetic alignment. During the Training phase 217, the Acoustic Model 122 and Language Model 124 are developed concurrently. The Acoustic Model 122 utilizes deep neural networks to map audio signals to Cantonese phonemes and English loanword profiles, while the Language Model 124 is tuned to the transition probabilities of switching between the two languages. Following training, Model 30 Evaluation 218 measures accuracy using Mixture Error Rate (MER) against a unseen test set to ensure precision before deployment. A key innovation of the closed-loop architecture is the (Re) Training loop 219, which creates a self-improving system. As the deployed ASR system 120 processes live 35 Cantonese audio 222 to generate Transcribed Cantonese text 224, the outputs and subsequent human corrections are cycled back into the Training phase 217. This recursive loop allows the Engine 110 to adapt to emerging professional terminology HK 30135233 A 8 and linguistic shifts without structural redesign. By maintaining this closed-loop lifecycle, the system achieves a sustainable accuracy advantage in the specialized domain of Cantonese-led professional communication. In some example experiments as performed by the inventors by use of other ASR 5 systems or other models, it was observed that several existing state-of-the-art systems produced less optimal results when tasked with professional domain recognition and code-switching. Examples of these results are shown in Figure 3, which illustrates a comparative performance matrix 300 that provides empirical evidence of the technical superiority of the present invention in processing indigenous Cantonese-led speech 10 containing English code-switching. In one example embodiment, a test case involving an original transcript 310 of a representative sentence: "平時生活上嘅一啲方案呢變成一啲原型，一啲 prototype." (Translation: "Some solutions in daily life become some prototypes, some 15 prototypes.") This sentence presents a specific "Linguistic Dissonance" challenge: the speaker uses the formal Cantonese term for prototype ("原型 ") 312 followed immediately by the English equivalent ("prototype") 314 as a clarifying code-switch. The matrix 300 highlights instances where different models when used as part of the 20 example experiment as performed by the inventors demonstrated less accuracy in localized contexts. For example, in the provided test set, Google Cloud 320 was observed to transcribe the final word as "prototy," which may indicate that the transition probability weights used in its language model were less optimal for this specific code-switch. In these tests, Microsoft Azure 330 appeared to phonetically 25 represent the Cantonese "原型" (prototype) as "圓形" (round shape); however, these words do not maintain the accurate meaning when considered literally within the semantic context of "solutions". Furthermore, it was noted that in this example experiment, Aliyun Speech 340 substituted "prototype" with "production," possibly reflecting a bias toward high-frequency general terms rather than specific professional 30 terminology. FanoLabs 350 was observed in this experiment to demonstrate a less optimal performance by producing phonetic approximations such as "鋪到題" (bou1 dou3 tai4). While the model 350 appears to have tried to phonetically represent the English acoustic signals, the resulting strings of Cantonese characters do not appear to present an accurate meaning. Similarly, Cantonese.ai 360 produced a less accurate 35 result in the inventor’s experiments by transcribing the final word as "polity". HK 30135233 A 9 In contrast, the LSCM-LLM 380 of the present invention is the only models that successfully match the original transcript 310. The LSCM-LLM 380 accurately transcribes both the Cantonese segments and the English term "prototype" by utilizing the specialized Acoustic Model 122 and Language Model 124. By specifically training on indigenous phonetic shifts, the LSCM-LLM 380 avoids the phonetic substitution 5 errors seen in competitive models. This performance is achieved through the Syntax and POS fine-tuning stage, which identifies the Cantonese particles (like "嘅" (ge3) and "呢" (ni1)) and anticipates the technical noun that follows. By treating the code-switch as a single, coherent linguistic 10 stream, the engine—particularly in the LSCM-LLM 380 version—avoids "Inter- sentential Phonetic Interference," the phenomenon where the trailing sounds of a Cantonese particle distort the start of an English word. This high-fidelity transcription serves as the essential "Ground Truth" foundation required for the subsequent formal translation phase. 15 In Hong Kong’s professional landscape, speakers frequently use English nouns for governance, legal roles, and administrative departments while maintaining Cantonese as the primary syntactic framework. 20 Figure 4 illustrates a comparative performance matrix 400 specifically designed to evaluate the engine's robustness when processing indigenous Cantonese-led speech that incorporates dense clusters of specialized legal and professional English terminology. 25 In one example embodiment, the scenario presented in Figure 4 involves a complex sentence 410: "你哋個 bylaws呢，所有你哋啲 membership呀、general counsel呀、 accounts呀、professional development，其實全部喺個 bylaws嗰到嘅" (Translation: "Regarding your bylaws, all your membership, general counsel, accounts, professional development, are actually all in those bylaws.") 30 The matrix 400 contrasts the original transcript 410 with existing state-of-the-art systems, revealing instances of reduced accuracy in professional terminology as observed in the inventor’s own testing and experiments. For example, Google Cloud 420 appeared to substitute the term "bylaws" with "file" or the phonetic loanword "35 快勞," suggesting that the variables used may have been biased toward general office vocabulary. While Microsoft Azure 430 was observed to capture several general terms, it produced near-miss phonetic errors such as transcribing "general counsel" as HK 30135233 A 10 "general council," which may lack the precision required for formal records. FanoLabs 440 similarly showed less accuracy in this corporate context, transcribing "bylaws" as "big lass" and "general counsel" as "general countial," which may indicate a need for more specialized domain-specific acoustic training. 5 In contrast, the embodiments of the present invention provide a technical solution to domain-specific code-switching interference by accurately capturing the sequence of professional terms including "membership" 412, "general counsel" 413, "accounts" 414, "professional development" 415, and the concluding reference to "bylaws" 416. The LSCM-neural 450 of the present invention successfully identifies the technical 10 density of these five English terms by utilizing the self-training methodology described in Figure 2. It maintains acoustic integrity even when English terms are immediately preceded or followed by Cantonese "filler" particles 417, 418, such as "呢" (ni1) and " 呀" (ge3), which typically confuse the phonetic boundaries for non-indigenous models. 15 The LSCM-LLM 460 of the present invention demonstrates near-perfect alignment with the ground truth. It correctly identifies the technical terms and avoids the phonetic traps of the existing state-of-the-art systems because its Language Model 124 assigns a significantly higher probability to "bylaws" 411, 416 when terms like "general counsel" 413 and "membership" 412 are detected in the same segment. Furthermore, 20 the system includes a (Re) Training loop 219 as part of its wider lifecycle, ensuring that the model remains updated with emerging professional jargon. By precisely preserving the indigenous particles 417, 418, the LSCM-LLM 460 provides the necessary stylistic cues for the subsequent translation phase. Thus, the present invention ensures a higher accuracy in technical noun recognition within a complex, code-switched 25 linguistic environment. Preferably, the Translator Module 130 leverages the inherent language capabilities of Large Language Models (LLMs) to improve the overall fine-tuned pipeline through a multi-stage linguistic transformation process. 30 Figure 5 illustrates a specialized tripartite fine-tuning architecture for the Large Language Models (LLMs) that power the engine's translation component. Unlike traditional systems that rely on generic mapping, this multi-stage strategy addresses the profound structural disparities between spoken Cantonese and formal written 35 Chinese. The process is divided into three distinct stages: Syntax / POS fine-tuning 510, MT fine-tuning 520, and post-processing correction 530, ensuring that the linguistic HK 30135233 A 11 nuances and stylistic requirements of professional documentation are met with technical precision. The first stage 510 focuses on "Syntax and POS Fine-Tuning" to master the skeletal structure of the dialect. Spoken Cantonese utilizes unique sentence-final particles and 5 aspect markers that lack direct mappings in formal text. In this stage, a first LLM 512 is trained on an indigenous dataset annotated with part-of-speech tags and syntactic trees. By mastering these hierarchical relationships first, the model 512 avoids the pitfall of literal translation—where vocabulary is correct but the word order remains "broken" or informal. This provides a stable foundation for the complex semantic 10 transformations required in subsequent stages. The second stage 520 involves "Machine Translation (MT) Fine-Tuning," transitioning the system from structural understanding to semantic mapping. The same LLM 512 is fine-tuned using a high-quality, cleansed bilingual corpus consisting of "informal-to-15 formal" sentence pairs. During this phase, the model 512 's weights are adjusted to recognize indigenous lexical patterns and map them to standardized formal equivalents. This stage is critical for maintaining the integrity of code-switched English technical terms while simultaneously transforming the surrounding Cantonese syntax into a professional Chinese register. 20 The final stage 530 introduces "Additional LLMs for Correction" to solve the problem of stylistic inconsistencies that often plague single-pass translations. This stage employs a generator-evaluator framework. The same finetuned LLM 512 produces a candidate formal translation, which is then scrutinized by the Evaluator / Corrector 25 LLMs 534. This secondary model 534 is fine-tuned specifically for error detection in formal Chinese, reviewing the output for missing punctuation—such as the distinctive enumeration comma ("、")—and identifying residual informal particles. If an error is detected, the system initiates an iterative refinement loop until a high-confidence "Formal Professional Standard" is achieved. This decoupled pipeline ensures that the 30 final output is vetted for syntactic correctness and semantic accuracy, making it suitable for mission-critical legal or corporate archives. Preferably, the syntax and Part-of-Speech (POS) fine-tuning stage 510 may further comprise three distinct prediction tasks executed in a single fine-tuning step: single-35 word syntax and POS prediction, a syntax and POS reordering task, and a random masking POS prediction task. HK 30135233 A 12 Figure 6 illustrates the internal linguistic processing logic of the engine, specifically the Part-of-Speech (POS) tagging and syntactic dependency parsing applied to informal Cantonese across three distinct scenarios. This phase represents the operational execution of the Stage 1 Fine-Tuning, decomposing spoken Cantonese into a structured map that allows the Large Language Model (LLM) to perform accurate 5 transformations into formal Chinese while preserving the interrogative, imperative, and conditional nuances of the speaker. By labeling indigenous markers within a dependency tree architecture, the system guides the translation engine through structures that standard toolkits often misidentify. 10 In one example embodiment, the first panel 610 analyzes a compound input involving aspectual and interrogative parsing: "噉我去咗邊呀? 快啲做功課啦。" (Translation: So where did I go? Hurry up and do the homework.) The LLM 612 identifies "噉" (gam2) as a discourse marker and specifically tags "咗" (zo2) as an aspectual auxiliary marker (AUX) indicating completed action. The terminal particle "呀" (aa3) is tagged as a 15 question particle. In the second clause, the LLM 612 identifies the adverbial "快啲" (faai3 di1) modifying the verb "做" (zo6), while "啦" (laa1) is tagged as an imperative discourse particle. This precise tagging ensures the translation reflects a past-tense question rather than a present-tense statement, maintaining the speaker's original temporal intent. 20 In one example embodiment, the second panel 620 demonstrates imperative and possessive parsing through the phrase "刷你嘅牙啦!" (Translation: Brush your teeth!). The LLM 622 identifies "刷 " (caat3) as the root verb and labels the indigenous possessive particle "嘅" (ge3) as a possessive marker (PART) linked to the pronoun "你25 " (nei5). The sentence-final particle "啦" (laa1) is labeled as a discourse particle. This structural mapping ensures the translation stage filters out informal particles in favor of formal syntactic structures while maintaining the "command" nature of the sentence. 30 In one example embodiment, the third panel 630 analyzes a complex conditional and modal statement: "我畀都未必係最正確嘅。" (Translation: Even if I give it, it might not be the most correct.) The LLM 632 identifies the particle "都 " (dou1) as a concessive marker, linking the act of "give" (畀) to the subsequent negation. The system tags "未必" (mei6 bit1) as an adverbial modifier of the copula "係" (hai6) to 35 indicate uncertainty, and identifies the superlative "最" (zeoi3) modifying the adjective "正確" (zing3 kok3). By identifying these specific modal tags, the engine transforms the input into a formal Chinese equivalent that preserves the original concessive tone. HK 30135233 A 13 Preferably, the post-processing correction stage 530 may utilize additional Large Language Models (LLMs) to identify and correct potential language errors, including grammatical mistakes, lexical errors, and syntactic irregularities. 5 Figure 7 illustrates the structural refinement and iterative correction pipeline 700 of the present invention. This stage addresses the challenge of "syntactic sparsity," where native Cantonese speakers often use context-heavy shorthand or omit specific verbs and formal punctuation that are strictly required in professional Written Chinese. While verbatim transcription captures spoken intent, the resulting text often appears 10 "broken" in a formal context. Figure 7 details how the system identifies these gaps and uses a recursive Large Language Model (LLM) loop to inject necessary grammatical components, ensuring the output meets the "Formal Professional Standard" required for legal and corporate documentation. 15 The architecture relies on a "Generator-Evaluator" model pairing. The Generator 712 produces an initial candidate translation based on the raw Cantonese transcript. The Evaluator 722, a specialized LLM fine-tuned on a corpus of informal-versus-formal Chinese pairs, then scans this output for "informal leaks"—residual particles that were not converted—and for syntactic omissions. As depicted in the upper panel 710 on the 20 left, the system processes a draft translation that reflects common spoken shorthand: "我喜歡和咖啡茶和果汁" (Translation: I like and coffee tea and juice). The Evaluator 722 identifies a critical "semantic gap" between the preference verb "喜歡" (like) and the objects "咖啡" (coffee), "茶" (tea), and "果汁" (juice). In formal Written Chinese, the specific consumption verb "喝" (to drink) is required for clarity even if omitted in 25 speech. As shown in the right panel 720, the Evaluator 722 triggers an iterative pass to categorize the nouns. Recognizing the objects as beverages, it selects the appropriate formal verb "喝 " and determines its correct placement following the predicate. 30 Simultaneously, the system performs Punctuation and Conjunction Optimization. It identifies that the repeated use of the conjunction "和" (and) is a stylistic carry-over from spoken patterns. In formal Chinese, the distinctive enumeration comma ("、") must be used to separate items in a list. Through this iterative correction, the model produces the refined output: "我喜歡喝咖啡、茶和果汁" (Translation: I like drinking 35 coffee, tea, and juice). HK 30135233 A 14 As depicted in the lower panel 730 on the left, the final result is a complete reconstruction into a professional standard that follows strict administrative stylistic rules. This mechanism provides a technical solution to the "translation fidelity" problem that plagues generic, single-pass LLMs, which often sound "unnatural" to native readers. By targeting the nuances of indigenous Cantonese shorthand, the 5 invention produces text indistinguishable from a professional human translation. This is particularly crucial for code-switching; the pipeline ensures that English technical terms are either translated or integrated into a perfectly fluid formal Chinese sentence structure, ensuring mission-critical documents are ready for official filing. 10 Figure 8 illustrates the comprehensive technical workflow and lifecycle 800 for the "Informal Cantonese to Formal Written Chinese Translator 130." Distinct from the acoustic focus of the ASR module 120 lifecycle, this architecture focuses on the semantic mapping and stylistic transformation required to bridge the gap between indigenous spoken dialects and professional written standards. The process originates 15 at the Data Collection stage 812, where the system aggregates a specialized Cleansed Bilingual Corpus 816. This corpus serves as the engine's primary knowledge base, consisting of parallel sentence pairs where raw Cantonese text—containing dialect- specific particles like "嘅" (ge3) and "喺"(hai2) —is mapped to equivalent formal Written Chinese. 20 Before ingestion, a Pre-processing engine 814 cleanses the data by standardizing character encodings and ensuring professional terminology is consistently mapped. During the Training phase 817, the Translation Model 132 undergoes supervised fine- tuning to minimize the "stylistic distance" between informal inputs and formal outputs. 25 A unique aspect of this training is the "Code-Switching Preservation" logic, which teaches the model to treat English professional terms as "anchors" that remain intact unless a high-quality formal equivalent is required. Once training is complete, the system moves to Model Evaluation 818, utilizing BLEU (Bilingual Evaluation Understudy) scores and human-in-the-loop assessments to ensure the absence of 30 "informal leakage." Upon passing evaluation thresholds, the system moves to the Deployment phase. In its operational state, the Translation module 820 receives Cantonese text 822 from the ASR subsystem 120 and utilizes the deployed Translation model 132 to produce the 35 final formal Written Chinese text 824. This output is stylistically indistinguishable from text originally drafted in formal Chinese, effectively erasing informal markers while preserving 100% of the speaker’s informational content. This level of refinement is HK 30135233 A 15 essential for generating archives and reports that must withstand scrutiny in legal and administrative environments. Similar to the ASR system 120, Figure 8 features a critical (Re) Training loop 819. As users or human-in-the-loop editors refine the formal Chinese output to fit specific 5 professional contexts, these corrected pairs are captured and cycled back into the Training phase 817. This recursive feedback ensures the Translation Model 132 evolves alongside the language, allowing it to learn new government terms or administrative jargon within a few training cycles. By treating the transformation as a dedicated translation problem rather than a simple character conversion, the invention 10 successfully solves the problem of "dialectal interference" in documentation. In our experiments, it was also observed that several existing state-of-the-art systems produced results that were less than optimal when handling localized administrative contexts. 15 Figure 9 illustrates a comparative performance matrix 900 that evaluates the translation accuracy of the present invention against global benchmark services when processing localized Hong Kong Cantonese. This case study focuses on the transformation of indigenous slang and culturally specific terminology into formal 20 written Chinese. In one example embodiment, the test scenario 910 involves a sentence regarding a local administrative context: "佢尋日就話買外賣都要嘟「安心出行」QR圖碼，但 食環署深夜就出新聞稿話唔用都可以接受。" (Translation: Yesterday he said 25 scanning the "LeaveHomeSafe" QR code was required for takeout, but the FEHD issued a press release saying it's optional.) This sentence presents three distinct technical challenges: the onomatopoeic indigenous verb "嘟 " (duk1) 911; the localized administrative mobile application name "安心出行" (LeaveHomeSafe) 912; and the requirement to convert spoken structures like "就話" (said) 913 and "就出" (issued) 30 914 into formal prose. The matrix 900 highlights instances where certain models demonstrated less accuracy in the experiments and tests performed by the inventors. For example, Baidu Translator 920 appeared to phonetically or literally represent " 安 心 出 行 " 35 (LeaveHomeSafe) as "好心出走" (kindly run away). While the model 920 appears to have tried to represent the characters, the resulting phrase does not maintain the HK 30135233 A 16 accurate meaning when considered literally in the context of the official mobile application. Furthermore, in our tests, Google Translate 930 and Microsoft Bing 940, while identifying the entity correctly, appeared to demonstrate less than optimal 5 performance in elevating the prose to a professional register. Google Translate 930 retained the informal "出新聞稿" (issue a press release) rather than utilizing the formal "發出" (issued), which may indicate that the variables used were less optimal for generating high-register documentation. These observations suggest that such models may produce "translationese" that lacks the professional tone required for 10 formal reporting in a Hong Kong context. In contrast, the embodiments of the present invention demonstrate a sophisticated understanding of indigenous context. The LSCM-neural 950 successfully bridges the gap between the onomatopoeic "嘟" (duk1) and the formal context of scanning. The 15 LSCM-LLM 960 provides the most polished result, replacing spoken particles with formal counterparts and transforming the informal "話" (said) into the formal "説" (said). It correctly utilizes the formal verb "發出 " (issued) and reconstructs the sentence flow to match the original transcript 910 in both meaning and professional tone. 20 Figure 9 proves that the invention provides a technical solution to the "dialect-to- formal-standard mapping" problem. Unlike generic models that treat Cantonese as a variation of Mandarin, the present invention—powered by the specialized fine-tuning and the Cleansed Bilingual Corpus 816—treats it as a distinct source language. By 25 performing "syntactic smoothing" and protecting administrative entities from literal translation errors, the system ensures high-fidelity output for legal, governmental, and corporate sectors where informal transcripts are unacceptable. This indigenous intelligence allows the engine to generate professional Chinese text that is ready for official publication without manual editing. 30 Figure 10 illustrates a comparative performance matrix 1000 evaluating the engine's capability to handle professional English code-switching. In administrative environments, speakers frequently embed English nouns and adjectives within a Cantonese syntactic frame. A major technical failure of existing systems is the inability 35 to determine whether to translate these terms or how to grammatically integrate them into formal Chinese. HK 30135233 A 17 In one example embodiment, the scenario 1010 involves a representative sentence: " 伍先生個 proposal 聽落幾 creative，但馮先生話唔符合市場預期。" (Translation: Mr. Ng's proposal sounds very creative, but Mr. Fung said it didn't meet market expectations). This sentence contains three specific challenges: the technical noun "proposal" 1011; the adjective "creative" 1012; and indigenous particles such as the 5 possessive "個" (‘s) 1013 and the descriptive "聽落幾" (sounds very) 1014. The matrix 1000 further contrasts the original transcript 1010 with existing state-of- the-art systems, revealing instances of reduced accuracy in professional English code- switching. For example, Baidu Translator 1020 demonstrated less accuracy by retaining 10 the English terms "proposal" and "creative" verbatim while using literal character substitution for the surrounding Cantonese. This model 1020 appears to have tried to represent the input, but the resulting output may lack the necessary formalization for professional documentation. 15 Similarly, in tests performed by the inventors, Google Translate 1030 was observed to produce a less than optimal result by not translating the English code-switched terms. Although it correctly identified the formal possessive "個"(‘s), the continued inclusion of English terms creates a "hybrid" text that lacks a professional polish. Microsoft Bing 1040 demonstrated a more refined performance by attempting to translate the terms 20 into "提案 " (proposal) and "創意 " (creative); however, the resulting sentence structure appeared to remain somewhat rigid in comparison to the original intent. These findings suggest that the variables used in these models may be less optimal for the seamless integration of mixed-language inputs into high-register Chinese. 25 The matrix 1000 reveals a lack of specialized code-switching logic in market-leading solutions. Baidu Translator 1020 fails by retaining "proposal" and "creative" verbatim while using literal character substitution, resulting in an output unsuitable for formal documentation. Google Translate 1030 similarly fails to translate the English terms; although it identifies the formal possessive "個" (‘s), the inclusion of English terms 30 creates a "hybrid" text that lacks professional polish. Microsoft Bing 1040 demonstrates higher intelligence by attempting to translate the terms into "提案" (proposal) and "創意" (creative), but the resulting structure often remains rigid. The embodiments of the present invention provide a technical solution to "code-35 switching dissonance." The LSCM-neural 1050 successfully bridges the linguistic gap by identifying the English terms within the Cantonese syntactic frame and providing an accurate translation that preserves the speaker's intent. Building on this, the LSCM- HK 30135233 A 18 LLM 1060 provides the most polished result: "伍先生的提案聽起來很有創意." Unlike baseline models, it identifies that in a professional context, "proposal" and "creative" should be translated as "提案" and "有創意." This is achieved through the MT Fine- Tuning stage described in Figure 5. The system replaces the spoken possessive "個" (‘s) 1013 with "的" (‘s) and transforms the informal "聽落幾" (sounds very) 1014 into the 5 formal "聽起來很有" (sounds very). Figure 10 proves the invention provides a professional-grade solution for mixed- language inputs. By treating English terms as part of a single semantic unit, the LSCM- neural 1050 and LSCM-LLM 1060 produce a unified formal output essential for 10 generating executive summaries or legal minutes. Additionally, the User Interface 140 allows the User 10 to toggle a "Preserve English Terms" mode, which would output " 伍先生的 proposal 聽起來很有創意" (Translation: Mr. Ng's proposal sounds very creative). This dual-mode capability ensures the system is adaptable to the varied professional standards of the Hong Kong business sector, ensuring mission-critical 15 documents are ready for official filing. Figure 11 illustrates the end-to-end operational sequence and method 1100 of use for the Indigenous Cantonese-led Transcription & Translation Engine 110. The workflow begins with Step 1110: File upload, where the user selects a multimedia file in formats 20 such as .mp3, .wav, or .mp4 for processing. Once the upload is initiated, the system proceeds to Step 1120: Transcribe audio, which activates the self-trained ASR engine 120 to begin decoding the speech signals. Throughout this phase, the Step 1130: Upload progress bar provides a real-time status update of the data ingestion and preliminary segmentation. 25 Upon completion of the ASR decoding, the interface populates Step 1140: Audio split and transcription results displayed area. This section provides a time-stamped, segmented view of the verbatim "informal" Cantonese transcript, allowing the user to review the acoustic accuracy of technical terms and indigenous particles. The user then 30 triggers the formalization phase by selecting Step 1150: Translate to formal Chinese, which utilizes the multi-stage LLM pipeline to transform the spoken dialect into a professional written standard. The resulting text is shown in Step 1160: Formal Chinese translation results displayed, 35 featuring a side-by-side comparison where the left pane holds the original transcript and the right pane displays the refined, formal Written Chinese. Finally, the user HK 30135233 A 19 completes the process at Step 1170: Export result to file, generating a professional document in formats like .txt or .docx that is ready for corporate or legal filing. It will be appreciated by persons skilled in the art that numerous variations and / or modifications may be made to the invention as shown in the specific embodiments 5 without departing from the spirit or scope of the invention as broadly described. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive. Any reference to prior art contained herein is not to be taken as an admission that the information is common general knowledge, unless otherwise indicated. 10 HK 30135233 A CLAIMS 1. A system for processing code-switching transcription, comprising: - a self-trained automatic speech recognition (ASR) module configured to transcribe an audio signal being represented in at least a first language into an informal transcript in at least the first language;5 - a translation module configured to selectively generate a formal translation in a second language based on the informal transcript in at least the first language; and - a user interface configured to receive the audio signal in at least the first language and present the formal translation generated by the translation module in the second language. 10 2. A system for processing code-switching transcription in accordance with claim 1, wherein the automatic speech recognition (ASR) module is configured to transcribe Cantonese audio signal into Cantonese transcript and the translation module is configured to translate the Cantonese transcript into written Chinese text. 15 3. A system for processing code-switching transcription in accordance with claim 2, wherein the automatic speech recognition (ASR) module further comprises: - an acoustic model configured to convert Cantonese audio signal into possible subword or phoneme sequences; and 20 - a language model configured to handle the word sequences based on their likelihood of occurring in natural Cantonese language. 4. A system for processing code-switching transcription in accordance with claim 2, wherein the translation module further comprises a translation model configured to 25 convert informal Cantonese transcript into formal written Chinese text. 5. A system for processing code-switching transcription in accordance with claim 2, wherein the automatic speech recognition (ASR) module is configured to selectively transcribe Cantonese portion of the audio signal into informal Cantonese transcript 30 and English portion of the audio signal into English transcript and the translation module is configured to translate the informal Cantonese transcript and English transcript into formal written Chinese text. 6. A system for processing code-switching transcription in accordance with claim 2, 35 wherein the audio signal includes indigenous Cantonese and the translation module is configured to translate indigenous Cantonese text into written Chinese text. 1 HK 30135233 A 2 7. A system for processing code-switching transcription in accordance with claim 3, wherein the automatic speech recognition (ASR) module is pretrained and finetuned by one or more training strategies so as to improve the transcription performance. 8. A system for processing code-switching transcription in accordance with claim 4,5 wherein the translation module is pretrained and finetuned on large language models (LLMs) so as to improve overall translation performance. 9. A system for processing code-switching transcription in accordance with claim 3, wherein the automatic speech recognition (ASR) module is pretrained by Cantonese 10 audio-transcript data pairs in a closed loop configuration. 10. A system for processing code-switching transcription in accordance with claim 4, wherein the translation module is pretrained by cleansed Cantonese-Chinese bilingual corpus data in a closed loop configuration. 15 HK 30135233 A FIG. 1 100 User Interface 140 Acoustic Model 122 Language Model 124 Translation Model 132 Self-trained Cantonese-English code-switching ASR 120 Informal Cantonese  formal written Chinese Translator 130 User 10 Transcription 102 Translation 104 Indigenous Cantonese-led Transcription & Translation Engine 110 1 HK 30135233 A FIG. 2 ASR system 120 Cantonese audio-transcript pairs 216 Acoustic Model 122 Model evaluation 218 ASR system 120 Cantonese audio 222 Transcribed Cantonese text 224 (Re) Training 219 Transcription Pre-processing engine 214 Data collection 212 Training Deployment Language Model 124 Training 217 2 HK 30135233 A FIG. 3 平時生活上嘅一啲方案呢變成一啲圓形一啲prototyGoogle Cloud 平時生活上嘅一啲方案呢，變成一啲圓形，一啲prototypeMicrosoft Azure 平時生活上嘅一啲方案咧變成一啲圓形一啲productionAliyun Speech 平時生活上嘅一啲方案呢變成一啲圓形一啲鋪到題FanoLabs 平時生活上嘅一啲方案呢，變成一啲圓形一啲politycantonese.ai 平時生活上嘅一啲方案呢變成一啲圓形一啲prototypeLSCM-neural 平時生活上嘅一啲方案呢，變成一啲原型，一啲prototypeLSCM-LLM 平時生活上嘅一啲方案呢變成一啲原型，一啲prototypeOriginal transcript 300 310 314 320 330 340 350 360 380 312 3 HK 30135233 A FIG. 4 你哋個file所有你哋啲membership general counsel account professional development其實全部喺個快勞嗰度 Google Cloud 你哋個bylaws呢，所有你哋啲membership呀，general council呀accounts呀 professional development其實全部喺個by law嗰度嘅。 Microsoft Azure 你哋個big lass呢，所有你哋啲membership呀，general countial呀account呀， professional development。 FanoLabs 你哋個bylaws呢，所有佢地啲membership呀general counsel啊accounts、 professional development其實全部喺嗰bylaws度嘅 LSCM-neural 你哋個bylaws呢，所有你哋啲membership呀，general counsel呀，accounts呀， professional development 其實全部喺嗰bylaws嗰度嘅 LSCM-LLM 你哋個bylaws呢，所有你哋啲membership呀、general counsel呀、accounts呀、 professional development，其實全部喺個bylaws嗰到嘅 Original transcript 400 410 420 430 440 450 460 411 412 413 414 415 417 418 416 4 HK 30135233 A FIG. 7 FIG. 5 FIG. 6 700 510 520 530 512 534 610 620 630 612 622 632 712 710 720 722 730 5 HK 30135233 A FIG. 8 Cleansed Bilingual corpus (Cantonese-Chinese) 816 Translation model 132 Model evaluation 818 Translation model 132 Cantonese text 822 Written Chinese text 824 (Re) Training 819 Translation 820 Pre-processing engine 814 Data collection 812 Training Deployment 800 Training 817 6 HK 30135233 A FIG. 9 他昨天就説買外賣都要嘟“好心出走”QR圖”號，但吃環署深夜就出新聞稿説不用都可以 接受。 Baidu Translator 他昨天就說買外賣都要嘟「安心出行」QR圖碼，但食環署深夜就出新聞稿說不用也可以接 受。 Google Translate 他昨天就說買外賣都要掃「安心出行」QR碼，但食環署深夜就發出新聞稿說不用也可以接 受。 Microsoft Bing 他昨天就説買外賣也要嘟「安心出行」QR圖碼,但食環署深夜就出新聞稿説不用也可以接受。LSCM-neural 他昨日説買外賣都要掃描「安心出行」QR圖碼，但食環署深夜就發出新聞稿説不用也可以 接受。 LSCM-LLM 佢尋日就話買外賣都要嘟「安心出行」QR圖碼，但食環署深夜就出新聞稿話唔用都可以接 受。 Cantonese 900 910 920 930 940 950 960 911 912913 914 7 HK 30135233 A FIG. 10 伍先生這個 proposal 聽起來挺 creative，但馮先生說不符合市場估計期。Baidu Translator 伍先生這個proposal聽起來挺creative，但馮先生說不符合市場預期。Google Translate 伍先生這個提案聽起來很有創意，但馮先生說不符合市場預期。Microsoft Bing 伍先生這個提案聽起來很創意，但馮先生説不符合市場預期。LSCM-neural 伍先生的提案聽起來很有創意，但馮先生説不符合市場預期。LSCM-LLM 伍先生呢個proposal聽落幾creative，但馮先生話唔符合市場預期。Cantonese 1000 1010 1020 1030 1040 1050 1060 1011 10121013 1014 8 HK 30135233 A FIG. 11 1. File upload 3. Upload progress 4. Audio split and transcription results displayed 2. Transcribe audio 5. Translate to formal Chinese 6. Formal Chinese translation results displayed 7. Export result to file 1100 1110 1120 1130 1140 1150 1160 1170 9 HK 30135233 A

Images