Lightweight multilingual context semantic continuity analysis method based on localized model

Abstract

The invention provides a lightweight multilingual context semantic continuity analysis method based on a localization model, and relates to the technical field of multilingual calculation crossover, the method comprises the following steps: on the basis of semantic integrity detection, carrying out preprocessing and adaptive blocking on a received multilingual text to be analyzed to obtain a text block sequence; dynamically selecting a final lightweight semantic understanding model from a pre-constructed lightweight multilingual semantic understanding model family based on the language features presented by the text block sequence in combination with a real-time hardware resource state of local equipment; and based on the final lightweight semantic understanding model, encoding the text block sequence, and extracting and fusing semantic features of different network depths of the final lightweight semantic understanding model to obtain a multi-scale semantic vector sequence. According to the method, the context semantic continuity of multilingual texts and multi-round dialogues can be efficiently and accurately analyzed.

Description

Technical Field

[0001] This invention relates to the field of multilingual computing technology, specifically to a lightweight multilingual contextual semantic continuity analysis method based on a localization model. Background Technology

[0002] With the deepening of global digital interaction, multilingual contextual semantic continuity analysis has become a core requirement in scenarios such as cross-border customer service and intelligent interaction of edge devices. Existing solutions rely on large-scale pre-trained models in the cloud, which poses risks of network latency and sensitive data leakage. Lightweight models only focus on monolingual or simple classification tasks and are difficult to adapt to complex scenarios with mixed multilingual text.

[0003] When handling multilingual inquiries from users regarding course content, textbook availability, and language support, the embedded intelligent consultation assistant of a cross-border education platform needs to accurately determine the semantic coherence of the context to provide a coherent response. However, this solution has technical flaws. The locally deployed, lightweight model cannot effectively integrate the multi-layered semantic features of multilingual texts at the lexical, syntactic, and discourse levels, making it difficult to capture the semantic connections between cross-language expressions, misjudging the logical coherence of the context, and ultimately providing fragmented responses that are out of touch with the user's needs. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a lightweight multilingual contextual semantic continuity analysis method based on a localization model, so as to achieve efficient and accurate analysis of the contextual semantic continuity of multilingual texts and multi-turn dialogues.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: Firstly, a lightweight multilingual contextual semantic continuity analysis method based on a localization model, the method comprising: Based on semantic integrity detection, the received multilingual text to be analyzed is preprocessed and adaptively segmented to obtain a text block sequence; Based on the language features presented by the text block sequence and combined with the real-time hardware resource status of the local device, the final lightweight semantic understanding model is dynamically selected from a pre-built family of lightweight multilingual semantic understanding models. Based on the final lightweight semantic understanding model, the text block sequence is encoded, and semantic features from different network depths of the final lightweight semantic understanding model are extracted and fused to obtain a multi-scale semantic vector sequence. By using multi-scale semantic vector sequences, the semantic similarity, entity overlap, and text surface feature differences between adjacent text blocks in the sequence are calculated to obtain the calculation results. The results are combined to obtain the local coherence metric. Based on the multi-scale semantic vector sequences, combined with the attention weights generated by the final lightweight semantic understanding model when processing text block sequences, the semantic association strength between non-adjacent text blocks is calculated. The long-range dependency metric is obtained by introducing a distance decay factor and topic consistency detection. Based on the text type of the multilingual text, appropriate adaptive weights are assigned to the local coherence measure and the long-range dependency measure. The local coherence measure and the long-range dependency measure are then weighted and fused using the adaptive weights to obtain the semantic continuity score of the overall contextual logical coherence and semantic fluency of the multilingual text.

[0006] Furthermore, based on semantic integrity detection, the received multilingual text to be analyzed is preprocessed and adaptively segmented to obtain a text block sequence, including: The received raw text is subjected to language detection and encoding standardization processing to obtain language type recognition results and intermediate text with unified encoding format; Based on the language type recognition results, a locally pre-stored lightweight semantic boundary detection model is called to process the intermediate text in order to obtain the location information of potential sentence boundaries, phrase boundaries or topic transition boundaries. Based on location information and combined with preset semantic integrity rules, the boundaries are filtered and merged to finally determine the semantic unit boundaries used for segmentation. By using semantic unit boundaries, the intermediate text is segmented to obtain a sequence of text blocks consisting of multiple text blocks.

[0007] Furthermore, based on the linguistic features presented in the text block sequence and combined with the real-time hardware resource status of the local device, the final lightweight semantic understanding model is dynamically selected from a pre-built family of lightweight multilingual semantic understanding models, including: Based on the text block sequence, the types of languages ​​included, the proportion of text blocks of each language, and the total length of the text block sequence are statistically analyzed to obtain a language feature vector describing the current analysis task; the real-time hardware resource status parameters of the local device are collected, including at least the available memory size, processor frequency, and current load, to obtain a resource status vector describing the device's real-time computing capabilities. The language feature vector and resource state vector are input into a preset dynamic selection decision model. The decision model selects a subset of candidate models from a pre-built model family that simultaneously meet the current language analysis requirements and the real-time resource constraints of the device, based on a predefined accuracy-efficiency balance strategy. By using a pre-trained lightweight performance predictor, the inference time and peak memory usage required to process the current text block sequence are predicted for each model in the candidate model subset. Combined with the preset model accuracy index, a multi-objective optimization algorithm selects the model with the highest comprehensive score as the final lightweight semantic understanding model.

[0008] Furthermore, based on the final lightweight semantic understanding model, the text block sequence is encoded. By extracting and fusing semantic features from different network depths of the final lightweight semantic understanding model, a multi-scale semantic vector sequence is obtained, including: The text block sequence is input into the final lightweight semantic understanding model. Through forward propagation, the complete encoding result containing the activation values ​​of the intermediate and output layers of all text blocks is obtained. From the complete encoding results, network layer outputs of different depths of the final lightweight semantic understanding model are selected according to a predetermined strategy, and hierarchical feature vectors representing lexical, syntactic and discourse semantics are extracted respectively. Dimensionality reduction and normalization operations are performed on the hierarchical feature vectors to obtain intermediate semantic vectors with uniform scale and density. The intermediate semantic vectors are aggregated through weighted fusion or concatenation operations to obtain the final semantic vector representing the multi-scale semantic information fused by each text block. The vectors are then arranged in the order of the text blocks in the original sequence to form a multi-scale semantic vector sequence.

[0009] Furthermore, by using multi-scale semantic vector sequences, the semantic similarity, entity overlap, and text surface feature differences between adjacent text blocks in the sequence are calculated to obtain the calculation results. The combined calculation results yield a local coherence metric, including: Based on the multi-scale semantic vector sequence, the cosine similarity of each pair of adjacent semantic vectors is calculated to obtain the semantic similarity calculation result; Using the same set of multi-scale semantic vector sequences associated with the semantic similarity calculation results, a lightweight entity recognition module is called to extract the named entity set corresponding to each text block and calculate the Jaccard similarity coefficient between the named entities of adjacent text blocks to obtain the entity overlap calculation result; and based on the original text block sequence corresponding to the semantic vector sequence, the edit distance between adjacent text blocks in terms of word composition and character sequence is directly calculated to obtain the text surface feature difference calculation result. The semantic similarity calculation result, entity overlap calculation result, and text surface feature difference calculation result are taken as inputs and fed into a pre-trained local coherence fusion model. The local coherence fusion model outputs a comprehensive scalar value as the final local coherence metric.

[0010] Furthermore, based on multi-scale semantic vector sequences and combined with the attention weights generated by the final lightweight semantic understanding model when processing text block sequences, the semantic association strength between non-adjacent text blocks is calculated. This is then corrected by introducing a distance decay factor and topic consistency detection to obtain a long-range dependency metric, including: Based on the multi-scale semantic vector sequence and the attention weight matrix generated by the final lightweight semantic understanding model when processing text block sequences, the product of the semantic cosine similarity and attention association strength between all non-adjacent text block pairs is calculated to obtain the initial set of long-range semantic association strengths. Based on the long-range semantic association strength set, a distance decay factor related to the relative distance in the sequence of non-adjacent text blocks is introduced to weight and correct the association strength of each pair of non-adjacent text blocks, thus obtaining the long-range semantic association strength set. Based on the long-range semantic association strength set and text block sequence, a lightweight topic consistency detection module is called to perform topic consistency judgment on each pair of non-adjacent text blocks, obtain the judgment result, and perform a second correction on the association strength according to the judgment result to obtain the final long-range semantic association strength set after topic consistency correction. Statistically aggregate all association strength values ​​in the final long-range semantic association strength set, calculate the weighted average of the association strength values, and use the weighted average as the long-range dependency metric that represents the long-range semantic dependencies of the entire text.

[0011] Furthermore, based on the text type of the multilingual text, appropriate adaptive weights are assigned to the local coherence measure and the long-range dependency measure; the local coherence measure and the long-range dependency measure are then weighted and fused using these adaptive weights to obtain a semantic continuity score for the overall contextual logical coherence and semantic fluency of the multilingual text, including: Based on the text type of the multilingual text, the corresponding basic weight allocation strategy is obtained from the preset adaptive weight mapping table, and fine-tuned by combining the language mixing complexity and length features of the text block sequence to obtain the final adaptive weight parameters used for weighted fusion. By using adaptive weight parameters, a weighted summation of local coherence and long-range dependency measures is calculated to obtain a preliminary comprehensive score for semantic continuity. Based on historical scoring data or preset scoring normalization rules, the initial semantic continuity comprehensive score is scaled and normalized to ensure that the semantic continuity comprehensive score falls within the preset standard scoring range, so as to obtain a standardized semantic continuity score. Based on the standardized semantic continuity score, corresponding local coherence and long-range dependency metrics, a final semantic continuity assessment report is obtained, which includes the overall score, sub-metrics, and a coherence analysis summary.

[0012] Secondly, a lightweight multilingual contextual semantic continuity analysis system based on a localization model includes: The acquisition module is used to preprocess and adaptively segment the received multilingual text to be analyzed based on semantic integrity detection to obtain a text block sequence; The selection module is used to dynamically select the final lightweight semantic understanding model from a pre-built family of lightweight multilingual semantic understanding models based on the language features presented in the text block sequence and in combination with the real-time hardware resource status of the local device. The fusion module is used to encode text block sequences based on the final lightweight semantic understanding model. By extracting and fusing semantic features from different network depths of the final lightweight semantic understanding model, a multi-scale semantic vector sequence is obtained. The calculation module is used to calculate the semantic similarity, entity overlap and text surface feature differences between adjacent text blocks in a multi-scale semantic vector sequence, and obtain the calculation results. The combined calculation results are used to obtain the local coherence metric. Based on the multi-scale semantic vector sequence, combined with the attention weight generated by the final lightweight semantic understanding model when processing the text block sequence, the semantic association strength between non-adjacent text blocks is calculated, and the long-range dependency metric is obtained by introducing a distance decay factor and topic consistency detection. The processing module is used to assign appropriate adaptive weights to the local coherence measure and the long-range dependency measure according to the text type of the multilingual text; and to perform weighted fusion of the local coherence measure and the long-range dependency measure through adaptive weights to obtain the semantic continuity score of the overall contextual logical coherence and semantic fluency of the multilingual text.

[0013] Thirdly, a computing device includes: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0014] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0015] The above-described solution of the present invention has at least the following beneficial effects: By employing techniques such as adaptive text segmentation based on semantic integrity, dynamic model selection combining text language features and real-time resource status of local devices, multi-scale vector extraction of semantic features from different network depths of the fusion model, long-range dependency detection with multi-dimensional local coherence measurement, attention weights, distance decay factors, and topic consistency correction, and standardized scoring with adaptive weighted fusion according to text type, this approach effectively overcomes the technical shortcomings of existing solutions, such as the difficulty of deploying large-scale pre-trained models locally on edge devices, the risk of network latency and sensitive data leakage in cloud services, and the inability of traditional lightweight models to accurately capture the multi-layered semantic features and long-range semantic dependencies of multilingual mixed text. It achieves the goal of efficiently running multilingual contextual semantic continuity analysis on local devices, ensuring data privacy and security as well as real-time analysis, and improving the accuracy, reliability, and scenario adaptability of multilingual text semantic coherence assessment. This provides high-quality semantic analysis support for multilingual interaction scenarios on edge devices. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a lightweight multilingual contextual semantic continuity analysis method based on a localization model, provided in an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of a lightweight multilingual contextual semantic continuity analysis system based on a localization model provided in an embodiment of the present invention. Detailed Implementation

[0018] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0019] like Figure 1 As shown, embodiments of the present invention propose a lightweight multilingual contextual semantic continuity analysis method based on a localization model, the method comprising the following steps: Step 1: Based on semantic integrity detection, preprocess and adaptively segment the received multilingual text to be analyzed to obtain a text block sequence; Step 2: Based on the language features presented by the text block sequence and combined with the real-time hardware resource status of the local device, dynamically select the final lightweight semantic understanding model from the pre-built family of lightweight multilingual semantic understanding models. Step 3: Based on the final lightweight semantic understanding model, the text block sequence is encoded. By extracting and fusing semantic features from different network depths of the final lightweight semantic understanding model, a multi-scale semantic vector sequence is obtained. Step 4: Using multi-scale semantic vector sequences, calculate the semantic similarity, entity overlap, and text surface feature differences between adjacent text blocks in the sequence to obtain the calculation results. Combine the calculation results to obtain the local coherence metric. Based on the multi-scale semantic vector sequences, combined with the attention weights generated by the final lightweight semantic understanding model when processing text block sequences, calculate the semantic association strength between non-adjacent text blocks. Correct the results by introducing a distance decay factor and topic consistency detection to obtain the long-range dependency metric. Step 5: Assign appropriate adaptive weights to the local coherence measure and the long-range dependency measure according to the text type of the multilingual text; and use the adaptive weights to perform weighted fusion of the local coherence measure and the long-range dependency measure to obtain the semantic continuity score of the overall contextual logical coherence and semantic fluency of the multilingual text.

[0020] In this embodiment of the invention, by employing techniques such as multilingual text preprocessing and adaptive segmentation based on semantic integrity detection, dynamic model selection combining text language features and real-time hardware resource status of local devices, multi-scale vector extraction integrating semantic features of different network depths of lightweight models, long-range dependency detection with multi-dimensional local coherence measurement, attention weights, distance decay factors, and topic consistency correction, and weighted fusion scoring with adaptive weight allocation according to text type, the invention effectively overcomes the technical problems of traditional fixed segmentation fragmenting semantics, model selection failing to adapt to resource and text requirements, insufficient information from single deep features, inaccurate capture of local and long-range semantic associations in multilingual texts, and poor scene adaptability due to fixed scoring weights. This achieves the goal of efficiently processing multilingual text semantic continuity analysis on local devices, ensuring data privacy and security as well as real-time analysis, improving the accuracy and scene adaptability of semantic coherence evaluation for multilingual mixed texts and long texts, and providing high-quality and reliable semantic analysis support for multilingual interaction scenarios on edge devices.

[0021] In a preferred embodiment of the present invention, step 1 above may include: Step 1.1 involves performing language detection and encoding standardization on the received raw text to obtain language type identification results and intermediate text with a unified encoding format. Specifically, this includes: receiving the user's original multilingual mixed consultation text obtained by the embedded intelligent consultation assistant of the cross-border education platform; starting the locally deployed lightweight language detection module; analyzing the text content segment by segment to identify all the language types contained therein; clearly distinguishing the distribution range of different languages ​​in the text; comprehensively detecting the encoding format of the raw text; and uniformly converting texts with different encoding standards into a common standard encoding format to eliminate subsequent processing obstacles caused by multilingual encoding differences. Finally, a clear and unambiguous language type identification result and intermediate text with a completely unified encoding format are obtained.

[0022] Step 1.2: Based on the language type recognition results, a locally pre-stored lightweight semantic boundary detection model is invoked to process the intermediate text to obtain the location information of potential sentence boundaries, phrase boundaries, or topic transition boundaries. Specifically, this includes: based on the language type recognition results, a locally pre-stored lightweight semantic boundary detection model adapted to the identified language is invoked. The lightweight semantic boundary detection model deeply analyzes key identifiers such as punctuation marks, modal particles, and logical connectors in the text, taking into account the expression habits and grammatical rules of different languages. At the same time, it accurately captures semantic features related to topic transitions, accurately identifies potential sentence boundaries, phrase boundaries, and topic transition boundaries in the text, and records the specific location information of these boundaries in the intermediate text in detail.

[0023] Step 1.3: Based on the location information and in conjunction with preset semantic integrity rules, the boundaries are filtered and merged to finally determine the semantic unit boundaries used for segmentation. Specifically, this includes: collecting all potential boundary location information, filtering each potential boundary one by one in conjunction with preset semantic integrity rules, and resolutely eliminating invalid boundaries that may cause semantic fragmentation, such as boundaries that would split the user's intent to inquire about the practical content of the course, or boundaries that would split inquiries related to the localized version of the textbook; reasonably merging adjacent boundaries that do not affect semantic integrity to avoid over-segmentation, focusing on ensuring the integrity of the user's three core inquiry intents regarding what practical content the course includes, whether the relevant textbook has a localized version, and whether multilingual learning support can be provided, and finally determining the segmentation boundaries that can divide independent and complete semantic units.

[0024] Step 1.4 involves segmenting the intermediate text using semantic unit boundaries to obtain a sequence of text blocks. Specifically, this includes: dividing the intermediate text, after unifying the encoding format, into ordered segments based on the final determined semantic unit boundaries. This accurately segments the user's multilingual mixed consultation text into multiple semantically independent and complete text blocks. Each text block corresponds to a specific consultation point or semantic fragment, such as consultation on course practical content, inquiry about localized versions of teaching materials, or requests for multilingual learning support. These text blocks are arranged according to their order in the original consultation to obtain a text block sequence with a clear structure and explicit semantics. This provides standardized input data for lightweight model encoding and multi-dimensional semantic coherence analysis.

[0025] In this embodiment of the invention, by employing technical means such as language detection and encoding standardization of the received original multilingual text, calling a local lightweight semantic boundary detection model based on language type to obtain potential boundary location information, filtering and merging boundaries in combination with semantic integrity rules, and segmenting the text according to the final semantic unit boundaries, the technical problems of inconsistent multilingual text encoding formats, semantic fragmentation caused by traditional fixed block segmentation methods, and incomplete semantics of text blocks due to lack of targeted boundary recognition are effectively overcome. This achieves the unification of multilingual text formats and accurate segmentation of semantic units, resulting in semantically independent and complete text block sequences. This provides a high-quality input foundation for subsequent lightweight model encoding, multi-dimensional semantic coherence measurement, and other steps, ensuring the accuracy and reliability of the entire multilingual context semantic continuity analysis process.

[0026] In a preferred embodiment of the present invention, step 2 above may include: Step 2.1: Based on the text block sequence, statistically analyze the language types included, the proportion of text blocks in each language, and the total length of the text block sequence to obtain a language feature vector describing the current analysis task. Collect the real-time hardware resource status parameters of the local device, including at least the available memory size, processor frequency, and current load, to obtain a resource status vector describing the device's real-time computing capabilities. Specifically, this includes: conducting comprehensive language feature statistics and device resource status collection on the text block sequence already acquired by the embedded intelligent consultation assistant of the cross-border education platform. In terms of language feature statistics, analyze each text block in the text block sequence one by one, identify and record all the language types contained therein, clearly distinguish which text blocks correspond to each language, ensure that no language involved in mixed consultation is missed, count the number of text blocks corresponding to each language, calculate the proportion of text blocks of each language to the total number of text blocks in the entire text block sequence, clearly present the specific composition of multilingual mixing, count the total number of text blocks contained in the text block sequence, and determine the total length of the text block sequence to reflect the text scale of the current analysis task. By integrating the statistically obtained list of language types, the proportion of text blocks in each language, and the total length of the text block sequence, a language feature vector is obtained that comprehensively and accurately describes the characteristics of the current multilingual mixed consultation and analysis task. In terms of device resource status collection, the hardware resource usage of the device where the embedded intelligent consultation assistant is located is queried in real time. The available memory size that is not currently occupied by other programs is recorded in detail, the current running frequency value of the processor is accurately obtained, and the current resource usage ratio of the processor is monitored and statistically analyzed, i.e., the current load. These real-time collected hardware resource parameters, such as available memory size, processor frequency value, and current processor load, are summarized to obtain a resource status vector that fully describes the real-time computing capabilities of the device.

[0027] Step 2.2 involves inputting the language feature vector and resource state vector into a pre-defined dynamic selection decision model. Based on a predefined precision-efficiency balancing strategy, the decision model selects a subset of candidate models from a pre-built model family that simultaneously meets the current language analysis requirements and the real-time resource constraints of the device. Specifically, this includes inputting the language feature vector and resource state vector into the pre-defined dynamic selection decision model, which incorporates a predefined precision-efficiency balancing strategy specifically for multilingual semantic continuity analysis scenarios. The core of this strategy is to maximize the adaptation to the limited hardware resources of embedded devices while ensuring the accuracy of multilingual mixed text analysis, without sacrificing the ability to fuse multilingual text features at the lexical, syntactic, and discourse levels, and without exceeding the device's resource capacity limit. The decision model first deeply analyzes the core requirements of the current analysis task based on the language feature vector, clarifying the specific requirements of the number of languages, the mixing ratio, and the text length in the current consultation text on the model's multilingual processing capabilities and text adaptation capabilities. For example, the model needs to accurately capture the semantic relationships between different language expressions and be able to process text block sequences of the current scale without semantic omissions. The decision model comprehensively grasps the requirements by combining the resource state vector. The device's real-time computing and storage limits are defined, specifying the maximum available memory the device can currently provide and the maximum computing load the processor can handle. Based on the analysis, the decision model selects from a pre-built family of lightweight models specifically designed for multilingual contextual semantic continuity analysis. Each model in the family is verified to simultaneously meet two core conditions: the model's multilingual processing capabilities and semantic feature fusion capabilities can cover the current language analysis needs; the model's memory usage does not exceed the device's current available memory; and the computing power requirement does not exceed the device's current remaining computing power. Finally, models that meet the conditions are selected to form a subset of candidate models that satisfy the current language analysis needs and the device's real-time resource constraints.

[0028] Step 2.3: Using a pre-trained lightweight performance predictor, for each model in the candidate model subset, predict the inference time and peak memory usage required to process the current text block sequence. Combined with pre-set model accuracy metrics, a multi-objective optimization algorithm selects the model with the highest overall score as the final lightweight semantic understanding model. Specifically, this includes: calling the pre-trained lightweight performance predictor, which has been trained on a large number of multilingual text analysis tasks and running data from different edge devices, possessing the ability to accurately predict model performance; for each model in the candidate model subset, the performance predictor, based on the language features, text size, and resource state vector of the current multilingual mixed consultation text block sequence from the cross-border education platform user, predicts the inference time required for the model to process the current text block sequence, the complete time from inputting the text block sequence into the model to outputting the initial encoding result, the peak memory usage during model operation, and the maximum memory usage on the device during model operation; and retrieving the pre-set accuracy metrics for each candidate model. These accuracy metrics are test results of the model during training, focusing on core performance dimensions such as the ability to fuse multi-level semantic features of multilingual text, capture cross-language semantic associations, and judge contextual logical coherence. They comprehensively reflect the model's analytical accuracy in processing multilingual mixed text. The inference time, peak memory usage, and accuracy metrics of each candidate model are used as the three core evaluation dimensions. A multi-objective optimization algorithm is used for comprehensive scoring. The algorithm assigns appropriate weights to the three evaluation dimensions based on the needs of cross-border education consultation scenarios, with accuracy having the highest weight to ensure the model accurately understands the semantic coherence of users' multilingual consultations. The weights for inference time and peak memory usage ensure the model can run efficiently on embedded devices without slow response or resource shortages. After calculating the comprehensive score for each candidate model using the multi-objective optimization algorithm, the model with the highest comprehensive score is selected. This model meets the high-precision analysis requirements of multilingual mixed consultation texts and adapts to the real-time resource status of embedded devices, serving as the final lightweight semantic understanding model.

[0029] In this embodiment of the invention, by employing techniques such as constructing language feature vectors based on statistical analysis of language types, the proportion of each language, and the total length of the sequence in text block sequences, constructing resource status vectors by collecting available memory, processor frequency, and current load on the local device, inputting these two types of vectors into a dynamic selection decision model to filter a subset of candidate models, predicting model inference time and peak memory usage using a lightweight performance predictor, and selecting the model with the highest comprehensive score through a multi-objective optimization algorithm in conjunction with model accuracy metrics, this invention effectively overcomes the technical problems of traditional model selection ignoring the matching degree between text language features and real-time device resources, and fixed models either exceeding the resource carrying capacity of edge devices or failing to meet the accuracy requirements of multilingual analysis. This achieves precise adaptation of model selection to multilingual analysis tasks and local device resources, ensuring the accuracy of multilingual text semantic analysis while minimizing resource consumption, ensuring the real-time performance and stability of the analysis process, and providing efficient and adaptable model support for semantic vector extraction and coherence measurement.

[0030] In a preferred embodiment of the present invention, step 3 above may include: Step 3.1: Input the text block sequence into the final lightweight semantic understanding model. Through forward propagation computation, obtain the complete encoding result containing the activation values ​​of the intermediate and output layers of all text blocks. Specifically, this includes: inputting the semantically complete text block sequence containing the user's core consultation intent, which has already been broken down by the embedded intelligent consultation assistant of the cross-border education platform, into the final lightweight semantic understanding model, which has been dynamically selected and determined. This model has been adapted to the characteristics of the current multilingual mixed consultation scenario and the resource constraints of embedded devices. It can accurately process multilingual semantic information while ensuring efficient resource utilization. After the model starts, it performs forward propagation computation on the text block sequence according to the preset computation process, performing semantic encoding layer by layer for each text block. It not only comprehensively calculates and obtains the activation values ​​of the model's output layer, but also completely retains all the activation data generated by each intermediate network layer during the encoding process, finally obtaining the complete encoding result containing the semantic representation information of all text blocks in each layer of the model.

[0031] Step 3.2: From the complete encoding results, select network layer outputs of different depths of the final lightweight semantic understanding model according to a predetermined strategy, and extract hierarchical feature vectors representing lexical, syntactic, and discourse-level semantics respectively. Specifically, this includes: based on a preset multi-scale feature extraction strategy, accurately selecting network layer outputs of different depths of the final lightweight semantic understanding model from the complete encoding results, selecting the output results of the shallow network of the model, and specifically extracting lexical-level semantic feature vectors. The focus is on capturing the semantic information of core keywords such as course practical content, localized textbook versions, and multilingual learning support in each text block, while fully adapting to the semantic representation needs of multilingual vocabulary to ensure the semantic representation of different languages. Lexical semantics can be accurately captured. By selecting the output of the middle layer network of the model, syntactic-level semantic feature vectors are extracted to clarify the grammatical structure, sentence type, and grammatical relationship between words in each text block. For example, the semantic expression logic of interrogative sentences in user consultation texts and the adaptation of different language sentence structures. By selecting the output of the deep network of the model, discourse-level semantic feature vectors are extracted to focus on the core consultation intent of each text block, which corresponds to the three core semantic demands of querying course practical content, inquiring about the localized version of the textbook, and requesting multilingual learning support. Through this process, hierarchical feature vectors corresponding to the three semantic dimensions of lexical, syntactic, and discourse levels are obtained.

[0032] The model selection for device capability awareness, multi-dimensional device capability profile, not only considers hardware parameters, but also constructs device semantic analysis performance profiles; the processing performance differences of different languages, the processing efficiency of different text lengths, the changes in resource availability at different time points, innovative strategies for dynamic model selection, and improved algorithms for initial model screening. M∈ , , , , ,in, Step 3.3 involves performing dimensionality reduction and normalization operations on the hierarchical feature vectors to obtain dense intermediate semantic vectors with uniform scale. Specifically, this includes performing systematic and meticulous dimensionality reduction and normalization operations on the extracted lexical, syntactic, and discourse-level hierarchical feature vectors. Since the feature vectors output by different network layers of the model differ in dimensionality, and the semantic features of multilingual texts may have inconsistent numerical scales, these differences can interfere with subsequent feature fusion and semantic association calculations. A dimensionality reduction method that is fully adapted to the computational needs of lightweight models is adopted to effectively reduce the dimensionality complexity of feature vectors and reduce computational resource consumption while preserving the core semantic information to the greatest extent possible. The feature vectors at each level after dimensionality reduction are normalized one by one, mapping the values ​​of all vectors to the same standard range, completely eliminating the computational interference caused by the differences in feature scales of multilingual features and the differences in feature dimensions at different levels, and finally obtaining intermediate semantic vectors with completely uniform scale, dense data, and complete core semantic information.

[0033] Step 3.4: Aggregate the intermediate semantic vectors through weighted fusion or concatenation operations to obtain the final semantic vector representing the multi-scale semantic information fused from each text block. Arrange these vectors according to the order of the text blocks in the original sequence to form a multi-scale semantic vector sequence. Specifically, based on the core requirements of multilingual contextual semantic continuity analysis and considering the characteristics of multilingual mixed-consultation scenarios for users on current cross-border education platforms, assign scientifically reasonable weights to the lexical, syntactic, and discourse-level intermediate semantic vectors. Emphasis is placed on enhancing the feature weights related to capturing cross-language semantic associations to ensure that the semantic connections between different language expressions are fully highlighted. The three dimensions are then fused using a weighted fusion method. The intermediate semantic vectors of each degree are organically aggregated into a comprehensive semantic vector. If the independence of semantic features is required to a high extent in some special scenarios, a concatenation operation is used to integrate the vector information. Finally, the final semantic vector of each text block is obtained, which integrates multi-scale semantic information. Each final semantic vector fully covers the core meaning of the corresponding text block's vocabulary, syntactic structure logic, and discourse intent. At the same time, it fully preserves the key information of multilingual semantic association. All final semantic vectors are arranged in sequence according to the order of the text blocks in the user's original multilingual mixed consultation, forming a multi-scale semantic vector sequence with a standardized structure, rich semantics, and multi-dimensional information integration.

[0034] Context vector construction involves maintaining a sequence of context vectors from multiple dialogues to capture long-distance semantic dependencies and store dialogue history. A sliding window mechanism is used to store the semantic vectors of the most recent N dialogues, enabling context fusion. Finally, an attention mechanism is used to fuse the semantic vectors of the current dialogue and historical dialogues. ,in, It is the context vector of the current dialogue, and N is the number of the most recent dialogue turns stored in the sliding window. It is attention weight. It is the semantic vector of historical dialogues. Long-range dependency analysis is used to calculate the long-range dependency measure between the current dialogue and historical dialogues. ,in, It is a measure of the long-range dependency between the current dialogue and the i-th round of historical dialogue. The expression representing the attentional association between unit t and unit i is a semantic association weight calculated based on the attention mechanism. This represents the semantic vector at time t. This represents the semantic vector corresponding to the i-th unit, carrying the semantic information of the i-th unit. It is the distance decay factor. It is the turn distance between the current dialogue turn t and the historical dialogue turn i. Indicates the distance decay coefficient; context consistency check: judges the coherence between the current conversation and the historical conversation through the overall fluency score; ,in, It is a score of the overall fluency of the current dialogue and the historical dialogue. These are weighting coefficients. Long-term dependency score.

[0035] In this embodiment of the invention, by employing the technical means of inputting a text block sequence into the final lightweight semantic understanding model and obtaining the activation values ​​of the intermediate and output layers through forward propagation, selecting the outputs of different depth network layers according to a predetermined strategy to extract lexical, syntactic, and discourse-level hierarchical feature vectors, performing dimensionality reduction and normalization on the hierarchical feature vectors, and aggregating the final semantic vectors of each text block through weighted fusion or concatenation operations and arranging them in their original order to form a multi-scale semantic vector sequence, the invention effectively overcomes the technical problems of traditional lightweight models that only extract single-depth semantic features, cannot integrate multi-layer semantic information of multilingual texts, and lead to inaccurate capture of semantic associations between cross-language expressions. This achieves comprehensive capture and efficient integration of semantic features of multilingual texts across various dimensions, resulting in a multi-scale semantic vector sequence that is both rich and consistent. This provides a high-quality feature foundation for measuring local coherence and long-range dependencies, and improves the accuracy and reliability of multilingual context semantic continuity analysis.

[0036] In a preferred embodiment of the present invention, step 4 above may include: Step 4.1 specifically includes: obtaining the constructed multi-scale semantic vector sequence. The semantic vector sequence contains lexical, syntactic, and discourse-level fused semantic information of each text block in the multilingual mixed consultation of users on the cross-border education platform, and strictly follows the text block order of the user's original consultation. For each pair of adjacent semantic vectors in the sequence, a cosine similarity calculation process adapted to the computational needs of the lightweight model is initiated. The semantic fit of adjacent text blocks in the unified semantic space is compared one by one. The semantic representation characteristics of multilingual texts are fully considered during the calculation process to ensure that the same core semantics expressed in different languages ​​can be accurately identified. For example, course-related content mentioned in different languages ​​in the user consultation can be determined to be semantically related. Finally, the semantic similarity calculation results of each pair of adjacent text blocks are obtained to obtain a complete semantic similarity sequence and multi-angle similarity calculation. Where LocalCoherence(i) is the local coherence at position i. These are weighting coefficients, which assign importance percentages to similarity metrics across different dimensions and are used for weighted summation. cos(Vi, Vi+1) is the semantic vector cosine similarity, Jaccard(Ei, Ei+1) is the entity overlap, and EditDistance(Ti, Ti+1) represents the text surface feature edit distance.

[0037] Step 4.2: Using the same set of multi-scale semantic vector sequences associated with the semantic similarity calculation results, a lightweight entity recognition module is invoked to extract the named entity set corresponding to each text block and calculate the Jaccard similarity coefficient between named entities of adjacent text blocks to obtain the entity overlap calculation result; and based on the original text block sequence corresponding to the semantic vector sequence, the edit distance between adjacent text blocks in terms of lexical composition and character sequence is directly calculated to obtain the text surface feature difference calculation result. Specifically, based on the same set of multi-scale semantic vector sequences used, a locally deployed lightweight entity recognition module is invoked. This module has been specifically optimized for multilingual texts in cross-border education scenarios, and can efficiently identify named entities in different languages ​​without consuming too many device resources. The module processes each text block one by one, accurately extracting the set of named entities related to user inquiries. These entities include core inquiry elements such as course practical content, localized versions of teaching materials, and multilingual learning support, while being compatible with entity representation forms in different languages. Subsequently, the Jaccard similarity coefficient of named entities between each pair of adjacent text blocks is calculated to quantify the degree of overlap between adjacent text blocks at the core entity level, thus obtaining the entity overlap calculation result. Based on the original text block sequence corresponding to the semantic vector sequence, the edit distance between adjacent text blocks in terms of lexical composition and character sequence is directly calculated. This distance can reflect the degree of difference between adjacent text blocks in terms of surface form, such as the difference in the permutation and combination of words in different languages, the difference in character length, etc. Finally, the text surface feature difference calculation result is obtained, supplementing the data of local coherence analysis from the two dimensions of entity association and surface features.

[0038] Step 4.3: The semantic similarity calculation results, entity overlap calculation results, and text surface feature difference calculation results are used as inputs to a pre-trained local coherence fusion model. The local coherence fusion model outputs a comprehensive scalar value as the final local coherence metric. Specifically, this includes: collecting the semantic similarity calculation results, entity overlap calculation results, and text surface feature difference calculation results, and using these three types of data, which reflect the association between adjacent text blocks from the three dimensions of core semantics, entity association, and surface features, respectively, as joint inputs to the pre-trained local coherence fusion model. This fusion model has been trained on a task involving the analysis of local coherence of massive amounts of multilingual mixed text. It is specifically adapted to multilingual interaction scenarios such as cross-border education. It can automatically learn the weight of different dimensions of data in the judgment of local coherence, effectively avoiding the problem of one-sided judgment caused by traditional models that rely on only a single dimension of data. The model performs comprehensive analysis and weighted fusion of three types of input data, fully considering the impact of semantic characteristics, entity association rules and surface feature differences of multilingual texts on local coherence. Finally, it outputs a comprehensive scalar value that can fully characterize the degree of local semantic coherence of adjacent text blocks. The comprehensive scalar value is the final measure of local coherence.

[0039] In this embodiment of the invention, because it employs techniques such as calculating the cosine similarity of adjacent semantic vectors based on multi-scale semantic vector sequences, extracting named entity sets from each text block using a lightweight entity recognition module and calculating the Jaccard similarity coefficient between named entities of adjacent text blocks, calculating the edit distance of adjacent text blocks in terms of lexical composition and character sequence based on the original text block sequence, and then inputting the semantic similarity calculation results, entity overlap calculation results, and text surface feature difference calculation results into a pre-trained local coherence fusion model to output a comprehensive scalar value, it effectively overcomes the limitation of traditional lightweight models that rely solely on a single dimension for measurement. This addresses the technical challenges of inconsistent local semantic coherence, difficulty in considering semantic entity relationships and surface feature differences in cross-language texts, and the one-sided and inaccurate judgment of local semantic coherence in cross-language scenarios. It achieves multi-dimensional and comprehensive measurement of the local coherence of adjacent text blocks in multilingual mixed texts, improving the accuracy of capturing local semantic relationships in cross-language scenarios. This provides reliable and accurate local support for evaluating the overall contextual logical coherence and semantic fluency of multilingual texts, helping the embedded intelligent consultation assistant of cross-border education platforms accurately grasp the contextual relationships of users' multilingual mixed consultations and avoid fragmented responses caused by misjudgments of local coherence.

[0040] In a preferred embodiment of the present invention, step 4 above may include: Step 4.4: Based on the multi-scale semantic vector sequence and the attention weight matrix generated by the final lightweight semantic understanding model when processing text block sequences, calculate the product of the semantic cosine similarity and attention association strength between all non-adjacent text block pairs to obtain the initial set of long-range semantic association strengths. Specifically, this includes: obtaining the constructed multi-scale semantic vector sequence and the attention weight matrix generated by the final lightweight semantic understanding model when processing user multilingual mixed consultation text block sequences. The multi-scale semantic vector sequence fully covers the lexical, syntactic, and discourse-level semantic features of each text block, reflecting the semantic information of the user's core consultation intent regarding course practical content, localized textbook versions, multilingual learning support, etc. The attention weight matrix clearly presents the degree of attention the model pays to the semantic associations between different text blocks during processing, and the text related to the core consultation topic. Each block corresponds to a higher attention weight. The process iterates through all non-adjacent text block pairs in the text block sequence. For each non-adjacent text block pair, the cosine similarity between their semantic vectors is calculated. The cosine similarity quantifies the degree of core semantic fit between the two text blocks in a unified semantic space. This involves assessing the semantic correlation between text blocks related to course practice content and text blocks related to multilingual learning support in user consultations. The attention association strength corresponding to each text block pair in the attention weight matrix is ​​extracted. This association strength reflects the model's confidence in determining a semantic association between the two non-adjacent text blocks. The semantic cosine similarity of each non-adjacent text block pair is multiplied by its corresponding attention association strength to obtain the initial long-range semantic association strength of the text block pair. These initial association strengths are then integrated and summarized to obtain the initial set of long-range semantic association strengths. This is the long-range association strength based on the attention mechanism. ,in, It represents the long-range dependency between the i-th and j-th text units. Enhanced attention weight calculation, combining semantic and structural information, These are the semantic vectors of the i-th and j-th text units. It is a distance decay factor, dynamically adjusted based on text type. Topic consistency detection, based on a lightweight topic model. It is the text sequence of the i-th and j-th text units.

[0041] Step 4.5: Based on the long-range semantic association strength set, a distance attenuation factor related to the relative distance in the sequence of non-adjacent text blocks is introduced to weight and correct the association strength of each pair of non-adjacent text blocks, resulting in the long-range semantic association strength set. Specifically, this includes: based on the initial long-range semantic association strength set, introducing a distance attenuation factor related to the relative distance in the sequence of non-adjacent text blocks, and weighting and correcting the initial association strength of each pair of non-adjacent text blocks. In natural language expression, the long-range semantic association between text blocks usually weakens gradually as their relative distance in the sequence increases. The distance attenuation factor is set to conform to this language rule. The relative distance in the sequence of each pair of non-adjacent text blocks is calculated, that is, the position difference between two text blocks in the text block sequence. For example, the relative distance between the first text block and the third text block in the user's inquiry is greater than the relative distance between the first and the second. Then, according to the preset attenuation rule, the corresponding distance attenuation factor is determined based on the relative distance. The greater the relative distance, the smaller the value of the attenuation factor, which means that the attenuation of the initial association strength is greater. The initial long-range semantic association strength of each pair of non-adjacent text blocks is multiplied by the corresponding distance attenuation factor to obtain the long-range semantic association strength after distance attenuation correction, ensuring that the association strength can objectively reflect the influence of distance on long-range semantic dependence. All the corrected association strengths are then reintegrated to obtain the set of long-range semantic association strengths after distance attenuation correction.

[0042] Step 4.6: Based on the long-range semantic association strength set and the text block sequence, a lightweight topic consistency detection module is invoked to perform topic consistency judgment on each pair of non-adjacent text blocks, obtain the judgment result, and perform a secondary correction on the association strength according to the judgment result to obtain the final long-range semantic association strength set after topic consistency correction. Specifically, this includes: invoking the locally deployed lightweight topic consistency detection module, which is specifically optimized for multilingual mixed text and can efficiently and accurately judge the topic association between text blocks without occupying too much embedded device resources. The long-range semantic association strength set and the corresponding original text block sequence are input into the module. The module performs topic consistency analysis on each pair of non-adjacent text blocks one by one. During the analysis, the module extracts the core topic keywords and semantic intent of each text block and combines them with the multilingual topic feature library to determine whether two non-adjacent text blocks revolve around the same core topic. For example, if the three text blocks of a user consultation all revolve around the core topic of course-related consultation, then the topic consistency judgment result of each pair of non-adjacent text blocks is consistent. For text block pairs whose topics are consistent, the association strength is maintained or appropriately increased. For text block pairs whose topics are inconsistent, the association strength is reduced, thus completing the secondary correction based on topic consistency. All association strengths after the secondary correction are integrated and summarized to obtain the final long-range semantic association strength set corrected for topic consistency, ensuring that the association strengths in the set can truly reflect the long-range semantic associations between non-adjacent text blocks based on topic consistency.

[0043] Step 4.7 involves statistically aggregating all association strength values ​​in the final long-range semantic association strength set and calculating the weighted average of the association strength values. This weighted average is then used as the long-range dependency metric representing the long-range semantic dependencies of the entire text. Specifically, this includes: performing a systematic statistical aggregation operation on all association strength values ​​in the final long-range semantic association strength set; assigning reasonable weighting coefficients to each association strength value in the set based on the semantic structure of the text block sequence and the core intent of the user's inquiry; summing all the weighted association strength values ​​and dividing by the sum of the weighting coefficients to calculate the weighted average of the association strength values. The weighted average comprehensively considers the long-range semantic association strength, distance influence, and topic consistency of all non-adjacent text block pairs, and can comprehensively and objectively represent the long-range semantic dependencies of the entire user's multilingual mixed inquiry text. Finally, this weighted average is determined as the long-range dependency metric.

[0044] In this embodiment of the invention, by employing techniques such as calculating the product of semantic cosine similarity and attention association strength of all non-adjacent text block pairs using an attention weight matrix generated by a multi-scale semantic vector sequence and a final lightweight semantic understanding model, introducing a distance decay factor related to the relative distance of non-adjacent text blocks for weighted correction, calling a lightweight topic consistency detection module to judge the topic consistency of non-adjacent text blocks and correct the association strength a second time, and statistically aggregating and calculating a weighted average of the final long-range semantic association strength value, the invention effectively overcomes the technical problems of traditional lightweight models that are difficult to accurately capture long-range semantic dependencies in multilingual texts, ignore the natural law of semantic association decay with distance, and lack of topic consistency verification leading to one-sided long-range association judgment. This achieves accurate measurement of long-range semantic association of non-adjacent text blocks in multilingual mixed texts, fully taking into account the attention attention, distance rationality, and topic consistency of semantic associations. This ensures that the long-range dependency measurement value can truly reflect the long-range semantic dependency relationship of the entire text, providing reliable long-range support for the overall contextual semantic continuity scoring of multilingual texts, and improving the comprehensiveness and accuracy of semantic continuity analysis of multilingual long texts and mixed texts.

[0045] In a preferred embodiment of the present invention, step 5 above may include: Step 5.1: Based on the text type of the multilingual text, obtain the corresponding basic weight allocation strategy from the preset adaptive weight mapping table, and fine-tune it by combining the language mixing complexity and length characteristics of the text block sequence to obtain the final adaptive weight parameters used for weighted fusion. Specifically, this includes: clarifying that the text type of the currently processed multilingual text is a multilingual mixed consultation text from a cross-border education platform user. The core feature of this type of text is that it involves multiple rounds of consultation around specific service needs and contains related requests expressed across languages. Based on this text type, retrieve the corresponding basic weight allocation strategy from the preset adaptive weight mapping table. The mapping table has preset basic weight ratios for local coherence measurement and long-range dependency measurement for different text types such as cross-border education and cross-border customer service. The basic weight strategy for cross-border education consultation text focuses on long-range dependency measurement to adapt to multiple rounds of consultation. To address the semantic association requirements across text blocks in the query, a comprehensive analysis of the language mixing complexity of the text block sequence is conducted. This analysis examines the number of language types contained in the text block sequence, the frequency of alternation between different language text blocks, and the tightness of semantic association between cross-language expressions. If frequent alternation of multiple languages ​​is detected in the text and the core consultation intent is dispersed across different language text blocks, the language mixing complexity is determined to be high. Simultaneously, the total length of the text block sequence is calculated to determine the number of text blocks. Based on the above analysis results, the basic weight allocation strategy is fine-tuned. The higher the language mixing complexity, the higher the weight ratio of long-range dependency metrics is appropriately increased to strengthen the consideration of long-range semantic association across languages. The longer the text block sequence, the more the local and long-range weight ratios are adjusted to avoid an imbalance in semantic association judgment due to increased text length. Finally, adaptive weight parameters suitable for the current multilingual mixed consultation text are obtained for weighted fusion.

[0046] Adaptive weight adjustment; + + ,in, It is the overall smoothness score. It is a weighting coefficient for local coherence. It is average local coherence. It focuses on long-range dependencies. =0.6, It is the average long-range dependence. Emphasis on local coherence =0.7, This indicates global consistency.

[0047] Step 5.2 involves using adaptive weight parameters to perform a weighted summation of the local coherence metric and the long-range dependency metric to obtain a preliminary comprehensive semantic continuity score. This includes retrieving both the local coherence metric and the long-range dependency metric. The local coherence metric comprehensively reflects the degree of association between adjacent text blocks in dimensions such as semantic similarity, entity overlap, and differences in text surface features, for example, the local semantic fit between text blocks related to course practical content and text blocks related to the localized version of the textbook. The long-range dependency metric reflects the strength of attentional association, the impact of distance decay, and topic consistency between non-adjacent text blocks. The semantic relevance level, such as the long-range semantic dependency between text blocks related to practical course content and text blocks related to multilingual learning support, is assessed. These two measures are multiplied by their corresponding weights in the adaptive weighting parameters, where the local coherence measure is multiplied by the local weight coefficient and the long-range dependency measure is multiplied by the long-range weight coefficient. The two products are then summed to obtain a preliminary comprehensive score that can initially reflect the semantic continuity level of the entire multilingual mixed consultation text. The preliminary comprehensive score has fully considered the language mixing characteristics, text length, and semantic relevance of the current text, avoiding the scoring bias caused by traditional fixed weights.

[0048] Step 5.3: Based on historical scoring data or preset scoring normalization rules, the preliminary semantic continuity comprehensive score is scaled and normalized to ensure it falls within a preset standard scoring range, thus obtaining a standardized semantic continuity score. Specifically, this includes collecting semantic continuity scoring data from similar multilingual mixed consultation texts historically processed by the embedded intelligent consultation assistant of the cross-border education platform. This includes scoring results for consultation texts with different language mixing complexities and text lengths, constructing a historical scoring dataset. Simultaneously, preset scoring normalization rules are retrieved. These rules define the standard range for semantic continuity scoring and the basis for adjusting scores in different scenarios. The standard range is set using... Extensive experience in assessing the semantic coherence of cross-border educational consultation texts was conducted to ensure that the scores objectively reflect the actual semantic fluency of the texts. Based on historical scoring datasets, the distribution differences between the current preliminary comprehensive scores and historical scores of similar texts were analyzed. If the preliminary scores exceed the reasonable distribution range of historical scores, their scales are adjusted according to the statistical characteristics of historical data. At the same time, the preliminary comprehensive scores are mapped to the preset standard scoring range in strict accordance with the preset scoring standardization rules, eliminating the problem of inconsistent scoring scales in different batches and scenarios, ensuring that the scores are comparable across regions, and finally obtaining a standardized semantic continuity score. The semantic continuity score can accurately quantify the overall semantic coherence level of the current multilingual mixed consultation texts.

[0049] Step 5.4: Based on the standardized semantic continuity score, the corresponding local coherence measure and long-range dependency measure are used to obtain the final semantic continuity assessment report, which includes the overall score, sub-measures and coherence analysis summary. Specifically, this includes: using the standardized semantic continuity score as the core, integrating the corresponding local coherence measure and long-range dependency measure, and constructing a complete semantic continuity assessment data system. The overall score is standardized into a semantic continuity score, which intuitively presents the semantic coherence of the entire multilingual mixed consultation text. The sub-measures detail the specific values ​​of local coherence and long-range dependency metrics, as well as the weighting percentages of the two metrics in the weighted fusion, clearly demonstrating the contribution of different dimensions to the overall score. The coherence analysis summary delves into the semantic connection logic behind the score, including the core semantic fit points of adjacent text blocks and the long-range semantic connection paths of non-adjacent text blocks. For example, it clearly points out the semantic connection basis between the three core consultation intentions: course practical content, localized textbook version, and multilingual learning support. It analyzes the semantic connection between cross-language expressions, identifies key text block combinations for semantic coherence in the text, and identifies areas where semantic breaks may exist. Integrating this information system yields the final semantic continuity assessment report, which includes the overall score, sub-measures, and coherence analysis summary. This report provides a comprehensive and detailed semantic analysis basis for the embedded intelligent consultation assistant, helping it accurately grasp the contextual logic of users' multilingual mixed consultations, avoid fragmented responses, and ensure the coherence and relevance of the response.

[0050] In this embodiment of the invention, the following technical means are employed: First, a basic weight allocation strategy is obtained from a preset adaptive weight mapping table based on multilingual text types. This strategy is then combined with the complexity of language mixing and text length characteristics to fine-tune the weight parameters. Second, a preliminary comprehensive score is obtained by weighted summation of local coherence and long-range dependency metrics using adaptive weight parameters. Third, the preliminary score is scaled and normalized according to historical scoring data or preset rules to ensure it falls within a standard range. Finally, an evaluation report containing the overall score, sub-items, and a coherence analysis summary is generated based on the standardized score and corresponding sub-item metrics. Therefore, this approach effectively overcomes the limitations of traditional scoring methods where fixed weights cannot adapt to different text types and languages. The technical issues of mixed-language scenarios and text length differences, lack of unified scoring standards leading to poor comparability, and evaluation results containing only single scores without detailed analysis and supporting evidence have been addressed. This paper addresses these issues by achieving scenario-adaptive and standardized scoring of multilingual text semantic continuity. The scoring results not only meet the semantic coherence assessment needs of different types of multilingual texts but also possess a unified and comparable scale. Furthermore, comprehensive evaluation reports provide clear supporting evidence and analytical references for subsequent semantic understanding and response. This significantly improves the accuracy, practicality, and interpretability of multilingual contextual semantic continuity assessment, helping intelligent consultation assistants in scenarios such as cross-border education platforms to accurately grasp user needs and provide coherent responses.

[0051] like Figure 2 As shown, embodiments of the present invention also provide a lightweight multilingual contextual semantic continuity analysis system based on a localization model, including: The acquisition module is used to preprocess and adaptively segment the received multilingual text to be analyzed based on semantic integrity detection to obtain a text block sequence; The selection module is used to dynamically select the final lightweight semantic understanding model from a pre-built family of lightweight multilingual semantic understanding models based on the language features presented in the text block sequence and in combination with the real-time hardware resource status of the local device. The fusion module is used to encode text block sequences based on the final lightweight semantic understanding model. By extracting and fusing semantic features from different network depths of the final lightweight semantic understanding model, a multi-scale semantic vector sequence is obtained. The calculation module is used to calculate the semantic similarity, entity overlap and text surface feature differences between adjacent text blocks in a multi-scale semantic vector sequence, and obtain the calculation results. The combined calculation results are used to obtain the local coherence metric. Based on the multi-scale semantic vector sequence, combined with the attention weight generated by the final lightweight semantic understanding model when processing the text block sequence, the semantic association strength between non-adjacent text blocks is calculated, and the long-range dependency metric is obtained by introducing a distance decay factor and topic consistency detection. The processing module is used to assign appropriate adaptive weights to the local coherence measure and the long-range dependency measure according to the text type of the multilingual text; and to perform weighted fusion of the local coherence measure and the long-range dependency measure through adaptive weights to obtain the semantic continuity score of the overall contextual logical coherence and semantic fluency of the multilingual text.

[0052] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A lightweight multilingual contextual semantic continuity analysis method based on a localization model, characterized in that, The method includes: Based on semantic integrity detection, the received multilingual text to be analyzed is preprocessed and adaptively segmented to obtain a text block sequence; Based on the language features presented by the text block sequence and combined with the real-time hardware resource status of the local device, the final lightweight semantic understanding model is dynamically selected from a pre-built family of lightweight multilingual semantic understanding models. Based on the final lightweight semantic understanding model, the text block sequence is encoded, and semantic features from different network depths of the final lightweight semantic understanding model are extracted and fused to obtain a multi-scale semantic vector sequence. By using multi-scale semantic vector sequences, the semantic similarity, entity overlap, and text surface feature differences between adjacent text blocks in the sequence are calculated to obtain the calculation results. The results are combined to obtain the local coherence metric. Based on the multi-scale semantic vector sequences, combined with the attention weights generated by the final lightweight semantic understanding model when processing text block sequences, the semantic association strength between non-adjacent text blocks is calculated. The long-range dependency metric is obtained by introducing a distance decay factor and topic consistency detection. Based on the text type of the multilingual text, appropriate adaptive weights are assigned to the local coherence measure and the long-range dependency measure. The local coherence measure and the long-range dependency measure are then weighted and fused using the adaptive weights to obtain the semantic continuity score of the overall contextual logical coherence and semantic fluency of the multilingual text.

2. The lightweight multilingual contextual semantic continuity analysis method based on a localization model according to claim 1, characterized in that, Based on semantic integrity detection, the received multilingual text to be analyzed is preprocessed and adaptively segmented to obtain a text block sequence, including: The received raw text is subjected to language detection and encoding standardization processing to obtain language type recognition results and intermediate text with unified encoding format; Based on the language type recognition results, a locally pre-stored lightweight semantic boundary detection model is called to process the intermediate text in order to obtain the location information of potential sentence boundaries, phrase boundaries or topic transition boundaries. Based on location information and combined with preset semantic integrity rules, the boundaries are filtered and merged to finally determine the semantic unit boundaries used for segmentation. By using semantic unit boundaries, the intermediate text is segmented to obtain a sequence of text blocks consisting of multiple text blocks.

3. The lightweight multilingual contextual semantic continuity analysis method based on a localization model according to claim 2, characterized in that, Based on the language features presented by the text block sequence and combined with the real-time hardware resource status of the local device, the final lightweight semantic understanding model is dynamically selected from a pre-built family of lightweight multilingual semantic understanding models, including: Based on the text block sequence, the types of languages ​​included, the proportion of text blocks of each language, and the total length of the text block sequence are statistically analyzed to obtain a language feature vector describing the current analysis task; the real-time hardware resource status parameters of the local device are collected, including at least the available memory size, processor frequency, and current load, to obtain a resource status vector describing the device's real-time computing capabilities. The language feature vector and resource state vector are input into a preset dynamic selection decision model. The decision model selects a subset of candidate models from a pre-built model family that simultaneously meet the current language analysis requirements and the real-time resource constraints of the device, based on a predefined accuracy-efficiency balance strategy. By using a pre-trained lightweight performance predictor, the inference time and peak memory usage required to process the current text block sequence are predicted for each model in the candidate model subset. Combined with the preset model accuracy index, a multi-objective optimization algorithm selects the model with the highest comprehensive score as the final lightweight semantic understanding model.

4. The lightweight multilingual contextual semantic continuity analysis method based on a localization model according to claim 3, characterized in that, Based on the final lightweight semantic understanding model, text block sequences are encoded. By extracting and fusing semantic features from different network depths of the final lightweight semantic understanding model, a multi-scale semantic vector sequence is obtained, including: The text block sequence is input into the final lightweight semantic understanding model. Through forward propagation, the complete encoding result containing the activation values ​​of the intermediate and output layers of all text blocks is obtained. From the complete encoding results, network layer outputs of different depths of the final lightweight semantic understanding model are selected according to a predetermined strategy, and hierarchical feature vectors representing lexical, syntactic and discourse semantics are extracted respectively. Dimensionality reduction and normalization operations are performed on the hierarchical feature vectors to obtain intermediate semantic vectors with uniform scale and density. The intermediate semantic vectors are aggregated through weighted fusion or concatenation operations to obtain the final semantic vector representing the multi-scale semantic information fused by each text block. The vectors are then arranged in the order of the text blocks in the original sequence to form a multi-scale semantic vector sequence.

5. The lightweight multilingual contextual semantic continuity analysis method based on a localization model according to claim 4, characterized in that, By using multi-scale semantic vector sequences, the semantic similarity, entity overlap, and text surface feature differences between adjacent text blocks in the sequence are calculated to obtain the results. The combined results yield a local coherence metric, including: Based on the multi-scale semantic vector sequence, the cosine similarity of each pair of adjacent semantic vectors is calculated to obtain the semantic similarity calculation result; Using the same set of multi-scale semantic vector sequences associated with the semantic similarity calculation results, a lightweight entity recognition module is called to extract the named entity set corresponding to each text block and calculate the Jaccard similarity coefficient between the named entities of adjacent text blocks to obtain the entity overlap calculation result; and based on the original text block sequence corresponding to the semantic vector sequence, the edit distance between adjacent text blocks in terms of word composition and character sequence is directly calculated to obtain the text surface feature difference calculation result. The semantic similarity calculation result, entity overlap calculation result, and text surface feature difference calculation result are taken as inputs and fed into a pre-trained local coherence fusion model. The local coherence fusion model outputs a comprehensive scalar value as the final local coherence metric.

6. The lightweight multilingual contextual semantic continuity analysis method based on a localization model according to claim 5, characterized in that, Based on multi-scale semantic vector sequences, and combined with the attention weights generated by the final lightweight semantic understanding model when processing text block sequences, the semantic association strength between non-adjacent text blocks is calculated. This is then corrected by introducing a distance decay factor and topic consistency detection to obtain long-range dependency metrics, including: Based on the multi-scale semantic vector sequence and the attention weight matrix generated by the final lightweight semantic understanding model when processing text block sequences, the product of the semantic cosine similarity and attention association strength between all non-adjacent text block pairs is calculated to obtain the initial set of long-range semantic association strengths. Based on the long-range semantic association strength set, a distance decay factor related to the relative distance in the sequence of non-adjacent text blocks is introduced to weight and correct the association strength of each pair of non-adjacent text blocks, thus obtaining the long-range semantic association strength set. Based on the long-range semantic association strength set and text block sequence, a lightweight topic consistency detection module is called to perform topic consistency judgment on each pair of non-adjacent text blocks, obtain the judgment result, and perform a second correction on the association strength according to the judgment result to obtain the final long-range semantic association strength set after topic consistency correction. Statistically aggregate all association strength values ​​in the final long-range semantic association strength set, calculate the weighted average of the association strength values, and use the weighted average as the long-range dependency metric that represents the long-range semantic dependencies of the entire text.

7. The lightweight multilingual contextual semantic continuity analysis method based on a localization model according to claim 6, characterized in that, Based on the text type of the multilingual text, appropriate adaptive weights are assigned to the local coherence and long-range dependency measures. These adaptive weights are then used to weight and fuse the local coherence and long-range dependency measures to obtain a semantic continuity score for the overall contextual logical coherence and semantic fluency of the multilingual text, including: Based on the text type of the multilingual text, the corresponding basic weight allocation strategy is obtained from the preset adaptive weight mapping table, and fine-tuned by combining the language mixing complexity and length features of the text block sequence to obtain the final adaptive weight parameters used for weighted fusion. By using adaptive weight parameters, a weighted summation of local coherence and long-range dependency measures is calculated to obtain a preliminary comprehensive score for semantic continuity. Based on historical scoring data or preset scoring normalization rules, the initial semantic continuity comprehensive score is scaled and normalized to ensure that the semantic continuity comprehensive score falls within the preset standard scoring range, so as to obtain a standardized semantic continuity score. Based on the standardized semantic continuity score, corresponding local coherence and long-range dependency metrics, a final semantic continuity assessment report is obtained, which includes the overall score, sub-metrics, and a coherence analysis summary.

8. A lightweight multilingual contextual semantic continuity analysis system based on a localization model, wherein the system implements the method as described in any one of claims 1 to 7, characterized in that, include: The acquisition module is used to preprocess and adaptively segment the received multilingual text to be analyzed based on semantic integrity detection to obtain a text block sequence; The selection module is used to dynamically select the final lightweight semantic understanding model from a pre-built family of lightweight multilingual semantic understanding models based on the language features presented in the text block sequence and in combination with the real-time hardware resource status of the local device. The fusion module is used to encode text block sequences based on the final lightweight semantic understanding model. By extracting and fusing semantic features from different network depths of the final lightweight semantic understanding model, a multi-scale semantic vector sequence is obtained. The calculation module is used to calculate the semantic similarity, entity overlap and text surface feature differences between adjacent text blocks in a multi-scale semantic vector sequence, and obtain the calculation results. The combined calculation results are used to obtain the local coherence metric. Based on the multi-scale semantic vector sequence, combined with the attention weight generated by the final lightweight semantic understanding model when processing the text block sequence, the semantic association strength between non-adjacent text blocks is calculated, and the long-range dependency metric is obtained by introducing a distance decay factor and topic consistency detection. The processing module is used to assign appropriate adaptive weights to the local coherence measure and the long-range dependency measure according to the text type of the multilingual text; and to perform weighted fusion of the local coherence measure and the long-range dependency measure through adaptive weights to obtain the semantic continuity score of the overall contextual logical coherence and semantic fluency of the multilingual text.

9. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images