Localizable intelligent multilingual spoken language partner training method, system and equipment

Abstract

The invention discloses a localizable intelligent multilingual spoken language partner training method, system and equipment, and belongs to the crossing field of intelligent language learning equipment and wireless communication technology.The method comprises the steps that firstly, user voice in the external environment is collected, and hardware-level switching is conducted between a Bluetooth mode and a 4G independent mode; in the 4G mode, real-time positioning is supported, a spoken language scene is recommended based on the position, and three partner training modes of real-time dialogue, scene presetting and new word prompting are provided; user learning data generated in the mode is synchronously stored in the cloud; and when the Bluetooth mode is switched to be connected with the mobile phone, the mobile phone APP can request and synchronously display complete historical learning data to form a continuous learning track. According to the invention, the functions of independent communication, positioning perception, scenarized learning and secure communication are integrated, the problems of strong equipment dependence, disjunction between learning contents and scenes, lack of security design and poor data continuity in the prior art are solved, and all-weather and all-scene intelligent spoken language learning and security guarantee are realized.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of intelligent language learning devices and wireless communication technology, and in particular relates to a location-based intelligent multilingual oral practice method, system and device. Background Technology

[0002] With the acceleration of globalization and the widespread adoption of mobile communication technology, the demand for anytime, anywhere, personalized, and scenario-based oral English learning is growing rapidly. Existing oral English practice solutions mainly suffer from the following shortcomings:

[0003] The first type is based on smartphone applications, which rely entirely on the phone for recording, processing, and interaction. Its limitations include: the phone must be carried and operated at all times, making it inconvenient and potentially dangerous in situations such as sports or driving; the phone's microphone and speaker are separate, resulting in poor voice interaction quality in noisy environments; and functionality is limited by the phone's network and battery life, making true "phone-free" learning impossible.

[0004] The second type is a lightweight solution based on Bluetooth headsets, which enables simple voice interaction by pairing with a mobile phone. Its disadvantages are: its functionality relies entirely on the mobile phone's Bluetooth connection, and it becomes ineffective once disconnected or the phone is not nearby; Bluetooth communication range is limited, making it unsuitable for independent use over a large area; and it lacks autonomous positioning and cellular network communication capabilities, thus failing to provide location-based services and emergency communication functions.

[0005] The third category is dedicated language learning devices, which usually have offline content, but lack interactive intelligence, are inconvenient to update, and most of them do not have modern mobile communication and positioning capabilities, making it difficult to provide a dynamic and contextualized learning experience and necessary security guarantees.

[0006] Furthermore, existing technologies generally lack safety features specifically designed for user groups such as children, the elderly, or outbound travelers. When users are using learning devices independently, they may be unable to quickly contact the outside world in case of an emergency, creating a safety blind spot.

[0007] Therefore, there is an urgent need for a wearable device and system that integrates independent cellular communication, high-precision positioning, intelligent training, and secure communication to solve the problems of strong device dependence, disconnect between scenarios and locations, and lack of security functions in existing technologies. Summary of the Invention

[0008] Purpose of the Invention: The purpose of this invention is to provide a location-based intelligent multilingual oral practice method, system, and device. This method and system integrate 4G and Bluetooth dual-mode communication and positioning modules, and innovatively introduces a hardware DIP switch and one-button calling function, aiming to achieve the following objectives:

[0009] It offers two flexible working modes: Bluetooth connection to mobile phones and independent 4G network connectivity, balancing performance and portability;

[0010] Utilizing real-time location information, it provides spoken language learning content that is deeply integrated with the user's geographical environment and cultural background;

[0011] In stand-alone mode, it provides reliable one-button emergency call capability, enhancing user safety;

[0012] Build an intelligent wearable learning terminal that integrates learning, positioning, and communication.

[0013] Technical Solution: The present invention provides a location-based intelligent multilingual oral practice method, wherein the method is executed collaboratively by a wearable device and a mobile terminal integrating 4G and Bluetooth dual-mode communication and positioning modules, and includes the following steps:

[0014] Step 1: Collect user voice from the external environment through the wearable device, and perform voice enhancement processing on the collected voice. The voice enhancement processing includes echo cancellation (AEC), automatic gain control (AGC), and background noise suppression (ANS).

[0015] Step 2: Execute 4G mode or Bluetooth mode according to the communication mode selected by the user via the DIP switch;

[0016] Step 3: In 4G mode, execute the corresponding coaching process according to the coaching mode selected by the user; the coaching process includes real-time coaching mode, preset scenario coaching mode, and prompt word scenario coaching mode.

[0017] Step 4: During the coaching process, the user initiates a one-click call via physical or virtual buttons on the device to establish a voice call connection with the pre-set paired mobile phone.

[0018] Step 5: Record and analyze user interaction data, and generate learning suggestions based on the analysis results. The learning suggestions include recommendations for switching or associating between different coaching modes.

[0019] Step 6: Display the real-time location of the wearable device via the mobile app;

[0020] Step 7: In 4G mode, the user's practice process data, learning records, and assessment results are synchronously stored on the cloud server. When the device switches to Bluetooth mode and connects to the mobile APP, the mobile APP sends a data synchronization request to the cloud server to obtain and display the user's complete historical learning data.

[0021] Furthermore, in step 1, the echo cancellation (AEC) employs an adaptive filter algorithm, and its error signal calculation formula is as follows:

[0022]

[0023] in, This is the error signal after echo cancellation. The original input signal, This is the predicted output signal of the filter;

[0024] The automatic gain control (AGC) dynamically adjusts the gain based on the root mean square (RMS) value of the signal. The gain calculation formula is as follows:

[0025]

[0026] in, The signal after dynamic gain adjustment. The target gain value, The root mean square value of the input signal;

[0027] The background noise suppression ANS uses spectral subtraction for noise suppression, which is achieved by modeling and subtracting the background noise spectrum, as shown in the formula:

[0028]

[0029] in, The signal spectrum after denoising. The original signal spectrum, This is the estimated noise spectrum.

[0030] Furthermore, in step 2, in the Bluetooth mode, the processed voice data is sent to the mobile APP software via the Bluetooth Low Energy (BLE) protocol. The cloud server performs voice recognition, dialogue generation, and audio synthesis, and returns the generated training content to the wearable device for playback. The specific processes for voice recognition, dialogue generation, and audio synthesis include:

[0031] Step 2.1, Speech Recognition: After receiving the speech data, the mobile app first uses an end-to-end speech recognition model deployed in the cloud to convert the speech signal into text. The model adopts an encoder-decoder structure based on an attention mechanism. The encoder extracts speech features, and the decoder outputs the corresponding text sequence, supporting real-time conversion of multiple languages. The speech recognition model is represented as follows:

[0032]

[0033] in, The input voice signal. For the target statement, The conditional probabilities of speech and sentences given for the acoustic model. These are the probabilities of the language model, obtained through training on a large-scale corpus. Let X be the marginal probability of the speech signal X, which is a constant during the decoding process and does not affect the maximum a posteriori probability estimation.

[0034] Step 2.2, Natural Language Understanding and Dialogue Generation: The recognized text is analyzed by the natural language understanding module to understand the user's intent and semantics. Combined with the context memory unit and the dialogue state tracking mechanism, the current dialogue topic and user needs are determined. Subsequently, based on the pre-trained generative dialogue model or retrieval dialogue engine, a response text in the target language that is context-appropriate and grammatically correct is generated.

[0035] Step 2.3, Audio Synthesis: The generated response text is input into the neural speech synthesis model. The model uses a Transformer-based acoustic model and vocoder to convert the text into a highly natural target language speech waveform with emotional intonation.

[0036] Step 2.4, Content Return: The synthesized audio stream is transmitted back to the wearable device in real time via Bluetooth link for playback, while the dialogue text and pronunciation evaluation prompts are displayed synchronously on the mobile APP interface.

[0037] Furthermore, in step 2, in the 4G mode, the wearable device works independently of the mobile phone, uploading voice data to the cloud server through the 4G communication module. The cloud server processes the data and sends the generated training content to the wearable device for playback.

[0038] Furthermore, in step 4, the one-click call includes the following steps:

[0039] Trigger a call command by pressing a button on the device;

[0040] Initiate a voice call request to a preset mobile phone number via 4G network;

[0041] The current coaching session is automatically paused during the call and resumed or prompted to continue after the call ends.

[0042] Furthermore, the one-click call also supports voice triggering. After the user says the preset wake-up word, they can say the phone number to make a call. During the call, the device automatically enables voice enhancement processing to ensure call clarity.

[0043] Furthermore, in step 5, the learning suggestions specifically include:

[0044] Step 5.1: Switch from real-time coaching mode to preset scenario mode:

[0045] When the system detects that the user's pronunciation error rate is higher than the preset threshold or the fluency of the conversation is poor during real-time tutoring, it will automatically recommend entering the preset scenario mode to carry out targeted reinforcement training for high-frequency error words or sentence patterns.

[0046] Step 5.2: Switch from the preset scene mode to the real-time coaching mode:

[0047] When a user completes one or more sets of scenario exercises in a preset scenario mode and the pronunciation accuracy and fluency reach the preset standards, the system recommends entering the real-time coaching mode, encouraging the user to conduct comprehensive ability training in a dialogue environment without a fixed script.

[0048] Step 5.3: Switch from real-time coaching mode or preset scenario mode to prompt word scenario mode:

[0049] When a user actively requests personalized practice, or when the system analyzes historical data and finds that the user has persistent weaknesses in specific topics, grammar, or vocabulary, it recommends entering the prompt word scenario mode to generate customized practice content based on the user's interests or ability shortcomings.

[0050] Step 5.4, Related Recommendation Content Generation Mechanism:

[0051] Based on the user's historical training data, real-time performance, personal preferences, and geographic location information, the system intelligently generates cross-mode navigation suggestions and dynamically adjusts the difficulty and topic of the recommended content. It employs a weighted collaborative filtering model for dynamic personalized recommendations, integrating user historical behavior and real-time location information. Its prediction scoring formula is as follows:

[0052]

[0053] in, For users For the scene Predicted interest scores For users and Behavioral similarity between them For users For the scene Actual interaction rating, The average rating of user u across all scenarios. Let u be the set of similar users. The average rating given by user v for all scenarios.

[0054] Furthermore, in step 7, the data synchronization request specifically involves: when the mobile app initiates a synchronization request in Bluetooth mode, after verifying the user's identity, encrypting and transmitting the corresponding historical data to the mobile app, and displaying it in the form of charts, lists, or reports.

[0055] The present invention also discloses a location-based intelligent multilingual oral practice system, including a wearable device integrating 4G and Bluetooth dual-mode communication and positioning modules, an optional connected mobile terminal and a cloud server;

[0056] The wearable device includes:

[0057] Voice acquisition unit;

[0058] Voice processing unit;

[0059] 4G and Bluetooth dual-mode communication and positioning unit, supporting mobile network connection, Bluetooth data transmission and real-time positioning;

[0060] The status switching module responds to DIP switch operations and controls the device to work in Bluetooth mode or 4G mode.

[0061] The call control unit is used to respond to one-click call commands in 4G mode and initiate voice calls with the paired mobile phone.

[0062] Voice playback unit;

[0063] The mobile terminal has an application installed to provide a complete coaching service and interface interaction in Bluetooth mode;

[0064] The cloud server is used to store the user's learning data in 4G mode and respond to the synchronization request of the mobile APP in Bluetooth mode, sending the user's historical learning data to the mobile APP for visualization.

[0065] The present invention also discloses a wearable device, including a housing, a magnetic or clamping structure disposed on the housing, and a voice acquisition unit, a voice processing unit, a voice playback unit, a 4G and Bluetooth dual-mode communication and positioning unit, a state switching module, and a call control unit integrated within the housing; the device surface is provided with a DIP switch and a call button.

[0066] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0067] 1. This invention pioneers a hardware-level seamless dual-mode switching mechanism: It achieves fast and reliable hardware-level switching between 4G and Bluetooth modes via a physical DIP switch, offering intuitive operation and instant response. Users can flexibly choose according to the scenario: use Bluetooth mode indoors or when the phone is nearby to save device power and utilize the phone's powerful computing capabilities; switch to 4G mode outdoors or when independent use is required, completely freeing users from the constraints of their phones and enabling true mobile learning.

[0068] 2. Enables contextualized learning based on real-world geolocation: In 4G standalone mode, the device's integrated high-precision positioning module can acquire the user's location in real time. The cloud system combines this location information to intelligently recommend spoken language scenarios and vocabulary related to that geographical location, cultural environment (such as airports, museums, restaurant districts), or local hot events. This transforms learning content from "static presets" to "dynamic generation," greatly enhancing the immersion, practicality, and enjoyment of learning.

[0069] 3. Embedded secure communication extends core value: In stand-alone mode, the device provides a hardware-level one-button emergency call function. When users (especially children, the elderly, or travelers) encounter difficulties or dangers, they can quickly establish a voice call with family members or emergency contacts through preset buttons, giving the product a crucial safety guarantee attribute and broadening its application scenarios and user base.

[0070] 4. Constructing an all-weather, all-scenario intelligent learning loop: The system deeply integrates four major functions: high-quality audio interaction, multi-mode intelligent tutoring, real-time location perception, and secure communication, creating a wearable intelligent terminal that integrates learning, navigation, and safety. It is not only a language learning tool, but also an intelligent assistant for users in unfamiliar environments.

[0071] 5. Achieve seamless synchronization of learning data across modes: Learning data generated in 4G standalone mode is automatically stored in the cloud. When switching to Bluetooth mode and connecting to a mobile phone, historical data can be fully synchronized to the mobile APP, forming a continuous learning record and growth trajectory, improving user experience and learning continuity. Attached Figure Description

[0072] Figure 1 This is a schematic diagram of the overall architecture of the location-enabled intelligent multilingual oral practice system provided in an embodiment of the present invention;

[0073] Figure 2 A flowchart of the Bluetooth mode operation provided in this embodiment of the invention;

[0074] Figure 3 A flowchart of the 4G standalone mode workflow provided in this embodiment of the invention;

[0075] Figure 4 This is a flowchart of the one-click call function provided in an embodiment of the present invention;

[0076] Figure 5 A flowchart illustrating a location-based speech recommendation scenario provided in an embodiment of the present invention;

[0077] Figure 6 This is a schematic diagram of the wearable device structure provided in an embodiment of the present invention;

[0078] Figure 7 The flowchart illustrates the workflow of three training modes under 4G conditions provided in this embodiment of the invention.

[0079] Figure 8 This is a flowchart illustrating the learning suggestion generation and mode transition recommendation process for implementing this invention.

[0080] Figure 9 This is a flowchart illustrating the cross-modal learning data synchronization process provided in an embodiment of the present invention. Detailed Implementation

[0081] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0082] Example 1: System Overall Architecture

[0083] The core of the location-based intelligent multilingual oral practice solution provided by this invention lies in a collaborative system of "end (wearable device) - cloud (mobile app)". For example... Figure 1 As shown, system 100 mainly includes a wearable device 110 and a training application (App) running on a mobile terminal 120 (such as a smartphone). The communication control module of wearable device 110 can control the communication connection method, which is divided into two types: 1) connecting to mobile terminal 120 via wireless communication (such as Bluetooth); 2) connecting directly to cloud services via a 4G module. These two connection methods are mutually exclusive.

[0084] The wearable device 110 is designed to be worn on the user's chest, secured by means such as magnets, clips, or lanyards. Its core function is as a high-quality audio interaction front-end: a built-in voice acquisition unit (such as a directional microphone array) clearly captures the user's voice at close range; a voice processing unit performs echo cancellation (AEC), automatic gain control (AGC), and background noise suppression (ANS) to ensure the quality of uploaded voice data; a voice playback unit (such as a miniature speaker) plays training audio from the app; a communication unit handles data transmission and reception with the mobile app; a 4G communication module connects to cloud services and enables data transmission and reception; a communication control module switches communication modes; and a call control unit connects to the control phone via the 4G module to enable call functionality.

[0085] The App on the mobile terminal 120 carries the main intelligent processing and business logic. It includes a pattern management interface, a speech recognition / synthesis engine, a machine translation module, a dialogue generation model, a scene construction engine, and a user evaluation module. The App receives clean speech data from the wearable device 110, performs in-depth processing, and sends the generated training audio or control commands back to the wearable device 110 for playback.

[0086] Example 2: Bluetooth Mode Workflow

[0087] like Figure 2 As shown, the Bluetooth mode workflow begins with the user switching the physical DIP switch on the device to Bluetooth mode.

[0088] Subsequently, the device automatically or manually establishes a BLE Bluetooth connection with the user's smartphone, which has the accompanying application installed. Once the connection is successful, the user can begin practicing speaking. As the user speaks, the device's built-in microphone array captures the voice signal, which is immediately enhanced in real time by the local speech processing unit, including echo cancellation, automatic gain control, and background noise suppression to improve voice quality.

[0089] The processed, clean audio data is transmitted in real-time to the mobile application via a BLE Bluetooth link. Upon receiving the audio stream, the mobile app further uploads it to a cloud server, leveraging the cloud's powerful computing capabilities for deep speech recognition, natural language understanding, and intelligent dialogue generation. The generated speech translation and scenario-based practice content are then synthesized into high-quality audio files or streams either from the cloud or locally on the mobile device.

[0090] Finally, the generated practice audio is transmitted back to the wearable device via Bluetooth and played to the user through the device's built-in miniature speaker; the generated text is displayed on the app. During this process, users can flexibly select different practice modes, view learning progress, and access analysis reports through the mobile app's graphical interface.

[0091] Furthermore, in Bluetooth mode, the mobile app can proactively initiate a historical data synchronization request to the cloud server. After verifying the user's identity, the cloud server encrypts and transmits all learning records, practice scores, scene history, and other data generated by the user in 4G standalone mode to the mobile app. Upon receiving the data, the app displays it to the user in a visual format (such as learning curves, report cards, and scene maps), achieving seamless integration of learning records across modes.

[0092] Example 3: 4G Standalone Mode Workflow

[0093] like Figure 3 As shown, when the user needs to use the device independently without the phone, they can switch the DIP switch to 4G mode to automatically disconnect the Bluetooth connection with the phone and enter independent working mode.

[0094] In this mode, the device first obtains the user's real-time geographical location information (such as GPS coordinates) through the integrated 4G communication and positioning module. When the user practices speaking, the device collects and enhances the voice, and no longer relies on Bluetooth, but directly uploads the voice data along with the location information to the cloud server via the built-in 4G module in encryption.

[0095] After receiving the data, the cloud server first performs keyword recognition to determine whether the user is modifying configurations or performing a coaching task. If it is modifying configurations, it will adjust the coaching mode, scene selection, etc., and return a success or failure notification tone.

[0096] If no keywords are identified, core processing is executed. On one hand, the speech is recognized and semantically analyzed to generate personalized practice dialogue content; on the other hand, based on the uploaded real-time location information, oral learning scenarios related to that geographical location are matched from the scene database (for example, if the user is near an airport, dialogue scenarios such as check-in, security check, and boarding are recommended). The cloud dynamically combines the generated intelligent practice content with the scene recommendations and sends instructions and audio data to the device via the 4G network.

[0097] During this process, the cloud server records the user's practice data in real time, including practice time, scenario type, dialogue content, pronunciation score, and incorrect vocabulary, and stores it in the user learning database categorized by user ID. This data continues to accumulate in 4G mode, building a complete learning profile for the user.

[0098] Example 4: One-click call function process

[0099] like Figure 4 As shown, during 4G independent coaching, if the user encounters an emergency or needs assistance, they can use the safe call function.

[0100] Users can trigger a call in two ways: first, by briefly pressing the dedicated physical "one-click call" button on the device to call a pre-stored mobile phone number; second, by saying the preset voice wake-up word and then saying the phone number to confirm the contact.

[0101] Upon triggering the call command, the device immediately suspends any ongoing training tasks to ensure call priority. Subsequently, the call control unit initiates a standard voice call request to the emergency contact's mobile phone via the 4G cellular network.

[0102] Once the call is established, the device's voice processing unit automatically activates an enhancement algorithm tailored to the call scenario, ensuring clear and smooth communication for both parties. After the call ends, the device will ask the user via voice prompt, "Do you want to resume learning?" If the user confirms, the tutor will seamlessly resume from where it was paused; if the user cancels, the current session will end.

[0103] Example 5: Location-based Recommendation of Spoken Language Scenarios

[0104] like Figure 5 As shown, in 4G standalone mode, the location-based service process continues to run in the background, providing users with dynamic learning content.

[0105] The device proactively obtains its precise location via GPS / BeiDou or base station positioning at configurable time intervals (e.g., every 5 minutes) or after detecting that the user has moved a certain distance. This location information is then uploaded to a cloud-based intelligent recommendation engine in real time.

[0106] The cloud-based engine matches the user's current location with a vast database of geographic information tags and spoken language scenarios. For example, when the system identifies that the user is near a famous museum, it can automatically recommend relevant multilingual dialogue practice such as "buying tickets," "asking about exhibits," and "discussing history."

[0107] After receiving a scenario recommendation prompt, users can choose to accept the recommendation, and the system will automatically guide them into that scenario for immersive oral practice; or they can ignore the recommendation and continue with their original learning plan. User choices and feedback will be recorded to continuously optimize future recommendation accuracy, achieving a personalized learning path that becomes "smarter with use."

[0108] Figure 6 A specific embodiment of the wearable device 110 is shown. A back clip 602 is provided on the housing 601 for attaching clothing. Internally, it integrates a main processing chip (voice processing unit) 603, a microphone (voice acquisition unit) 604, a speaker (voice playback unit) 605, a Bluetooth module (communication unit) 606, a 4G module (communication unit) 607, a Bluetooth / 4G module switch 608, and a one-button wake-up button 609. This device is optimized for clear pickup of voices spoken to the chest and comfortable playback near the ear.

[0109] Example 6: Three training modes under 4G conditions

[0110] In 4G standalone mode, the system supports three training modes, which users can select and switch between via voice wake-up words or device buttons. The following will be combined with the attached... Figure 7 Detailed description.

[0111] I. Real-time coaching process

[0112] This mode simulates a "conversation partner" who is always online and can respond to the user's messages in real time.

[0113] S301: The user speaks a sentence into the wearable device 110 (e.g., "The weather is nice today" in Chinese). The voice acquisition unit of the wearable device 110 acquires the voice, and the voice processing unit immediately performs enhancement processing (eliminating environmental noise and adjusting volume).

[0114] S302: The processed voice data is sent to the mobile app via Bluetooth.

[0115] S303: After receiving the voice data, the App initiates real-time coaching processing. The end-to-end speech conversion model takes the source language voice data as direct input and directly generates the coaching voice waveform in the target language through a deep neural network.

[0116] S304: The generated training audio data is sent from the App back to the wearable device 110.

[0117] S305: The voice playback unit of the wearable device 110 plays the foreign language audio. The user can then hear a natural response, as if from a conversational partner.

[0118] II. Scenario Selection and Training Process

[0119] This model performs deep learning for specific topics (such as restaurant ordering, hotel check-in, and business meetings).

[0120] S401: Users set up scenarios (such as "restaurant mode") using wake words.

[0121] S402: The cloud server generates a complete set of learning materials based on the user-selected scenario. First, it generates a scenario description including the location, characters, and dialogue targets. Then, based on this description, it uses a dialogue generation model to create a logically coherent, idiomatic multi-turn dialogue text in the target Chinese language.

[0122] S403: The user enters the "follow-along practice" stage. The device automatically plays the generated sentence, and then the user listens to the demonstration pronunciation played by the device.

[0123] S404: After each sentence is played, the wearable device 110 collects the user's voice recording and sends it back to the cloud server.

[0124] S405: The cloud server recognizes the user's spoken words and performs speech recognition to obtain the text spoken by the user. It then compares this text with the played audio text to evaluate the pronunciation accuracy of each word. If correct, it plays the next sentence; otherwise, it repeats the sentence.

[0125] S406: After a round of dialogue practice, the user's overall score and pronunciation improvement suggestions can be played.

[0126] III. Prompt Word Practice Process

[0127] This mode provides practice and learning based on prompts given by the user.

[0128] S501: The user enters this mode by using a wake word, says a prompt word, and initiates "prompt word practice".

[0129] S502: The cloud server performs in-depth analysis based on the prompt words selected by the user: through dictionary API and corpus analysis, it obtains the part of speech (noun), common collocations, applicable contexts (business, diplomacy, daily bargaining) and cultural background.

[0130] S503: Based on the analysis results, construct a customized oral practice scenario. For example, generate the following scenario description for "advanced": "Two children are exchanging information about advanced things."

[0131] S504: Similar to the preset scenario process, the App generates a word including "advanced" and related expressions based on this customized description, and pronounces them in sequence for the user to learn.

[0132] S505: After each sentence is played, the wearable device 110 collects the user's voice recording and sends it back to the cloud server.

[0133] S506: The cloud server recognizes the user's spoken words and performs speech recognition to obtain the text spoken by the user. It then compares this text with the played audio text to evaluate the pronunciation accuracy of each word. If correct, it plays the next sentence; otherwise, it repeats the sentence.

[0134] S507: After a round of dialogue practice, the user's overall score and pronunciation improvement suggestions can be played.

[0135] Example 7: Cross-modal learning data synchronization process

[0136] This embodiment describes a seamless synchronization mechanism for learning data between 4G mode and Bluetooth mode, such as... Figure 9 As shown, the specific process is as follows:

[0137] S701: When users engage in spoken language practice in 4G standalone mode, all interactive data (including voice recordings, scene selections, pronunciation scores, and location information) are uploaded to the cloud server in real time and stored in the user learning database according to the user account ID.

[0138] S702: When the user ends 4G mode, they can switch the device's DIP switch to Bluetooth mode and establish a Bluetooth connection with the mobile app.

[0139] S703: After the mobile APP detects the device mode switch and successfully connects, it automatically or according to the user's instructions sends a historical data synchronization request to the cloud server. The request includes user authentication information.

[0140] S704: The cloud server verifies the legitimacy of the user's identity and retrieves all historical learning data generated by the user in 4G mode from the user's learning database.

[0141] S705: The cloud server encrypts and packages the retrieved learning data (which may include structured learning reports, original audio clips, scoring details, scene trajectories, etc.) and transmits it to the mobile APP via the Internet.

[0142] S706: After receiving and decrypting the data, the mobile APP parses and reassembles it, and displays it to the user in various visual formats, including but not limited to: learning time curves, pronunciation accuracy statistics, historical list of scene practice, geographical location learning map, and vocabulary mastery progress.

[0143] S707: Users can view detailed learning reports through a mobile app to understand their learning outcomes in 4G standalone mode, and adjust their subsequent learning plans or select recommended learning paths based on historical data.

[0144] Example 7 also includes a learning suggestion generation and pattern switching recommendation mechanism, such as Figure 8 As shown, the specific process is as follows:

[0145] When analyzing user learning data, the system monitors user performance metrics in the current mode in real time, including pronunciation accuracy, fluency, error frequency, and scene completion rate. Based on this data, the system generates intelligent learning suggestions according to preset transition conditions and algorithms, and recommends users to switch to a more suitable practice mode at appropriate times via voice prompts or push notifications through the mobile app interface. The specific transition logic includes:

[0146] (1) Real-time tutoring mode → Preset scenario mode: When the system detects that the user is making continuous mistakes in a certain type of sentence pattern or vocabulary, it will automatically recommend entering the relevant preset scenario for special practice;

[0147] (2) Preset scenario mode → Real-time coaching mode: When the user achieves the learning goal in the scenario, the system encourages the user to enter the real-time mode for practical dialogue;

[0148] (3) Real-time / Preset Scene Mode → Prompt Word Scene Mode: When the user expresses interest or the system detects a knowledge gap, it is recommended to use the prompt word mode for personalized extended training;

[0149] (4) Dynamic adjustment of recommended content: The system combines information such as user's historical performance, geographical location, and learning preferences to update recommended content in real time, ensuring the personalization and efficiency of the learning path.

[0150] This mechanism is related to the cross-modal data synchronization process ( Figure 9 Collaborative work forms a closed loop of "learning-assessment-recommendation-synchronization," enhancing the user's learning experience and effectiveness.

[0151] Example 8: Implementation details of the speech enhancement processing algorithm and its role in the system

[0152] In speech enhancement processing, echo cancellation (AEC), automatic gain control (AGC), and background noise suppression (ANS) are fundamental to achieving high-quality voice interaction. This processing can be implemented using the following algorithms:

[0153] Echo cancellation (AEC): Employs adaptive filter algorithms, such as the Least Mean Square Error (LMS) algorithm, to update filter weights in real time to eliminate echoes. The error signal is calculated using the following formula:

[0154]

[0155] in, This is the error signal after echo cancellation. The original input signal, This is the predicted output signal of the filter. The algorithm ensures that while the device plays the practice audio, it can effectively separate and eliminate echoes, avoiding interference from the speech signal in the local loop and providing a clean input source for subsequent speech recognition.

[0156] Automatic gain control (AGC): Dynamically adjusts the gain based on the root mean square value of the signal to ensure the speech signal remains within an appropriate range. The gain calculation formula is:

[0157]

[0158] in, The target gain value, This is the root mean square value of the input signal. The algorithm can adapt to different user speaking volumes and environmental changes, avoiding the impact of excessively weak or strong speech signals on recognition accuracy and improving the system's robustness in complex environments.

[0159] Background noise suppression (ANS): This method uses spectral subtraction to suppress noise by modeling and subtracting the background noise spectrum, thereby improving speech intelligibility. The basic formula is:

[0160]

[0161] in, The signal spectrum after denoising. The original signal spectrum, The noise spectrum is estimated. This algorithm is particularly suitable for noisy environments such as outdoors and public places, and can significantly reduce the impact of background noise on speech acquisition and recognition, ensuring the clarity and coherence of the training dialogue.

[0162] In summary, the speech enhancement algorithm described in this embodiment is the key technical foundation supporting the high-precision speech acquisition and interaction of this invention in various environments, and is directly related to the accuracy of subsequent speech recognition, dialogue generation and user evaluation, as well as user experience.

[0163] Example 9: Implementation details of speech recognition and dialogue generation algorithms and their role in the system

[0164] In the speech recognition process, the cloud server employs a speech recognition model based on deep neural networks, such as convolutional neural networks (CNNs) or recurrent neural networks (RNNs), to achieve high-accuracy speech-to-text conversion. The model is trained using the maximum likelihood estimation method, and its probability model can be expressed as:

[0165]

[0166] in, The input voice signal. For the target statement, The conditional probabilities of speech and sentences given for the acoustic model. The probabilities of the language model are obtained through training on a large-scale corpus. This model, through end-to-end learning, can effectively capture the temporal and semantic features in speech signals and accurately convert users' speech into text information.

[0167] This recognition result is a prerequisite for all subsequent intelligent coaching functions:

[0168] In real-time coaching mode, the recognized text is directly input into the end-to-end speech conversion model to generate a speech response in the target language.

[0169] In preset scenarios and prompt word scenario modes, the recognized text is used to understand user intent, assess pronunciation accuracy, and drive the dialogue generation engine to build coherent and authentic practice content.

[0170] In a voice-triggered scenario for one-click calling, the recognized text is used to parse the phone number spoken by the user, enabling voice dialing.

[0171] Therefore, the speech recognition and probability model described in this embodiment is the core bridge connecting user voice input and system intelligent feedback, and its performance directly determines the response speed, accuracy and naturalness of the coaching.

[0172] Example 10: Implementation details of a location-based learning-based scene recommendation algorithm and its role in the system

[0173] In location-based learning-based scenario recommendation, the system employs a weighted collaborative filtering model for dynamic personalized recommendations. This algorithm effectively integrates user historical behavior and real-time location information, and its prediction scoring formula is as follows:

[0174]

[0175] in For users For the scene Predicted interest scores For users and Behavioral similarity between them (calculated based on historical practice records, scene preferences, etc.) For users For the scene The algorithm assesses actual interaction scores. By analyzing which scenarios similar users prefer in similar locations, it achieves personalized and contextualized content recommendations.

[0176] This recommendation mechanism is deeply integrated with the location awareness capability of this invention:

[0177] Real-time location input: The device obtains GPS / base station positioning data through 4G / Bluetooth dual-mode communication and positioning unit, which serves as the key context input for the recommendation algorithm.

[0178] Scene matching: The system maintains a spoken scene database labeled with geographical locations (such as "airport check-in", "restaurant ordering", "museum tour" etc.). The algorithm filters out a set of candidate scenes with high geographical relevance based on the user's current location.

[0179] Personalized ranking: In the above candidate set, a collaborative filtering model is used in conjunction with the user's historical preference data to perform personalized scoring and ranking of the scenarios, and the scenarios with the highest scores are recommended to the user first.

[0180] Dynamic updates: User feedback data, such as acceptance of recommended scenarios, ignoring of recommendations, and completion of practice, will be updated in real time to optimize subsequent recommendations, forming a virtuous cycle of "the more you use it, the more accurate it becomes".

[0181] Therefore, the recommendation algorithm described in this embodiment is the technical engine for realizing the core innovation of "scenario-based learning based on real geographical location". It transforms static learning content into a dynamic, personalized learning experience that is deeply integrated with the user's environment, significantly improving the immersion, practicality and user stickiness of learning.

[0182] In summary, this invention, through innovative hardware design and system integration, organically integrates intelligent oral practice, real-time location tracking, and emergency communication functions into a wearable device, creating a safe, convenient, and scenario-based next-generation language learning solution. It not only improves learning efficiency but also expands the product's application value and user base.

[0183] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A location-based intelligent multilingual oral practice method, characterized in that, The method is executed collaboratively by a wearable device and a mobile terminal that integrate 4G and Bluetooth dual-mode communication and positioning modules, and includes the following steps: Step 1: Collect user voice from the external environment through the wearable device, and perform voice enhancement processing on the collected voice. The voice enhancement processing includes echo cancellation (AEC), automatic gain control (AGC), and background noise suppression (ANS). Step 2: Execute 4G mode or Bluetooth mode according to the communication mode selected by the user via the DIP switch; Step 3: In 4G mode, execute the corresponding coaching process according to the coaching mode selected by the user; the coaching process includes real-time coaching mode, preset scenario coaching mode, and prompt word scenario coaching mode. Step 4: During the coaching process, the user initiates a one-click call via physical or virtual buttons on the device to establish a voice call connection with the pre-set paired mobile phone. Step 5: Record and analyze user interaction data, and generate learning suggestions based on the analysis results. The learning suggestions include recommendations for switching or associating between different coaching modes. Step 6: Display the real-time location of the wearable device via the mobile app; Step 7: In 4G mode, the user's practice process data, learning records, and assessment results are synchronously stored on the cloud server. When the device switches to Bluetooth mode and connects to the mobile APP, the mobile APP sends a data synchronization request to the cloud server to obtain and display the user's complete historical learning data.

2. The location-based intelligent multilingual oral practice method according to claim 1, characterized in that, In step 1, the echo cancellation (AEC) employs an adaptive filter algorithm, and its error signal calculation formula is as follows: ； in, This is the error signal after echo cancellation. The original input signal, This is the predicted output signal of the filter; The automatic gain control (AGC) dynamically adjusts the gain based on the root mean square (RMS) value of the signal. The gain calculation formula is as follows: ； in, The signal after dynamic gain adjustment. The target gain value, The root mean square value of the input signal; The background noise suppression ANS uses spectral subtraction for noise suppression, which is achieved by modeling and subtracting the background noise spectrum, as shown in the formula: ； in, The signal spectrum after denoising. The original signal spectrum, This is the estimated noise spectrum.

3. The location-based intelligent multilingual oral practice method according to claim 1, characterized in that, In step 2, in the Bluetooth mode, the processed voice data is sent to the mobile APP via Bluetooth BLE. The cloud server performs speech recognition, dialogue generation, and audio synthesis, and then returns the generated training content to the wearable device for playback. The specific processes for speech recognition, dialogue generation, and audio synthesis include: Step 2.1, Speech Recognition: After receiving the speech data, the mobile app first uses an end-to-end speech recognition model deployed in the cloud to convert the speech signal into text. The model adopts an encoder-decoder structure based on an attention mechanism. The encoder extracts speech features, and the decoder outputs the corresponding text sequence, supporting real-time conversion of multiple languages. The speech recognition model is represented as follows: ； in, The input voice signal. For the target statement, The conditional probabilities of speech and sentences given for the acoustic model. These are the probabilities of the language model, obtained through training on a large-scale corpus. For speech signals The marginal probability; Step 2.2, Natural Language Understanding and Dialogue Generation: The recognized text is analyzed by the natural language understanding module to understand the user's intent and semantics. Combined with the context memory unit and the dialogue state tracking mechanism, the current dialogue topic and user needs are determined. Subsequently, based on the pre-trained generative dialogue model or retrieval dialogue engine, a response text in the target language that is context-appropriate and grammatically correct is generated. Step 2.3, Audio Synthesis: The generated response text is input into the neural speech synthesis model. The model uses a Transformer-based acoustic model and vocoder to convert the text into a target language speech waveform with naturalness and emotional intonation. Step 2.4, Content Return: The synthesized audio stream is transmitted back to the wearable device in real time via Bluetooth link for playback, while the dialogue text and pronunciation evaluation prompts are simultaneously displayed on the mobile APP interface.

4. The location-based intelligent multilingual oral practice method according to claim 1, characterized in that, In step 2, in the 4G mode, the wearable device works independently of the mobile phone, uploading voice data to the cloud server through the 4G communication module. The cloud server processes the data and sends the generated training content to the wearable device for playback.

5. The location-based intelligent multilingual oral practice method according to claim 1, characterized in that, Step 4, the one-click call includes the following steps: Trigger a call command by pressing a button on the device; Initiate a voice call request to a preset mobile phone number via 4G network; The current coaching session is automatically paused during the call and resumed or prompted to continue after the call ends.

6. The location-based intelligent multilingual oral practice method according to claim 5, characterized in that, The one-click call also supports voice triggering. After the user says the preset wake-up word, they can say the phone number to make a call. During the call, the device automatically enables voice enhancement processing to ensure call clarity.

7. The location-based intelligent multilingual oral practice method according to claim 1, characterized in that, In step 5, the learning suggestions specifically include: Step 5.1: Switch from real-time coaching mode to preset scenario mode: When the system detects that the user's pronunciation error rate is higher than the preset threshold or the fluency of the conversation is poor during real-time tutoring, it will automatically recommend entering the preset scenario mode to carry out targeted reinforcement training for high-frequency error words or sentence patterns. Step 5.2: Switch from the preset scene mode to the real-time coaching mode: When a user completes one or more sets of scenario exercises in a preset scenario mode and the pronunciation accuracy and fluency reach the preset standards, the system recommends entering the real-time coaching mode, allowing the user to conduct comprehensive ability training in a dialogue environment without a fixed script. Step 5.3: Switch from real-time coaching mode or preset scenario mode to prompt word scenario mode: When a user actively requests personalized practice, or when the system analyzes historical data and finds that the user has persistent weaknesses in specific topics, grammar, or vocabulary, it recommends entering the prompt word scenario mode to generate customized practice content based on the user's interests or ability shortcomings. Step 5.4, Related Recommendation Content Generation Mechanism: Based on the user's historical training data, real-time performance, personal preferences, and geographic location information, the system intelligently generates cross-mode navigation suggestions and dynamically adjusts the difficulty and topic of the recommended content. It employs a weighted collaborative filtering model for dynamic personalized recommendations, integrating user historical behavior and real-time location information. Its prediction scoring formula is as follows: ； in, For users For the scene Predicted interest scores For users and Behavioral similarity between them For users For the scene Actual interaction rating, The average rating of user u across all scenarios. Let u be the set of similar users. The average rating for user v across all scenarios.

8. The location-based intelligent multilingual oral practice method according to claim 1, characterized in that, In step 7, the data synchronization request specifically involves: when the mobile app initiates a synchronization request in Bluetooth mode, after verifying the user's identity, the corresponding historical data is encrypted and transmitted to the mobile app, and displayed in the form of charts, lists, or reports.

9. A location-based intelligent multilingual oral practice system, used to implement the method as described in claim 1, characterized in that, This includes wearable devices with integrated 4G and Bluetooth dual-mode communication and positioning modules, optional connected mobile terminals, and cloud servers; The wearable device includes: Voice acquisition unit; Voice processing unit; 4G and Bluetooth dual-mode communication and positioning unit, supporting mobile network connection, Bluetooth data transmission and real-time positioning; The status switching module responds to DIP switch operations and controls the device to work in Bluetooth mode or 4G mode. The call control unit is used to respond to one-click call commands in 4G mode and initiate voice calls with the paired mobile phone. Voice playback unit; The mobile terminal has an application installed to provide a complete coaching service and interface interaction in Bluetooth mode; The cloud server is used to store the user's learning data in 4G mode and respond to the synchronization request of the mobile APP in Bluetooth mode, sending the user's historical learning data to the mobile APP for visualization.

10. A wearable device for integrating the system of claim 9, characterized in that, It includes a housing, a magnetic or clamping structure mounted on the housing, and a voice acquisition unit, a voice processing unit, a voice playback unit, a 4G and Bluetooth dual-mode communication and positioning unit, a status switching module, and a call control unit integrated within the housing; the device surface is equipped with a DIP switch and a call button.

Images