Implementation method and device of a dual-screen same-display real-time translator

Abstract

The present invention provides a method and device for realizing a dual-screen same-display real-time translator, including the following steps: Configure hardware: Select the SSD2381 chip to drive two touch screen modules, and the two touch screen modules display the same content; Installation of touch screen modules: Make a rotatable aluminum alloy bracket and install the two touch screen modules on the bracket. The present invention realizes multitasking through a multitasking algorithm. The multitasking algorithm can monitor the resource usage of the chip in real time and dynamically allocate resources according to the requirements of tasks. Through the time slice rotation and priority preemption mechanism, it ensures that each task can run efficiently and in parallel, improves the overall performance, reduces the system bottleneck caused by task waiting, and moreover, the multitasking algorithm can also achieve load balancing, evenly distribute tasks to each processing unit of the system, avoid the situation where some processing units are overloaded while other processing units are idle, and improve the overall performance and efficiency.

Description

Technical Field

[0001] The present invention relates to a method and a device, specifically to a method and a device for realizing a dual - screen simultaneous display real - time translator, belonging to the technical field of speech translation. Background Art

[0002] A real - time translator is an electronic device that can instantaneously translate one language into another. It usually has functions such as speech recognition, text translation, and speech synthesis, and can achieve instant translation of spoken or written language, helping people with different language backgrounds to communicate. Real - time translators have a wide range of applications in fields such as travel, business, international conferences, and education.

[0003] When a real - time translator is working, first, the human speech signal is converted into a digital signal that can be processed by a computer through speech recognition technology. This process includes microphone acquisition of the speech signal, pre - processing of the signal (such as noise removal, amplification, and filtering), feature extraction (such as spectrum, cepstrum, etc.), and pattern matching, that is, matching the extracted features with a pre - trained speech model to recognize the corresponding words and phrases, and then converting the source - language text into the target - language text. Through a series of complex technical processing steps, the real - time translator realizes the function of instantaneously translating one language into another, greatly promoting international communication and cooperation.

[0004] In the prior art, the system performance of a dual - screen simultaneous display real - time translator often faces bottlenecks. The root cause of this problem lies in the lack of an efficient multi - task processing mechanism. The absence of this mechanism directly leads to the lag of the overall system performance and response speed, thereby affecting the user experience and the practicality of the device:

[0005] When a dual - screen simultaneous display real - time translator processes multiple tasks simultaneously, such as speech recognition, machine translation, screen display update, and possible network communication, if there is no reasonable and efficient multi - task processing mechanism to coordinate the execution of these tasks, the system is prone to problems such as resource conflicts and task blocking. These problems not only reduce the execution efficiency of individual tasks but also cause the entire system to freeze or crash. Therefore, introducing an efficient multi - task processing mechanism is the key to solving the system performance bottleneck of the dual - screen simultaneous display real - time translator. For this reason, a method and a device for realizing a dual - screen simultaneous display real - time translator are proposed. Summary of the Invention

[0006] In view of this, the present invention provides a method and a device for realizing a dual - screen simultaneous display real - time translator to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial alternative.

[0007] The technical solution of the embodiment of the present invention is realized as follows: A method for realizing a dual - screen simultaneous display real - time translator includes the following steps:

[0008] Configure the hardware: Select the SSD2381 chip to drive two touch screen modules, and the two touch screen modules display the same content;

[0009] Install the touch screen module: Make a rotatable aluminum alloy bracket and install the two touch screen modules on the bracket;

[0010] Configure the sound collection module: Layout the microphones according to the orientation of the touch screen module, and reduce the impact of environmental noise on the sound collection module through a denoising algorithm;

[0011] Establish a multitasking mechanism: The SSD2381 chip provides the function of task scheduling, distributes tasks to available processor cores, and realizes multitasking through a multitasking algorithm;

[0012] Real-time translation: Use the LSTM model to realize the real-time translation of sound signals;

[0013] System testing: Conduct functional testing, performance testing, and stability testing. Functional testing includes screen display testing, sound collection and translation testing, and user interaction testing. Performance testing includes processing speed testing and power consumption testing. Stability testing includes long-term operation testing and environmental adaptability testing.

[0014] Further preferably, when configuring the hardware, use the Mipi and I2C interfaces to connect the touch screen module to the SSD2381 chip, and write the driver program of the SSD2381 to realize the functions of screen initialization, image data transmission, and touch event processing.

[0015] Further preferably, when installing the touch screen module, install a rotating shaft on the aluminum alloy bracket to allow the touch screen module to rotate freely, and a locking knob for fixing the touch screen module is provided on the bracket.

[0016] Further preferably, when configuring the sound collection module, select a microphone with low noise, high sensitivity, and wide frequency response characteristics;

[0017] The denoising algorithm includes the following steps:

[0018] Signal preprocessing: Perform frame processing on the collected noisy speech signal, select the frame length to be 20 - 30 milliseconds for each frame, the frame shift to be 8 - 11 milliseconds, and perform windowing processing on each frame of the signal;

[0019] Noise estimation: During the inactive period of the speech signal, use the LMS algorithm to estimate the power spectral density of the background noise;

[0020] Adaptive filtering: Use the estimated noise characteristics to design an adaptive filter, which is used to dynamically adjust its parameters according to the input noisy speech signal;

[0021] Spectrum subtraction: Perform Fourier transform on the speech signal after adaptive filtering to obtain the spectrum representation. According to the noise estimation result, subtract the estimated noise spectrum from the spectrum of the speech signal;

[0022] Inverse Fourier transform and reconstruction: Perform inverse Fourier transform on the processed spectrum to obtain the denoised speech signal in the time domain. Perform overlap-and-add or overlap-save processing on the signal after inverse transform to restore the continuity of the original signal;

[0023] Post-processing: Perform post-processing on the denoised speech signal, including removing the residual DC component and performing gain adjustment.

[0024] Further preferably, when establishing the multi-task processing mechanism, divide the SSD2381 chip into multiple independent control regions, and each control region is responsible for a specific function;

[0025] Adopt an asynchronous programming model so that the system will not be blocked when a task is waiting for I / O operations or other resources;

[0026] The multi-task processing algorithm includes the following steps:

[0027] Task division and priority setting: Divide the functions of the dual-screen simultaneous display real-time translator into multiple independent task modules, including speech recognition tasks, machine translation tasks, and display control tasks. Set a priority for each task according to the importance and real-time requirements of the task;

[0028] Task queue management: Create a task queue for storing tasks to be executed. Each task in the task queue contains task identification, priority, and execution function information. Sort the task queue according to the priority of the tasks and give priority to executing high-priority tasks;

[0029] Round-robin time slice scheduling: Allocate a fixed time slice for each task. The size of the time slice is set according to the processing power of the chip and the complexity of the task. The chip executes the tasks in the queue in a round-robin manner;

[0030] Priority preemption mechanism: On the basis of round-robin time slice scheduling, introduce a priority preemption mechanism. When a high-priority task arrives, if the time slice of the currently executing task has not been used up, immediately interrupt the current task, insert the high-priority task at the front of the queue and execute it. The interrupted task will continue to execute within its remaining time slice or be re-allocated a time slice as needed;

[0031] Task synchronization and communication: Design a synchronization and communication mechanism between tasks. For tasks that need to interact frequently, design a dedicated data buffer or shared memory area to store and access data;

[0032] Resource Management and Optimization: Monitor the resource usage of the chip, including CPU utilization and memory occupancy. For tasks with high resource consumption, implement them asynchronously.

[0033] Further preferably, during real-time translation, use the LSTM model to achieve real-time translation of voice signals, which specifically includes the following steps:

[0034] Dataset Preparation: Collect and prepare a multilingual dialogue dataset for training the LSTM model;

[0035] Model Training: Use a deep learning framework to train the LSTM model and optimize the model parameters;

[0036] Model Evaluation and Optimization: Evaluate the model performance through a test set and optimize the model according to the evaluation results;

[0037] Real-time Translation: Embed the trained LSTM model into the SSD2381 chip to achieve real-time translation of voice signals.

[0038] Further preferably, the function test specifically includes:

[0039] Screen Display Test: Test the display effect of the screen at different angles and lighting conditions;

[0040] Voice Acquisition and Translation Test: Test the voice acquisition effect of the microphone, the translation accuracy and speed of the real-time translation module;

[0041] User Interaction Test: Test the usability and response speed of the user interface;

[0042] The performance test specifically includes:

[0043] Processing Speed Test: Test the processing speed of the device under different loads;

[0044] Power Consumption Test: Test the power consumption of the device and optimize the power management strategy.

[0045] Further preferably, the stability test specifically includes:

[0046] Long-term Running Test: Let the device run for a long time and observe whether any abnormal situations occur;

[0047] Environmental Adaptability Test: Test the stability and reliability of the device in environments with different temperatures, humidities, and electromagnetic interferences.

[0048] A dual-screen simultaneous display real-time translation device, including a first touch screen, a second touch screen, an SSD2381 chip, and a digital microphone;

[0049] The digital microphones are respectively installed inside the first touch screen and the second touch screen, and the signal terminals of the SSD2381 chip are respectively connected to the signal terminals of the first touch screen and the second touch screen.

[0050] Further preferably, the digital microphones are used to collect the voice data of the user, and the voice data is sent to the SSD2381 chip;

[0051] The SSD2381 chip is used to realize real-time translation of the sound signal by using the LSTM model, and the translated data is respectively displayed through the first touch screen and the second touch screen.

[0052] Due to the above technical solutions adopted in the embodiments of the present invention, it has the following advantages:

[0053] First, the present invention provides a task scheduling function through the SSD2381 chip, allocates tasks to available processor cores, realizes multi-task processing through a multi-task processing algorithm, can monitor the resource usage of the chip in real time, and dynamically allocates resources according to the requirements of tasks. Through the time slice rotation and priority preemption mechanisms, it ensures that each task can run efficiently and in parallel, improves the overall performance. The scheduling mechanism in the algorithm can arrange the execution order according to the priority of tasks, and high-priority tasks can obtain processing resources first, thus ensuring the timely execution of key tasks and reducing system bottlenecks caused by task waiting. Moreover, the multi-task processing algorithm can also achieve load balancing, evenly distribute tasks to each processing unit of the system, avoid the situation where some processing units are overloaded while other processing units are idle, and improve the overall performance and efficiency.

[0054] Second, the present invention reduces the influence of environmental noise on the sound acquisition module through a denoising algorithm, can adaptively adjust the filtering parameters according to the change of environmental noise, so as to maintain a good denoising effect in different noise environments. At the same time, combining the advantages of adaptive filtering and spectral subtraction, it can effectively remove background noise and residual noise, improve the signal-to-noise ratio and intelligibility of the voice signal. Moreover, the computational complexity of the denoising algorithm is moderate, and it can be quickly processed in a real-time translator to meet the requirements of real-time communication.

[0055] The above summary is only for the purpose of the specification and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0057] Figure 1 It is a flowchart of the steps of a method for implementing a dual-screen simultaneous display real-time translator of the present invention;

[0058] Figure 2 It is a flowchart of the steps of the denoising algorithm of the present invention;

[0059] Figure 3 It is a flowchart of the steps of the multi-task processing algorithm of the present invention;

[0060] Figure 4 It is a schematic diagram of the principle structure of a dual-screen simultaneous display real-time translator device of the present invention;

[0061] Figure 5 It is a schematic diagram of the working process of a dual-screen simultaneous display real-time translator device of the present invention. Detailed implementation manners

[0062] In the following, only some exemplary embodiments are briefly described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the present invention. Therefore, the drawings and the description are considered to be exemplary in nature rather than restrictive.

[0063] The embodiments of the present invention will be described in detail below with reference to the drawings.

[0064] As Figures 1 - 3 shown, the embodiments of the present invention provide a method for implementing a dual-screen simultaneous display real-time translator, including the following steps:

[0065] Configure hardware:

[0066] Select the SSD2381 chip to drive two touch screen modules. The two touch screen modules display the same content. By clicking on the touch screen with a finger, the content displayed on the screen can be manipulated;

[0067] Confirm the interface compatibility between the touch screen module and the SSD2381 chip. Connect the touch screen module to the SSD2381 chip using Mipi and I2C interfaces. Write the driver program for the SSD2381 to implement functions such as screen initialization, image data transmission, and touch event handling. The SSD2381 chip is built-in with a high-performance 4-core 64-bit ARM Cortex-A55 kernel processor with a main frequency up to 1.5 GHz. Such a high-performance processor core can ensure that the translator performs excellently when dealing with complex translation tasks and provides fast and accurate translation results.

[0068] Installation of the touch screen module:

[0069] Make a rotatable aluminum alloy bracket. By using aluminum alloy material, the overall weight can be reduced. Install two touch screen modules on the bracket, install a rotating shaft on the aluminum alloy bracket to allow the touch screen module to rotate freely, and set a locking knob on the bracket to fix the touch screen module to ensure that it can be stably fixed after adjusting the angle.

[0070] Configuration of the sound collection module:

[0071] Layout the microphones according to the orientation of the touch screen module, and reduce the influence of environmental noise on the sound collection module through a denoising algorithm;

[0072] Through the denoising algorithm, the filtering parameters can be adaptively adjusted according to the change of environmental noise, so as to maintain a good denoising effect in different noise environments. At the same time, combining the advantages of adaptive filtering and spectral subtraction, it can effectively remove background noise and residual noise, improve the signal-to-noise ratio and intelligibility of the voice signal, and the calculation complexity of the denoising algorithm is moderate, which can be quickly processed in a real-time translator to meet the needs of real-time communication;

[0073] Among them, the denoising algorithm includes the following steps:

[0074] Signal preprocessing: Perform frame processing on the collected noisy speech signal. The length of each frame is selected to be 20 - 30 milliseconds, and the frame shift is 8 - 11 milliseconds. Window the signal of each frame to reduce the discontinuity at the frame edge;

[0075] Noise estimation: During the inactive period of the speech signal, use the LMS algorithm to estimate the power spectral density of the background noise. The accuracy of noise estimation is crucial for the subsequent denoising effect, so it is necessary to ensure accurate identification and extraction of noise samples during the inactive period;

[0076] Adaptive filtering: Use the estimated noise characteristics to design an adaptive filter. This filter is used to dynamically adjust its parameters according to the input noisy speech signal to remove the background noise to the greatest extent. The output of the adaptive filter will be the preliminarily denoised speech signal;

[0077] Spectrum subtraction: Perform a Fourier transform on the speech signal after adaptive filtering to obtain a spectral representation. According to the noise estimation result, subtract the estimated noise spectrum from the spectrum of the speech signal to achieve further denoising;

[0078] The LMS algorithm is an adaptive filtering algorithm that iteratively adjusts the filter coefficients to minimize the mean square value of the output error;

[0079] To avoid introducing musical noise, the spectrum after subtraction can be smoothed, such as using techniques like spectral weighting or non-linear transformation;

[0080] Inverse Fourier transform and reconstruction: Perform an inverse Fourier transform on the processed spectrum to obtain the denoised speech signal in the time domain. Perform overlap-add or overlap-save processing on the signal after the inverse transform to restore the continuity of the original signal;

[0081] Post-processing: Perform post-processing on the denoised speech signal, including removing the residual DC component and performing gain adjustment to ensure the sound quality and intelligibility of the output signal;

[0082] When configuring the sound acquisition module, select a microphone with low noise, high sensitivity, and wide frequency response characteristics.

[0083] Establish a multi-tasking mechanism:

[0084] The SSD2381 chip provides a task scheduling function, allocates tasks to available processor cores, and implements multi-tasking through a multi-tasking algorithm;

[0085] The multi-tasking algorithm can monitor the resource usage of the chip in real time and dynamically allocate resources according to the requirements of the tasks. Through the time slice rotation and priority preemption mechanisms, it ensures that each task can run efficiently and in parallel, improving the overall performance. The scheduling mechanism in the algorithm can arrange the execution order according to the priority of the tasks. High-priority tasks can obtain processing resources first, thus ensuring the timely execution of critical tasks and reducing system bottlenecks caused by task waiting. Moreover, the multi-tasking algorithm can also achieve load balancing, evenly distribute tasks to each processing unit of the system, avoid the situation where some processing units are overloaded while others are idle, and improve the overall performance and efficiency;

[0086] When establishing the multi-tasking mechanism, divide the SSD2381 chip into multiple independent control regions, and each control region is responsible for a specific function;

[0087] Adopt an asynchronous programming model, so that the system will not be blocked when tasks are waiting for I / O operations or other resources;

[0088] The multi-tasking algorithm includes the following steps:

[0089] Task Division and Priority Setting: Divide the functions of the dual-screen simultaneous display real-time translator into multiple independent task modules, including speech recognition tasks, machine translation tasks, and display control tasks. Set a priority for each task according to the importance and real-time requirements of the task. Among them, the speech recognition and display control tasks are set as the first and second priorities respectively to ensure real-time performance and user experience;

[0090] Task Queue Management: Create a task queue for storing tasks to be executed. Each task in the task queue contains task identification, priority, and execution function information. Sort the task queue according to the priority of the tasks and give priority to executing high-priority tasks;

[0091] Round Robin Scheduling: Allocate a fixed time slice for each task. The size of the time slice is set according to the processing power of the chip and the complexity of the task. The chip executes the tasks in the queue in a round-robin manner. When the time slice of a certain task is used up, if the task is not completed, it will be put back to the end of the queue to wait for the next scheduling;

[0092] Priority Preemption Mechanism: On the basis of round-robin scheduling, introduce a priority preemption mechanism. When a high-priority task arrives, if the time slice of the currently executing task has not been used up, immediately interrupt the current task, insert the high-priority task to the front of the queue and execute it. The interrupted task will continue to execute within its remaining time slice or be re-allocated a time slice as needed;

[0093] Task Synchronization and Communication: Design a synchronization and communication mechanism between tasks to ensure data consistency and collaboration between tasks. For example, synchronization primitives such as semaphores and message queues can be used to achieve synchronization and communication between tasks. For tasks that need to interact frequently, design dedicated data buffers or shared memory areas to store and access data;

[0094] Resource Management and Optimization: Monitor the resource usage of the chip, including CPU usage rate and memory occupancy rate. For tasks that consume a large amount of resources, implement them in an asynchronous execution manner;

[0095] Fault Recovery and Fault Tolerance Handling:

[0096] Design a fault recovery and fault tolerance handling mechanism to handle possible abnormal situations during task execution. For example, use strategies such as retry mechanisms and backup tasks to ensure the reliable execution of tasks. For critical tasks, design redundant backups and failover mechanisms to improve the reliability and stability of the system.

[0097] Real-time Translation: Use the LSTM model to achieve real-time translation of voice signals, specifically including the following steps:

[0098] Dataset Preparation: Collect and prepare a multilingual dialogue dataset for training the LSTM model;

[0099] Model Training: Use a deep learning framework to train the LSTM model and optimize its parameters;

[0100] Model Evaluation and Optimization: Evaluate the model performance through a test set and optimize the model according to the evaluation results;

[0101] Real-time Translation: Embed the trained LSTM model into the SSD2381 chip to achieve real-time translation of voice signals;

[0102] The LSTM model is a long short-term memory artificial neural network model.

[0103] System Testing: Conduct functional testing, performance testing, and stability testing;

[0104] Functional testing includes screen display testing, voice collection and translation testing, and user interaction testing. Among them,

[0105] Screen Display Testing: Test the display effect of the screen at different angles and light conditions to ensure that users can obtain a clear visual experience in different environments, including:

[0106] Angle Testing: Use a viewing angle measuring instrument to measure the changes in brightness and color parameters of the screen at different angles to determine the viewing angle range of the screen. For example, observe the content displayed on the screen at vertical, horizontal, 45-degree tilt, 60-degree tilt, etc. angles to check whether there are obvious changes in the brightness and color of the image;

[0107] Light Testing: In a darkroom environment, use a luminance meter to measure the brightness values of the screen in full black and full white states, as well as the brightness changes under different light conditions. At the same time, observe the color reproduction and contrast performance of the screen under different lights;

[0108] Voice Collection and Translation Testing: Test the voice collection effect of the microphone, the translation accuracy and speed of the real-time translation module, including:

[0109] Voice Collection Testing: Use professional audio testing equipment to record voice samples of the microphone at different distances, different volumes, and different speaking speeds, and analyze its clarity, fidelity, and noise suppression ability;

[0110] Translation Testing: Prepare voice samples in multiple languages, translate them through the real-time translation module, and compare the accuracy and speed of the translation results with the standard answers. At the same time, invite testers with multilingual backgrounds to participate in the testing to verify the performance of the translation module in different language environments;

[0111] User Interaction Testing: Test the usability and response speed of the user interface to ensure that users can complete operations easily and quickly, including:

[0112] Usability Testing: Invite users with different backgrounds to participate in the test, observe their operation habits, difficulties and problems when using the user interface, and collect their feedback;

[0113] Response Speed Testing: Use professional performance testing tools to simulate user operations and measure the response speed of the user interface, such as measuring the time difference between clicking a button and the interface update, and the smoothness when sliding the screen;

[0114] Performance testing includes processing speed testing and power consumption testing, where

[0115] Processing Speed Testing: Test the processing speed of the device under different loads to ensure that real-time translation and image processing can proceed smoothly, including:

[0116] Load Testing: By simulating multiple users accessing the system simultaneously and performing operations such as translation and image processing, measure performance metrics such as the system's response time and throughput, and use professional load testing tools to simulate user behavior and generate test reports;

[0117] Benchmark Testing: Under certain software, hardware, and network environments, simulate a certain number of virtual users running one or more services, and use the test results as baseline data. In subsequent optimization or system evaluation processes, determine whether the optimization has achieved the desired effect or provide decision-making data for system selection by running the same service scenarios and comparing the test results;

[0118] Power Consumption Testing: Test the power consumption of the device and optimize the power management strategy to reduce energy consumption, including:

[0119] Power Consumption Measurement: Use power consumption measurement equipment to measure the power consumption values of the system in different working states, such as the power consumption changes when in standby, running the translation module, and performing image processing;

[0120] Power Management Strategy Testing: By adjusting the system's power management strategy, such as reducing the screen brightness and closing unnecessary background applications, observe the reduction in power consumption and verify the effectiveness of the strategy;

[0121] Stability testing includes long-term running testing and environmental adaptability testing, where

[0122] Long-Term Running Testing: Let the device run for a long time and observe whether any abnormal situations occur, including:

[0123] Continuous operation test: Let the device run continuously for 72 hours, observe whether there are any abnormal situations such as crashes or lags, and use monitoring tools to record the device's operating status and error information;

[0124] Log analysis: Analyze the log files of the device during operation, search for potential problems and abnormal behaviors, and analyze the log information before the system crashes to determine the cause of the crash and solutions;

[0125] Environmental adaptability test: Test the stability and reliability of the device in environments with different temperatures, humidities, and electromagnetic interferences, including:

[0126] Temperature test: Place the device in different temperature environments, including high temperature, low temperature, and rapid temperature change conditions, observe the device's operating status and performance changes, and a temperature test chamber can be used to simulate different temperature environments;

[0127] Humidity test: Place the device in a high humidity environment, observe the moisture-proof performance and stability of the system, and use humidity test equipment to simulate a high humidity environment;

[0128] Electromagnetic interference test: Place the device in a strong electromagnetic interference environment, such as electromagnetic pulses and radio frequency interference, observe the device's anti-interference ability and stability, and electromagnetic interference test equipment can be used to simulate a strong electromagnetic interference environment.

[0129] As Figures 4 - 5 shown, the embodiment of the present invention provides a dual-screen simultaneous display real-time translator device, including a first touch screen, a second touch screen, an SSD2381 chip, and a digital microphone;

[0130] The digital microphones are respectively installed inside the first touch screen and the second touch screen. The signal terminals of the SSD2381 chip are respectively connected to the signal terminals of the first touch screen and the second touch screen. By using the SSD2381 chip, a dual-screen simultaneous display device is realized. Then, the digital microphones of the two screens are used to pick up sound respectively, collect the voices of people in different languages, and through artificial intelligence algorithms, automatic language recognition and voice-to-text translation effects are achieved. Then, the two output texts are superimposed and displayed on the two screens, enabling people in two different languages to communicate face-to-face without barriers;

[0131] This translator can be used in places such as ports, scenic spots, stores, and foreign-related hotels. When guests in different languages come, as long as the guests say a word, the system can automatically identify the guests' languages within one second; then the store clerks and customers can communicate without barriers based on the translator.

[0132] In one embodiment, the digital microphone is used to collect the user's voice data, and the voice data is sent to the SSD2381 chip;

[0133] The SSD2381 chip is used to realize the real-time translation of voice signals by using the LSTM model. The translated data is displayed through the first touch screen and the second touch screen respectively. The SSD2381 drives the two touch screens simultaneously, and the same content is displayed on the two touch screens. By clicking on the touch screen with a finger, the content displayed on the screen can be manipulated.

[0134] When in use: the two screens are displayed in different directions, and the user can adjust the display direction and angle within a certain range. There are two groups of microphones beside each screen, which can collect the voice inputs from user A and user B respectively according to the screen orientation. The long short-term memory network algorithm is used to perform real-time detection and analysis on the voices collected from both sides. The system can automatically identify the language types on both sides, translate the conversation process into two languages and display them on the screen at the same time. People on both sides of the screen can see the real-time conversation record after translation and communicate without obstacles.

[0135] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various changes or substitutions, and these should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for implementing a dual-screen simultaneous display real-time translator, characterized in that, It includes the following steps: Select the SSD2381 chip to drive two touch screen modules, and the two touch screen modules display the same content; Make a rotatable aluminum alloy bracket and install the two touch screen modules on the bracket; Layout the microphones according to the orientation of the touch screen modules, and reduce the impact of environmental noise on the sound collection module through a denoising algorithm; The SSD2381 chip provides a task scheduling function, distributes tasks to available processor cores, and realizes multitasking through a multitasking algorithm; Use the LSTM model to realize real-time translation of sound signals; Conduct function tests, performance tests and stability tests. The function tests include screen display tests, sound collection and translation tests, and user interaction tests. The performance tests include processing speed tests and power consumption tests. The stability tests include long-term operation tests and environmental adaptability tests; When establishing a multitasking mechanism, divide the SSD2381 chip into multiple independent control areas, and each control area is responsible for a specific function; Adopt an asynchronous programming model so that the system will not be blocked when tasks are waiting for I / O operations or other resources; The multitasking algorithm includes the following steps: Task division and priority setting: Divide the functions of the dual-screen same-display real-time translator into multiple independent task modules, including speech recognition tasks, machine translation tasks, and display control tasks. Set a priority for each task according to the importance and real-time requirements of the task; Task queue management: Create a task queue for storing tasks to be executed. Each task in the task queue contains task identification, priority, and execution function information. Sort the task queue according to the priority of the tasks, and give priority to executing high-priority tasks; Round-robin scheduling: Allocate a fixed time slice for each task. The size of the time slice is set according to the processing power of the chip and the complexity of the task. The chip executes the tasks in the queue in a round-robin manner; Priority preemption mechanism: On the basis of round-robin scheduling, introduce a priority preemption mechanism. When a high-priority task arrives, if the time slice of the currently executing task has not been used up, immediately interrupt the current task, insert the high-priority task at the front of the queue and execute it, and the interrupted task will continue to execute within its remaining time slice or be re-allocated a time slice as needed; Task synchronization and communication: Design a synchronization and communication mechanism between tasks. For tasks that need to interact frequently, design a dedicated data buffer or shared memory area to store and access data; Resource management and optimization: Monitor the resource usage of the chip, including CPU usage and memory occupancy. For tasks that consume a large amount of resources, use an asynchronous execution method to implement.

2. The method for implementing a dual-screen simultaneous display real-time translator according to claim 1, wherein: When configuring the hardware, use the Mipi and I2C interfaces to connect the touch screen module to the SSD2381 chip and write the driver program for the SSD2381.

3. The method for implementing a dual-screen simultaneous display real-time translator according to claim 1, wherein: When installing the touch screen module, install a rotating shaft on the aluminum alloy bracket. The touch screen module can rotate freely, and there is a locking knob on the bracket to fix the touch screen module.

4. A method for implementing a dual-screen simultaneous display real-time translator according to claim 1, characterized in that: Reduce the impact of environmental noise on the sound collection module through the denoising algorithm; Specifically, it includes the following steps: Signal preprocessing: The collected noisy speech signal is framed, with the length of each frame selected as 20 - 30 milliseconds and the frame shift as 8 - 11 milliseconds, and each frame of the signal is windowed; Noise estimation: During the inactive period of the speech signal, the power spectral density of the background noise is estimated using the LMS algorithm; Adaptive filtering: Using the estimated noise characteristics, an adaptive filter is designed, which is used to dynamically adjust its parameters according to the input noisy speech signal; Spectrum subtraction: The speech signal after adaptive filtering is Fourier-transformed to obtain a spectral representation, and according to the noise estimation result, the estimated noise spectrum is subtracted from the spectrum of the speech signal; Inverse Fourier transform and reconstruction: The processed spectrum is inverse Fourier-transformed to obtain the denoised speech signal in the time domain, and the overlapped add or overlapped save process is performed on the signal after inverse transformation to restore the continuity of the original signal; Post-processing: The denoised speech signal is post-processed, including removing the residual DC component and performing gain adjustment.

5. A method for implementing a dual-screen simultaneous display real-time translator according to claim 1, characterized in that: During real-time translation, the LSTM model is used to achieve real-time translation of sound signals, specifically including the following steps: Dataset preparation: Collect and prepare a multi-language dialogue dataset for training the LSTM model; Model training: Use a deep learning framework to train the LSTM model and optimize the model parameters; Model evaluation and optimization: Evaluate the model performance through the test set and optimize the model according to the evaluation results; Real-time translation: Embed the trained LSTM model into the SSD2381 chip to achieve real-time translation of sound signals.

6. The method for implementing a dual-screen simultaneous display real-time translator according to claim 1, wherein: The function test specifically includes: Screen display test; Sound collection and translation test; User interaction test; The performance test specifically includes: Processing speed test; Power consumption test.

7. A method for implementing a dual-screen simultaneous display real-time translator according to claim 1, characterized in that: The stability test specifically includes: Long-term operation test; Environmental adaptability test.

8. A dual-screen simultaneous display real-time translation device, which is applied to a method for realizing a dual-screen simultaneous display real-time translation machine as described in any one of claims 1-7, and is characterized in that, It includes a first touch screen, a second touch screen, an SSD2381 chip, and a digital microphone; The digital microphones are respectively installed inside the first touch screen and the second touch screen, and the signal terminals of the SSD2381 chip are respectively connected to the signal terminals of the first touch screen and the second touch screen.

9. The dual-screen simultaneous display real-time translation device according to claim 8, characterized in that: The digital microphone is used to collect the speech data of the user, and the speech data is sent to the SSD2381 chip; The SSD2381 chip is used to achieve real-time translation of sound signals using the LSTM model, and the translated data is respectively displayed through the first touch screen and the second touch screen.

Images