An extended display method and system

Abstract

The application relates to an extended display method and system, which comprises a display module, a mobile terminal display screen and a shared screen, the display module comprises at least two display screens; the extended display method comprises the following steps: acquiring all user information, screen information, a source language, a first target language and a second target language; acquiring audio information input by an audio input device; converting the audio information into original text, first translation text and second translation text according to the source language, the first target language and the second target language; and respectively pushing the original text, the first translation text and the second translation text to corresponding display modules, mobile terminal display screens and shared screens. According to the application, the original text, the first translation text and the second translation text are formed according to the set language, and are pushed to the display module, the mobile terminal display screen and the shared screen. Personal screen display extension, shared screen extension and personal mobile terminal device display extension can be realized, and the display extension modes are various.

Description

Technical Field

[0001] This invention relates to the field of electronic device display extension technology, and in particular to an extended display method and system. Background Technology

[0002] In daily work, academic conferences, and international business negotiations, language barriers may arise. Simultaneous interpretation using translation equipment is often necessary, with the translated content displayed on a screen. However, a single screen can only display a limited amount of content, making comparison difficult. Expandable screens, on the other hand, can only display content on a personal monitor and require specific expandable screens, placing high demands on equipment and limiting expansion options. For applications involving extensive negotiations and discussions, relying solely on personal screens for expansion leads to low communication efficiency and hinders meeting progress. Summary of the Invention

[0003] In order to overcome the shortcomings and deficiencies of the existing technology, the purpose of this invention is to provide an extended display method and system.

[0004] To achieve the above objectives, an extended display method of the present invention includes a display module, a mobile terminal display screen, and a shared screen, wherein the display module includes at least two display screens; The extended display method includes the following steps: Obtain all user information, screen information, source language and first target language corresponding to the user information, and second target language corresponding to the screen information; Acquire audio information input from the audio input device; Based on the acquired source language, first target language, and second target language, the audio information is converted into the original text, the first translated text, and the second translated text, respectively. Based on the user information, the original text and the first translated text are displayed to the corresponding display module. The original text and the first translated text are displayed on the mobile terminal screen based on user information; Based on the screen information, the original text, the first translated text, and the second translated text are displayed on the corresponding shared screen.

[0005] The beneficial effects of this invention are as follows: This invention provides an extended display method that acquires accessed user information, screen information, and a source language and a first target language corresponding to the user information, as well as a second target language corresponding to the screen information. It also acquires input audio information and then forms original text, a first translated text, and a second translated text based on the source language, the first target language, and the second target language. These original text, first translated text, and second translated text are then pushed to the display module, the mobile terminal display screen, and the shared screen, respectively, based on connections to different display modules, mobile terminal display screens, and shared screens. The display module includes at least two displays, allowing multiple different contents to be displayed simultaneously. This enables content comparison and better understanding, facilitating meetings and work processes. Furthermore, it can connect to mobile terminal display screens and shared screens, offering diverse expansion methods with fewer limitations. For application scenarios requiring extensive negotiations and discussions, shared screens allow everyone to view content simultaneously, providing text in multiple languages, including original text, first translated text, and second translated text, enabling rapid understanding of the content. This improves communication efficiency and ensures the smooth and efficient progress of meetings. For users unable to attend in person, relevant content can be displayed on a mobile terminal screen, offering greater flexibility as it is not limited by location. This invention's extended display method offers diverse expansion options and can be applied to a variety of different scenarios.

[0006] Preferably, the display screen includes a first screen and a second screen; obtaining all accessed user information, screen information, source language and first target language corresponding to the user information, and second target language corresponding to the screen information further includes the following steps: Identify and store the number of user information entries, the corresponding source language, and the first target language; Identify and store the access signals from the first screen and the second screen; Identify and store the access signal from the mobile terminal display screen; Identify the access signal of the shared screen, obtain screen information and the corresponding second target language, and store them.

[0007] During the process of acquiring user information, screen information, and the set first and second target languages, the access signals of the connected display screen, mobile terminal display screen, and shared screen are identified and stored. When displaying the original text, the first translated text, and the second translated text, the output is performed according to the corresponding access signal. The display expansion is highly flexible, and expansion methods can be added or removed as needed. This extended display method of the present invention does not waste resources, and expansion methods can be added when necessary.

[0008] Preferably, the extended display method further includes the following steps: Read the access signals from the first screen, the second screen, the mobile terminal display screen, and the shared screen. When only the access signal of the first screen is received, both the original text and the first translated text are displayed on the first screen; When the access signal from the first screen and the access signal from the second screen are received, the original text and the first translated text are displayed on the first screen and the second screen, respectively. When a shared screen access signal is received, the original text, the first translated text, and the second translated text are displayed in separate columns on the shared screen; When an access signal is received from the mobile terminal display screen, the original text and the first translated text are displayed on the mobile terminal display screen.

[0009] When the display module and the mobile terminal display screen are connected, the original text and the first translated text will be displayed. When the shared screen is connected, a second target language is set, and the shared screen can display the original text, the first translated text, and the second translated text. Furthermore, depending on the incoming signal, the corresponding text is output, achieving extended display. The display expansion is highly flexible, and the expansion methods can be added or removed as needed without wasting resources, thus improving display efficiency.

[0010] Preferably, the extended display method further includes the following steps: Connect the display module to the shared screen; Based on the screen information, control the shared screen to display the original text, the first translated text, and the second translated text in separate columns.

[0011] By connecting the display module to the shared screen, content that was originally pushed to the display module will also be pushed to the shared screen. The shared screen can expand the display to multiple languages ​​and can be viewed by everyone, improving communication and work efficiency.

[0012] Preferably, connecting the display module to the shared screen further includes the following steps: Entering the same group code or scanning the QR code will display the module being connected to the shared screen.

[0013] The display module can quickly connect to the shared screen by entering the same group code or QR code. Connecting via group code or QR code is efficient and convenient.

[0014] Preferably, converting the audio information into original text, first translated text, and second translated text according to the acquired source language, first target language, and second target language respectively further includes the following steps: Input a large language model; the large language model stores the original text as context data after it has been corrected and optimized by the large language model. Obtain the current audio information, optimize and correct the audio information using a large language model, and output the original text. The original text is optimized and corrected using context data, and the optimized and corrected original text is output and stored as new context data in the large language model. Translate the original text after contextual data optimization and correction, and output the first translated text and / or the second translated text.

[0015] By inputting a large language model, and considering potential accents and speech impediments in user-input audio, basic corrections and optimizations are first performed, outputting the original text. Then, the stored optimized original text is retrieved as context data. This context data is then used to further optimize and correct the currently output original text before outputting it again. This output original text is stored as new context data in the large language model, serving as the basis for context data correction for the next input audio information. The original text, after basic and context optimizations, is then translated to obtain a first and a second translated text, each corresponding one-to-one with the first and second target languages. Through the corrections made by the large language model and the combination of context data, the accuracy of the output original text, the first translated text, and / or the second translated text is improved, enhancing communication efficiency and information accuracy.

[0016] Preferably, translating the original text after contextual data optimization and correction, and outputting the first translated text and / or the second translated text, further includes the following steps: Connect to at least one translation model; Select and match translation models; The original text is input into the translation model, which outputs the first translated text and / or the second translated text.

[0017] The system inputs at least one translation model and can select an appropriate model based on the content, domain, and dialogue scenario of the source text. After matching, it translates the source text and outputs a first translated text and / or a second translated text. This ability to select the appropriate translation model based on different situations improves translation quality and efficiency, optimizes resources, and avoids wasting translation resources.

[0018] Preferably, converting the audio information into original text, first translated text, and second translated text according to the acquired source language, first target language, and second target language respectively further includes the following steps: Input the dictionary model for the corresponding professional field; The system acquires audio information and corrects and optimizes it using a dictionary model, outputting the original text, the first translated text, and / or the second translated text.

[0019] By pre-inputting a dictionary model for a specific field, and combining this model with voice dialogues or meetings, it is possible to identify specialized terms in the voice input more quickly, significantly improving the accuracy of terminology recognition. Furthermore, it can enhance semantic understanding and contextual coherence, making the conversion of audio into original text, first translation text, and second translation text more accurate.

[0020] Preferably, acquiring the audio information input by the audio input device further includes the following steps: Interleaving the first and second channel signals from the audio input device and converting them into data streams; The data stream is split, and the even-numbered bytes are extracted as the first audio information of the first channel, while the odd-numbered bytes are extracted as the second audio information of the second channel.

[0021] Interleaving the first and second audio channels improves data processing efficiency, reducing the number of calls and buffer switching overhead compared to reading the two channels separately. Interleaving ensures that the first and second channel data are physically adjacent at the same timestamp, guaranteeing timing synchronization and preventing audio breaks or translation errors due to buffer misalignment during decoding. Furthermore, by default, interleaving simplifies subsequent processing and reduces code complexity. It also supports simultaneous input from two audio channels from different devices, improving meeting efficiency.

[0022] An extended display system of the present invention includes: The identification and receiving module is used to acquire all user information, screen information, source language and first target language corresponding to the user information, and second target language corresponding to the screen information; The acquisition module is used to acquire the audio information input by the audio input device; The processing module is used to convert the audio information into the original text, the first translated text, and the second translated text respectively, based on the acquired source language, the first target language, and the second target language; The display module includes at least two displays for receiving and displaying the original text and the first translated text. A mobile terminal display screen is used to receive and display the original text and the first translated text. Share the screen to receive and display the original text and the first and second translated texts.

[0023] The beneficial effects of this invention are as follows: This invention provides an extended display system that, through an identification receiving module, acquires accessed user information, screen information, and a source language and a first target language corresponding to the user information, as well as a second target language corresponding to the screen information. An acquisition module acquires input audio information. A processing module generates original text, a first translated text, and a second translated text based on the source language, the first target language, and the second target language. These original text, the first translated text, and the second translated text are then pushed to the display modules, mobile terminal displays, and shared screens, respectively, based on connections to different display modules, mobile terminal displays, and shared screens. The display module includes at least two displays, allowing multiple different contents to be displayed simultaneously. This enables content comparison and better understanding, facilitating meetings and work processes. Furthermore, it can connect to mobile terminal displays and shared screens, offering diverse expansion methods with fewer limitations. For application scenarios requiring extensive negotiations and discussions, shared screens allow everyone to view content simultaneously, providing text in multiple languages, including original text, the first translated text, and the second translated text, enabling rapid understanding of the content. This improves communication efficiency and ensures the smooth and efficient progress of meetings. For users unable to attend in person, relevant content can be displayed on a mobile terminal screen, offering greater flexibility as it is not limited by location. This invention's extended display system offers diverse expansion methods and can be applied in a variety of different scenarios. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating an extended display method according to the present invention; Figure 2 This is a schematic diagram of the process of obtaining all user information, screen information, source language and first target language corresponding to the user information, and second target language corresponding to the screen information according to the present invention. Figure 3 This is a flowchart illustrating an extended display method according to the present invention; Figure 4 This is a flowchart illustrating an extended display method according to the present invention; Figure 5 This is a schematic diagram of the process of converting audio information into original text, first translated text and second translated text respectively based on the acquired source language, first target language and second target language according to the present invention; Figure 6 This is a flowchart illustrating the process of translating the original text after contextual data optimization and correction, and outputting a first translated text and / or a second translated text according to the present invention. Figure 7 This is a flowchart illustrating how audio information is converted into original text, first translated text, and second translated text based on the acquired source language, first target language, and second target language, respectively, according to the present invention. Figure 8 This is a flowchart illustrating the acquisition of audio information input by an audio input device according to the present invention. Figure 9 This is a schematic diagram illustrating the steps of acquiring audio information input by an audio input device according to the present invention; Figure 10 This is a schematic diagram illustrating the multi-user display module or mobile terminal display screen access and shared screen access of the present invention; Figure 11 This is a schematic diagram illustrating the steps of correcting and optimizing the original text and outputting the original text, the first translated text, and the second translated text in the large model of the present invention. Figure 12 This is a schematic diagram illustrating the steps of the access display module, mobile terminal display screen, and shared screen of the present invention. Figure 13 This is a schematic diagram of the extended display system of the present invention; Figure 14 This is a schematic diagram of the processing module of the present invention.

[0025] The reference numerals in the figures include: 1. Recognition and Reception Module; 2. Acquisition Module; 3. Processing Module; 31. Large Language Model; 32. Speech-to-Text Module; 33. Translation Module; 34. Dictionary Model; 35. Split Processor; 36. First Sub-processor; 37. Second Sub-processor; 4. Display Module; 41. First Screen; 42. Second Screen; 5. Mobile Terminal Display Screen; 6. Shared Screen; 7. Audio Input Device; Detailed Implementation

[0026] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0027] Please see Figures 1 to 14The present invention provides an extended display system comprising an identification and receiving module 1, an acquisition module 2, a processing module 3, a display module 4, a mobile terminal display screen 5, and a shared screen 6. The identification and receiving module 1 acquires all accessed user information, screen information, a source language corresponding to the user information, a first target language, and a second target language corresponding to the screen information. The acquisition module 2 acquires audio information input by an audio input device 7. The processing module 3 converts the audio information into original text, a first translated text, and a second translated text, respectively, based on the acquired source language, first target language, and second target language. The display module 4 includes at least two display screens, one for receiving and displaying the original text and the other for displaying the first translated text. The mobile terminal display screen 5 receives and displays the original text and the first translated text. The shared screen 6 receives and displays the original text, the first translated text, and the second translated text.

[0028] The identification and receiving module 1, acquisition module 2, processing module 3, display module 4, and audio input device 7 can be integrated or configured independently. Similarly, these modules can be integrated with the mobile terminal display screen 5 or configured independently. Display module 4 and mobile terminal display screen 5 are for individual user use, while shared screen 6 is for shared use by users in the same space. Users can choose display module 4 or mobile terminal display screen 5 as needed. Shared screen 6 is accessed based on actual communication needs.

[0029] Please see Figure 13 Specifically, the identification receiving module 1 identifies the user information of the connected user, the user's set first target language, and the screen information and corresponding second target language of the connected shared screen 6. The source language corresponding to the user information can be identified by the acquisition module 2 when input through the audio input device 7, or it can be set by the user. The display module 4 and the mobile terminal display screen 5 are owned by the user and can be bound to and stored with the user information. The display module 4 and the mobile terminal display screen 5 can be electrically connected to the identification receiving module 1. After connection, the identification receiving module 1 can obtain the relevant information of the connected user. When the user clicks to input the first target language on the display module 4 or the mobile terminal display screen 5, the user information is bound to the first target language and transmitted to the identification receiving module 1 for storage. In use, the user can select either the display module 4 or the mobile terminal display screen 5.

[0030] This invention discloses an extended display system. A receiving module 1 acquires user information, screen information, and a source language and a first target language corresponding to the user information, as well as a second target language corresponding to the screen information. An acquisition module 2 acquires input audio information. A processing module 3 generates original text, a first translated text, and a second translated text based on the source language, the first target language, and the second target language. These original text, first translated text, and second translated text are then pushed to their respective display modules 4, mobile terminal displays 5, and shared screens 6, depending on their connection to these modules. Each display module 4 includes at least two displays, allowing multiple different contents to be displayed simultaneously. This enables content comparison and better understanding, facilitating meetings and work. Furthermore, it can connect to mobile terminal displays 5 and shared screens 6, offering diverse expansion methods with fewer limitations. For applications requiring extensive negotiations and discussions, the shared screen 6 allows everyone to view the content simultaneously, providing multiple languages ​​such as original text, first translated text, and second translated text for quick understanding. This improves communication efficiency and ensures the smooth and efficient conduct of meetings. For users unable to attend in person, relevant content can be displayed on the mobile terminal screen 5, making it more flexible and not limited by location. The extended display system of this invention offers diverse expansion methods and can be applied in a variety of different scenarios.

[0031] Please see Figure 1 As shown, an extended display method of the present invention includes the following steps: Step S100: Obtain all user information, screen information, source language and first target language corresponding to the user information, and second target language corresponding to the screen information; Step S200: Obtain the audio information input by the audio input device 7; Step S300: Based on the acquired source language, first target language, and second target language, the audio information is converted into the original text, the first translated text, and the second translated text, respectively. Step S400a: Display the original text and the first translated text to the corresponding display module 4 according to the user information; Step S400b: Display the original text and the first translated text to the mobile terminal display screen 5 according to the user information; Step S400c: Display the original text, the first translated text, and the second translated text to the corresponding shared screen 6 according to the screen information.

[0032] Among them, the user information corresponds to the relevant user information of the access display module 4 or the mobile terminal display screen 5. The user can set the first target language and the source language on the display module 4 or the mobile terminal display screen 5. The source language can also be obtained after being input through the audio input device 7 and recognized. The screen information corresponds to the relevant screen information of the shared screen 6, such as the name and parameters of the shared screen 6 after access. The second target language is the language of the output display set on the shared screen 6.

[0033] In step S300, the first target language and the second target language are received, and then translated into the first translated text and the second translated text respectively. When the shared screen 6 is not connected, the second target language is not received, and therefore the second translated text will not be generated.

[0034] It is understood that steps S400a, S400b, and S400c do not represent the triggering order of display module 4, mobile terminal display screen 5, and shared screen 6. Furthermore, steps S400a, S400b, and S400c are triggered according to the access status of display module 4, mobile terminal display screen 5, and shared screen 6, and there may be cases of full triggering and partial triggering.

[0035] This invention provides an extended display method that acquires user information, screen information, and a source language and a first target language corresponding to the user information, as well as a second target language corresponding to the screen information. It also acquires input audio information and then forms original text, a first translated text, and a second translated text based on the source language, the first target language, and the second target language. These original text, first translated text, and second translated text are then pushed to the display modules 4, mobile terminal display screen 5, and shared screen 6 respectively, based on their connections to different display modules 4, mobile terminal display screen 5, and shared screen 6. The display module 4 includes at least two displays, allowing multiple different contents to be displayed simultaneously. This enables content comparison and better understanding, facilitating meetings and work. Furthermore, it can connect to the mobile terminal display screen 5 and the shared screen 6, offering diverse expansion methods with fewer limitations. For application scenarios requiring extensive negotiations and discussions, the shared screen 6 allows everyone to view the content simultaneously, providing text in multiple languages ​​such as original text, first translated text, and second translated text, enabling rapid understanding of the content. This improves communication efficiency and ensures the smooth and efficient progress of meetings. For users unable to attend in person, relevant content can be displayed on the mobile terminal screen 5, offering greater flexibility as it is not limited by location. This invention's extended display method offers diverse expansion options and can be applied to various scenarios.

[0036] Please see Figure 2The display screen includes a first screen 41 and a second screen 42; step S100, obtaining all accessed user information, screen information, source language and first target language corresponding to the user information, and second target language corresponding to the screen information, further includes the following steps: Step S101: Identify and store the number of user information entries, the source language corresponding to the user information, and the first target language; Step S102: Identify and store the access signal of the first screen 41 and the access signal of the second screen 42; Step S103: Identify and store the access signal of the mobile terminal display screen 5; Step S104: Identify the access signal of the shared screen 6, obtain screen information and the second target language corresponding to the screen information, and store them.

[0037] The identification and receiving module 1 identifies the access signals from the connected display screen, mobile terminal display screen 5, and shared screen 6, etc. After identification, it stores the access signals and retrieves user information, screen information, and the set first and second target languages ​​based on the access signals, storing them accordingly. The source language can be input by the user and received and stored by the identification and receiving module 1, or it can be input via audio input device 7, where the source language is identified, matched with the corresponding user, and transmitted to the identification and receiving module 1 for reception and storage. When displaying the original text, the first translated text, and the second translated text, the output is based on the corresponding access signals. The display expansion is highly flexible, allowing for the addition or removal of expansion methods as needed without wasting resources, and expansion methods can be added when necessary.

[0038] In this embodiment, users enter the same group by scanning a QR code, entering the same group code, or clicking a link through the selected display module 4 or mobile terminal display screen 5. The identification and receiving module 1 obtains all user information within the same group, the number of users, and the user's first target language, as well as the access signal and second target language of the shared screen 6. A roster is established, storing user information, the first target language, and the access status of the corresponding display module 4 and mobile terminal display screen 5, as well as the access status and corresponding second target language of the shared screen 6. The roster is updated in real time to detect changes in the access signal and adjust the contents accordingly. For example, during a meeting, if users initially enter the same group, the roster stores the information of all users who have entered the same group. If a new user joins during the meeting, their information is added to the roster, and a first translated text is generated corresponding to the user's first target language. The original text and the first translated text are then transmitted to the user's display module 4 or mobile terminal display screen 5, allowing the user to quickly follow the progress of the meeting and facilitate its advancement. If a user leaves during the negotiation process, their information will be removed from the roster to avoid wasting resources translating for those who have left, thus facilitating a faster completion of the original text and the first translation. Shared screens that join or leave midway will also be added and removed in the same way, which will not be elaborated upon here.

[0039] Please see Figure 3 The extended display method also includes the following steps: Step S501: Read the access signal of the first screen 41, the access signal of the second screen 42, the access signal of the mobile terminal display screen 5, and the access signal of the shared screen 6. In step S502a, when only the access signal of the first screen 41 is received, both the original text and the first translated text are displayed on the first screen 41. In step S502b, when the access signal from the first screen 41 and the access signal from the second screen 42 are received, the original text and the first translated text are displayed on the first screen 41 and the second screen 42, respectively. In step S502c, when the access signal of the shared screen 6 is received, the original text, the first translated text, and the second translated text are displayed in columns on the shared screen 6. In step S502d, when the access signal of the mobile terminal display screen 5 is received, the original text and the first translated text are displayed on the mobile terminal display screen 5.

[0040] Please see Figure 10 and Figure 12Specifically, the system reads in real time the access signals from each user's first screen 41, second screen 42, mobile terminal display screen 5, and shared screen 6. Taking a meeting between users A, B, and C as an example, user A accesses the group through display module 4, user B through display module 4, and user C through mobile terminal display screen 5, simultaneously accessing shared screen 6. After users A, B, and C scan the same QR code to join the group, their relevant information, such as name, company, and position, is obtained. User A inputs their first target language (e.g., Chinese) through their display module 4, user B inputs their first target language (e.g., Japanese) through their display module 4, and user C inputs their first target language (e.g., English) through mobile terminal display screen 5. The second target language set on the shared screen 6 is Korean. All of this information is stored in the roster. During the meeting, User A's display module 4 connected to both the first screen 41 and the second screen 42 simultaneously, while User B's display module 4 only connected to the first screen 41 and also connected to the shared screen 6. User A speaks through audio input device 7. Acquisition module 2 receives the audio information and identifies the source language of User A corresponding to the audio information. Then, it converts the audio information into original text based on User A's source language. At the same time, according to the first target language of User B and User C and the second target language of shared screen 6, the original text is translated into Japanese version, English version, and Korean version. The original text and the Japanese version of the first translation text are pushed to User B's display module 4. Since User B is only connected to the first screen 41, the original text and the first translation text are displayed on the first screen 41 simultaneously in a column or scrolling manner. User A's display module 4 only displays the original text on the first screen 41. User C's mobile terminal display screen 5 displays the original text and the English version of the first translation text in a column or scrolling manner. The shared screen 6 displays the original text, the Japanese version of the first translation text, and the Korean version of the second translation text in a column. When user B speaks, since both the first screen 41 and the second screen 42 of user A's display module 4 are connected, the original text of user B's speech in the language version will be displayed on user A's first screen 41, and the first translated text in Chinese will be displayed on the second screen 42.

[0041] During the meeting, if user B connects to the second screen 42, the connection signal of the second screen 42 will be stored. Subsequently, the first screen 41 will display the original text, and the second screen 42 will display the first translated text. If user C exits midway, all relevant information about user C will be removed from the roster, and the first translated text in English will no longer be translated. If user D is added, relevant information about user D will be added to the roster, and the first translated text in the first target language of user D will be translated. If shared screen 6 exits midway through the meeting and its connection signal is no longer detected, its relevant information will be removed. Similarly, other shared screens 6 can be added midway through the meeting, their information will be added, and the second translated text will be translated accordingly.

[0042] In short, during the meeting, the system monitors the connection status of each display module 4—the first screen 41, the second screen 42, the mobile terminal display screen 5, and the shared screen 6—in real time and displays accordingly. Simultaneously, the first screen 41 and the second screen 42 display the original text and the first translated text respectively, eliminating the need to cram them onto the same screen for easier viewing. Furthermore, the two displays facilitate the expansion of other functions. The shared screen 6 allows for the setting of more languages ​​to facilitate communication, provides access for more users, and facilitates negotiations and meeting progress. It can also detect the connection signal in real time and output corresponding text based on the situation, enabling expanded display. The display expansion is highly flexible, allowing for the addition or removal of expansion methods as needed without wasting resources and improving display efficiency.

[0043] Furthermore, the display module 4, equipped with a first screen 41 and a second screen 42, can support two users simultaneously using the same display module 4 for negotiations. Specifically, in a face-to-face negotiation, the first screen 41 faces one of the negotiators, and the second screen 42 faces the other. Normally, the text displayed on the first screen 41 and the second screen 42 is in the same direction, which causes the text seen by the other negotiator to be reversed, making it difficult for them to read the text and resulting in low communication efficiency. Therefore, by clicking the settings and sending a software command, the content of one of the displays on the first screen 41 or the second screen 42 can be reversed, allowing the other negotiator to see the text correctly and directly read the text content without having to repeatedly flip the translator or move around frequently to see the text. When users sit face-to-face, they can negotiate better and more efficiently.

[0044] Please see Figure 4 The extended display method also includes the following steps: Step S601: Connect the display module 4 to the shared screen 6; Step S602: Based on the screen information, control the shared screen to display the original text, the first translated text, and the second translated text in 6 columns.

[0045] The shared screen 6 is connected via display module 4. After connection, the access signal of the shared screen 6 is transmitted to the recognition and receiving module 1 to complete the access of the shared screen 6 and acquire screen information and the second target language. This allows the content originally pushed to display module 4 to also be pushed to the shared screen 6. The shared screen 6 displays the original text corresponding to the connected display module 4, the first translated text, and the second translated text corresponding to the second target language in columns. The shared screen 6 can achieve multilingual display expansion and can be viewed by everyone, improving communication and work efficiency. In another embodiment, the shared screen 6 can also be directly connected, and the access signal can be transmitted to the recognition and receiving module 1. After connection, the second target language is set. After the speaker speaks, during output, only the original text in the speaker's source language and the second translated text corresponding to the second target language are displayed on the shared screen 6, or only the second translated text corresponding to the second target language is displayed.

[0046] Step S601, connecting the display module 4 to the shared screen 6, also includes the following steps: entering the same group code or scanning the QR code to connect the display module 4 to the shared screen 6.

[0047] Display module 4 can quickly connect to shared screen 6 by entering the same group code or QR code. Connecting via group code input or QR code scanning is efficient and convenient.

[0048] Please see Figure 5 Step S300, converting the audio information into original text, first translated text, and second translated text according to the acquired source language, first target language, and second target language respectively, further includes the following steps: Step S301: Input the large language model 31; wherein, the large language model 31 stores the original text as context data after being corrected and optimized by the large language model 31. Step S302: Obtain the current audio information, optimize and correct the audio information using the large language model 31, and output the original text. Step S303: Optimize and correct the original text using context data, output the optimized and corrected original text, and store it as new context data in the large language model 31; Step S304: Translate the original text after optimization and correction of context data, and output the first translated text and / or the second translated text.

[0049] By inputting into the large language model 31, based on potential issues such as accents and speech impediments in the user's audio input, basic corrections and optimizations are first performed, outputting the original text. Then, the stored optimized original text is retrieved as context data. This context data is then used to further optimize and correct the currently output original text before outputting it again. This output original text is then stored as new context data in the large language model 31, serving as the basis for correcting the context data of the next input audio information. The original text, after basic correction and context optimization, is then translated to obtain a first translated text and a second translated text that correspond one-to-one with the set first and second target languages. Through the corrections made by the large language model 31 and the corrections combined with context data, the accuracy of the output original text, the first translated text, and / or the second translated text can be improved, thereby enhancing communication efficiency and information accuracy.

[0050] Please see Figure 11 In this embodiment, specifically, the large language model 31 is pre-inputted and stored in the processing module 3. The processing module 3 also includes a speech-to-text module 32, which is used to convert audio signals into text. During the meeting, the user speaks through the audio input device 7. After the acquisition module 2 receives the audio signal, it transmits it to the processing module 3. The processing module 3 first triggers the speech-to-text module 32. After receiving the audio signal, the speech-to-text module 32 first outputs an initial version of the original text (i.e., uncorrected and unoptimized text) to the user's display module 4 or mobile terminal display screen 5. Figure 11The left side of the output (directly outputting the original text) indicates that the speech-to-text module 32 has received the audio signal and is operating normally, improving the user experience, reducing output latency, and reducing user waiting time. Meanwhile, the uncorrected and unoptimized original text enters the processing branch. First, the source language of the audio information is detected. If the source language matched by the audio information is not configured with a processing strategy, it is directly output according to the current original text. If the source language matched by the audio information is configured with a processing strategy, it proceeds to the next step, calling the large language model 31. If the call fails, the current original text is directly output. If the call succeeds, the large language model 31 first performs basic corrections such as accent and speech impediments on the original text, then reads the stored context data, and combines the context data to correct and optimize the original text. This corrected and optimized original text is stored as the context data for the next input original text. Simultaneously, the corrected original text is input to the translation module 33 for translation, and the first and second translated texts are output according to the access signal. In this embodiment, the original text is iteratively processed multiple times using continuously updated context data during the meeting to obtain a highly accurate original text. This is then translated to obtain a highly accurate first and second translated text. The end user can then see the speaker's name, speaking time, and the original text of their speech, along with the corresponding first translated text, on their display module 4 or mobile terminal display screen 5. The shared screen 6 will also display the speaker's name, speaking time, and the original text of their speech, along with the corresponding first and second translated texts.

[0051] Please see Figure 6 Step S304, translating the original text after context data optimization and correction, and outputting the first translated text and / or the second translated text, further includes the following steps: Step S3041: Connect at least one translation model; Step S3042: Select and match the translation model; Step S3043: Input the original text into the translation model and output the first translated text and / or the second translated text.

[0052] The system inputs at least one translation model and can select an appropriate model based on the content, domain, and dialogue scenario of the source text. After matching, it translates the source text and outputs a first translated text and / or a second translated text. This ability to select the appropriate translation model based on different situations improves translation quality and efficiency, optimizes resources, and avoids waste.

[0053] Specifically, the translation module 33 stores various translation models, including but not limited to conventional translation models (mainly applicable to ordinary sentence dialogue translation), real-time assisted translation models (mainly applied to interpreting, optimizing tone), long text translation models (used to handle long paragraph dialogues, combining context to ensure the quality of long text translation), and other translation models. The translation module 33 displays and identifies the language and content of the original text, matches the appropriate translation model, inputs the original text into the matching translation model, and translates the corresponding first and second translated texts in the target language one by one. Finally, the user information, the corresponding first and second translated texts, and the speaking timestamp are packaged and output to the corresponding display module 4, mobile terminal display screen 5, and shared screen 6.

[0054] Please see Figure 7 Step S300, converting the audio information into original text, first translated text, and second translated text according to the acquired source language, first target language, and second target language respectively, further includes the following steps: Step S305: Input the dictionary model 34 corresponding to the professional field; Step S306: Obtain audio information and correct and optimize it using dictionary model 34 to output the original text, the first translated text, and / or the second translated text.

[0055] By pre-inputting a dictionary model 34 for a specific professional field, during voice dialogues or meetings, the dictionary model 34 can be combined to more quickly identify professional terms in the voice input, significantly improve the accuracy of professional term recognition, and enhance semantic understanding and contextual coherence, making the conversion of audio into original text, first translated text, and second translated text more accurate.

[0056] Specifically, processing module 3 stores a dictionary model 34 for various professional fields, including but not limited to fields such as medicine, furniture, and architecture. The dictionary model 34 contains professional terms from multiple different fields, which can be user-defined or directly integrated from existing dictionary models 34. Through user-defined dictionaries, users can access dictionary management to create or edit professional terms, commonly used phrases within the company, or industry terminology. After creating or importing a dictionary model 34, it is stored in processing module 3. When enabling a dictionary model 34, one or more dictionary models 34 can be selected, or the professional field can be identified directly based on the audio information to enable the corresponding dictionary model 34. Upon receiving the audio signal, the service provider is identified, and different processing strategies are matched according to different service providers. These strategies include enhanced professional terminology keyword recognition and injection of professional terms. Professional terms are identified through different processing strategies, and the corresponding professional terms are extracted from dictionary models 34 and output to form the original text. In translation module 33, the terminology list and mapping table from dictionary models 34 are injected into translation module 33 to ensure that the professional terms in the output first and second translated texts correspond to the original text.

[0057] The dictionary model 34 and the large language model 31 can run sequentially, with the order set as needed, or they can be processed in parallel and synchronously. The input models include, but are not limited to, the dictionary model 34 and the large language model 31, and models can be added or removed according to the actual use case.

[0058] Please see Figure 8 , Figure 9 and Figure 14 Step S200, obtaining the audio information input by the audio input device 7 further includes the following steps: Step S201: Interleave the signals of the first channel and the second channel of the audio input device 7 and convert them into a data stream; Step S202: Split the data stream, extract the even-numbered bytes of data into the first audio information of the first channel, and extract the odd-numbered bytes of data into the second audio information of the second channel.

[0059] Interleaving the first and second audio channels improves data processing efficiency, reducing the number of calls and buffer switching overhead compared to reading the two channels separately. Interleaving ensures that the first and second channel data are physically adjacent at the same timestamp, guaranteeing timing synchronization and preventing audio breaks or translation errors caused by buffer misalignment during decoding. Furthermore, by default, interleaving simplifies subsequent processing and reduces code complexity. It also supports simultaneous input from two audio input devices (7), improving meeting efficiency.

[0060] In this embodiment, the audio input device 7 includes dual microphones, etc., and includes a first channel and a second channel. Dual-beam directional sound pickup technology is used to record a user's voice through the first channel and the second channel. Audio input can be performed through the first channel and the second channel respectively, or independently through either the first channel or the second channel, or simultaneously through both. The processing module 3 includes a split processor 35, a first sub-processor 36, and a second sub-processor 37. Specifically, an analog signal is input through audio input device 7. After receiving the analog signal, acquisition module 2 converts it into a digital signal, then interleaves the signals from the first and second channels and converts them into a data stream. This data stream is arranged in the order of first channel-second channel-first channel-second channel. The data stream is input to processing module 3. The split processor 35 of processing module 3 first deinterleaves the data stream in 16-bit frames, splitting it into a first channel data stream and a second channel data stream. The amplitude of each channel is calculated. Calculating the amplitude of the first and second channels can be used to identify whether the input is human voice or ambient noise, and can also detect whether audio input device 7 is damaged and needs replacement. If the amplitude indicates that the first channel is a valid input (indicating human voice input) and the second channel is an invalid input (indicating only ambient noise), then in subsequent processing, only the first audio information is processed. When the amplitude determines that both the first and second channels are valid inputs (voice input), the first and second audio information are processed separately in subsequent processing. Even if two users speak at the same time using the first and second channels respectively, the processing module 3 can distinguish them based on amplitude and volume, preventing audio confusion. In this embodiment, deinterleaving is performed using 16-bit frames. A 16-bit frame provides approximately 96dB of dynamic range, sufficient to cover the perceptible dynamic range of human speech, while each sampling point occupies only 2 bytes. Alternatively, 8-bit, 24-bit, or 32-bit frames can also be used.

[0061] The acquisition module 2 reads the input status of the first and second channels of the audio input device 7 alternately in real time. The split processor 35 splits the data stream and identifies and judges the first and second audio information. Taking the first and second audio information as valid inputs as an example, the split data streams are sent to the first sub-processor 36 and the second sub-processor 37 of the processing module 3 respectively. The first sub-processor 36 and the second sub-processor 37 are independent of each other. The first sub-processor 36 and the second sub-processor 37 are both equipped with a speech-to-text module 32, which performs speech-to-text conversion on the first channel data stream and the second channel data stream respectively, generating the first original text and the second original text respectively. The first original text is marked with a first mark, and the second original text is marked with a second mark. The first mark and the second mark contain the user information of the speaker. The first mark and the second mark are used to distinguish the first original text and the second original text to avoid confusion and affect the correction and translation in subsequent processing. After generating the first and second source texts, the first and second source texts are associated to form a conversational source text in chronological order. This source text is then sent to the large language model 31 for basic corrections, such as typo correction. The large language model 31 retrieves the stored context data and optimizes and corrects the source text to obtain the optimized source text. This optimized source text is then used as the new context data storage. The source text is sent to the translation module 33, where a translation model is selected based on the length and content of the source text. Based on the access signals from the display module 4 and the shared screen 6, as well as the stored user information and screen information, the translation module translates the source text to form the corresponding first and second translated texts. Both the first and second translated texts are conversational translated texts arranged in chronological order. When each user is configured with a separate audio input device 7 for input, taking the first audio information as valid input and the second audio information as invalid input as an example, only the first audio information is input to the first subprocessor 36 for processing and generating the original text. The original text is then sent to the large language model 31 for basic correction, such as typo correction. The large language model 31 retrieves the stored context data and optimizes and corrects the original text to obtain the optimized original text. This original text is used as the new context data storage. The original text is sent to the translation module 33, and the translation model is selected according to the length and content of the original text. Based on the access signals from the display module 4 and the shared screen 6, as well as the stored user information, the translation is performed to form the corresponding first translated text and second translated text. The first translated text and second translated text are both conversational translated texts arranged in chronological order.

[0062] Combination Figures 1 to 14The extended display method and system of the present invention operate as follows: The system identifies the access signals of the first screen 41 and second screen 42 of the display module 4, the mobile terminal display screen 5, and the shared screen 6 accessed by the user. Based on the access signals, it acquires relevant user information, the user's set first target language, and the screen information and second target language of the shared screen 6. The user speaks through the audio input device 7, and the speech is converted into original text by the speech-to-text module 32. The original text is then input into the large language model 31 and the dictionary model 34. The original text is corrected and optimized using context data, basic corrections, and technical terms, and the optimized original text is output. This optimized original text is stored as new context data. A translation model is selected based on the content of the original text, and the text is input into the matching translation model. The first translated text and the second translated text are then translated according to the first and second target languages. Finally, the original text, the first translated text, and the second translated text are distributed according to the access signals. For users joining midway through the process, the display module 4, mobile terminal display screen 5, and shared screen 6 add the new access signal and user information to the roster. Subsequent processing will generate the first and second translated texts accordingly. For users leaving midway through the process, all related information is removed from the roster, and this removed information will not be translated or pushed again. Simultaneously, it supports two users inputting audio signals synchronously. The audio signal data stream is input through interleaved reading. After receiving the data stream, the processing module 3 splits it into first and second audio information, then processes them independently to generate first original text with a first marker and second original text with a second marker. The context data with the first and second markers is then combined and processed to correct the first and second original texts before outputting them. Finally, translation is performed to generate the first and second translated texts.

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An extended display method, characterized in that, It includes a display module (4), a mobile terminal display screen (5), and a shared screen (6), wherein the display module (4) includes at least two displays; The extended display method includes the following steps: Obtain all user information, screen information, source language and first target language corresponding to the user information, and second target language corresponding to the screen information; Acquire the audio information input by the audio input device (7); Based on the acquired source language, first target language, and second target language, the audio information is converted into the original text, the first translated text, and the second translated text, respectively. The original text and the first translated text are displayed to the corresponding display module (4) according to the user information; The original text and the first translated text are displayed on the mobile terminal display screen (5) according to the user information; Based on the screen information, the original text, the first translated text, and the second translated text are displayed on the corresponding shared screen (6).

2. The extended display method according to claim 1, characterized in that, The display screen includes a first screen (41) and a second screen (42); obtaining all user information, screen information, source language and first target language corresponding to the user information, and second target language corresponding to the screen information further includes the following steps: Identify and store the number of user information entries, the corresponding source language, and the first target language; Identify and store the access signals of the first screen (41) and the second screen (42); Identify and store the access signal of the mobile terminal display screen (5); Identify the access signal of the shared screen (6), obtain screen information and the second target language corresponding to the screen information, and store them.

3. The extended display method according to claim 2, characterized in that, It also includes the following steps: Read the access signal of the first screen (41), the access signal of the second screen (42), the access signal of the mobile terminal display screen (5) and the access signal of the shared screen (6); When only the access signal of the first screen (41) is received, both the original text and the first translated text are displayed on the first screen (41); When the access signal from the first screen (41) and the access signal from the second screen (42) are received, the original text and the first translated text are displayed on the first screen (41) and the second screen (42) respectively. When the access signal of the shared screen (6) is received, the original text, the first translated text and the second translated text are displayed in columns on the shared screen (6); When the access signal is received from the mobile terminal display screen (5), the original text and the first translated text are displayed on the mobile terminal display screen (5).

4. The extended display method according to claim 1, characterized in that, It also includes the following steps: Connect the display module (4) to the shared screen (6); Based on the screen information, control the shared screen (6) to display the original text, the first translated text, and the second translated text in columns.

5. The extended display method according to claim 4, characterized in that, Connecting the display module (4) to the shared screen (6) also includes the following steps: Entering the same group code or scanning the QR code will display module (4) connected to the shared screen (6).

6. The extended display method according to claim 1, characterized in that, Converting audio information into original text, first translated text, and second translated text based on the acquired source language, first target language, and second target language also includes the following steps: Input the large language model (31); wherein the large language model (31) stores the original text that has been corrected and optimized by the large language model (31) as context data; Obtain the current audio information, optimize and correct the audio information through the large language model (31), and output the original text. The original text is optimized and corrected using context data, and the optimized and corrected original text is output and stored as a new context data in the large language model (31); Translate the original text after contextual data optimization and correction, and output the first translated text and / or the second translated text.

7. The extended display method according to claim 6, characterized in that, Translating the original text after contextual data optimization and correction, and outputting the first translated text and / or the second translated text, also includes the following steps: Connect to at least one translation model; Select and match translation models; The original text is input into the translation model, which outputs the first translated text and / or the second translated text.

8. The extended display method according to claim 1, characterized in that, Converting audio information into original text, first translated text, and second translated text based on the acquired source language, first target language, and second target language also includes the following steps: Input the dictionary model for the corresponding professional field (34); The audio information is obtained and corrected and optimized using a dictionary model (34) to output the original text, the first translated text, and / or the second translated text.

9. The extended display method according to claim 1, characterized in that, Acquiring the audio information input by the audio input device (7) further includes the following steps: Interleaving the first channel signal and the second channel signal of the audio input device (7) and converting them into a data stream; The data stream is split, and the even-numbered bytes are extracted as the first audio information of the first channel, while the odd-numbered bytes are extracted as the second audio information of the second channel.

10. An extended display system, characterized in that, include: The identification receiving module (1) is used to obtain all user information, screen information, source language and first target language corresponding to the user information, and second target language corresponding to the screen information; The acquisition module (2) is used to acquire the audio information input by the audio input device (7); The processing module (3) is used to convert the audio information into the original text, the first translated text and the second translated text respectively according to the source language, the first target language and the second target language. The display module (4) includes at least two displays for receiving and displaying the original text and the first translated text; The mobile terminal display screen (5) is used to receive and display the original text and the first translated text; Shared screen (6) is used to receive and display the original text and the first and second translated texts.

Images