Data transmission method and apparatus

Abstract

A data transmission method and apparatus, which are used for flexibly sharing a conference file in a conference. The method comprises: a media service apparatus determining first media data displayed on a screen of a first terminal, wherein the first media data comprises text content of a first language; if a second terminal chooses to use a second language, then the media service apparatus determining second media data corresponding to the first media data, wherein the second media data comprises text content of the second language, and the first terminal and the second terminal are connected to the same conference; and the media service apparatus controlling the screen display content of the second terminal according to the second media data. When the first terminal initiates data sharing, if the second terminal and the first terminal use different languages, media data of the first language that is displayed on the screen of the first terminal can be converted into media data of the second language, and the media data of the second language is displayed on the second terminal, thereby realizing flexible transmission of the media data in a video conference scenario.

Description

Data transmission method and device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on June 29, 2023, with application number 202310786174.4 and application name “A method and device for multi-channel sharing of video conferences”, the entire contents of which are incorporated by reference into this application; this application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on September 18, 2023, with application number 202311203704.4 and application name “A method and device for data transmission”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of communication technology, and in particular to a data transmission method and device. Background Art

[0004] In a video conferencing scenario, multiple endpoints can access the same conference through a conference server. During a video conference, the target endpoints among the multiple endpoints can share conference files.

[0005] When the conference files used in the meeting are displayed on the target terminal, the user can operate the target terminal to perform screen sharing to realize sharing of the conference files displayed on the screen of the target terminal. For example, the target terminal can send the display content on the screen to the conference server, and the conference server sends the display content on the target terminal screen to other terminals in the conference. Through this conference file sharing method, the display content on the screens of multiple terminals in the conference is exactly the same; when the language used by the participants in the meeting is different from the language in the shared conference file, the participants cannot understand the content in the shared conference file (for example, when the conference file shared by the target terminal is a Chinese file, if the participant is an English user, the participant cannot understand the content in the Chinese file). Therefore, the current conference file sharing method is not flexible enough.

[0006] Summary of the Invention

[0007] The present application provides a data transmission method and device for flexibly sharing conference files in a conference.

[0008] In a first aspect, an embodiment of the present application provides a data transmission method, which can be executed by a media service device, wherein the media service device determines first media data displayed on the screen of a first terminal, where the first media data includes text content in a first language; if the second terminal chooses to use a second language, the media service device generates second media data corresponding to the first media data, where the second media data includes text content in the second language; the first terminal and the second terminal are connected to the same conference; the media service device can control the screen display content of the second terminal based on the second media data.

[0009] Through the above method, when the embodiment of the present application initiates data sharing on the first terminal, when the language used by the second terminal is different from that used by the first terminal, for example, the first terminal uses the first language and the second terminal uses the second language; the media service device can generate second media data (including text data in the second language) corresponding to the first media data based on the first media data (including text data in the first language) displayed on the screen of the first terminal, and the second media data can be displayed on the screen of the second terminal, so that the user of the second terminal can watch the media data suitable for his or her own language habits during the meeting, thereby realizing flexible transmission of media data in the video conferencing scenario and further improving the user experience.

[0010] In a possible implementation, the first media data is a first video frame in a target video obtained by recording the screen of the first terminal, and the second media data is a second video frame.

[0011] In the above manner, data synchronization between the first terminal and the second terminal is achieved by recording the screen of the first terminal.

[0012] In one possible implementation, the media service device receives a target video sent by the first terminal, where the target video includes multiple video frames; the media service device determines the first video frame from the target video, where the first video frame is a video frame in the target video whose display content has changed compared to a previous frame.

[0013] In the above manner, the media service device uses the video frame in which the display content of the target video is changed compared with the previous frame as the first video frame, which can reduce the amount of data synchronized to the second terminal.

[0014] In a possible implementation, the media service device translates text content in the first video frame to obtain a second video frame corresponding to the first video frame.

[0015] Through the above method, the media service device translates the text content in the first video frame into text content in the second language, so that the second video frame synchronized to the second terminal includes text content in the second language, which can facilitate the user of the second terminal to watch media data suitable for their own language habits and improve the user experience.

[0016] In a possible implementation, the first media data is text data in a first online document, and the second media data is text data in a second online document; the text data in the first online document and the text data in the second online document represent the same meaning.

[0017] Through the above-mentioned method, the media service device provides a solution for synchronizing the display content of the first terminal and the second terminal through the online document.

[0018] In one possible implementation, the media service device determines the text data in the second online document based on the address information and document location information of the second online document; wherein the document location information is determined based on the text data of the first online document currently displayed on the screen of the first terminal.

[0019] In the above manner, the media service device determines the second media data according to the address information and the document location information of the second online document, thereby ensuring synchronization of display content on the first terminal and the second terminal.

[0020] In a possible implementation, the document location information is first display location information of the first online document, where the first display location information represents the page number and / or paragraph of the text data displayed on the screen of the first terminal in the first online document.

[0021] In one possible implementation, the document location information is second display location information of the second online document, where the second display location information represents the page number and / or paragraph of the text displayed on the screen of the second terminal in the second online document; the second display location information is determined based on the first display location information of the first online document, where the first display location information represents the page number and / or paragraph of the text displayed on the screen of the first terminal in the first online document.

[0022] In a possible implementation, the media service device may send document location information to the second terminal.

[0023] In the above manner, the media service device sends the document location information to the second terminal, and the second terminal can determine the second media data according to the document location information, thereby ensuring synchronization of display content between the first terminal and the second terminal.

[0024] In one possible implementation, the media service device controls the screen display content of the second terminal based on the second media data and the collaboration information sent by the first terminal; wherein the collaboration information is used to indicate the location and / or operation content of the operation triggered by the user on the screen of the first terminal.

[0025] In the above manner, the media service device synchronizes the collaboration information triggered by the user on the first terminal to the second terminal, thereby ensuring data synchronization of the content displayed on the first terminal and the second terminal during the conference.

[0026] In a possible implementation, the media service device synthesizes the second media data and the collaboration information, and controls the screen display content of the second terminal according to the synthesized data.

[0027] In a possible implementation, when the media service device is deployed separately from the second terminal, the media service device may send collaboration information and the second media data to the second terminal. The collaboration information may be used to be displayed superimposed with the second media data on the screen of the second terminal.

[0028] In the second aspect, an embodiment of the present application further provides a data transmission device, which has the behavioral function of implementing the media service device in the method example of the first aspect above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The functions can be implemented by hardware, or by hardware executing corresponding software implementations. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the structure of the data transmission device includes a determination unit, a processing unit, and a display control unit, which can perform the corresponding functions in the method example of the first aspect above. Please refer to the detailed description in the method example for details, which will not be repeated here.

[0029] In a third aspect, an embodiment of the present application further provides a computing device having the function of implementing the behavior of the terminal device in the method example of the first aspect above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The structure of the computing device includes a processor and a memory, and the processor is configured to support the computing device to perform the corresponding functions in the first aspect above. The memory is coupled to the processor and stores the computer program instructions and data necessary for the computing device. The structure of the computing device also includes a communication interface for communicating with other devices.

[0030] In a fourth aspect, an embodiment of the present application further provides a computing device cluster, which has the function of implementing the behavior of the terminal device in the method example of the first aspect above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The computing device cluster includes at least one computing device, and the structure of any computing device includes a processor and a memory. The processor in any computing device is configured to support the computing device to perform part or all of the functions of the first aspect and each possible implementation of the first aspect above. The memory is coupled to the processor, which stores the computer program instructions and data necessary for the computing device. The structure of the computing device also includes a communication interface for communicating with other devices.

[0031] In a fifth aspect, the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the method in the above-mentioned first aspect and various possible implementations of the first aspect.

[0032] In a sixth aspect, the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first aspect and various possible implementations of the first aspect.

[0033] In a seventh aspect, the present application also provides a computer chip, which is connected to a memory and is used to read and execute a software program stored in the memory to execute the method in the above-mentioned first aspect and various possible implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1A is a schematic diagram of the architecture of a data transmission system provided in an embodiment of the present application;

[0035] FIG1B is a schematic diagram of the architecture of a data transmission system provided in an embodiment of the present application;

[0036] FIG2 is a flow chart of a data transmission method provided in an embodiment of the present application;

[0037] FIG3 is a schematic diagram of a display interface of a terminal provided in an embodiment of the present application;

[0038] FIG4 is a schematic diagram of data sharing of a terminal provided in an embodiment of the present application;

[0039] FIG5 is a schematic diagram of a display page of a conference APP provided in an embodiment of the present application;

[0040] FIG6 is a schematic diagram of the architecture of a data transmission system provided in an embodiment of the present application;

[0041] FIG7 is a schematic diagram of screen display content of a terminal provided in an embodiment of the present application;

[0042] FIG8 is a schematic diagram of a data transmission process according to an embodiment of the present application;

[0043] FIG9 is a schematic diagram of a data transmission process according to an embodiment of the present application;

[0044] FIG10 is a schematic diagram of a data transmission process according to an embodiment of the present application;

[0045] FIG11 is a schematic diagram of screen display content of a terminal provided in an embodiment of the present application;

[0046] FIG12 is a schematic diagram of the architecture of a data transmission system provided in an embodiment of the present application;

[0047] FIG13 is a schematic diagram of a data transmission process according to an embodiment of the present application;

[0048] FIG14 is a schematic diagram of a data transmission process according to an embodiment of the present application;

[0049] FIG15 is a schematic structural diagram of a data transmission device provided in an embodiment of the present application;

[0050] FIG16 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0051] FIG17 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. The terms used in the implementation methods of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0053] In the embodiments of the present application, "multiple" refers to two or more. In view of this, in the embodiments of the present application, "multiple" can also be understood as "at least two". "At least one" can be understood as one or more, for example, one, two or more. For example, including at least one means including one, two or more, and does not limit which ones are included. For example, including at least one of A, B and C, then the included ones may be A, B, C, A and B, A and C, B and C, or A, B and C. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / ", unless otherwise specified, generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0054] Unless otherwise specified, ordinal numbers such as "first" and "second" in the embodiments of the present application are used to distinguish multiple objects and are not used to limit the order, timing, priority or importance of multiple objects.

[0055] Figure 1A shows a schematic diagram of the architecture of a data transmission system. The data transmission system includes a media server 10 and at least two terminals 11 (e.g., terminal 11_1, terminal 11_2, and terminal 11_3). The at least two terminals 11 can access the media server 10 via a communication network and can participate in a conference created by the media server 10.

[0056] At least two terminals 11 include a first terminal, which may be a terminal that initiates data sharing (such as terminal 11_1), and the other terminals are second terminals (such as terminal 11_2 and terminal 11_3). When terminal 11_1 initiates data sharing, the media service device 10 determines the first media data displayed on the screen of terminal 11_1, and the first media data includes text data in the first language. If terminal 11_2 chooses to use the second language, the media service device 10 generates second media data corresponding to the first media data, and the second media data includes text data in the second language. The media service device 10 controls the screen display content of terminal 11_2 based on the second media data; illustratively, the media service device 10 sends the second media data to terminal 11_2, and the second media data can be displayed on the screen of terminal 11_2; or, the media service device 10 sends the address information of the second media data to terminal 11_2, and terminal 11_2 can obtain the second media data based on the address information and display the second media data on the screen.

[0057] In an embodiment of the present application, when data sharing is initiated on the first terminal, if the language used by the second terminal is different from that used by the first terminal, for example, the first terminal uses the first language and the second terminal uses the second language; the media service device can generate second media data (including text content in the second language) corresponding to the first media data based on the first media data (including text content in the first language) displayed on the screen of the first terminal, and the second media data can be displayed on the screen of the second terminal, so that the user of the second terminal can watch the media data suitable for his or her own language habits during the meeting, thereby realizing flexible transmission of media data in the video conferencing scenario and further improving the user experience.

[0058] As shown in the architecture of the data transmission system in FIG1A , at least two terminals 11 and the media service device 10 can be deployed separately and independently. Alternatively, in one possible implementation, the media service device 10 can be deployed on the terminal 11. For example, as shown in the structure of the data transmission system in FIG1B , the data transmission system includes a media service device 10 and at least two terminals 11 (for example, represented as terminal 11_1 and terminal 11_2), and the media service device 10 is distributedly deployed on each terminal; or each terminal 11 in the data transmission system includes a media service device 10 (for example, terminal 11_1 includes a media service device 10a, and terminal 11_2 includes a media service device 10b). At least two terminals 11 can access the same conference.

[0059] At least two terminals 11 include a first terminal, which may be the terminal initiating data sharing (e.g., terminal 11_1), and the other terminals are second terminals (e.g., terminal 11_2). When terminal 11_1 initiates data sharing, a media server 10a deployed in terminal 11_1 may determine the first media data displayed on the screen of terminal 11_1, where the first media data includes text data in a first language. If terminal 11_2 chooses to use a second language, media server 10a generates second media data corresponding to the first media data (or media server 10a sends the first media data to media server 10b, which generates second media data corresponding to the first media data), where the second media data includes text data in a second language. Media server 10b controls the screen display content of terminal 11_2 based on the second media data; illustratively, media server 10b controls the display of the second media data on the screen of the second terminal, or the media server obtains the second media data based on the address information of the second media data and displays the second media data on the screen of the second terminal.

[0060] It should be noted that the data transmission system architecture applicable to the above-mentioned present application can be as shown in Figure 1A or Figure 1B, but is not limited to the architecture shown in Figure 1A or Figure 1B, and the specific deployment form of the device in the data transmission system is not limited in the actual scenario.

[0061] The terminal 11 may be a conference terminal device, or the terminal 11 may also be a portable terminal device, such as a mobile phone, tablet computer, laptop computer, wearable device, smart screen device, etc. The media service device 10 may be a physical server, or a device cluster composed of multiple physical devices.

[0062] The media server 10 may be implemented by software or hardware.

[0063] As an example, the implementation of the media service device 10 is described below.

[0064] The media service device 10 is an example of a software functional unit. The media service device 10 may include code running on a computing instance. The computing instance may be at least one of a physical host (computing device), a virtual machine, a container, and other computing devices. Furthermore, the above-mentioned computing device may be one or more. For example, the media service device 10 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the application may be distributed in the same region or in different regions. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, and each AZ includes one data center or multiple data centers with close geographical locations. Generally, a region may include multiple AZs.

[0065] Similarly, the multiple hosts / virtual machines / containers used to run this code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is located within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0066] As an example of a hardware functional unit, the media service device 10 may include at least one computing device, such as a server. Alternatively, the media service device 10 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0067] The multiple computing devices included in the media service apparatus 10 can be distributed in the same region or in different regions. The multiple computing devices included in the media service apparatus 10 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the media service apparatus 10 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0068] The following describes the data transmission method provided in an embodiment of the present application by taking the data transmission system architecture shown in FIG1A as an example.

[0069] As shown in FIG2 , a data transmission method provided in an embodiment of the present application may include the following steps:

[0070] Step 201: The media service device determines first media data displayed on the screen of the first terminal.

[0071] In the data transmission method flow shown in Figure 2, a first terminal and a second terminal can access a media server. The first terminal and the second terminal are in the same conference created by the media server. In this conference, the first terminal can be the initiator of data sharing. In this conference, there can be one or more second terminals. This embodiment of the application does not limit the number of second terminals.

[0072] During implementation, users can operate their terminals, run the conference app (Meeting) on ​​the terminal, and join the meeting through the conference app. As shown in Figure 3, users can click the conference app installed on their terminals to start running the conference app; on the conference app's display page, they can enter the meeting number, or the conference number and password, to join the meeting.

[0073] It should be noted that the terminal that creates a conference through the media service device can be either the first terminal or the second terminal. For example, after the first terminal creates a conference, it notifies the user operating the second terminal of the conference number or the conference number and password of the created conference. Alternatively, the terminal that creates a conference can be a third-party terminal other than the first terminal and the second terminal. After the conference is created through the media service device, the conference number or the conference number and password of the created conference are notified to the users operating the first terminal and the second terminal.

[0074] In the following introduction, for the convenience of description, the user operating the first terminal may be referred to as the first user, and the user operating the second terminal may be referred to as the second user.

[0075] After the first and second terminals join the conference, if the first terminal is the initiator of data sharing, the user operating the first terminal can trigger the data sharing instruction and select the conference files to be shared in the conference. As shown in the data sharing diagram in Figure 4, the first terminal runs the conference app. The first user can trigger the data sharing instruction by clicking the "Share" control on the conference app display page. The first user can then select the conference files to be shared from the conference app display page to initiate data sharing.

[0076] In one possible implementation, the first terminal sends a data sharing instruction to the media service device. In response to the data sharing instruction, the media service device determines first media data displayed on the screen of the first terminal. Optionally, the first media data includes text content in a first language.

[0077] Step 202: If the second terminal chooses to use the second language, the media service device generates second media data corresponding to the first media data.

[0078] Optionally, when the second terminal accesses the conference, it can select the language to be used in the conference. For example, as shown in FIG5 , the display page of the conference app may include an option to select the conference language. The second user can select the language to be used for the second terminal. For example, if the second user's usual language is English, the second user can select "English".

[0079] When the first media data displayed on the screen of the first terminal includes text content in the first language, if the second user's commonly used language is the second language, and the language selected for use by the second terminal is the second language; the first language and the second language are different languages ​​(for example, the first language is Chinese and the second language is English). At this time, since the second user's commonly used language is the second language, if the first media data is displayed directly on the screen of the second terminal, the second user may not be able to understand the content displayed on the screen, affecting the second user experience. Based on this, when the media service device determines that the language selected for use by the second terminal is different from the language used in the text of the first media data displayed on the screen of the first terminal, the media service device determines the second media data corresponding to the first media data.

[0080] Optionally, the second media data includes text content in a second language.

[0081] It should be noted that the text content in the first language included in the first media data and the text content in the second language included in the second media data have the same meaning. It can be understood that after translating the text content in the first language into the second language, the text content in the second language can be obtained.

[0082] Step 203: The media service device controls the screen display content of the second terminal according to the second media data.

[0083] After determining the second media data corresponding to the first media data, the media service device may control the screen display content of the second terminal according to the second media data.

[0084] The media service device may control the screen display content of the second terminal in such a manner that the media service device sends the second media data to the second terminal, and the second terminal performs screen display according to the second media data; or the media service device may send the address information of the second media data to the second terminal, and after receiving the second media data, the second terminal obtains the second media data according to the address information of the second media data and displays it on the screen.

[0085] Since in an embodiment of the present application, in a conference including a first terminal and a second terminal, when the media service device determines that the language used in the first media data displayed in the first terminal serving as the data sharing terminal is different from the language selected by the second terminal, the second media data corresponding to the first media data can be determined based on the language selected by the second terminal, and the second media data is displayed on the screen of the second terminal, thereby meeting the needs of users of multiple different languages ​​in the conference and improving the user experience of the participants.

[0086] The data transmission scheme provided in the embodiment of the present application is that the first terminal and the second terminal are connected to the same conference, and when the first terminal initiates data sharing, the display content of the screens of the first terminal and the second terminal is synchronized through the media service device. When the text content of the first media data displayed on the first terminal is in the first language and the second terminal chooses to use the second language, the media service device can control the screen display content of the second terminal according to the second media data including the text content in the second language. Based on the above-mentioned data transmission scheme, the embodiment of the present application can have multiple implementation methods; for example, in implementation method one, the media service device translates the text content of the first media data to obtain the second media data, and sends the translated second media data to the second terminal; in implementation method two, by providing multi-language online documents, the first terminal uses the online document in the first language, and the media service device indicates the online document in the second language to the second terminal.

[0087] Different implementation modes are described below.

[0088] Implementation method one:

[0089] In the first embodiment, the first media data displayed on the screen of the first terminal is a first video frame in a target video obtained by recording the screen of the first terminal, and the second media data is a second video frame.

[0090] A first user operates a first terminal to display a conference document on a screen of the first terminal. The conference document may include, but is not limited to, image files, video files, audio files, and office documents (such as Word files, Excel files, Portable Document Format (PDF) files, and PowerPoint (PPT) files). The conference document includes text content in a first language. For example, if the first language is Chinese, the text content in the conference document is Chinese text.

[0091] When the conference file is displayed on the screen of the first terminal, the first terminal performs screen recording to obtain a target video.

[0092] An optional implementation method is that the first terminal sends a target video to a media service device, the media service device determines a first video frame from the target video, and when the second terminal chooses to use a second language, determines a second video frame corresponding to the first video frame, and the second video frame includes text content in the second language; after determining the second video frame, the media service device sends the second video frame to the second terminal.

[0093] It should be noted that when the media service device is implemented by hardware, the functions performed by the above media service device can be performed by one device, such as a media server; or can be performed by multiple devices in cooperation, such as a media server and a media gateway.

[0094] The following description will be made by taking the example of a media service device being executed by multiple devices in cooperation.

[0095] Optionally, the first embodiment of the present application may be applied to the architecture of a data transmission system as shown in FIG6 . The data transmission system includes a media server 61, a media gateway 62, a first terminal 63, and a second terminal 64. The media server 61 and the media gateway 62 may cooperate to perform the functions of the aforementioned media service device.

[0096] The media server 61 receives the target video sent by the first terminal 63 and forwards the target video of the first terminal 63 to the media gateway 62; the media gateway 62 determines a first video frame from the target video, and when the second terminal 64 chooses to use the second language, determines a second video frame corresponding to the first video frame; the media gateway 62 sends the second video frame to the media server 61, and after receiving the second video frame, the media server 61 sends the second video frame to the second terminal 64; the second terminal 64 displays the second video frame on the screen.

[0097] After receiving the target video forwarded by the media server, the media gateway determines the first video frame from the target video. The first video frame is a video frame in the target video whose display content has changed compared to the previous frame. Exemplarily, the target video includes video frame 1, video frame 2, video frame 3, video frame 4, video frame 5, video frame 6, video frame 7, video frame 8, and video frame 9. If the display content of video frame 3 differs from the display content of video frame 2, and the display content of video frame 8 differs from the display content of video frame 9, then video frame 3 and video frame 8 are the first video frames.

[0098] The media gateway can determine the first video frame from the target video in a variety of different ways, which are described in detail below.

[0099] Method 1 for determining the first video frame: The media gateway determines the first video frame from the target video according to the bit rate corresponding to the target video.

[0100] Optionally, the media gateway detects the bit rate of each video frame in the target video, and when detecting that the bit rate of the target video increases, determines that the video frame with the increased bit rate is the first video frame.

[0101] Because the first video frame that changes compared to the previous frame typically contains more details, a higher bit rate is required to encode and transmit these details to maintain stable video frame quality. During video encoding, the bit rate determines the number of bits transmitted per second. When the content in the image becomes more complex or the motion increases, the encoder needs to allocate more bits to represent these changes to maintain image quality. Alternatively, the encoder requires a higher bit rate to capture and transmit the changing information to avoid quality issues such as blurring, distortion, or pixelation. Therefore, when the media gateway detects a sudden increase in the bit rate, it determines the first video frame.

[0102] Method 2 for determining the first video frame: the media gateway compares the frame data of each video frame in the target video and determines the first video frame from the target video.

[0103] Optionally, the media gateway compares frame data of every two adjacent video frames in the target video to determine a change rate between every two adjacent video frames; and determines a video frame whose change rate compared to the previous frame is greater than a threshold as the first video frame.

[0104] During implementation, the media gateway can calculate the similarity between the frame data of two adjacent video frames based on an image similarity algorithm. A higher similarity indicates a lower rate of change between the two adjacent video frames; conversely, a lower similarity indicates a greater rate of change between the two adjacent video frames. Therefore, for example, the media gateway can use the inverse of the similarity as the rate of change.

[0105] After determining the first video frame, the media gateway determines the language selected by the second terminal. When determining that the second terminal selects the second language, the media gateway determines a second video frame corresponding to the first video frame.

[0106] In one possible implementation, the media gateway translates the text content in the first video frame to obtain a second video frame corresponding to the first video frame.

[0107] In implementation, the media gateway translates the text content in the first video frame into the second language, thereby obtaining a second video frame including the text content in the second language.

[0108] The media gateway can obtain a second video frame containing text content in a second language by invoking an AI service. The AI ​​service can be a service integrated with the media gateway, for example, a translation tool installed on the media gateway; or the AI ​​service can be a cloud service that performs translation functions, which the media gateway can invoke to translate the text content in the first video frame.

[0109] When the AI ​​service generates the second video frame, it can extract text content from the first video frame, translate the text content in the first language into the second language, and then fuse the text content in the second language with the content other than text in the first video frame to generate the second video frame. Alternatively, the AI ​​service can directly replace the text content in the first language in the first video frame with text content in the second language.

[0110] The above describes the process by which the media gateway determines the first video frame and the second video frame when the first video frame is a video frame in the target video whose displayed content has changed compared to the previous frame. In embodiments of the present application, the first video frame may also be every frame in the target video. It can be understood that the media gateway translates the text content in every frame in the target video to obtain the corresponding second video frame.

[0111] After determining the second video frame corresponding to the first video frame, the media gateway sends the second video frame to the media server. In the first embodiment of the present application, with respect to step 203, the media server may control the screen display content of the second terminal by sending the second video frame to the second terminal. Accordingly, after receiving the second video frame, the second terminal may display the received second video frame on its screen.

[0112] In addition, in an embodiment of the present application, during the data sharing process initiated by the first terminal, the first user can perform collaborative operations on the screen of the first terminal. For example, when the first terminal is a computer, the first user can move the mouse during the data sharing process or make real-time annotations in the meeting document.

[0113] The collaborative information generated by the collaborative operation triggered by the first user on the screen of the first terminal is used to indicate the location and / or operation content of the operation triggered by the first user on the screen of the first terminal (such as the mouse position on the screen, and / or the location and content of the instant annotation). It is usually drawn by the operating system on the upper layer of the screen and does not belong to the image displayed on the screen. Therefore, the target video obtained by screen recording by the first terminal does not include the collaborative information.

[0114] In the embodiment of the present application, the instant annotation can be a vector graphic or dot matrix graphic formed by the first user using a mouse, stylus or input text on the first terminal, which is overlaid on the background screen as an upper layer. For example, the instant annotation triggered by the first user on the screen of the first terminal can be an annotation that highlights part of the content on the screen. As shown in Figure 7, the screen display content of the first terminal, the first user can draw a rectangular box on the text displayed on the screen of the first terminal to highlight the text. The rectangular box is the instant annotation triggered by the first user.

[0115] Based on this, in the embodiment of the present application, the first terminal can also send collaboration information to the media server. After receiving the collaboration information, the media server synchronizes the collaboration information of the first terminal to other terminals connected to the conference. The following describes several ways in which the media server synchronizes the collaboration information of the first terminal to the second terminal.

[0116] Synchronization mode 1: The media server sends the collaboration information of the first terminal to the second terminal, and the second terminal superimposes the collaboration information and the second video frame for display.

[0117] In synchronization mode 1, after receiving the collaboration information sent by the first terminal, the media server sends the collaboration information to the second terminal. Optionally, the media server may send the second video frame and the collaboration information to the second terminal simultaneously, or the media server may send the second video frame and the collaboration information to the second terminal separately, which is not limited in this embodiment of the present application.

[0118] After receiving the second video frame and the cooperation information, the second terminal superimposes the cooperation information and the second video frame for display.

[0119] Optionally, the collaboration information includes mouse information and / or annotation information, wherein the mouse information includes position information of the mouse on the screen of the first terminal, and the annotation information includes annotation content and / or annotation position information.

[0120] If the collaboration information received by the second terminal includes mouse information and annotation information, the second terminal may treat the second video frame, the mouse information, and the annotation information as a layer respectively, and render the three layers together for superimposed display.

[0121] Synchronization mode 2: The media server sends the cooperation information of the first terminal to the media gateway, and the media gateway synthesizes the cooperation information into the second video frame.

[0122] In synchronization mode 2, after receiving the collaboration information sent by the first terminal, the media server sends the collaboration information to the media gateway; optionally, the media server can send the target video and collaboration information to the media gateway at the same time, or the media server can send the target video and collaboration information to the media gateway separately, which is not limited in the embodiment of the present application.

[0123] After determining the first video frame from the target video, the media gateway translates the text content in the first video frame to obtain a reference video frame, wherein the reference video frame includes text content in the second language; the media gateway synthesizes the collaboration information with the reference video frame to obtain a second video frame.

[0124] Optionally, the collaboration information includes mouse information and / or annotation information, wherein the mouse information includes position information of the mouse on the screen of the first terminal, and the annotation information includes annotation content and / or annotation position information.

[0125] If the collaboration information includes mouse information and annotation information, the media gateway may treat the reference video frame, the mouse information, and the annotation information as a layer respectively, and synthesize the three layers into a second video frame.

[0126] In synchronization mode 2, the media gateway sends the synthesized second video frame to the media server, and the media server sends the second video frame to the second terminal. The second terminal can directly display the received second video frame on the screen.

[0127] It should be noted that the media server in the embodiment of the present application can use any of the above methods to synchronize the collaboration information of the first terminal to the second terminal. For some special terminals, such as when the second terminal is a terminal that does not have the ability to overlay collaboration information (the second terminal does not have the ability to overlay the second video frame with the collaboration information), the media server uses the above synchronization method 2 to synchronize the collaboration information of the first terminal to the second terminal.

[0128] The following describes the data transmission process of the first embodiment of the present application under the data transmission system architecture shown in FIG6 in conjunction with FIG8 . In the process shown in FIG8 , the media server uses the above-mentioned synchronization method 1 to synchronize the collaboration information of the first terminal to the second terminal.

[0129] Step 801: The first terminal performs screen recording to obtain a target video.

[0130] The target video obtained by screen recording by the first terminal may include multiple video frames.

[0131] Step 802a: The first terminal sends the target video to the media server.

[0132] Step 802b: The first terminal sends collaboration information to the media server.

[0133] The collaboration information is information representing the location and / or content of the operation triggered by the first user on the screen of the first terminal, generated based on the collaborative operation triggered by the first user on the screen of the first terminal.

[0134] It should be noted that step 802a and step 802b can be performed in one step, or step 802a and step 802b can be performed in two steps, and the order of performing step 802a and step 802b is not limited.

[0135] Step 803: The media server sends the target video to the media gateway.

[0136] Step 804: The media gateway determines a first video frame from the target video.

[0137] The first video frame is a video frame in which display content of the target video is changed compared to the previous frame.

[0138] The manner in which the media gateway translates the text content in the first video frame may refer to the various first video frame determination manners described above.

[0139] Step 805: When the second terminal chooses to use the second language, the media gateway translates the text content in the first video frame to obtain a second video frame corresponding to the first video frame.

[0140] The manner in which the media gateway translates the text content in the first video frame can be referred to the above introduction.

[0141] Step 806: The media gateway sends the second video frame to the media server.

[0142] Step 807a: The media server sends the second video frame to the second terminal.

[0143] Step 807b: The media server sends the cooperation information of the first terminal to the second terminal.

[0144] It should be noted that step 807a and step 807b can be performed in one step, or step 807a and step 807b can be performed in two steps, and the order of performing step 807a and step 807b is not limited.

[0145] Step 808: The second terminal displays the second video frame and the collaboration information of the first terminal in an overlaid manner on the screen.

[0146] 9 , the data transmission process of the first embodiment of the present application under the data transmission system architecture shown in FIG6 will be introduced. In the process shown in FIG9 , the media server uses the above-mentioned synchronization method 2 to synchronize the collaboration information of the first terminal to the second terminal.

[0147] Step 901: The first terminal performs screen recording to obtain a target video.

[0148] The target video obtained by screen recording by the first terminal may include multiple video frames.

[0149] Step 902a: The first terminal sends the target video to the media server.

[0150] Step 902b: The first terminal sends collaboration information to the media server.

[0151] The collaboration information is information representing the location and / or content of the operation triggered by the first user on the screen of the first terminal, generated based on the collaborative operation triggered by the first user on the screen of the first terminal.

[0152] It should be noted that step 902a and step 902b can be performed in one step, or step 902a and step 902b can be performed in two steps, and the order in which step 902a and step 902b are performed is not limited.

[0153] Step 903a: The media server sends the target video to the media gateway.

[0154] Step 903b: The media server sends the cooperation information of the first terminal to the media gateway.

[0155] It should be noted that step 903a and step 903b can be performed in one step, or step 903a and step 903b can be performed in two steps, and the order of performing step 903a and step 903b is not limited.

[0156] Step 904: The media gateway determines a first video frame from the target video.

[0157] The first video frame is a video frame in which display content of the target video is changed compared to the previous frame.

[0158] The manner in which the media gateway translates the text content in the first video frame may refer to the various first video frame determination manners described above.

[0159] Step 905: When the second terminal chooses to use the second language, the media gateway translates the text content in the first video frame to obtain a reference video frame.

[0160] The reference video frame includes text content in the second language.

[0161] Step 906: The media gateway synthesizes the collaboration information and the reference video frame to obtain a second video frame.

[0162] Step 907: The media gateway sends the second video frame to the media server.

[0163] Step 908: The media server sends the second video frame to the second terminal.

[0164] Step 909: The second terminal displays the second video frame on the screen.

[0165] It should be noted that when the media service device is executed by a single device, such as a media server, the functions performed by the media server and the media gateway described above can both be performed by the media server. The functions performed by the media server can be found in the functions performed by the media server and the media gateway described above, and will not be repeated here.

[0166] The following describes the data transmission process of the first embodiment of the present application in conjunction with a specific application scenario. In this application scenario, terminal 1, terminal 2, terminal 3, and terminal 4 access the same conference through a media server, and the conference supports multiple languages. Among them, terminal 1, terminal 2, and terminal 3 are terminals with the ability to overlay collaborative information, and terminal 4 is a terminal without the ability to overlay collaborative information. The data transmission flow chart shown in Figure 10 may include the following steps:

[0167] Step 1001: Terminal 1, Terminal 2, Terminal 3, and Terminal 4 send a conference access request to a media server, wherein the conference access request carries the language type selected by each terminal.

[0168] Exemplarily, the language selected for use by terminal 1 and terminal 2 is Chinese, and the language selected for use by terminal 3 and terminal 4 is English.

[0169] Step 1002a: Terminal 1 initiates a data sharing instruction and sends a target video to the media server.

[0170] The conference file used in the conference is displayed on the screen of terminal 1, and terminal 1 can record the screen to obtain the target video.

[0171] Step 1002b: Terminal 1 sends collaboration information to the media server.

[0172] The user can operate the conference file displayed on the screen of terminal 1, such as moving the mouse or making real-time annotations in the conference file. In order to ensure data synchronization between terminals, terminal 1 needs to send the collaboration information generated by the user operation to the media server.

[0173] It should be noted that step 1002a and step 1002b can be executed in one step, or step 1002a and step 1002b can be executed in two steps, and the order of executing step 1002a and step 1002b is not limited.

[0174] Step 1003a: The media server sends the target video to terminal 2.

[0175] Step 1003b: The media server sends the collaboration information of terminal 1 to terminal 2.

[0176] It should be noted that step 1003a and step 1003b can be executed in one step, or step 1003a and step 1003b can be executed in two steps, and the order of executing step 1003a and step 1003b is not limited.

[0177] Step 1004: Terminal 2 overlays and displays each video frame in the target video and the collaboration information of Terminal 1 on the screen.

[0178] Step 1005a: The media server sends the target video to the media gateway.

[0179] Step 1005b: The media server sends the collaboration information of terminal 1 to the media gateway.

[0180] It should be noted that step 1005a and step 1005b can be executed in one step, or step 1005a and step 1005b can be executed in two steps, and the order of executing step 1005a and step 1005b is not limited.

[0181] Step 1006: The media gateway determines a first video frame from the target video.

[0182] The first video frame is a video frame in which display content of the target video is changed compared to the previous frame.

[0183] The manner in which the media gateway translates the text content in the first video frame may refer to the various first video frame determination manners described above.

[0184] Step 1007: The media gateway translates the Chinese text in the first video frame into English text to obtain a reference video frame.

[0185] Step 1008: The media gateway uses the reference video frame as a second video frame corresponding to the first video frame, and sends the second video frame to the media server.

[0186] Step 1009a: The media server sends the second video frame to terminal 3.

[0187] Step 1009b: The media server sends the collaboration information of terminal 1 to terminal 3.

[0188] Step 1010: Terminal 3 displays the second video frame and the collaboration information of Terminal 1 superimposed on the screen.

[0189] Step 1011: The media gateway synthesizes the reference video frame and the collaboration information of terminal 1 to obtain a second video frame.

[0190] Step 1012: The media gateway sends the second video frame to the media server.

[0191] It should be noted that the content of the second video frame in step 1012 is different from that of the second video frame in step 1008. The second video frame in step 1012 is a video frame without synthesized collaboration information, and the second video frame in step 1008 is a video frame after synthesizing collaboration information.

[0192] Step 1013: The media server sends the second video frame to terminal 4.

[0193] Step 1014: Terminal 4 displays the second video frame on the screen.

[0194] Assume that the screen display of Terminal 1 is as shown in Figure 7, which includes the text "Happy Children's Day." The user adds a "rectangular box" annotation to the screen display of Terminal 1. The media server synchronizes the screen display of Terminal 1 with Terminals 2, 3, and 4. If Terminal 2 selects Chinese as its language, the screen display of Terminal 2 will be the same as that of Terminal 1. If Terminals 3 and 4 select English as their languages, the media server will translate the Chinese text "Happy Children's Day" included in the screen display of Terminal 1 into English via the media gateway. The screen displays of Terminals 3 and 4 may then appear as shown in Figure 11.

[0195] Implementation method 2:

[0196] In the second embodiment, the first media data displayed on the screen of the first terminal is the text data of a first online document, and the second media data is the text data of a second online document. The text data of the first online document and the text data of the second online document are in different languages ​​but represent the same meaning. For example, if the text data of the first online document is in Chinese, the text data in the second online document is a translation of the Chinese text data in the first online document.

[0197] Among them, online documents may include but are not limited to: Word documents, Excel documents, PDF documents, and PPT documents.

[0198] In the second embodiment, the functions of the above-mentioned media service device can be executed by a media server. The following description takes the execution by the media server as an example.

[0199] Optionally, the second embodiment of the present application may be applicable to a data transmission system architecture as shown in FIG12 , wherein the data transmission system includes a media server 121 , an online document service device 122 , a first terminal 123 and a second terminal 124 .

[0200] The online document service device 122 can store multiple online documents in different languages. The text data in each online document in different languages ​​represents the same meaning. It can be understood that the multiple online documents in different languages ​​are obtained by translating the text data of a single online document. For example, the multiple online documents in different languages ​​stored in the online document service device 122 include Chinese, English, Japanese, Russian, and the like.

[0201] The online document service device 122 generates corresponding online document address information for each language type of online document. The online document service device 122 sends the address information of each language type of online document to the media server 121. During the conference, the media server 121 can synchronize the screen display content between the first terminal and the second terminal through the online document.

[0202] The following describes in detail the process of the media server synchronizing screen display content between the first terminal and the second terminal.

[0203] The first terminal sends a conference access request to the media server, which carries the language type selected by the first terminal. The media server sends address information of a first online document corresponding to the language type selected by the first terminal to the first terminal according to the language type selected by the first terminal.

[0204] After the first terminal receives the address information of the first online document, in response to the online document opening operation triggered by the first user, the first terminal displays the first online document on a screen.

[0205] Accordingly, the second terminal sends a conference access request to the media server, which carries the language type selected by the second terminal. The media server can then determine that the first terminal and the second terminal are accessing the same conference. Optionally, after determining that the second terminal is accessing the conference, the media server can determine a second online document corresponding to the first online document based on the first online document selected by the first terminal, where the text data of the first online document is in the first language, the text data of the second online document is in the second language, and the text data of the first and second online documents represent the same meaning.

[0206] In one possible implementation, after determining that the second terminal has joined the conference, the media server sends the address information of the second online document to the second terminal based on the second language selected by the second terminal. Alternatively, after determining that the first terminal has initiated data sharing, the media server sends the address information of the second online document to the second terminal.

[0207] During implementation, the first terminal may send a data sharing instruction to the media server. Upon receiving the data sharing instruction from the first terminal, the media server may send a notification message to the second terminal, instructing the second terminal to display the second online document. During implementation, the second terminal may display a prompt message on its screen, prompting the second user to open the second online document. In response to the second user triggering the online document opening operation, the second terminal displays the second online document on its screen.

[0208] To ensure data synchronization between the first terminal and the second terminal, an optional implementation is that the media server may generate second media data corresponding to the first media data based on text data of the first online document currently displayed on the screen of the first terminal.

[0209] During implementation, the media server may determine the text data in the second online document based on the address information and document location information of the second online document, where the document location information is determined based on the text data of the first online document currently displayed on the screen of the first terminal. For example, if the text data of the first online document currently displayed on the screen of the first terminal is the content of paragraphs 2-3 of page 1, the media server may determine the document location information based on the text data of the first online document currently displayed on the screen of the first terminal, and based on the document location information, determine from the second online document the content corresponding to paragraphs 2-3 of page 1 of the first online document as the second media data.

[0210] The first terminal may send first display position information to the media server. The first display position information indicates the page number and / or paragraph of the first online document in which the text displayed on the screen of the first terminal resides. The media server may determine document location information based on the first display position information. The media server then sends the document location information to the second terminal. The document location information is used to determine the displayed text in the second online document. The displayed text is the text displayed on the screen of the second terminal.

[0211] Exemplarily, the first display position information includes the page number and paragraph number of the text currently displayed on the first terminal screen in the first online document; for example, the first display position information includes paragraphs 5-10 on page 1.

[0212] Optionally, the document location information sent by the media server to the second terminal may be first display location information of the first online document; alternatively, the document location information sent by the media server to the second terminal may be second display location information of the second online document, where the second display location information is determined based on the first display location information of the first online document. These are described below.

[0213] 1. The document location information is the first display location information of the first online document.

[0214] Because the text content of the first online document and the text content of the second online document represent the same meaning, during the conference, the first terminal and the second terminal need to synchronously display the text with the same meaning. However, the position of the text content with the same meaning in the first online document and the second online document may be the same or different.

[0215] When the positions of text contents with the same meaning in the first online document and the second online document are the same, after the media server sends the first display position information to the second terminal, the second terminal displays the text contents corresponding to the first display position information in the second online document on the screen.

[0216] Exemplarily, the media server sends the first display position information including paragraphs 5-10 of page 1 to the second terminal, and the second terminal displays the text content of paragraphs 5-10 of page 1 of the second online document on the screen.

[0217] When text content with the same meaning is located in different positions in the first online document and the second online document, after the media server sends the first display position information to the second terminal, the second terminal converts the first display position information into second display position information, and displays the text content corresponding to the second display position information in the second online document on the screen.

[0218] Optionally, the second terminal may pre-store a mapping relationship between the locations of text content with the same meaning in the first online document and the locations in the second online document. For example, if paragraph 5 on page 1 of the first online document corresponds to paragraph 1 on page 2 of the second online document, it can be understood that the text content in paragraph 5 on page 1 of the first online document represents the same meaning as the text content in paragraph 1 on page 2 of the second online document.

[0219] 2. The document location information is the second display location information of the second online document.

[0220] Because the text content of the first online document and the text content of the second online document represent the same meaning, during the conference, the first terminal and the second terminal need to synchronously display the text with the same meaning. However, the position of the text content with the same meaning in the first online document and the second online document may be the same or different.

[0221] When text content with the same meaning is located at the same position in the first online document and in the second online document, after the media server receives the first display position information sent by the first terminal, the media server may use the first display position information as the second display position information and send the second display position information to the second terminal; and the second terminal displays the text content corresponding to the second display position information in the second online document on the screen.

[0222] When text content with the same meaning is located in different positions in the first online document and the second online document, after receiving the first display position information sent by the first terminal, the media server converts the first display position information into second display position information and sends the second display position information to the second terminal; the second terminal displays the text content corresponding to the second display position information in the second online document on the screen.

[0223] Optionally, the media server may pre-store a mapping relationship between the position of text content with the same meaning in the first online document and the position in the second online document; the media server converts the first display position information into the second display position information based on the mapping relationship.

[0224] Based on the above manner, text contents representing the same meaning are displayed on the screens of the first terminal and the second terminal through the media server, so that the first terminal and the second terminal maintain data synchronization during the conference.

[0225] In addition, in an embodiment of the present application, during the data sharing process initiated by the first terminal, the first user can perform collaborative operations on the screen of the first terminal. For example, when the first terminal is a computer, the first user can move the mouse during the data sharing process or make real-time annotations in the first online document.

[0226] The collaborative information generated by the collaborative operation triggered by the first user on the screen of the first terminal is used to represent the location and / or operation content of the operation triggered by the first user on the screen of the first terminal (such as the mouse position on the screen, and / or the location and content of the instant annotation).

[0227] The instant annotation in the embodiment of the present application may be a vector graphic or a dot matrix graphic formed by the first user using a mouse, a stylus or inputting text on the first terminal, which is overlaid on the background screen as an upper layer.

[0228] In order to ensure data synchronization between the first terminal and the second terminal, the first terminal in the embodiment of the present application can also send collaboration information to the media service device. After receiving the collaboration information, the media server synchronizes the collaboration information of the first terminal to the second terminal accessing the conference.

[0229] During implementation, the first terminal may send the first display position information and the collaboration information to the media server at the same time, or the first terminal may send the first display position information and the collaboration information to the media server separately, which is not limited in this embodiment of the present application.

[0230] After receiving the collaboration information sent by the first terminal, the media server sends the collaboration information to the second terminal. Optionally, the media server can send the document location information and the collaboration information to the second terminal at the same time, or the media server can send the document location information and the collaboration information to the second terminal separately, which is not limited in this embodiment of the application.

[0231] After receiving the document location information and the collaboration information, the second terminal overlays and displays the text content of the second online document determined according to the document location information and the collaboration information.

[0232] Optionally, the collaboration information includes mouse information and / or annotation information, wherein the mouse information includes position information of the mouse on the screen of the first terminal, and the annotation information includes annotation content and / or annotation position information.

[0233] If the collaboration information received by the second terminal includes mouse information and annotation information, the second terminal may render the text content of the second online document, mouse information, and annotation information determined according to the document location information as a layer, and render the three layers together for superimposed display.

[0234] It should be noted that when the second terminal is a terminal that does not have the ability to overlay collaborative information, the media server can send the document location information and collaborative information to the media gateway. The media gateway determines the text content of the second online document based on the document location information, and synthesizes the text content of the second online document with the text content of the second online document to obtain the target video frame. The media gateway sends the target video frame to the second terminal through the media server, and the second terminal directly displays the target video frame, thereby ensuring that the display content of the second terminal is synchronized with the display content of the first terminal.

[0235] The following describes the data transmission process of the second embodiment of the present application under the data transmission system architecture shown in FIG12 , in conjunction with FIG13 .

[0236] Step 1301: In response to a first user's operation of opening an online document, a first online document is displayed on a screen of a first terminal.

[0237] Step 1302: The first terminal sends a data sharing instruction to the media server.

[0238] Exemplarily, the first terminal can access the conference by running a conference APP. After the first terminal accesses the conference, the first user can initiate data sharing by clicking the "Share" control in the conference APP display page; the first terminal sends a data sharing instruction to the media server.

[0239] Step 1303: If the second terminal chooses to use the second language, the media server determines a second online document corresponding to the first online document.

[0240] The text contents in the first online document and the second online document are in different languages, but the meanings represented by the text contents are the same.

[0241] Step 1304: The media server sends the address information of the second online document to the second terminal.

[0242] In practice, for a conference accessed by a first terminal and a second terminal, if the media server stores the address of an online document in different languages ​​corresponding to a conference file, the media server can also send the address of the second online document to the second terminal after the second terminal accesses the conference. In this way, after receiving the data sharing instruction from the first terminal, the media server sends a notification message to the second terminal. After the second terminal receives the notification message, a prompt message can be displayed on the display interface of the conference app, prompting the second user to open the second online document.

[0243] Step 1305: In response to the second user's operation of opening the online document, the second online document is displayed on the screen of the second terminal.

[0244] Step 1306a: The first terminal sends first display position information to the media server, where the first display position information represents the page number and / or paragraph of the text displayed on the screen of the first terminal in the first online document.

[0245] Step 1306b: The first terminal sends collaboration information to the media server, where the collaboration information is used to indicate the location and / or content of the operation triggered by the first user on the screen of the first terminal.

[0246] It should be noted that step 1306a and step 1306b can be executed in one step, or step 1306a and step 1306b can be executed in two steps, and the order of executing step 1306a and step 1306b is not limited.

[0247] Step 1307: The media server determines document location information according to the first display position information.

[0248] During implementation, the media server can directly use the first display position information as the document location information, or the media server can determine the second display position information of the second online document corresponding to the first display position information based on the mapping relationship between the position of text content with the same meaning in the first online document and the position in the second online document, and use the second display position information as the document location information.

[0249] Step 1308a: The media server sends document location information to the second terminal.

[0250] Step 1308b: The media server sends the cooperation information of the first terminal to the second terminal.

[0251] It should be noted that step 1308a and step 1308b can be executed in one step, or step 1308a and step 1308b can be executed in two steps, and the order of executing step 1308a and step 1308b is not limited.

[0252] Step 1309: The second terminal displays the text content of the second online document determined according to the document location information and the collaboration information of the first terminal in an overlapping manner on the screen.

[0253] The following describes the data transmission process of the first embodiment of the present application in conjunction with a specific application scenario. In this application scenario, terminal 1, terminal 2, and terminal 3 access the same conference through a media server, and the conference supports multiple languages. Among them, terminal 1, terminal 2, and terminal 3 are terminals with the ability to overlay collaborative information. The operator of terminal 1 is user 1, the operator of terminal 2 is user 2, and the operator of terminal 3 is user 3. The data transmission flow chart shown in Figure 14 may include the following steps:

[0254] Step 1401: Terminal 1, Terminal 2, and Terminal 3 send a conference access request to a media server, where the conference access request carries the language type selected by each terminal.

[0255] Exemplarily, the language selected for use by terminal 1 and terminal 2 is Chinese, and the language selected for use by terminal 3 is English.

[0256] Step 1402 : The media server sends the address information of the first online document to Terminal 1 and Terminal 2 , and sends the address information of the second online document to Terminal 3 .

[0257] The text content of the first online document is in Chinese, and the second online document is an English version of the document with the same meaning as the text content of the first online document.

[0258] Step 1403: In response to the online document opening operation by user 1, the first online document is displayed on the screen of terminal 1.

[0259] Step 1404: Terminal 1 sends a data sharing instruction to the media server.

[0260] Step 1405: The media server sends a notification message to Terminal 2 and Terminal 3.

[0261] The notification message is used to prompt the user to open the online document.

[0262] Step 1406 : In response to the online document opening operation by user 2 , the first online document is displayed on the screen of terminal 2 .

[0263] Step 1407 : In response to the online document opening operation by user 3 , the second online document is displayed on the screen of terminal 3 .

[0264] Step 1408a: Terminal 1 sends first display position information to the media server, where the first display position information represents the page number and / or paragraph of the text displayed on the screen of Terminal 1 in the first online document.

[0265] Step 1408b: Terminal 1 sends collaboration information to the media server. The collaboration information is used to indicate the location and / or content of the operation triggered by user 1 on the screen of terminal 1.

[0266] It should be noted that step 1408a and step 1408b can be performed in one step, or step 1408a and step 1408b can be performed in two steps, and the order in which step 1408a and step 1408b are performed is not limited.

[0267] Step 1409a: The media server sends first display position information to terminal 2.

[0268] Step 1409b: The media server sends the collaboration information of terminal 1 to terminal 2.

[0269] It should be noted that step 1409a and step 1409b can be executed in one step, or step 1409a and step 1409b can be executed in two steps, and the order of executing step 1409a and step 1409b is not limited.

[0270] Step 1410a: The media server sends document location information to terminal 3.

[0271] The document location information is determined by the media server according to the first display location information sent by the terminal 1 . The specific determination method can be found in the above description.

[0272] Step 1410b: The media server sends the collaboration information of terminal 1 to terminal 3.

[0273] It should be noted that step 1410a and step 1410b can be performed in one step, or step 1410a and step 1410b can be performed in two steps, and the order in which step 1410a and step 1410b are performed is not limited.

[0274] Step 1411: Terminal 2 displays the text content of the first online document determined according to the first display position information and the collaboration information of Terminal 1 in an overlapping manner on the screen.

[0275] Step 1412: Terminal 3 displays the text content of the second online document determined according to the document location information and the collaboration information of Terminal 1 in a superimposed manner on the screen.

[0276] In conjunction with the above method embodiments, embodiments of the present application also provide a data transmission device. In some embodiments, as shown in FIG15 , the data transmission device 1500 may include a determination unit 1501, a processing unit 1502, and a display control unit 1503. The data transmission device 1500 can be used to implement the functions of the above method embodiments, thereby achieving the beneficial effects of the above method embodiments.

[0277] The determination unit 1501 is used to determine the first media data displayed on the screen of the first terminal, where the first media data includes text content in a first language; the processing unit 1502 is used to generate second media data corresponding to the first media data if the second terminal chooses to use a second language, where the second media data includes text content in a second language; the first terminal and the second terminal are connected to the same conference; the display control unit 1503 is used to control the screen display content of the second terminal according to the second media data.

[0278] In a possible implementation, the first media data is a first video frame in a target video obtained by recording the screen of the first terminal, and the second media data is a second video frame.

[0279] In a possible implementation, the determining unit 1501 is configured to:

[0280] receiving a target video sent by the first terminal, where the target video includes multiple video frames;

[0281] The first video frame is determined from the target video, where the first video frame is a video frame in the target video whose display content is changed compared to a previous frame.

[0282] In a possible implementation, the processing unit 1502 is configured to:

[0283] The text content in the first video frame is translated to obtain a second video frame corresponding to the first video frame.

[0284] In a possible implementation, the first media data is text data in a first online document, and the second media data is text data in a second online document; the text data in the first online document and the text data in the second online document represent the same meaning.

[0285] In a possible implementation, the processing unit 1502 is configured to:

[0286] The text data in the second online document is determined according to the address information and document location information of the second online document; wherein the document location information is determined according to the text data of the first online document currently displayed on the screen of the first terminal.

[0287] In a possible implementation, the display control unit 1503 is further configured to:

[0288] Document location information is sent to the second terminal, where the document location information is used to determine text data in the second online document, and the displayed text is text displayed on the screen of the second terminal.

[0289] In a possible implementation, the document location information is first display location information of the first online document, where the first display location information represents the page number and / or paragraph of the text displayed on the screen of the first terminal in the first online document.

[0290] In a possible implementation, the document location information is second display location information of the second online document, where the second display location information indicates the page number and / or paragraph of the text displayed on the screen of the second terminal in the second online document; the second display location information is determined based on the first display location information of the first online document, where the first display location information indicates the page number and / or paragraph of the text displayed on the screen of the first terminal in the first online document.

[0291] In a possible implementation, the display control unit 1503 is configured to:

[0292] Control the screen display content of the second terminal according to the second media data and the collaboration information sent by the first terminal; wherein the collaboration information is used to indicate the location and / or operation content of the operation triggered by the user on the screen of the first terminal.

[0293] The determining unit 1501, the processing unit 1502, and the display control unit 1503 can all be implemented via software or hardware. For example, the implementation of the processing unit 1502 will be described below using the processing unit 1502 as an example. Similarly, the implementation of the determining unit 1501 and the display control unit 1503 can refer to the implementation of the processing unit 1502.

[0294] As an example of a software functional unit, the processing unit 1502 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the processing unit 1502 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same AZ or in different AZs, and each AZ includes one data center or multiple geographically close data centers. Generally, a region may include multiple AZs.

[0295] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same VPC or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0296] As an example of a hardware functional unit, a module may include at least one computing device, such as a server. Alternatively, processing unit 1502 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0297] The multiple computing devices included in processing unit 1502 can be distributed in the same region or in different regions. The multiple computing devices included in module A can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in module A can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0298] It should be noted that, in other embodiments, the processing unit 1502 can be used to execute any step in the data transmission method, the determining unit 1501 can be used to execute any step in the data transmission method, and the display control unit 1503 can be used to execute any step in the data transmission method. The steps that the determining unit 1501, the processing unit 1502, and the display control unit 1503 are responsible for implementing can be specified as needed. The full functionality of the data transmission device is achieved by having the determining unit 1501, the processing unit 1502, and the display control unit 1503 respectively implement different steps in the data transmission method.

[0299] This application also provides a computing device 1600. As shown in Figure 16, computing device 1600 can be used to implement the functions of the media service device in the above-mentioned embodiment, and includes: a bus 1601, a processor 1602, a memory 1603, and a communication interface 1604. Processor 1602, memory 1603, and communication interface 1604 communicate with each other via bus 1601. Computing device 1600 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 1600.

[0300] Bus 1601 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG16 shows only one line, but this does not imply that there is only one bus or only one type of bus. Bus 1601 may include a path for transmitting information between various components of computing device 1600 (e.g., memory 1603, processor 1602, and communication interface 1604).

[0301] The processor 1602 may include any one or more processors such as a CPU, a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0302] The memory 1603 may include a volatile memory, such as a random access memory (RAM). The processor 1602 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0303] The memory 1603 stores executable program code, and the processor 1602 executes the executable program code to respectively implement the functions of the aforementioned determination unit 1501, processing unit 1502, and display control unit 1503, thereby implementing the data transmission method. In other words, the memory 1603 stores instructions for executing the data transmission method.

[0304] The communication interface 1604 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1600 and other devices or a communication network.

[0305] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. At least one computing device in the computing device cluster cooperates to implement the functions of the computing device in the above embodiments. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0306] As shown in Figure 17, the computing device cluster includes at least one computing device 1600. The memory 1603 in one or more computing devices 1600 in the computing device cluster may store the same instructions for executing the data transmission method.

[0307] In some possible implementations, the memory 1603 of one or more computing devices 1600 in the computing device cluster may also store partial instructions for executing the data transmission method. In other words, the combination of one or more computing devices 1600 can jointly execute the instructions for executing the data transmission method.

[0308] It should be noted that the memory 1603 in different computing devices 1600 in the computing device cluster can store different instructions, each for performing a portion of the functions of the computing device. In other words, the instructions stored in the memory 1603 in different computing devices 1600 can implement the functions of one or more of the determining unit 1501, the processing unit 1502, and the display control unit 1503.

[0309] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network, which may be a wide area network or a local area network.

[0310] Embodiments of the present application also provide a computer program product comprising instructions. The computer program product may be software or a program product comprising instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute an application deployment method, or an application deployment method.

[0311] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct a computing device to execute an application deployment method, or instructs a computing device to execute an application deployment method.

[0312] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0313] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or box in the flow chart and / or block diagram, as well as the combination of the flow chart and / or box in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more flow charts and / or one or more boxes in the block diagram.

[0314] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0315] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0316] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A data transmission method, characterized in that: The method is applied to a media service device, and the method includes: Determining first media data displayed on a screen of a first terminal, wherein the first media data includes text content in a first language; If the second terminal chooses to use the second language, second media data corresponding to the first media data is generated, and the second media data includes text content in the second language; the first terminal and the second terminal are connected to the same conference; Controlling screen display content of the second terminal according to the second media data.

2. The method according to claim 1, characterized in that The first media data is a first video frame in a target video obtained by recording a screen of the first terminal, and the second media data is a second video frame.

3. The method according to claim 2, characterized in that The determining the first media data displayed on the screen of the first terminal includes: Receiving a target video sent by the first terminal, where the target video includes multiple video frames; The first video frame is determined from the target video, where the first video frame is a video frame in the target video whose display content is changed compared to a previous frame.

4. The method according to claim 2 or 3, characterized in that The generating second media data corresponding to the first media data includes: The text content in the first video frame is translated to generate a second video frame corresponding to the first video frame.

5. The method according to claim 1, characterized in that The first media data is text data in a first online document, and the second media data is text data in a second online document; the text data in the first online document and the text data in the second online document represent the same meaning.

6. The method according to claim 5, characterized in that The generating second media data corresponding to the first media data includes: The text data in the second online document is determined according to the address information and document location information of the second online document; wherein the document location information is determined according to the text data of the first online document currently displayed on the screen of the first terminal.

7. The method according to any one of claims 2 to 6, characterized in that: The controlling the screen display content of the second terminal according to the second media data includes: Control the screen display content of the second terminal according to the second media data and the collaboration information sent by the first terminal; wherein the collaboration information is used to indicate the position and / or operation content of the operation triggered by the user on the screen of the first terminal.

8. A data transmission device, characterized in that: The device comprises: a determining unit, configured to determine first media data displayed on a screen of a first terminal, wherein the first media data includes text content in a first language; a processing unit, configured to generate second media data corresponding to the first media data if the second terminal selects to use a second language, wherein the second media data includes text content in the second language; the first terminal and the second terminal are connected to the same conference; A display control unit is used to control the screen display content of the second terminal according to the second media data.

9. The device according to claim 8, characterized in that The first media data is a first video frame in a target video obtained by recording a screen of the first terminal, and the second media data is a second video frame.

10. The device according to claim 9, characterized in that The determining unit is used for: Receiving a target video sent by the first terminal, where the target video includes multiple video frames; The first video frame is determined from the target video, where the first video frame is a video frame in the target video whose display content is changed compared to a previous frame.

11. The device according to claim 9 or 10, characterized in that The processing unit is used for: The text content in the first video frame is translated to generate a second video frame corresponding to the first video frame.

12. The device according to claim 8, characterized in that The first media data is text data in a first online document, and the second media data is text data in a second online document; the text data in the first online document and the text data in the second online document represent the same meaning.

13. The device according to claim 12, characterized in that The processing unit is used for: The text data in the second online document is determined according to the address information and document location information of the second online document; wherein the document location information is determined according to the text data of the first online document currently displayed on the screen of the first terminal.

14. The device according to any one of claims 8 to 13, characterized in that: The display control unit is used for: Control the screen display content of the second terminal according to the second media data and the collaboration information sent by the first terminal; wherein the collaboration information is used to indicate the position and / or operation content of the operation triggered by the user on the screen of the first terminal.

15. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: The storage medium stores a computer program or an instruction. When the computer program or the instruction is executed by the communication device, the method according to any one of claims 1 to 7 is implemented.

17. A computer program product, characterized in that When the computer program product is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7.

Images