Multimedia processing method and multimedia interactive system

Abstract

The present application discloses a multimedia processing method and a multimedia interactive system. The multimedia processing method includes: a terminal device and a wireless broadcasting device establish a connection, and transmit a transcription instruction to the wireless broadcasting device; during the process of the terminal device playing a video, the wireless broadcasting device connected to the terminal device continuously receives and compresses the voice information of the video to continuously transmit the compressed information back to the terminal device; the terminal device decompresses the compressed information received in real time to obtain information to be translated; the terminal device transmits the information to be translated to a cloud server for translation to obtain text information in the target language; and the terminal device displays the text information on the screen of the video it plays. Therefore, the problem that the video played by the terminal device has no subtitles or the subtitles are not in a language familiar to the user is solved, and the user's experience of watching the video is improved.

Description

Technical Field

[0001] The present application relates to the field of multimedia technology, and in particular to a multimedia processing method and a multimedia interactive system. Background Art

[0002] With the continuous development of computer technology and communication technology, more and more people use terminal devices to watch videos and play audio through wireless connection to wireless broadcasting devices to obtain clearer or high-quality sound. However, foreign videos (such as movies) watched by users may not have subtitles, or the subtitles are not in a language familiar to the user, which makes the user have a poor experience in watching foreign videos.

[0003] Therefore, how to provide a multimedia processing method and a multimedia interactive system to improve the viewing experience of users who use terminal equipment with wireless broadcasting devices to watch foreign videos is an urgent problem to be solved by those skilled in the art. Summary of the invention

[0004] The embodiments of the present application provide a multimedia processing method and a multimedia interaction system, which can solve the problem in the prior art that users who use terminal devices with wireless broadcasting devices to watch foreign videos have a poor viewing experience because the videos they watch have no subtitles or the subtitles are not in a familiar language.

[0005] In order to solve the above technical problems, this application is implemented as follows:

[0006] The present application provides a multimedia processing method, which includes: establishing a connection between a terminal device and a wireless broadcasting device, and transmitting a transcription instruction to the wireless broadcasting device; during the process of the terminal device playing a video, the wireless broadcasting device connected to the terminal device continuously receives and compresses the voice information of the video, so as to continuously return the compressed information to the terminal device; the terminal device decompresses the compressed information received in real time to obtain information to be translated; the terminal device transmits the information to be translated to a cloud server for translation, so as to obtain text information in a target language; and the terminal device displays the text information on the screen of the video it plays.

[0007] The present application provides a multimedia interactive system, which includes: a wireless broadcasting device, a terminal device and a cloud server. The terminal device is used to establish a connection with the wireless broadcasting device and transmit a transcription instruction to the wireless broadcasting device. The cloud server is connected to the terminal device via a network. In the process of the terminal device playing a video, the wireless broadcasting device connected to the terminal device and receiving the transcription instruction continuously receives and compresses the voice information of the video to continuously transmit the compressed information back to the terminal device; the terminal device decompresses the compressed information received in real time to obtain information to be translated; the terminal device transmits the information to be translated to the cloud server for translation to obtain text information in the target language; and the terminal device displays the text information on the screen of the video it plays.

[0008] In the embodiment of the present application, the wireless broadcasting device obtains the voice information of the video played in real time by the terminal device, and transmits back the compressed information of the corresponding voice information to the terminal device, so that the terminal device decompresses the compressed information and translates it through the cloud server, and obtains the text information in the target language corresponding to the played video (that is, the text information in the language familiar to the user), and then displays the text information on the screen of the played video to improve the user's experience of watching the video. In addition, since the voice information of the video played in real time by the terminal device is obtained through the wireless broadcasting device, the multimedia processing method and multimedia interaction system of the embodiment of the present application are suitable for application in the case where the terminal device playing the video cannot directly obtain the voice information of the video watched by the user (for example: the terminal device cannot directly obtain the voice information of the video watched by the user due to permission issues). BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0010] Figure 1 A block diagram of an embodiment of a multimedia interactive system according to the present application; and

[0011] Figure 2 FIG. 4 is a flow chart of an embodiment of a multimedia processing method according to the present application. DETAILED DESCRIPTION

[0012] The embodiments of the present invention will be described below with reference to the accompanying drawings. In these drawings, the same reference numerals represent the same or similar components or method flows.

[0013] It must be understood that the words "comprise", "include" and the like used in this specification are used to indicate the existence of specific technical features, values, method steps, job processing, components and / or assemblies, but do not exclude the addition of more technical features, values, method steps, job processing, components, assemblies, or any combination of the above.

[0014] It should be understood that when a component is described as being "connected" or "coupled" to another component, it may be directly connected or coupled to the other component, and intermediate components may be present. Conversely, when a component is described as being "directly connected" or "directly coupled" to another component, there are no intermediate components.

[0015] See also Figure 1 , which is a block diagram of an embodiment of a multimedia interactive system according to the present application. Figure 1 As shown, the multimedia interactive system 1 includes: a wireless broadcasting device 11, a terminal device 12 and a cloud server 13. Among them, the terminal device 12 can selectively connect to the wireless broadcasting device 11 wirelessly, and the cloud server 13 can connect to the wireless broadcasting device 11 through a network (such as Figure 1 The cloud-shaped diagram in the figure) connects the terminal device 12.

[0016] In actual implementation, the wireless broadcasting device 11 may be, but not limited to, a wireless headset or a wireless electronic device with a broadcasting function, such as smart glasses or wireless speakers that can broadcast; the terminal device 12 may be, but not limited to, a mobile terminal, a tablet computer, a desktop computer, a laptop / notebook computer or a smart TV; the wireless communication protocol between the wireless broadcasting device 11 and the terminal device 12 may include, but not limited to, WiFi, Bluetooth or ZigBee; the network through which the terminal device 12 connects to the cloud server 13 may include, but not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network or a wireless self-organizing network (Ad Hoc network).

[0017] In this embodiment, the terminal device 12 is used to establish a connection with the wireless broadcasting device 11 and transmit a transcription instruction to the wireless broadcasting device 11 .

[0018] In one embodiment, the terminal device 12 cannot directly transmit instructions to the wireless broadcasting device 11 wirelessly connected thereto, therefore, the terminal device 12 may download and install an application for controlling the wireless broadcasting device 11 to transmit the transcription instructions to the wireless broadcasting device 11 connected thereto through the application.

[0019] In this embodiment, during the video playback process of the terminal device 12, the wireless broadcasting device 11 connected to the terminal device 12 and receiving the transcription instruction continuously receives and compresses the voice information of the video to continuously transmit the compressed information back to the terminal device 12; the terminal device 12 decompresses the compressed information received in real time to obtain the information to be translated; the terminal device 12 transmits the information to be translated to the cloud server 13 for translation to obtain text information in the target language; the terminal device 12 displays the text information on the screen of the video it plays. Therefore, the multimedia interactive system 1 can use the wireless broadcasting device 11 as a medium for capturing and transmitting the voice information of the video and the cloud server 13 to perform the translation function, so that the terminal device 12 can provide the user with subtitles in a familiar language corresponding to the video he is watching, thereby improving the user's experience of watching the video.

[0020] In one embodiment, the wireless communication protocol between the wireless broadcasting device 11 and the terminal device 12 may be Bluetooth, and therefore, the terminal device 12 may establish a Bluetooth audio transmission model protocol (Advanced Audio Distribution Profile, A2DP) connection with the wireless broadcasting device 11, so that the terminal device 12 can transmit the voice information of the video it is playing to the wireless broadcasting device 11 based on the A2DP connection.

[0021] In one embodiment, since the wireless broadcasting device 11 needs to transmit compressed information back to the terminal device 12 while receiving the voice information of the video from the terminal device 12, in order to prevent the transmission of the compressed information returned by the wireless broadcasting device 11 to the terminal device 12 from occupying too much communication bandwidth between the wireless broadcasting device 11 and the terminal device 12, thereby congesting the transmission of the voice information transmitted by the terminal device 12 to the wireless broadcasting device 11, the transmission of the compressed information returned by the wireless broadcasting device 11 to the terminal device 12 needs to be performed in the time gap of the voice information transmitted by the terminal device 12 to the wireless broadcasting device 11. Specifically, the terminal device 12 encapsulates the data of the voice information into data packets, and then uses the communication protocol between the terminal device 12 and the wireless broadcasting device 11 to sequentially transmit the data packets of the voice information to the wireless broadcasting device 11. Since the transmission of the data packets of the voice information has time gaps (i.e., the terminal device 12 does not transmit the data packets of the voice information uninterruptedly), the wireless broadcasting device 11 can encapsulate the compressed information into data packets after decapsulating the received data packets of the voice information and compressing the information, and then use the communication protocol between the terminal device 12 and the wireless broadcasting device 11 to sequentially transmit the data packets of the compressed information back to the terminal device 12 in the time gap of the transmission of the data packets of the voice information (i.e., the transmission of the data packets of the compressed information is interspersed between the transmission of the data packets of the voice information). In other words, the wireless broadcasting device 11 continuously transmits the data packets of the compressed information back to the terminal device 12 during the gaps of continuously receiving the data packets of the voice information.

[0022] In one embodiment, the wireless broadcasting device 11 can be used to continuously receive the voice information and compress the voice information with a default compression ratio through a voice compression algorithm. The voice compression algorithm can be any algorithm that can meet the auditory requirements (user satisfaction), such as G.711A, G.711U, G.729, 324M and other voice compression algorithms. The default compression ratio can be but is not limited to 16:1. The actual voice compression algorithm and the default compression ratio can be adjusted according to actual needs.

[0023] In one embodiment, each data packet of the compressed information may include a CRC16 check code, and the terminal device 12 is further used to continuously determine whether the CRC16 check code included in each data packet of the compressed information received in real time has passed the check, so as to decompress the data packets of the compressed information received in real time that have passed the check, and notify the wireless broadcasting device 11 to retransmit the data packets of the compressed information that have failed the check. Therefore, the occurrence of subsequent decompression failures caused by transmission errors of the compressed information can be prevented.

[0024] In one embodiment, the wireless communication protocol between the wireless broadcasting device 11 and the terminal device 12 may be Bluetooth. Therefore, the wireless broadcasting device 11 may also continuously transmit the compressed information back to the terminal device 12 based on the Serial Port Profile (SPP) protocol.

[0025] In one embodiment, when the terminal device 12 transmits the information to be translated to the cloud server 13 for translation, it also transmits a language instruction (i.e., the language instruction is used to specify the target language) to the cloud server 13, so that the cloud server 13 translates the received information to be translated into text information in the target language based on the language instruction. The target language may be, but is not limited to, Chinese. In one example, the process of the cloud server 13 translating the received information to be translated into text information in the target language may include: converting the received information to be translated into text information in the same language as the voice information through speech to text technology; and translating the text information in the same language as the voice information into text information in the target language through text translation technology.

[0026] In another embodiment, the target language may be a default language, and therefore, the terminal device 12 only needs to transmit the information to be translated to the cloud server 13 (i.e., there is no need to transmit the language instruction to the cloud server 13) to obtain the text information in the target language returned by the cloud server 13.

[0027] In one embodiment, the terminal device 12 can display the text information in the form of a floating window on the screen of the video being played. In another embodiment, the terminal device 12 can directly superimpose the text information on the screen of the video being played in a manner similar to augmented reality technology.

[0028] See also Figure 2 , which is a flowchart of an embodiment of a multimedia processing method according to the present application. Figure 2 As shown, the multimedia processing method 2 can be applied to Figure 1The multimedia interactive system 1 and the multimedia processing method 2 are as follows: the terminal device 12 establishes a connection with the wireless broadcasting device 11 and transmits a transcription instruction to the wireless broadcasting device 11 (step 21); during the video playback process of the terminal device 12, the wireless broadcasting device 11 connected to the terminal device 12 continuously receives and compresses the voice information of the video to continuously transmit the compressed information back to the terminal device 12 (step 22); the terminal device 12 decompresses the compressed information received in real time to obtain the information to be translated (step 23); the terminal device 12 transmits the information to be translated to the cloud server 13 for translation to obtain text information in the target language (step 24); and the terminal device 12 displays the text information on the screen of the video it plays (step 25). That is to say, the multimedia interactive system 1 can use the wireless broadcasting device 11 as a medium for capturing and transmitting the voice information of the video, and then use the cloud server 13 to perform the translation function, so that the terminal device 12 can provide the user with subtitles in a familiar language corresponding to the video he is watching. The detailed description has been explained in the above paragraph and will not be repeated here.

[0029] In one embodiment, the terminal device 12 transmitting the transcription instruction to the wireless broadcasting device 11 in step 21 may include: the terminal device 12 downloads and installs an application for controlling the wireless broadcasting device 11, so as to transmit the transcription instruction to the wireless broadcasting device 11 connected thereto through the application. The detailed description has been described in the above paragraphs and will not be repeated here.

[0030] In one embodiment, the step 21 of establishing a connection between the terminal device 12 and the wireless broadcasting device 11 may include: the terminal device 12 and the wireless broadcasting device 11 establish a Bluetooth audio transmission model (A2DP) protocol connection. The detailed description has been described in the above paragraphs and will not be repeated here.

[0031] In one embodiment, step 22 may include: the wireless broadcasting device 11 continuously transmits the compressed information data packet to the terminal device 12 during the interval of continuously receiving the voice information data packet. The detailed description has been explained in the above paragraphs and will not be repeated here.

[0032] In one embodiment, step 22 may include: the wireless broadcasting device 11 continuously transmits the compressed information to the terminal device 12 based on the Serial Port Profile (SPP) protocol. The detailed description has been described in the above paragraphs and will not be repeated here.

[0033] In one embodiment, step 22 may include: the wireless broadcasting device 11 continuously receives the voice information, and compresses the voice information at a default compression ratio using a voice compression algorithm. The detailed description has been described in the above paragraphs and will not be repeated here.

[0034] In one embodiment, each data packet of the compressed information may include a CRC16 check code, and step 23 of the terminal device 12 decompressing the compressed information received in real time may include: the terminal device 12 continuously determines whether the CRC16 check code included in each data packet of the compressed information received in real time passes the check, so as to decompress the data packet of the compressed information received in real time that passes the check, and notifies the wireless broadcasting device 11 to retransmit the data packet of the compressed information that fails the check. The detailed description has been described in the above paragraphs and will not be repeated here.

[0035] In one embodiment, step 24 may include: when the terminal device 12 transmits the information to be translated to the cloud server 13 for translation, it also transmits a language instruction to the cloud server 13, so that the cloud server 13 translates the received information to be translated into text information in the target language based on the language instruction. The detailed description has been explained in the above paragraph and will not be repeated here.

[0036] In one embodiment, step 25 may include: the terminal device 12 displays the text information in the form of a floating window on the screen of the video being played.

[0037] In another embodiment, step 25 may include: the terminal device 12 directly superimposes the text information onto the screen of the played video in a manner similar to augmented reality technology.

[0038] In summary, in the embodiment of the present application, the wireless broadcasting device obtains the voice information of the video played in real time by the terminal device, and transmits back the compressed information of the corresponding voice information to the terminal device, so that the terminal device decompresses the compressed information and translates it through the cloud server, and obtains the text information in the target language corresponding to the played video (that is, the text information in the language familiar to the user), and then displays the text information on the screen of the played video to improve the user's experience of watching the video. In addition, since the voice information of the video played in real time by the terminal device is obtained through the wireless broadcasting device, the multimedia processing method and multimedia interaction system of the embodiment of the present application are suitable for application in the case where the terminal device playing the video cannot directly obtain the voice information of the video watched by the user (for example: the terminal device cannot directly obtain the voice information of the video watched by the user due to permission issues).

[0039] Although the above-described components are included in the drawings of the present application, it does not exclude the use of more additional components to achieve better technical effects without violating the spirit of the invention.

[0040] Although the present invention is described using the above embodiments, it should be noted that these descriptions are not intended to limit the present invention. On the contrary, the present invention covers obvious modifications and similar arrangements that are obvious to those skilled in the art. Therefore, the scope of the claims should be interpreted in the broadest way to include all obvious modifications and similar arrangements.

Claims

1. A multimedia processing method, characterized in that: include: The terminal device establishes a connection with the wireless broadcasting device and transmits a transcription instruction to the wireless broadcasting device; During the video playback process of the terminal device, the wireless broadcasting device connected to the terminal device continuously receives and compresses the voice information of the video to continuously transmit the compressed information back to the terminal device, wherein each data packet of the compressed information includes a CRC16 check code; The terminal device decompresses the compressed information received in real time to obtain information to be translated; The terminal device transmits the information to be translated to the cloud server for translation to obtain text information in the target language; and The terminal device displays the text information on the screen of the video being played; The step of the terminal device decompressing the compressed information received in real time comprises: the terminal device continuously determines whether the CRC16 check code included in each data packet of the compressed information received in real time passes the check, so as to decompress the data packet of the compressed information received in real time that passes the check, and notifies the wireless broadcasting device to retransmit the data packet of the compressed information that fails the check; Among them, the step of the wireless broadcasting device connected to the terminal device continuously receiving and compressing the voice information of the video to continuously return the compressed information to the terminal device includes: the wireless broadcasting device continuously returns the data packets of the compressed information to the terminal device during the intervals of continuously receiving the data packets of the voice information.

2. The multimedia processing method according to claim 1, characterized in that: The step of establishing a connection between the terminal device and the wireless broadcasting device and transmitting a transcription instruction to the wireless broadcasting device includes: The terminal device downloads and installs an application for controlling the wireless broadcasting device, so as to transmit the transcription instruction to the wireless broadcasting device connected thereto through the application.

3. The multimedia processing method according to claim 1, characterized in that: The step of establishing a connection between the terminal device and the wireless broadcasting device includes: The terminal equipment and the wireless broadcasting device establish a Bluetooth audio transmission model protocol connection.

4. The multimedia processing method according to claim 1, characterized in that: The step of the wireless broadcasting device connected to the terminal device continuously receiving and compressing the voice information of the video to continuously transmit the compressed information back to the terminal device includes: The wireless broadcasting device continuously transmits the compressed information back to the terminal device based on the serial port specification protocol.

5. The multimedia processing method according to claim 1, characterized in that: The step of the wireless broadcasting device connected to the terminal device continuously receiving and compressing the voice information of the video includes: The wireless broadcasting device continuously receives the voice information and compresses the voice information at a default compression ratio using a voice compression algorithm.

6. A multimedia interactive system, characterized in that: include: Wireless broadcasting device; A terminal device, used to establish a connection with the wireless broadcasting device and transmit a transcription instruction to the wireless broadcasting device; as well as A cloud server connected to the terminal device via a network; Wherein, during the process of the terminal device playing the video, the wireless broadcasting device connected to the terminal device and receiving the transcription instruction continuously receives and compresses the voice information of the video to continuously transmit the compressed information back to the terminal device; the terminal device decompresses the compressed information received in real time to obtain information to be translated; the terminal device transmits the information to be translated to the cloud server for translation to obtain text information in the target language; and the terminal device displays the text information on the screen of the video it plays; Wherein, each data packet of the compressed information includes a CRC16 check code, and the terminal device is further used to continuously determine whether the CRC16 check code included in each data packet of the compressed information received in real time passes the check, so as to decompress the data packet of the compressed information received in real time that passes the check, and notify the wireless broadcasting device to retransmit the data packet of the compressed information that fails the check; The wireless broadcasting device continuously transmits the compressed information data packet to the terminal device during the interval of continuously receiving the voice information data packet.

7. The multimedia interactive system according to claim 6, characterized in that: The terminal device is also used to download and install an application for controlling the wireless broadcasting device, so as to transmit the transcription instruction to the wireless broadcasting device connected thereto through the application.

8. The multimedia interactive system according to claim 6, characterized in that: The terminal device is also used to establish a Bluetooth audio transmission model protocol connection with the wireless broadcasting device.

9. The multimedia interactive system according to claim 6, characterized in that: The wireless broadcasting device is also used for continuously transmitting the compressed information back to the terminal device based on the serial port specification protocol.

10. The multimedia interactive system according to claim 6, characterized in that: The wireless broadcasting device is used to continuously receive the voice information and compress the voice information at a default compression ratio using a voice compression algorithm.

Images