Method, apparatus, device, storage medium and computer program product for multimedia synchronous playing

Abstract

The application relates to a multimedia synchronous playing method, device, equipment, storage medium and computer program product. The method comprises the following steps: acquiring multimedia data of a target multimedia from a data source equipment, and analyzing the multimedia data to obtain continuous content data, and determining the analysis sequence between each content data; determining the independent timestamp corresponding to each content data, and sequentially encapsulating the content data and the corresponding independent timestamp into the multimedia data frame of each content data based on the analysis sequence; and sequentially sending each multimedia data frame to a master equipment and a slave equipment, so that the slave equipment determines the slave playing progress of playing the target multimedia based on the sequentially received multiple multimedia data frames, and adjusts the slave playing progress according to the playing progress difference between the slave playing progress and the master playing progress of the master equipment playing the target multimedia, so that the master equipment and the slave equipment synchronously play the target multimedia. The method can improve the reliability of multimedia synchronous playing.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a method, apparatus, device, storage medium, and computer program product for synchronous multimedia playback. Background Technology

[0002] With the continuous development of internet technology, wireless communication can provide a more free and convenient multimedia playback experience. However, during multimedia playback, multiple wireless playback devices may be simultaneously playing the same multimedia data, which raises the issue of synchronizing playback progress between wireless playback devices.

[0003] Currently, it's possible to establish wireless communication connections between smart devices and multiple wireless playback devices. When a smart device plays multimedia data, the multimedia data is then transmitted wirelessly to the master device of the wireless playback device. The master device then sends the received multimedia data to the slave devices, allowing both the master and slave devices to play the same multimedia data. However, the cascading relationship between the smart device, master device, and slave devices, and the inherent time lag in data exchange, can lead to desynchronization when multiple wireless devices play the same multimedia simultaneously, reducing the reliability of synchronized playback. Therefore, improving the reliability of synchronized multimedia playback is a pressing issue that needs to be addressed. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, device, storage medium, and computer program product for multimedia synchronous playback that improves the reliability of multimedia synchronous playback, addressing the aforementioned technical problems.

[0005] Firstly, this application provides a method for synchronized multimedia playback. The method is applied to an adapter device connected to a data source device, and the adapter device provides a wireless communication connection between a master device and a slave device; the method includes:

[0006] Obtain multimedia data of the target multimedia from the data source device, parse the multimedia data to obtain multiple consecutive content data, and determine the parsing order between each content data.

[0007] Determine the independent timestamp corresponding to each content data, and encapsulate multiple content data and their corresponding independent timestamps into a multimedia data frame corresponding to each content data based on the parsing order.

[0008] Each multimedia data frame is sent sequentially to the master device and the slave device, so that the slave device can determine the playback progress of the target multimedia based on the multiple multimedia data frames received in sequence, and adjust the playback progress according to the difference between the playback progress of the slave device and the main playback progress of the target multimedia played by the master device, so that the master device and the slave device can play the target multimedia synchronously.

[0009] Secondly, this application also provides another method for synchronized multimedia playback. The method is applied to a slave device, which wirelessly connects to a master device via an adapter, and the adapter is connected to a data source device; the method includes:

[0010] The system receives multiple consecutive multimedia data frames of the target multimedia device, which are sent sequentially by the adapter device. The target multimedia device plays the multimedia. Each multimedia data frame includes content data and an independent timestamp corresponding to the content data.

[0011] Each multimedia data frame is parsed according to the receiving order to determine the playback progress of the target multimedia; the playback progress is used to characterize the independent timestamp corresponding to the multimedia data frame parsed by the slave device.

[0012] Obtain the main playback progress of the target multimedia on the main device; the main playback progress is used to characterize the independent timestamps corresponding to the multimedia data frames parsed by the main device;

[0013] The playback progress is adjusted according to the difference between the playback progress of the slave device and the main playback progress, so that the main device and the slave device can play the target multimedia synchronously.

[0014] Thirdly, this application also provides a multimedia synchronous playback device. The device is applied to an adapter device, which is connected to a data source device, and the adapter device provides a wireless communication connection between the master device and the slave device; the device includes:

[0015] The data processing module is used to acquire multimedia data of the target multimedia from the data source device, parse the multimedia data to obtain multiple consecutive content data, and determine the parsing order between each content data.

[0016] The data encapsulation module is used to determine the independent timestamp corresponding to each content data, and encapsulate multiple content data and the independent timestamp corresponding to each content data into a multimedia data frame corresponding to each content data according to the parsing order.

[0017] The data transmission module is used to send each multimedia data frame sequentially to the master device and the slave device, so that the slave device can determine the playback progress of the target multimedia based on the multiple multimedia data frames received in sequence, and adjust the playback progress according to the difference between the playback progress of the slave device and the main playback progress of the target multimedia played by the master device, so that the master device and the slave device play the target multimedia synchronously.

[0018] Fourthly, this application also provides another multimedia synchronous playback device. The device is applied to a slave device, which wirelessly connects to a master device via an adapter, and the adapter is connected to a data source device; the device includes:

[0019] The data receiving module is used to receive multiple consecutive multimedia data frames of the target multimedia sent sequentially by the adapter device; the target multimedia is played on the data source device; the multimedia data frame includes content data and an independent timestamp corresponding to the content data;

[0020] The data parsing module is used to parse each multimedia data frame according to the receiving order to determine the playback progress of the target multimedia; the playback progress is used to characterize the independent timestamp corresponding to the multimedia data frame parsed by the slave device.

[0021] The data receiving module is also used to obtain the main playback progress of the target multimedia played by the main device; the main playback progress is used to characterize the independent timestamps corresponding to the multimedia data frames parsed by the main device;

[0022] The synchronous playback module is used to adjust the playback progress of the slave device according to the difference between the playback progress of the slave device and the main playback progress, so that the main device and the slave device can play the target multimedia synchronously.

[0023] Fifthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0024] Obtain multimedia data of the target multimedia from the data source device, parse the multimedia data to obtain multiple consecutive content data, and determine the parsing order between each content data.

[0025] Determine the independent timestamp corresponding to each content data, and encapsulate multiple content data and their corresponding independent timestamps into a multimedia data frame corresponding to each content data based on the parsing order.

[0026] Each multimedia data frame is sent sequentially to the master device and the slave device, so that the slave device can determine the playback progress of the target multimedia based on the multiple multimedia data frames received in sequence, and adjust the playback progress according to the difference between the playback progress of the slave device and the main playback progress of the target multimedia played by the master device, so that the master device and the slave device can play the target multimedia synchronously.

[0027] Sixthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0028] The system receives multiple consecutive multimedia data frames of the target multimedia device, which are sent sequentially by the adapter device. The target multimedia device plays the multimedia. Each multimedia data frame includes content data and an independent timestamp corresponding to the content data.

[0029] Each multimedia data frame is parsed according to the receiving order to determine the playback progress of the target multimedia; the playback progress is used to characterize the independent timestamp corresponding to the multimedia data frame parsed by the slave device.

[0030] Obtain the main playback progress of the target multimedia on the main device; the main playback progress is used to characterize the independent timestamps corresponding to the multimedia data frames parsed by the main device;

[0031] The playback progress is adjusted according to the difference between the playback progress of the slave device and the main playback progress, so that the main device and the slave device can play the target multimedia synchronously.

[0032] Seventhly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0033] Obtain multimedia data of the target multimedia from the data source device, parse the multimedia data to obtain multiple consecutive content data, and determine the parsing order between each content data.

[0034] Determine the independent timestamp corresponding to each content data, and encapsulate multiple content data and their corresponding independent timestamps into a multimedia data frame corresponding to each content data based on the parsing order.

[0035] Each multimedia data frame is sent sequentially to the master device and the slave device, so that the slave device can determine the playback progress of the target multimedia based on the multiple multimedia data frames received in sequence, and adjust the playback progress according to the difference between the playback progress of the slave device and the main playback progress of the target multimedia played by the master device, so that the master device and the slave device can play the target multimedia synchronously.

[0036] Eighthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0037] The system receives multiple consecutive multimedia data frames of the target multimedia device, which are sent sequentially by the adapter device. The target multimedia device plays the multimedia. Each multimedia data frame includes content data and an independent timestamp corresponding to the content data.

[0038] Each multimedia data frame is parsed according to the receiving order to determine the playback progress of the target multimedia; the playback progress is used to characterize the independent timestamp corresponding to the multimedia data frame parsed by the slave device.

[0039] Obtain the main playback progress of the target multimedia on the main device; the main playback progress is used to characterize the independent timestamps corresponding to the multimedia data frames parsed by the main device;

[0040] The playback progress is adjusted according to the difference between the playback progress of the slave device and the main playback progress, so that the main device and the slave device can play the target multimedia synchronously.

[0041] Ninthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0042] Obtain multimedia data of the target multimedia from the data source device, parse the multimedia data to obtain multiple consecutive content data, and determine the parsing order between each content data.

[0043] Determine the independent timestamp corresponding to each content data, and encapsulate multiple content data and their corresponding independent timestamps into a multimedia data frame corresponding to each content data based on the parsing order.

[0044] Each multimedia data frame is sent sequentially to the master device and the slave device, so that the slave device can determine the playback progress of the target multimedia based on the multiple multimedia data frames received in sequence, and adjust the playback progress according to the difference between the playback progress of the slave device and the main playback progress of the target multimedia played by the master device, so that the master device and the slave device can play the target multimedia synchronously.

[0045] Tenthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0046] The system receives multiple consecutive multimedia data frames of the target multimedia device, which are sent sequentially by the adapter device. The target multimedia device plays the multimedia. Each multimedia data frame includes content data and an independent timestamp corresponding to the content data.

[0047] Each multimedia data frame is parsed according to the receiving order to determine the playback progress of the target multimedia; the playback progress is used to characterize the independent timestamp corresponding to the multimedia data frame parsed by the slave device.

[0048] Obtain the main playback progress of the target multimedia on the main device; the main playback progress is used to characterize the independent timestamps corresponding to the multimedia data frames parsed by the main device;

[0049] The playback progress is adjusted according to the difference between the playback progress of the slave device and the main playback progress, so that the main device and the slave device can play the target multimedia synchronously.

[0050] The aforementioned method, apparatus, computer device, storage medium, and computer program product for synchronous multimedia playback acquire multimedia data of the target multimedia from a data source device through an adapter device, parse the multimedia data to obtain multiple consecutive content data, and determine the parsing order between each content data; determine the independent timestamp corresponding to each content data, and encapsulate the multiple content data and the independent timestamp corresponding to each content data into a multimedia data frame corresponding to each content data according to the parsing order; and send each multimedia data frame sequentially to the master device and the slave device, so that the slave device determines the secondary playback progress of the target multimedia based on the multiple multimedia data frames received sequentially, and adjusts the secondary playback progress according to the playback progress difference between the secondary playback progress and the master playback progress of the target multimedia played by the master device, so that the master device and the slave device play the target multimedia synchronously. By employing the aforementioned method for synchronized multimedia playback, the adapter device can sequentially encapsulate multiple content data points and their corresponding independent timestamps into multimedia data frames for each content data point, based on the parsing order of the multimedia data of the target multimedia. In other words, the multimedia data frames for each content data point are ordered sequentially according to the content data sequence. These encapsulated multimedia data frames are then sent sequentially to both the master and slave devices. This allows the master and slave devices to simultaneously play the same target multimedia with the data source device. Furthermore, the master device's playback progress is used as the basis for adjusting the slave device's playback progress. This ensures that the time difference between the slave device's playback progress and the master device's playback progress does not affect the synchronized playback effect, thus guaranteeing synchronized playback of the target multimedia by both the master and slave devices and improving the reliability of synchronized multimedia playback. Attached Figure Description

[0051] Figure 1 This is an application environment diagram of a multimedia synchronous playback method in one embodiment;

[0052] Figure 2 This is a schematic diagram illustrating the Bluetooth pairing process between a Bluetooth master device and a Bluetooth slave device in one embodiment.

[0053] Figure 3 This is a flowchart illustrating the interaction process of a device during multimedia synchronous playback in one embodiment.

[0054] Figure 4 This is a flowchart illustrating a method for synchronous multimedia playback in one embodiment;

[0055] Figure 5 This is a schematic diagram of the data structure of a multimedia data frame in one embodiment;

[0056] Figure 6 This is a schematic diagram illustrating the data frame encapsulation and data frame transmission process in one embodiment;

[0057] Figure 7 This is a schematic diagram illustrating the process of determining the master and slave devices in one embodiment;

[0058] Figure 8 Here is a flowchart of frequency band allocation in one embodiment;

[0059] Figure 9 This is an example of an interaction flowchart for determining the master and slave devices in one embodiment;

[0060] Figure 10 This is a schematic diagram of the master-slave device switching process in one embodiment;

[0061] Figure 11 This is a schematic diagram of the data structure of a heartbeat data frame in one embodiment;

[0062] Figure 12 This is a flowchart illustrating the process of a master and slave device sending heartbeat data frames in one embodiment;

[0063] Figure 13 This is a flowchart illustrating the updating of data frame loss count and data timeout count based on the master heartbeat data frame in one embodiment.

[0064] Figure 14 This is a flowchart of the master-slave switching determination in one embodiment;

[0065] Figure 15 This is a flowchart of master-slave switching and device address broadcasting in one embodiment;

[0066] Figure 16 This is a flowchart illustrating a method for synchronous multimedia playback in another embodiment;

[0067] Figure 17 This is an interactive flowchart illustrating the communication and interaction between the adapter device, the master device, and the slave device in one embodiment to adjust the playback progress.

[0068] Figure 18 This is a flowchart illustrating the adjustment based on playback progress in one embodiment;

[0069] Figure 19 This is a flowchart illustrating the process of obtaining the main playback progress of the target multimedia on the main device in one embodiment.

[0070] Figure 20 This is a schematic diagram of the data structure of a synchronization data frame in one embodiment;

[0071] Figure 21 This is a complete flowchart of a method for synchronous multimedia playback in one embodiment;

[0072] Figure 22 This is a structural block diagram of a multimedia synchronous playback device in one embodiment;

[0073] Figure 23 This is a structural block diagram of a multimedia synchronous playback device in another embodiment;

[0074] Figure 24 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0076] With the continuous development of internet technology, wireless communication can provide a more free and convenient multimedia playback experience. However, during multimedia playback, multiple wireless playback devices may simultaneously play the same multimedia data, raising the issue of playback progress synchronization between these devices. Currently, a wireless communication connection can be established between a smart device and multiple wireless playback devices. When multimedia data is played through the smart device, it is wirelessly transmitted to the master device of the wireless playback devices. The master device then sends the received multimedia data to the slave devices, allowing both the master and slave devices to play the same multimedia data. Alternatively, considering the time consumption of multimedia data relay, multimedia data can be directly transmitted from the smart device to two different wireless playback devices, which then play the same multimedia data. However, since there is a cascading relationship between the smart device, the master device, and the slave devices, and the data interaction process inevitably involves information transmission time differences, asynchronous playback may occur when multiple wireless devices play the same multimedia simultaneously, reducing the reliability of synchronized multimedia playback. Secondly, even if the same multimedia data is directly transmitted to both the master and slave devices, delays exist in receiving the data and in parsing and playing it, leading to playback asynchrony and reducing the reliability of synchronized multimedia playback. Therefore, improving the reliability of synchronized multimedia playback is a pressing issue that needs to be addressed.

[0077] This application provides a method for efficient multimedia synchronous playback. The multimedia synchronous playback method provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another server.

[0078] Specifically, this application is specifically applied to terminal 102, which is described as an adapter device. Terminal 102 obtains multimedia data of the target multimedia from the data source device, parses the multimedia data to obtain multiple consecutive content data, determines the parsing order between each content data, and determines the independent timestamp corresponding to each content data. Based on the parsing order, the multiple content data and the independent timestamp corresponding to each content data are encapsulated into a multimedia data frame corresponding to each content data. Then, each multimedia data frame is sent sequentially to the master device and the slave device. At this time, the master device and the slave device can play the same target multimedia simultaneously with the data source device in sequence. Based on this, the slave device determines the secondary playback progress of the target multimedia based on the multiple multimedia data frames received sequentially, and adjusts the secondary playback progress according to the playback progress difference between the secondary playback progress and the main playback progress of the target multimedia played by the master device, so that the master device and the slave device play the target multimedia synchronously. Therefore, the time difference between the secondary playback progress of the slave device and the main playback progress of the master device does not affect the synchronous playback effect, that is, it ensures that the master device and the slave device play the target multimedia synchronously, thereby improving the reliability of multimedia synchronous playback.

[0079] In this embodiment, terminal 102 can be an adapter device, a data source device, a master device, or a slave device. The adapter device is specifically a wireless receiver, a small device that can be plugged into the Universal Serial Bus (USB) interface of other terminal devices to achieve wireless communication. The data source device is not limited to various desktop computers, laptops, smartphones, and tablets capable of multimedia data playback control. The master and slave devices include, but are not limited to, wireless headphones, wireless mice, wireless keyboards, wireless game controllers, various IoT devices, and portable wearable devices capable of multimedia data playback. The aforementioned IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers. The multimedia synchronous playback method provided in this embodiment can be applied to various multimedia playback scenarios, including but not limited to audio playback, video playback, and audio-visual playback. In other words, the multimedia in this embodiment can be audio, video, and audio-visual content, etc., without limitation.

[0080] Since the multimedia synchronous playback method provided in this application embodiment requires an adapter, can be applied to different wireless connection devices, and is based on wireless connection communication, the technical terms involved in multimedia synchronous playback in this application embodiment are explained below:

[0081] 1. 2.4G Dongle.

[0082] A 2.4G dongle, also known as a 2.4GHz wireless receiver, is a small device that plugs into the USB port of a desktop computer, laptop, smartphone, or other terminal device to enable wireless communication. Therefore, a 2.4G dongle can receive 2.4GHz wireless signals from wireless devices (such as wireless headphones, wireless mice, wireless keyboards, wireless game controllers, etc.) and then convert the received 2.4GHz signal into input for the USB device. This allows devices without a built-in wireless receiver (such as some older computers or desktop computers) to establish wireless communication connections with wireless devices. Furthermore, considering the good penetration and coverage of 2.4GHz wireless signals, a 2.4G dongle can typically provide a stable wireless connection. Therefore, the adapter device in this embodiment can be a 2.4G dongle.

[0083] 2. True Wireless Stereo (TWS) microphone speaker.

[0084] A TWS microphone speaker is a speaker device that combines TWS technology and a microphone. Therefore, a TWS microphone speaker consists of two wirelessly connected speakers and one or more microphones. The working principle of a TWS microphone speaker is that the speakers receive audio signals from an audio source device (such as a mobile phone, tablet, etc.) via Bluetooth or other wireless connection technology, and then synchronize the audio between the two speakers to achieve stereo playback. Simultaneously, the microphones can capture sound and transmit it wirelessly to the audio source device, enabling voice input or singing functionality.

[0085] 3. Bluetooth pairing.

[0086] Bluetooth pairing is a security process; the purpose of Bluetooth pairing is to establish a trusted link for secure wireless communication between two wireless devices. For ease of understanding, as... Figure 2The diagram illustrates the Bluetooth pairing process between a Bluetooth master device and a Bluetooth slave device. First, a search and connection process is performed. This means that before pairing, both Bluetooth devices (the master and slave devices) need to be searched and connected, ensuring they are both set to discoverable mode. This allows Bluetooth communication between the master device 201 and the slave device 202. At this point, either Bluetooth device searches for the target device. Once the searching device finds the target device, it initiates a Bluetooth pairing request. For example, if both the master device 201 and the slave device 202 are set to discoverable mode, they sequentially press and hold the pairing button and reset the pairing information to enter pairing mode. After the master device 201 discovers the slave device 202, it initiates a Bluetooth pairing request, which includes a verification code.

[0087] Therefore, after a Bluetooth pairing request is accepted, the receiving device will verify the connection using a checksum. This verification ensures that all devices communicating via Bluetooth are legitimate, and confirms the establishment of a Bluetooth connection only if the devices are legitimate. Specifically, after the Bluetooth master device 201 initiates a Bluetooth pairing request to the Bluetooth slave device 202, the slave device 202 extracts the checksum from the request and verifies the legitimacy of the Bluetooth master device 201 based on the checksum. Upon successful verification, the Bluetooth slave device and the Bluetooth master device will begin the Bluetooth pairing process. During this process, the Bluetooth slave device and the Bluetooth master device will exchange keys (specifically, a pairing code). This pairing code is the key for communication between the devices; only Bluetooth devices that obtain the pairing code can communicate via Bluetooth. After Bluetooth slave device 202 successfully verifies the legitimacy of Bluetooth master device 201 based on the checksum, it sends pairing information to Bluetooth master device 201. This pairing information carries a key. Bluetooth master device 201 receives the pairing information and verifies the key based on it. Specifically, it verifies whether the key has timed out. If it has, it needs to resend and receive the Bluetooth pairing request, repeating the steps described above until the key has not timed out. After completing the key verification and confirming successful pairing, Bluetooth master device 202 can also send a pairing success message back to Bluetooth slave device 202, so that both Bluetooth master device 201 and Bluetooth slave device 202 confirm successful pairing. At this time, Bluetooth slave device 202 also needs to verify whether the key has timed out. If it has, it also needs to resend and receive the Bluetooth pairing request, repeating the steps described above until the key has not timed out. Finally, upon successful pairing, the Bluetooth master device 201 and the Bluetooth slave device 202 will establish an encrypted link. The Bluetooth master device 201 and the Bluetooth slave device 202 need to communicate and interact through this encrypted link to ensure the security of wireless communication between them. Therefore, after completing device pairing and encryption, the multimedia synchronous playback method provided in this embodiment can be implemented.

[0088] This application requires data interaction between the data source device, the adapter device, the master device, and the slave device to achieve synchronized multimedia playback. Therefore, the interaction process between the data source device, the adapter device, the master device, and the slave device during synchronized multimedia playback in the embodiments of this application is described below. Figure 3The diagram illustrates the interaction flow of the devices during synchronized multimedia playback. First, when the target multimedia is played on the data source device 302, the adapter device 304 collects the multimedia data of the target multimedia being played on the data source device 302. Then, the multimedia data of the target multimedia is divided into multiple consecutive content data, and the parsing order between each content data is determined. The adapter device 304 then determines the independent timestamp corresponding to each content data through the device's local timestamp. The device's local timestamp can be consistent with the general timestamp, or it can be the timestamp used for device timing; there is no limitation here. At this time, based on the parsing order, the adapter device 304 encapsulates the multiple content data and the independent timestamp corresponding to each content data into a multimedia data frame corresponding to each content data, and then sends each multimedia data frame sequentially to the master device 306, the slave device 308, and the slave device 310.

[0089] Based on this, the master device 306 parses each multimedia data frame in the receiving order to determine the main playback progress of the target multimedia. Similarly, slave devices 308 and 310 can also parse each multimedia data frame in the receiving order to determine the secondary playback progress of the target multimedia played by slave device 308 and slave device 310, respectively. The master device 306 plays the target multimedia according to the main playback progress and sends the main playback progress to the adapter device 304. The adapter device 304 encapsulates the main playback progress into a synchronization data frame and sends the synchronization data frame to slave devices 308 and 310. The slave device 308 receives the synchronization data frame, parses it to obtain the main playback progress, and then adjusts its secondary playback progress according to the difference between its secondary playback progress and the main playback progress. At this point, the master device 306 and slave device 308 can play the target multimedia synchronously. Similarly, the slave device 310 receives and parses the synchronization data frames to obtain the main playback progress. At this time, the slave device 310 needs to adjust its playback progress according to the difference between its own playback progress and the main playback progress. Then, the master device 306 and the slave device 310 can play the target multimedia synchronously. This completes the synchronous playback of multimedia between the master device and multiple slave devices.

[0090] The following examples illustrate this in detail: In one example, as... Figure 4 As shown, a method for synchronous playback of multimedia is provided, which can be applied to... Figure 1Taking terminal 102 as an example, and specifically as an adapter device, the adapter device is connected to the data source device and provides a wireless communication connection between the master device and the slave device. It is understood that this method can also be applied to a system including terminal 102 and server 104, and implemented through the interaction between terminal 102 and server 104. In this embodiment, the method includes the following steps:

[0091] Step 402: Obtain multimedia data of the target multimedia from the data source device, parse the multimedia data to obtain multiple consecutive content data, and determine the parsing order between each content data.

[0092] The target multimedia is the multimedia being played by the data source device. The target multimedia can be audio, video, or audio-visual. The multimedia data can correspond to the audio data of the audio, the video data of the video, and the audio-visual data of the audio-visual. The multimedia data can be the complete data of the target multimedia or a part of the complete data. For example, if the target multimedia is 4 minutes of audio, the multimedia data can be the audio data of the 4 minutes of audio, or the multimedia data can be the audio data of 10 seconds of the 4 minutes of audio. There is no limitation here.

[0093] Secondly, the content data includes multimedia content. Therefore, multiple consecutive pieces of content data can constitute continuous multimedia content within the target multimedia. For example, if the multimedia data is 10 seconds of audio data within a 4-minute audio file, then the content data can be audio frames. Multiple consecutive pieces of content data are multiple consecutive audio frames, and these multiple consecutive audio frames constitute 10 seconds of audio data. Furthermore, considering that multiple consecutive pieces of content data can specifically constitute continuous multimedia content within the target multimedia, to ensure sequential playback of the target multimedia, there must be a corresponding parsing order among the multiple pieces of content data obtained from the multimedia data parsing. This parsing order is the playback order of the multiple pieces of content data. The content data in the initial order of the parsing order is the initial frame in the multimedia data, while the content data in the final order of the parsing order is the final frame in the multimedia data.

[0094] Specifically, when the data source device is playing target multimedia, the adapter device can obtain the multimedia data of the target multimedia from the data source device. Considering that the adapter device can be a wireless receiver directly built into the data source device, it can directly obtain the multimedia data of the target multimedia from the data source device. However, for data source devices without a built-in wireless receiver, the adapter device can be the 2.4G Dongle described in the previous embodiment, meaning the adapter device needs to collect the multimedia data of the target multimedia through the USB interface of the data source device. The method of obtaining multimedia data is not limited here.

[0095] Furthermore, after acquiring the multimedia data of the target multimedia, the adapter device parses the multimedia data to obtain multiple consecutive content data. During the parsing process, the parsing order between each content data is determined, that is, the playback data of each content data in the multimedia data is determined as the parsing order between each content data. For example, in 1 second of audio data, the audio is played according to content data A1, content data A2, content data A3, and so on up to content data A101. Then, content data A1 is the initial audio frame in the 1 second of audio data, and content data A101 is the last audio frame in the 1 second of audio data. The parsing order of multiple audio frames (i.e., multiple content data) is content data A1, content data A2, content data A3, and so on up to content data A101.

[0096] Step 404: Determine the independent timestamp corresponding to each content data, and encapsulate multiple content data and the independent timestamp corresponding to each content data into a multimedia data frame corresponding to each content data according to the parsing order.

[0097] The independent timestamp is the device-local timestamp of the adapted device. This local timestamp can be the same as the universal timestamp, or it can be the timestamp used for device timing. Specifically, the independent timestamp is a string of numbers. For example, if the device-local timestamp is the same as the universal timestamp, then it is the number of seconds since Coordinated Universal Time (UTC), that is, the number of seconds since January 1, 1970. The timing reference for the independent timestamp is not limited here. Secondly, the multimedia data frame includes the corresponding content data, the independent timestamp corresponding to the content data, and the multimedia data frame may also include a checksum and an error correction code.

[0098] Specifically, the adapter determines an independent timestamp for each piece of content data. This means the adapter determines the independent timestamp for each piece of content data sequentially according to the parsing order. The time interval between the independent timestamps of the content data is consistent with the playback time interval between the content data in the multimedia data. For example, if the parsing order of multiple content data obtained from 1 second of audio data is content data A1, content data A2, content data A3, and so on up to content data A101, and the playback time interval between content data A1 and content data A101 is 10ms, then the time interval between the independent timestamps of content data A1 and A2 is 10ms, and the time interval between the independent timestamps of content data A2 and A3 is also 10ms. That is, when the independent timestamp of content data A1 is 0, the independent timestamp of content data A2 is 10ms (i.e. 0.01s), while the independent timestamp of content data A3 is 20ms (i.e. 0.02s), the independent timestamp of content data A4 is 30ms (i.e. 0.03s), and so on. The independent timestamp of content data A101 is 100ms (i.e. 1s), which means that content data A101 is the last audio frame in the 1s audio data.

[0099] Furthermore, the adapter sequentially encapsulates multiple content data points and their corresponding independent timestamps into multimedia data frames for each content data point, based on the parsing order. In other words, the adapter encapsulates the content data and its corresponding independent timestamp for each content data point to obtain the multimedia data frame. Therefore, the multimedia data frame includes the content data and its independent timestamp. For example, multimedia data frame B1 for content data A1 includes content data A1 and its independent timestamp "0", while multimedia data frame B2 for content data A2 includes content data A2 and its independent timestamp "10ms".

[0100] Considering that the adapter device provides a wireless communication connection between the master and slave devices, when the wireless communication between the master and slave devices is specifically Bluetooth communication, as described above, the master and slave devices need to communicate and interact through an encrypted link during the Bluetooth communication process to ensure the security of wireless communication between the master and slave devices. Therefore, the adapter device can also synchronously encapsulate the checksum and error correction code during the encapsulation process. Thus, multimedia data frames can also include checksums and error correction codes. For ease of understanding, as follows... Figure 5 The data structure of the multimedia data frame shown is as follows: the data header of the multimedia data frame is a checksum, the data trailer of the multimedia data frame is an error correction code, and the multimedia data frame includes content data and an independent timestamp corresponding to the content data.

[0101] The aforementioned checksum is used to verify the integrity of multimedia data frames, preventing data loss or errors during transmission from the adapter device to the master and slave devices. Therefore, when the master and slave devices detect an anomaly in the multimedia data frame using the checksum, they can request the adapter device to re-transmit the multimedia data frame. The error correcting code, on the other hand, is used by the master and slave devices to correct errors that occur during multimedia data frame transmission. If the error cannot be corrected, the adapter device can be requested to re-transmit the multimedia data frame.

[0102] Step 406: Each multimedia data frame is sent sequentially to the master device and the slave device, so that the slave device determines the playback progress of the target multimedia based on the multiple multimedia data frames received sequentially, and adjusts the playback progress according to the difference between the playback progress of the slave device and the main playback progress of the target multimedia played by the master device, so that the master device and the slave device play the target multimedia synchronously.

[0103] Specifically, the adapter sends each multimedia data frame sequentially to the master device and the slave device. That is, the adapter sends each multimedia data frame to the master device and the slave device in the order of its independent timestamp, i.e., sequentially sending multimedia data frame B1 of content data A1, multimedia data frame B2 of content data A2, multimedia data frame B3 of content data A3, and multimedia data frame B101 of content data A101 to the master device and the slave device. Then, based on the sequentially received multimedia data frames, the master device determines the main playback progress of the target multimedia and forwards this main playback progress to the slave device through the adapter. The slave device, based on the sequentially received multimedia data frames, determines the secondary playback progress of the target multimedia and adjusts it according to the difference between the secondary playback progress and the main playback progress of the target multimedia played by the master device. This ensures that the master device and the slave device play the target multimedia synchronously. The specific adjustment method is described in detail in subsequent embodiments.

[0104] In one optional embodiment, during the establishment of a wireless communication connection between the master device and the slave device, each device is allocated a corresponding frequency band. Therefore, the adapter device will send each multimedia data frame to the master device sequentially according to the order of the independent timestamps carried by each multimedia data frame, based on the frequency band corresponding to the master device. Similarly, it will send each multimedia data frame to the slave device sequentially according to the order of the independent timestamps carried by each multimedia data frame, based on the frequency band corresponding to the slave device.

[0105] For ease of understanding, such as Figure 6The flowchart shown illustrates the data frame encapsulation and transmission process of the adapter device. When the target multimedia is played on the data source device 602, the adapter device 604 obtains the multimedia data of the target multimedia from the data source device 602, parses the multimedia data to obtain multiple consecutive content data, and determines the independent timestamp corresponding to each content data. For each content data, the adapter device encapsulates the content data and the independent timestamp into a corresponding multimedia data frame. If frequency band 1 is allocated to the master device 606, frequency band 6 is allocated to the slave device 608, and frequency band 11 is allocated to the slave device 610, then the adapter device 604 will sequentially send each multimedia data frame to the master device 606 via frequency band 1, and the master device 606 will sequentially receive each multimedia data frame via frequency band 1. Similarly, the adapter device 604 will sequentially send each multimedia data frame to the slave device 608 via frequency band 6, and the master device 608 will sequentially receive each multimedia data frame via frequency band 6. The adapter 604 will send each multimedia data frame sequentially to the slave device 610 via frequency band 11, and the master device 610 will receive each multimedia data frame sequentially via frequency band 11.

[0106] It is understood that the foregoing examples are for the purpose of understanding this solution and should not be construed as specific limitations on this solution.

[0107] In the aforementioned method for synchronous multimedia playback, the adapter device can sequentially encapsulate multiple content data and their corresponding independent timestamps into multimedia data frames corresponding to each content data based on the parsing order of the multimedia data of the target multimedia. This means that the multimedia data frames corresponding to each content data are ordered sequentially according to the content data sequence. These encapsulated multimedia data frames are then sent sequentially to the master and slave devices. Therefore, the master and slave devices can simultaneously play the same target multimedia with the data source device in sequence. The master device's playback progress of the target multimedia is used as the benchmark for the slave device's progress adjustment. Adjusting the slave device's playback progress of the target multimedia ensures that the time difference between the slave device's playback progress and the master device's playback progress does not affect the synchronous playback effect. This guarantees that the master and slave devices play the target multimedia synchronously, thereby improving the reliability of synchronous multimedia playback.

[0108] Since the prerequisite for multimedia synchronous playback in the aforementioned embodiments is that the adapter device provides a wireless communication connection between the master device and the slave device, and it is also necessary to determine the master device and slave device among each wireless connection device, the following describes how to establish a wireless communication connection and determine the master and slave devices: In one embodiment, such as Figure 7 As shown, methods for synchronized multimedia playback also include:

[0109] Step 702: In response to the pairing requests of the multiple devices to be paired, a corresponding frequency band is allocated to each device to be paired, so that wireless communication connection is provided between each device to be paired through the allocated frequency band.

[0110] The pairing request is used to instruct the device to be paired to establish a wireless communication connection with other devices to be paired. Taking Bluetooth communication as an example, the pairing request is the Bluetooth pairing request described in the previous embodiment. Secondly, the frequency band specifically refers to the communication frequency band, which is a continuous range of electromagnetic waves, i.e., the aforementioned frequency band used for wireless communication. Wireless routers typically have two frequency bands: 2.4GHz and 5GHz. Since the adapter device in this embodiment can be a 2.4GHz dongle, it operates within the 2.4GHz communication frequency band. The 2.4GHz operating frequency band is specifically 2400~2483.5MHz, with each channel occupying approximately 20MHz. Therefore, 2.4GHz can be divided into 14 channels, with the 14th channel typically not used for data transmission. Therefore, different frequency bands should be allocated to different devices to be paired to avoid frequency band interference affecting the reliability of data transmission.

[0111] Specifically, when each device to be paired needs to establish a wireless communication connection with other devices, it can initiate a pairing request. The adapter device then receives and responds to each pairing request from the various devices. This involves triggering the pairing controls on both the adapter device and each device to be paired. These triggering operations can include long presses and continuous clicks, among others, and are not limited here. The adapter device then receives each device's pairing request and responds to it to reset the device pairing information. At this point, it needs to identify unassigned frequency bands from the frequency band table and assign a corresponding frequency band (i.e., the pairing band) to each device to complete the pairing. The adapter device can then provide wireless communication connections between the paired devices using the assigned frequency band. For ease of understanding, as follows... Figure 8 The flowchart shown illustrates the frequency band allocation process. It triggers the pairing controls for adapter 801, device 802, device 803, and device 804. At this point, all four devices begin pairing. The unassigned frequency bands in the frequency band table include bands 1, 6, and 11. Therefore, band 1 can be assigned to device C1, band 6 to device C2, and band 11 to device C3, thus completing the pairing communication.

[0112] Since there are multiple pairing requests from different devices, the following describes how to allocate frequency bands: In a specific embodiment, in response to the pairing requests from multiple devices, a corresponding frequency band is allocated to each device, including: in response to the pairing requests from multiple devices, determining the request timestamp of each pairing request; the request timestamp is either the request reception timestamp or the request initiation timestamp; and allocating the corresponding frequency band to each device according to the request timestamp of each pairing request.

[0113] Specifically, in response to pairing requests from multiple devices to be paired, the adapter determines the request timestamp for each pairing request. This means the adapter can determine the request reception timestamp of the received pairing request, or it can extract the request initiation timestamp from the pairing request. Therefore, the request timestamp can be either the request reception timestamp or the request initiation timestamp. Based on this, the adapter allocates the corresponding frequency band to each device to be paired sequentially according to the request timestamp of each pairing request. In other words, the adapter allocates unassigned frequency bands from the frequency band table to each device to be paired according to the order described by the request timestamps of each pairing request. For example, again... Figure 8 As an example, the request reception timestamp of device 802 to be matched is the earliest, while that of device 804 is the latest. The order is: device 802, device 803, and device 804. Therefore, frequency band 1 can be allocated to device 802 first, then frequency band 6 to device 803, and finally frequency band 11 to device 804. It is understandable that in practical applications, frequency bands can be randomly allocated to the devices to be matched; this is not a limitation here.

[0114] Step 704: Determine the device to be matched whose assigned frequency band is a preset frequency band as the master device, and determine the device to be matched whose assigned frequency band is not a preset frequency band as the slave device.

[0115] Specifically, the adapter identifies devices assigned to a preset frequency band as the master device, and devices not assigned to the preset frequency band as slave devices. For ease of understanding, let's use frequency band 1 as the preset frequency band, and... Figure 8 The flowchart of frequency band allocation shown will be further explained. Since frequency band 1 is allocated to device 802, frequency band 6 is allocated to device 803, and frequency band 11 is allocated to device 804, device 802, which is allocated frequency band 1 (i.e., the preset frequency band), can be identified as the master device, while devices 803 and 804 are slave devices. It can be understood that the preset frequency band can also be frequency band 13 or other frequency bands within the available frequency band range; this is not limited here.

[0116] Step 706: Broadcast the master device address of the master device so that the master device carries the address of the device to be verified when sending the main playback progress, and enables the slave device to perform device address matching based on the master device address and the address of the device to be verified.

[0117] Specifically, after the adapter device determines the master device and slave device, it first determines the master device address of the master device and then broadcasts the master device address to both the master device and the slave device. This allows the device to be matched, identified as the master device, to confirm its own master status through the master device address. In subsequent processes, after the master device determines the main playback progress, it needs to send the main playback progress and the address of the device to be verified (i.e., the master device address stored in the master device itself) to the adapter device. This allows the adapter device to forward the main playback progress and the address of the device to be verified to the slave device. Since the slave device can determine that it is not the master device through the master device address broadcast by the adapter device, it determines itself as a slave device. After determining the secondary playback progress, the slave device needs to perform device address matching using the received master device address and the address of the device to be verified. If the device address matching is successful, the secondary playback progress is adjusted based on the difference between the secondary and main playback progress.

[0118] To facilitate understanding of the aforementioned process, based on Figure 8 The flowchart of frequency band allocation shown will be further explained, such as... Figure 9 The flowchart shown illustrates the interaction process for determining the master and slave devices. If the device to be matched in frequency band 1 is determined as the master device, that is, if device 802 is determined as the master device, then device 803 and device 804 are slave devices. At this time, the adapter device 801 also needs to determine the master device address of device 802, and then broadcast the master device address of device 802 to device 802 (master device), device 803 (slave device), and device 804 (slave device).

[0119] It is understood that the foregoing examples are for the purpose of understanding this solution and should not be construed as specific limitations on this solution.

[0120] In this embodiment, different frequency bands should be allocated to different devices to be matched to avoid frequency band interference affecting the reliability of data transmission, that is, to improve the reliability of multimedia data frame interaction between master and slave devices, and thus improve the reliability of multimedia synchronous playback.

[0121] Considering the possibility of communication problems with the master device in practical applications, that is, in addition to dividing the master and slave devices by frequency band, master-slave switching can also be performed based on actual data transmission. This is described below: In one embodiment, such as... Figure 10As shown, methods for synchronized multimedia playback also include:

[0122] Step 1002: Receive multiple master heartbeat data frames sequentially fed back by the master device, and determine the master data information of each master heartbeat data frame.

[0123] The master heartbeat data frame is the heartbeat data frame fed back by the master device. A heartbeat data frame includes at least the sending device's address, the sequence number of the heartbeat data frame, a checksum, and an error correction code. Therefore, the master heartbeat data frame includes at least the master device's address, the sequence number of the master heartbeat data frame, a checksum, and an error correction code. The master data information includes at least the data information contained in the master heartbeat data frame, as well as the sending and receiving information of the master heartbeat data frame. Therefore, the master data information includes at least the master device's address, the sequence number of the master heartbeat data frame, the sending timestamp of the master heartbeat data frame, the receiving timestamp of the master heartbeat data frame, a checksum, and an error correction code.

[0124] Specifically, the master device sends master heartbeat data frames to the adapter device sequentially at preset time intervals. The master device sends these master heartbeat data frames to the adapter device sequentially at preset time intervals based on the frequency band it is matched with. For example, multiple master heartbeat data frames include: master heartbeat data frame D1, master heartbeat data frame D2, master heartbeat data frame D3, and master heartbeat data frame D4. The data sequence number in each master heartbeat data frame indicates its order within the multiple master heartbeat data frames. For instance, if the data sequence number of master heartbeat data frame D1 is 1, then master heartbeat data frame D1 is the first frame in the multiple master heartbeat data frames. If the data sequence number of master heartbeat data frame D2 is 2, then master heartbeat data frame D2 is the second frame in the multiple master heartbeat data frames, and so on. Similarly, the slave device also sends slave heartbeat data frames to the adapter device sequentially at preset time intervals. The slave device sends these slave heartbeat data frames to the adapter device sequentially at preset time intervals based on the frequency band it is matched with. The aforementioned preset time interval can be 5ms, 10ms, etc., and is not limited here.

[0125] Based on this, since the master device sends master heartbeat data frames to the adapter device sequentially at preset time intervals, the adapter device can receive multiple master heartbeat data frames sequentially fed back by the master device, that is, receive multiple master heartbeat data frames sequentially fed back by the master device according to the frequency band matched by the master device. Then, for each master heartbeat data frame, the master data information of each master heartbeat data frame is determined, that is, the data information included in each master heartbeat data frame is extracted, and the transmit and receive information recorded for each master heartbeat data frame is recorded, such as recording the receive timestamp of the master heartbeat data frame, etc. To facilitate understanding of the heartbeat data frame, such as... Figure 11The heartbeat data frame shown has a data header containing a checksum, a data trailer containing an error correction code, and includes the device address and data sequence number. The master data information is located within... Figure 11 The heartbeat data frame shown carries information in the case of feedback from the master device, namely, the master data information, which includes at least the master device's device address, the data sequence number of the master heartbeat data frame, the checksum, and the error correction code.

[0126] For ease of understanding, such as Figure 12 The flowchart shown illustrates the transmission of heartbeat data frames between master and slave devices. It includes an adapter device 1201, a master device 1202, a slave device 1203, and a slave device 1204. The master device 1202 sends a master heartbeat data frame once, waits a preset time interval, and then sends another master heartbeat data frame. During this time, the sequence number in the master heartbeat data frame is incremented, allowing the adapter device 1201 to receive multiple master heartbeat data frames sent by the master device 1202. Similarly, the slave device 1203 sends a slave heartbeat data frame once, waits a preset time interval, and then sends another slave heartbeat data frame. During this time, the sequence number in the slave heartbeat data frame is incremented, allowing the adapter device 1201 to receive multiple slave heartbeat data frames sent by the slave device 1203. The method by which the adapter device 1201 receives multiple slave heartbeat data frames sent by the slave device 1204 is similar and will not be described further here.

[0127] Based on this, the adapter needs to determine whether the master-slave switching conditions are met through multiple master data information. If the conditions are met, step 1004 is executed to perform the master-slave device conversion. If not, the original master and slave devices continue to play multimedia synchronously.

[0128] Step 1004: Based on the master data information, if the master device meets the master-slave switching conditions, perform master-slave device conversion between the master device and each slave device.

[0129] Specifically, when the master data information meets the master-slave switching conditions, the adapter device performs a master-slave device conversion between the master device and each slave device. That is, the adapter device switches the master device to a slave device and then determines the master device from among the multiple slave devices. Then, it broadcasts the device address to be updated to the master device so that all wireless connected devices update their stored master device addresses, completing the master-slave device conversion. In other words, the subsequent process uses the updated master device and slave devices for synchronized multimedia playback.

[0130] The following describes how to determine if the master-slave switching conditions are met: In a specific embodiment, the method for determining if the master device meets the master-slave switching conditions based on each master data information includes at least one of the following: when the data sequence number of each master data information is sorted in a jump arrangement, the master device is determined to meet the master-slave switching conditions; when the reception time difference between the reception timestamps of each master data information is greater than the time difference threshold, the master device is determined to meet the master-slave switching conditions.

[0131] In this case, the skip arrangement is a non-sequential arrangement. For example, if the data sequence numbers of each master data information are: master heartbeat data frame D1 data sequence number "1", master heartbeat data frame D2 data sequence number "2", master heartbeat data frame D3 data sequence number "3", and master heartbeat data frame D4 data sequence number "4", then the sequential arrangement is 1234. The non-sequential arrangement can be: 1243, 1324, 1342, 2134, etc., which are not in the order of 1234. Specifically, when the adapter device receives multiple master heartbeat data frames sequentially, it will determine the master data information of each master heartbeat data frame. At this time, the data sequence numbers in the master data information can be arranged according to the receiving order of the master heartbeat data frames to obtain the sorting method of the data sequence numbers. When the sorting method of the data sequence numbers of each master data information is a skip arrangement, it is determined that the master device has lost frames when sending heartbeat data frames, and thus it is determined that the master device meets the master-slave switchover conditions. For example, when the adapter receives the master heartbeat data frame D1, it can record the data sequence number "1". Then, upon receiving the master heartbeat data frame D3, it can record the data sequence number "3". If it receives the master heartbeat data frame D2, it can record the data sequence number "2", and upon receiving the master heartbeat data frame D4, it can record the data sequence number "4". At this point, the data sequence numbers of each master data information are arranged in a 1324 pattern, meaning the sorting method is a jump arrangement. This indicates that the master device has experienced frame loss while sending heartbeat data frames, thus confirming that the master device meets the master-slave switchover conditions.

[0132] Considering the instability of network communication, assuming the data sequence numbers of each master data item are sorted using a jump arrangement, we first determine if the master device is experiencing frame loss when sending heartbeat data frames. At this point, we can further update the master device's frame loss count. Only when the master device's frame loss count reaches a frame loss threshold is it determined that the master device meets the master-slave switchover conditions. For example, if the master device first determines that heartbeat data frame loss has occurred, the frame loss count is set to 1. If frame loss occurs again subsequently, the frame loss count is updated to 2. Taking a frame loss threshold of 5 as an example, this means that when the master device's frame loss count reaches 5, it is determined that the master device meets the master-slave switchover conditions. The aforementioned frame loss threshold can also be flexibly adjusted to values ​​such as 3 and 4 based on actual application requirements; no limitation is made here.

[0133] Secondly, the time difference threshold can be consistent with or close to the preset time interval. For example, if the preset time interval is 10ms, then the time difference threshold can be 10ms or 12ms. That is, the master device will send master heartbeat data frames to the adapter device sequentially according to the preset time interval, so the time difference for the adapter device to receive the master heartbeat data frames should be close to the preset time interval. Specifically, during the process of sequentially receiving multiple master heartbeat data frames, the adapter device can record the timestamp of each received master heartbeat data frame, and then compare the timestamp of the received master heartbeat data frame with the timestamp of the previous master heartbeat data frame. If the timestamp is greater than the time difference threshold, it is determined that the master device has timed out sending the heartbeat data frame, and thus the master device meets the master-slave switchover conditions.

[0134] Considering the instability of network communication, if the receiving time difference exceeds a time difference threshold, we first determine that the master device has timed out sending heartbeat data frames. At this point, we can further update the master device's data timeout count. Only when the master device's data timeout count reaches a timeout threshold are we determined that the master device meets the master-slave switchover conditions. For example, if the master device first determines that a heartbeat data frame has timed out, the master device's data timeout count is set to 1. If a timeout occurs again, the master device's data timeout count is updated to 2. Taking a timeout threshold of 3 as an example, when the master device's data timeout count is updated to 3, it is determined that the master device meets the master-slave switchover conditions. The aforementioned timeout threshold can also be 4, 5, or other values ​​that can be flexibly adjusted based on actual application requirements; no limitation is made here.

[0135] For ease of understanding, such as Figure 13 The flowchart shown illustrates the process of updating the number of data loss frames and the number of data timeouts based on the master heartbeat data frames. Specifically, the sorting method of the data sequence numbers of each master data information is a jump arrangement to determine frame loss, and the adapter device updates the number of data loss frames accordingly. Similarly, if the reception time difference between the reception timestamps of each master data information exceeds a time difference threshold, a timeout is determined, and the adapter device updates the number of data timeouts accordingly. The received master heartbeat data frames are then continuously processed in the aforementioned manner.

[0136] Furthermore, the adapter devices can also jointly consider the number of data frame drops and the number of data timeouts. That is, they make a joint judgment based on the total number of data frame drops and data timeouts. Specifically, after determining, through the aforementioned methods, that the master device has experienced frame drops or timeouts in sending heartbeat data frames, the update of the data frame drop count and data timeout count considers the total number of these events. In other words, it is necessary to check if the total number reaches a threshold. Only when the total number of data frame drops and data timeouts reaches the threshold is it determined that the master device meets the master-slave switchover conditions. For ease of understanding, as follows... Figure 14 In the master-slave handover determination process shown, the adapter device determines the master device 1401, slave device 1402, and slave device 1403 using the method described in the aforementioned embodiments. Then, the adapter device receives heartbeat data frames according to the different frequency bands allocated to the master device 1401, slave device 1402, and slave device 1403, respectively. Then, based on... Figure 13 The method shown is similar to determine frame loss and timeout situations. At this time, the total number of data frame loss and data timeouts of the master device 1401 (E1), the total number of data frame loss and data timeouts of the slave device 1402 (E2), and the total number of data frame loss and data timeouts of the slave device 1403 (E3) are recorded. Then, it is determined whether the total number of data frame loss and data timeouts has reached the total number threshold. That is, if it is determined that the total number of data frame loss and data timeouts has reached the total number threshold, it is determined that the master-slave switchover condition is met. If it has not been met, heartbeat data frames continue to be received and the total number of data frame loss and data timeouts is updated.

[0137] The following describes the specific implementation method for master-slave device switching: In an optional embodiment, the method for multimedia synchronous playback further includes: for each slave device, receiving multiple slave heartbeat data frames sequentially fed back by the slave devices, and determining the slave data information of each slave heartbeat data frame.

[0138] Specifically, the adapter receives multiple slave heartbeat data frames sequentially fed back by each slave device and determines the slave data information for each slave heartbeat data frame. That is, the slave devices also send slave heartbeat data frames to the adapter sequentially at preset time intervals. The adapter can then receive these multiple slave heartbeat data frames sequentially fed back by the slave devices, i.e., according to the frequency band matched to each slave device. Then, for each slave heartbeat data frame, the adapter determines the slave data information for that specific slave heartbeat data frame. The method of receiving slave heartbeat data frames and determining slave data information is similar to the method of receiving master heartbeat data frames and determining master data information, and will not be described in detail here.

[0139] Based on this, a master-slave device conversion is performed between the master device and each slave device, including: selecting slave devices whose data sequence numbers are sorted sequentially as candidate slave devices; selecting the candidate slave device with the smallest reception time difference between the reception timestamps of each slave data information as the target slave device; switching the master device to a slave device, switching the target slave device to the updated master device, and broadcasting the updated master device address as the updated master device address.

[0140] Specifically, since the slave data information is similar to the master data information, the adapter device determines the slave data information of each slave heartbeat data frame during the process of sequentially receiving multiple slave heartbeat data frames. At this point, the data sequence numbers in the slave data information are arranged according to the receiving order of the slave heartbeat data frames to obtain a sorting method. The slave devices whose data sequence numbers are arranged sequentially are selected as candidate slave devices. In other words, the adapter device does not consider slave devices that experience frame loss during heartbeat data frame transmission. Then, during the process of sequentially receiving multiple slave heartbeat data frames, the adapter device records the receiving timestamp of each received slave heartbeat data frame. It then compares the receiving timestamp of the slave heartbeat data frame with the receiving timestamp of the previous slave heartbeat data frame. Therefore, the candidate slave devices are sorted according to the value of the receiving time difference, and the candidate slave device with the smallest value is selected as the target slave device. The target slave device is the updated master device.

[0141] Furthermore, considering that determining whether the master device meets the master-slave switchover conditions before the conversion can be done by determining at least one of the number of data frame drops and the number of data timeouts, the same method can be used to determine the number of data frame drops and the number of data timeouts for the slave device. Based on this, in practical applications, the adapter device can select slave devices with both a data frame drop count less than the frame drop threshold and a data timeout count less than the timeout threshold as candidate slave devices, and then determine the candidate slave device with the smallest total of data frame drops and data timeouts as the target slave device. The method for determining the target slave device is not limited here.

[0142] Furthermore, the adapter switches from master to slave, and then switches the target slave to the updated master. The updated master's address is then broadcast as the updated master's address. In other words, after the master-slave switch is complete, the updated master determines and sends the master playback progress as described in subsequent embodiments, while the wireless connection device switched to slave determines the slave playback progress as described in subsequent embodiments, and adjusts the slave playback progress according to the slave and master playback progress to achieve synchronized multimedia playback.

[0143] For ease of understanding, such as Figure 15The flowchart shown illustrates the master-slave handover and device address broadcasting process. The adapter device determines the master device 1401, slave device 1402, and slave device 1403 using the method described in the preceding embodiments. Then, the adapter device receives heartbeat data frames according to the different frequency bands allocated to the master device 1401, slave device 1402, and slave device 1403, respectively. Then, based on... Figure 13 The method shown is similar to that used to determine frame loss and timeout situations. Then, it is determined whether the total number of data frame loss and data timeouts reaches the total number threshold. That is, if the total number of data frame loss and data timeouts E1 of the master device 1401 reaches the total number threshold, it is determined that the master-slave switchover conditions are met. Then, the target slave device is determined from slave devices 1402 and 1403 in the manner described above. Taking slave device 1402 as the target slave device as an example, during the master-slave switchover process, the master device 1401 needs to be switched to slave device 1402, and the slave device 1402 determined as the target slave device needs to be switched to the updated master device. Then, the device address of slave device 1402 is used as the updated master device address, and the device address of slave device 1402 is broadcast.

[0144] It is understood that the foregoing examples are for the purpose of understanding this solution and should not be construed as specific limitations on this solution.

[0145] In this embodiment, heartbeat data frames from the master and slave devices are received at preset time intervals, and the communication reliability of the master and slave devices is evaluated from two dimensions: data frame loss and data timeout. That is, if the communication of the master device may have problems, the master and slave devices are switched to ensure that the determined master device is a stable and reliable wireless communication device. This ensures the reliability of the master device in sending the main playback progress during multimedia synchronous playback, and further improves the reliability of media synchronous playback.

[0146] The foregoing embodiments described data processing and master-slave device switching operations performed by the adapter device during multimedia synchronous playback. However, this application embodiment specifically describes multimedia synchronous playback between the slave device and the master device. Therefore, the process of multimedia synchronous playback by the slave device is explained in detail through the following embodiments: In one embodiment, as... Figure 16 As shown, a method for synchronous playback of multimedia is provided, which can be applied to... Figure 1Taking terminal 102 as an example, and specifically as a slave device, the slave device establishes a wireless communication connection with the master device through an adapter device, which in turn connects to the data source device. It is understood that this method can also be applied to a system including terminal 102 and server 104, and is implemented through the interaction between terminal 102 and server 104. In this embodiment, the method includes the following steps:

[0147] Step 1602: Receive multiple consecutive multimedia data frames of the target multimedia sent sequentially by the adapter device; the target multimedia is played on the data source device; the multimedia data frames include content data and independent timestamps corresponding to the content data.

[0148] The target multimedia refers to the multimedia being played on the data source device, which can be audio, video, or a combination of both. Secondly, the multimedia data frame specifically refers to the data frame corresponding to the content data. The multimedia data frame includes the corresponding content data, an independent timestamp corresponding to the content data, and may also include a checksum and an error correction code. The data structure of the target multimedia and the multimedia data frame is similar to that described in the previous embodiments and is not limited here.

[0149] Specifically, as described in the foregoing embodiments, the adapter device sequentially encapsulates multiple content data and their corresponding independent timestamps into multimedia data frames corresponding to each content data based on the parsing order. After completing the encapsulation, the adapter device sequentially sends each multimedia data frame to both the master device and the slave device. Therefore, the master device can receive multiple consecutive multimedia data frames of the target multimedia sequentially sent by the adapter device, and the slave device can also receive multiple consecutive multimedia data frames of the target multimedia sequentially sent by the adapter device. Furthermore, during the establishment of a wireless communication connection between the master device and the slave device, each device is allocated a corresponding frequency band. Therefore, the adapter device can also sequentially send each multimedia data frame to the slave device according to the order of the independent timestamps carried by each multimedia data frame, based on the frequency band corresponding to the slave device. At this time, the slave device can receive multiple consecutive multimedia data frames of the target multimedia sequentially sent by the adapter device according to the corresponding frequency band.

[0150] Step 1604: Parse each multimedia data frame according to the receiving order to determine the playback progress of the target multimedia; the playback progress is used to characterize the independent timestamp corresponding to the multimedia data frame parsed by the slave device.

[0151] The receiving order refers to the sequence in which multiple multimedia data frames are received. Secondly, the playback progress is used to represent the independent timestamps corresponding to the multimedia data frames parsed by the slave device. Specifically, when the slave device receives multiple consecutive multimedia data frames, it determines the receiving order between them. For ease of understanding, if the adapter device sequentially sends multimedia data frame B1 of content data A1, multimedia data frame B2 of content data A2, multimedia data frame B3 of content data A3, and then multimedia data frame B101 of content data A101 to both the master and slave devices, then the slave device can sequentially receive multimedia data frames B1, B2, B3, and finally B101. That is, the receiving order for the aforementioned multiple multimedia data frames is from multimedia data frame B1 to multimedia data frame B101.

[0152] Furthermore, through methods such as Figure 5 As shown in the data structure of the multimedia data frame, the header of the multimedia data frame is a checksum, the trailer is an error correction code, and the multimedia data frame includes content data and an independent timestamp corresponding to the content data. Therefore, the slave device parses each multimedia data frame in the order of reception to obtain the independent timestamp corresponding to each multimedia data frame. At this time, the independent timestamp is determined as the playback progress of the target multimedia on the slave device. For example, to further illustrate the previous example, if the reception order of the aforementioned multimedia data frames is multimedia data frame B1 to multimedia data frame B101, that is, the slave device parses multimedia data frames B1 to multimedia data frame B101 in sequence, and can obtain the independent timestamp "0" for multimedia data frame B1, the independent timestamp "10ms" for multimedia data frame B2, the independent timestamp "20ms" for multimedia data frame B3, and so on, up to the independent timestamp "1000ms" for multimedia data frame B101.

[0153] Furthermore, it is understandable that before the slave device obtains the main playback progress of the target multimedia from the master device, it first plays the multimedia data frames corresponding to the independent timestamps represented by the slave playback progress.

[0154] Step 1606: Obtain the main playback progress of the target multimedia on the main device; the main playback progress is used to characterize the independent timestamps corresponding to the multimedia data frames parsed by the main device.

[0155] The main playback progress is used to characterize the independent timestamps corresponding to the multimedia data frames parsed by the master device. Specifically, the main playback progress is similar to the secondary playback progress; that is, the main playback progress is determined by the master device parsing each multimedia data frame in the order of reception. After determining the main playback progress, the master device sends it to the adapter device in real time. The adapter device then forwards the main playback progress to each secondary device through the corresponding frequency band. Therefore, the secondary devices can obtain the main playback progress of the target multimedia played by the master device, that is, the secondary devices receive the main playback progress of the target multimedia played by the master device forwarded by the adapter device through the corresponding frequency band.

[0156] It is understandable that the aforementioned main playback progress can be encapsulated in a synchronization data frame. That is, the main device sends the main playback progress to the adapter device, the adapter device encapsulates the main playback progress to obtain a synchronization data frame, and then sends the synchronization data frame to the slave device, so that the slave device can parse the synchronization data frame to obtain the main playback progress of the target multimedia played by the main device.

[0157] Step 1608: Adjust the playback progress of the slave device according to the difference between the playback progress of the slave device and the main playback progress, so that the main device and the slave device can play the target multimedia synchronously.

[0158] Specifically, the slave device adjusts its playback progress according to the difference between its own playback progress and the master playback progress, ensuring synchronized playback of the target multimedia between the master and slave devices. In other words, the slave device determines the playback progress difference between the target multimedia played on the master and slave devices based on the time difference between the independent timestamps represented by the slave playback progress and those represented by the master playback progress. When the playback progress difference is small, playback is performed directly using the content data corresponding to the independent timestamps represented by the slave playback progress, allowing the slave device to synchronize with the master device. Conversely, when the playback progress difference is large, playback is performed using the content data corresponding to the adjusted independent timestamps represented by the slave playback progress, again ensuring synchronized playback of the target multimedia between the slave and master devices.

[0159] For ease of understanding, such as Figure 17The flowchart shown illustrates the interaction between the adapter device, master device, and slave device to adjust the playback progress. The adapter device 1702 sequentially sends each multimedia data frame to the master device 1704, slave device 1706, and slave device 1708. The master device 1704 parses the multiple multimedia data frames in the order of receipt to obtain the content data and the main playback progress. Then, the master device 1704 sends the main playback progress to the adapter device 1702. The adapter device 1702 forwards the main playback progress to the slave devices 1706 and 1708. Therefore, the slave device 1706 can parse the multiple multimedia data frames in the order of receipt to obtain the content data and the secondary playback progress, and receive the main playback progress from the adapter device 1702. Thus, it adjusts the secondary playback progress based on the difference between the main playback progress and the secondary playback progress. Since the processing steps of the slave devices 1706 and 1708 are similar, the method of adjusting the secondary playback progress of the slave device 1708 will not be described in detail here.

[0160] Furthermore, the following details how to adjust the playback progress to synchronize the target multimedia playback between the device and the main device: In a specific embodiment, the playback progress is adjusted according to the playback progress difference between the playback progress of the secondary playback progress and the main playback progress, including: determining the independent timestamp represented by the playback progress of the secondary playback progress and the progress time difference between the independent timestamp represented by the main playback progress; when the progress time difference is within the first time difference range formed by the minimum offset time difference and the maximum offset time difference, the independent timestamp represented by the playback progress of the secondary playback progress is determined as the target timestamp, and the content data corresponding to the target timestamp is played.

[0161] Specifically, since the playback progress is used to represent the independent timestamp corresponding to the multimedia data frame parsed by the slave device, while the main playback progress is used to represent the independent timestamp corresponding to the multimedia data frame parsed by the master device, the playback progress difference is specifically the progress time difference between the independent timestamp represented by the slave playback progress and the independent timestamp represented by the main playback progress. In other words, the slave device determines the progress time difference as the difference between the independent timestamp represented by the slave playback progress and the independent timestamp represented by the main playback progress. For example, if the slave playback progress of slave device 1 represents the independent timestamp of multimedia data frame B3 as "20ms", and the main playback progress represents the independent timestamp of multimedia data frame B3 as "20ms", then the progress time difference between slave device 1 and the master device is 0ms (20ms - 20ms). Secondly, if the slave playback progress of slave device 2 represents the independent timestamp of multimedia data frame B3 as "20ms", and the main playback progress represents the independent timestamp of multimedia data frame B5 as "40ms", then the progress time difference between slave device 2 and the master device is -20ms (20ms - 40ms).

[0162] Furthermore, the device needs to evaluate the numerical comparison between the progress time difference and the minimum and maximum offset time differences. The aforementioned minimum offset time difference can be -3ms, and the maximum offset time difference can be 3ms. These minimum and maximum offset time differences can be determined based on the target multimedia's playback time requirements and playback scenario. The aforementioned example should not be construed as a limitation on the minimum and maximum offset time differences. Therefore, the minimum and maximum offset time differences can constitute a first time difference range. For example, taking a minimum offset time difference of -3ms and a maximum offset time difference of 3ms as an example, the first time difference range is [-3ms, 3ms].

[0163] Therefore, when the progress time difference is within the first time difference range, it can be assumed that the playback progress difference between the master device and the slave device is small. The slave device then determines the independent timestamp representing the playback progress as the target timestamp and plays the content data corresponding to the target timestamp, enabling the slave device to synchronously play the target multimedia with the master device. For example, further illustrating the previous example, if the progress time difference between slave device 1 and the master device is 0ms, then slave device 1 can determine the independent timestamp "20ms" as the target timestamp and play the content data corresponding to the target timestamp (i.e., content data A3 in multimedia data frame B3).

[0164] When the progress time difference is not within the first time difference range, playback needs to be performed using the content data corresponding to the independent timestamp represented by the adjusted playback progress. This is described below: In an optional embodiment, the multimedia synchronous playback method further includes: when the progress time difference is within the second time difference range formed by the minimum time difference and the minimum offset time difference, accelerating the playback of the secondary playback progress; when the progress time difference is less than the minimum time difference or greater than the maximum time difference, discarding the content data corresponding to the independent timestamp represented by the secondary playback progress and playing the next frame of the discarded content data; when the progress time difference is within the third time difference range formed by the maximum offset time difference and the maximum time difference, updating the main playback progress based on a preset duration, and adjusting the secondary playback progress based on the playback progress difference between the updated main playback progress and the secondary playback progress.

[0165] The minimum time difference is less than the minimum offset time difference. For example, if the minimum offset time difference is -3ms, the minimum time difference can be -30ms. Therefore, the second time difference range formed by the minimum time difference and the minimum offset time difference can be [-30ms, -3ms]. Similarly, the maximum time difference is greater than the maximum offset time difference. For example, if the maximum offset time difference is 3ms, the maximum time difference can be 30ms. Therefore, the third time difference range formed by the maximum time difference and the maximum offset time difference can be [3ms, 30ms].

[0166] Specifically, if the progress time difference is not within the first time difference range, the slave device further determines the range difference between the progress time difference and the minimum and maximum time differences. If the progress time difference is within the second time difference range formed by the minimum time difference and the minimum offset time difference, it means that the slave device's playback progress is later than the master device's playback progress. In this case, the slave device needs to accelerate the playback progress. The aforementioned acceleration playback processing can directly involve discarding non-multimedia data frames (such as data connection frames) to make the slave playback progress consistent with the master playback progress, or accelerating the decoding rate of multimedia data frames and the playback rate of content data to make the slave playback progress match the master playback progress. The specific acceleration playback processing method is not limited here.

[0167] Furthermore, if the progress time difference is less than the minimum time difference, it means that the slave device's playback progress is significantly later than the master device's playback progress. In this case, normal accelerated playback may not be able to keep up with the master playback progress. The slave device needs to start playing the content data corresponding to the independent timestamp represented by the master playback progress. Alternatively, if the progress time difference is greater than the maximum time difference, it means that the slave device's playback progress is significantly earlier than the master device's playback progress. In this case, after playing the content data corresponding to the independent timestamp represented by the playback progress, the slave device discards that frame of content data and waits. That is, it waits until the content data corresponding to the independent timestamp represented by the master playback progress is the discarded content data before playing the next frame of content data after the discarded content data.

[0168] Based on this, when the progress time difference falls within the third time difference range formed by the maximum offset time difference and the maximum time difference, it indicates that the slave device's playback progress is earlier than the master device's playback progress. In this case, the slave device needs to slow down the playback progress. That is, the slave device can update the master playback progress based on a preset duration, and then determine the playback progress difference (i.e., the adjusted progress time difference) between the independent timestamp represented by the updated master playback progress and the independent timestamp represented by the slave playback progress. At this point, it is also necessary to determine whether the adjusted progress time difference meets the aforementioned adjustment conditions, i.e., whether it meets any of the following adjustment conditions: the adjusted progress time difference is within the first time difference range, the adjusted progress time difference is within the second time difference range, the adjusted progress time difference is less than the minimum time difference, or the adjusted progress time difference is greater than the maximum time difference. If so, the corresponding adjustment is performed according to the met condition. If the adjusted progress time difference is still within the third time difference range, the master playback progress needs to be updated again based on the preset duration, and the aforementioned steps are repeated until the aforementioned adjustment conditions are met. The aforementioned preset duration can be 1ms, 100 microseconds (μs), etc., and is not limited here.

[0169] For ease of understanding, such as Figure 18 The flowchart shown illustrates the playback progress adjustment process. It involves obtaining an independent timestamp 1801 representing the main playback progress from the device, parsing the content data, determining the corresponding independent timestamp 1802, and then calculating the difference between independent timestamps 1802 and 1801 to obtain the progress time difference. If the progress time difference falls within a first time difference range, the independent timestamp 1802 representing the playback progress is determined as the target timestamp, and the content data corresponding to the target timestamp is played. Next, if the progress time difference falls within a second time difference range, playback is accelerated for independent timestamp 1802. If the progress time difference falls within a third time difference range, timestamp 1801 is updated with a preset duration, the progress time difference is recalculated, and the content data is played based on the recalculated progress time difference. Finally, if the progress time difference is less than the minimum time difference or greater than the maximum time difference, the content data is discarded, and the progress adjustment playback process described in the aforementioned embodiment is performed.

[0170] It is understood that the foregoing examples are for the purpose of understanding this solution and should not be construed as specific limitations on this solution.

[0171] In the above-mentioned method for synchronous multimedia playback, since the adapter device sends the encapsulated multimedia data frames to the master device and the slave device in sequence, the slave devices can play the same target multimedia simultaneously with the data source device in sequence. At this time, the slave device uses the main playback progress of the target multimedia played by the master device as the basis for adjusting the slave device's playback progress, thereby ensuring that the time difference between the slave device's playback progress and the master device's main playback progress does not affect the synchronous playback effect. In other words, it ensures that the master device and the slave device play the target multimedia synchronously, thereby improving the reliability of synchronous multimedia playback.

[0172] The foregoing embodiments describe how the main playback progress can be encapsulated in a synchronous data frame. Therefore, the following describes how to obtain the main playback progress: In one embodiment, as... Figure 19 As shown, the main playback progress of the target multimedia on the main device is obtained, including:

[0173] Step 1902: Obtain the synchronization data frame including the main playback progress of the target multimedia played on the main device, and parse the synchronization data frame to obtain the main playback progress and verification data.

[0174] The synchronization data frame includes the main playback progress and verification data. The verification data includes at least the first pairing code of the master device and the address of the device to be verified. Specifically, the master device parses each multimedia data frame in the receiving order to determine the secondary playback progress of the target multimedia, and simultaneously sends the main playback progress to the adapter device. Therefore, the adapter device obtains the synchronization data frame by scoring based on the main playback progress and verification data, and sends the synchronization data frame to the secondary device. At this time, the secondary device can obtain the synchronization data frame including the main playback progress of the target multimedia played by the master device, and parse the synchronization data frame to obtain the main playback progress and verification data.

[0175] The master device sends the main playback progress to the adapter device sequentially based on the frequency band matched by the master device, while the adapter device sends synchronization data frames to the slave device through the frequency band matched by the slave device. For ease of understanding, as follows... Figure 20 The data structure of the synchronization data frame shown is as follows: the data header of the synchronization data frame is the first pairing code in the verification data, the data tail of the synchronization data frame is the error correction code, and the multimedia data frame also includes the address of the master device to be verified and the main playback progress.

[0176] Based on this, the playback progress is adjusted according to the difference between the secondary playback progress and the main playback progress, including:

[0177] Step 1904: If the verification result of the verification data indicates successful verification, adjust the playback progress according to the difference between the playback progress of the secondary playback progress and the main playback progress.

[0178] Specifically, the device verifies the verification data to obtain a verification result. If the verification result indicates successful verification, the playback progress is adjusted according to the difference between the main playback progress and the secondary playback progress, as described in the aforementioned embodiments. Conversely, if the verification result indicates failed verification, the device directly plays the target multimedia according to the secondary playback progress. The following details how to determine if the verification result indicates successful verification:

[0179] In one alternative embodiment, the verification data includes a first pairing code of the master device. The first pairing code is a pairing code determined when the master device and the slave device establish wireless communication.

[0180] Based on this, the method for synchronous multimedia playback also includes: determining a second pairing code during the process of establishing wireless communication with the master device; performing device pairing verification on the first pairing code and the second pairing code to obtain a pairing verification result; and determining the verification result of the verification data to indicate successful verification when the pairing verification result indicates that the slave device and the master device are connected wirelessly through the same adapter.

[0181] Specifically, since the master device and the slave device exchange pairing codes when establishing wireless communication, the slave device determines the second pairing code during the process of establishing wireless communication with the master device. Then, it performs device pairing verification on the first pairing code and the second pairing code of the master device in the verification data to obtain the pairing verification result. If the pairing verification result indicates that the slave device and the master device are wirelessly connected through the same adapter, it means that the slave device and the master device corresponding to the synchronization data frame are wirelessly connected through the same adapter. At this time, the verification result of the verification data indicates that the verification is successful.

[0182] In one optional embodiment, the verification data includes the device address to be verified of the master device. The device address to be verified is the device address of the master device that sent the main playback progress, and the aforementioned device address is specifically the Media Access Control Address (MAC address).

[0183] Based on this, the method for synchronous multimedia playback also includes: obtaining the master device address determined during the master-slave device configuration process; performing device address matching between the master device address and the address of the device to be verified to obtain the address matching result; and determining the verification result of the verification data to indicate successful verification if the address matching result indicates that the address of the device to be verified is consistent with the master device address.

[0184] As described above, after determining the master device during the master-slave device configuration process, the adapter broadcasts the device address of the determined master device to both the master and slave devices with wireless communication connections. Therefore, the slave device can obtain and store the master device address determined during the master-slave device configuration process. Then, the slave device extracts the device address to be verified from the verification data and performs device address matching between the master device address and the device address to be verified to obtain the address matching result. If the address matching result indicates that the device address to be verified matches the master device address, it means that the slave device and the master device corresponding to the synchronization data frame have a wireless communication connection, and the wireless communication device sending the main playback progress is the master device. At this time, the verification result of the verification data indicates that the verification is successful.

[0185] It is understood that the foregoing examples are for the purpose of understanding this solution and should not be construed as specific limitations on this solution.

[0186] In this embodiment, the main playback progress is obtained by synchronous data frames to ensure the security of the transmission process. Furthermore, the obtained verification data is parsed to verify that the wireless communication device sending the main playback progress is the main device and belongs to the same wireless communication environment as the slave device. This verifies the reliability of the obtained main playback progress, thereby ensuring the reliability of adjusting the slave playback progress according to the playback progress difference between the slave and main playback progress, and further improving the reliability of multimedia synchronous playback.

[0187] Based on the detailed description of the foregoing embodiments, the complete process of the multimedia synchronous playback method in the embodiments of this application will be described below. In one embodiment, such as Figure 21 As shown, a method for synchronized multimedia playback is provided, applicable to a system including a data source device, an adapter device, a master device, and at least one slave device, and implemented through interaction between the data source device, the adapter device, the master device, and at least one slave device. In this embodiment, the method includes the following steps:

[0188] Step 2101: In response to the pairing requests of the multiple devices to be paired, the adapter allocates a corresponding frequency band to each device to be paired, so as to provide wireless communication connection between each device to be paired through the allocated frequency band.

[0189] Step 2102: The adapter determines the device to be matched that is assigned a preset frequency band as the master device, and determines the device to be matched that is assigned a frequency band other than the preset frequency band as the slave device.

[0190] Step 2103: The adapter device broadcasts the master device address to the master device so that the master device carries the address of the device to be verified when sending the main playback progress, and the slave device performs device address matching based on the master device address and the address of the device to be verified.

[0191] Step 2104: The adapter device obtains the multimedia data of the target multimedia from the data source device, parses the multimedia data to obtain multiple consecutive content data, and determines the parsing order between each content data.

[0192] Step 2105: The adapter determines the independent timestamp corresponding to each content data, and encapsulates multiple content data and the independent timestamp corresponding to each content data into a multimedia data frame corresponding to each content data according to the parsing order.

[0193] Step 2106: The adapter sends each multimedia data frame sequentially to the master device and the slave device.

[0194] Step 2107: The main device receives multiple consecutive multimedia data frames of the target multimedia sent sequentially by the adapter device; it parses each multimedia data frame according to the receiving order, determines the main playback progress of the target multimedia, and sends the main playback progress to the adapter device.

[0195] Step 2108: The adapter device encapsulates a synchronization data frame based on the main playback progress and the verification data of the main device, and sends the synchronization data frame to the slave device.

[0196] Step 2109: Receive multiple consecutive multimedia data frames of the target multimedia from the adapter device in sequence; parse each multimedia data frame according to the receiving order to determine the playback progress of the target multimedia.

[0197] Step 2110: Obtain synchronization data frames from the device, including the main playback progress of the target multimedia played on the main device, and parse the synchronization data frames to obtain the main playback progress and verification data.

[0198] Step 2111: If the verification result of the verification data indicates successful verification, adjust the playback progress according to the difference between the playback progress of the slave device and the main playback progress, so that the main device and the slave device can play the target multimedia synchronously.

[0199] It should be understood that the specific implementation methods of steps 2101 to 2111 are similar to those of the aforementioned embodiments, and will not be repeated here.

[0200] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0201] Based on the same inventive concept, this application also provides a multimedia synchronous playback device for implementing the multimedia synchronous playback method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more multimedia synchronous playback device embodiments provided below can be found in the limitations of the multimedia synchronous playback method described above, and will not be repeated here.

[0202] In one embodiment, such as Figure 22 As shown, a multimedia synchronous playback device is provided. This device is applied to an adapter device, which is connected to a data source device and provides a wireless communication connection between the master device and the slave device. The multimedia synchronous playback device includes: a data processing module 2202, a data encapsulation module 2204, and a data transmission module 2206, wherein:

[0203] The data processing module 2202 is used to acquire multimedia data of the target multimedia from the data source device, parse the multimedia data to obtain multiple consecutive content data, and determine the parsing order between each content data.

[0204] The data encapsulation module 2204 is used to determine the independent timestamp corresponding to each content data, and encapsulate multiple content data and the independent timestamp corresponding to each content data into a multimedia data frame corresponding to each content data according to the parsing order.

[0205] The data transmission module 2206 is used to send each multimedia data frame sequentially to the master device and the slave device, so that the slave device can determine the playback progress of the target multimedia based on the multiple multimedia data frames received sequentially, and adjust the playback progress according to the playback progress difference between the slave playback progress and the master playback progress of the target multimedia played by the master device, so that the master device and the slave device play the target multimedia synchronously.

[0206] In one embodiment, the media synchronization playback device further includes a device matching module;

[0207] The device matching module is used to respond to the pairing requests of multiple devices to be matched, allocate a corresponding frequency band to each device to be matched, so as to provide wireless communication connection between each device to be matched through the allocated frequency band; determine the device to be matched whose allocated frequency band is a preset frequency band as the master device, and determine the device to be matched whose allocated frequency band is not a preset frequency band as the slave device; broadcast the master device address of the master device, so that the master device carries the address of the device to be verified when sending the main playback progress, and enables the slave device to perform device address matching based on the address of the device to be verified.

[0208] In one embodiment, the device matching module is specifically used to respond to the pairing requests from multiple devices to be matched, determine the request timestamp of each pairing request, and the request timestamp is either the request receiving timestamp or the request initiation timestamp; according to the request timestamp of each pairing request, the corresponding frequency band is allocated to each device to be matched in sequence.

[0209] In one embodiment, the media synchronization playback device further includes a master-slave switching module;

[0210] The master-slave switching module is used to receive multiple master heartbeat data frames sequentially fed back by the master device, and determine the master data information of each master heartbeat data frame; based on the master data information, if the master device meets the master-slave switching conditions, the module performs master-slave switching between the master device and each slave device.

[0211] In one embodiment, the method of determining whether the master device meets the master-slave switching conditions based on each master data information includes at least one of the following: when the sorting method of the data sequence number of each master data information is a jump arrangement, the master device is determined to meet the master-slave switching conditions based on each master data information; when the reception time difference between the reception timestamps of each master data information is greater than the time difference threshold, the master device is determined to meet the master-slave switching conditions based on each master data information.

[0212] In one embodiment, the data processing module is further configured to receive, for each slave device, a plurality of slave heartbeat data frames sequentially fed back by the slave devices, and determine the slave data information of each slave heartbeat data frame.

[0213] The master-slave switching module is specifically used to select the slave devices whose data sequence numbers are sorted sequentially as candidate slave devices; select the candidate slave device with the smallest reception time difference between the reception timestamps of each slave data information as the target slave device; switch the master device to the slave device, switch the target slave device to the updated master device, and broadcast the device address of the updated master device as the updated master device address.

[0214] In one embodiment, such as Figure 23 As shown, another multimedia synchronous playback device is provided. This device is used in a slave device, which wirelessly connects to the master device via an adapter. The adapter is connected to a data source device. The multimedia synchronous playback device includes: a data receiving module 2302, a data parsing module 2304, a data receiving module 2306, and a synchronous playback module 2308, wherein:

[0215] The data receiving module 2302 is used to receive multiple consecutive multimedia data frames of the target multimedia sent sequentially by the adapter device; the target multimedia is played on the data source device; the multimedia data frame includes content data and an independent timestamp corresponding to the content data;

[0216] The data parsing module 2304 is used to parse each multimedia data frame according to the receiving order to determine the playback progress of the target multimedia; the playback progress is used to characterize the independent timestamp corresponding to the multimedia data frame parsed by the slave device.

[0217] The data receiving module 2306 is also used to obtain the main playback progress of the target multimedia played by the main device; the main playback progress is used to characterize the independent timestamps corresponding to the multimedia data frames parsed by the main device;

[0218] The synchronous playback module 2308 is used to adjust the playback progress of the slave device according to the difference between the playback progress of the slave device and the main playback progress, so that the main device and the slave device can play the target multimedia synchronously.

[0219] In one embodiment, the synchronous playback module is specifically used to determine the progress time difference between the independent timestamp representing the playback progress and the independent timestamp representing the main playback progress; when the progress time difference is within the first time difference range formed by the minimum offset time difference and the maximum offset time difference, the independent timestamp representing the playback progress is determined as the target timestamp, and the content data corresponding to the target timestamp is played.

[0220] In one embodiment, the synchronous playback module is further configured to: accelerate playback of the secondary playback progress when the progress time difference is within a second time difference range consisting of the minimum time difference and the minimum offset time difference; discard the content data corresponding to the independent timestamp represented by the secondary playback progress when the progress time difference is less than the minimum time difference or greater than the maximum time difference, and play the next frame of content data of the discarded content data; and update the main playback progress based on a preset duration when the progress time difference is within a third time difference range consisting of the maximum offset time difference and the maximum time difference, and adjust the secondary playback progress based on the playback progress difference between the updated main playback progress and the secondary playback progress.

[0221] In one embodiment, the data receiving module is specifically used to acquire synchronous data frames including the main playback progress of the target multimedia played by the main device, and to parse the synchronous data frames to obtain the main playback progress and verification data.

[0222] The synchronized playback module is specifically used to adjust the playback progress of the secondary playback based on the difference between the secondary playback progress and the main playback progress when the verification result of the verification data indicates that the verification is successful.

[0223] In one embodiment, the media synchronization playback device further includes a verification module;

[0224] The verification data includes the first pairing code of the master device;

[0225] The data parsing module is also used to determine the second pairing code during the process of establishing wireless communication with the master device;

[0226] The verification module is used to perform device pairing verification on the first pairing code and the second pairing code to obtain the pairing verification result; if the pairing verification result indicates that the slave device and the master device are wirelessly connected through the same adapter, the verification result of the verification data indicates that the verification is successful.

[0227] In one embodiment, the verification data includes the address of the device to be verified from the master device;

[0228] The data receiving module is also used to obtain the master device address determined during the master-slave device configuration process;

[0229] The verification module is also used to determine the verification result of the verification data to indicate successful verification when the address matching result indicates that the address of the device to be verified is consistent with the address of the master device.

[0230] In one embodiment, a computer device is provided. This embodiment uses the computer device as a terminal as an example for description, and its internal structure diagram can be as follows: Figure 24As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method for synchronous multimedia playback. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0231] Those skilled in the art will understand that Figure 24 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0232] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0233] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0234] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0235] It should be noted that the multimedia synchronous playback (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0236] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, etc., and are not limited to these.

[0237] The technical features in the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0238] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for synchronous playback of multimedia, characterized in that, The method is applied to an adapter device, which is connected to a data source device and provides a wireless communication connection between the master device and the slave device; the method includes: The multimedia data of the target multimedia is obtained from the data source device, and the multimedia data is parsed to obtain multiple consecutive content data, and the parsing order between each content data is determined. Determine the independent timestamp corresponding to each of the content data, and based on the parsing order, encapsulate the multiple content data and the independent timestamp corresponding to each of the content data into a multimedia data frame corresponding to each of the content data. Each multimedia data frame is sequentially sent to the master device and the slave device, so that the slave device determines the secondary playback progress of the target multimedia based on the multiple multimedia data frames received in sequence, and adjusts the secondary playback progress according to the playback progress difference between the secondary playback progress and the main playback progress of the target multimedia played by the master device, so that the master device and the slave device play the target multimedia synchronously.

2. The method according to claim 1, characterized in that, The method further includes: In response to the pairing requests of multiple devices to be paired, a corresponding frequency band is allocated to each device to be paired, so that wireless communication connectivity is provided between each device to be paired through the allocated frequency band; The device to be matched that is assigned a frequency band that is a preset frequency band is designated as the master device, and the device to be matched that is assigned a frequency band that is not a preset frequency band is designated as the slave device. The master device's address is broadcast so that the master device carries the device address to be verified when sending the main playback progress, and the slave device performs device address matching on the device address to be verified based on the master device's address.

3. The method according to claim 2, characterized in that, The step of responding to pairing requests from multiple devices to be paired, and allocating a corresponding frequency band for each device to be paired, includes: In response to pairing requests from multiple devices to be paired, a request timestamp for each pairing request is determined; the request timestamp is either a request receipt timestamp or a request initiation timestamp. According to the request timestamp of each pairing request, the corresponding frequency band is assigned to each device to be matched in sequence.

4. The method according to claim 1, characterized in that, The method further includes: Receive multiple master heartbeat data frames sequentially fed back by the master device, and determine the master data information of each master heartbeat data frame; If the master device meets the master-slave switching conditions based on the master data information, the master device and each of the slave devices are converted into master-slave devices.

5. The method according to claim 4, characterized in that, The method for determining whether the master device meets the master-slave switching conditions based on the master data information includes at least one of the following: If the sorting method of the data sequence numbers of each master data information is a jump arrangement, it is determined that the master device meets the master-slave switching conditions; If the time difference between the received timestamps of each master data information is greater than the time difference threshold, the master device is determined to meet the master-slave switching conditions.

6. The method according to claim 5, characterized in that, The method further includes: For each of the slave devices, multiple slave heartbeat data frames are received sequentially from the slave devices, and slave data information for each slave heartbeat data frame is determined. Master-slave conversion between the master device and each of the slave devices includes: Select the slave devices whose data sequence numbers are sorted sequentially as candidate slave devices; The candidate slave device with the smallest reception time difference between the reception timestamps of each of the aforementioned data information is selected as the target slave device; The master device is switched to a slave device, and the target slave device is switched to the updated master device. The device address of the updated master device is then broadcast as the updated master device address.

7. A method for synchronous playback of multimedia, characterized in that, The method is applied to a slave device, which wirelessly connects to a master device via an adapter device, and the adapter device is connected to a data source device; the method includes: The system receives a series of consecutive multimedia data frames of a target multimedia device, which are sequentially sent by the adapter device. The target multimedia device plays the multimedia. Each multimedia data frame includes content data and an independent timestamp corresponding to the content data. Each multimedia data frame is parsed according to the receiving order to determine the playback progress of the target multimedia; the playback progress is used to characterize the independent timestamp corresponding to the multimedia data frame parsed by the slave device. The main playback progress of the target multimedia is obtained by the main device; the main playback progress is used to characterize the independent timestamps corresponding to the multimedia data frames parsed by the main device. The playback progress is adjusted according to the difference between the secondary playback progress and the primary playback progress, so that the primary device and the secondary device can play the target multimedia synchronously.

8. The method according to claim 7, characterized in that, The step of adjusting the secondary playback progress according to the playback progress difference between the secondary playback progress and the primary playback progress includes: Determine the progress time difference between the independent timestamp representing the playback progress and the independent timestamp representing the main playback progress; When the progress time difference is within the first time difference range formed by the minimum offset time difference and the maximum offset time difference, the independent timestamp represented by the playback progress is determined as the target timestamp, and the content data corresponding to the target timestamp is played.

9. The method according to claim 8, characterized in that, The method further includes: When the progress time difference is within the second time difference range formed by the minimum time difference and the minimum offset time difference, the playback progress is accelerated. In either case where the progress time difference is less than the minimum time difference or the progress time difference is greater than the maximum time difference, discard the content data corresponding to the independent timestamp represented by the playback progress, and play the next frame of content data of the discarded content data. When the progress time difference is within the third time difference range formed by the maximum offset time difference and the maximum time difference, the main playback progress is updated based on a preset duration, and the secondary playback progress is adjusted based on the playback progress difference between the updated main playback progress and the secondary playback progress.

10. The method according to claim 7, characterized in that, The step of obtaining the main playback progress of the target multimedia played on the main device includes: Acquire synchronization data frames including the main playback progress of the target multimedia played by the main device, and parse the synchronization data frames to obtain the main playback progress and verification data; The step of adjusting the secondary playback progress according to the playback progress difference between the secondary playback progress and the primary playback progress includes: If the verification result of the verification data indicates successful verification, the secondary playback progress is adjusted according to the playback progress difference between the secondary playback progress and the primary playback progress.

11. The method according to claim 10, characterized in that, The verification data includes the first pairing code of the master device; The method further includes: During the process of establishing wireless communication with the master device, a second pairing code is determined; Perform device pairing verification on the first pairing code and the second pairing code to obtain the pairing verification result; If the pairing verification result indicates that the slave device and the master device are wirelessly connected through the same adapter, the verification result of the verification data indicates that the verification was successful.

12. The method according to claim 10, characterized in that, The verification data includes the address of the device to be verified of the master device; The method further includes: Obtain the master device address determined during the master-slave device configuration process; The master device address and the device address to be verified are matched to obtain the address matching result. If the address matching result indicates that the address of the device to be verified is consistent with the address of the master device, the verification result of the verification data is determined to indicate that the verification was successful.

13. A multimedia synchronous playback device, characterized in that, The device is applied to an adapter device, the adapter device is connected to a data source device, and the adapter device provides a wireless communication connection between the master device and the slave device; the device includes: The data processing module is used to acquire multimedia data of the target multimedia from the data source device, parse the multimedia data to obtain multiple consecutive content data, and determine the parsing order between each content data. The data encapsulation module is used to determine the independent timestamp corresponding to each of the content data, and encapsulate the plurality of content data and the independent timestamp corresponding to each of the content data into a multimedia data frame corresponding to each of the content data according to the parsing order. The data transmission module is used to sequentially send each of the multimedia data frames to the master device and the slave device, so that the slave device determines the secondary playback progress of the target multimedia based on the multiple multimedia data frames received in sequence, and adjusts the secondary playback progress according to the playback progress difference between the secondary playback progress and the main playback progress of the target multimedia played by the master device, so that the master device and the slave device play the target multimedia synchronously.

14. A multimedia synchronous playback device, characterized in that, The device is applied to a slave device, which wirelessly connects to a master device via an adapter, and the adapter is connected to a data source device; the device includes: A data receiving module is used to receive a series of consecutive multimedia data frames of a target multimedia device sent sequentially by the adapter device; the target multimedia device is played on the data source device; the multimedia data frame includes content data and an independent timestamp corresponding to the content data; The data parsing module is used to parse each multimedia data frame according to the receiving order to determine the playback progress of the target multimedia; the playback progress is used to characterize the independent timestamp corresponding to the multimedia data frame parsed by the slave device. The data receiving module is further configured to obtain the main playback progress of the target multimedia played by the main device; the main playback progress is used to characterize the independent timestamps corresponding to the multimedia data frames parsed by the main device; The synchronous playback module is used to adjust the secondary playback progress according to the playback progress difference between the secondary playback progress and the primary playback progress, so that the primary device and the secondary device can synchronously play the target multimedia.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 12.

Images