Screen projection method and device and electronic equipment
By dynamically switching between audio and video clocks for frame synchronization, the method addresses playback delays in screen projection by ensuring accurate synchronization, particularly when audio data is absent in subsequent frames.
Patent Information
- Application Number
- CN202410578359.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-10
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-05-10
AI Technical Summary
Existing screen projection methods face delays due to the inability to perform frame synchronization when audio data is absent after the first frame, leading to playback issues.
The method involves using an audio clock for frame synchronization if the first frame includes both video and audio data, and switching to a video clock for subsequent frames with only video data, ensuring accurate synchronization and reducing playback delays.
This approach allows for accurate frame synchronization and reduces playback delays by dynamically adapting the synchronization clock based on the presence of audio and video data, ensuring smooth playback.
Smart Images

Figure CN119071551A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of screen projection technology, and in particular to a screen projection method, device and electronic device. Background Art
[0002] When a PC or mobile phone is casting a screen, if the first frame of data contains video data and audio data when the sender (PC or mobile phone) starts casting, and there is no audio data in the data of the frames after the first frame, only video data, then when the receiver (TV) receives the frame data sent by the sender, since there is audio data in the data of the first frame, the receiver uses the clock of the audio data for frame synchronization. In this case, since there is no audio data in the data of the frames after the first frame, the clock of the audio data cannot be obtained, and thus frame synchronization cannot be performed, resulting in a large delay in the casting process. Summary of the invention
[0003] In order to solve the above technical problems, the present disclosure provides a screen projection method, device and electronic device.
[0004] The technical solution of the present disclosure is as follows:
[0005] In a first aspect, the present disclosure provides a display device, comprising: a processor, configured to control a communicator to obtain projection data of a multimedia file to be projected in response to a projection operation; the processor, further configured to parse the projection data obtained by the communicator, and if the frame data of the first frame in the projection data includes video data and audio data, use an audio clock corresponding to the audio data as a main clock for frame synchronization; the processor, further configured to change the main clock from the audio clock to a video clock corresponding to the video data, and perform frame synchronization according to the video clock, if it is determined that the frame data of frames other than the first frame only includes video data; the processor, further configured to perform frame synchronization according to a target clock, and control the display to play the projection data; wherein the target clock includes any one of an audio clock and a video clock.
[0006] In a second aspect, the present disclosure provides a screen projection method, comprising: in response to a screen projection operation, obtaining projection data of a multimedia file to be projected; parsing the projection data, and if the frame data of the first frame in the projection data includes video data and audio data, using an audio clock corresponding to the audio data as a main clock for frame synchronization; if it is determined that the frame data of frames other than the first frame only include video data, changing the main clock from the audio clock to a video clock corresponding to the video data, and performing frame synchronization according to the video clock; performing frame synchronization according to a target clock, and playing the projection data; wherein the target clock includes any one of an audio clock and a video clock.
[0007] In a third aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, the memory being used to store a computer program; the processor being used to enable the electronic device to implement any one of the screen projection methods provided in the first aspect when executing the computer program.
[0008] In a fourth aspect, the present disclosure provides a computer-readable storage medium, comprising: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a computing device, the computing device implements any screen projection method provided in the first aspect above.
[0009] In a fifth aspect, the present invention provides a computer program product, which, when executed on a computer, enables the computer to execute any one of the screen projection methods provided in the first aspect.
[0010] It should be noted that the above-mentioned computer instructions can be stored in whole or in part on the first computer-readable storage medium. Among them, the first computer-readable storage medium can be packaged together with the processor of the projection device, or it can be packaged separately from the processor of the projection device, and this disclosure does not limit this.
[0011] The description of the second, third, fourth and fifth aspects in the present disclosure can refer to the detailed description of the first aspect; and the beneficial effects of the description of the second, third, fourth and fifth aspects can refer to the beneficial effect analysis of the first aspect, which will not be repeated here.
[0012] In the present disclosure, the name of the above-mentioned projection device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of the present disclosure, they belong to the scope of the present disclosure and its equivalent technologies.
[0013] These and other aspects of the present disclosure will become more apparent from the following description.
[0014] Compared with the prior art, the technical solution provided by the present invention has the following advantages:
[0015] It can be seen that the screen projection method provided by the present disclosure obtains the screen projection data of the multimedia file to be projected in response to the screen projection operation; the screen projection data is parsed, and if the frame data of the first frame in the screen projection data includes video data and audio data, the audio clock corresponding to the audio data is used as the main clock for frame synchronization; if it is determined that the frame data of other frames except the first frame only includes video data, the main clock is changed from the audio clock to the video clock corresponding to the video data, and frame synchronization is performed according to the video clock. This can avoid the inability to perform preset operations on video frames when there is no audio or video, so that frame synchronization can be performed accurately and the playback delay can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0018] Figure 1 One of the flowcharts of the screen projection method provided in the embodiment of the present application;
[0019] Figure 2 One of the structural schematic diagrams of the display device in the screen projection method provided in the embodiment of the present application;
[0020] Figure 3 The second structural schematic diagram of the display device in the screen projection method provided in the embodiment of the present application;
[0021] Figure 4 One of the flowcharts of the screen projection method provided in the embodiment of the present application;
[0022] Figure 5 The second flowchart of the screen projection method provided in the embodiment of the present application;
[0023] Figure 6 The third flowchart of the screen projection method provided in the embodiment of the present application;
[0024] Figure 7 A fourth flowchart of the screen projection method provided in an embodiment of the present application;
[0025] Figure 8 A fifth flowchart of the screen projection method provided in an embodiment of the present application;
[0026] Fig. 9 A schematic diagram of the structure of a display device provided in an embodiment of the present application;
[0027] Fig.10 A schematic diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0030] The display device provided in the embodiments of the present application may have various implementation forms, for example, it may be a television, a smart TV, a laser projection device, a monitor, an electronic whiteboard, an electronic table, etc. Figure 1 and Figure 2 This is a specific implementation of the display device of the present application.
[0031] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0032] Figure 1 is a schematic diagram of an operation scenario between a display device and a control device according to one or more embodiments of the present application, such as Figure 1 As shown, the user can operate the display device 200 through the mobile terminal 300 and the control device 100. The control device 100 can be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication, Bluetooth protocol communication, wireless or other wired methods to control the display device 200. The user can input user instructions through buttons on the remote controller, voice input, control panel input, etc. to control the display device 200. In some embodiments, a mobile terminal, a tablet computer, a computer, a laptop computer, and other smart devices can also be used to control the display device 200.
[0033] In some embodiments, the electronic device provided by the embodiment of the present application may be the above-mentioned display device 200. Among them, when the user needs to cast the screen on the display device 200, the content to be cast can be cast to the display device 200 through electronic devices such as mobile phones. At this time, the display device 200 responds to the screen casting operation and obtains the screen casting data of the multimedia file to be cast; the display device 200 parses the screen casting data, and if the frame data of the first frame in the screen casting data includes video data and audio data, the audio clock corresponding to the audio data is used as the main clock for frame synchronization; if the display device 200 determines that the frame data of other frames except the first frame only contains video data, the main clock is changed from the audio clock to the video clock corresponding to the video data, and frame synchronization is performed according to the video clock; the display device 200 performs frame synchronization according to the target clock and plays the screen casting data. In this way, the screen casting is completed.
[0034] It can be seen that the screen projection method provided by the embodiment of the present disclosure, because the display device 200 usually uses the audio clock as the main clock for frame synchronization during the process of playing the projection data, and when the frame data of other frames except the first frame only contains video data, the corresponding audio clock cannot be obtained at this time, so that the preset operation cannot be performed on the video frame. For example, when the preset operation is a frame drop operation, it is necessary to parse and play each video frame, which will increase the time for playing the video frame. Since the time for playing the video frame is longer, the progress of the playback of the projection end (such as a mobile phone or other electronic device) is different from the progress of the playback of the display device 200, resulting in a playback delay. To this end, the screen projection method provided by the embodiment of the present disclosure, in response to the screen projection operation, obtains the projection data of the multimedia file to be projected; parses the projection data, and if the frame data of the first frame in the projection data includes video data and audio data, the audio clock corresponding to the audio data is used as the main clock for frame synchronization; if it is determined that the frame data of other frames except the first frame only contains video data, the main clock is changed from the audio clock to the video clock corresponding to the video data, and frame synchronization is performed according to the video clock. This can avoid the situation where the preset operation of the video frame cannot be performed when there is no audio or video, so that frame synchronization can be performed accurately and the playback delay can be reduced.
[0035] Figure 2 FIG. 2 shows a hardware configuration block diagram of a display device 200 according to an exemplary embodiment. Figure 2The display device 200 shown includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface 280. The controller includes a central processing unit, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output. The display 260 can be a display with a touch function, such as a touch display. The tuner 210 receives broadcast television signals by wired or wireless reception, and demodulates audio and video signals from multiple wireless or wired broadcast television signals, such as and EPG data signals. The detector 230 is used to collect signals of the external environment or interact with the outside. The controller 250 and the tuner 210 can be located in different split devices, that is, the tuner 210 can also be in an external device of the main device where the controller 250 is located, such as an external set-top box.
[0036] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.
[0037] In some examples, the display device 200 of one or more embodiments is a TV set 1, and the operating system of the TV set 1 is an Android system. Figure 3 As shown, the television 1 can be logically divided into an application layer (Applications) (referred to as "application layer") 21, an application framework layer (Application Framework) (referred to as "framework layer") 22, an Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer") 23, and a kernel layer 24.
[0038] Among them, the application layer 21 includes one or more applications. The application can be a system application or a third-party application. For example, the application layer 21 includes a first application, and the first application can provide a screen projection function. The framework layer 22 provides an application programming interface (API) and a programming framework for the application of the application layer 21. The system runtime layer 23 provides support for the upper layer, namely the framework layer 22. When the framework layer 22 is used, the Android operating system will run the C / C++ library contained in the system runtime layer 23 to implement the functions to be implemented by the framework layer 22. The kernel layer 24 serves as a software middleware between the hardware layer and the application layer 21, and is used to manage and control hardware and software resources.
[0039] In some examples, taking mircast as an example, after TV 1 is turned on, when the user is using TV 1, if the user needs to project the display content on other devices (such as the so-called projection terminal) to TV 1 for display, the user can select the control corresponding to the first application provided by TV 1, and the processing unit 201 of TV 1 responds to the selection operation of the target control for providing the projection function, creates a player (such as: the processing unit 201 creates a player in the Java interface layer of the application layer 21, and the process of creating the player includes: creating a surface, setMediaDataSource (such as setting the projection data sent by the projection terminal to be stored in a buffer, and setting the player to read the projection data in the buffer), setting low latency attributes (such as turning off motion compensation, image quality processing, etc.), and creating an audio and video in the local framework (native layer) of the application layer 21 The audio and video processing module is used to read the projection data in the buffer and input the projection data to the demultiplexer (DEMUX) for demultiplexing to obtain the first data. The audio and video processing module is also used to input the first data to the decoder (decoder) for decoding to obtain the second data (such as Medciacode). Afterwards, the audio and video processing module is also used to perform image recognition on the second data to obtain the first processing result of the first frame image of the current frame; at the same time, the audio and video processing module is also used to store the second data in the specified position of the framework layer 22. The synchronous rendering module is used to read the second data stored in the specified position (such as the ES cache module), and perform audio and video synchronization (Avsync) on the second data to obtain the third data. The synchronous rendering module is also used to render (Render) the third data to obtain the projection image). After creating the player and establishing a communication connection with the projection end, the processing unit 201 of the TV 1 controls the acquisition unit 202 to acquire the projection data of the multimedia file to be projected; the processing unit 201 of the TV 1 parses the projection data acquired by the acquisition unit 202, and if the frame data of the first frame in the projection data includes video data and audio data, the audio clock corresponding to the audio data is used as the main clock for frame synchronization; if the processing unit 201 of the TV 1 determines that the frame data of other frames except the first frame only includes video data, the main clock is changed from the audio clock to the video clock corresponding to the video data, and frame synchronization is performed according to the video clock; the processing unit 201 of the TV 1 performs frame synchronization according to the target clock, and controls the display unit 203 to play the projection data.
[0040] Specifically, the storage unit 204 of the television 1 is used to store data such as a computer program of the screen projection method provided in the embodiment of the present disclosure.
[0041] In the following embodiments, the method of the embodiments of the present application is explained by taking the case where the execution subject of the screen projection method provided by the embodiments of the present disclosure is the above-mentioned display device 200, and the display device 200 is the TV 1 as an example.
[0042] The present application embodiment provides a screen projection method, such as Figure 4 As shown, the screen projection method may include S11-S14.
[0043] S11. In response to the screen projection operation, obtain the projection data of the multimedia file to be projected.
[0044] S12. Analyze the projection data. If the frame data of the first frame in the projection data includes video data and audio data, use the audio clock corresponding to the audio data as the main clock for frame synchronization.
[0045] S13: If it is determined that the frame data of the frames other than the first frame only include video data, the main clock is changed from the audio clock to the video clock corresponding to the video data, and frame synchronization is performed according to the video clock.
[0046] S14. Perform frame synchronization according to the target clock and play the projection data; wherein the target clock includes any one of the audio clock and the video clock.
[0047] In some examples, the mobile phone and the TV control and transmit the video stream through the Real Time Streaming Protocol (RTSP). When the TV receives the screen projection data packet sent by the mobile phone, it creates a player, sets low latency attributes when configuring the player, creates a read thread, and reads the data in real time. After reading the data, it performs demux processing and parses the information in the stream, such as the video-related format of the video (width, height, frame rate, video encoding format) and the audio-related format of the audio (number of channels, audio format, etc.), and then creates audio and video decoders according to the relevant video and audio formats. At the same time, the TV determines that the first frame of the received screen projection data packet contains both audio and video data, and the audio clock is used as the main clock, otherwise the video clock is used as the main clock. Otherwise, it switches to 2channel and creates a thread to detect the type of data stream received every 5 seconds. If it is found that there is only video data but no audio data at this time, the clock during synchronization is changed to the video clock. If there is both video and audio, the synchronization clock is dynamically updated to the audio clock again. It can be seen that the screen projection method provided by the embodiment of the present disclosure obtains the screen projection data of the multimedia file to be projected in response to the screen projection operation; the screen projection data is parsed, and if the frame data of the first frame in the screen projection data includes video data and audio data, the audio clock corresponding to the audio data is used as the main clock for frame synchronization; if it is determined that the frame data of other frames except the first frame only includes video data, the main clock is changed from the audio clock to the video clock corresponding to the video data, and frame synchronization is performed according to the video clock. This can avoid the situation where the preset operation cannot be performed on the video frame when there is no audio or video, so that frame synchronization can be performed accurately and the playback delay can be reduced.
[0048] In some possible implementation examples, combined with Figure 4 ,like Figure 5 As shown, the screen projection method provided by the embodiment of the present disclosure also includes S15.
[0049] S15: If it is determined that the frame data of the frames other than the first frame include video data and audio data, the main clock is changed from the video clock to the audio clock corresponding to the audio data, and frame synchronization is performed according to the audio clock.
[0050] In some examples, when the TV 1 is playing the projection data, part of the frame data may contain video data and audio data, and part of the frame data may contain only video data. To this end, in the projection method provided by the embodiment of the present disclosure, when the main clock is an audio clock, if the received frame data only contains video data, the main clock is changed from the audio clock to the video clock corresponding to the video data. When the main clock is a video clock, if the received frame data contains video data and audio data, the main clock is changed from the video clock to the audio clock corresponding to the audio data. In this way, the main clock can be accurately acquired.
[0051] In some possible implementation examples, combined with Figure 4 ,like Figure 6 As shown, the above S14 can be specifically implemented through the following S140-S143.
[0052] S140, perform frame synchronization according to the target clock and obtain the parsed projection data.
[0053] S141. When rendering the parsed projection data, obtain the total number of video frames that are not currently decoded and rendered.
[0054] S142: Based on the total number, obtain the display duration of the video frame.
[0055] S143. Play the video frames in the projection data according to the display duration.
[0056] In some examples, when the display duration of a video frame is obtained based on the total number, it can be determined whether to adjust the display duration of the video frame based on the size relationship between the total number and the preset threshold value, such as: when the preset threshold value is 1, the TV 1 determines that the total number is greater than 1, and adjusts the display duration of the video frame from the first duration to the second duration, and uses the second duration as the display duration of the video frame in the next frame. Since the total number of video frames that are not currently decoded and rendered is greater than the preset threshold, it means that part of the screen projection data sent by the screen projection end (such as a mobile phone) has not been displayed in time. Therefore, the TV 1 can adjust the decoding and rendering time of the video frame by adjusting the display duration of the video frame in the received screen projection data, such as: when the TV 1 determines that the total number is greater than 1, the display duration of the video frame is adjusted from the first duration to the second duration. Since the second duration is less than the first duration, the decoding speed and rendering speed of the video frame can be accelerated, thereby avoiding the problem of delay.
[0057] In some examples, when obtaining the display duration of a video frame based on the total number, the difference between the current display duration of the video frame and the ratio of the total number and 1000 can be calculated, and the difference is used as the display duration of the video frame in the next frame, that is, the display duration of the video frame in the next frame is equal to t represents the current display duration of the video frame, and A represents the total number.
[0058] In some examples, when obtaining the display duration of the video frame based on the total number, the duration relationship table can be queried based on the total number to determine the preset duration corresponding to the total number. Afterwards, the preset duration is used as the display duration of the frame, that is, the display duration of the video frame is adjusted from the first duration to the preset duration, and the preset duration is used as the display duration of the video frame in the next frame.
[0059] Exemplarily, the duration relationship table is shown in Table 1.
[0060] Table 1
[0061] total Preset duration 1 30ms 2 25ms 3 20ms 4 5ms
[0062] Typically, the display duration of a frame is 33.33ms. In order to prevent the problem of slow decoding due to an old underlying chip without low latency, the display method provided in the embodiment of the present disclosure dynamically configures the display duration of the frame so that the decoded YUV data can be rendered as quickly as possible and the YUV buffer can be poured out, so that the frame data cached in the es cache module can be sent to the decoder for decoding and rendering as quickly as possible to reduce latency.
[0063] For example, when the total is 1, it can be seen from Table 1 that the preset duration is 30ms, and the display duration of the frame before adjustment is 33.33ms. Therefore, the display duration of the video frame can be shortened to reduce the delay.
[0064] In some examples, when obtaining the display duration of a video frame based on the total number, the difference between the current display duration of the video frame and the ratio of the total number and 1000 can be calculated, and the difference can be used as the target duration; at the same time, the duration relationship table can be queried based on the total number to determine the preset duration corresponding to the total number; at the same time. When it is determined that the total number is greater than 1, the display duration of the video frame is adjusted from the first duration to the second duration. Thereafter, the average of the target duration, the preset duration, and the second duration is used as the display duration of the video frame in the next frame. Alternatively, the minimum value of the target duration, the preset duration, and the second duration is used as the display duration of the video frame in the next frame.
[0065] In some examples, the first duration is an initial display duration of the video frame, such as 33.33 ms.
[0066] In some possible implementation examples, combined with Figure 6 ,like Figure 7 As shown, the above S142 can be specifically implemented through the following S1420.
[0067] S1420: When the total number is greater than 1, adjust the display duration of the video frame from the first duration to a second duration; wherein the second duration is smaller than the first duration.
[0068] In some possible implementation examples, combined with Figure 7 ,like Figure 8 As shown, the screen projection method provided by the embodiment of the present disclosure also includes S16.
[0069] S16. After adjusting the display duration of the video frame from the first duration to the second duration, if it is determined that the total is less than or equal to 1, adjusting the display duration of the video frame from the second duration to the first duration.
[0070] In some examples, when rendering after video decoding, the number of video es buffers received at this time is detected in real time. If the number of video es buffers is greater than 1, it means that the underlying decoding is too slow. For example, if the underlying chip is old and has no low latency, the decoding will be too slow and the es data will be cached. At this time, the decoded YUV data is rendered as soon as possible, and the YUVbuffer is poured out to send the video frames stored in the ES cache module to the decoder and render as soon as possible to reduce latency.
[0071] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.
[0072] The embodiment of the present application can divide the functional modules of the projection device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.
[0073] like Fig. 9 As shown, an embodiment of the present application provides a structural diagram of a display device 200 , which includes a processor 101 , a communicator 102 , and a display 103 .
[0074] The processor is configured to control the communicator to obtain the projection data of the multimedia file to be projected in response to the projection operation; the processor is also configured to parse the projection data obtained by the communicator, and if the frame data of the first frame in the projection data includes video data and audio data, use the audio clock corresponding to the audio data as the main clock for frame synchronization; the processor is also configured to change the main clock from the audio clock to the video clock corresponding to the video data if it is determined that the frame data of other frames except the first frame only includes video data, and perform frame synchronization according to the video clock; the processor is also configured to perform frame synchronization according to the target clock and control the display to play the projection data; wherein the target clock includes any one of the audio clock and the video clock.
[0075] In some feasible examples, the processor is further configured to change the main clock from the video clock to the audio clock corresponding to the audio data if it is determined that the frame data of frames other than the first frame contains video data and audio data, and perform frame synchronization according to the audio clock.
[0076] In some feasible examples, the processor is further configured to perform frame synchronization according to the target clock and control the communicator to obtain the parsed projection data; the processor is further configured to obtain the total number of video frames that are not currently decoded and rendered when rendering the parsed projection data obtained by the communicator; the processor is further configured to obtain the display duration of the video frame based on the total number; the processor is further configured to control the display to play the video frames in the projection data according to the display duration.
[0077] In some feasible examples, the processor is further configured to adjust the display duration of the video frame from a first duration to a second duration when the total is greater than 1; wherein the second duration is smaller than the first duration.
[0078] In some feasible examples, the processor is further configured to, after adjusting the display duration of the video frame from the first duration to the second duration, if it is determined that the total is less than or equal to 1, adjust the display duration of the video frame from the second duration to the first duration.
[0079] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and its role will not be repeated here.
[0080] Of course, the display device 200 provided in the embodiment of the present application includes but is not limited to the above modules, for example, the display device 200 may also include a memory 104. The memory 104 may be used to store the program code of the display device 200, and may also be used to store data generated by the display device 200 during operation, such as data in a write request.
[0081] As an example, combining Figure 3 The function implemented by the acquisition unit 202 in the TV 1 is the same as the function of the communicator 102, the function implemented by the processing unit 201 is the same as the function of the processor 101, the function implemented by the display unit 203 is the same as the function of the display 103, and the function implemented by the storage unit 204 is the same as the function of the memory 104.
[0082] like Fig.10 As shown, the embodiment of the present application also provides a chip system, which can be applied to the display device 200 in the aforementioned embodiment. The chip system includes at least one processor 1501 and at least one interface circuit 1502. The processor 1501 can be the processor in the above-mentioned display device 200. The processor 1501 and the interface circuit 1502 can be interconnected through a line. The processor 1501 can receive and execute computer instructions from the memory of the above-mentioned display device 200 through the interface circuit 1502. When the computer instructions are executed by the processor 1501, the display device 200 can execute the various steps executed by the display device 200 in the above-mentioned embodiment. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiment of the present application.
[0083] The embodiment of the present application also provides a computer-readable storage medium for storing computer instructions executed by the above-mentioned display device 200.
[0084] The embodiment of the present application also provides a computer program product, including computer instructions executed by the above-mentioned display device 200.
[0085] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A display device, characterized in that: include: The processor is configured to control the communicator to obtain the projection data of the multimedia file to be projected in response to the projection operation; The processor is further configured to parse the projection data acquired by the communicator, and if the frame data of the first frame in the projection data includes video data and audio data, use the audio clock corresponding to the audio data as the main clock for frame synchronization; The processor is further configured to, if it is determined that the frame data of frames other than the first frame only include video data, change the master clock from the audio clock to a video clock corresponding to the video data, and perform frame synchronization according to the video clock; The processor is also configured to perform frame synchronization according to a target clock and control the display to play the projection data; wherein the target clock includes any one of the audio clock and the video clock.
2. The display device according to claim 1, characterized in that The processor is further configured to change the main clock from the video clock to an audio clock corresponding to the audio data if it is determined that the frame data of frames other than the first frame include video data and audio data, and perform frame synchronization according to the audio clock.
3. The display device according to claim 1, characterized in that The processor is further configured to perform frame synchronization according to the target clock and control the communicator to obtain the parsed projection data; The processor is further configured to obtain a total number of video frames that are not currently decoded and rendered when rendering the parsed projection data obtained by the communicator; The processor is further configured to obtain a display duration of the video frame based on the total number; The processor is further configured to control the display to play the video frames in the projection data according to the display duration.
4. The display device according to claim 3, characterized in that The processor is further configured to adjust the display duration of the video frame from a first duration to a second duration when the total is greater than 1; wherein the second duration is smaller than the first duration.
5. The display device according to claim 4, characterized in that The processor is further configured to, after adjusting the display duration of the video frame from the first duration to the second duration, if it is determined that the total is less than or equal to 1, adjust the display duration of the video frame from the second duration to the first duration.
6. A screen projection method, characterized in that: include: In response to the screen projection operation, obtaining projection data of the multimedia file to be projected; Parsing the projection data, if the frame data of the first frame in the projection data includes video data and audio data, using the audio clock corresponding to the audio data as the master clock for frame synchronization; If it is determined that the frame data of the frames other than the first frame only include video data, the master clock is changed from the audio clock to a video clock corresponding to the video data, and frame synchronization is performed according to the video clock; Frame synchronization is performed according to a target clock, and the projection data is played; wherein the target clock includes any one of the audio clock and the video clock.
7. The screen projection method according to claim 6, characterized in that: The method further comprises: If it is determined that the frame data of frames other than the first frame include video data and audio data, the master clock is changed from the video clock to an audio clock corresponding to the audio data, and frame synchronization is performed according to the audio clock.
8. The screen projection method according to claim 6, characterized in that: The performing frame synchronization according to the target clock and playing the screen projection data includes: Perform frame synchronization according to the target clock and obtain the parsed projection data; When rendering the parsed projection data, obtaining the total number of video frames that are not currently decoded and rendered; Based on the total number, obtaining the display duration of the video frame; Play the video frames in the projection data according to the display duration.
9. The screen projection method according to claim 8, characterized in that: The obtaining, based on the total number, a display duration of the video frame, comprises: When the total number is greater than 1, the display duration of the video frame is adjusted from the first duration to a second duration; wherein the second duration is smaller than the first duration.
10. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the screen projection method described in any one of claims 6-9.
Citation Information
Patent Citations
Audio and video synchronous playing method and device, terminal and storage medium
CN110324689A
Media audio and video synchronization method and system, and electronic equipment
CN115914708A
Screen projection method, screen projection system and electronic equipment
CN117956212A
Method for synchronizing video with audio in decoding system
CN1436001A
Synchronized, variable-speed playback of digitally recorded audio and video
US5583652A
