Signal processing method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to a signal processing method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: determining a current video source frame and a current display frame corresponding to the current video source frame; and modifying the timestamp of the vertical synchronization signal of the current display frame in response to the fact that the accumulated time error between the current video source frame and the current display frame reaches the target offset, so that the modified timestamp is aligned with the display moment of the corresponding video source frame. The method and the device can be applied to scenes such as an intelligent cabin, the modified timestamp can be aligned with the display moment of the corresponding video source frame by accurately modifying the timestamp of the vertical synchronizing signal of the current display frame, so that frame loss, smooth transition of the video and the difference of the display frame rate are avoided, the fluency and the stability of video playing are improved, and the user experience is improved. The visual experience of the user is improved, and the display performance of the target device is enhanced.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of computer graphics, and in particular, to a signal processing method and device, electronic equipment, storage medium and program product. BACKGROUND

[0002] Video playing is a high-frequency use scenario for users, and its smoothness directly affects the user experience. Currently, there is often an unavoidable deviation between the display frame rate of a device and the video source frame rate. With the passage of time, this deviation causes the time offset to accumulate continuously, and then causes a cumulative time gap to periodically appear when the application program performs vertical synchronization, so that the application program cannot output graphics on time, and finally causes the frame loss problem, affecting the smoothness of video playing. SUMMARY

[0003] To overcome the problems in the related art, the present disclosure provides a signal processing method and device, electronic equipment, storage medium and program product.

[0004] According to a first aspect of an embodiment of the present disclosure, a signal processing method is provided, determining a current video source frame and a current display frame corresponding to the current video source frame; in response to the accumulated time error between the current video source frame and the current display frame reaching a target offset, modifying the timestamp of the vertical synchronization signal of the current display frame to align the modified timestamp with the display time of the corresponding video source frame.

[0005] In the above technical solution, by determining the current video source frame and the corresponding display frame, and monitoring the accumulated time error between the two in real time, when the error reaches the target offset, the timestamp of the vertical synchronization signal of the current display frame is accurately modified, which can align the modified timestamp with the display time of the corresponding video source frame, thereby avoiding subsequent frame loss, smoothing the difference between the video and the display frame rate, improving the smoothness and stability of video playing, improving the visual experience of users, and enhancing the display performance of the target device.

[0006] In some possible implementation manners, the modifying the timestamp of the vertical synchronization signal of the current display frame in response to the accumulated time error between the current video source frame and the current display frame reaching a target offset comprises: in response to the accumulated time length since the target time reaching an overtime waiting time length, modifying the timestamp of the vertical synchronization signal of the current display frame; wherein the target time is a time when the timestamp of the display frame is aligned with the display time of the corresponding video source frame, and the overtime waiting time length is a corresponding interval time length when the accumulated time error between the video source frame and the display frame reaches the target offset since the target time.

[0007] In the technical solution, the target offset is used to determine the timeout waiting time length, and the timeout waiting time length is used to determine the modification time of the timestamp, so that the time node can be accurately controlled, the cumulative error can be avoided to cause abnormal playing, the synchronization of the subsequent video source frame and the display frame can be ensured, and the playing fluency and quality are improved In some possible implementations, the method further includes: determining a video source frame rate and a display frame rate; delaying or advancing the timestamp of the vertical synchronization signal of the current display frame by the target offset based on the video source frame rate and the display frame rate.

[0008] In the technical solution, the video source frame rate and the display frame rate are determined, and the timestamp of the vertical synchronization signal is flexibly adjusted by the target offset, so that the picture synchronization problem caused by the frame rate difference can be accurately solved, the video playing fluency can be ensured, and the user watching experience is improved.

[0009] In some possible implementations, the method further includes: if the video source frame rate is less than the display frame rate, delaying the timestamp of the vertical synchronization signal of the current display frame by the target offset to delay the timestamp of the subsequent vertical synchronization signal; if the video source frame rate is greater than the display frame rate, advancing the timestamp of the vertical synchronization signal of the current display frame by the target offset to advance the timestamp of the subsequent vertical synchronization signal.

[0010] In the technical solution, the timestamp correction direction (delay or advance) is dynamically adjusted according to the relative size of the video source frame rate and the display frame rate, so that the synchronization strategy can be accurately adapted to different frame rate scenes, and the display compatibility and user experience are improved.

[0011] In some possible implementations, the method further includes: determining a maximum value of a variable frame frequency of display hardware; determining the reciprocal of the maximum value of the variable frame frequency of the display hardware as the target offset.

[0012] In the technical solution, the timing adjustment can be effectively recognized and executed by the hardware, and the adjustment range can be accurately controlled within a reasonable range.

[0013] In some possible implementations, the timeout waiting time length is determined by the following method: determining a video source frame rate and a display frame rate; determining the timeout waiting time length according to the video source frame rate and the display frame rate.

[0014] In the technical solution, the timeout waiting duration can be accurately determined, and the timestamp can be accurately adjusted.

[0015] In some possible implementation, the determining the timeout waiting duration according to the video source frame rate and the display frame rate comprises: determining a first single-frame duration of the video source according to the video source frame rate; determining a second single-frame duration of the display frame according to the display frame rate; determining a frame length deviation of the first single-frame duration and the second single-frame duration; determining the timeout waiting duration according to a ratio of the target offset and the frame length deviation, and the second single-frame duration.

[0016] In the technical solution, the best intervention time can be accurately predicted, and resource waste or lag caused by early or late adjustment can be avoided.

[0017] According to a second aspect of the embodiments of the present disclosure, a signal processing apparatus is provided, and the signal processing apparatus comprises: a first determining module configured to determine a current video source frame and a current display frame corresponding to the current video source frame; an adjusting module configured to, in response to a cumulative time error between the current video source frame and the current display frame reaching a target offset, modify a timestamp of a vertical synchronization signal of the current display frame, so that the modified timestamp is aligned with a display time of a corresponding video source frame.

[0018] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, and the electronic device comprises: display hardware supporting a multiple tearing effect signal; a processor; a memory for storing processor executable instructions; The processor is configured to execute the executable instructions in the memory to implement the steps of the signal processing method provided in the first aspect of the present disclosure.

[0019] According to a fourth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the signal processing method provided in the first aspect of the present disclosure.

[0020] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, and the computer program product comprises a computer program, which, when executed by a processor, implements the steps of the signal processing method provided in the first aspect of the present disclosure.

[0021] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure, in which, like reference numerals designate corresponding parts throughout the several views.

[0023] Figure 1 is a flow chart of a signal processing method according to an exemplary embodiment.

[0024] Figure 2 is a diagram of vertical synchronization signal and application frame submission time before timestamp correction according to an exemplary embodiment.

[0025] Figure 3 is a diagram of vertical synchronization signal and application frame submission time after timestamp correction according to an exemplary embodiment.

[0026] Figure 4 is a diagram of application and Vsync (vertical synchronization signal) status before timestamp modification according to an exemplary embodiment.

[0027] Figure 5 is a diagram of application and Vsync status after timestamp modification according to an exemplary embodiment.

[0028] Figure 6 is a block diagram of a signal processing apparatus according to an exemplary embodiment.

[0029] Figure 7 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0030] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is merely exemplary and is not intended to limit the disclosure to only the disclosed embodiments. Rather, the disclosure is intended to embrace all alternatives, modifications, and variations in accordance with the disclosure as defined by the appended claims.

[0031] It should be noted that all the actions of obtaining signals, information or data in the present disclosure are carried out in accordance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization given by the owner of the corresponding device.

[0032] In the use of electronic devices, video playing is a high-frequency scenario, and its fluency is related to user experience. However, the frame rate of the device display and the frame rate of the video source often inevitably have deviations.

[0033] When the video is output at a standard 60FPS, the underlying often has difficulty in accurately implementing a refresh rate of 60Hz, and may brush the picture at 60.1Hz, which causes the display frame to be faster than the video source frame. The time taken to output 60 frames of video is 1s, and the time taken to display 60 frames is only 998ms, and a time gap is generated between the two. This gap will cause a 16ms cumulative gap every 8s, which makes the application unable to display the picture on time, and ultimately causes the frame loss phenomenon, affecting the fluency of video playing.

[0034] In order to solve the above problems, the present disclosure provides a signal processing method to improve the fluency of video playing and reduce the frame loss problem.

[0035] It should be noted that the signal processing method provided by the present disclosure is not only applicable to 60Hz video, but also applicable to any frame rate scenario, such as 24Hz, 30Hz, etc. The signal processing method provided by the present disclosure is applicable to scenarios where the video source frame rate and the underlying display frame rate are close, such as video source frame rate 30Hz and display frame rate 30.05Hz, etc. In other words, the example of 60Hz should not be understood as limiting the application field and protection scope of the present disclosure.

[0036] The signal processing method of the present disclosure can be applied to a target device, for example, the target device can be a smartphone, a smart tablet, a smart cockpit, or an electronic device for video playing. The display hardware (such as a display screen) of the target device supports multiple tearing effect signals (Multiple Tearing Effect, Multi-TE), that is, the graphics processing unit (GPU) is allowed to submit multiple frame data within a single screen refresh cycle. Therefore, the underlying Multi-TE hardware can ensure that the target device can receive variable-length frames without causing tearing.

[0037] Figure 1 is a flowchart of a signal processing method according to an exemplary embodiment. As shown in Figure 1 the method can include steps S101-S102.

[0038] In step S101, the current video source frame and the current display frame corresponding to the current video source frame are determined.

[0039] In step S102, in response to the cumulative time error between the current video source frame and the current display frame reaching a target offset, the timestamp of the vertical synchronization signal of the current display frame is modified to align the modified timestamp with the display time of the corresponding video source frame.

[0040] In an embodiment, the target offset can be pre-set based on actual requirements. Alternatively, the target offset can be determined by determining a maximum value of the variable frame rate of the display hardware, and determining the reciprocal of the maximum value of the variable frame rate of the display hardware as the target offset.

[0041] For example, the maximum value of the variable frame rate of the display hardware can be 360Hz, and the target offset can be determined to be about 2.777777ms. For the display hardware of 360Hz, 2.777777ms is the minimum vertical synchronization interval that can be responded to, and any smaller offset cannot be recognized and executed by the hardware. Therefore, determining the reciprocal of the maximum value of the variable frame rate of the display hardware as the target offset can ensure that the timing adjustment can be recognized and executed by the hardware, thereby achieving effective synchronization.

[0042] When the accumulated time error between the current video source frame and the current display frame reaches the target offset, it can be determined that the current video source frame and the corresponding display frame have a large offset, and the underlying Multi-TE hardware can support corresponding adjustment implementation. At this time, modifying the timestamp of the vertical synchronization signal of the current display frame can align the modified timestamp with the display time of the corresponding video source frame, thereby avoiding a large deviation between subsequent video source frames and corresponding display frames.

[0043] In the above technical solution, by determining the current video source frame and the corresponding display frame, and monitoring the accumulated time error between the two in real time, when the error reaches the target offset, the timestamp of the vertical synchronization signal of the current display frame is accurately modified, which can align the modified timestamp with the display time of the corresponding video source frame, thereby avoiding subsequent frame loss, smoothing the difference between the video and the display frame rate, improving the smoothness and stability of video playback, improving the user's visual experience, and enhancing the display performance of the target device.

[0044] In some possible implementations, in step S102, in response to the accumulated time error between the current video source frame and the current display frame reaching the target offset, the timestamp of the vertical synchronization signal of the current display frame is modified, including: In response to the accumulated duration since the target time reaching the timeout waiting duration, the timestamp of the vertical synchronization signal of the current display frame is modified.

[0045] The target time is a time when the timestamp of the display frame is aligned with the display time of the corresponding video source frame, and the timeout waiting duration is a corresponding interval duration when the accumulated time error of the video source frame and the display frame reaches the target offset since the target time.

[0046] In an embodiment, the target time can be the initial time of video playback.

[0047] For example, when the user clicks the play button, the current time can be recorded as the initial target time (e.g., 0 ms), and a timer can be started. When the cumulative duration since the target time reaches the timeout waiting duration, the timestamp of the vertical synchronization signal of the current display frame (i.e., the recorded vertical synchronization signal of the latest frame) can be modified, triggering the first modification of the timestamp to achieve the first synchronization correction.

[0048] In yet another embodiment, the target time can be the time corresponding to the last modified timestamp.

[0049] For example, after completing a synchronization correction, the current time can be re-timed, and the cumulative duration can be re-accumulated. When the cumulative duration again reaches the timeout waiting duration, the timestamp of the vertical synchronization signal of the current display frame can be modified to trigger synchronization correction again. Over time, periodic dynamic synchronization can be formed.

[0050] In this way, the initial time of video playback and / or the time corresponding to the last modified timestamp can be used as a reference to balance the startup stability and dynamic adaptability, avoid error accumulation, ensure synchronization accuracy, improve picture smoothness, and ensure display effect.

[0051] In an embodiment, the timestamp of the vertical synchronization signal of the next frame can be determined according to the timestamp of the vertical synchronization signal of the current frame and the display frame period, i.e., the delivery time.

[0052] In an embodiment, the timeout waiting duration can be determined by the following method: determine the video source frame rate and the display frame rate; determine the timeout waiting duration according to the video source frame rate and the display frame rate.

[0053] For example, the first single-frame duration of the video source can be determined according to the video source frame rate, the second single-frame duration of the display frame can be determined according to the display frame rate, and the frame length deviation of the first single-frame duration and the second single-frame duration can be determined. The timeout waiting duration can be determined according to the ratio of the target offset and the frame length deviation, and the second single-frame duration.

[0054] Wherein, the single-frame duration is the inverse of the corresponding frame rate. If the video source frame rate is 60 FPS, the first single-frame duration of the video source is approximately 16.66 ms; the display frame rate can be 60.1 Hz, and the second single-frame duration of the display frame is approximately 16.63 ms. The frame length deviation of the first single-frame duration and the second single-frame duration is 0.03 ms, the target offset is 2.777777 ms as given above, and the ratio of the target offset and the frame length deviation is approximately 92.59. The product of the ratio and the second single-frame duration (approximately 1539 ms) can be determined as the timeout waiting duration.

[0055] That is, the timeout waiting duration can be determined by the following formula :

[0056] wherein, is a target offset, is a frame length difference between the first single-frame length (which can be understood as a video source frame period) and the second single-frame length, is the second single-frame length, which can also be understood as a display frame period.

[0057] In this way, based on the frame rate difference, the optimal intervention opportunity can be accurately predicted, and resource waste or lag caused by too early or too late adjustment can be avoided.

[0058] In some possible implementations, the timestamp of the vertical synchronization signal of the current display frame can be modified in the following manner, including: determining a video source frame rate and a display frame rate; based on the video source frame rate and the display frame rate, delaying or advancing the timestamp of the vertical synchronization signal of the current display frame by a target offset.

[0059] In the above technical solution, by explicitly determining the video source and display frame rates, the timestamp of the vertical synchronization signal is flexibly adjusted by a target offset, which can accurately solve the problem of picture asynchronization caused by frame rate difference, ensure smooth video playback, and improve user viewing experience.

[0060] In an embodiment, if the video source frame rate is less than the display frame rate, the timestamp of the vertical synchronization signal of the current display frame is delayed by a target offset, so as to delay the timestamp of the subsequent vertical synchronization signal.

[0061] Figure 2 Corresponding to the scenario in which the video source frame rhythm is slower than the display frame. Figure 2 The dashed line in the middle represents the moment when the accumulated length reaches the timeout waiting time, and the thick short line represents the actual position of the vertical synchronization signal of the current display frame to be adjusted. By moving the timestamp of the signal in the positive direction of the time axis by a target offset, the timestamp can be accurately delayed, so that the emission moment of the subsequent vertical synchronization signal is re-aligned with the frame display rhythm expected by the application program.

[0062] In another embodiment, if the video source frame rate is greater than the display frame rate, the timestamp of the vertical synchronization signal of the current display frame is advanced by a target offset, so as to advance the timestamp of the subsequent vertical synchronization signal.

[0063] Figure 3 Corresponding to the scenario in which the video source frame rhythm is faster than the display frame. Figure 3The middle dotted line also represents the timeout waiting trigger point, and the thick short line represents the actual position of the vertical synchronization signal of the current display frame to be adjusted. By moving the signal timestamp in the negative direction of the time axis by the target offset, the timestamp can be accurately advanced, so that the issuance time of the subsequent vertical synchronization signal is re-aligned with the frame display rhythm expected by the application program.

[0064] In the above technical solution, the direction of timestamp correction (delay or advance) is dynamically adjusted according to the relative size of the video source frame rate and the display frame rate, which can ensure that the synchronization strategy accurately adapts to different frame rate scenarios, and improves display compatibility and user experience.

[0065] In some possible implementations, in response to the frame rate deviation of the video source frame rate and the display frame rate being greater than a preset value, the step of determining the timeout waiting duration according to the video source frame rate and the display frame rate is performed.

[0066] In an embodiment, the preset value can be pre-set based on actual needs. If the frame rate deviation of the video source frame rate and the display frame rate is greater than the preset value, it can be determined that a frame loss may occur once in a short period. At this time, the timeout waiting duration can be determined to actively perform synchronization correction to avoid frequent frame loss.

[0067] If the frame rate deviation of the video source frame rate and the display frame rate is less than or equal to the preset value, it can be determined that the video source frame rate and the display frame rate are close to or the same, and a frame loss may occur once in a long time, or there is no frame loss problem. At this time, in order to avoid excessive occupation of system resources, the subsequent synchronization optimization step can not be performed, and system resources are left to other tasks, so that energy efficiency optimization can be realized while ensuring basic smoothness.

[0068] In some possible implementations, the signal processing method provided by the present disclosure further includes: determining the video source frame rate according to the time of submitting the video source frame of the first preset number of frames; or obtaining the video source frame rate through an application program for playing a video.

[0069] In an embodiment, the first preset number of frames can be pre-set based on actual needs, for example, can be set to 60. The determination of the video source frame rate can be realized by using the formula frame rate = number of frames / total time. For example, the submission time of the recent 60 video source frames can be recorded through recordQueueBufferTime each time the application program calls a function (such as setTransactionState), and the actual frame rate of the video source can be calculated in real time based on the formula "frame rate = 60 / total time". In this way, by relying on the actual buffer submission time, the video source frame rate is determined by counting the total time of the first preset number of frames, which can dynamically adapt to changes and ensure the accuracy and timeliness of the video source frame rate data.

[0070] In yet another embodiment, the video source frame rate can be directly obtained through the application program, for example, the default video source frame rate of the application program is determined as the video source frame rate to be used subsequently. In this way, the key parameters for subsequent synchronization optimization can be quickly and accurately provided.

[0071] In the above technical solution, by counting the first preset number of frame submission times or directly obtaining the video source frame rate from the application program, real-time and accurate acquisition of the video source frame rate can be realized, a reliable basis for the synchronization strategy is provided, and the accuracy of the synchronization control is improved.

[0072] In some possible implementation manners, the signal processing method provided by the present disclosure further includes: Determine the display frame rate according to the time information of the vertical synchronization signal of the second preset number of frames.

[0073] In an embodiment, the second preset number of frames can be pre-set based on actual needs, for example, it can be set to 6. The most recent 6 consecutive vertical synchronization signals can be selected as the sampling data, the sum of the time intervals of the 6 vertical synchronization signals is determined, and then the display frame rate with high accuracy and timeliness is determined to provide an accurate basis for subsequent timestamp offset adjustment.

[0074] In some possible implementation manners, the target device is in a scenario of playing a video under a target application program.

[0075] In an embodiment, the target application program can be an album, and the signal processing method provided by the present disclosure can take effect when playing a video under the album. For example, the judgment of the scenario can be realized by judging whether the current image layer is the image layer of the media viewer. For example, when the target application program creates a layer, the video source frame rate can be passed through a function (such as setRecordQueueBufferFrameRate), and then it is determined that the scenario is started; when the layer is destroyed, the video source frame rate is reset to zero through a function, and then it is determined that the scenario is closed.

[0076] It should be noted that the signal processing method provided by the present disclosure can be applied not only to the album playing a video, but also to any video scenario, even a game scenario. The illustration taking the album as the target application program should not be understood as limiting the application field and protection scope of the present disclosure.

[0077] In the above technical solution, through scenario design, the adaptability and effectiveness of the signal processing method in the target application program can be ensured, and then the visual experience and operation fluency of the user in the scenario are improved.

[0078] Figure 4 A schematic diagram of the application program and Vsync state before timestamp modification according to an example embodiment. As shown in FIG. 1, the application program is in a state of playing a video, and the Vsync state is in a state of playing a video.Figure 4 As shown, frame drops occur before the timestamp is modified. At this time, Vsync-sf and the underlying Vsync are mismatched, causing frame drops during Vsync synchronization. To resolve this issue, the following solution provided in this disclosure can be implemented: 1. Obtain the video source frame rate: When using `setTransactionState`, the video source frame rate is obtained via `recordQueueBufferTime`. The reason for using `setTransactionState` is that the application can call `setTransactionState` every time the queueBuffer is given to the source file (sf). `recordQueueBufferTime` records the arrival time of the latest 60 video source frames, thus determining the video source frame rate. Alternatively, the video source frame rate can be obtained directly through the application.

[0079] 2. Enable shiftVsync optimization: a. Each frame can use nextAnticipatedVSyncTimeFrom to calculate the next Vsync. If the corresponding function is enabled, and the target device is in a preset scene and has variable frame rate capability (such as 360Hz TE), then shiftVsync can be started to achieve adjustment.

[0080] b. processShiftVsync can first obtain the sf frame rate (i.e., the display frame rate). The vsync model stores the current underlying frame rate period slope (the value is the same as the target offset). processShiftVsync can use the latest 6 frames to calculate the sf frame rate.

[0081] c. It can obtain the current system time and calculate the timeout for the next execution of shiftVsync optimization: calculateTimeoutForShiftVsync.

[0082] If the source video frame rate and the display frame rate are equal, there is no need to call shiftVsync. If the deviation between the source video frame rate and the display frame rate is less than a preset value and not zero, the frame rate deviation is small, and to avoid wasting resources, shiftVsync can be omitted. If the deviation between the source video frame rate and the display frame rate is greater than a preset value, shiftVsync can be called to achieve the offset. The offset direction can be determined by comparing the magnitudes of the source video frame rate and the display frame rate.

[0083] If the video source frame rate is less than the display frame rate, forward shift is called, and shiftVsyncSequenceAdd is invoked; if the video source frame rate is greater than the display frame rate, reverse shift is called, and shiftVsyncSequenceSub is invoked.

[0084] The shift operation can not be performed when the timeout time is determined for the first time, and the shift operation is performed when the timeout time is reached, and the timeout time is calculated again by calling calculateTimeoutForShiftVsync.

[0085] 3. Implementing ShiftVsync: For example, shiftVsyncSequenceAdd, the step of modifying the vsyncTime (i.e. the timestamp of the vertical synchronization signal) of the current frame to delay the target offset, so that the subsequent vertical synchronization signal is delayed, and the emission of the vertical synchronization signal is re-aligned with the expected frame submission rhythm of the application.

[0086] Figure 5 A schematic diagram of the application and Vsync state after the timestamp is modified according to an exemplary embodiment is shown. As Figure 5 shown, the frame loss problem is significantly improved, and the smoothness and stability of video playback are improved.

[0087] Figure 6 A block diagram of a signal processing device according to an exemplary embodiment is shown. As Figure 6 shown, the signal processing device 600 includes: A first determination module 601 is configured to determine a current video source frame and a current display frame corresponding to the current video source frame. An adjustment module 602 is configured to modify the timestamp of the vertical synchronization signal of the current display frame to align the modified timestamp with the display time of the corresponding video source frame, in response to the cumulative time error between the current video source frame and the current display frame reaching a target offset.

[0088] In the above technical solution, by determining the current video source frame and the corresponding display frame, and monitoring the cumulative time error between the two in real time, when the error reaches the target offset, the timestamp of the vertical synchronization signal of the current display frame is accurately modified, so that the modified timestamp is aligned with the display time of the corresponding video source frame, thereby avoiding frame loss, smoothing the difference between the video and display frame rates, improving the smoothness and stability of video playback, improving the user's visual experience, and enhancing the display performance of the target device.

[0089] In some possible implementations, the adjustment module 602 is configured to modify the timestamp of the vertical synchronization signal of the current display frame by: in response to a cumulative time length from a target time reaching a timeout waiting time length, modify a timestamp of a vertical synchronization signal of the current display frame; wherein the target time is a time when a timestamp of a display frame aligns with a display time of a corresponding video source frame, and the timeout waiting time length is a corresponding interval time length when a cumulative time error of the video source frame and the display frame reaches a target offset from the target time.

[0090] In some possible implementation manners, the adjusting module 602 comprises: a first determining sub-module, configured to determine a video source frame rate and a display frame rate; an adjusting sub-module, configured to delay or advance a timestamp of a vertical synchronization signal of a current display frame by the target offset based on the video source frame rate and the display frame rate.

[0091] In some possible implementation manners, the adjusting sub-module is configured to delay or advance the timestamp of the vertical synchronization signal of the current display frame by the target offset by the following manner: if the video source frame rate is less than the display frame rate, delay the timestamp of the vertical synchronization signal of the current display frame by the target offset to delay a timestamp of a subsequent vertical synchronization signal; if the video source frame rate is greater than the display frame rate, advance the timestamp of the vertical synchronization signal of the current display frame by the target offset to advance a timestamp of a subsequent vertical synchronization signal.

[0092] In some possible implementation manners, the signal processing apparatus 600 further comprises: a second determining module, configured to determine a maximum value of a variable frame frequency of display hardware, and determine a reciprocal of the maximum value of the variable frame frequency of the display hardware as the target offset.

[0093] In some possible implementation manners, the timeout waiting time length is determined by the following manner: determine a video source frame rate and a display frame rate; determine the timeout waiting time length according to the video source frame rate and the display frame rate.

[0094] In some possible implementation manners, the determining the timeout waiting time length according to the video source frame rate and the display frame rate comprises: determine a first single-frame time length of a video source according to the video source frame rate; determine a second single-frame time length of a display frame according to the display frame rate; determine a frame length deviation of the first single-frame time length and the second single-frame time length; determine the timeout waiting time length according to a ratio of the target offset and the frame length deviation and the second single-frame time length.

[0095] With regard to the apparatus in the above-described embodiments, a specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and thus will not be described in detail here.

[0096] Figure 7 is a block diagram of an electronic device 800 according to an example embodiment. Referring to Figure 7 , the electronic device 800 can include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0097] The processing component 802 usually controls overall operations of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to complete all or part of the steps of the signal processing methods described above. In addition, the processing component 802 can include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0098] The memory 804 is configured to store various types of data to support operations of the electronic device 800. Examples of these data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0099] The power supply component 806 provides power for the various components of the electronic device 800. The power supply component 806 can include a power supply management system, one or more power supplies, and other components associated with generating, managing and distributing power for the electronic device 800.

[0100] The multimedia component 808 includes a display for the electronic device 800 to provide an output interface between the electronic device 800 and a user. The display supports multiple touch modes. In some embodiments, the display can include a liquid crystal display (LCD) and a touch panel (TP). If the display includes a touch panel, the display can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors for sensing touch, swiping and gestures on the touch panel. The touch sensors can not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0101] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) for receiving an external audio signal when the electronic device 800 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0102] The input / output interface 812 provides an interface between the processing component 802 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0103] The sensor component 814 includes one or more sensors for providing various state assessments for the electronic device 800. For example, the sensor component 814 can detect an open / closed state of the electronic device 800, relative positioning of components, such as a display and a keypad of the electronic device 800, a change in position of the electronic device 800 or a component of the electronic device 800, presence or absence of user contact with the electronic device 800, orientation or acceleration / deceleration of the electronic device 800, and a temperature change of the electronic device 800. The sensor component 814 can include a proximity sensor configured to detect presence of a nearby object without any physical touch. The sensor component 814 can further include a light sensor, such as a CMOS or CCD image sensor, for use in an imaging application. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0104] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0105] In an exemplary embodiment, the electronic device 800 can be implemented with one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements, for performing the above-described signal processing methods.

[0106] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 804 including instructions, is also provided, which can be executed by the processor 820 of the electronic device 800 to complete the above-described signal processing methods. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0107] In another exemplary embodiment, the present disclosure also provides a computer-readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the signal processing method provided by the present disclosure.

[0108] In another exemplary embodiment, the present disclosure also provides a computer program product containing a computer program capable of being executed by a programmable device, the computer program having code portions for performing the above-described signal processing method when executed by the programmable device.

[0109] It should be understood that the features of various embodiments of the present disclosure described herein can be combined with each other, unless specifically noted otherwise. As used in this document, the term “and / or” includes any one of the associated listed items, as well as any combination of any two or more of the associated listed items; similarly, “at least one of’ includes any one of the associated listed items, as well as any combination of any two or more of the associated listed items.

[0110] Although terms such as "first", "second", and "third" can be used herein to describe various elements, components, regions, layers or sections, these elements, components, regions, layers or sections should not be limited to the above terms. Instead, these terms are only used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Thus, the first element, component, region, layer or section mentioned in the examples described herein can also be referred to as the second element, component, region, layer or section without departing from the teachings of the examples. In addition, the terms "first", "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or an indicated number of technical features. Thus, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description herein, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise explicitly specified.

[0111] Furthermore, the word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as advantageous over other aspects or designs. Rather, the exemplary word is used herein to present concepts in a concrete manner. As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless specified otherwise, or unless clear from context, "X employs A or B" is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied under any of the foregoing instances. In addition, the articles "a" and "an" as used in this application and the appended claims should generally be construed to mean "one or more" unless specified otherwise or clear from context to be directed to a singular form.

[0112] Likewise, although the present disclosure has been described and illustrated with respect to one or more implementations, equivalent alterations and modifications will become apparent to those skilled in the art that do not depart from the true spirit and scope of the disclosure. The present disclosure includes all such modifications and alterations and is limited only by the scope of the following claims. In particular regard to the various functions performed by the above described components (e.g., elements, resources, etc.), the terms (including a reference to a "means") used to describe such components are intended to correspond, unless otherwise indicated, to any component which performs the described function (e.g., a functional equivalent), even though not structurally equivalent to the disclosed structure. In addition, although a particular feature of the disclosure can have been disclosed with respect to only one of several implementations, other implementations can include the particular feature. For example, the disclosure can be implemented with respect to other implementations that incorporate the particular feature, and that implement other features as disclosed herein, and each of the various implementations have a reasonable expectation of support. Furthermore, to the extent that the terms "includes", "including", "has", "have", "having", or variants thereof are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term "comprising".

[0113] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features of the disclosure disclosed herein. It is intended that the present disclosure be considered as including any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such steps, compositions, components, and / or elements known in the art to be appropriate. It is specifically intended that the present disclosure include all such modifications and alterations in the application, processes, and procedures as fall within the usual scope of the technology and are appreciated by those skilled in the art. The specification and examples are to be considered exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

[0114] It is to be understood that the present disclosure is not limited to the precise construction described and shown herein and that changes can be made in various embodiments without departing from the scope of the disclosure. The scope of the present disclosure is limited only by the claims appended hereto.

Claims

1. A signal processing method, characterized in that, include: Determine the current video source frame and the current display frame corresponding to the current video source frame; In response to the cumulative time error between the current video source frame and the current display frame reaching the target offset, the timestamp of the vertical synchronization signal of the current display frame is modified so that the modified timestamp is aligned with the display time of the corresponding video source frame.

2. The signal processing method according to claim 1, characterized in that, The step of modifying the timestamp of the vertical synchronization signal of the current display frame in response to the cumulative time error between the current video source frame and the current display frame reaching a target offset includes: In response to the cumulative duration since the target time reaching the timeout waiting period, the timestamp of the vertical synchronization signal of the current display frame is modified; wherein, the target time is the time when the timestamp of the display frame is aligned with the display time of the corresponding video source frame, and the timeout waiting period is the interval duration corresponding to when the cumulative time error between the video source frame and the display frame reaches the target offset since the target time.

3. The signal processing method according to claim 2, characterized in that, Modifying the timestamp of the vertical synchronization signal of the current display frame includes: Determine the source video frame rate and the display frame rate; Based on the video source frame rate and the display frame rate, the timestamp of the vertical synchronization signal of the current display frame is delayed or advanced by the target offset.

4. The signal processing method according to claim 3, characterized in that, The step of delaying or advancing the timestamp of the vertical synchronization signal of the current display frame by the target offset based on the video source frame rate and the display frame rate includes: If the video source frame rate is less than the display frame rate, the timestamp of the vertical synchronization signal of the current display frame is delayed by the target offset to delay the timestamp of the subsequent vertical synchronization signal. If the video source frame rate is greater than the display frame rate, the timestamp of the vertical synchronization signal of the current display frame is advanced by the target offset to advance the timestamp of the subsequent vertical synchronization signal.

5. The signal processing method according to any one of claims 1-4, characterized in that, The method further includes: Determine the maximum value of the variable frame rate of the display hardware; The target offset is determined by taking the reciprocal of the maximum value of the variable frame rate of the display hardware.

6. The signal processing method according to any one of claims 2-4, characterized in that, The timeout duration is determined in the following way: Determine the source video frame rate and the display frame rate; The timeout duration is determined based on the video source frame rate and the display frame rate.

7. The signal processing method according to claim 6, characterized in that, Determining the timeout duration based on the video source frame rate and the display frame rate includes: The duration of the first single frame of the video source is determined based on the video source frame rate. The duration of the second single frame of the display frame is determined based on the display frame rate. Determine the frame length deviation between the first single-frame duration and the second single-frame duration; The timeout waiting time is determined based on the ratio of the target offset to the frame length deviation and the second single frame duration.

8. A signal processing apparatus, characterized in that... The signal processing device includes: The first determining module is used to determine the current video source frame and the current display frame corresponding to the current video source frame; An adjustment module is used to modify the timestamp of the vertical synchronization signal of the current display frame in response to the cumulative time error between the current video source frame and the current display frame reaching a target offset, so that the modified timestamp is aligned with the display time of the corresponding video source frame.

9. The signal processing apparatus according to claim 8, characterized in that, The adjustment module is used to modify the timestamp of the vertical synchronization signal of the current display frame in the following manner: In response to the cumulative duration since the target time reaching the timeout waiting period, the timestamp of the vertical synchronization signal of the current display frame is modified; wherein, the target time is the time when the timestamp of the display frame is aligned with the display time of the corresponding video source frame, and the timeout waiting period is the interval duration corresponding to when the cumulative time error between the video source frame and the display frame reaches the target offset since the target time.

10. The signal processing apparatus according to claim 9, characterized in that, The adjustment module includes: The first determining submodule is used to determine the video source frame rate and the display frame rate; The adjustment submodule is used to delay or advance the timestamp of the vertical synchronization signal of the current display frame by the target offset based on the video source frame rate and the display frame rate.

11. The signal processing apparatus according to claim 9 or 10, characterized in that, The timeout duration is determined in the following way: Determine the source video frame rate and the display frame rate; The timeout duration is determined based on the video source frame rate and the display frame rate.

12. An electronic device, characterized in that, include: Display hardware that supports multiple tearing effect signals; processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions in the memory to implement the steps of the signal processing method according to any one of claims 1-7.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the signal processing method according to any one of claims 1-7.

14. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the steps of the signal processing method according to any one of claims 1-7.

Images