A dynamic optimization method for subtitle display in free zoom mode
By dynamically adjusting the display position of subtitles and creating floating windows in the desktop system, the problem of occlusion or incomplete display in the video playback application on the desktop platform is solved, and the effective display of subtitles when any window size changes is achieved.
Patent Information
- Application Number
- CN202410585404.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-13
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-05-13
AI Technical Summary
Video playback applications running on desktop platforms may cause problems such as subtitles obscuring the video display area or incomplete display, especially when the application window is zoomed, because mobile applications only adapt to fixed-scale display windows.
Dynamically obtain the updated area and unupdated area of the target desktop window, adjust the display position of subtitles. If it cannot be adjusted, create a floating window to display subtitles in the desktop system to ensure that the subtitles can be displayed effectively without obstructing the video when the size of the window changes without blocking the video. area.
Without modifying the application, the problem of subtitles obstructing the video display area or incomplete display is effectively avoided, ensuring that subtitles can always be displayed normally in free zoom mode.
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of computer software development, and in particular relates to a dynamic optimization method for subtitle display in a free zooming mode. Background Art
[0002] In order to use Android applications on desktop platforms, an Android compatible environment is generally used to run Android applications in the form of desktop application windows. Existing Android compatible environment implementation solutions include xDroid. xDroid includes xDroidUI and xDroidServer. xDroidUI is an Android application display unit, and xDroidServer is an Android application compatible service.
[0003] Commonly used video playback applications usually display subtitles and bullet screens at the same time when playing videos. Subtitles and bullet screens are collectively referred to as subtitles below. Subtitles are usually displayed in the blank area just above or just below the video area. For display windows with fixed ratios such as mobile phones, the relative positions of the above subtitles can generally be displayed normally, that is, the subtitles will not block the video display area. However, when such video playback applications are running on desktop platforms, since mobile video playback applications are often only adapted to several commonly used display window ratios, problems such as subtitles blocking the video display area or incomplete subtitle display may occur during the scaling of the application window. Summary of the invention
[0004] In view of this, the present invention provides a dynamic optimization method for subtitle display in free zoom mode, which effectively avoids the problems of subtitles blocking the video display area or incomplete subtitle display caused by free zooming of mobile terminal software without modifying the application.
[0005] The present invention provides a method for dynamically optimizing subtitle display in a free zoom mode, comprising the following steps:
[0006] Step 1, the Android application display unit generates a target desktop window corresponding to the target Android application, traverses the view tree of the target desktop window, records the view corresponding to the video playback component as the target video playback view, and records the view corresponding to the subtitle rendering component as the target subtitle view;
[0007] Step 2: When the window size of the target desktop window changes, the update area of the target video playback view is obtained, and the difference between the rectangular area corresponding to the target video playback view and the update area is the unupdated area; a continuous black area is located in the unupdated area as a black border area, and the unupdated area is divided into an upper area and a lower area according to the relative position with the update area;
[0008] Step 3: If the target subtitle view overlaps with the rectangular area corresponding to the update area, execute step 4; otherwise, end this process;
[0009] Step 4: If the width of the target subtitle view cannot accommodate the subtitles, the font size of the subtitles is reduced. If the target subtitle view still cannot accommodate the subtitles when the font size is the minimum font, the Android application display unit creates a floating window in the desktop system to display the subtitles. Otherwise, the subtitles are displayed according to the determined font size. If the width of the current target subtitle view can accommodate the subtitles, step 5 is executed.
[0010] Step 5: If neither the upper area nor the lower area can accommodate the target subtitle view, the Android application display unit creates a floating window in the desktop system to display the subtitles; otherwise, if the upper area can accommodate the target subtitle view, the lower edge of the target subtitle view is aligned with the lower edge of the black border area in the upper area to complete the display; if the lower area can accommodate the target subtitle view, the upper edge of the target subtitle view is aligned with the upper edge of the black border area in the lower area to complete the display.
[0011] Furthermore, in step 1, the method of recording the view corresponding to the video playback component as the target video playback view is as follows: if the view inherits from the VideoView class, then the view is the target video playback view.
[0012] Furthermore, the method of recording the view corresponding to the video playback component as the target video playback view in step 1 is: modifying the setDisplay function implementation of the MediaPlayer class of the Android framework, and when it is detected that the SurfaceView corresponding to the parameter SurfaceHolder of the setDisplay function of the MediaPlayer class is a view, the view is the target video playback view.
[0013] Furthermore, the method of recording the view corresponding to the subtitle rendering component as the target subtitle view in step 1 is: if the view inherits from the text view class and the name of the parent view of the view is related to the subtitles, then the view is recorded as the target subtitle view.
[0014] Furthermore, the method of recording the view corresponding to the subtitle rendering component as the target subtitle view in step 1 is: if the ID of the view is related to the subtitle, the view is recorded as the target subtitle view.
[0015] Furthermore, the method of recording the view corresponding to the subtitle rendering component as the target subtitle view in step 1 is: if the view is located below the target video playback view and the width of the view is close to the width of the target video playback view, then the view is recorded as the target subtitle view.
[0016] Furthermore, the method of recording the view corresponding to the subtitle rendering component as the target subtitle view in step 1 is as follows: when the target video playback view renders a frame of data, the view that inherits from the text view class and updates its text content is recorded as the target subtitle view.
[0017] Furthermore, the method of locating the continuous black area in the non-updated area in step 2 is to adopt a row-by-row parallel scanning method.
[0018] Furthermore, the method for the Android application display unit to create a floating window in the desktop system to display subtitles is: the Android application compatible service first enables the target subtitle view not to perform rendering operations, and sends subtitle data related to the rendering operation to the Android application display unit, and then the Android application display unit creates a floating window based on the subtitle data to complete the display.
[0019] Furthermore, the method of creating the floating window according to the subtitle data by the Android application display unit is: when the desktop system is Linux, the Android application display unit uses Qt components to create the floating window. Beneficial Effects
[0020] The present invention dynamically obtains the updated area and the unupdated area of the target video playback view when the window size of the target desktop window changes, and when dynamically determining that the target subtitle view overlaps with the updated area, determines the adjustment method of the target subtitle view according to the display relationship between the target subtitle view and the subtitles and the position relationship between the target subtitle view and the unupdated area, and creates a floating window in the desktop system to display the subtitles when adjustment is impossible, thereby effectively avoiding the problems of subtitles blocking the video display area or incomplete subtitle display caused by free scaling of mobile terminal software without modifying the application. DETAILED DESCRIPTION
[0021] The present invention is described in detail with reference to the following embodiments.
[0022] The present invention provides a dynamic optimization method for subtitle display in a free scaling mode, the core idea of which is: by dynamically obtaining the updated area and the unupdated area of the target video playback view when the window size of the target desktop window changes, and when dynamically determining that the target subtitle view overlaps with the updated area, the adjustment method of the target subtitle view is determined according to the display relationship between the target subtitle view and the subtitles and the position relationship between the target subtitle view and the unupdated area, and when adjustment is impossible, a floating window is created in the desktop system to complete the display of the subtitles.
[0023] The present invention provides a method for dynamically optimizing subtitle display in a free zoom mode, which specifically includes the following steps:
[0024] Step 1: Start the Android compatible environment, and start the target Android application in the Android compatible environment. The Android application display unit generates a target desktop window corresponding to the target Android application; traverse the view tree of the target desktop window, find the view corresponding to the video playback component, and mark the view as the target video playback view. The target video playback view can be divided into a video area, a black border area, and other areas. The other areas refer to all areas except the video area and the black border area.
[0025] Specifically, for the video playback view implemented based on the video playback class that comes with the Android framework, the following method can be used to find the view corresponding to the video playback component from the view tree of the target desktop window: if the view inherits from the VideoView class, then the view is the target video playback view; or if it is detected that the SurfaceView corresponding to the parameter SurfaceHolder of the setDisplay function of the MediaPlayer class is a certain view, then the view is the target video playback view. This judgment process can be implemented by modifying the setDisplay function of the MediaPlayer class of the Android framework.
[0026] Step 2: traverse the view tree of the target desktop window, find the view corresponding to the subtitle rendering component, and mark the view as the target subtitle view.
[0027] Specifically, by analyzing the view-related attributes, it is possible to statically determine whether a view is a target subtitle view, including the following methods:
[0028] If the view inherits from the text view class and the name of the parent view of the view is related to subtitles, the view is marked as the target subtitle view. Alternatively, if the view ID is related to subtitles, the view is marked as the target subtitle view, for example, the view ID contains subtitle. In addition, since subtitles are usually displayed at the bottom of the video playback area, the layout file of the target desktop window is analyzed. If a view in it is located below the target video playback view and the width of the view is similar to the width of the target video playback view, the view is marked as the target subtitle view.
[0029] If the above static methods cannot determine whether the view is the target subtitle view, it can be determined dynamically during video playback. The specific process is: every time the target video playback view renders a frame of data, a view inherited from the text view class will update its text content, and the view will be marked as the target subtitle view.
[0030] Step 3. When the window size of the target desktop window changes, the update area of the target video playback view is obtained, and the update area is the area in the target video playback view where the display content changes. The difference between the rectangular area corresponding to the target video playback view and the update area is the unupdated area, and the unupdated area is the black border area and other areas. In the unupdated area, a continuous black area is located in a row-by-row parallel scanning manner, and the black area is the current black border area. The unupdated area is divided into an upper area and a lower area according to whether it is above or below the update area.
[0031] Step 4: Get the display position of the current target subtitle view. If the target subtitle view overlaps with the rectangular area corresponding to the update area, execute step 5. Otherwise, do not execute any operation and exit this process.
[0032] Step 5. If the width of the current target subtitle view cannot accommodate the subtitles displayed by it, reduce the font size used for the subtitles. If the width of the current target subtitle view still cannot accommodate the subtitles when the font size is the minimum font, create a floating window in the desktop system through the Android application display unit to display the subtitle content. Otherwise, display the subtitles according to the determined font size. If the width of the current target subtitle view can accommodate the subtitles displayed by it, execute step 6.
[0033] In addition, when the width of the current target subtitle view cannot accommodate the subtitles displayed by it, the width of the target subtitle view can be increased. If the target subtitle view still cannot accommodate the subtitles when the width is increased to the maximum value that the target desktop window can accommodate, a floating window is created in the desktop system through the Android application display unit to display the subtitle content. Otherwise, the target subtitle view is set to a certain width to display the subtitles.
[0034] Step 6: If neither the upper area nor the lower area can accommodate the target subtitle view, a floating window is created in the desktop system through the Android application display unit to display the subtitle content; otherwise, if the upper area can accommodate the target subtitle view, the bottom edge of the target subtitle view is aligned with the bottom edge of the black border area in the upper area; if the lower area can accommodate the target subtitle view, the top edge of the target subtitle view is aligned with the top edge of the black border area in the lower area.
[0035] In the above steps 5 and 6, before creating a floating window in the desktop system through the Android application display unit to display the subtitle content, the target subtitle view is prevented from performing rendering operations through the Android application compatible service, and the subtitle data related to the rendering operation is sent to the Android application display unit, and the Android application display unit then creates a floating window based on the subtitle data.
[0036] Furthermore, when the desktop system is Linux, the Android application display unit uses Qt components to create a floating window. Example
[0037] In this embodiment, a dynamic optimization method for subtitle display in a free zoom mode provided by the present invention is used to realize free zooming of Android applications in a Linux desktop system in combination with xDroid. The specific steps are as follows:
[0038] S1. Traverse the view tree of the current desktop window and dynamically determine the view corresponding to the video playback component. This view is generally inherited from SurfaceView and is recorded as videoView. Specifically, the video playback view implemented based on the video playback class of the Android framework can use the following two methods to determine whether the view is used to play the video:
[0039] S1.1. Traverse the current number of views and record the currently traversed view as view;
[0040] S1.2. Based on the Java reflection mechanism, if view inherits from the VideoView class, videoView is assigned to view and the loop ends.
[0041] Another way is to modify the setDisplay function of the MediaPlayer class of the Android framework. If the SurfaceView corresponding to the SurfaceHolder parameter of the setDisplay function is equal to view, then assign videoView to view and end the loop.
[0042] S2. Traverse the view tree of the current desktop window and dynamically determine the view corresponding to the subtitle rendering component. This view is generally inherited from TextView and is recorded as subtitleView.
[0043] Specifically, the following methods can be used to determine whether a view is used to render a video:
[0044] S2.1. The currently traversed view is recorded as view.
[0045] S2.2. If view inherits from TextView and is placed in a parent view named subtitle_container, assign subtitleView to view and end the loop.
[0046] Alternatively, if the ID of view is very special, such as @+id / subtitle_view, subtitleView is assigned to view and the loop ends.
[0047] Alternatively, since subtitles are usually displayed at the bottom of the video playback area, if view is located below videoView in the layout file and its width is similar to the area of videoView, assign subtitleView to view and end the loop.
[0048] If subtitleView is not obtained through the above methods, it is determined dynamically when the video is playing. That is, when subtitleView renders a frame of data each time, TextView will normally update its text content, so the view of the updated content is assigned to subtitleView.
[0049] S3. During dynamic scaling, that is, each time the window changes, perform the following steps:
[0050] S3.1. Determine the range of the upper and lower black borders in the video playback view, that is, the unrefreshed part, mainly the upper and lower boundaries of the area. Divide all areas of the entire window into video area, black border area and other areas, specifically:
[0051] Determine the update area of videoView by monitoring the invalidate function of videoView. Subtract the update area from the rectangular area of videoView to get the sum of the black border area and other areas. Record this area as areaNotUpdate.
[0052] Get the content of the current frame of data, and quickly locate the continuous black area by line-by-line parallel scanning in areaNotUpdate. The black area is recorded as blackArea. The CPU SIMD can be used to accelerate the above line-by-line parallel scanning operation.
[0053] S3.2. If the rectangular area of subtitleView overlaps with the rectangular area of videoView, that is, the subtitle view appears in the video area, then:
[0054] Dynamically adjust the vertical coordinate of subtitleView, and prioritize it in the black border area; if the black border area cannot accommodate it completely, continue to adjust the vertical coordinate to move subtitleView to the black border area and other areas, and occupy the black border area as much as possible; if the sum of the black border area and other areas cannot accommodate subtitleView, use xDroidUI of xDroid to create a floating Linux window using Qt components in the Linux system to display subtitles, and the Linux window is located below or above videoView;
[0055] If the width of subtitleView is not enough to accommodate the subtitles, the font size is dynamically reduced; if it is reduced to the lower limit of the font, a floating Linux window is created in the Linux system using Qt components through xDroid's xDroidUI to display the subtitles.
[0056] The lower limit of the font is the minimum value of the font size that can be seen clearly.
[0057] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A dynamic optimization method for subtitle display in free zoom mode, characterized in that: The following steps are involved: Step 1, the Android application display unit generates a target desktop window corresponding to the target Android application, traverses the view tree of the target desktop window, records the view corresponding to the video playback component as the target video playback view, and records the view corresponding to the subtitle rendering component as the target subtitle view; Step 2: When the window size of the target desktop window changes, the update area of the target video playback view is obtained, and the difference between the rectangular area corresponding to the target video playback view and the update area is the unupdated area; a continuous black area is located in the unupdated area as a black border area, and the unupdated area is divided into an upper area and a lower area according to the relative position with the update area; Step 3: If the target subtitle view overlaps with the rectangular area corresponding to the update area, execute step 4; otherwise, end this process; Step 4: If the width of the target subtitle view cannot accommodate the subtitles, the font size of the subtitles is reduced. If the target subtitle view still cannot accommodate the subtitles when the font size is the minimum font, the Android application display unit creates a floating window in the desktop system to display the subtitles. Otherwise, the subtitles are displayed according to the determined font size. If the width of the current target subtitle view can accommodate the subtitles, step 5 is executed. Step 5: If the upper area and the lower area cannot accommodate the target subtitle view, the Android application display unit creates a floating window in the desktop system to display the subtitles; Otherwise, if the upper area can accommodate the target subtitle view, the lower edge of the target subtitle view is aligned with the lower edge of the black border area in the upper area to complete the display; if the lower area can accommodate the target subtitle view, the upper edge of the target subtitle view is aligned with the upper edge of the black border area in the lower area to complete the display; The method of recording the view corresponding to the subtitle rendering component as the target subtitle view in step 1 is as follows: if the view inherits from the text view class and the name of the parent view of the view is related to the subtitle, the view is recorded as the target subtitle view.
2. The dynamic optimization method according to claim 1, characterized in that: The method of recording the view corresponding to the video playback component as the target video playback view in step 1 is: if the view inherits from the VideoView class, then the view is the target video playback view.
3. The dynamic optimization method according to claim 1, characterized in that: The method of recording the view corresponding to the video playback component as the target video playback view in step 1 is: modify the setDisplay function implementation of the MediaPlayer class of the Android framework, and when it is detected that the SurfaceView corresponding to the parameter SurfaceHolder of the setDisplay function of the MediaPlayer class is a view, the view is the target video playback view.
4. The dynamic optimization method according to claim 1, characterized in that: The method of recording the view corresponding to the subtitle rendering component as the target subtitle view in step 1 is: if the ID of the view is related to the subtitle, the view is recorded as the target subtitle view.
5. The dynamic optimization method according to claim 1, characterized in that: The method of recording the view corresponding to the subtitle rendering component as the target subtitle view in step 1 is as follows: if the view is located below the target video playback view and the width of the view is similar to the width of the target video playback view, the view is recorded as the target subtitle view.
6. The dynamic optimization method according to claim 1, characterized in that: The method of recording the view corresponding to the subtitle rendering component as the target subtitle view in step 1 is: when the target video playback view renders a frame of data, the view that inherits from the text view class and updates its text content is recorded as the target subtitle view.
7. The dynamic optimization method according to claim 1, characterized in that: The method of locating the continuous black area in the unupdated area in step 2 is to adopt a row-by-row parallel scanning method.
8. The dynamic optimization method according to claim 1, characterized in that: The method for creating a floating window in a desktop system by the Android application display unit to display subtitles is as follows: first, the Android application compatible service causes the target subtitle view to not perform rendering operations, and sends subtitle data related to the rendering operation to the Android application display unit, and then the Android application display unit creates a floating window based on the subtitle data to complete the display.
9. The dynamic optimization method according to claim 8, characterized in that: The method of creating the floating window according to the subtitle data by the Android application display unit is: when the desktop system is Linux, the Android application display unit uses the Qt component to create the floating window.
Citation Information
Patent Citations
Text processing method and device, text playing method and device and text playing control system
CN112632902A
TextView free scaling method based on implicit rendering
CN115599275A
Automatic judgment method for free scaling of application adaptation
CN116931933A
Subtitle display method and display equipment
CN117615199A
Method, equipment, and medium for displaying content
US20200401758A1
