Subtitle adjustment method and device, storage medium and electronic equipment

Abstract

The invention discloses a subtitle adjustment method and device, a storage medium and electronic equipment, and relates to the technical field of video processing, and the method comprises the steps: obtaining video stream data and subtitle stream data corresponding to a target video; performing alignment processing on the video stream data and the subtitle stream data to obtain a plurality of video frames corresponding to the target video and subtitle windows corresponding to the plurality of video frames; performing color similarity analysis on the plurality of video frames and the subtitle window, and determining a target video frame meeting a color similarity condition in the plurality of video frames; and generating operation prompt information of the subtitle window based on the target video frame, wherein the operation prompt information is used for prompting to adjust the subtitle window under the condition that the target video plays the target video frame. Compared with the prior art, if the condition that the subtitles are similar to the colors of the video pictures is recognized in the video playing process, a user can be prompted to carry out targeted adjustment on the subtitles, the readability of the subtitles is improved, and then the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a method, apparatus, storage medium and electronic device for adjusting subtitles. Background Technology

[0002] Video subtitles translate, explain, or enhance the viewer's understanding by providing textual information. They typically appear at the bottom of the video screen and can be embedded in the video file or be external files, such as SRT or ASS formats. Video subtitles can support multiple languages ​​to help users with hearing impairments or those watching videos in noisy environments.

[0003] Currently, the main way to adjust subtitles is by having users modify the displayed information. However, this method only allows users to make overall adjustments to the subtitles. If the subtitles and video screen colors are similar during video playback, users cannot make targeted adjustments, which can reduce the readability of the subtitles and affect the user experience. Summary of the Invention

[0004] In view of this, this application provides a subtitle adjustment method, apparatus, storage medium, and electronic device. The main purpose is to improve the technical problem that in the current technology, users can only make overall adjustments to the display of subtitles. If the subtitles and video screen colors are similar during video playback, users cannot make targeted adjustments to the subtitles, which will lead to a decrease in the readability of the subtitles and affect the user experience.

[0005] Firstly, this application provides a method for adjusting subtitles, including: Obtain the video stream data and subtitle stream data corresponding to the target video; The video stream data and the subtitle stream data are aligned to obtain multiple video frames corresponding to the target video and subtitle windows corresponding to the multiple video frames. A color similarity analysis is performed on the plurality of video frames and the subtitle window to determine the target video frame that meets the color similarity condition among the plurality of video frames; Operation prompts for generating the subtitle window based on the target video frame are used to prompt adjustments to the subtitle window when the target video plays the target video frame. Secondly, this application provides a subtitle adjustment device, comprising: The acquisition module is configured to acquire the video stream data and subtitle stream data corresponding to the target video. The alignment module is configured to align the video stream data and the subtitle stream data to obtain multiple video frames corresponding to the target video and subtitle windows corresponding to the multiple video frames. The determination module is configured to perform color similarity analysis on the plurality of video frames and the subtitle window to determine the target video frame that meets the color similarity condition among the plurality of video frames; The generation module is configured to generate operation prompt information for the subtitle window based on the target video frame. The operation prompt information is used to prompt adjustments to the subtitle window when the target video plays the target video frame.

[0006] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the subtitle adjustment method described in the first aspect.

[0007] Fourthly, this application provides an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the subtitle adjustment method described in the first aspect.

[0008] Fifthly, this application provides a computer program product, which includes a computer program that, when executed by a processor, implements the subtitle adjustment method described in the first aspect.

[0009] By employing the above technical solutions, this application provides a subtitle adjustment method, apparatus, storage medium, and electronic device, comprising: acquiring video stream data and subtitle stream data corresponding to a target video; aligning the video stream data and subtitle stream data to obtain multiple video frames corresponding to the target video and subtitle windows corresponding to the multiple video frames; performing color similarity analysis on the multiple video frames and subtitle windows to determine the target video frame among the multiple video frames that meets the color similarity condition; and generating operation prompt information for the subtitle window based on the target video frame, the operation prompt information being used to prompt adjustment of the subtitle window when the target video is played. Compared with the prior art, this application aligns the video stream data and subtitle stream data of the target video to obtain multiple video frames corresponding to the target video and subtitle windows corresponding to the multiple video frames after alignment. Furthermore, it can perform color similarity analysis based on the multiple video frames and corresponding subtitle windows to determine the target video frame that meets the color similarity condition. This allows it to prompt adjustment of the subtitle window when the user plays the target video and the target video frame. This enables the application to prompt the user to make targeted adjustments to the subtitles if it detects that the subtitles are similar in color to the video screen during video playback, thereby improving the readability of the subtitles and enhancing the user experience. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating a subtitle adjustment method provided in an embodiment of this application is shown; Figure 2 A flowchart illustrating a subtitle adjustment method provided in an embodiment of this application is shown; Figure 3 A flowchart illustrating an example of server-side subtitle processing provided in an embodiment of this application is shown. Figure 4 This illustration shows an interactive flow diagram of a subtitle processing example provided in an embodiment of this application; Figure 5 This paper shows a schematic diagram of the structure of a subtitle adjustment device provided in an embodiment of this application; Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0013] The embodiments of this application will now be described in more detail with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0014] To address the technical problem in existing technologies where users can only adjust subtitles as a whole, and where subtitles are similar in color to the video screen during playback, preventing targeted adjustments and thus reducing readability and impacting user experience, this embodiment provides a subtitle adjustment method. Figure 1 As shown, the method includes: Step 101: Obtain the video stream data and subtitle stream data corresponding to the target video.

[0015] In this embodiment of the application, the target video can be the video that the user is currently playing. In this embodiment, the target video can be a live video (such as a live sports event) or a recorded video (such as a movie or TV series). It should be noted that the specific type of the target video is not specifically limited in this embodiment of the application.

[0016] For example, if the current user is playing live sports event 1, then live sports event 1 can be the target video in this application embodiment; if the current user is playing movie 1, then movie 1 can be the target video in this application embodiment, and so on, without further examples here.

[0017] In some examples, the video stream data can be the video stream data of the target video. Specifically, the video stream data can be the continuous frame sequence data corresponding to the video frames of the target video. The video stream data can be transmitted in an encoding format (such as H.264 or H.265).

[0018] In this embodiment, the subtitle stream data can be the subtitle stream data of the target video. Specifically, the subtitle stream data can be subtitle text data corresponding to the video stream, such as embedded subtitles or external subtitle files. The subtitle stream data can include timestamps and text content.

[0019] For example, if the current user is playing live event 1, then live event 1 can be the target video in this application embodiment. Then, the live screen data corresponding to live event 1, the subtitle text data of the host's content, and the bullet screen data of other users can be obtained. The live screen data can be the video stream data in this application embodiment, and the subtitle text data of the host's content and the bullet screen data of other users can be the subtitle stream data in this application embodiment.

[0020] For example, if the current user is playing movie 1, then movie 1 can be the target video in this application embodiment. Then, the playback screen data corresponding to movie 1, the subtitle text data of the characters in the movie and the bullet screen data of other users can be obtained. Then, the playback screen data corresponding to movie 1 can be the video stream data in this application embodiment, and the subtitle text data of the characters in the movie and the bullet screen data of other users can be the subtitle stream data in this application embodiment.

[0021] Step 102: Align the video stream data and subtitle stream data to obtain multiple video frames corresponding to the target video and subtitle windows corresponding to the multiple video frames.

[0022] In this embodiment of the application, aligning the video stream data and the subtitle stream data can specifically involve synchronizing and matching the video stream data and the subtitle stream data according to the timestamp, ensuring that each frame of video is aligned with the corresponding subtitle text in time.

[0023] In some examples, the subtitle window can be an independent area on the video screen where subtitles are displayed. The subtitle window can be in a fixed position (such as the bottom of the video frame) or in a position that can be dragged by the user. It should be noted that in the embodiments of this application, the subtitle window can support operations such as scaling and color adjustment.

[0024] As an alternative approach, if the current user is playing live event 1, then live event 1 can be the target video in this application embodiment. After determining the video stream data 1 and subtitle stream data 1 corresponding to live event 1, the video stream data 1 and subtitle stream data 1 can be aligned according to the timestamp, thereby obtaining multiple consecutive video frames corresponding to video stream data 1 and the subtitle window in each consecutive video frame.

[0025] As an alternative approach, if the current user is playing movie 1, then movie 1 can be the target video in this application embodiment. After determining the video stream data 2 and subtitle stream data 2 corresponding to movie 1, the video stream data 2 and subtitle stream data 2 can be aligned according to the timestamp, thereby obtaining multiple consecutive video frames corresponding to video stream data 2 and the subtitle window in each consecutive video frame.

[0026] Step 103: Perform color similarity analysis on multiple video frames and subtitle windows to determine the target video frame that meets the color similarity criteria among the multiple video frames.

[0027] In this embodiment of the application, color similarity analysis of multiple video frames and subtitle windows can be specifically performed by calculating the similarity between the color of the video screen in the area covered by the subtitle window and the color of the subtitle, and then using the calculated similarity to determine whether there are similar colors between the video frames and the subtitle window.

[0028] In some examples, the target video frame can be a video frame whose color is similar to that of the subtitle window after color analysis. For example, if the current user is playing a live event 1, the mechanical energy color similarity analysis can be performed on multiple consecutive video frames 1 after alignment processing of the live event 1 and the subtitle window 1 in each consecutive video frame, and then the video frame whose color is similar to that of the subtitle window 1 can be identified in multiple consecutive video frames 1 as the target video frame in this application embodiment.

[0029] For example, if the current user is playing movie 1, color similarity analysis can be performed on multiple consecutive video frames 2 after movie 1 has been aligned and the subtitle window 2 in each consecutive video frame, and then the video frame with a color similar to the subtitle window 1 can be identified in the multiple consecutive video frames 2 as the target video frame in this application embodiment.

[0030] Step 104: Generate subtitle window operation prompts based on the target video frame.

[0031] The operation prompts are used to suggest adjustments to the subtitle window when the target video is playing a target video frame.

[0032] In this embodiment, corresponding subtitle window operation prompts can be generated for the identified target video frame, and then the subtitle window can be adjusted when the target video frame is played.

[0033] In some examples, the video system server can analyze and compare video frames and subtitles. When the color similarity between the subtitle window area and the video screen exceeds a set threshold (such as 0.8, which can be flexibly adjusted), the server notifies the client to display the window where the subtitle is dragged and pops up a prompt (i.e., the operation prompt information in the embodiment of this application) to remind the user that the position of the subtitle can be moved.

[0034] It should be noted that, in order to avoid the video system server repeatedly analyzing and comparing video frames and subtitles and reduce resource consumption, it can be handled in two ways depending on the video type: one is live video (such as sports games), and the other is recorded video (such as movies and TV series).

[0035] As an alternative approach, for live video (such as sports matches), the video system server can analyze and compare video frames and subtitles in real time, and push subtitle operation prompts when the similarity threshold is exceeded. It can also record the video time points during the live video broadcast where users are reminded to drag subtitles and the user's operation behavior on the subtitles. When the user watches the replay later, the subtitle operation prompts are given based on the stored video and subtitle analysis and comparison data. It is only necessary to record and update the latest subtitle operation behavior of the user. When the user plays the video again, the subtitle operation prompts are pushed based on the most recent user behavior.

[0036] As an alternative approach, for pre-recorded videos (such as movies and TV series), the video system server can analyze and compare video frames and subtitles in advance, record and store all video time points that require users to drag subtitles. When a user watches the video for the first time, the server can directly provide subtitle operation prompts based on the stored video and subtitle analysis and comparison data, and record and update the user's first subtitle operation behavior. When the user plays the video again, the server can push subtitle operation prompts based on the most recent user behavior.

[0037] As an alternative approach, if the current user is watching a live event 1, the mechanical energy color similarity analysis can be performed on multiple consecutive video frames 1 after alignment processing of the live event 1 and the subtitle window 1 in each consecutive video frame. Then, video frames with colors similar to the subtitle window 1 can be identified among the multiple consecutive video frames 1 as target video frames in this embodiment. This can then generate subtitle window prompt information for the live event 1, prompting the user watching the live event 1 to adjust the subtitle window to improve the readability of the subtitles in the live event 1.

[0038] As an alternative approach, if the user is currently playing movie 1, color similarity analysis can be performed on multiple consecutive video frames 2 after alignment processing of movie 1 and the subtitle window 2 in each consecutive video frame. Then, video frames with colors similar to subtitle window 1 can be identified among the multiple consecutive video frames 2 as target video frames in this embodiment. Subtitle window prompt information for movie 1 can then be generated. When the user has seen or is about to see the target video frame, the user is prompted to adjust the subtitle window to improve the readability of the subtitles in movie 1.

[0039] Compared with existing technologies, this embodiment aligns the video stream data and subtitle stream data of the target video to obtain multiple video frames and subtitle windows corresponding to the aligned target video. Then, color similarity analysis can be performed based on these multiple video frames and corresponding subtitle windows to identify target video frames that meet the color similarity criteria. This allows the embodiment to prompt the user to adjust the subtitle window when the user plays the target video and the target video frame. Therefore, if the embodiment detects that the subtitles are similar in color to the video screen during video playback, it can prompt the user to make targeted adjustments to the subtitles, improving readability and thus enhancing the user experience.

[0040] As a refinement and extension of the above embodiments, when executing the "operation prompt information for generating a subtitle window based on the target video frame", this embodiment may also employ the following methods, but is not limited thereto, such as... Figure 2 As shown, the method includes: Step 201: Based on the historical movement information of the subtitle window, determine the target position of the subtitle window after it has moved.

[0041] As an alternative approach, taking live video A as an example, when user a watches live video A for the first time, the video system server analyzes and compares the video frames and subtitles in real time. For example, if the subtitles are detected to have a color conflict with the video screen in the area at 00:05 (exceeding the similarity threshold), the server can push subtitle operation prompts (i.e., operation prompt information in the embodiment of this application) and subtitle window operation tools to user a's terminal device.

[0042] For example, based on the above example, after user A receives the prompt at 00:05, he / she can drag the subtitle to the upper left corner (i.e., the target position in this embodiment of the application). It should be noted that the target position in this embodiment of the application can be any position on the video screen. During the user's zooming process, the length and width of the window where the subtitle is located are limited to not exceed the current video screen, and are aligned according to the subtitle font.

[0043] For example, based on the above example, the user can also adjust the length and width of the subtitle window to convert the subtitle from one line to multiple lines. Then, after a certain period of time (such as 3 seconds) after user a completes the operation, the subtitle selection frame and prompt words can be automatically hidden, leaving only the subtitle text displayed.

[0044] As an alternative approach, based on the above example, the video system server can record user A's actions and, according to the subtitle position area currently set by user A, detect color conflicts between subsequent video frames and subtitles. If any conflicts are found, the server can push subtitle operation prompts and tools to user A again and record user A's actions based on the user A's actions.

[0045] For example, based on the above example, when user A watches the video again, by default, the subtitles and operation prompts will be displayed according to the user A's last operation recorded by the system. If user A initializes the historical configuration, the operation steps in this step will be followed as if user A watched the video for the first time. If user A changes the position or other information of the subtitles again during the viewing period based on the last operation, the system will detect the color conflict between the subsequent video frames and the subtitles according to the latest subtitle position area set by user A. If there is a conflict, the subtitle operation prompts and tools will be pushed to user A again, and user A's operation behavior will be recorded according to the operation behavior.

[0046] As an optional approach, such as Figure 3 As shown, when the video system server pushes subtitle operation prompts and tools to user A, user A can process subtitles on the video client on the terminal device by integrating the subtitle software development kit (SDK). This can include, but is not limited to, the following processing steps: Figure 4 As shown, it includes: Step 1: Taking the live video of the event as an example, when the video transcoding system receives the external live video stream, it needs to transcode the video stream and subtitle stream into a format that can be directly processed by various models within the system, and push it to the corresponding processing module through the RTMP channel. It should be noted that in this embodiment, only the subtitle stream needs to be edited.

[0047] Step 2: When the video system server pushes a subtitle operation prompt to user A, the client SDK can authenticate with the server and then user A's client synchronously loads and displays the integrated subtitle SDK toolbar resources, including functions such as dragging the subtitle area position and modifying the color of subtitle blocks.

[0048] Step 3: Assuming user a selects to drag and lock the subtitle area at the current drag position, the client sends the endpoint of the user's drag to the server and stores the coordinate information of the subtitle area as position information 2 (i.e., the target position in this application implementation). This does not overwrite the original position information of the subtitle area, which is assumed to be position information 1. Similarly, when user a selects to edit the subtitle, the client records the edited subtitle information and sends it to the server for storage as subtitle information 2 (i.e., the target position in this application implementation).

[0049] Step 4: When user A needs to operate on the subtitle area again, such as moving its position, there are two scenarios: First, the user wants to operate on the basis of their previous subtitle editing, clicking on the newly dragged subtitle on the screen to drag and edit it. In this case, the client records the relevant information according to steps 1 and 2 above and pushes it to the server. Second, the user is not used to the changes in subtitle position, color, font size, or style due to personal long-term operating habits. In this case, the server displays the subtitle that the user wants to operate on in the default position based on the original position information 1 and subtitle information 1. At the same time, it resets the user's custom subtitle position, clears the position information 2 (i.e., the target position in this embodiment) and subtitle information 2 recorded in step 2. If the user customizes the subtitle again, the above steps continue to be processed.

[0050] Step 5: After user A finishes processing the subtitles, the processed subtitle stream is obtained. The server establishes two independent data streams—the main video stream and the auxiliary subtitle stream—within the same RTMP connection channel, and performs RTMP multiplexing for push. This maintains protocol efficiency while achieving logical separation, allowing users to process the subtitle stream independently.

[0051] Step 6: The server delivers video streams and auxiliary subtitle streams of different bitrates to the CDN, and the CDN distributes and pushes them to the client based on the user's terminal network / location, etc.

[0052] Step 202: Determine the target area covered by the subtitle window in the target video frame at the target location.

[0053] Step 203: Generate position operation prompts for the subtitle window based on the target location, and generate color operation prompts for the subtitle window based on the target area.

[0054] Optionally, when performing the "generating color operation prompt information for the subtitle window based on the target area", this embodiment may use the following method, but is not limited to it: the method includes: determining at least one target replacement color corresponding to the subtitle in the subtitle window based on the target area; generating color operation prompt information corresponding to the subtitle window based on the at least one target replacement color.

[0055] As an optional approach, the subtitle manipulation tool provided to users in this application embodiment can, in addition to dragging and stretching subtitles, also provide subtitle color adjustment. For example, color recommendations can be made based on the color conflict between the subtitle in the current video frame and the color of the covered image.

[0056] Optionally, when performing the "determining at least one target replacement color corresponding to the subtitle in the subtitle window based on the target area", this embodiment may use the following method, but is not limited thereto: the method includes: determining the brightness data of at least one pixel corresponding to the target area; determining the average brightness and contrast data corresponding to the target area based on the brightness data; and determining at least one target replacement color corresponding to the subtitle in the subtitle window based on the average brightness and contrast data.

[0057] In some examples, the system server extracts the video image features corresponding to the subtitle area set by the user, and analyzes the brightness and contrast of the video image. Among them, the brightness analysis can be to calculate the average brightness value of the current frame and the neighboring area of ​​the video, and then convert the pixel values ​​of the current frame and the neighboring area from the red-green-blue (RGB) space to the hue-saturation-brightness (HSV) space, and then calculate the brightness (V) value of each pixel (i.e. the brightness data of at least one pixel in the embodiments of this application). Finally, the average value of the brightness values ​​of all pixels is calculated as the average brightness value of the current frame and the neighboring area (i.e. the average brightness value in the embodiments of this application).

[0058] For example, based on the above examples, the contrast analysis in this application embodiment can be to calculate the contrast of the current frame of the video and its neighboring regions, calculate the minimum brightness value (determined based on the brightness data of at least one pixel in this application embodiment) and the maximum brightness value (determined based on the brightness data of at least one pixel in this application embodiment) of the pixel values ​​of the current frame and its neighboring regions, and then calculate the contrast (i.e., the contrast data in this application embodiment) using Formula 1, which is shown in detail below: Contrast ratio = (Maximum brightness value - Minimum brightness value) / (Maximum brightness value + Minimum brightness value + ε) (Formula 1) In Formula 1, ε is a very small positive number to prevent the denominator from being 0.

[0059] It should be noted that after the contrast between the subtitle color and the covered image in the current video frame can be calculated using Formula 1 (i.e., the contrast data in this application embodiment), the recommended order of subtitle colors (i.e., at least one target replacement color in this application embodiment) can be arranged according to the contrast from largest to smallest, for example: white -> red -> orange -> green -> blue -> purple. Then, when pushing subtitle operation prompts and tools to user a, white will be displayed first so that the user can choose it first.

[0060] Optionally, when performing the "generate color operation prompt information corresponding to the subtitle window based on at least one target color replacement", this embodiment may adopt the following method, but is not limited thereto. The method includes: dividing the subtitle window into at least one text block; determining the subtitle color and video background color corresponding to each text block in the at least one text block; performing color similarity analysis on each text block based on the subtitle color and video background color, and generating color operation prompt information for each text block in the subtitle window based on the analysis results.

[0061] It should be noted that the embodiments of this application differ from the conventional direct manipulation of video stream subtitle colors. Based on the operation of the subtitle small window, the local area of ​​the subtitle can be optimized to avoid color conflicts between the subtitle and the screen. This reduces the number of times users drag the subtitle position within the same video viewing period. When the number of times users drag is reduced, the overhead of the server needing to re-analyze and compare the video and subtitles each time can also be reduced.

[0062] For example, in this embodiment, the video subtitle coverage area can be divided into multiple small areas (considering user readability, the smallest block is 1-2 Chinese characters, and for English, it is 2-4 English subtitles, which can be adjusted appropriately), and then the color can be adjusted independently for each small area. Specifically, the subtitle coverage area can be divided into several small areas of equal size (i.e., at least one text block in this embodiment). Then, a color conflict analysis between the subtitle and the screen is performed on each small area. Based on the analysis results of each small area, a recommended order of subtitle colors is arranged for each area from largest to smallest contrast (e.g., orange -> red -> blue -> green -> purple). When a local subtitle and screen color conflict occurs, a subtitle adjustment prompt is pushed to the user, prompting the user whether to adjust the color of some subtitles, and the user's operation behavior is recorded based on the user's operation.

[0063] Optionally, after performing the "alignment processing of video stream data and subtitle stream data to obtain multiple video frames corresponding to the target video and subtitle windows corresponding to the multiple video frames", this embodiment may adopt the following method, but is not limited thereto: the method includes: sending the aligned video stream data and subtitle stream data to the client respectively, and the client is used to display the subtitle window corresponding to the subtitle stream data on the upper layer of the video frame screen corresponding to the video stream data.

[0064] For example, after aligning the video stream and subtitle stream by timestamp, the video stream and subtitle stream can be pushed to the video client synchronously and separately, where the subtitles can be displayed in a small window within the corresponding video frame.

[0065] It should be noted that the embodiments of this application can realize windowed management of video subtitles. When the user is not watching the video in full screen, the user is allowed to display the subtitles below the video within the current mobile phone screen window, as shown in the figure below. At this time, the subtitle window can be dragged out of the screen, making the display clearer and completely unobstructed. When the user adopts this method, the screen and subtitles are separated, so it is not necessary to perform real-time analysis and comparison of video frames and subtitles on the video system server in step two, which greatly reduces resource consumption.

[0066] It should be noted that video subtitles translate, explain, or enhance the viewer's understanding by providing textual information. They typically appear at the bottom of the video screen, can be embedded in the video file or be external files, support multiple languages, and assist users with hearing impairments or those watching videos in noisy environments. In related technologies, video subtitle display solutions have continuously evolved with the development of digital media, from early static text to today's dynamic subtitles, including scrolling subtitles, pop-up subtitles, and hidden subtitles, to adapt to different scenarios and viewer needs. However, current video subtitle displays often use fixed colors or user-preset schemes. When the subtitle color is similar to the background, readability decreases, such as when white subtitles are displayed in a light-colored scene. Subtitles cannot adapt to dynamic scenes in real time, such as camera cuts or changes in scene brightness. Related technologies do not consider local interference from the surrounding colors of the subtitle area, such as subtitles overlaid on complex textures. Furthermore, current video subtitle display solutions in the field include fixed color schemes, user-preset schemes, fixed color with shadow schemes, and static adjustment schemes. These schemes rely on manual presets and cannot adapt to changes in dynamic scenes.

[0067] Based on the above, the shortcomings of the related video subtitle display technology are as follows: Although it can adaptively adjust the appearance parameters of the subtitles according to user preferences and environmental conditions, the solution still relies on the collection of user behavior data and environmental data. It may not be able to perceive the local interference of colors around the subtitle area in real time and accurately. For example, when the subtitles are covered on complex textures, it may lead to a decrease in subtitle readability. The position and color of the subtitles cannot adapt to the real-time dynamic changes of the video screen, such as video shot switching and changes in scene brightness, which will cause the subtitles to conflict with the background color and affect the readability of the subtitles. The lack of consideration for the local interference of colors around the subtitle area may lead to a decrease in the contrast between the subtitles and the background when the subtitles are covered on complex textures, affecting the viewing experience.

[0068] It should be noted that this application embodiment can display subtitles separately floating on top of the video. When there is a color conflict between the subtitles and the video screen, the user can drag the position of the subtitles to move, scale, and edit the colors. This solves the problem of synchronization between the video screen and the subtitles. This application embodiment also extracts the image features of the part of the image covered by the subtitles in the current video frame and calculates the color conflict with the background based on the color space model, recommending the most reasonable subtitle color setting options to the user. This application embodiment can also separate the video stream and the subtitle stream, with the subtitles floating on top of the video screen. The subtitles can be separated from the video by dragging a small window, and the subtitles can appear at any position on the video screen according to the user's drag. At the same time, in terms of the subtitle color recommendation method, color recommendations are made based on the color conflict between the subtitle color and the color of the covered image in the current video frame, which can better adapt to real-time dynamic images, such as camera transitions and changes in scene brightness, making it more practical.

[0069] Compared with existing technologies, this embodiment aligns the video stream data and subtitle stream data of the target video to obtain multiple video frames and subtitle windows corresponding to the aligned target video. Then, color similarity analysis can be performed based on these multiple video frames and corresponding subtitle windows to identify target video frames that meet the color similarity criteria. This allows the embodiment to prompt the user to adjust the subtitle window when the user plays the target video and the target video frame. Therefore, if the embodiment detects that the subtitles are similar in color to the video screen during video playback, it can prompt the user to make targeted adjustments to the subtitles, improving readability and thus enhancing the user experience.

[0070] Furthermore, as Figure 1 and Figure 2 The specific implementation of the method shown in this embodiment provides a subtitle adjustment device, such as... Figure 5 As shown, the device includes: an acquisition module 31, an alignment module 32, a determination module 33, and a generation module 34.

[0071] The acquisition module 31 is configured to acquire the video stream data and subtitle stream data corresponding to the target video; Alignment module 32 is configured to align the video stream data and the subtitle stream data to obtain multiple video frames corresponding to the target video and subtitle windows corresponding to the multiple video frames; The determination module 33 is configured to perform color similarity analysis on the plurality of video frames and the subtitle window to determine the target video frame that meets the color similarity condition among the plurality of video frames; The generation module 34 is configured to generate operation prompt information for the subtitle window based on the target video frame. The operation prompt information is used to prompt the subtitle window to be adjusted when the target video plays the target video frame.

[0072] In some examples of this embodiment, the generation module 34 is specifically configured to determine the target position of the subtitle window based on the historical movement information of the subtitle window; determine the target area covered by the subtitle window at the target position in the target video frame; generate position operation prompt information for the subtitle window based on the target position; and generate color operation prompt information for the subtitle window based on the target area.

[0073] In some examples of this embodiment, the generation module 34 is further configured to determine at least one target replacement color corresponding to the subtitle in the subtitle window based on the target region; and generate color operation prompt information corresponding to the subtitle window based on the at least one target replacement color.

[0074] In some examples of this embodiment, the generation module 34 is further configured to: determine the brightness data of at least one pixel corresponding to the target area; determine the average brightness and contrast data corresponding to the target area based on the brightness data; and determine at least one target replacement color corresponding to the subtitle in the subtitle window based on the average brightness and the contrast data.

[0075] In some examples of this embodiment, the generation module 34 is further configured to divide the subtitle window into at least one text block; determine the subtitle color and video background color corresponding to each text block in the at least one text block; perform color similarity analysis on each text block based on the subtitle color and the video background color; and generate color operation prompt information for each text block in the subtitle window based on the analysis results.

[0076] In some examples of this embodiment, the alignment module 32 is also configured to send the aligned video stream data and subtitle stream data to the client respectively. The client is used to display the subtitle window corresponding to the subtitle stream data on top of the video frame corresponding to the video stream data.

[0077] It should be noted that other corresponding descriptions of the functional units involved in the subtitle adjustment device provided in this embodiment can be found in [reference needed]. Figure 1 and Figure 2 The corresponding descriptions in [the document] will not be repeated here.

[0078] Based on the above, Figure 1 and Figure 2Accordingly, this embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. Figure 1 and Figure 2 The method shown.

[0079] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0080] like Figure 6 The diagram shown is a hardware structure schematic of an electronic device according to the present invention, comprising: At least one processor 401; and, A memory 402 is communicatively connected to at least one of the processors 401; wherein, The memory 402 stores instructions that can be executed by at least one of the processors to enable the at least one of the processors to perform the subtitle adjustment method as described above.

[0081] Figure 6 Take a processor 401 as an example.

[0082] The electronic device may also include an input device 403 and a display device 404.

[0083] The processor 401, memory 402, input device 403, and display device 404 can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0084] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the subtitle adjustment method in the embodiments of this application, for example, Figure 1 and Figure 2 The method flow is shown. The processor 401 executes various functional applications and data processing by running non-volatile software programs, instructions, and modules stored in the memory 402, thereby implementing the subtitle adjustment method in the above embodiments.

[0085] Memory 402 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created according to the use of the subtitle adjustment method, etc. Furthermore, memory 402 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 402 may optionally include memory remotely located relative to processor 401, and these remote memories may be connected via a network to the apparatus performing the subtitle adjustment method. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0086] Input device 403 can receive user clicks and generate signal inputs related to user settings and function control for subtitle adjustment methods. Display device 404 may include display devices such as a display screen.

[0087] When one or more modules are stored in the memory 402, and are run by one or more processors 401, the subtitle adjustment method in any of the above method embodiments is executed.

[0088] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0089] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0090] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0091] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms, or it can be implemented by hardware. By applying the solution of this embodiment, compared with the prior art, this embodiment aligns the video stream data and subtitle stream data of the target video to obtain multiple video frames corresponding to the target video and subtitle windows corresponding to the multiple video frames after alignment. Then, color similarity analysis can be performed based on the multiple video frames and the corresponding subtitle windows to determine the target video frames that meet the color similarity conditions. Then, when the user plays the target video frame, the user can be prompted to adjust the subtitle window. This embodiment can prompt the user to make targeted adjustments to the subtitles if it is found that the subtitles are similar in color to the video screen during video playback, thereby improving the readability of the subtitles and improving the user experience.

[0092] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0093] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for adjusting subtitles, characterized in that, include: Obtain the video stream data and subtitle stream data corresponding to the target video; The video stream data and the subtitle stream data are aligned to obtain multiple video frames corresponding to the target video and subtitle windows corresponding to the multiple video frames. A color similarity analysis is performed on the plurality of video frames and the subtitle window to determine the target video frame that meets the color similarity condition among the plurality of video frames; Operation prompts for generating the subtitle window based on the target video frame are used to prompt adjustments to the subtitle window when the target video plays the target video frame.

2. The method according to claim 1, characterized in that, The operation prompt information for generating the subtitle window based on the target video frame includes: Based on the historical movement information of the subtitle window, the target position of the subtitle window is determined; Determine the target area covered by the subtitle window at the target location in the target video frame; The system generates position operation prompts for the subtitle window based on the target location, and generates color operation prompts for the subtitle window based on the target area.

3. The method according to claim 2, characterized in that, Based on the target area, color operation prompts for the subtitle window are generated, including: Based on the target area, at least one target replacement color corresponding to the subtitle in the subtitle window is determined; Based on the at least one target color replacement, generate color operation prompt information corresponding to the subtitle window.

4. The method according to claim 3, characterized in that, Determining at least one target replacement color corresponding to the subtitle in the subtitle window based on the target region includes: Determine the brightness data of at least one pixel corresponding to the target region; Based on the brightness data, determine the average brightness and contrast data corresponding to the target area; Based on the average brightness and the contrast data, at least one target replacement color corresponding to the subtitle in the subtitle window is determined.

5. The method according to claim 3, characterized in that, The step of generating color operation prompt information corresponding to the subtitle window based on the at least one target color replacement includes: The subtitle window is divided into at least one text block; Determine the subtitle color and video background color corresponding to each text block in the at least one text block; Based on the subtitle color and the video background color, a color similarity analysis is performed on each text block, and color operation prompt information for each text block in the subtitle window is generated based on the analysis results.

6. The method according to any one of claims 1 to 5, characterized in that, After aligning the video stream data and the subtitle stream data to obtain multiple video frames corresponding to the target video and subtitle windows corresponding to the multiple video frames, the method further includes: The aligned video stream data and subtitle stream data are sent to the client respectively. The client is used to display the subtitle window corresponding to the subtitle stream data on top of the video frame corresponding to the video stream data.

7. A subtitle adjustment device, characterized in that, include: The acquisition module is configured to acquire the video stream data and subtitle stream data corresponding to the target video. The alignment module is configured to align the video stream data and the subtitle stream data to obtain multiple video frames corresponding to the target video and subtitle windows corresponding to the multiple video frames. The determination module is configured to perform color similarity analysis on the plurality of video frames and the subtitle window to determine the target video frame that meets the color similarity condition among the plurality of video frames; The generation module is configured to generate operation prompt information for the subtitle window based on the target video frame. The operation prompt information is used to prompt adjustments to the subtitle window when the target video plays the target video frame.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

9. An electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 6.

10. A computer program product, the computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images