Subtitle display method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a subtitle display method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: in response to an update display request for a currently played target video, obtaining a subtitle area of an original video frame matched with the update display request and a target area of the original video frame, the target area being a mapping area of updated and displayed content in a playing interface in the original video frame; identifying the original video frame to obtain an overlapping area of the target area and the caption area, and obtaining the display integrity of the caption in the target area based on the overlapping area and the caption area; and if the display integrity meets the condition of embedded display, embedding and displaying the subtitle content in the subtitle area in an image picture corresponding to the update display request in the playing interface. By adopting the method, the subtitle content in the video file can be completely presented when the image is updated and displayed.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a subtitle display method, device, computer equipment, storage medium and program product. Background Art

[0002] With the rapid development of the video field, video frames often present rich content in the form of pictures and texts. According to the embedding form of text subtitles in the image, subtitles can be divided into hard-pressed subtitles and external subtitles.

[0003] Taking hard-compressed subtitles as an example, in traditional technology, subtitles and video content are compressed into one video file. In this way, when playing each frame of the video file, the image itself carries the subtitle content, and the subtitles and the picture can be presented at the same time. However, when displaying the image in the video file, if the image is updated and displayed, the subtitle content in the video file may not be fully presented. Summary of the invention

[0004] Based on this, it is necessary to provide a subtitle display method, device, computer equipment, storage medium and program product that can fully present the subtitle content in the video file when the image is updated and displayed to address the above technical problems.

[0005] The present application provides a subtitle display method. The method comprises:

[0006] In response to an update display request for a target video currently being played, obtaining a subtitle area of ​​an original video frame and a target area of ​​the original video frame that match the update display request, wherein the target area is a mapping area of ​​the content displayed after the update in the playback interface in the original video frame;

[0007] Identify the original video frame to obtain an overlapping area between the target area and the subtitle area, and obtain a display completeness of the subtitle in the target area based on the overlapping area and the subtitle area;

[0008] If the display completeness satisfies the embedded display condition, the subtitle content in the subtitle area is embedded and displayed in the image picture corresponding to the update display request in the playback interface.

[0009] The present application also provides a subtitle display device. The device comprises:

[0010] A request response module, configured to respond to an update display request for a target video currently being played, and obtain a subtitle area of ​​an original video frame that matches the update display request and a target area of ​​the original video frame, wherein the target area is a mapping area of ​​the content displayed after the update in the playback interface in the original video frame;

[0011] A video frame recognition module, used to recognize the original video frame, obtain the overlapping area between the target area and the subtitle area, and obtain the display completeness of the subtitle in the target area based on the overlapping area and the subtitle area;

[0012] The subtitle display module is used to embed the subtitle content in the subtitle area into the image screen corresponding to the update display request in the playback interface when the display completeness meets the embedded display condition.

[0013] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned subtitle display method when executing the computer program.

[0014] A computer-readable storage medium stores a computer program, which implements the steps of the above-mentioned subtitle display method when executed by a processor.

[0015] A computer program product includes a computer program, which implements the steps described in the above-mentioned subtitle display method when executed by a processor.

[0016] The above-mentioned subtitle display method, device, computer equipment, storage medium and program product, in response to an update display request for the target video currently being played, obtains the subtitle area of ​​the original video frame matching the update display request and the target area of ​​the original video frame, the target area being the mapping area of ​​the content displayed after the update in the playback interface in the original video frame; identifies the original video frame, obtains the overlapping area of ​​the target area and the subtitle area, and obtains the display completeness of the subtitle in the target area based on the overlapping area and the subtitle area; if the display completeness meets the conditions for embedded display, the subtitle content in the subtitle area is embedded in the image screen corresponding to the update display request in the playback interface. In this way, when the screen in the target video is updated and displayed, on the basis of obtaining the target area of ​​the content displayed after the update in the playback interface and the subtitle area of ​​the original video frame, the original video frame is identified to determine whether there is subtitle occlusion in the target area. Further, in the case of subtitle occlusion in the target area, the overlapping area of ​​the subtitle area and the target area is determined, and based on the overlapping area and the subtitle area, the display completeness of the subtitle in the target area is obtained as a quantitative indicator parameter of the degree of subtitle occlusion in the target area. Then, when the display completeness meets the conditions for embedded display, the complete subtitle content in the subtitle area is embedded in the image screen corresponding to the update display request and displayed in the playback interface, which is equivalent to that no matter how the target area of ​​the original video frame is updated and displayed in the playback interface, the content presented in the playback interface includes the complete subtitle information of the original video frame. In addition, the step of embedding the subtitle content is performed when the display completeness meets the conditions for embedded display, and the conditions for embedded display in the embodiment of the present application can be flexibly set based on different application scenarios to meet the needs of complete subtitle display in different application scenarios, and to a certain extent improve the scenario applicability of the subtitle display method. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] FIG1 is a diagram showing an application environment of a subtitle display method according to an embodiment;

[0018] FIG2 is a schematic diagram of a flow chart of a subtitle display method in one embodiment;

[0019] FIG3 is a schematic diagram of triggering a zoom display request in a currently playing video frame in one embodiment;

[0020] FIG4 is a schematic diagram of triggering a zoom display request in a currently playing video frame in another embodiment;

[0021] FIG5 is a schematic diagram of the position of a reference coordinate system in an original video frame in one embodiment;

[0022] FIG6 is a schematic diagram of overlapping areas in an original video frame in one embodiment;

[0023] FIG7 is a schematic diagram of overlapping areas in an original video frame in another embodiment;

[0024] FIG8 is a schematic diagram showing a completeness of 100% in one embodiment;

[0025] FIG9 is a flow chart of a subtitle display method in another embodiment;

[0026] FIG10 is a comparative schematic diagram of non-global display in one embodiment;

[0027] FIG11 is a block diagram of a subtitle display device in one embodiment;

[0028] FIG12 is a diagram showing the internal structure of a computer device in one embodiment;

[0029] FIG. 13 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0031] The subtitle display method provided in the embodiment of the present application can be applied in the application environment shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, can be integrated on the server 104, or can be placed on the cloud or other servers. Among them, the terminal 102 can be but is not limited to various desktop computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers, or it can be a cloud server.

[0032] Both the terminal and the server can be used independently to execute the subtitle display method provided in the embodiments of the present application.

[0033] For example, in response to an update display request for a target video currently being played, the terminal obtains the subtitle area of ​​the original video frame and the target area of ​​the original video frame that match the update display request, where the target area is a mapping area of ​​the content displayed in the playback interface after being updated in the original video frame, and then identifies the original video frame to obtain an overlapping area between the target area and the subtitle area, and based on the overlapping area and the subtitle area, obtains the display completeness of the subtitles in the target area, and when the display completeness meets the conditions for embedded display, embeds the subtitle content in the subtitle area into an image screen corresponding to the update display request in the playback interface and displays it.

[0034] The terminal and the server can also be used together to execute the subtitle display method provided in the embodiments of the present application.

[0035] For example, the terminal obtains the target video from the server, and in response to the update display request for the currently playing target video, obtains the subtitle area of ​​the original video frame and the target area of ​​the original video frame that match the update display request, and the target area is the mapping area of ​​the content displayed in the playback interface after the update in the original video frame. The server identifies the original video frame, obtains the overlapping area of ​​the target area and the subtitle area, obtains the display completeness of the subtitles in the target area based on the overlapping area and the subtitle area, and when the display completeness meets the embedded display condition, embeds the subtitle content in the subtitle area into the image screen corresponding to the update display request in the playback interface. The server sends the image screen with the embedded subtitle content to the terminal, and the terminal displays the received image screen in the playback interface.

[0036] In one embodiment, as shown in FIG2 , a subtitle display method is provided, and the method is applied to a computer device as an example for explanation. The computer device may be a terminal or a server, and the method may be executed by the terminal or the server itself alone, or may be implemented through interaction between the terminal and the server. Referring to FIG2 , the subtitle display method includes the following steps:

[0037] Step S202, in response to an update display request for the currently playing target video, obtain the subtitle area of ​​the original video frame and the target area of ​​the original video frame that match the update display request, where the target area is a mapping area of ​​the content displayed after the update in the playback interface in the original video frame.

[0038] The target video is a video stream including multiple continuous video frames, which can be played continuously and in real time on the playback interface; the target video can be a TV series, a movie, a live video stream, a game video stream, etc.

[0039] The update display request of the target video refers to the triggering of the processing operation of the target video based on the display demand, including but not limited to the zoom display request triggered by the equal-proportional zoom operation, the zoom display request triggered by the custom parameter, the sliding display request triggered by the interest area display operation, the zoom display request triggered by the equal-proportional zoom operation, etc. In the embodiment of the present application, there is no restriction on the triggering subject of the update display request, for example, the update display request is triggered by the user, or the update display request is automatically triggered by the server adaptive playback request corresponding to the playback interface.

[0040] In actual applications, the update display request may be triggered when the player is not playing the target video. In this scenario, the original video frame refers to any video frame in the target video. It may also be triggered on the currently playing video frame during the playing of the target video. In this scenario, the original video frame refers to the currently playing video frame, or the backward video frame to be played after the currently playing video frame.

[0041] Taking the update display request as an enlarged display request triggered on the currently playing video frame as an example, the method of obtaining the subtitle area and target area of ​​the original video frame is explained: in response to the enlarged display request for the currently playing video frame, the content displayed after the enlarged update in the playback interface is determined, and then the content displayed after the enlarged update in the playback interface is matched with the content in the original video frame to determine the target area from the original video frame. In addition, a text recognition algorithm is used to detect the subtitle area in the original video frame to determine the subtitle area in the original video frame.

[0042] Taking the update display request as a sliding display request triggered on the currently playing video frame as an example, the method of obtaining the subtitle area and target area of ​​the original video frame is explained: in response to the sliding display request for the currently playing video frame, the content displayed after the sliding update in the playback interface is determined, and then the content displayed after the sliding update in the playback interface is matched with the content in the original video frame, and the target area is determined from the original video frame. In addition, a text recognition algorithm is used to detect the subtitle area in the original video frame to determine the subtitle area in the original video frame.

[0043] It should be noted that, considering that in the scenario where the update display request is a reduction display request, there is no need to execute embedded subtitle content, and there is no need to calculate the subtitle area, in an embodiment of the present application, the step of obtaining the subtitle area of ​​the original video frame can be performed after determining that the type of the update display request is an enlargement display request or a movement display request, so as to avoid the waste of computing resources caused by calculating the subtitle area in the scenario of the reduction display request.

[0044] Of course, as an optional embodiment, when computing resources are sufficient or the time segment of the target video is short, a text recognition algorithm can be used in advance to obtain the subtitle area of ​​each video frame from the target video before responding to the update display request, so as to improve the response speed of the update display request and improve the subtitle display efficiency.

[0045] Step S204, identifying the original video frame, obtaining the overlapping area between the target area and the subtitle area, and obtaining the display completeness of the subtitles in the target area based on the overlapping area and the subtitle area.

[0046] The original video frame includes a target area and at least one subtitle area, that is, the original video frame may include multiple subtitle areas. In the case where the original video frame includes multiple subtitle areas, the overlap area refers to multiple overlap areas between the target area and each subtitle area. In the embodiment of the present application, the size of the overlap area is less than or equal to the size of the subtitle area, and the size of the overlap area is less than or equal to the size of the target area.

[0047] After determining the subtitle area of ​​the original video frame and the target area of ​​the original video frame, the original video frame is identified to determine the position information of the subtitle area and the position information of the target area, and based on the position information of the subtitle area and the position information of the target area, the image portion distributed simultaneously in the subtitle area and the target area is determined as an overlapping area.

[0048] Furthermore, the feature parameters of the overlapped area are matched with the feature parameters of the subtitle area, and the ratio of the feature parameters of the overlapped area to the feature parameters of the subtitle area is calculated to obtain the display completeness of the subtitle in the target area. The subtitle refers to the complete text part of the subtitle area in the original video frame, and the display completeness of the subtitle in the target area can be represented in the form of a percentage, specifically a value of [0, 100%], such as 0%, 15%, 58.4%, 100%, etc.

[0049] In an optional embodiment, taking the characteristic parameter as the number of pixels as an example, the ratio of the number of pixels in the overlapped area to the number of pixels in the subtitle area is calculated to determine the display completeness of the subtitles in the target area. Alternatively, taking the characteristic parameter as the pixel area as an example, the ratio of the pixel area in the overlapped area to the pixel area in the subtitle area is calculated to determine the display completeness of the subtitles in the target area.

[0050] Step S206: If the display completeness satisfies the embedded display condition, the subtitle content in the subtitle area is embedded and displayed in the image screen corresponding to the update display request in the playback interface.

[0051] The embedded display condition refers to the upper limit requirement for allowing the playback interface to block the subtitle content, which can be a lower limit threshold of the display completeness set in advance according to the target video playback scenario.

[0052] For example, in a scenario where a user is watching a live broadcast, the condition for embedded display can be set to a display completeness of less than 0%, that is, when no subtitles are displayed in the target area, embedded subtitle display is performed; in a TV drama playback scenario, the condition for embedded display can be set to a display completeness of less than 80%, that is, when 80% or more of the subtitles are not displayed in the target area, embedded subtitle display is performed; in a sports event playback scenario, the condition for embedded display can be set to a display completeness of less than 100%, that is, when the complete subtitles cannot be presented in the target area, embedded subtitle display is performed.

[0053] In the embodiment of the present application, the display completeness meets the conditions for embedded display, which indicates that the updated content displayed in the playback interface is blocked by subtitles, that is, the subtitles in the original video frame "extend" the playback interface, resulting in the inability to display complete subtitles in the playback interface. Based on this, in order to ensure the integrity of the subtitle content in the playback interface, the subtitle content in the subtitle area can be re-embedded into the image screen corresponding to the update display request.

[0054] In another scenario, if the display completeness does not meet the conditions for embedded display, it means that the current target area does not need to embed subtitles. In this case, the target area can be rendered so that the content of the target area fills the playback interface, and an image corresponding to the update display request is displayed in the playback interface.

[0055] In the above-mentioned subtitle display method, when updating and displaying the screen in the target video, the original video frame is identified on the basis of obtaining the target area of ​​the content displayed after the update in the playback interface in the original video frame and the subtitle area of ​​the original video frame, and judging whether there is subtitle occlusion in the target area. Further, in the case where there is subtitle occlusion in the target area, the overlapping area of ​​the subtitle area and the target area is determined, and based on the overlapping area and the subtitle area, the display completeness of the subtitle in the target area is obtained as a quantitative indicator parameter of the degree of subtitle occlusion in the target area. Then, in the case where the display completeness meets the conditions for embedded display, the complete subtitle content in the subtitle area is embedded in the image screen corresponding to the update display request, and displayed in the playback interface, which is equivalent to no matter how the target area of ​​the original video frame is updated and displayed in the playback interface, the content presented in the playback interface includes the complete subtitle information of the original video frame. In addition, the step of performing subtitle content embedding is performed when the display completeness meets the conditions for embedded display, and the conditions for embedded display in the embodiment of the present application can be flexibly set based on different application scenarios to meet the needs of complete display of subtitles in different application scenarios, and to a certain extent, improve the scene applicability of the subtitle display method.

[0056] The above embodiment does not limit the type of update display request, which may be a zoom display request, a sliding display request, etc. For different types of update display requests, the method of obtaining the target area of ​​the original video frame matching the update display request is also different.

[0057] In one embodiment, obtaining a target area of ​​an original video frame includes:

[0058] The target position point of the updated content displayed in the playback interface is determined as the update center; and the target area is determined in the original video frame according to the update center and the update parameters carried in the update display request.

[0059] Among them, the target position point refers to the middle pixel point of the content displayed in the playback interface after updating. In the embodiment of the present application, the position point of the target position point in the original video frame is determined as the update center, which is also the center point of the target area.

[0060] Optionally, the update display request is an enlarged display request for updating the display according to a preset ratio. For a scenario in which the update display request is an enlarged display request according to a preset ratio, the update center is the central pixel point of the original video frame, and the update parameter carried in the update display request is the magnification factor, which can be a positive number greater than 1, such as 1.2 times, 1.4 times, 2 times, etc.

[0061] When determining the target area, the central pixel point of the original video frame is used as the central point of the target area, and the ratio between the pixel area of ​​the original video frame and the pixel area of ​​the target area is used as the update parameter to frame the target area from the original video frame.

[0062] Optionally, the update display request is a sliding display request, the update center refers to the middle pixel point of the content displayed after the sliding update in the playback interface, and the default update parameter is 1. This is because the sliding display request involves the movement of the image content in the playback interface, but does not involve the enlargement or reduction of the original video frame, so there is no need to set the update parameter.

[0063] When determining the target area, the content displayed in the playback interface after sliding update is matched with the content in the original video frame, the picture area of ​​the content displayed in the playback interface in the original video frame is determined, and the determined picture area is determined as the target area.

[0064] In the embodiment of the present application, the target position point of the content displayed after the update in the playback interface is determined as the update center, and the target area is determined in the original video frame according to the update center and the update parameters carried in the update display request. In this way, the center point of the target area is determined from the original video frame according to the update center, and the size of the original video frame is deformed according to the update parameters to map the picture corresponding to the updated content displayed in the playback interface to the original video frame, thereby quickly and accurately determining the target area.

[0065] In practical applications, the update display request can be triggered in a variety of ways, such as the sliding trigger in the above embodiment, the selection update parameter trigger, etc. Next, a feasible way to obtain the target area when the update display request is triggered by two-finger sliding is described.

[0066] In one embodiment, the update display request is a zoom display request; obtaining a target area of ​​the original video frame includes:

[0067] In response to an operation on a target video, a zoom-in display request is generated; based on an operation position corresponding to the zoom-in display request, a region to be zoomed in on the original video frame is determined, and the region to be zoomed in is determined as a target region.

[0068] The enlarged display request may be generated in response to a user placing two fingers on the playback interface and then sliding the two fingers in relative directions. A complete enlarged display request corresponds to two initial operation points, at least one termination operation point, and at least one sliding path, wherein the initial operation point and the termination operation point do not completely overlap.

[0069] For example, as shown in FIG3, FIG3 is a schematic diagram of a zoom display request in the process of playing a current video frame on the playback interface. The user places two fingers on the playback interface to generate two initial operation points, which are indicated by two vertices of a dotted box in FIG3. Then the user slides the two fingers in two relative directions to generate two sliding paths, and the end points of the two sliding paths are two termination operation points. In this case, a complete zoom display request corresponds to two initial operation points, two termination operation points, and two sliding paths.

[0070] For example, as shown in FIG4, FIG4 is another schematic diagram of a zoom display request in the process of playing a current video frame. The user places two fingers on the playback interface to generate a first initial operation point and a second initial operation point. Then, the first initial operation point is kept unchanged, and the second initial operation point is used as the starting point of the sliding path. The sliding is performed in a direction away from the first initial operation point to generate a sliding path and obtain a termination operation point. In this case, a complete zoom display request corresponds to two initial operation points, two termination operation points (one is the first initial operation point that remains unchanged, and the other is the end point of the sliding path), and a sliding path.

[0071] It should be noted that the enlarged display request generated based on the two operation points is not limited to being performed by the user using two fingers, but can also be obtained by operating a stylus on the playback interface.

[0072] In an embodiment of the present application, the operation position corresponding to the enlarged display request is a combination of two operation points, such as two initial operation points, or two termination operation points; the area to be enlarged is an area determined according to the two operation points, such as a rectangular area constructed with the two operation points as vertices, a circular area constructed with the distance between the two operation points as the diameter, and the two operation points are at the boundary of the circular area.

[0073] Based on the operation position corresponding to the enlarged display request, the area covered by the operation position in the playback interface is determined, the area covered by the operation position is determined as the area to be enlarged for the original video frame, and the area to be enlarged is further determined as the target area.

[0074] In an embodiment of the present application, in response to an operation on a target video, an enlarged display request is generated, and based on the operation position corresponding to the enlarged display request, the area to be enlarged for the original video frame is determined, and the area to be enlarged is determined as the target area. In this way, the area to be enlarged can be intuitively determined from the original video frame according to the operation position corresponding to the enlarged display request generation process, and then the target area is obtained, thereby improving the accuracy of the target area. In addition, the generation process of the enlarged display request is a dynamic process, involving multiple operation points, so that multiple operation positions can be selected to obtain the area to be enlarged that meets different requirements, and the method of determining the target area in this way is also more flexible.

[0075] After determining the subtitle area and the target area, the original video frame needs to be further identified to determine whether the subtitle area and the target area overlap. If so, the overlapping area of ​​the subtitle area and the target area needs to be further determined.

[0076] In one embodiment, the original video frame is identified to obtain the overlapping area of ​​the target area and the subtitle area, including:

[0077] In a reference coordinate system determined based on the original video frame, first position information corresponding to the target area and second position information corresponding to the subtitle area are obtained; according to the first position information and the second position information, the intersection coordinates of the target area and the subtitle area are determined, and according to the intersection coordinates, the overlapping area is determined.

[0078] Among them, the reference coordinate system is a plane coordinate system, and the construction process of the reference coordinate system can be: determine any pixel point of the original video frame as the coordinate origin of the reference coordinate system, determine two mutually perpendicular directions in the original video frame as two coordinate axes of the reference coordinate system, and construct the reference coordinate system based on the coordinate origin and each coordinate axis. Usually, in order to facilitate calculation, the reference coordinate system can be constructed in the manner of Figure 5, which is a schematic diagram of the reference coordinate system and the original video frame, and the lower left corner of the original video frame is set as the coordinate origin, the direction of the horizontal axis of the original video frame is set as the horizontal axis of the reference coordinate system, and the direction of the vertical axis of the original video frame is set as the vertical axis of the reference coordinate system.

[0079] After the reference coordinate system is constructed, the first position information corresponding to the target area and the second position information corresponding to the subtitle area can be further determined. In the embodiment of the present application, the first position information and the second position information can be directly represented by the pixel coordinates in the reference coordinate system, or by the relative positions of the target area and the subtitle area in the original video frame.

[0080] According to the first position information and the second position information, the coordinates of the intersection of the target area and the subtitle area are determined as the first vertex coordinate set of the overlapping area. Then, according to the relative positions of the target area and the subtitle area and the relative sizes of the target area and the subtitle area, the vertex coordinates corresponding to the target area in the overlapping area or the vertex coordinates corresponding to the subtitle area in the overlapping area are determined as the second vertex coordinate set of the overlapping area. Then, the corresponding closed area is determined according to the first vertex coordinate set and the second vertex coordinate set as the overlapping area.

[0081] Please refer to Figure 6, which is a schematic diagram of the overlapping area. In Figure 6, the target area is above the subtitle area, and the pixel width of the target area is smaller than the pixel width of the subtitle area. In this case, the two intersection points of the target area and the subtitle area (indicated by squares in Figure 6), the lower left corner coordinates of the target area (indicated by a circle in Figure 6), and the lower right corner coordinates of the target area (indicated by a circle in Figure 6) are determined as the four vertex coordinates of the overlapping area, and the rectangular area corresponding to the four vertex coordinates is determined as the overlapping area.

[0082] Please refer to Figure 7, which is a schematic diagram of the overlapping area. In Figure 7, the target area is above the subtitle area, and the pixel width of the target area is greater than the pixel width of the subtitle area. In this case, the two intersection points of the target area and the subtitle area (indicated by squares in Figure 7), the upper left corner coordinates of the subtitle area (indicated by a circle in Figure 7), and the upper right corner coordinates of the subtitle area (indicated by a circle in Figure 7) are determined as the four vertex coordinates of the overlapping area, and the rectangular area corresponding to the four vertex coordinates is determined as the overlapping area.

[0083] In another playback scenario, the target area is on the left side of the subtitle area, and the pixel height of the target area is greater than the pixel height of the subtitle area. In this case, the two intersection points of the target area and the subtitle area, the upper left corner coordinates of the subtitle area, and the right small corner coordinates of the subtitle area are determined as the four vertex coordinates of the overlapping area, and the rectangular area corresponding to the four vertex coordinates is determined as the overlapping area.

[0084] In the embodiment of the present application, in the reference coordinate system determined based on the original video frame, the first position information corresponding to the target area and the second position information corresponding to the subtitle area are obtained, and the intersection coordinates of the target area and the subtitle area are quickly determined based on the first position information and the second position information, and the overlapping area is determined based on the intersection coordinates. In this way, in the same reference coordinate system, with the pixel point as the smallest unit, the first position information corresponding to the target area and the second position information corresponding to the subtitle area are accurately determined, and then the coordinates of each vertex of the overlapping area including the intersection coordinates are determined, which provides a true and reliable coordinate basis for the determination of the overlapping area and subsequent calculations.

[0085] As can be seen from the above embodiments, the first position information of the target area and the second position information of the subtitle area can be represented in the form of pixel coordinates, or can be represented by the relative positions of the target area and the subtitle area in the original video frame. Next, an embodiment is used to illustrate an achievable method of using relative positions to represent position information.

[0086] In one embodiment, in a reference coordinate system determined based on an original video frame, obtaining first position information of a target area and second position information of a subtitle area includes:

[0087] In a reference coordinate system determined by the original video frame, a first vertex coordinate set of the original video frame, a second vertex coordinate set of the target area, and a third vertex coordinate set of the subtitle area are obtained; a first relative coordinate set of the target area is determined according to the first vertex coordinate set and the second vertex coordinate set, and first position information is determined based on the first relative coordinate set; and a second relative coordinate set of the subtitle area is determined according to the first vertex coordinate set and the third vertex coordinate set, and second position information is determined based on the second relative coordinate set.

[0088] The coordinate forms in the first vertex coordinate set, the second vertex coordinate set and the third vertex coordinate set are represented by pixel coordinates. The first vertex coordinate set includes four vertex coordinates of the original video frame in the reference coordinate system, the second vertex coordinate set includes four vertex coordinates of the target area in the reference coordinate system, and the third vertex coordinate set includes four vertex coordinates of the subtitle area in the reference coordinate system.

[0089] The first relative coordinate set represents the relative position of the target area in the original video frame and can be expressed in percentage. The first relative coordinate set includes the vertex coordinates of two vertices in any diagonal direction of the target area, or can be the coordinates of four vertices of the target area.

[0090] The first relative coordinate set is obtained in the following manner: obtaining a first horizontal axis distance between the minimum horizontal axis coordinate in the first vertex coordinate set and the minimum horizontal axis coordinate in the second vertex coordinate set, a second horizontal axis distance between the maximum horizontal axis coordinate in the first vertex coordinate set and the minimum horizontal axis coordinate in the second vertex coordinate set, a first vertical axis distance between the minimum vertical axis coordinate in the first vertex coordinate set and the minimum vertical axis coordinate in the second vertex coordinate set, and a second vertical axis distance between the maximum vertical axis coordinate in the first vertex coordinate set and the minimum vertical axis coordinate in the second vertex coordinate set; determining the ratio between the first horizontal axis distance and the horizontal axis pixels of the original video frame as the first horizontal axis coordinate of the target area; and determining the ratio between the second horizontal axis distance and the horizontal axis pixels of the original video frame as the second horizontal axis coordinate of the target area. ; The ratio between the first longitudinal axis distance and the longitudinal axis pixels of the original video frame is determined as the first longitudinal axis coordinate of the target area; The ratio between the second longitudinal axis distance and the longitudinal axis pixels of the original video frame is determined as the second longitudinal axis coordinate of the target area; According to the first transverse axis coordinate of the target area (expressed as M1), the second transverse axis coordinate of the target area (expressed as M2), the first longitudinal axis coordinate of the target area (expressed as N1) and the second longitudinal axis coordinate of the target area (expressed as N2), the four vertex coordinates of the target area in the original video frame are obtained, which are (M1, N1), (M1, N2), (M2, N1), (M2, N2), that is, the first relative coordinate set, and then the first relative coordinate set is determined as the first position information of the target area.

[0091] The second relative coordinate set represents the relative position of the subtitle area in the original video frame and can be expressed in percentage. The second relative coordinate set includes the vertex coordinates of two vertices in any diagonal direction of the subtitle area, or can be the coordinates of four vertices of the subtitle area.

[0092] The second relative coordinate set is obtained by: obtaining a third horizontal axis distance between the minimum horizontal axis coordinate in the first vertex coordinate set and the minimum horizontal axis coordinate in the third vertex coordinate set, a fourth horizontal axis distance between the maximum horizontal axis coordinate in the first vertex coordinate set and the minimum horizontal axis coordinate in the third vertex coordinate set, a third vertical axis distance between the minimum vertical axis coordinate in the first vertex coordinate set and the minimum vertical axis coordinate in the third vertex coordinate set, and a fourth vertical axis distance between the maximum vertical axis coordinate in the first vertex coordinate set and the minimum vertical axis coordinate in the third vertex coordinate set;

[0093] Determine the ratio between the third horizontal axis distance and the horizontal axis pixels of the original video frame as the first horizontal axis coordinate of the subtitle area, determine the ratio between the fourth horizontal axis distance and the horizontal axis pixels of the original video frame as the second horizontal axis coordinate of the subtitle area, determine the ratio between the third vertical axis distance and the vertical axis pixels of the original video frame as the first vertical axis coordinate of the subtitle area, and determine the ratio between the fourth vertical axis distance and the vertical axis pixels of the original video frame as the second vertical axis coordinate of the subtitle area;

[0094] According to the first horizontal axis coordinate of the subtitle area (expressed as X1), the second horizontal axis coordinate of the subtitle area (expressed as X2), the first vertical axis coordinate of the subtitle area (expressed as Y1) and the second vertical axis coordinate of the subtitle area (expressed as Y2), the four vertex coordinates of the subtitle area in the original video frame are obtained, which are (X1, Y1), (X1, Y2), (X2, Y1), and (X2, Y2), that is, the second relative coordinate set, and then the second relative coordinate set is determined as the second position information of the subtitle area.

[0095] In an embodiment of the present application, in a reference coordinate system determined by the original video frame, a first vertex coordinate set of the original video frame, a second vertex coordinate set of the target area, and a third vertex coordinate set of the subtitle area are obtained; a first relative coordinate set of the target area is determined based on the first vertex coordinate set and the second vertex coordinate set, and first position information is determined based on the first relative coordinate set; and a second relative coordinate set of the subtitle area is determined based on the first vertex coordinate set and the third vertex coordinate set, and second position information is determined based on the second relative coordinate set. Since the first position information represents the relative position of the target area in the original video frame, and the second position information represents the relative position of the subtitle area in the original video frame, even if the reference coordinate system changes, it will not affect the specific contents of the first position information and the second position information, and there is no need to recalculate the first position information and the second position information, thus saving computing resources.

[0096] In addition, after obtaining the first position information and the second position information represented by relative positions, the third position information of the overlapping area determined according to the first position information and the second position information is also represented by the relative position of the overlapping area relative to the original video frame, so as to quickly and accurately locate the overlapping area in the original video frame.

[0097] In the process of identifying the original video frame, if there is no overlapping area between the target area and the subtitle area, it means that there is no subtitle content in the target area at all, and the subtitle embedding step needs to be performed. If there is an overlapping area between the target area and the subtitle area, it means that there may be subtitle occlusion in the target area, or there may not be subtitle occlusion. At this time, it is necessary to determine the completeness of the subtitle display based on the overlapping area and the subtitle area, and further judge the degree of subtitle occlusion in the target area. The following describes the feasible method of obtaining the display completeness when there is an overlapping area between the target area and the subtitle area.

[0098] In one embodiment, based on the overlapped area and the subtitle area, obtaining the display completeness of the subtitles in the target area includes:

[0099] In a reference coordinate system determined based on the original video frame, the overlapped pixel area of ​​the overlapped area and the subtitle pixel area of ​​the subtitle area are obtained; and the ratio of the overlapped pixel area to the subtitle pixel area is determined as the display completeness of the subtitles in the target area.

[0100] When determining the overlapping area, the method for determining the overlapping pixel area in the reference coordinate system determined based on the original video frame can be: determining the pixel length and pixel width of the overlapping area, calculating the overlapping pixel area of ​​the overlapping area, or determining the number of pixels and the unit pixel area of ​​the overlapping area, and calculating the overlapping pixel area of ​​the overlapping area.

[0101] In the case of determining the subtitle area, the method of determining the subtitle pixel area in the reference coordinate system determined based on the original video frame can be: determining the pixel length and pixel width of the subtitle area, calculating the subtitle pixel area of ​​the subtitle area, or determining the number of pixels and the unit pixel area of ​​the subtitle area, and calculating the subtitle pixel area of ​​the subtitle area.

[0102] It is emphasized again that since the overlapped area is the area of ​​the subtitle area and the target area, the overlapped pixel area of ​​the overlapped area is less than or equal to the subtitle pixel area of ​​the subtitle area, and the overlapped pixel area of ​​the overlapped area is less than or equal to the target pixel area of ​​the target area. Based on this, the ratio of the overlapped pixel area to the subtitle pixel area is calculated, and a positive number less than or equal to 1 is obtained as the display completeness of the subtitles in the target area.

[0103] In an embodiment of the present application, in a reference coordinate system determined based on the original video frame, the overlapping pixel area of ​​the overlapping area and the subtitle pixel area of ​​the subtitle area are obtained to ensure the objectivity and accuracy of the overlapping pixel area and the subtitle pixel area. On this basis, the ratio of the overlapping pixel area to the subtitle pixel area is calculated, and the result of the display completeness of the subtitles in the target area is also true and accurate.

[0104] Next, another implementation method for obtaining the display completeness when there is an overlapped area between the target area and the subtitle area is described.

[0105] In one embodiment, based on the overlapped area and the subtitle area, obtaining the display completeness of the subtitles in the target area includes:

[0106] In a reference coordinate system determined based on the original video frame, first position information corresponding to the target area and second position information corresponding to the subtitle area are obtained; when the first position information and the second position information meet preset conditions, it is determined that the display completeness is 100%.

[0107] Among them, the reference coordinate system is a plane coordinate system, and the construction process of the reference coordinate system can be: determine any pixel point of the original video frame as the coordinate origin of the reference coordinate system, determine two mutually perpendicular directions in the original video frame as two coordinate axes of the reference coordinate system, and construct the reference coordinate system based on the coordinate origin and each coordinate axis. Usually, in order to facilitate calculation, the reference coordinate system can be constructed in the manner of Figure 5, which is a schematic diagram of the reference coordinate system and the original video frame. In Figure 5, the lower left corner of the original video frame is set as the coordinate origin, the horizontal direction of the original video frame is set as the horizontal axis of the reference coordinate system, and the vertical direction of the original video frame is set as the vertical axis of the reference coordinate system.

[0108] When the reference coordinate system is constructed, the first position information and the second position information may be directly represented by pixel coordinates in the reference coordinate system, or by relative positions of the target area and the subtitle area in the original video frame.

[0109] Next, the vertex coordinates of the first position information are compared with the vertex coordinates of the second position information. When the first position information and the second position information meet preset conditions, it is determined that the target area includes a subtitle area, and the display completeness of the subtitles in the target area is 100%.

[0110] Among them, the preset conditions include horizontal axis coordinate conditions and vertical axis coordinate comparison conditions of the first position information and the second position information, for example, the interval range corresponding to all horizontal axis coordinates of the first position information is larger than the interval range corresponding to all horizontal axis coordinates of the second position information, and the interval range corresponding to all vertical axis coordinates of the first position information is larger than the interval range corresponding to all vertical axis coordinates of the second position information.

[0111] In an embodiment of the present application, in a reference coordinate system determined based on the original video frame, first position information corresponding to the target area and second position information corresponding to the subtitle area are obtained; when the first position information and the second position information meet preset conditions, the display completeness is determined to be 100%. In this way, the display completeness is determined directly according to the position coordinates of the target area and the subtitle area, without introducing the overlapping pixel area calculation process of the overlapping area, the subtitle pixel area calculation process of the subtitle area, and the ratio calculation process of the overlapping pixel area and the subtitle pixel area, thereby saving a large amount of computing resources. In addition, compared with the method of using computing resources to calculate the display completeness, the method of using comparison to obtain the display completeness in the embodiment of the present application has a faster acquisition speed, thereby improving the acquisition efficiency of the display completeness.

[0112] Next, the contents of the preset conditions in the above-mentioned embodiment are described. In one embodiment, the preset conditions include:

[0113] The maximum horizontal coordinate and the minimum horizontal coordinate of the subtitle area are both within the coordinate interval determined by the horizontal coordinates corresponding to the target area; and the maximum vertical coordinate and the minimum vertical coordinate of the subtitle area are both within the coordinate interval determined by the vertical coordinates corresponding to the target area.

[0114] Among all the coordinates of the target area, the maximum horizontal coordinate, the minimum horizontal coordinate, the maximum vertical coordinate, and the minimum vertical coordinate are determined, and the coordinate interval corresponding to each horizontal coordinate of the target area is obtained based on the maximum horizontal coordinate and the minimum horizontal coordinate of the target area, and the coordinate interval corresponding to each vertical coordinate of the target area is obtained based on the maximum vertical coordinate and the minimum vertical coordinate of the target area.

[0115] The maximum horizontal coordinate and the minimum horizontal coordinate of the subtitle area are both within the coordinate interval determined by the horizontal coordinates corresponding to the target area, which is equivalent to that the horizontal coordinates of the subtitle area are all within the target area (including the boundary of the target area); the maximum vertical coordinate and the minimum vertical coordinate of the subtitle area are both within the coordinate interval determined by the vertical coordinates corresponding to the target area, which is equivalent to that the vertical coordinates of the subtitle area are all within the target area (including the boundary of the target area).

[0116] For example, the subtitle area and the target area are both rectangular areas, the upper left corner coordinates of the target area are (M1, N1), the lower right corner coordinates are (M2, N2), where M1 is less than M2, N1 is greater than N2, the upper left corner coordinates of the subtitle area are (X1, Y1), the lower right corner coordinates are (X2, Y2), where X1 is less than X2, and Y1 is greater than Y2, the preset conditions for display completeness of 100% include: M1≤X1≤M2, M1≤X2≤M2, N1≤Y1≤N2, N1≤Y2≤N2. As shown in Figure 8, Figure 8 is a schematic diagram of display completeness of 100%.

[0117] In the embodiment of the present application, the maximum horizontal coordinate and the minimum horizontal coordinate of the subtitle area are both within the coordinate interval determined by the horizontal coordinates corresponding to the target area; and the maximum vertical coordinate and the minimum vertical coordinate of the subtitle area are both within the coordinate interval determined by the vertical coordinates corresponding to the target area as the preset conditions for the display completeness to be 100%, which is equivalent to comprehensively and completely limiting the relative position of the subtitle area and the target area from both the horizontal and vertical dimensions to ensure the accuracy of the display completeness.

[0118] For playback terminals of different types and sizes, there are also differences in their playback interfaces, and there may be a case where the size of the target area does not match the size of the playback interface. Based on this, before the image corresponding to the target area is played on the playback interface, the target area can be adaptively adjusted according to the playback interface, so that the image presented on the playback interface is more coordinated and reasonable.

[0119] In one embodiment, before embedding the subtitle content in the subtitle area into the image picture corresponding to the update display request displayed in the playback interface, the method further includes:

[0120] The target area is updated to obtain a rendered image that matches the playback interface; the rendered image is determined as an image screen corresponding to the update display request.

[0121] When updating the target area, a processing strategy that matches the target area and the playback interface can be determined based on the aspect ratio of the target area, the aspect ratio of the playback interface, the size of the target area, and the size of the playback interface. By executing the processing strategy, the target area is rendered to obtain an image corresponding to the update display request.

[0122] When the aspect ratio of the target area matches the aspect ratio of the playback interface, and the size of the target area is smaller than the size of the playback interface, the target area can be enlarged by interpolation in equal proportion so that the size of the target area matches the size of the playback interface to obtain a rendered image, for example, the target area is enlarged to obtain a rendered image with the same size as the playback interface. Then, the rendered image is determined as the image screen corresponding to the update display request, which is equivalent to enlarging the target area so that the enlarged target area (the image screen corresponding to the update display request) covers the entire playback interface.

[0123] In the case where the aspect ratio of the target area does not match the aspect ratio of the playback interface, the length or width of the target area can be adaptively adjusted according to the aspect ratio of the playback interface to obtain a rendered image that matches the aspect ratio of the playback interface, that is, an image screen corresponding to the update display request. In this case, the length of the rendered image will fill the length of the playback interface, but the width of the rendered image will not fill the width of the playback interface, or the length of the rendered image will not fill the length of the playback interface, but the width of the rendered image will fill the width of the playback interface.

[0124] In another optional embodiment, if the aspect ratio of the target area does not match the aspect ratio of the playback interface, the length of the target area can be processed according to the ratio between the length of the playback interface and the length of the target area; at the same time, the width of the target area can be processed according to the ratio between the width of the playback interface and the width of the target area. In this way, the rendered image (the image corresponding to the update display request) obtained will have a certain degree of content deformation, but can cover the entire playback interface.

[0125] In an embodiment of the present application, the target area is adaptively adjusted according to the playback interface to obtain a rendered image that matches the playback interface, and then the image screen corresponding to the update display request is determined, which can make the image screen presented by the playback interface more coordinated and reasonable. In addition, the adaptive adjustment method of the embodiment of the present application is more flexible and can be applied to the playback interface of playback terminals of different types and different aspect ratios. While improving the picture display effect, it expands the terminal application scenarios of image screen display. In addition, the step of updating the target area is performed before the step of embedding the subtitle content, which avoids the problem of deformation of the embedded subtitle content as the target area is deformed, thereby ensuring that the complete and accurate subtitle content is always displayed in the image screen to be played in the playback interface.

[0126] When the display completeness meets the conditions for embedded display, the subtitle content can be embedded in the image screen in a variety of ways. The following describes a variety of possible ways to embed the subtitle content.

[0127] In one embodiment, embedding the subtitle content in the subtitle area into an image screen corresponding to the update display request displayed in the playback interface includes:

[0128] In the image picture corresponding to the update display request, an embedding area is determined; the subtitle content is displayed in the embedding area in the form of plug-in subtitles, and an image picture of the original video frame to be played in the playback interface is obtained.

[0129] The image corresponding to the update display request refers to the image to be displayed on the playback interface after the target area is updated. The embedded area refers to an area in the image for displaying complete subtitle content. On the basis that the display priority of the complete subtitle content in the embedded area is higher than the display priority of the content in the image, there is no restriction on the method of determining the embedded area.

[0130] Optionally, it is detected whether there are subtitles in the image screen corresponding to the update display request. If so, the area where the subtitles are located is determined as the embedded area, and then the subsequently embedded subtitle content is displayed in the embedded area in the form of external subtitles, covering the incomplete original subtitles in the image screen, and retaining the non-subtitle area in the image screen to the greatest extent; if not, a fixed position in the image screen is determined as the embedded area, such as the lower area in the image screen, the non-interest area in the image screen, etc., and the subtitle content is displayed in the embedded area in the form of external subtitles.

[0131] Optionally, relative position information of the subtitle area in the original video frame is obtained, and then the area in the image screen that matches the relative position information is determined as the embedded area, and the subtitle content is displayed in the embedded area in the form of external subtitles.

[0132] In an embodiment of the present application, in an image screen corresponding to an update display request, an embedded area is determined, and subtitle content is displayed in the embedded area in the form of plug-in subtitles, so as to obtain an image screen of the original video frame to be played in the playback interface. In this way, when the image screen corresponding to the original video frame is displayed on the playback interface, the content of the playback interface can not only satisfy the update display request for the target area in the original video frame, but also display complete subtitles.

[0133] In one embodiment, embedding the subtitle content in the subtitle area into an image screen corresponding to the update display request displayed in the playback interface includes:

[0134] Generate a subtitle image according to the subtitle content; the size of the subtitle image is less than or equal to the size of the image screen corresponding to the update display request; merge the subtitle image and the image screen corresponding to the update display request to obtain the image screen of the original video frame to be played in the playback interface.

[0135] To ensure the integrity of the subtitle content in the fused image, the size of the subtitle image is smaller than or equal to the size of the image screen corresponding to the update display request, so that the fused image can retain the complete subtitle content.

[0136] Optionally, the minimum bounding rectangle corresponding to the subtitle content is obtained, and the non-subtitle area in the minimum bounding rectangle is filled with transparency. Then, the minimum bounding rectangle including the subtitle content and the transparent filling area is determined as the original subtitle image, and the original subtitle image is adaptively scaled to obtain a subtitle image with a size less than or equal to that of the image screen. Then, the subtitle image is fused with the image screen corresponding to the update display request, so that the fused image retains both the image features in the image screen and the text features in the subtitle image.

[0137] In the embodiment of the present application, a subtitle image is generated according to the subtitle content, and the subtitle image and the image corresponding to the update display request are merged to obtain the image of the original video frame to be played in the playback interface. In this way, when the image corresponding to the original video frame is displayed on the playback interface, the content of the playback interface can not only meet the update display request of the target area, but also display the complete subtitles.

[0138] The original video frame refers to the currently playing video frame in the target video, or the backward video frame to be played after the currently playing video frame. Then, after determining that the original video frame needs to be embedded with subtitle content, and the subtitle content is embedded in the image screen corresponding to the playback interface, if the update display request does not change, it is necessary to continue to judge the conditions for embedding and displaying the backward video frame to be played corresponding to the original video frame, and when the conditions for embedding and display are met, continue to perform the subtitle content embedding operation of the backward video frame to be played corresponding to the original video frame. Since the target video usually includes a large number of video frames, and the computing resources consumed by the judgment and embedding operations of each video frame are large, the common operations of each backward video frame to be played corresponding to the original video frame can be extracted and processed uniformly to save computing resources.

[0139] In one embodiment, the subtitle display method further includes:

[0140] Obtain a target area and a subtitle area of ​​a backward video frame to be played corresponding to an original video frame in a target video; for any backward video frame to be played, update the target area of ​​the backward video frame to be played to obtain a rendered image matching a playback interface, embed the subtitle content in the subtitle area of ​​the backward video frame to be played and display it in the rendered image, and obtain an image screen of the backward video frame to be played in the playback interface.

[0141] The backward video frame to be played refers to a video frame in the target video whose playing order is after the original video frame.

[0142] After determining that the display completeness corresponding to the original video frame in the target video meets the conditions for embedded display, it is assumed that the display completeness of the backward video frame to be played corresponding to the original video frame in the target video meets the embedded display conditions. In this way, when processing the backward video frame to be played, there is no need to calculate and compare the display completeness, and the rendering of the target area of ​​the backward video frame to be played and the embedding of the subtitle content can be directly performed.

[0143] Among them, the rendering steps of the target area of ​​the backward video frame to be played can be the same as the rendering steps of the target area in the original video frame. For details, please refer to the embodiment of rendering the target area of ​​the original video frame in the aforementioned embodiment, which will not be repeated here.

[0144] The method of embedding the subtitle content of the backward video frame to be played is the same as the method of embedding the subtitle content in the original video frame. It can be embedded in the form of external subtitles or generated in the form of image fusion. For details, please refer to the corresponding embodiment of embedding and displaying the subtitle content of the original video frame in the aforementioned embodiment, which will not be repeated here.

[0145] In the embodiment of the present application, when it is determined that the original video frame meets the embedded display conditions, it is assumed that each backward video frame to be played corresponding to the original video frame in the target video meets the embedded display conditions. In this way, when processing each backward video frame to be played, there is no need to calculate and compare the display completeness, and the target area of ​​the backward video frame to be played is directly updated to obtain a rendered image that matches the playback interface, saving computing resources. On this basis, the update processing of the target area in the backward video frame to be played is the same as the update processing of the target area in the original video frame, and the embedding method of the subtitle content in the backward video frame to be played is the same as the embedding method of the subtitle content in the original video frame, so as to further save computing resources.

[0146] During the playback of the target video, an update display request may be triggered once or multiple times, and different update display requests correspond to different playback video frames, and the types of update display requests triggered each time may be the same or different. For example, during the playback of the target video, a zoom-in display request may be triggered first, and then a zoom-out display request may be triggered, or a sliding display request may be triggered first, and then a zoom-in display request may be triggered, or multiple zoom-in display requests may be triggered continuously, etc.

[0147] In one embodiment, after embedding the subtitle content in the subtitle area into the image picture corresponding to the update display request displayed in the play interface, the method further includes:

[0148] In response to a second update display request for a target video, a subtitle area and a target area of ​​a target video frame matching the second update display request are obtained, wherein the target area is a mapping area of ​​the content displayed in the target video frame after being updated to match the second update display request in the playback interface; the target video frame is a backward video frame of the original video frame in the target video; based on the target area of ​​the target video frame and the subtitle area of ​​the target video frame, an update completeness of the subtitles in the target area of ​​the target video frame is obtained; if the update completeness meets the conditions for embedded display, the subtitle content in the subtitle area of ​​the target video frame is embedded and displayed in an image screen corresponding to the second update display request in the playback interface; if the update completeness does not meet the conditions for embedded display, the image screen corresponding to the second update display request is directly displayed in the playback interface.

[0149] Among them, the second update display request can be triggered by a processing operation on the target video based on the display requirements, including but not limited to a zoom display request triggered by a proportional zoom operation, a zoom display request triggered by a custom parameter, a sliding display request triggered by an area of ​​interest display operation, a zoom out display request triggered by a proportional zoom out operation, and the like.

[0150] In response to the second update display request for the target video, for the backward video frame of the original video frame in the target video, the target position point of the content displayed after the update in the playback interface is determined as the target update center, and the target area of ​​the target video frame is determined from the target video frame according to the target update center and the update parameters carried in the second update display request. In the reference coordinate system determined based on the target video frame, the update completeness corresponding to the target area of ​​the target video frame and the subtitle area of ​​the target video frame is obtained, and whether the update completeness meets the conditions for embedded display is evaluated. If so, the subtitle content in the subtitle area of ​​the target video frame is embedded and displayed in the image screen corresponding to the second update display request in the playback interface; if not, the image screen corresponding to the second update display request is directly displayed in the playback interface.

[0151] Taking the update display request as a zoom-in display request and the second update display request as a zoom-out display request triggered for the currently playing video frame as an example, in response to the second update display request for the target video, the target area in the target video frame is determined, and the target area in the target video frame is updated to obtain an image corresponding to the second update display request; then the overlapping area between the target area in the target video frame and the subtitle area in the target video frame is obtained to obtain the update completeness of the subtitles in the target area in the target video frame; then, whether the update completeness meets the conditions for embedded display is evaluated, and if so, the subtitle content in the subtitle area of ​​the target video frame is embedded and displayed in the image corresponding to the second update display request in the playback interface; if not, the image corresponding to the second update display request is directly displayed in the playback interface.

[0152] Taking the update display request as the enlarged display request, the second update display request as the enlarged display request triggered for the currently playing video frame, and the target video frame as the backward video frame of the currently playing video frame as an example, in response to the second update display request triggered for the currently playing video frame, the target area in the target video frame is determined, and the target area in the target video frame is updated to obtain an image corresponding to the second update display request; then the overlapping area between the target area in the target video frame and the subtitle area in the target video frame is obtained to obtain the update completeness of the subtitle in the target area in the target video frame; then, whether the update completeness meets the conditions for embedded display is evaluated, and if so, the subtitle content in the subtitle area of ​​the target video frame is embedded and displayed in the image corresponding to the second update display request in the playback interface; if not, the image corresponding to the second update display request is directly displayed in the playback interface.

[0153] It should be noted that, in the case where both the second update display request and the update display request are enlarged display requests, if the display completeness of the subtitles in the target area of ​​the original video frame meets the conditions for embedded display when the update display request (enlarged display request) is first responded to, since the second update display request is made on the basis of the first update display request, it means that the display completeness of the subtitles in the target area of ​​the target video frame associated with the second update display request also meets the conditions for embedded display.

[0154] Then, when responding to the second update display request (enlarged display request), there is no need to identify the target video frame. Instead, it is directly confirmed that the display completeness of the subtitles in the target area of ​​the target video frame meets the conditions for embedded display, and the subtitle content in the subtitle area of ​​the target video frame is embedded and displayed in the image screen corresponding to the update display request in the playback interface.

[0155] In an embodiment of the present application, in response to a second update display request for a target video, the backward video frame of the original video frame in the target video is used as the target video frame, and the subtitle area and the target area of ​​the target video frame matching the second update display request are obtained, and the target area is the mapping area of ​​the content displayed in the target video frame after the update matching the second update display request in the playback interface; based on the target area of ​​the target video frame and the subtitle area of ​​the target video frame, the update completeness of the subtitle in the target area of ​​the target video frame is obtained; if the update completeness meets the conditions for embedded display, the subtitle content in the subtitle area of ​​the target video frame is embedded and displayed in the image screen corresponding to the second update display request in the playback interface; if the update completeness does not meet the conditions for embedded display, the image screen corresponding to the second update display request is directly displayed in the playback interface. In this way, during the playback of the target video, the content presented in the playback interface of the current playback video frame and the backward video frame to be played of the current playback video frame can be adjusted in real time according to the currently triggered update display request, so that the playback interface continues to present the subtitle content corresponding to the playback video frame and complete.

[0156] In a specific embodiment, the subtitle display method can be applied to a scene of magnifying and displaying a TV series, as shown in FIG9 , and includes the following steps:

[0157] Step S901, reading and decompressing the video file.

[0158] Step S902, obtaining the audio and video of the target video.

[0159] Step S903, processing audio and video to complete audio and video synchronization.

[0160] Steps S901-S903 belong to the pre-process of the player, and the video file can be pulled by the player from a storage system such as the cloud or server.

[0161] Step S904: Detect the image screen in the playback interface.

[0162] If the content of the image screen is smaller than the content of the original video frame corresponding to the image screen, it means that the playback interface is displayed globally and no subsequent steps are required; if the content of the image screen is smaller than the content of the original video frame corresponding to the image screen, it means that the image screen in the playback interface is enlarged and displayed, then execute step S911.

[0163] See Figure 10, which is a comparative schematic diagram of non-global display. The left side of Figure 10 is the original video frame displayed on the playback interface, and the right side is the image screen that updates and displays the original video frame, that is, the area framed by dotted lines in the original video frame. It should be noted that the dotted lines in the original video frame in Figure 10 are only schematic diagrams and are not displayed during actual playback. As can be seen from Figure 10, the content of the image screen is smaller than the schematic diagram of the content of the original video frame corresponding to the image screen. Specifically, in the image screen obtained by enlarging the original video frame, the subtitle area in the original video frame is stretched out of the playback interface.

[0164] Step S905: identifying the subtitle area in the original video frame.

[0165] In the reference coordinate system constructed based on the original video frame, an artificial intelligence algorithm is used to identify the subtitle area of ​​the original video frame to obtain a vertex coordinate set of the subtitle area. The coordinates in the vertex coordinate set of the subtitle area use the proportional value of the subtitle area relative to the original video frame, rather than the absolute value.

[0166] Step S906, identifying a target area of ​​the image in the playback interface in the original video frame.

[0167] The target position point of the image screen in the playback interface is determined as the update center; the target area is determined in the original video frame according to the update center and the update parameters carried in the update display request; in the reference coordinate system constructed based on the original video frame, an artificial intelligence algorithm is used to perform edge recognition on the target area in the original video frame to obtain a vertex coordinate set of the target area. The coordinates in the vertex coordinate set of the target area use the ratio value of the target area relative to the original video frame, rather than the absolute value.

[0168] Step S907, determining whether the subtitle area is completely displayed in the target area.

[0169] If the maximum horizontal coordinate and the minimum horizontal coordinate of the subtitle area are both within the coordinate interval determined by the horizontal coordinates corresponding to the target area; and the maximum vertical coordinate and the minimum vertical coordinate of the subtitle area are both within the coordinate interval determined by the vertical coordinates corresponding to the target area, it is determined that the target area fully displays the subtitle area.

[0170] If the maximum horizontal coordinate or the minimum horizontal coordinate of the subtitle area is not in the coordinate interval determined by the horizontal coordinates corresponding to the target area, or the maximum vertical coordinate or the minimum vertical coordinate of the subtitle area is not in the coordinate interval determined by the vertical coordinates corresponding to the target area, it is determined that the target area does not fully display the subtitle area.

[0171] If the target area completely displays the subtitle area, step S911 is directly executed.

[0172] Step S908: if the target area does not completely display the subtitle area, extract the subtitle content in the subtitle area.

[0173] In the original video frame, an artificial intelligence algorithm is used to extract text from the subtitle area to obtain the subtitle text in the subtitle area.

[0174] Step S909: embed the subtitle content into the image screen in the playback interface.

[0175] Steps S904 to S909 are the content of the video rendering executed by the player, which is applied to the video frames corresponding to the image screen in the playback interface, and the backward video frames to be played.

[0176] Step S910, rendering the audio.

[0177] Step S911, synchronously playing the rendered audio and video in the playback interface.

[0178] In an embodiment of the present application, for a scenario where the image screen is not displayed globally during video playback, that is, the playback interface cannot display all image content, for example, when the user zooms in with two fingers, the image content will be magnified, causing part of the image to pop out of the playback interface. The detection program is started to detect the relative position of the subtitle area in the original video frame, and when it is detected that the area that is populated in the playback interface includes the subtitle area, the text information of the subtitles in the subtitle area is extracted, and then the part of the text is redisplayed on the playback interface, thereby achieving the effect that the complete subtitles are always displayed on the playback interface no matter how the image is displayed.

[0179] In a specific embodiment, the subtitle display method can be applied to a TV drama playback scenario, in which the subtitles are the lines of the characters in the TV drama or the content of the TV drama interludes. Specifically, the terminal player displays the first episode of the TV drama. When the user is browsing the 20th frame of the video frame of the first episode of the TV drama, he selects the area of ​​interest with two fingers in the current playback interface and performs a sliding zoom operation, generating a zoom display request. In response to the zoom display request, the player determines whether the area of ​​interest includes the complete subtitle text of the 20th frame. If the complete subtitle text is included, the area of ​​interest in each backward video frame to be played after the 20th frame is determined according to the relative position of the area of ​​interest in the 20th frame in the 20th frame, and the area of ​​interest in each backward video frame to be played is rendered, so that the area of ​​interest in each backward video frame to be played is displayed on the playback interface, and the rendering of the 21st frame is displayed in the playback interface. The processed picture, the picture after rendering the 22nd frame, etc.; if there is no complete subtitle text, according to the relative position of the interest area in the 20th frame in the 20th frame, determine the interest area in each backward video frame to be played after the 20th frame, and perform subtitle embedding processing on each backward video frame to be played after the 20th frame. Take the 21st frame as an example to illustrate the subtitle embedding processing method: after rendering the interest area of ​​the 21st frame so that the interest area of ​​the 21st frame covers the playback interface and obtains the rendering picture of the 21st frame, identify the subtitle content of the 21st frame, and embed the subtitle content in the rendering picture, display the rendering picture with embedded subtitle content on the playback interface, and obtain the playback picture corresponding to the 21st frame in the playback interface. Then, the above-mentioned processing method for the 21st frame is used to process the 22nd frame, the 23rd frame and other backward video frames to be played as the corresponding playback pictures of each backward video frame to be played in the playback interface. In this way, for each video frame before the 20th frame (including the 20th frame), the player plays it in the size of the original video frame, and for each video frame after the 20th frame, the player presents it in the enlarged playback screen, and at the same time presents the complete subtitle text in the playback screen. This is equivalent to the user watching the first episode of the TV series. The subtitles seen in the first 20 frames are the hard-pressed subtitles in the original video frames, and the subtitles seen after the 23rd frame are the plug-in subtitles newly embedded by the player. Furthermore, after the user finishes watching the first episode of the TV series, when continuing to watch the second episode of the TV series, if the user's update display request is not detected, each frame in the second episode of the TV series is processed in the same way as the 21st frame of the first episode, and the subtitle content of each video frame of the second episode is continuously embedded in the playback screen corresponding to the video frame in the form of plug-in subtitles, thereby achieving the effect that no matter how the screen in the playback interface changes, the player can present the user with a complete subtitle text that matches the screen.

[0180] In a specific embodiment, the subtitle display method can be applied to a live broadcast of a sports event. In this scenario, the subtitles are commentary texts and game scores of the sports event. Taking a basketball game as an example, a terminal player displays a basketball game (basketball games usually include four sections, each section lasting 12 minutes). After browsing the third section and before the start of the fourth section, the user selects a double magnification display request on the terminal player interface. In response to the double magnification display request, the player obtains the video of the fourth section from the cloud, and uses the center position of the original video frame as the magnification center to double-magnify each video frame of the fourth section to obtain an enlarged picture of each video frame. Then, it is determined whether the game score and commentary text in the enlarged picture of each video frame are blocked. If not, the enlarged picture of each video frame is directly used as the image picture to be displayed in the playback interface. At this time, the commentary text and game score in the normal basketball game video including the fourth section seen by the user in the player are in the form of hard-pressed subtitles. If blocked, the game score and commentary text are identified from each video frame, and the game score and commentary text are embedded in the enlarged picture of each video frame. At this time, in the video of the fourth section seen by the user in the player, the commentary text and game score in each video frame are in the form of external subtitles. The subtitle display method provided in the embodiment of the present application can support users to perform various magnification and movement processing on the picture in the user playback interface, while ensuring that the playback interface presents complete subtitle text that matches the picture.

[0181] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0182] Based on the same inventive concept, the embodiment of the present application also provides a subtitle display device for implementing the subtitle display method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more subtitle display device embodiments provided below can refer to the limitations of the subtitle display method above, and will not be repeated here.

[0183] In one embodiment, as shown in FIG. 11 , a subtitle display device is provided, including: a request response module 1101, a video frame recognition module 1102, and a subtitle display module 1103, wherein:

[0184] The request response module 1101 is used to respond to the update display request for the currently played target video, obtain the subtitle area of ​​the original video frame and the target area of ​​the original video frame that match the update display request, and the target area is the mapping area of ​​the content displayed after the update in the playback interface in the original video frame;

[0185] The video frame recognition module 1102 is used to recognize the original video frame, obtain the overlapping area between the target area and the subtitle area, and obtain the display completeness of the subtitle in the target area based on the overlapping area and the subtitle area;

[0186] The subtitle display module 1103 is used to embed and display the subtitle content in the subtitle area in the image screen corresponding to the update display request in the playback interface when the display completeness meets the embedded display condition.

[0187] In one embodiment, the request response module 1101 includes:

[0188] An update center determination unit, used to determine a target location point of the content displayed after update in the playback interface as an update center;

[0189] The first target determination unit is used to determine a target area in the original video frame according to the update center and the update parameters carried in the update display request.

[0190] In one embodiment, the update display request is an enlarged display request; the request response module 1101 includes:

[0191] A request generating unit, configured to generate a zoom display request in response to an operation on a target video;

[0192] The to-be-enlarged region determining unit is used to determine the to-be-enlarged region for the original video frame based on the operation position corresponding to the enlarged display request, and determine the to-be-enlarged region as the target region.

[0193] In one embodiment, the video frame identification module 1102 includes:

[0194] A position information determining unit, configured to obtain first position information corresponding to the target area and second position information corresponding to the subtitle area in a reference coordinate system determined based on the original video frame;

[0195] The overlap region determining unit is used to determine the intersection coordinates of the target region and the subtitle region according to the first position information and the second position information, and determine the overlap region according to the intersection coordinates.

[0196] In one embodiment, a position information determination unit is used to obtain a first vertex coordinate set of an original video frame, a second vertex coordinate set of a target area, and a third vertex coordinate set of a subtitle area in a reference coordinate system determined by the original video frame; determine a first relative coordinate set of the target area according to the first vertex coordinate set and the second vertex coordinate set, determine first position information based on the first relative coordinate set; and determine a second relative coordinate set of the subtitle area according to the first vertex coordinate set and the third vertex coordinate set, and determine second position information based on the second relative coordinate set.

[0197] In one embodiment, the video frame identification module 1102 includes:

[0198] A pixel area acquisition unit, used to acquire the overlap pixel area of ​​the overlap area and the subtitle pixel area of ​​the subtitle area in a reference coordinate system determined based on the original video frame;

[0199] The completeness calculation unit is used to determine the ratio of the overlapping pixel area to the subtitle pixel area as the display completeness of the subtitle in the target area.

[0200] In one embodiment, the completeness calculation unit is used to obtain first position information corresponding to the target area and second position information corresponding to the subtitle area in a reference coordinate system determined based on the original video frame; when the first position information and the second position information meet preset conditions, the display completeness is determined to be 100%.

[0201] In one embodiment, the preset conditions include:

[0202] The maximum horizontal coordinate and the minimum horizontal coordinate of the subtitle area are both within the coordinate interval determined by the horizontal coordinates corresponding to the target area; and the maximum vertical coordinate and the minimum vertical coordinate of the subtitle area are both within the coordinate interval determined by the vertical coordinates corresponding to the target area.

[0203] In one embodiment, the subtitle display device further includes: an update processing module and an image frame determination module, wherein:

[0204] An update processing module is used to update the target area to obtain a rendering image that matches the playback interface;

[0205] The image frame determination module is used to determine the rendered image as the image frame corresponding to the update display request.

[0206] In one embodiment, the subtitle display module 1103 includes:

[0207] An embedding area determination unit, used to determine an embedding area in an image picture corresponding to the update display request;

[0208] The first subtitle embedding unit is used to display the subtitle content in the embedding area in the form of plug-in subtitles to obtain an image screen of the original video frame to be played in the playback interface.

[0209] In one embodiment, the subtitle display module 1103 includes:

[0210] A subtitle image generating unit, used to generate a subtitle image according to the subtitle content; the size of the subtitle image is less than or equal to the size of the image screen corresponding to the update display request;

[0211] The second subtitle embedding unit is used to merge the subtitle image with the image picture corresponding to the update display request to obtain the image picture of the original video frame to be played in the playback interface.

[0212] In one embodiment, the subtitle display device further includes: a backward region acquisition module and a backward video frame processing module, wherein:

[0213] A backward region acquisition module is used to acquire a target region and a subtitle region of a backward to-be-played video frame corresponding to an original video frame in a target video;

[0214] The backward video frame processing module is used to update the target area of ​​any backward video frame to be played, obtain a rendered image matching the playback interface, embed the subtitle content in the subtitle area of ​​the backward video frame to be played in the rendered image, and obtain the image screen of the backward video frame to be played in the playback interface.

[0215] In one embodiment, the subtitle display device further includes: a second request response module, an update completeness acquisition module, a first judgment module and a second judgment module, wherein:

[0216] A second request response module is used to respond to a second update display request for a target video, obtain a subtitle area of ​​a target video frame and a target area of ​​the target video frame that match the second update display request, wherein the target area is a mapping area of ​​the content displayed after being updated and matched with the second update display request in the playback interface in the target video frame; the target video frame is a backward video frame of an original video frame in the target video;

[0217] An update completeness acquisition module, used to obtain the update completeness of the subtitles in the target area of ​​the target video frame based on the target area of ​​the target video frame and the subtitle area of ​​the target video frame;

[0218] A first judgment module is used to embed and display the subtitle content in the subtitle area of ​​the target video frame in an image picture corresponding to the second update display request in the playback interface if the update completeness meets the embedded display condition;

[0219] The second judgment module is used to directly display the image frame corresponding to the second update display request in the playback interface if the update completeness does not meet the embedded display condition.

[0220] The above-mentioned subtitle display device, when updating and displaying the screen in the target video, obtains the target area of ​​the content displayed after the update in the playback interface in the original video frame and the subtitle area of ​​the original video frame, identifies the original video frame, and determines whether there is subtitle occlusion in the target area. Further, in the case of subtitle occlusion in the target area, the overlapping area of ​​the subtitle area and the target area is determined, and based on the overlapping area and the subtitle area, the display completeness of the subtitle in the target area is obtained as a quantitative indicator parameter of the degree of subtitle occlusion in the target area. Then, in the case where the display completeness meets the conditions for embedded display, the complete subtitle content in the subtitle area is embedded in the image screen corresponding to the update display request, and displayed in the playback interface, which is equivalent to no matter how the target area of ​​the original video frame is updated and displayed in the playback interface, the content presented in the playback interface includes the complete subtitle information of the original video frame. In addition, the step of performing subtitle content embedding is performed when the display completeness meets the conditions for embedded display, and the conditions for embedded display in the embodiment of the present application can be flexibly set based on different application scenarios to meet the needs of complete subtitle display in different application scenarios, and to a certain extent, improve the scene applicability of the subtitle display method.

[0221] Each module in the above-mentioned subtitle display device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0222] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG12. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store subtitle display data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a subtitle display method is implemented.

[0223] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG13. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a subtitle display method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0224] Those skilled in the art will understand that the structures shown in FIG. 12 and FIG. 13 are merely block diagrams of partial structures related to the scheme of the present application, and do not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0225] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0226] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0227] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0228] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0229] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0230] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0231] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A subtitle display method, characterized in that: The method comprises: In response to an update display request for a target video currently being played, obtaining a subtitle area of ​​an original video frame and a target area of ​​the original video frame that match the update display request, wherein the target area is a mapping area of ​​the content displayed after the update in the playback interface in the original video frame; Identify the original video frame to obtain an overlapping area between the target area and the subtitle area, and obtain a display completeness of the subtitle in the target area based on the overlapping area and the subtitle area; If the display completeness satisfies the embedded display condition, the subtitle content in the subtitle area is embedded and displayed in the image picture corresponding to the update display request in the playback interface.

2. The method according to claim 1, characterized in that Acquiring the target area of ​​the original video frame includes: Determine a target location point of the updated content displayed in the playback interface as an update center; The target area is determined in the original video frame according to the update center and the update parameters carried in the update display request.

3. The method according to claim 1, characterized in that The update display request is a zoom display request; obtaining the target area of ​​the original video frame includes: In response to the operation on the target video, generating the enlarged display request; Based on the operation position corresponding to the enlarged display request, a region to be enlarged for the original video frame is determined, and the region to be enlarged is determined as the target region.

4. The method according to claim 1, characterized in that: The identifying the original video frame to obtain the overlapping area between the target area and the subtitle area includes: In a reference coordinate system determined based on the original video frame, obtaining first position information corresponding to the target area and second position information corresponding to the subtitle area; The intersection coordinates of the target area and the subtitle area are determined according to the first position information and the second position information, and the overlap area is determined according to the intersection coordinates.

5. The method according to claim 4, characterized in that The step of acquiring the first position information of the target area and the second position information of the subtitle area in a reference coordinate system determined based on the original video frame includes: In the reference coordinate system determined by the original video frame, obtaining a first vertex coordinate set of the original video frame, a second vertex coordinate set of the target area, and a third vertex coordinate set of the subtitle area; A first relative coordinate set of the target area is determined according to the first vertex coordinate set and the second vertex coordinate set, and the first position information is determined based on the first relative coordinate set; and a second relative coordinate set of the subtitle area is determined according to the first vertex coordinate set and the third vertex coordinate set, and the second position information is determined based on the second relative coordinate set.

6. The method according to claim 1, characterized in that The obtaining, based on the overlapped area and the subtitle area, the display completeness of the subtitle in the target area includes: In a reference coordinate system determined based on the original video frame, obtaining an overlap pixel area of ​​the overlap region and a subtitle pixel area of ​​the subtitle region; The ratio of the overlapping pixel area to the subtitle pixel area is determined as the display completeness of the subtitle in the target area.

7. The method according to claim 1, characterized in that The obtaining, based on the overlapped area and the subtitle area, the display completeness of the subtitle in the target area includes: In a reference coordinate system determined based on the original video frame, obtaining first position information corresponding to the target area and second position information corresponding to the subtitle area; When the first position information and the second position information meet a preset condition, it is determined that the display completeness is 100%.

8. The method according to claim 7, characterized in that The preset conditions include: The maximum horizontal coordinate and the minimum horizontal coordinate of the subtitle area are both within the coordinate interval determined by the horizontal coordinates corresponding to the target area; and The maximum vertical coordinate and the minimum vertical coordinate of the subtitle area are both within the coordinate interval determined by the vertical coordinates corresponding to the target area.

9. The method according to claim 1, characterized in that: Before embedding the subtitle content in the subtitle area into the image picture corresponding to the update display request displayed in the playback interface, the method further includes: Performing an update process on the target area to obtain a rendering image matching the playback interface; The rendered image is determined as an image frame corresponding to the update presentation request.

10. The method according to claim 1, characterized in that The step of embedding the subtitle content in the subtitle area into the image picture corresponding to the update display request displayed in the playback interface includes: Determining an embedding area in an image picture corresponding to the update display request; The subtitle content is displayed in the embedded area in the form of external subtitles, and the image picture of the original video frame to be played in the playback interface is obtained.

11. The method according to claim 1, characterized in that: The step of embedding the subtitle content in the subtitle area into the image picture corresponding to the update display request displayed in the playback interface includes: generating a subtitle image according to the subtitle content; wherein the size of the subtitle image is smaller than or equal to the size of the image screen corresponding to the update display request; The subtitle image and the image picture corresponding to the update display request are merged to obtain the image picture of the original video frame to be played in the playback interface.

12. The method according to claim 1, characterized in that The method further comprises: Acquire a target area and a subtitle area of ​​a backward to-be-played video frame corresponding to the original video frame in the target video; For any backward video frame to be played, the target area of ​​the backward video frame to be played is updated to obtain a rendered image matching the playback interface, and the subtitle content in the subtitle area of ​​the backward video frame to be played is embedded and displayed in the rendered image to obtain an image screen of the backward video frame to be played in the playback interface.

13. The method according to claim 1, characterized in that After embedding the subtitle content in the subtitle area into the image picture corresponding to the update display request displayed in the playback interface, the method further includes: In response to a second update display request for the target video, obtaining a subtitle area of ​​a target video frame matching the second update display request and a target area of ​​the target video frame, wherein the target area is a mapping area of ​​the content displayed in the playback interface after being updated and matching the second update display request in the target video frame; the target video frame is a backward video frame of the original video frame in the target video; Based on the target area of ​​the target video frame and the subtitle area of ​​the target video frame, obtaining the update completeness of the subtitles in the target area of ​​the target video frame; If the update completeness satisfies the embedded display condition, embedding and displaying the subtitle content in the subtitle area of ​​the target video frame in the image picture corresponding to the second update display request in the playback interface; If the update completeness does not meet the embedded display condition, the image frame corresponding to the second update display request is directly displayed in the playback interface.

14. A subtitle display device, characterized in that: The device comprises: A request response module, configured to respond to an update display request for a target video currently being played, and obtain a subtitle area of ​​an original video frame that matches the update display request and a target area of ​​the original video frame, wherein the target area is a mapping area of ​​the content displayed after the update in the playback interface in the original video frame; A video frame recognition module, used to recognize the original video frame, obtain the overlapping area between the target area and the subtitle area, and obtain the display completeness of the subtitle in the target area based on the overlapping area and the subtitle area; The subtitle display module is used to embed the subtitle content in the subtitle area into the image screen corresponding to the update display request in the playback interface when the display completeness meets the embedded display condition.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 13 are implemented.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images