Method for converting horizontal screen to vertical screen for broadcasting based on character tracking

By identifying the speaking subject in the video and cropping, the poor experience of horizontal videos on the mobile phone in vertical viewing is solved, and a better user experience is achieved.

CN119094648APending Publication Date: 2024-12-06SHANGHAI MEDIA TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411015404.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

When watching horizontal TV programs on the mobile phone, the viewing experience in the vertical screen scene is poor, and the existing technology cannot effectively crop the video to adapt to the vertical screen proportions, resulting in black edges or partial screen display.

Method used

By identifying the other person to be judged in the video, using the speech count parameters and the lost count parameters for cropping, determining the speaking subject and performing secondary composition, adjusting the video proportion to suit vertical viewing.

Benefits of technology

It effectively retains the subject in the video, improves the user's experience during vertical viewing, and provides a viewing effect that is more in line with the audience's habits.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094648A_ABST
    Figure CN119094648A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of video clipping, in particular to a horizontal-to-vertical-screen broadcasting method based on character tracking, which comprises the following steps: aiming at a to-be-clipped video, sequentially identifying a to-be-distinguished character in a single-frame image of each frame, and distributing a speaking counting parameter to the to-be-distinguished character; in each single-frame image, the mouth shape length-width ratio of each person to be discriminated is judged, and then speaking counting parameters of the person to be discriminated are updated according to a judgment result; and determining a cutting object character in the single-frame image according to the speaking counting parameter, and cutting the cutting object character. In order to solve the problem that the effect of watching landscape interview content in a mobile phone vertical screen scene is poor, an actual speaking main body in an interview interface is recognized and serves as a cutting object character, and then secondary composition and cutting are performed on a video, so that landscape video content can be re-composed into content suitable for being watched in a vertical screen, and the effect of watching landscape interview content in a mobile phone vertical screen scene is improved. And the main body in the watching process is effectively reserved, and the user experience is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of video editing, and in particular to a method for converting horizontal screen to vertical screen broadcasting based on character tracking. Background Art

[0002] Among the terminal broadcasting devices for TV programs, the proportion of mobile phones as display terminals is increasing. This is because smartphones have become an essential communication device for the public. As smartphones become more and more powerful, network communication speeds and methods become more and more popular, and they are easy to carry and can be moved at any time, many people are watching TV programs directly on their mobile phones.

[0003] Ordinary TV programs are usually 16:9 or 4:3, while the common display screen ratios of mobile phones are 9:16, 9:18, 9:19, etc. Under normal circumstances, if a mobile phone wants to play a TV program, as long as it is displayed in horizontal mode, no matter which ratio is mentioned above, even after scaling, the error can be ignored. However, in reality, most people do not have the habit of watching TV in horizontal mode. This may be due to the inconvenience of carrying in horizontal mode or the difficulty of placing in horizontal mode. At this time, when watching TV programs, viewers would rather watch in vertical mode. To watch the same program, in vertical mode, the player can only open an area in the middle of the screen with the width of the screen as the width and 9 / 16 of the width as the height as the playback area, and the other areas are black backgrounds or display other content "masked wide-screen film" effects, or only part of the entire TV program screen can be played. This is obviously not in line with the habits of some viewers who like to watch TV in vertical mode. Summary of the invention

[0004] In view of the above problems existing in the prior art, a method for converting horizontal screen to vertical screen broadcasting based on character tracking is now provided.

[0005] The specific technical solutions are as follows:

[0006] A method for converting horizontal screen to vertical screen broadcasting based on character tracking, comprising:

[0007] Step S1: for the video to be cropped, identifying the person to be identified in each single frame image in turn, and assigning a speech count parameter to the person to be identified;

[0008] Step S2: in each single-frame image, judging the length-to-width ratio of the mouth shape of each of the characters to be judged, and then updating the speech counting parameter of the characters to be judged according to the judgment result;

[0009] Step S3: determining the cropping target person in the single-frame image according to the speech count parameter, and cropping the cropping target person.

[0010] On the other hand, the step S1 comprises:

[0011] Step S11: extracting one frame of the single-frame image from the video to be cropped in sequence;

[0012] Step S12: using a face recognition model to identify the person to be identified and adding a label box;

[0013] Before starting recognition, assigning a label number to the person to be identified in advance, and generating the corresponding speech count parameter;

[0014] Step S13: Return to step S11 until all the single-frame images are traversed and then turn to step S2.

[0015] On the other hand, for each of the characters to be identified, a loss count parameter is also created respectively;

[0016] The step S12 comprises:

[0017] Step S121: using a face recognition model to identify the face area appearing in the single frame image;

[0018] Step S122: matching the facial region with the person to be identified respectively, and clearing the loss count parameter of the matched person to be identified;

[0019] Step S123: determining whether there is a missing person in the person to be identified that is not matched in the single frame image;

[0020] If yes, add one to the loss count parameter corresponding to the lost person, and then go to step S13;

[0021] If not, go to step S13;

[0022] After executing step S13 and before executing step S2, the to-be-identified person whose loss count parameter reaches the loss threshold is removed.

[0023] On the other hand, the step S122 includes:

[0024] Step S1221: performing a first match on each of the facial regions with the person to be identified, and marking the facial regions that have not been successfully matched as unmatched facial regions, and marking the person to be identified that has not been successfully matched as an unmatched person;

[0025] Step S1222: For the unmatched person, extract the person position coordinates of the unmatched person from the previous single-frame image, and point the nearest unmatched facial area to the person position coordinates for matching.

[0026] On the other hand, the step S2 comprises:

[0027] Step S21: calculating the lip shape aspect ratio for each of the characters to be identified in the single frame image;

[0028] Step S22: determining whether the person to be identified has an open mouth according to the length-to-width ratio of the mouth shape;

[0029] If yes, add 1 to the value of the speech count parameter, and then go to step S23;

[0030] If not, add one to the value of the speech count parameter, and then go to step S23;

[0031] Step S23: Return to step S21 to traverse all the single-frame images and then transition to step S3.

[0032] On the other hand, the step S21 comprises:

[0033] Step S211: for the current single-frame image, respectively identify each of the characters to be identified to obtain the mouth contour points;

[0034] Step S212: extracting the vertex positions of the mouth corners on both sides and the vertex positions of the upper and lower lips from the mouth contour points as contour information;

[0035] Step S213: Calculate the aspect ratio of the mouth shape according to the contour information.

[0036] On the other hand, the step S3 comprises:

[0037] Step S31: determining a plurality of time intervals in the video to be cropped and the speaking subjects in the time intervals according to the speech count parameter;

[0038] Step S32: taking the speaking subject as a cropping object person in each of the time intervals;

[0039] Step S33: cropping the video to be cropped with the cropping object person as the center of the picture.

[0040] On the other hand, in the step S31, when the speaking subject is recognized for the first time, the time point when the speaking subject disappears in the single frame image is used as the end point of the time interval.

[0041] The above technical solution has the following advantages or beneficial effects:

[0042] In order to solve the problem that the vertical screen scene of mobile phones in the prior art is not effective for watching horizontal screen interview content, this solution introduces a process for identifying the characters to be identified appearing in the screen. By identifying the actual speaking subject in the interview interface and using it as the cropping object, the video is then recomposed and cropped, so that the horizontal screen video content can be recomposed into content suitable for vertical screen viewing, and the subject in the viewing process is effectively retained, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The embodiments of the present invention will be described more fully with reference to the attached drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.

[0044] Figure 1 is an overall schematic diagram of an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of step S1 in an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of step S12 in an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of step S122 in an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of step S2 in an embodiment of the present invention;

[0049] Figure 6 This is a schematic diagram of step S21 in an embodiment of the present invention;

[0050] Figure 7 Schematic diagram of step S3 in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0053] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0054] The present invention comprises:

[0055] A method for converting horizontal screen to vertical screen broadcasting based on character tracking, such as Figure 1 As shown, including:

[0056] Step S1: for the video to be cropped, identify the person to be identified in each single frame image in turn, and assign a speech count parameter to the person to be identified;

[0057] Step S2: in each single frame image, the length-width ratio of the mouth shape of each character to be judged is judged respectively, and then the speech count parameter of the character to be judged is updated according to the judgment result;

[0058] Step S3: determining the cropping target person in the single-frame image according to the speech count parameter, and cropping the cropping target person.

[0059] Specifically, in order to address the problem in the prior art that the vertical screen scene of mobile phones is not effective in watching horizontal screen interview content, this solution introduces a process for identifying the characters to be identified appearing in the screen. By identifying the actual speaking subject in the interview interface and using it as the cropping object, the video is then recomposed and cropped, so that the horizontal screen video content can be recomposed into content suitable for vertical screen viewing, and the subject in the viewing process is effectively retained, thereby improving the user experience.

[0060] The video to be cropped is a pre-produced TV program in horizontal screen mode, which has been edited relative to the original material, such as arranging the camera positions so that the picture focuses on the speaking subject, inserting B-roll based on the main perspective, and other processing.

[0061] The person to be judged is the person who appears in the image. After statistics and research on user habits, the inventor found that when the audience watches some studio or indoor close-up programs, people's focus is actually only on the person who is currently speaking, and they will not pay too much attention to the surrounding details. Take news programs as an example. The protagonist on the scene is only the announcer. Even if it is a normal full-screen TV program, people's attention is always on an area centered on the announcer's face. Take outdoor interview programs. Although the protagonists on the scene have become two people, in general, the audience's attention is mainly focused on an area centered on the person who is currently speaking. When the speaker changes, the audience's attention will also switch accordingly, but it is still always an area centered on the person who is currently speaking.

[0062] Taking into account the various inconveniences in converting horizontal screen programs to vertical screen format, this solution chooses to recompose and crop the video around the speaking subject based on the original horizontal screen program to form a new vertical screen program, which is convenient for viewing in a vertical screen scene.

[0063] In one embodiment, Figure 2 As shown, step S1 includes:

[0064] Step S11: extracting single-frame images one by one from the video to be cropped;

[0065] Step S12: using a face recognition model to identify the person to be identified and adding a label box;

[0066] Before starting recognition, assign a label number to the person to be identified and generate a corresponding speech count parameter;

[0067] Step S13: Return to step S11 until all single-frame images are traversed and then turn to step S2.

[0068] Specifically, in order to achieve a better recognition process of the speaking subject, in this embodiment, it is chosen to first assign a label number to the person to be identified and generate a corresponding speaking count parameter. Generally speaking, in order to achieve a better recognition effect, for a panoramic interview interface, the faces of all the people to be identified can be identified in the first frame and the label numbers can be assigned in sequence. The speaking count parameter is a counter that reflects the amount of speech of the person to be identified within a period of time, so as to identify the subject that needs to be targeted during the cropping process. After identifying the people to be identified in all single-frame images and adding the label box corresponding to the face, the recognition of the speaking subject is achieved by further identifying and counting the facial movements.

[0069] During the implementation process, you also need to set the default recognition area first. In normal pictures or movies, the entire screen is the normal screen area, but in TV programs, some areas are not the original TV program screen area. Each TV station will add station logos, QR codes, subtitles, weather forecasts, emergency notices, etc. according to their needs. If you don't want to display these contents after cropping, you can directly exclude these areas in the initial preset. By default, 100 columns of pixels on both sides of the original TV screen are removed.

[0070] Also, when outputting the annotation box, the rectangular area where the recognized face is located in the form of an array (usually the coordinates of its "upper left" and "lower right" positions), through which its "physical center" can be calculated ((x 左上 +x 右下 ) / 2,(y 左上 +y 右下 ) / 2), and the confidence level (0%-100%) of "the image in this area is a face". The rectangular areas where the confidence level is less than 60% are directly deleted.

[0071] In one embodiment, for each character to be judged, a loss count parameter is also created;

[0072] like Figure 3 As shown, step S12 includes:

[0073] Step S121: using a face recognition model to identify the facial region appearing in the single frame image;

[0074] Step S122: matching the facial regions with the persons to be identified respectively, and clearing the missing count parameters of the matched persons to be identified;

[0075] Step S123: determining whether there is a missing person in the person to be identified that is not matched in the single frame image;

[0076] If yes, the missing count parameter corresponding to the missing person is increased by one, and then the process goes to step S13;

[0077] If not, go to step S13;

[0078] After executing step S13 and before executing step S2, the characters to be identified whose loss count parameters reach the loss threshold are removed.

[0079] Specifically, considering that in some interview content, there may be audiences or other equivalent non-targeted objects, a missing count parameter is created for each person to be identified during the recognition process, which is also a counter. During the recognition process, the face recognition model is used to identify the facial area appearing in a single frame image in chronological order and match it with the person to be identified.

[0080] Generally speaking, in a normal TV program, the picture in consecutive frames will remain on a specific object; when the camera position is switched, it will usually switch back to the same object after a period of time. Unrelated characters will not appear repeatedly.

[0081] In order to reduce the time for subsequent identification and determination of cropped objects, in this embodiment, during the matching process, statistics are also collected on the unmatched persons to be identified in each match, and the missing persons to be identified in the current single-frame image are identified, and their missing count parameters are increased by one;

[0082] When the lost person is matched again, the lost count parameter of the matched person to be identified is cleared.

[0083] After traversing all single-frame images, the characters to be identified whose loss count parameter reaches the loss threshold are removed as irrelevant characters, such as the characters to be identified that have not been matched for 10 consecutive frames.

[0084] In one embodiment, Figure 4 As shown, step S122 includes:

[0085] Step S1221: each facial region is matched with the person to be identified for the first time, and the facial region that is not successfully matched is marked as an unmatched facial region, and the person to be identified that is not successfully matched is marked as an unmatched person;

[0086] Step S1222: For the unmatched person, extract the person position coordinates of the unmatched person from the previous single-frame image, and point the nearest unmatched facial area to the person position coordinates for matching.

[0087] Specifically, in order to achieve a better character matching effect, in this embodiment, the above process is adopted for a single-frame image, wherein each facial region can be matched with the character to be identified for the first time according to the recognition result of the model, thereby realizing continuous tracking of a character in a fixed position or with specific facial features. When there are still unmatched objects, the unmatched facial regions are marked as unmatched facial regions, and the unmatched characters to be identified are marked as unmatched characters. Among them, the unmatched characters should have appeared at least once in the previous single-frame image. For the unmatched characters, the character position coordinates of the unmatched characters are extracted from the previous single-frame image, and the Euclidean distance of the center coordinates of each unmatched facial region relative to the character position coordinates is calculated respectively. When the Euclidean distance of the nearest unmatched facial region relative to the character position coordinates is less than 20 pixels, the nearest unmatched facial region is pointed to the character position coordinates for matching.

[0088] In one embodiment, Figure 5 As shown, step S2 includes:

[0089] Step S21: calculating the lip shape aspect ratio of each person to be identified in a single frame image;

[0090] Step S22: determining whether the person to be judged has his mouth open according to the length-width ratio of the mouth shape;

[0091] If yes, add one to the value of the speech count parameter, and then go to step S23;

[0092] If not, the value of the speech count parameter is reduced by one, and then the process goes to step S23;

[0093] Step S23: Return to step S21 to traverse all single-frame images and then return to step S3.

[0094] Specifically, in order to achieve a better recognition effect, in this embodiment, on the basis of extracting the facial rectangular area of ​​each person to be identified in a single-frame image, the mouth is identified for each facial rectangular area, and then the length-width ratio of the mouth shape corresponding to the person to be identified is calculated. When the length-width ratio of the mouth shape is greater than the threshold, such as 0.6, it indicates that the person is currently opening his mouth, and the value of the speech count parameter is increased by one; if it is less than the threshold, it indicates that the person is closing his mouth, and the value of the speech count parameter is reduced by one. By traversing continuous single-frame images, a sequence of speech count parameters changing over time can be formed. Subsequently, by comparing the speech count parameters of different people to be identified at different time points, the actual speaker can be determined.

[0095] In one embodiment, Figure 6 As shown, step S21 includes:

[0096] Step S211: for the current single frame image, respectively identify each person to be identified to obtain the mouth contour points;

[0097] Step S212: extracting the vertex positions of the mouth corners on both sides and the vertex positions of the upper and lower lips from the mouth contour points as contour information;

[0098] Step S213: Calculate the aspect ratio of the mouth shape according to the contour information.

[0099] Specifically, during the recognition process, for each facial rectangular area of ​​the person to be identified, further facial recognition can be performed. For example, taking onnxruntime as an example, the number of points returned for each face is 68, of which 22 points are the contour points of the mouth. For the mouth contour points, the vertex positions of the corners of the mouth on both sides and the vertex positions of the upper and lower lips can be extracted as contour information. Then, a rectangular figure can be drawn according to the vertex positions of the corners of the mouth on both sides and the vertex positions of the upper and lower lips, and the aspect ratio of the mouth shape can be calculated for the rectangular figure.

[0100] In one embodiment, Figure 7 As shown, step S3 includes:

[0101] Step S31: determining multiple time intervals in the video to be cropped and the speaking subjects in the time intervals according to the speech count parameter;

[0102] Step S32: taking the speaking subject as the cropping object person in each time interval;

[0103] Step S33: cropping the video to be cropped with the cropping object person as the center of the picture.

[0104] Specifically, in order to achieve a better cropping effect, in this embodiment, after obtaining the speech count parameter, the speaker with the largest number of speeches in each single frame image is determined according to the speech count parameter. To improve the accuracy of cropping, the speech count parameter can also be compared with the threshold rectangle, for example, only when it is greater than 5, it is determined that the person is speaking. Then, the time interval is divided according to different speakers. In each time interval, the speaker is used as the cropping object person, and the cropping object person is used as the center of the screen to crop the video to be cropped.

[0105] In one embodiment, in step S31, when the speaking subject is recognized for the first time, the time point at which the speaking subject disappears in the single frame image is used as the end point of the time interval.

[0106] Among them, in order to achieve a better cropping effect, in this embodiment, after the speaking subject is identified, the speaking count parameter of the speaking subject in the time interval is also forcibly assigned to a larger value, so as to give priority to it as the selection object for the subsequent face, thereby avoiding the screen from frequently switching between multiple faces, thereby causing screen jitter, flashing and other adverse operations that affect the audience's viewing effect, until the next speaking subject is selected after the face disappears.

[0107] In the actual cutting process, the following methods are used for cutting:

[0108] Run the clipping algorithm, the formula is: y 手机 =y 节目视频 ;x 手机左 =x 人脸 -y 节目视频 *9 / 16 / 2. Its practical significance is that the height of the vertical screen of the mobile phone is the height of the original video of the TV program. Because it needs to fill the full screen, the width of the vertical screen of the mobile phone can only be y 节目视频 *9 / 16. The football should be placed on the center of the screen (not the center of the screen), because the left edge of the vertical screen of the mobile phone needs to be the x coordinate of the football minus half of the vertical screen width of the mobile phone. If the value is less than 0, it is forced to be equal to 0 to avoid cutting to the invalid area on the left side of the original video. If the value is greater than x 节目视频 -The width of the vertical screen of the mobile phone (y 节目视频 *9 / 16), then force x 手机左 =x 节目视频 -y 节目视频 *9 / 16 / 2. To avoid cropping to the invalid area on the right side of the original video. Regardless of the above, the right border is: x 手机右 =x 手机左 +The width of the vertical screen of the mobile phone (y 节目视频 *9 / 16).

[0109] It is not difficult to see that 人脸No matter what the value of does not affect the final cropping, this is because no matter where the ordinate is, the cropping area is the height of the entire screen. 人脸 The value of directly determines the center position of the area to be cropped. However, when tracking a close-range target, if this value is used accurately, once the frequency of this value changes (such as when a person is interviewing, his head keeps shaking left and right), the cropped area will also change, which will cause the audience to feel dizzy when watching for a long time. For this reason, the algorithm is optimized at the beginning of the operation, that is, if the range is still within the last cropping range, the cropping range will remain unchanged.

[0110] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for converting horizontal screen to vertical screen broadcasting based on character tracking, characterized in that: include: Step S1: for the video to be cropped, identifying the person to be identified in each single frame image in turn, and assigning a speech count parameter to the person to be identified; Step S2: in each single-frame image, judging the length-to-width ratio of the mouth shape of each of the characters to be judged, and then updating the speech counting parameter of the characters to be judged according to the judgment result; Step S3: determining the cropping target person in the single-frame image according to the speech count parameter, and cropping the cropping target person.

2. The method for converting horizontal screen to vertical screen broadcasting according to claim 1, characterized in that: The step S1 comprises: Step S11: extracting one frame of the single-frame image from the video to be cropped in sequence; Step S12: using a face recognition model to identify the person to be identified and adding a label box; Before starting recognition, assigning a label number to the person to be identified in advance, and generating the corresponding speech count parameter; Step S13: Return to step S11 until all the single-frame images are traversed and then turn to step S2.

3. The method for converting horizontal screen to vertical screen broadcasting according to claim 1, characterized in that: For each of the characters to be judged, a loss count parameter is also created respectively; The step S12 comprises: Step S121: using a face recognition model to identify the face area appearing in the single frame image; Step S122: matching the facial region with the person to be identified respectively, and clearing the loss count parameter of the matched person to be identified; Step S123: determining whether there is a missing person in the person to be identified that is not matched in the single frame image; If yes, add one to the loss count parameter corresponding to the lost person, and then go to step S13; If not, go to step S13; After executing step S13 and before executing step S2, the to-be-identified person whose loss count parameter reaches the loss threshold is removed.

4. The method for converting horizontal screen to vertical screen broadcasting according to claim 3, characterized in that: The step S122 includes: Step S1221: performing a first match on each of the facial regions with the person to be identified, and marking the facial regions that have not been successfully matched as unmatched facial regions, and marking the person to be identified that has not been successfully matched as an unmatched person; Step S1222: For the unmatched person, extract the person position coordinates of the unmatched person from the previous single-frame image, and point the nearest unmatched facial area to the person position coordinates for matching.

5. The method for converting horizontal screen to vertical screen broadcasting according to claim 1, characterized in that: The step S2 comprises: Step S21: calculating the lip shape aspect ratio for each of the characters to be identified in the single frame image; Step S22: determining whether the person to be identified has an open mouth according to the length-to-width ratio of the mouth shape; If yes, add 1 to the value of the speech count parameter, and then go to step S23; If not, the value of the speech counting parameter is reduced by one, and then the process goes to step S23; Step S23: Return to step S21 to traverse all the single-frame images and then transition to step S3.

6. The method for converting horizontal screen to vertical screen broadcasting according to claim 5, characterized in that: The step S21 comprises: Step S211: for the current single-frame image, respectively identify each of the characters to be identified to obtain the mouth contour points; Step S212: extracting the vertex positions of the mouth corners on both sides and the vertex positions of the upper and lower lips from the mouth contour points as contour information; Step S213: Calculate the aspect ratio of the mouth shape according to the contour information.

7. The method for converting horizontal screen to vertical screen broadcasting according to claim 1, characterized in that: The step S3 comprises: Step S31: determining a plurality of time intervals in the video to be cropped and the speaking subjects in the time intervals according to the speech count parameter; Step S32: taking the speaking subject as a cropping object person in each of the time intervals; Step S33: cropping the video to be cropped with the cropping object person as the center of the picture.

8. The method for converting horizontal screen to vertical screen broadcasting according to claim 7, characterized in that: In the step S31, when the speaking subject is recognized for the first time, the time point when the speaking subject disappears in the single frame image is used as the end point of the time interval.