Video picture display method and device, electronic equipment and readable medium

Abstract

The invention relates to a video picture display method and device, electronic equipment and a readable medium, and the method comprises the steps: obtaining a plurality of key frames from a to-be-displayed video picture, and enabling the key frames to comprise a target object; extracting a target frame where the target object is located from each key frame, and determining an actual object proportion of the target object according to each target frame; comparing the actual object proportion with a preset target object proportion, and determining an adjustment parameter according to a comparison result; and adjusting the display scale of the video picture according to the adjustment parameter so as to enable the video picture to be normally displayed. The current proportion of the target object in the video picture is compared with the preset proportion, then the adjustment parameter is calculated according to the difference between the two proportions, and finally the display proportion of the whole video picture is adjusted through the adjustment parameter, so that the technical effect of quickly and accurately correcting the display proportion of the video picture is achieved. The problem of picture deformation during video playing caused by incorrect video proportion is solved.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a method, device, electronic device and readable medium for displaying video images. Background Art

[0002] With the development of smart display terminals, more and more users begin to upload videos to online video platforms and social media platforms for playback. However, the videos uploaded by users may come from different devices and shooting environments, which will lead to differences in video ratios. Then, when the platform displays these videos, the screen will be distorted and needs to be adapted. However, the proportions may be unbalanced during the adaptation process. In order to meet specific playback requirements or visual effects, the aspect ratio of each frame needs to be manually adjusted. Since a video includes a large number of frames, the workload of video processing is very large. If scripts and automated tools are used to crop or fill the video to adjust the display ratio, the correct proportion of the video display cannot be automatically determined, and cropping and filling will destroy the integrity of the original video.

[0003] Therefore, based on the fact that the incorrect video ratio causes the picture to be deformed during video playback, how to quickly and accurately correct the display ratio of the video picture has become a problem that needs to be solved urgently.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The present application provides a method, device, electronic device and readable medium for displaying a video screen to solve the above-mentioned technical problem of "incorrect video ratio causing screen deformation during video playback".

[0006] According to one aspect of an embodiment of the present application, the present application provides a method for displaying a video screen, including: obtaining multiple key frames from a video screen to be displayed, wherein the key frames include a target object; extracting a target frame where the target object is located from each key frame, and determining the actual object ratio of the target object based on each target frame; comparing the actual object ratio with a preset target object ratio, and determining an adjustment parameter based on the comparison result; adjusting the display ratio of the video screen according to the adjustment parameter so that the video screen is displayed normally.

[0007] Optionally, multiple key frames are obtained from the video picture to be displayed, including: extracting multiple candidate frames from the video picture; detecting whether there is a target object in each candidate frame; if there is a target object in the candidate frame, marking the candidate frame with the target object to mark the target border where the target object is located on the candidate frame; and determining the candidate frame marked with the target border as a key frame.

[0008] Optionally, extracting multiple candidate frames from the video picture includes at least one of the following methods: extracting each video frame from the video picture in sequence according to a preset interval, and using each video frame as a candidate frame, wherein the preset interval is a preset time interval or a preset frame number interval; randomly extracting a target number of multiple video frames from the video picture, and using the multiple video frames as candidate frames.

[0009] Optionally, before determining the candidate frame marked with the target border as a key frame, the method also includes verifying the target object in the target border in the following manner: extracting multiple first feature points in the target border, and determining a first relative position relationship between each first feature point; judging whether there is a corresponding relationship between each first feature point and each second feature point, and judging whether the first relative position relationship between each first feature point is the same as the second relative position relationship between each second feature point, wherein the second feature point is a key feature point pre-defined for the target object, and the second relative position relationship is pre-established based on each key feature point; if there is a corresponding relationship between the first feature point and the second feature point, and the first relative position relationship is the same as the second relative position relationship, then it is determined that the target object in the target border has passed the verification.

[0010] Optionally, determining the actual object ratio of the target object according to each target border includes: obtaining the pixel height and pixel width of each target border; dividing the pixel width of each target border by the pixel height to obtain multiple first ratios; filtering out abnormal values ​​in each first ratio to obtain at least one second ratio; and determining the actual object ratio according to a preset calculation strategy and the second ratio.

[0011] Optionally, comparing the actual object ratio with a preset target object ratio, and determining the adjustment parameter according to the comparison result, includes: if the actual object ratio is different from the target object ratio, calculating the deformation ratio according to the actual object ratio and the target object ratio; and determining the adjustment parameter according to the actual object ratio, the target object ratio and the deformation ratio. Optionally, determining the adjustment parameter according to the actual object ratio, the target object ratio and the deformation ratio includes: if the deformation ratio is greater than the target value, determining a width reduction value according to the actual object ratio and the target object ratio, and determining the width reduction value as the adjustment parameter; if the deformation ratio is less than the target value, determining a height reduction value according to the actual object ratio and the target object ratio, and determining the height reduction value as the adjustment parameter.

[0012] According to another aspect of an embodiment of the present application, the present application provides a display device for a video screen, including: an acquisition module, used to acquire multiple key frames from the video screen to be displayed, wherein the key frames include a target object; an extraction module, used to extract a target frame where the target object is located from each key frame, and determine the actual object ratio of the target object based on each target frame; a comparison module, used to compare the actual object ratio with a preset target object ratio, and determine an adjustment parameter based on the comparison result; and an adjustment module, used to adjust the display ratio of the video screen according to the adjustment parameter so that the video screen is displayed normally.

[0013] According to another aspect of an embodiment of the present application, the present application provides an electronic device, including a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, the memory and the processor communicate through the communication bus and the communication interface, and the steps of the above method are implemented when the processor executes the computer program.

[0014] According to another aspect of an embodiment of the present application, the present application also provides a computer-readable medium having a non-volatile program code executable by a processor, and the program code enables the processor to execute the above method.

[0015] The above technical solution provided by the embodiment of the present application has the following advantages compared with the related art:

[0016] The present application discloses a method for displaying a video screen, including: obtaining multiple key frames from a video screen to be displayed, wherein the key frames include a target object; extracting a target frame where the target object is located from each key frame, and determining the actual object ratio of the target object according to each target frame; comparing the actual object ratio with a preset target object ratio, and determining an adjustment parameter according to the comparison result; adjusting the display ratio of the video screen according to the adjustment parameter, so that the video screen is displayed normally. By comparing the current ratio of the target object in the video screen with the preset ratio, and then calculating the adjustment parameter according to the difference between the two ratios, and finally adjusting the display ratio of the entire video screen by adjusting the parameter, the technical effect of quickly and accurately correcting the display ratio of the video screen can be achieved, and the problem of screen deformation during video playback caused by incorrect video ratio is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A flowchart of an optional method for displaying a video screen according to an embodiment of the present application;

[0020] Figure 2 A flowchart of another optional method for displaying a video screen according to an embodiment of the present application;

[0021] Figure 3 A block diagram of an optional video display device provided according to an embodiment of the present application;

[0022] Figure 4 A schematic diagram of an optional electronic device structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0024] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application and have no specific meaning. Therefore, "module" and "component" can be used interchangeably.

[0025] With the development of smart display terminals, more and more users begin to upload videos to online video platforms and social media platforms for playback. However, the videos uploaded by users may come from different devices and shooting environments, which will lead to differences in video ratios. Then, when the platform displays these videos, the screen will be distorted and needs to be adapted. However, the proportions may be unbalanced during the adaptation process. In order to meet specific playback requirements or visual effects, the aspect ratio of each frame needs to be manually adjusted. Since a video includes a large number of frames, the workload of video processing is very large. If scripts and automated tools are used to crop or fill the video to adjust the display ratio, the correct proportion of the video display cannot be automatically determined, and cropping and filling will destroy the integrity of the original video.

[0026] Therefore, based on the fact that the incorrect video ratio causes the picture to be deformed during video playback, how to quickly and accurately correct the display ratio of the video picture has become a problem that needs to be solved urgently.

[0027] In order to solve the problem mentioned in the background technology, according to one aspect of the embodiment of the present application, a method for displaying a video screen is provided, such as Figure 1 As shown, including:

[0028] Step 101, obtaining a plurality of key frames from a video image to be displayed, wherein the key frames include a target object;

[0029] Step 103, extracting the target frame where the target object is located from each key frame, and determining the actual object ratio of the target object according to each target frame;

[0030] Step 105, comparing the actual object ratio with the preset target object ratio, and determining the adjustment parameter according to the comparison result;

[0031] Step 107, adjusting the display ratio of the video picture according to the adjustment parameters so that the video picture is displayed normally.

[0032] By adopting the video screen display method provided in the present application, the video can be adjusted to the correct display ratio before it is played, so that the video can be adapted for playback on any platform, and there is no need for manual frame-by-frame adjustment, which can save a lot of manpower.

[0033] As a preferred embodiment of the present application, the present application uses human face as the target object to illustrate the entire embodiment. In fact, the target object can also be other elements, such as animals, plants, and fruits, etc. The corresponding elements can be selected as the target object according to different application scenarios. In other words, the present application is not limited to the processing of video images containing people.

[0034] This application realizes automatic correction of video ratio by performing face detection, multi-frame sampling, face ratio calculation, outlier filtering, comparison and correction on the video screen to be played. This application combines modern computer vision technology and statistical methods to automatically adjust the video ratio efficiently and accurately, especially suitable for processing large amounts of video content, thereby significantly improving efficiency and user experience.

[0035] This application needs to first determine whether the ratio of the current video screen to be played is the correct playback ratio. The technical concept is to extract multiple key frames in the video screen to analyze the degree of deformation. According to the degree of deformation, the adjustment parameters can be determined. The adjustment parameters are used to correct the ratio of the entire video screen so that the video screen can be displayed normally.

[0036] As an optional embodiment, multiple key frames are obtained from a video screen to be displayed, including: extracting multiple candidate frames from the video screen; detecting whether there is a target object in each candidate frame; if there is a target object in the candidate frame, marking the candidate frame with the target object to mark a target border where the target object is located on the candidate frame; and determining the candidate frame marked with the target border as a key frame.

[0037] Since the actual object ratio (ie, face aspect ratio) of the target object in the key frame needs to be extracted later, only candidate frames containing the target object can be used as key frames.

[0038] For example, using computer vision technology, such as the dlib library and OpenCV, through algorithms such as image feature point detection or Convolutional Neural Networks (CNN), the target object (ie, face) in the video frame can be detected efficiently and accurately.

[0039] Marking the candidate frame with the target object means marking the minimum rectangular frame where the target object is located. In addition, the boundary of the face can also be located and marked as the target frame, but the width and height of the face boundary are the same as the width and height of the minimum rectangular frame where the face is located, so this application directly marks the minimum rectangular frame as the target frame.

[0040] As an optional embodiment, extracting multiple candidate frames from a video picture includes at least one of the following methods: extracting each video frame from the video picture in sequence according to a preset interval, and using each video frame as a candidate frame, wherein the preset interval is a preset time interval or a preset frame number interval; randomly extracting a target number of multiple video frames from the video picture, and using the multiple video frames as candidate frames.

[0041] In order to avoid the accuracy of the ratio calculation being affected by special cases or outliers in a single frame, it is necessary to extract frames from multiple time points in the video to obtain more diverse data samples.

[0042] The candidate frames may be extracted randomly or at preset intervals.

[0043] When the preset interval is a preset time interval, it is necessary to first determine a fixed time interval, such as extracting one frame per second; then use video processing software or a programming library to read the video screen in the video file; then, according to the preset time interval, extract the corresponding frames from the video screen in turn; finally, save these extracted frames as candidate frames.

[0044] When the preset interval is a preset frame number interval, it is necessary to first determine a fixed frame number interval (for example, extract one frame every 5 frames); then use video processing software or a programming library to read the video screen in the video file; then, according to the preset frame number interval, extract the corresponding frames from the video in turn; finally, save these extracted frames as candidate frames.

[0045] By extracting frames at preset intervals, you can ensure that the time interval between frames is fixed, which helps to maintain the regularity of the extracted frames. You can flexibly adjust the preset interval duration according to the length and content of the video to control the number and density of extracted frames.

[0046] When randomly extracting a target number of multiple video frames from a video screen, it is necessary to first determine the target number of candidate frames to be extracted based on task requirements; then use a random number generation algorithm or library function to generate a random number sequence within a specified range, and these random numbers correspond to the frame numbers or timestamps in the video; then extract the corresponding frames from the video based on the generated random number sequence; finally, save these extracted frames as candidate frames.

[0047] Randomly extracting frames can ensure that the extracted frames are randomly distributed on the video content, thereby increasing the diversity and representativeness of the extracted frames, helping to capture different scenes and details in the video and improving the comprehensiveness of video content extraction.

[0048] After obtaining the candidate frames, it is necessary to mark the target borders of the candidate frames and find the key frames. Since the key frames are representative frame images extracted from the video images, and this application needs to use the target borders where the target objects are located, the key frames must contain the target objects, that is, the faces. Therefore, after obtaining the candidate frames, in order to ensure that the faces detected in the candidate frames are faces, the candidate frames need to be verified. If the verification is passed, no processing is required. If the candidate frame verification fails, it means that the candidate frame does not contain a face, then the candidate frame needs to be eliminated to ensure the accuracy of the implementation of the entire solution.

[0049] Therefore, after selecting and marking the candidate frames with the target objects, it is necessary to verify whether the elements within the marked target borders in the candidate frames are faces. Next, how to verify the selected candidate frames is described.

[0050] As an optional embodiment, before determining the candidate frame marked with the target border as a key frame, the method also includes verifying the target object in the target border in the following manner: extracting multiple first feature points in the target border, and determining a first relative position relationship between each first feature point; judging whether there is a corresponding relationship between each first feature point and each second feature point, and judging whether the first relative position relationship between each first feature point is the same as the second relative position relationship between each second feature point, wherein the second feature point is a key feature point pre-defined for the target object, and the second relative position relationship is pre-established based on each key feature point; if there is a corresponding relationship between the first feature point and the second feature point, and the first relative position relationship is the same as the second relative position relationship, then it is determined that the target object in the target border has passed the verification.

[0051] Feature point extraction is one of the key technologies in face recognition technology. By extracting the feature points of the face, it can provide important recognition basis for face recognition.

[0052] For example, the Facemark class in OpenCV or the facial feature point detection algorithm in the Dlib library can be used to extract feature points in the target border, and key feature points (ie, first feature points) representing eyes, nose, and mouth can be selected from the extracted feature points.

[0053] In the present application, the second feature point is the key feature point pre-defined for the target object, specifically including the geometric features of the face, including the position and size relationship of facial organs, such as eyes, nose, and mouth.

[0054] The first feature point needs to correspond to the second feature point, which means that the feature point corresponding to the second feature point needs to be detected in the target frame. For example, if the pre-defined second features are left eye, right eye, nose and mouth, then the first feature points extracted should also be left eye, right eye, nose and mouth.

[0055] After extracting each first feature point, it is necessary to determine the first relative position relationship of each first feature point. For example, among the extracted first feature points, select one or more points as reference points, and establish a local coordinate system with these reference points as the origin; according to the established coordinate system, calculate the relative position relationship of other feature points relative to the reference point, and the relative position relationship can be distance, angle or direction; use mathematical expressions or data structures to describe these relative position relationships. For example, vectors, matrices or graph structures can be used to represent the relative position relationship between feature points.

[0056] The relative position relationship is the relative position between each feature point, which can be defined in detail or roughly (it can be defined in advance according to actual needs). For example, the mouth is located directly below the nose, and the center point of the mouth and the lower end of the nose (such as below the tip of the nose) are roughly on the same vertical line; the left eye is symmetrical with the right eye, and the axis of symmetry is the line where the center point of the nose and the center point of the mouth are located.

[0057] The second relative position relationship is pre-defined based on the second feature points, and the first relative position relationship is determined based on the extracted first feature points. The first relative position relationship needs to be the same as the second relative position relationship to determine that the candidate frame does contain a face.

[0058] It should be noted that the first feature point and the second feature point are both defined according to the target object. If the target object is not a face, the corresponding feature points and the relative position relationship also need to be redefined.

[0059] By further performing face recognition on the selected candidate frames, it can be ensured that the final selected key frames must include faces, which can reduce the errors in subsequent calculation and adjustment parameters, making the entire picture adjustment plan more accurate.

[0060] As an optional embodiment, determining the actual object ratio of the target object according to each target border includes: obtaining the pixel height and pixel width of each target border; dividing the pixel width of each target border by the pixel height to obtain multiple first ratios; filtering out abnormal values ​​in each first ratio to obtain at least one second ratio; and determining the actual object ratio according to a preset calculation strategy and the second ratio.

[0061] Faces in video frames have relatively fixed proportions in most cases and can therefore be used as reference points for correction.

[0062] After detecting the face area, the aspect ratio is directly calculated. The natural aspect ratio of the face is highly consistent among the general population, so it can be used as a standard reference for video ratio correction.

[0063] Since the target border in the present application is the smallest rectangular frame where the target object is located, the pixel height of the target border can be regarded as the pixel height of the face, and the pixel width of the target border can be regarded as the pixel width of the face.

[0064] The pixel height and pixel width can be calculated by the coordinate information of the target frame. For example, the coordinates of the upper left corner and the lower right corner of the target frame are (x1, y1) and (x2, y2) respectively, then the pixel width = x2-x1, and the pixel height = y2-y1.

[0065] For each target frame, the pixel width is divided by the pixel height, and the ratio of the width to the height (ie, the first ratio) is the actual aspect ratio of the face in the target frame.

[0066] Due to noise in the image, changes in the posture of the target object, inaccurate border extraction, etc., there may be outliers in each first ratio, so it is also necessary to filter out the outliers in the first ratio to ensure the validity of the data.

[0067] Optionally, outliers can be identified by statistical methods (such as mean, standard deviation, median, quartiles, etc.) or machine learning algorithms (such as clustering, classification, etc.).

[0068] Optionally, a threshold value (eg, twice the average value of each first ratio value) may be preset to determine the first ratio value exceeding the threshold value as an abnormal value.

[0069] The identified outliers are removed from the set of first ratios to obtain a set of ratios after the outliers are removed, wherein the set includes at least one second ratio.

[0070] Specifically, determining the actual object ratio according to the preset calculation strategy and the second ratio is actually selecting a calculation method as the preset calculation strategy to select a representative value from multiple second ratios as the actual object ratio.

[0071] For example, the actual object ratio can be obtained by selecting the median. The selected second ratios are sorted according to the numerical value, and the total number of numerical values, that is, N, is calculated; if N is an odd number, the second ratio at the position (N+1) / 2 after sorting is selected as the actual object ratio; if N is an even number, the average of the two numerical values ​​at the positions N / 2 and (N / 2)+1 after sorting is selected as the actual object ratio.

[0072] For example, the actual object ratio can also be obtained by calculating an average value. The sum of the second ratios is calculated, and then the sum is divided by the total number of the second ratios to obtain an average value as the actual object ratio.

[0073] The actual object ratio is the aspect ratio of the face in each video frame.

[0074] By eliminating outliers, the errors caused by outliers can be reduced, making the final selection results more accurate.

[0075] After outliers are identified, the median can be selected as the final calculation result. The median is a robust statistic that is insensitive to outliers. Even if there are some extreme values ​​in the data, the median value will not be greatly affected. Therefore, selecting the median as the measurement value after removing outliers can enhance the robustness of the final calculation result.

[0076] As an optional embodiment, the actual object ratio is compared with the preset target object ratio, and the adjustment parameters are determined according to the comparison result, including: if the actual object ratio is different from the target object ratio, the deformation ratio is calculated according to the actual object ratio and the target object ratio; the adjustment parameters are determined according to the actual object ratio, the target object ratio and the deformation ratio.

[0077] Since the human face is displayed in the video screen, the degree of deformation of the human face ratio may be equal to the degree of deformation of the video screen.

[0078] If the video aspect ratio deviates from the standard value (i.e., the target object ratio), the extent to which the video is stretched or compressed is determined. Based on this inference, the aspect ratio parameters of the video are adjusted to restore it to the correct ratio, thereby achieving the effect of correcting the display ratio of the video screen.

[0079] A standard face width-to-height ratio is preset as a reference (i.e., the target object ratio). This standard value can be determined based on the statistical results of a large amount of face data, or it can be set based on a specific application scenario or standard, and this application does not limit this.

[0080] The deformation ratio is calculated based on the actual object ratio and the target object ratio. The actual object ratio is compared with the target object ratio to calculate the deformation ratio, which reflects the degree to which the video image is stretched or compressed in the width or height direction.

[0081] According to the deformation ratio, it can be determined whether the current display ratio of the video screen is stretched or compressed, so as to determine the adjustment parameters according to the actual object ratio and the target object ratio. The adjustment parameters are actually used to adjust the actual object ratio to the target object ratio. Since the face and the video screen are in a part-to-whole relationship, the deformation degree of the video screen is the same as that of the face, so the adjustment parameters can be used to adjust the display ratio of the video screen, thereby adjusting each display element in the video screen. When the video screen is displayed in a normal ratio, the face will be displayed in the correct ratio.

[0082] As an optional embodiment, the adjustment parameters are determined according to the actual object ratio, the target object ratio and the deformation ratio, including: if the deformation ratio is greater than the target value, the width reduction value is determined according to the actual object ratio and the target object ratio, and the width reduction value is determined as the adjustment parameter; if the deformation ratio is less than the target value, the height reduction value is determined according to the actual object ratio and the target object ratio, and the height reduction value is determined as the adjustment parameter.

[0083] The target value is usually set to 1. If the deformation ratio is greater than 1, it means that the video is stretched horizontally. If the deformation ratio is less than 1, it means that the video is stretched vertically.

[0084] When the deformation ratio is greater than 1, it means that the video is stretched in width but may be relatively short in height. At this time, a width reduction value needs to be calculated. This value is determined based on the difference between the actual object ratio (i.e., the aspect ratio of the current video) and the target object ratio (i.e., the standard face aspect ratio). This width reduction value is set as an adjustment parameter to adjust the display width of the video screen. During the adjustment process, the display width of the video screen is reduced so that the display ratio of the adjusted video screen is a normal ratio.

[0085] When the deformation ratio is less than 1, it means that the video is stretched in height, but may be relatively narrow in width. At this time, a height reduction value needs to be calculated, which is also determined based on the difference between the actual object ratio and the target object ratio. Then this height reduction value is set as the adjustment parameter of the video to reduce the display height of the video during the adjustment process, so that the display ratio of the adjusted video screen is normal.

[0086] Optionally, when the deformation ratio is greater than 1, the adjustment parameter may also be a height reduction value, and when the deformation ratio is less than 1, the adjustment parameter may also be a width increase value.

[0087] Optionally, the adjustment parameters provided in the present application are adjustment values ​​of height or width. In actual situations, the adjustment parameters may also include adjustment values ​​of height and width at the same time, that is, the display height and display width are adjusted at the same time, as long as the adjusted display ratio is a normal ratio.

[0088] Figure 2 The flowchart of the method for displaying a video screen with a face as the target object provided by the present application includes the following steps as shown in the figure:

[0089] Step 201, video frame extraction;

[0090] Step 202, multi-frame sampling: uniformly / randomly select multiple frames;

[0091] Step 203, face detection: detecting faces in the frame and extracting feature points;

[0092] Step 204, face ratio calculation: calculating the aspect ratio of each face frame;

[0093] Step 205, outlier filtering: filtering outlier data using statistical methods;

[0094] Step 206, aspect ratio comparison and correction: compare the calculation result with the standard value and correct the video ratio.

[0095] Through face detection, multi-frame sampling, face ratio calculation, outlier filtering, comparison and correction, an automated video ratio correction method is constructed. Combining modern computer vision technology and statistical methods, it can automatically adjust the video ratio efficiently and accurately, and is particularly suitable for processing large amounts of video content, thereby significantly improving efficiency and user experience.

[0096] The present application discloses a method for displaying a video screen, including: obtaining multiple key frames from a video screen to be displayed, wherein the key frames include a target object; extracting a target frame where the target object is located from each key frame, and determining the actual object ratio of the target object according to each target frame; comparing the actual object ratio with a preset target object ratio, and determining an adjustment parameter according to the comparison result; adjusting the display ratio of the video screen according to the adjustment parameter, so that the video screen is displayed normally. By comparing the current ratio of the target object in the video screen with the preset ratio, and then calculating the adjustment parameter according to the difference between the two ratios, and finally adjusting the display ratio of the entire video screen by adjusting the parameter, the technical effect of quickly and accurately correcting the display ratio of the video screen can be achieved, and the problem of screen deformation during video playback caused by incorrect video ratio is solved.

[0097] According to another aspect of the embodiment of the present application, the present application provides a video display device, such as Figure 3 As shown, including:

[0098] An acquisition module 302 is used to acquire a plurality of key frames from a video picture to be displayed, wherein the key frames include a target object;

[0099] An extraction module 304 is used to extract a target frame where the target object is located from each key frame, and determine the actual object ratio of the target object according to each target frame;

[0100] A comparison module 306 is used to compare the actual object ratio with the preset target object ratio and determine the adjustment parameter according to the comparison result;

[0101] The adjustment module 308 is used to adjust the display ratio of the video picture according to the adjustment parameters so that the video picture can be displayed normally.

[0102] It should be noted that the acquisition module 302 in this embodiment can be used to execute step 101 in the embodiment of the present application, the extraction module 304 in this embodiment can be used to execute step 103 in the embodiment of the present application, the comparison module 306 in this embodiment can be used to execute step 105 in the embodiment of the present application, and the adjustment module 308 in this embodiment can be used to execute step 107 in the embodiment of the present application.

[0103] Optionally, the acquisition module 302 includes:

[0104] The extraction submodule is used to extract multiple candidate frames from the video screen; detect whether there is a target object in each candidate frame;

[0105] A marking submodule, for marking the candidate frame containing the target object if the target object exists in the candidate frame, so as to mark the target frame where the target object is located on the candidate frame;

[0106] The determination submodule is used to determine the candidate frames marked with the target border as key frames.

[0107] Optionally, the extraction submodule is also used to extract multiple candidate frames from the video screen in at least one of the following ways: extracting each video frame from the video screen in sequence according to a preset interval, and using each video frame as a candidate frame, wherein the preset interval is a preset time interval or a preset frame number interval; randomly extracting a target number of multiple video frames from the video screen, and using the multiple video frames as candidate frames.

[0108] Optionally, the acquisition module 302 also includes a verification submodule for verifying the target object in the target border in the following manner before determining the candidate frame marked with the target border as a key frame: extracting multiple first feature points in the target border, and determining a first relative position relationship between each first feature point; judging whether there is a corresponding relationship between each first feature point and each second feature point, and judging whether the first relative position relationship between each first feature point is the same as the second relative position relationship between each second feature point, wherein the second feature point is a key feature point pre-defined for the target object, and the second relative position relationship is pre-established based on each key feature point; if there is a corresponding relationship between the first feature point and the second feature point, and the first relative position relationship is the same as the second relative position relationship, then it is determined that the target object in the target border has passed the verification.

[0109] Optionally, the extraction module 304 is also used to obtain the pixel height and pixel width of each target border; divide the pixel width of each target border by the pixel height to obtain multiple first ratios; filter out abnormal values ​​in each first ratio to obtain at least one second ratio; determine the actual object ratio according to the preset calculation strategy and the second ratio.

[0110] Optionally, the comparison module 306 is further configured to calculate the deformation ratio according to the actual object ratio and the target object ratio if the actual object ratio is different from the target object ratio; and determine the adjustment parameter according to the actual object ratio, the target object ratio and the deformation ratio. Optionally, the adjustment parameter is determined according to the actual object ratio, the target object ratio and the deformation ratio, including: if the deformation ratio is greater than the target value, a width reduction value is determined according to the actual object ratio and the target object ratio, and the width reduction value is determined as the adjustment parameter; if the deformation ratio is less than the target value, a height reduction value is determined according to the actual object ratio and the target object ratio, and the height reduction value is determined as the adjustment parameter.

[0111] It should be noted here that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments.

[0112] According to another aspect of the embodiments of the present application, the present application provides an electronic device, such as Figure 4 As shown, it includes a memory 401, a processor 403, a communication interface 405 and a communication bus 407. The memory 401 stores a computer program that can be run on the processor 403. The memory 401 and the processor 403 communicate through the communication interface 405 and the communication bus 407. When the processor 403 executes the computer program, the steps of the above method are implemented.

[0113] The memory and processor in the above electronic device communicate via a communication bus and a communication interface. The communication bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus may be divided into an address bus, a data bus, a control bus, etc.

[0114] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0115] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0116] According to another aspect of the embodiments of the present application, a computer-readable medium having a non-volatile program code executable by a processor is provided.

[0117] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0118] When the embodiments of the present application are specifically implemented, reference may be made to the above-mentioned embodiments, which have corresponding technical effects.

[0119] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0120] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0121] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0122] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0123] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0124] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0126] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk. It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0127] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for displaying a video screen, characterized in that: include: Acquire a plurality of key frames from a video screen to be displayed, wherein the key frames include a target object; Extracting a target frame where the target object is located from each of the key frames, and determining an actual object ratio of the target object according to each of the target frames; Comparing the actual object ratio with a preset target object ratio, and determining an adjustment parameter according to the comparison result; The display ratio of the video picture is adjusted according to the adjustment parameters so that the video picture is displayed normally.

2. The method according to claim 1, characterized in that The step of obtaining a plurality of key frames from a video picture to be displayed includes: Extracting multiple candidate frames from the video picture; Detecting whether the target object exists in each of the candidate frames; If the target object exists in the candidate frame, marking the candidate frame in which the target object exists, so as to mark the target frame where the target object is located on the candidate frame; The candidate frame marked with the target frame is determined as the key frame.

3. The method according to claim 2, characterized in that The extracting of multiple candidate frames from the video picture includes at least one of the following methods: Extracting each video frame from the video picture in sequence according to a preset interval, and using each video frame as the candidate frame, wherein the preset interval is a preset time interval or a preset frame number interval; A target number of multiple video frames are randomly extracted from the video picture, and the multiple video frames are used as the candidate frames.

4. The method according to claim 2, characterized in that: Before determining the candidate frame marked with the target border as the key frame, the method further includes verifying the target object in the target border in the following manner: Extracting a plurality of first feature points in the target frame, and determining a first relative position relationship between each of the first feature points; Determine whether there is a corresponding relationship between each of the first feature points and each of the second feature points, and determine whether the first relative position relationship between each of the first feature points is the same as the second relative position relationship between each of the second feature points, wherein the second feature points are key feature points pre-defined for the target object, and the second relative position relationship is pre-established based on each of the key feature points; If the first feature point and the second feature point have the corresponding relationship, and the first relative position relationship is the same as the second relative position relationship, it is determined that the target object in the target bounding box has passed the verification.

5. The method according to claim 1, characterized in that The determining the actual object ratio of the target object according to each of the target frames includes: Obtain the pixel height and pixel width of each target border; Divide the pixel width of each target frame by the pixel height to obtain a plurality of first ratios; Filter out abnormal values ​​in each of the first ratios to obtain at least one second ratio; The actual object ratio is determined according to a preset calculation strategy and the second ratio.

6. The method according to claim 1, characterized in that The comparing the actual object ratio with the preset target object ratio and determining the adjustment parameter according to the comparison result includes: If the actual object ratio is different from the target object ratio, calculating the deformation ratio according to the actual object ratio and the target object ratio; The adjustment parameter is determined according to the actual object ratio, the target object ratio and the deformation ratio.

7. The method according to claim 6, characterized in that The determining the adjustment parameter according to the actual object ratio, the target object ratio and the deformation ratio includes: If the deformation ratio is greater than the target value, determining a width reduction value according to the actual object ratio and the target object ratio, and determining the width reduction value as the adjustment parameter; If the deformation ratio is smaller than the target value, a height reduction value is determined according to the actual object ratio and the target object ratio, and the height reduction value is determined as the adjustment parameter.

8. A video display device, characterized in that: include: An acquisition module, used for acquiring a plurality of key frames from a video picture to be displayed, wherein the key frames include a target object; An extraction module, used for extracting a target frame where the target object is located from each of the key frames, and determining an actual object ratio of the target object according to each of the target frames; A comparison module, used to compare the actual object ratio with a preset target object ratio, and determine an adjustment parameter according to the comparison result; The adjustment module is used to adjust the display ratio of the video picture according to the adjustment parameters so that the video picture can be displayed normally.

9. An electronic device, comprising a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, and the memory and the processor communicate through the communication bus and the communication interface, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer readable medium having a non-volatile program code executable by a processor, characterized in that: The program code enables the processor to execute the method according to any one of claims 1 to 7.

Images