Video content integrity evaluation method and electronic device

Abstract

A method for evaluating video content. In this method, an electronic device can obtain element groups from the video to be tested based on elements identified in the image frames of the video to be tested, and determine the content evaluation result of the video to be tested based on the element group attributes of the element groups. By implementing the technical solution provided by this application, the electronic device can automatically evaluate the video to be tested, improving the efficiency of video review.

Description

Technical Field

[0001] The present application relates to the field of multimedia technology, and in particular to a method for evaluating the integrity of video content and an electronic device. Background Art

[0002] With the widespread adoption of smartphones and the increasing maturity of the mobile internet, the volume of videos produced and shared by users has increased significantly. The average daily video upload volume on video platforms continues to rise. This substantial increase in video volume poses a challenge to the platforms' video quality review capabilities. Video platforms need to assess the completeness of user-uploaded videos. The primary goal is to verify whether any missing elements (subtitles, subject matter, characters, etc.) in the video are causing a poor viewing experience or hindering user comprehension.

[0003] The current method for reviewing the completeness of video content provided by the industry is mainly manual review. The steps are as follows: (1) The video review task is issued to the reviewer; (2) The reviewer watches the video on the review platform and can choose to watch it at normal speed, accelerated speed, or skip. (3) If the reviewer finds that the video content is incomplete and affects the viewing experience of the video viewers, it will be marked as "video content incomplete".

[0004] However, with the explosive growth in the number of short videos, faced with the demand for large-scale video review, the labor costs are high, and the efficiency of manual review can no longer meet the review needs of video platforms. Summary of the Invention

[0005] The embodiments of the present application provide a method and electronic device for evaluating the completeness of video content, which are used to automatically review videos uploaded by users and improve the review efficiency of video platforms.

[0006] In a first aspect, the present application provides a method for evaluating video picture content, which includes: an electronic device identifying elements in at least two image frames in a video to be detected, the at least two image frames including a first image frame and a second image frame, and the element is a basic element in the image frame; the electronic device determines an element group in the video to be detected based on the identified elements, including a first element group, the first element group including the first element in the first image frame and the first element in the second image frame; the electronic device determines the element group attributes of the first element group, the element group attributes of the first element group including a first incomplete parameter value, and the first incomplete parameter value is used to indicate the degree of incompleteness of the first element group; the electronic device determines the picture content evaluation result of the video to be detected based at least on the element group attributes of the first element group.

[0007] In the above embodiment, by automatically acquiring element groups in the video to be detected and determining the element group attributes of the element groups, the image content evaluation result of the video to be detected can be determined based on the element group attributes of the element groups. The evaluation result of the video to be detected can be directly obtained without manual intervention, allowing the video platform to automatically review the video to be detected, thereby improving the video platform's review efficiency.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the electronic device determines the picture content evaluation result of the video to be detected based on at least the element group attributes of the first element group, specifically including: the electronic device determines the impact of the first element group on the picture content of the video to be detected based on the element group attributes of the first element group; the electronic device determines the picture content evaluation result of the video to be detected based on at least the impact of the first element group on the picture content of the video to be detected.

[0009] In the above embodiment, the electronic device can first determine the impact of the first element group on the content of the video picture to be detected based on the element group attributes of the first element group, and then determine the picture content evaluation result of the video to be detected based on the impact result. It can more clearly determine the impact of each element group on the video picture content, so that the evaluation result of the video picture content is more accurate.

[0010] In combination with some embodiments of the first aspect, in some embodiments, the electronic device determines the impact of the first element group on the content of the video picture to be detected based on the element group attributes of the first element group, specifically including: the electronic device determines the impact of the first element group on the content of the video picture to be detected based on the element group attributes of the first element group and the preset element group impact picture integrity rules.

[0011] In the above embodiment, the preset element group influence rule for picture integrity can be used to quickly and accurately judge the influence of the element group on the content of the video picture to be detected. By judging whether the element group has an influence on the content of the video picture to be detected, the element group that has an influence on the video picture to be detected can be screened out to participate in subsequent calculations, thereby reducing the computational complexity of subsequent calculations.

[0012] In combination with some embodiments of the first aspect, in some embodiments, the electronic device determines the picture content evaluation result of the video to be detected based on at least the impact result of the first element group on the picture content of the video to be detected, specifically including: the electronic device determines whether the video to be detected is complete based on at least the impact result of the first element group on the picture content of the video to be detected; and / or, the electronic device determines the score of the video to be detected based on at least the impact result of the first element group on the picture content of the video to be detected and the classification information of the video to be detected; and / or, the electronic device determines the modification suggestions for the video to be detected based on at least the impact result of the first element group on the picture content of the video to be detected and the element group attributes of the first element group, including the modification suggestions for the first element group.

[0013] In the above embodiment, the video platform determines whether the detection video is complete and can feed back the completeness result to the user or video producer to complete the review; the video platform determines the score of the detection video and can feed back the score result to the user or video producer, which helps the user or video producer to understand the completeness of the video to be detected more objectively and visually; the video platform determines the modification suggestions for the video to be detected and can feed back the modification suggestions to the user as a specific evaluation result, which helps the user or video producer to make targeted modifications according to the modification suggestions and improve the user experience.

[0014] In combination with some embodiments of the first aspect, in some embodiments, the electronic device determines the picture content evaluation result of the video to be detected based on at least the element group attributes of the first element group, specifically including: the electronic device determines whether the video to be detected is complete based on at least the element group attributes of the first element group; and / or, the electronic device determines the score of the video to be detected based on at least the element group attributes of the first element group and the classification information of the video to be detected; and / or, the electronic device determines modification suggestions for the video to be detected based on at least the element group attributes of the first element group, including modification suggestions for the first element group.

[0015] In the above embodiment, the video platform determines whether the detection video is complete and can feed back the completeness result to the user or video producer to complete the review; the video platform determines the score of the detection video and can feed back the score result to the user or video producer, which helps the user or video producer to connect the completeness of the video to be detected more objectively and visually; the video platform determines the modification suggestions for the video to be detected and can feed back the modification suggestions to the user as a specific evaluation result, which helps the user or video producer to make targeted modifications according to the modification suggestions and improve the user experience.

[0016] In combination with some embodiments of the first aspect, in some embodiments, the modification suggestion of the first element group includes at least one of: passing, not processing, cropping, smearing, deleting frames, and blurring.

[0017] In the above embodiment, the video platform can determine the modification suggestions for the element group in the video to be detected, and can feed back the modification suggestions for the element group to the video producer or user, so that the video producer and user can obtain targeted, concise, and fool-proof modification suggestions, and can make corresponding modifications to the element group based on the modification suggestions, thereby improving the completeness of the video and enhancing the user experience.

[0018] In combination with some embodiments of the first aspect, in some embodiments, the electronic device determines the element group in the video to be detected based on the identified elements, including the first element group, specifically including: the electronic device forms an element group with the elements representing the same basic element in the at least two image frames identified.

[0019] In the above embodiment, the video platform determines the element group by element, which preserves the spatiotemporal integrity of the elements in the video to be detected, and helps to improve the accuracy of the evaluation results of the content of the video to be detected.

[0020] In combination with some embodiments of the first aspect, in some embodiments, the electronic device will identify the elements representing the same basic element in the at least two image frames to form an element group, including a first element group, specifically including: the electronic device determines the first element similarity between the first element in the first image frame and the first element in the second image frame, and the element similarity is used to indicate the degree of similarity between the element images; the electronic device determines that the first element similarity is greater than a preset similarity threshold, and forms the first element in the first image frame and the first element in the second image frame into a first element group.

[0021] In the above embodiment, the video platform determines element groups by element, preserving the spatiotemporal integrity of the elements in the video being tested. In situations where the video being tested may be affected by editing or filming techniques, using element groups rather than elements as the basic unit of video evaluation can effectively reflect the completeness of the video's visual content.

[0022] In combination with some embodiments of the first aspect, in some embodiments, the electronic device determines the element group in the video to be detected based on the identified elements, including the step of the first element group, and the method further includes: the electronic device calculates the completeness of the element group in the video to be detected, and the completeness is used to indicate whether the element group is complete; the electronic device filters out an incomplete element group from the element group in the video to be detected, including the first element group, and the completeness of the incomplete element group is incomplete.

[0023] In the above embodiment, by judging the completeness of the element groups, the element groups with incomplete completeness are screened for subsequent processing, which reduces the number of element groups in the subsequent processing and can effectively reduce the amount of calculation.

[0024] In combination with some embodiments of the first aspect, in some embodiments, the electronic device determines the element group in the video to be detected based on the identified elements, including the step of the first element group. The method also includes: the electronic device filters out a suspected incomplete element group from the element group in the video to be detected, including the first element group, and the suspected incomplete element group is an element group whose position is less than a preset second distance threshold from the edge of the picture.

[0025] In the above embodiment, considering that in actual situations, elements that affect the integrity of the content of the video picture to be detected often appear at the edge of the picture, the element groups are filtered by judging their positions, which reduces the number of element groups in subsequent processing and can effectively reduce the amount of calculation.

[0026] In combination with some embodiments of the first aspect, in some embodiments, after the electronic device filters out suspected incomplete element groups from the element groups in the video to be detected, the method also includes: the electronic device calculates the completeness of the suspected incomplete element groups in the video to be detected, and the completeness is used to indicate whether the element group is complete; the electronic device filters out incomplete element groups from the suspected incomplete element groups in the video to be detected, including the first element group, and the completeness of the incomplete element group is incomplete.

[0027] In the above embodiment, by judging the completeness of the element group, and considering that the element group with complete completeness does not affect the picture content of the video to be detected, the element group with incomplete completeness is screened out to participate in subsequent calculations, thereby reducing the number of element groups in subsequent processing and effectively reducing the amount of calculation.

[0028] In combination with some embodiments of the first aspect, in some embodiments, the element group attributes of the first element group also include a second incomplete parameter value, which is used to represent at least one of the incomplete type, duration, importance, surrounding element group, and time period type of the first element group.

[0029] In the above embodiment, by determining the second incomplete parameter value in the element group attribute of the element group, more information about the element group is obtained, thereby enabling a more accurate determination of the picture content evaluation result of the detection video.

[0030] In combination with some embodiments of the first aspect, in some embodiments, the classification information of the video to be detected includes: the subject of the video to be detected and / or the label of the video to be detected.

[0031] In the above embodiment, the video platform obtains the classification information of the video to be detected, and distinguishes the videos to be detected with different themes and tags according to the classification information, so as to evaluate the picture content of the video to be detected in a more targeted manner and obtain more accurate evaluation results.

[0032] In combination with some embodiments of the first aspect, in some embodiments, before the step of the electronic device identifying elements in at least two image frames in the video to be detected, the method also includes: the electronic device extracts frames from the original video to obtain the video to be detected; or, the electronic device cuts the original video into N videos to be detected; or, after the electronic device cuts the original video into N intermediate videos, it extracts frames from the N intermediate videos to obtain N videos to be detected; N is a positive integer.

[0033] In the above embodiment, by performing frame extraction operation on the original video, the number of image frames in the video to be detected is reduced, thereby reducing the number of element groups involved in the calculation, thereby reducing the amount of calculation; by cutting the original video, at least one segment of video to be detected is obtained after cutting, thereby reducing the amount of calculation, and at the same time, by utilizing parallel computing, the calculation time is reduced.

[0034] In a second aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code comprising computer instructions, the one or more processors calling the computer instructions to enable the electronic device to execute: identifying elements in at least two image frames in a video to be detected, the at least two image frames comprising a first image frame and a second image frame, the element being a basic element in the image frame; determining an element group in the video to be detected based on the identified elements, comprising a first element group, the first element group comprising the first element in the first image frame and the first element in the second image frame; determining an element group attribute of the first element group, the element group attribute of the first element group comprising a first incomplete parameter value, the first incomplete parameter value being used to represent the degree of incompleteness of the first element group; determining a picture content evaluation result of the video to be detected based at least on the element group attribute of the first element group.

[0035] In the above embodiment, the electronic device automatically obtains the element group in the video to be detected and determines the element group attributes of the element group, and can determine the picture content evaluation result of the video to be detected based on the element group attributes of the element group. The evaluation result of the video to be detected can be directly obtained without human intervention, so that the video platform can automatically review the video to be detected, thereby improving the review efficiency of the video platform. In addition, the electronic device can be a networked server or a computer of a user or video producer, which is convenient for the user or video producer to evaluate the video to be detected anytime and anywhere, thereby improving the user experience.

[0036] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are specifically used to call the computer instructions to enable the electronic device to execute: determining the impact of the first element group on the picture content of the video to be detected based on the element group attributes of the first element group; and determining the picture content evaluation result of the video to be detected based on at least the impact of the first element group on the picture content of the video to be detected.

[0037] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are specifically used to call the computer instructions to enable the electronic device to execute: determining the impact of the first element group on the content of the video picture to be detected based on the element group attributes of the first element group and the preset element group impact on the picture integrity rules.

[0038] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are specifically used to call the computer instructions to enable the electronic device to execute: determining whether the video to be detected is complete based on at least the impact of the first element group on the content of the video to be detected; and / or determining the score of the video to be detected based on at least the impact of the first element group on the content of the video to be detected and the classification information of the video to be detected; and / or determining modification suggestions for the video to be detected based on at least the impact of the first element group on the content of the video to be detected and the element group attributes of the first element group, including modification suggestions for the first element group.

[0039] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are specifically used to call the computer instructions to enable the electronic device to execute: determining whether the video to be detected is complete based on at least the element group attributes of the first element group; and / or determining the score of the video to be detected based on at least the element group attributes of the first element group and the classification information of the video to be detected; and / or determining modification suggestions for the video to be detected based on at least the element group attributes of the first element group, including modification suggestions for the first element group.

[0040] In combination with some embodiments of the second aspect, in some embodiments, the modification suggestion of the first element group includes at least one of: passing, not processing, cropping, smearing, deleting frames, and blurring.

[0041] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are specifically used to call the computer instructions to enable the electronic device to execute: forming an element group from the elements representing the same basic element in the at least two identified image frames.

[0042] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are specifically used to call the computer instructions to enable the electronic device to execute: determining the first element similarity between the first element in the first image frame and the first element in the second image frame, the element similarity is used to represent the degree of similarity between the element images; determining that the first element similarity is greater than a preset similarity threshold, and forming the first element in the first image frame and the first element in the second image frame into a first element group.

[0043] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are also used to call the computer instructions to cause the electronic device to execute: determining that the distance between the first position and the second position is less than a preset first distance threshold; the first position is the position of the first element in the first image frame in the first image frame, and the second position is the position of the first element in the second image frame in the second image frame.

[0044] In combination with some embodiments of the second aspect, in some embodiments, it is characterized in that the one or more processors are also used to call the computer instructions to enable the electronic device to execute: calculating the completeness of the element group in the video to be detected, and the completeness is used to indicate whether the element group is complete; screening out incomplete element groups from the element groups in the video to be detected, including the first element group, and the completeness of the incomplete element group is incomplete.

[0045] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are also used to call the computer instructions to enable the electronic device to execute: screening out suspected incomplete element groups from the element groups in the video to be detected, including the first element group, and the suspected incomplete element group is an element group whose position is less than a preset second distance threshold from the edge of the picture.

[0046] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are also used to call the computer instructions to enable the electronic device to execute: calculating the completeness of the suspected incomplete element group in the video to be detected, and the completeness is used to indicate whether the element group is complete; screening out incomplete element groups from the suspected incomplete element groups in the video to be detected, including the first element group, and the completeness of the incomplete element group is incomplete.

[0047] In combination with some embodiments of the second aspect, in some embodiments, the element group attributes of the first element group also include a second incomplete parameter value, which is used to represent at least one of the incomplete type, duration, importance, surrounding element group, and time period type of the first element group.

[0048] In conjunction with some embodiments of the second aspect, in some embodiments, the classification information of the video to be detected includes: the subject of the video to be detected and / or the label of the video to be detected.

[0049] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are also used to call the computer instructions to enable the electronic device to execute: extracting frames from the original video to obtain the video to be detected; or, cutting the original video into N videos to be detected; or, after cutting the original video into N intermediate videos, extracting frames from the N intermediate videos to obtain N videos to be detected; N is a positive integer.

[0050] In a third aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device, and the chip system includes one or more processors, which are used to call computer instructions to enable the electronic device to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0051] In a fourth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when the computer program product is run on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0052] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions. When the instructions are executed on an electronic device, the electronic device executes the method described in the first aspect and any possible implementation of the first aspect.

[0053] It is understandable that the electronic device provided in the second aspect, the chip system provided in the third aspect, the computer program product provided in the fourth aspect, and the computer storage medium provided in the fifth aspect are all used to execute the methods provided in the embodiments of the present application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 The following shows the scenario of manual video review in the prior art;

[0055] Figures 2 to 6 A set of exemplary user interface diagrams in the embodiments of the present application;

[0056] Figure 7 A flowchart of a method for evaluating video content integrity in an embodiment of the present application;

[0057] Figure 8 This is an exemplary schematic diagram of video cutting in an embodiment of the present application;

[0058] Figure 9 This is an exemplary schematic diagram of the non-adaptive frame extraction method in an embodiment of the present application;

[0059] Figure 10 This is an exemplary schematic diagram of the adaptive frame extraction method in an embodiment of the present application;

[0060] Figure 11 This is an exemplary schematic diagram of cutting and extracting frames of an original video in an embodiment of the present application;

[0061] Figure 12 and Figure 13 This is an exemplary schematic diagram of performing element detection in a single image frame in an embodiment of the present application;

[0062] Figure 14 This is a schematic diagram of an exemplary scenario for obtaining an element group in an embodiment of the present application;

[0063] Figure 15 This is another exemplary scenario diagram for obtaining an element group in an embodiment of the present application;

[0064] Figure 16 This is an exemplary schematic diagram of the integrity determination result of an element group in an embodiment of the present application;

[0065] Figure 17 This is a schematic diagram of an exemplary structure of the rule that element groups affect picture integrity in an embodiment of the present application;

[0066] Figure 18 This is an exemplary schematic diagram of the effect of element groups on picture integrity in an embodiment of the present application;

[0067] Figure 19 This is an exemplary schematic diagram of the BP neural network model in the embodiment of the present application;

[0068] Figure 20 A schematic diagram of the architecture of a modification suggestion generation model in an embodiment of the present application;

[0069] Figure 21 A schematic diagram of modification suggestions for an original video image output by a modification suggestion generation model in an embodiment of the present application;

[0070] Figure 22 A schematic structural diagram of an electronic device 100 provided in an embodiment of the present application;

[0071] Figure 23 is a schematic block diagram of a software structure of the electronic device 100 according to an embodiment of the present invention;

[0072] Figure 24 This is another hardware structure diagram of the electronic device 200 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The terms used in the following examples of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of this application, the singular expressions "a," "an," "the," "above," "the," and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0074] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0075] For ease of understanding, the following first introduces the relevant terms and concepts involved in the embodiments of the present application:

[0076] (1) Elements and element types:

[0077] In the embodiments of the present application, an element is a basic element in an image frame that can carry or transmit visual information. An image frame is the smallest unit that constitutes a video.

[0078] In some embodiments of the present application, elements may include a subject, accompanying objects, text, etc. in an image frame. The subject is the primary object that a video producer or user focuses on during filming. Generally, the subject is a single object or a group of objects, such as a person, an animal, a plant, or even an abstract object. Accompanying objects are secondary objects that a video producer or user focuses on during filming. Accompanying objects are objects used as backgrounds. Generally, accompanying objects are multiple independent objects, such as daily necessities, buildings, artworks, scenic spots, and historical sites.

[0079] In the embodiment of the present application, the element type is used to distinguish the element classifications that convey different types of visual information.

[0080] In some embodiments of the present application, element types may include people (skin color recognition such as Caucasians, Asians, and blacks), faces (recognition of men, women, old, young, etc.), text, other post-production text, subtitles, copyright logos, station logos, watermarks, plant subjects (recognition of plant varieties such as dandelions, chrysanthemums, etc.), animal subjects (recognition of specific animal varieties such as dogs, cats, camels, etc.), landscape subjects, etc.

[0081] For example, if the element type of element A is text, it means that element A is the filmed element; if element B is a subtitle, it means that element B is a text subtitle added later after the video is shot, and its position is mostly at the bottom and centered of the video; if element C is other post-production text, it means that element C is text added later after the video is shot, but its position is different from that of the subtitle.

[0082] Specifically, in the embodiment of the present application, by performing data processing on the image frame, or by performing image processing on the image presented by the image frame, such as element detection, the elements contained in the image frame can be obtained. For example, Figures 12 to 13 As shown, image processing (element detection) is performed on the image frame to obtain elements contained in the image frame. The detected elements include: 1301 Text 1, 1302 Icon 1, 1303 Animal 1, 1304 Food 1, and 1305 Text 2. The element type of each detected element can also be identified. The identification results include: 1301 Text 1 belongs to other post-production text, 1302 Icon belongs to a copyright logo, 1303 Animal 1 belongs to an animal (dog), 1304 Food 1 belongs to food, and 1305 Text 2 belongs to a subtitle.

[0083] When the elements contained in the image frame are obtained, the element attributes can be obtained. The element attributes are used to quantify the relationship between elements and the relationship between elements and the video image. The element attributes can include many parameters, such as the position of the element and the degree of damage of the element.

[0084] (2) Element group:

[0085] 2.1. Definition of element group:

[0086] In the embodiment of the present application, the same or identical elements in multiple image frames of a video can be grouped into an element group. It is understandable that the type of the element group is the same as the element type of the elements in the element group.

[0087] When video producers or users shoot or post-process videos, they are often affected by shooting or editing techniques such as montage, camera movements, and transitions. Therefore, analyzing only the integrity of elements cannot effectively reflect the completeness of the video's visual content. Element groups are defined to take into account the spatiotemporal integrity of elements in a video.

[0088] Among them, the element group includes: first, a number of image frames with a time sequence relationship, where the image frames can be continuous or discontinuous; second, the same element D exists in at least two of the above-mentioned image frames, where the element D can be complete or incomplete; finally, the D-Group element group is a collection of D elements in at least two image frames.

[0089] 2.2. Element group attributes:

[0090] The element group attributes of an element group contain the correspondence between parameters used to represent the characteristics of the element group and the parameter values ​​of the corresponding parameters. For example, the element group attributes of an element group may include many parameters: the area occupied by the element group on the screen, the position of the element group, the percentage of the element group missing, the type of the element group missing, the duration of the element group, the importance of the element group, the surrounding element groups of the element group, the time period type of the element group, etc.

[0091] For example, as shown in Table 1 below, it is a schematic example of element group attributes in an embodiment of the present application:

[0092]

[0093] Table 1

[0094] As shown in Table 1, the element group attributes of the first element group include a defect ratio parameter value of 1 / 2 and a duration parameter value of 10 seconds.

[0095] It is understandable that the element group attributes of the element group can also be expressed in many other ways, such as arrays, matrices, etc., which are not limited here.

[0096] The element group attributes of an element group are related to the element attributes of the elements in the element group. The element group attributes of an element group are used to quantify the relationship between the element groups, the relationship between the element groups and the video screen, and the like.

[0097] In some embodiments of the present application, the parameters included in the element group attributes may be the same as the parameters in the preset element group impacting the integrity of the picture. The electronic device may determine whether the element group has an impact on the integrity of the video picture content by comparing the parameter value of the parameter in the element group attributes of the element group with the threshold value or reference value of the parameter in the preset element group impacting the integrity of the picture. For details, please refer to the description of the preset element group impacting the integrity of the picture below (4), which will not be repeated here.

[0098] 2.3. Determination of element groups:

[0099] The element group type of an element group is the same as the element type of the elements in the element group and has the same meaning.

[0100] In the embodiment of the present application, there are many ways to obtain the same element D from at least two image frames and form the same elements D in different image frames into a D-Group element group, such as obtaining element groups by using element similarity, obtaining element groups by using a clustering algorithm, etc. The following uses the example of obtaining element groups by using element similarity to introduce the method of obtaining element groups:

[0101] Using element similarity to obtain element groups: Image features of all elements in all image frames are extracted using image or data processing algorithms. An element is randomly selected as a reference element, and the similarity between the image features of this reference element and the image features of the element being compared is calculated. When the similarity exceeds a preset threshold, the reference element and the element being compared are considered identical. Element similarity is the distance between the image features of the reference element and the element being compared. This distance can be expressed in various ways, including cosine distance, Euclidean distance, and Manhattan distance.

[0102] In the embodiments of the present application, there are many ways to select the elements to be compared. For example, the method for selecting the elements to be compared can be: traversing and selecting elements in image frames different from the image frame in which the reference element is located as the elements to be compared; or traversing and selecting elements in image frames different from the image frame in which the reference element is located and of the same element type as the reference element as the elements to be compared; or traversing and selecting elements in image frames different from the image frame in which the reference element is located and of the same element type as the reference element and of a position close to the reference element as the elements to be compared.

[0103] Determining whether the reference element and the compared element are located in close proximity: A first distance threshold is pre-set. Based on the element detection results, it can be determined that the reference element is located at the first element position and the compared element is located at the second element position. The distance between the first and second elements is calculated to determine whether it is less than the preset first distance threshold. If the distance is less than the preset first distance threshold, the reference element and the compared element are considered to be located in close proximity. The value of the first distance threshold may be related to information such as the frame rate of the video.

[0104] For example, Figure 14As shown, there are three image frames, including the first, second, and third frames. Element 1401A in the first frame is selected as the reference element, and all elements in the second and third frames are selected as compared elements. The similarity between the reference element and each compared element is calculated. When the element similarity is greater than a preset element similarity threshold of 0.85, reference element 1401A and the compared element are considered to be the same element, that is, reference element 1401A and the compared element belong to the same element group. Element 1401B in the second frame has an element similarity of 0.95 with reference element 1401A, and element 1401C in the third frame has an element similarity of 0.94 with reference element 1401A. Since the element similarity of elements 1401B and 1401C with reference element 1401A is greater than 0.85, elements 1401A, 1401B, and 1401C form an element group, which can be denoted as 1401-Group.

[0105] (3) Completeness of element group:

[0106] In the embodiment of the present application, the integrity of the element group is used to indicate whether the element group is complete. The integrity of the element group can be a binary value, including complete and incomplete.

[0107] In the implementation of this application, the completeness of the element group can be determined in a variety of ways, such as using a proportional relationship, using projection features, etc., which are not limited here.

[0108] For example, as an embodiment of the present application, an optional method for determining the completeness of an element group by using a proportional relationship in combination with projection features is described below:

[0109] Element groups of different element group types may correspond to different preset standard ratio intervals and preset suspected standard ratio intervals. The preset suspected standard ratio interval includes the preset standard ratio interval. For example, the element group D-Group has an element group type of Type-D, a corresponding preset standard ratio interval of [0.8 to 1.2], and a corresponding preset suspected standard ratio interval of [0.3 to 3].

[0110] When the aspect ratio or height-to-width ratio of an element group exceeds the preset suspected standard ratio range, the integrity of the element group can be determined to be incomplete. When the aspect ratio or height-to-width ratio of an element group is within the preset standard ratio range, the integrity of the element group can be determined to be complete.

[0111] When the aspect ratio or the height-to-width ratio of a group of elements exceeds the preset standard ratio range but is within the preset suspected standard ratio range, the projection features of each element in the group of elements can be calculated, and the projection features can be statistically analyzed. Then, the integrity of the group of elements can be determined according to the statistical results. For example, if the aspect ratio of the element group D-Group is 1.5, which exceeds the preset standard ratio range [0.8 to 1.2], but is within the preset suspected standard ratio range [0.3 to 3], the projection features of each element in the element group D-Group can be calculated, and the projection feature distribution curve of the element group D-Group can be calculated. By calculating the distance between the projection feature distribution curve of D-Group and the preset threshold curve, when the distance is greater than the threshold distance, it is considered that the element group D-Group is incomplete. Among them, the distance has various forms, including Euclidean distance, Manhattan distance, etc.

[0112] The feature distribution curve can be characterized in the form of histogram statistics. The preset threshold curve corresponds to a complete "J" shape. That is, when the histogram of the projection features of the element group D-Group is a complete "J" shape, the integrity of the element group D-Group can be determined to be complete; if the gap of the "J" shape in the histogram of the projection features of the element group D-Group exceeds the preset statistical threshold, the integrity of the element group D-Group can be determined to be incomplete.

[0113] In some embodiments of the present application, in the process of determining the integrity of a group of elements, relevant parameters (such as the残缺比例, the残缺类型) of the degree of mutilation of the group of elements can be directly obtained. In some embodiments of the present application, the degree of mutilation of the group of elements can also be determined separately, which is not limited here.

[0114] In some embodiments of the present application, whether to continue to judge whether a group of elements affects the integrity of the picture can be determined according to the integrity of the group of elements. If the integrity of a group of elements is complete, it can be directly considered that the group of elements does not affect the integrity of the picture; if the integrity of a group of elements is incomplete, it can be continued to judge whether the group of elements affects the integrity of the picture, that is, to determine the influence result of the group of elements on the video picture content. The method of judging whether the group of elements affects the integrity of the picture can specifically refer to the description in the following (4) preset rules for the group of elements to affect the integrity of the picture, which will not be elaborated here.

[0115] It should be noted that there are some Chinese characters in the original text that seem to be incorrect or incomplete. I have translated them as they are. You may need to check and correct them in the original text for a more accurate understanding.In some embodiments of the present application, the element groups located at the edge of the picture can be first screened as suspected incomplete element groups based on the position of the element groups in the picture presented by the image frame, and then the completeness of these suspected incomplete element groups can be determined. In some embodiments of the present application, the completeness of the element groups located at the edge of the picture can be directly determined as incomplete based on the position of the element groups in the picture presented by the image frame. In some embodiments of the present application, the completeness of all element groups can also be directly determined. This is not limited here.

[0116] (4) Rules for the influence of preset element groups on picture integrity:

[0117] In the embodiment of the present application, the rule of element group affecting picture integrity is pre-set and is used to record the thresholds and / or reference values ​​of parameters of element groups of each element group type that affect the integrity of video picture content.

[0118] For example, Figure 16 The figure shows a schematic diagram of a structure of a rule affecting picture integrity in an embodiment of the present application. The rule affecting picture integrity by an element group can include a variety of different element group types. The element group type is the same as the element type of the element. For example, the element group types may include: people, faces, text, other post-production text, subtitles, copyright marks, station logos, watermarks, plant-based subjects, animal-based subjects, landscape-based subjects, etc. The element groups of each element group type may correspond to multiple parameters, for example, the parameters may include: incompleteness ratio, incompleteness type, duration, time period type, importance, surrounding element groups, etc.

[0119] Among them, the incompleteness ratio indicates the incompleteness value of the element group, for example, the value range can be 0-1, and the larger the value, the greater the degree of incompleteness; the incompleteness type indicates the incomplete direction of the element group, and the value can be horizontal or vertical, etc.; the duration indicates the shortest time of incompleteness of the elements in the element group; the value of the time period type is any time period within the video length, and the common value is several seconds after the start of the video and several seconds before the end of the video; the importance is used to indicate the importance of the element group in the video; the surrounding element group is the surrounding element group of the element group.

[0120] In the element group impact screen integrity rule, the different parameters corresponding to the element group type of each element group may include the threshold value and / or reference value of the parameter of the element group of the element group type. For example, Table 2 below is an example of the preset element group impact screen integrity rule in an embodiment of the present application:

[0121]

[0122] Table 2

[0123] For example, in the preset element group affecting the integrity of the picture shown in Table 2, the element group type of other post-text corresponds to five parameters: incompleteness ratio, incompleteness type, duration, time period type, importance, and surrounding element group. Among them, the threshold value of the corresponding incompleteness ratio parameter is greater than or equal to 1 / 3, the threshold value of the corresponding duration parameter is greater than or equal to 15 seconds, the reference value of the corresponding incompleteness type parameter is horizontal, the reference value of the corresponding time period type parameter is arbitrary, the reference value of the corresponding importance parameter is greater than 3, and the reference value of the corresponding surrounding element group parameter is none. Among them, the threshold value of the parameter indicates that if the value of the parameter of the element group of the element group type exceeds the threshold value, the element group may be judged to affect the integrity of the picture content, and the reference value of the parameter can be used to limit the conditions for the judgment.

[0124] In an embodiment of the present application, the electronic device can determine whether the element group has an impact on the integrity of the video picture content based on the acquired parameter values ​​of the element group and the preset element group impact on picture integrity rules, that is, determine the impact result of the element group on the video picture.

[0125] It is understood that the impact of an element group on the integrity of video content can be expressed in a variety of ways. For example, the result can be a binary value, including: impact or no impact; the result can be a multivariate value, including: no impact, some impact, significant impact, impact. For ease of explanation, the impact of an element group on video content integrity is described as a binary value, including: the element group affects the integrity of the content, and the element group does not affect the integrity of the content.

[0126] It is understandable that, depending on different actual requirements, the parameters in the preset element group affecting the picture integrity rule may be more or less, which is not limited here.

[0127] For example, the following Table 3 is another example of the rule of the preset element group affecting the picture integrity in the embodiment of the present application.

[0128]

[0129] Table 3

[0130] The preset rules for element groups affecting picture integrity shown in Table 3 only include thresholds for the incompleteness ratio parameter. The threshold for the incompleteness ratio parameter corresponding to element groups whose element group type is "Other Post-production Text" is greater than or equal to 1 / 3. This means that for an element group whose element group type is "Other Post-production Text" in a video, if its incompleteness ratio exceeds 1 / 3, it can be determined that this element group affects the picture content integrity. If its incompleteness ratio does not exceed 1 / 3, it can be determined that this element group does not affect the picture content integrity.

[0131] For example, the following Table 4 is another example of the rules of the preset element groups affecting the picture integrity in the embodiment of the present application.

[0132]

[0133] Table 4

[0134] The preset rules for the element groups affecting the integrity of the picture shown in Table 4 include thresholds for the incompleteness ratio parameter and the duration parameter. Among them, the threshold for the incompleteness ratio parameter corresponding to the element group whose element group type is other post-production text is greater than or equal to 1 / 3, and the threshold for the duration parameter is greater than or equal to 15 seconds. For an element group whose element group type is other post-production text in a video, if its incompleteness ratio exceeds 1 / 3 and its duration is greater than or equal to 15 seconds, it can be determined that the element group will affect the integrity of the picture content. Similarly, if its incompleteness ratio does not exceed 1 / 3, no matter how long it lasts, it can be determined that the element group will not affect the integrity of the video picture content; if its duration is less than 15 seconds, no matter how long it lasts, it can be determined that the element group will not affect the integrity of the video picture content.

[0135] For example, the following Table 5 is another example of the rules of the preset element groups affecting the picture integrity in the embodiment of the present application.

[0136]

[0137] Table 5

[0138] The preset rules for the element group affecting the integrity of the picture shown in Table 5 include thresholds of two parameters and a reference value of one parameter, wherein the thresholds of the parameters are the threshold of the defect ratio parameter, the threshold of the duration parameter, and the reference value of the defect type parameter. Among them, the threshold of the defect ratio parameter corresponding to the element group whose element group type is other post-production text is greater than or equal to 1 / 3, the threshold of the duration parameter is greater than or equal to 15 seconds, and the defect type is horizontal. For an element group whose type is other post-production text in a video, if its defect type is horizontal, and the defect ratio exceeds 1 / 3 and the duration is greater than or equal to 15 seconds, it can be determined that the element group will affect the integrity of the picture content; if its defect type is not horizontal, it can be determined that the element group will not affect the integrity of the picture content.

[0139] For example, the following Table 6 is another example of the rule of the preset element group affecting the picture integrity in the embodiment of the present application.

[0140] The preset rules for element groups affecting picture integrity shown in Table 6 include two sets of rules for element groups of the same element type. Each set of rules includes two parameter thresholds and a parameter reference value of the same type, denoted as Rule 1 and Rule 2, respectively. The parameter thresholds are the thresholds for the defect ratio and duration, and the parameter reference value is the reference value for the defect type.

[0141] When the defect type parameter of an element group is horizontal, whether the element affects the integrity of the image needs to refer to the parameter thresholds in Rule 1: the threshold of the defect ratio parameter is greater than or equal to 1 / 3, and the threshold of the duration parameter is greater than or equal to 15 seconds. When the defect type parameter is vertical, whether the element affects the integrity of the image needs to refer to the parameter thresholds in Rule 2: the threshold of the defect ratio parameter is greater than or equal to 1 / 5, and the threshold of the duration parameter is greater than or equal to 10 seconds.

[0142]

[0143] Table 6

[0144] For element groups whose element group type is other late text, when its incomplete type is horizontal, its incomplete ratio is greater than or equal to 1 / 3, and its duration is greater than or equal to 15 seconds, it can be determined that this element group will affect the integrity of the picture content; when its incomplete type is horizontal, its incomplete ratio is less than 1 / 3 or its duration is less than 15 seconds, it can be determined that this element group will not affect the integrity of the picture content.

[0145] For element groups whose elements are similar to other later texts, when their incomplete type is vertical, their incomplete ratio is greater than or equal to 1 / 3, and their duration is greater than or equal to 15 seconds, it can be determined that this element group will affect the integrity of the picture content; when their incomplete type is vertical, their incomplete ratio is less than 1 / 3 or their duration is less than 15 seconds, it can be determined that this element group will not affect the integrity of the picture content.

[0146] For example, the preset rules for element groups affecting screen integrity shown in Table 2 above include reference values ​​for two parameters and thresholds for multiple parameters. The parameter thresholds are the threshold for the incompleteness ratio and the threshold for the duration. The reference values ​​are the reference values ​​for the incompleteness type, the reference value for the time period type, the reference value for the importance, and the reference value for the surrounding element groups. Specifically, if the incompleteness type of an element group with element type other post-text is horizontal, the importance is greater than 3, the incompleteness ratio exceeds 1 / 3, and the duration is greater than or equal to 15 seconds, then the element group can be determined to affect screen integrity; if its importance is less than or equal to 3, then the element group can be determined to not affect screen integrity; if its incompleteness type is vertical, then the element group can be determined to not affect screen integrity.

[0147] It is understandable that the preset rule of element group affecting picture integrity may also include thresholds or reference values ​​of other parameters related to the element group, which is not limited here.

[0148] The rule of element groups affecting picture integrity involved in this application may be expressed as a function, a mapping relationship, etc. in other embodiments.

[0149] It is worth noting that the thresholds and / or reference values ​​of the influential parameters in the rule of the element group affecting the picture integrity involved in the present application may be derived from the element group attributes of the element group.

[0150] (5) Completeness judgment model:

[0151] In an embodiment of the present application, the completeness judgment model is used to determine whether the picture content of the video is complete.

[0152] The input of the integrity judgment model may include: the effect of the element group on the video image content. The output of the integrity judgment model includes: the video image content is complete, and the video image content is incomplete.

[0153] The completeness assessment model is built using machine learning or deep learning methods in the field of artificial intelligence. This model is based on training data, which is labeled as either complete or incomplete video content.

[0154] It is understood that the input of the integrity judgment model may also include a variety of other data. For example, in some embodiments of the present application, the input of the integrity judgment model may also include element groups; in some embodiments of the present application, the input of the integrity judgment model may also include video classification information; in some embodiments of the present application, the input of the integrity judgment model may also include element group attributes of element groups. This is not limited here.

[0155] There are many ways to obtain video classification information. For example, you can obtain video classification information based on the video's subject; based on the column or section selected by the user or video producer when uploading the video; based on the video's name; and based on the user's or video producer's historical upload history and user profile. This is not limited here. Video classification information can include animals, digital products, people, daily life, food, and so on.

[0156] (6) Scoring model:

[0157] In an embodiment of the present application, the scoring model is used to determine the score of the completeness of the video's picture content.

[0158] The input of the scoring model includes the effect of the element group on the video content. The output of the scoring model is the score of the video content completeness.

[0159] It is understood that the input of the scoring model may also include a variety of other data. For example, in some embodiments of the present application, the input of the integrity judgment model may also include element groups; in some embodiments of the present application, the input of the integrity judgment model may also include video classification information; and in some embodiments of the present application, the input of the integrity judgment model may also include element group attributes of element groups. This is not limited here.

[0160] It is understandable that when scoring the completeness of a video's content, the video's classification information can be taken into account. For example, if the video's classification information is digital, elements such as digital products, subtitles, and post-production text will have a greater impact on the video's content completeness score. Whereas, if the video's classification information is people, elements such as people will have a greater impact on the video's content completeness score.

[0161] The scoring model can be a model constructed using the BP neural network model in the field of artificial intelligence. The model is completed based on training data. The training data is the first incomplete parameter group of the element group extracted from the video, the element group attributes, etc., and the label of the training data is the score of the completeness of the video picture content. The schematic diagram of the BP neural network model is as follows Figure 18 shown.

[0162] In the embodiment of the present application, the scoring model can be a model constructed using other machine learning methods or other deep learning methods in the field of artificial intelligence.

[0163] (7) Modify the suggestion generation model:

[0164] In an embodiment of the present application, a modification suggestion generation model is used to determine modification suggestions for the video's screen content.

[0165] The inputs to the modification suggestion generation model include whether the element group affects the picture integrity and the element group attributes. The output of the scoring model is a modification suggestion for the picture integrity of the video content. The modification suggestion includes a modification suggestion for the element group. The modification suggestion can be presented in the form of an audit report, which includes the modification suggestion for the element group and the element group attributes.

[0166] It is understood that the input to the modification suggestion generation model may also include a variety of other data. For example, in some embodiments of the present application, the input to the integrity judgment model may also include element groups; in some embodiments of the present application, the input to the integrity judgment model may also include video classification information. This is not limited here.

[0167] Suggested modifications for element groups include: No Suggestion (Suggest), Pass (Pass), Cut (Cut), Cover (Cover), Delete (Delete), Blur (Blur), etc. Pass (Pass) means the element group will not affect the integrity of the video content; Cut (Cut) means the video edge can be cropped based on the height or width of the element group boundary box; Cover (Cover) means the element group can be covered by other icons; Delete (Delete) means the video content integrity can be improved by deleting the image frame; Blur (Blur) means the element group can be blurred by mosaic or Gaussian blur to improve the integrity of the video content.

[0168] The modification suggestion generation model can be a model constructed using the decision tree algorithm in the field of artificial intelligence. The schematic diagram of the decision tree algorithm is as follows: Figure 18 As shown. Figure 18 As shown, the node attributes of the decision tree can be derived from the element group attributes, the node attributes of the decision tree can also be derived from the reference value and / or threshold of the element group's influence on the picture integrity rule. The node attributes of the decision tree can also be derived from the classification information of the video.

[0169] In an embodiment of the present application, the modification suggestion generation model can be a model constructed using other machine learning methods or other deep learning methods in the field of artificial intelligence.

[0170] In some embodiments of the present application, the inputs of the above-mentioned (5) completeness judgment model, (6) scoring model, and (7) modification suggestion generation model may include the element group attributes of the element group, but not the impact of the element group on the video screen content. The electronic device can directly determine the evaluation result of the video screen content based on the element group attributes of the element group: for example, whether it is complete or the score, without first determining the impact of the element group on the video screen content. This is not limited here.

[0171] The following first introduces the scenario of manual video review in the existing technology.

[0172] Figure 1 The following illustrates a scenario of manually reviewing a video in the prior art.

[0173] like Figure 1As shown in the figure, the current method of video content integrity review provided by the industry is mainly manual review. When reviewing videos, video reviewers often use methods such as browsing videos at double speed and skipping frames, which can easily miss or ignore video image frames that do not meet video review standards. Secondly, when reviewing videos, video reviewers need to manually browse the content of the video, which is limited by working hours and video length. As a result, the efficiency of manual review is far from meeting the needs of video review. Thirdly, different video reviewers have different specific evaluation criteria when reviewing videos, and cannot provide targeted modification guidance, which reduces the effectiveness of video review and is not conducive to maintaining the ecosystem of original video content.

[0174] In response to the above-mentioned problems existing in the current video review field, this application provides a method and electronic device for evaluating the completeness of video screen content. The method and electronic device for evaluating the completeness of video screen content provided by this application can automatically review videos uploaded by video producers, provide an evaluation result on whether the video screen content is complete, and improve the efficiency of video review; further, the method and electronic device for evaluating the completeness of video screen content provided by this application can provide a score for the completeness of the video screen; further, the method and electronic device for evaluating the completeness of video screen content provided by this application can provide guidance on targeted modifications to the video.

[0175] It can be understood that the electronic device in the embodiment of the present application can implement the video picture content integrity evaluation method in the embodiment of the present application by running the video picture content integrity evaluation system.

[0176] Figures 2 to 6 A set of exemplary user interface diagrams in an embodiment of the present application.

[0177] like Figure 2 As shown, users or video producers can upload the original video to be reviewed to the video content integrity assessment system. Optionally, when uploading the original video, users or video producers can also upload the original video's classification information to the video content integrity assessment system. The classification information may include: the original video's theme, tags, category, etc.

[0178] like Figure 3 and Figure 4 As shown, after the user or video producer uploads the original video to be reviewed to the video screen content integrity evaluation system in the electronic device, the electronic device will display the review result for the original video. The review result includes: the video screen content is complete or the video screen content is incomplete. Among them, the electronic device in the embodiment of the present application evaluates the video screen content integrity of the original video as shown in FIG. Figure 7 shown.

[0179] like Figure 5As shown, after reviewing the original video, the electronic device can optionally assign a completeness score based on the completeness of the original video's image content. The completeness score is used to evaluate the completeness of the original video's image content. Video producers or users can use this completeness score to gain a more concrete understanding of the completeness of the original video, helping them improve the quality of their original videos.

[0180] like Figure 6 As shown, optionally, after reviewing the original video, the system will give an audit report for the element group that affects the completeness of the original video's picture content. The audit report includes: the element group that affects the original video's picture content, the position of the element group in the original video, the duration and first appearance time of the element group in the original video, the importance of the element group in the original video, and the recommended modification suggestions for the element group. Among them, the modification suggestions include: passing, cropping, covering, smearing, blurring, deleting frames, etc. The system reviews the original video and gives corresponding modification suggestions for the completeness of the original video's picture content. The modification suggestions are presented in the form of an audit report, which can effectively help video producers or users modify the original video to improve the completeness of the original video's picture content.

[0181] The following describes the video content integrity assessment method and electronic device provided by this application.

[0182] Figure 7 This is a flow chart of a method for evaluating the completeness of video content in an embodiment of the present application.

[0183] It is worth noting that the video to be detected can be the original video, the first video or the second video in this application.

[0184] S701 and S7021 process the video to obtain image frames. S7022 and S7023 obtain element groups. S7024 and S7025 filter out incomplete element groups. S7026 calculates the element group attributes of the element groups. S7027 calculates whether the element groups affect the integrity of the image. S703 generates the results of the video content integrity assessment.

[0185] S701: Cut the original video to obtain N first video segments.

[0186] After receiving a video from a video generator or user that is awaiting completeness review, the electronic device uses the video as the original video and processes the original video, wherein the processing includes: cutting. The cutting operation specifically includes: dividing the original video into N segments based on information such as the time length of the original video and the video transition points, where N is a positive integer greater than or equal to 1. When N is equal to 1, the original video is not cut at this time, and the first video is the original video. When N is greater than 1, the original video is cut at this time, and N segments of the first video are obtained after cutting.

[0187] It can be understood that, on the one hand, cutting the original video into N first videos and performing subsequent integrity assessment processing on the N first videos can fully utilize the parallel computing capabilities of electronic devices and improve computing efficiency; on the other hand, when cutting the original video according to information such as the video transition points, the influence of montage techniques, camera movements, transitions and other shooting or editing techniques on the spatiotemporal integrity of the element group (element) is fully considered, which effectively improves the accuracy of the electronic device's assessment of the completeness of the video content.

[0188] In some embodiments, when an electronic device receives a video from a video producer or user that is waiting for completeness review, it also receives video classification information uploaded by the video producer or user, where the classification information includes the theme name of the video, the tags of the video, and other information.

[0189] In some embodiments, when an electronic device receives a video from a video generator or user that is uploaded and awaiting completeness review, it can obtain classification information of the video through an image processing algorithm.

[0190] Figure 8 This is an exemplary schematic diagram of video cutting in an embodiment of the present application.

[0191] like Figure 8 As shown, the original video is cut into three videos, namely the first video 1, the first video 2, and the first video 3.

[0192] In some embodiments, when there is no transition point information in the original video, the original video can be cut according to the duration of the video, or the video can be not cut (when N is equal to 1). For example, if the original video is 10 minutes and 30 seconds long and there is no transition point information in the original video, the original video can be cut into three first videos of equal duration, where each first video is 210 seconds long.

[0193] In some embodiments, the duration or duration range of the first video after the original video is cut can be pre-defined. For example, if the duration of the first video is set to 3 minutes, and the original video is 10 minutes and 30 seconds long, the original video can be cut into four segments, with the first segment of the first video being 0 to 3 minutes, the second segment of the first video being 3 to 6 minutes, the third segment of the first video being 6 to 9 minutes, and the fourth segment of the first video being 9 minutes to 10 minutes and 30 seconds.

[0194] In some embodiments, overlapping portions may be used between the multiple first video segments after cutting the original video.

[0195] In some embodiments, when the original video contains transition point information, the video can be cut based on the transition point information. Further, after the video is initially cut based on the transition point information, at least one intermediate video is obtained, and the intermediate video can be cut again based on the video length to obtain the first video.

[0196] After obtaining the N first videos, the electronic device may perform step S702 on all the N first videos or some of the N first videos.

[0197] S702: includes steps S7021, S7022, S7023, S7024, S7025, S7026, and S7027, which are performed sequentially. Steps S7021 and S7024 are optional steps that do not affect the integrity of the solution.

[0198] S7021: extract frames from the first video to obtain a second video.

[0199] After obtaining the first video, the electronic device extracts frames from the first video, and the video data after the frame extraction is recorded as the second video.

[0200] The frame extraction operation specifically includes: non-adaptive frame extraction and adaptive frame extraction, and the object of frame extraction is the first video; non-adaptive frame extraction specifically includes: extracting an image frame from the object of frame extraction at a fixed time interval or frame interval; adaptive frame extraction specifically includes: dynamically adjusting the time interval or frame interval between the next extracted image frame and the current image frame based on the data difference (picture difference) between the current image frame and the next extracted image frame.

[0201] It can be understood that by extracting frames from the first video, the number of image frames in the second video that needs to be processed in subsequent steps is reduced, and the computational complexity of the electronic device is reduced.

[0202] Figure 9 This is an exemplary schematic diagram of the non-adaptive frame extraction method in an embodiment of the present application.

[0203] like Figure 9As shown, if the L1th frame of the first video is used as the first frame to be extracted and the frame interval is set to L2, then the frame to be extracted is the L1+(I-1)*(L2+1)th frame of the first video. For example, if L1=1 and L2=2, the number of frames to be extracted is 1, 4, 7, etc. By setting the frame interval according to the original video frame rate, non-adaptive frame extraction can be achieved.

[0204] In some embodiments, the frame extraction frequency may be 1 frame per second.

[0205] Figure 10 This is an exemplary schematic diagram of the adaptive frame extraction method in an embodiment of the present application.

[0206] like Figure 10 As shown, the adaptive frame extraction includes: adaptively adjusting the frame extraction interval according to the data gap (picture gap) between the image frames of the first video. For example, the first frame of the first video is selected as the first frame to be extracted, and the frame is used as the reference frame to calculate the data gap between the subsequent frames and the reference frame. When comparing the second to fourth frames, the data gap does not exceed the threshold; when comparing the fifth frame, the data gap between the image frame of the fifth frame and the image frame of the first frame is greater than the preset threshold, and the image frame of the fifth frame is selected as the second frame to be extracted. At this time, the fifth frame is selected as the reference frame to calculate the data gap between the subsequent frames and the reference frame, and then the seventh frame is selected as the third frame to be extracted. Therefore, the electronic device selects the first, fifth, and seventh frames as the second video through adaptive frame extraction.

[0207] The data gap in the adaptive frame extraction method is the Euclidean distance, absolute value distance, Chebyshev distance, etc. between the image frames of two frames; the data gap in the adaptive frame extraction method is the gap between the picture content (image content) presented by the image frames of two frames, wherein the gap can be obtained by a quasi-image matching algorithm.

[0208] Figure 11 This is an exemplary schematic diagram of the original video being cut and framed in an embodiment of the present application.

[0209] like Figure 11 As shown, after the original video is cut and frame extracted, the second video 1, the second video 2, and the second video 3 are obtained, which are used for subsequent processing respectively.

[0210] In some embodiments, the original video may be directly used as the second video for subsequent processing without being cut and / or frame extracted.

[0211] In some embodiments, the original video may be subsequently processed as the second video after only the cutting operation.

[0212] In some embodiments, the original video may be processed as the second video only after undergoing a frame extraction operation.

[0213] It is worth noting that step S7021 may not be executed, in which case the second video is the first video.

[0214] Wherein S7022: detecting elements in each image frame in the second video.

[0215] Each image frame in the second video data is traversed, and when any image frame is traversed, element detection is performed on the image frame. After element detection, the elements contained in the image frame, the element types of the elements, and the element attributes of the elements are obtained. The element attributes of the elements include: the position of the element, etc.

[0216] The definitions of elements and element types can be found in the above term explanation (1) Elements and element types, which will not be repeated here.

[0217] Element detection includes text detection, face detection, subject detection, icon detection, watermark detection, and QR code detection. Subject detection includes vegetable, fruit, and animal recognition, plant recognition, and vehicle recognition. In some embodiments, the position of an element on the screen can be represented by the coordinates of a rectangular selection box, where the range of the element can be based on the range of the rectangular selection box.

[0218] Understandably, judging the integrity of original video content based on the integrity of elements within a single image frame has certain limitations. Elements within the original video have spatiotemporal integrity properties. When video producers or users create original videos through filming and editing techniques such as montage, camera movements, and animation transitions, judging the integrity of the original video content based on the integrity of groups of elements within a single image frame is less effective.

[0219] Figure 12 and Figure 13 This is an exemplary schematic diagram of performing element detection in a single image frame in an embodiment of the present application.

[0220] For example, if there are more than one image frame in the second video, all the image frames are traversed and element detection is performed on each frame to obtain the elements and element attributes that appear in the image content corresponding to the image frame. Figure 12 As shown in FIG, an image frame is selected for element detection, including text detection, QR code detection, subject detection, etc. Figure 12The image frame shown is used for element detection. Using text detection, the detected elements include: 1301 text 1 - "Original Video 1", 1305 text 2 - "Dog is a good friend of man"; using QR code detection, no elements are detected; using subject detection, the detected elements include: 1303 animal 1 - dog, 1304 food 1 - dog food; using icon detection, the detected elements include: 1302 icon - "HUAWE".

[0221] Further, the element type of the element is obtained. 1301 Text 1 - "Original Video 1" belongs to other post-production text, 1302 Icon - "Huawei HUAWE" belongs to the copyright logo, 1303 Animal 1 belongs to animals, 1304 Food 1 belongs to food, and 1305 Text 2 - "Dogs are good friends of humans" belongs to subtitles.

[0222] After performing text detection, QR code detection, subject detection, and icon detection on the image frame, the detection results of the image frame are as follows: Figure 13 In this image frame, all elements can be selected with a rectangular selection box, and the attributes of the elements and the element types to which the elements belong can be obtained.

[0223] S7023: Select at least two image frames in the second video to obtain an element group.

[0224] At least two image frames in the second video data are selected, and element groups are obtained by element similarity, clustering algorithm, etc. When obtaining the element groups, some element group attributes such as element group position, etc. can also be obtained at the same time.

[0225] The element group, selection method, element similarity definition, and method for calculating element similarity can be referred to the above term explanation (2) element group content, which will not be repeated here.

[0226] The method of obtaining the element group by using the clustering algorithm includes: superimposing the image frames of the selected frames, and then applying the clustering algorithm to the superimposed image frames, whereby the clustering result is the grouping result of the element group.

[0227] In some embodiments, when using a clustering algorithm such as the K-means algorithm, the number of clusters in the clustering algorithm, ie, the number of element groups, can be predetermined based on the element detection result of step S7022 and the image frame of the selected frame.

[0228] It can be understood that the element group preserves the spatiotemporal integrity of the elements in the video and can be used as a basic unit for judging the completeness of the video content.

[0229] Figure 14 This is a schematic diagram of an exemplary scenario for obtaining element groups in an embodiment of the present application.

[0230] like Figure 14 As shown, after obtaining the second video data, which includes three image frames, after each image frame undergoes element detection in S7022, the elements detected in each image frame can be selected using a dotted rectangular frame.

[0231] There are three image frames, including a first image frame, a second image frame, and a third image frame. Element 1401A in the first image frame is selected as a reference element, and all elements in the second and third image frames are selected as compared elements. The similarity between the reference element and each compared element is calculated. When the element similarity is greater than a preset element similarity threshold of 0.85, reference element 1401A and the compared element are considered to be the same element, that is, reference element 1401A and the compared element belong to the same element group. Element 1401B in the second image frame has an element similarity of 0.95 with reference element 1401A, and element 1401C in the third image frame has an element similarity of 0.94 with reference element 1401A. Since the element similarities of elements 1401B and 1401C with reference element 1401A are greater than 0.85, elements 1401A, 1401B, and 1401C form an element group, which can be denoted as 1401-Group-"Original Video 1." Similarly, we can obtain element groups 1402-Group-“HUAWE”, 1403-Group-Dog, 1404-Group-Dog Food, and 1405-Group-“Dog is a good friend of man”. The process of obtaining the element groups will not be repeated here.

[0232] Figure 15 This is another exemplary scenario diagram for obtaining element groups in an embodiment of the present application.

[0233] like Figure 15 As shown, after obtaining the second video data, which includes three image frames, after each image frame undergoes element detection in step S7022, the elements detected in each image frame can be selected using a dotted rectangular frame.

[0234] The three image frames are superimposed and then clustered using a pre-determined number of clusters (5) to obtain element groups. The element groups include: 1401-Group-"Original Video 1", 1402-Group-"HUAWE", 1403-Group-"Dogs", 1404-Group-"Dog Food", and 1405-Group-"Dogs are Man's Best Friends".

[0235] S7024: Obtain suspected incomplete element groups based on element group positions.

[0236] When the position of an element group is close to or covers the edge of the video screen, the element group can be considered as a suspected incomplete element group.

[0237] If the distance between the position of the element group and the border of the video screen is less than a preset second distance threshold, it is considered that the position of the element group is close to and covers the edge of the screen.

[0238] According to the position of the element obtained in step S7022 and the element group to which the element belongs determined in step S7023, the position of the element group can be obtained. The position of the element group is the range occupied by the union of all elements belonging to the element group. The position of the element group can be expressed in various forms. For example, the range can be based on the range selected by the rectangular selection box, specifically expressed as a set of four-dimensional coordinates (x, y, h, w), where (x, y) is the coordinate of the upper left corner of the rectangular selection box, the coordinate takes the lower left corner of the screen as the origin, the x-axis direction is from left to right, the y-axis direction is from bottom to top, h is the height of the rectangular selection box, and w is the width of the rectangular selection box. Based on x and h, it can be calculated whether the element group is close to or covers the edge of the video screen in the x-axis (horizontal) direction; based on y and w, it can be calculated whether the element group is close to or covers the edge of the video screen in the y-axis (vertical) direction.

[0239] Exemplarily, for example, if the screen of the second video is a rectangle, its height is H, its width is W, and the preset second distance threshold is threshold1. If there is an element group, and the coordinates of the element group are (x1, y1, h1, w1), then the boundary between the element group and the video screen is threshold2 = min{min{abs(y1-h1), abs(H-y1)}, min{abs(W-x1-w1), x1}}, where min represents the minimum value and abs represents the absolute value. When threshold2 is less than or equal to threshold1, it can be considered that the element group is close to and covers the edge of the video screen, and the element group can be considered as a suspected incomplete element group; when threshold2 is greater than threshold1, the element group can not be considered as a suspected incomplete element group.

[0240] like Figure 14 as well as Figure 15 As shown, the suspected incomplete element groups include: 1401-Group-"Original Video 1", 1402-Group-"Huawei HUAWE", and 1405-Group-"Dogs are good friends of humans".

[0241] It is worth noting that step S7024 may not be performed, that is, the element groups may not be screened. In this case, the suspected incomplete element groups are all element groups.

[0242] It can be understood that by screening suspected incomplete element groups, the number of element groups to be processed in subsequent steps is further reduced, and the amount of calculation is reduced.

[0243] S7025: Determine the completeness of the suspected incomplete element group and obtain the incomplete element group

[0244] The process of determining the integrity of a suspected incomplete element group can be referred to in the above term explanation (3) Integrity of an element group, which will not be repeated here.

[0245] When the completeness of the suspected incomplete element group is complete, the suspected incomplete element group is a complete element group; when the completeness of the suspected incomplete element group is incomplete, the suspected incomplete element group is an incomplete element group.

[0246] It is worth noting that step S7025 may not be performed. In this case, all suspected incomplete element groups are incomplete element groups.

[0247] It can be understood that the integrity of an element group is different from the integrity of the elements. The integrity of an element group is a measure of whether the element group affects the integrity of the video content, taking into account the temporal and spatial integrity of the element group. A complete element group is one that does not affect the integrity of the video content, while an incomplete element group is one that may affect the integrity of the video content.

[0248] In some embodiments, during the process of calculating the completeness of a suspected incomplete element group, an incompleteness ratio parameter of the element group can be calculated. For example, the degree of incompleteness of the element group can be calculated based on the distance between the proportional relationship of the element group and a preset proportional relationship threshold. The degree of incompleteness of the element group can also be calculated based on the distance between the proportional relationship of the element group, the projection feature, and a preset proportional relationship and projection feature threshold. The degree of incompleteness includes, for example, the incompleteness ratio.

[0249] Figure 16 This is an exemplary schematic diagram of the integrity judgment result of an element group in an embodiment of the present application.

[0250] like Figure 16 As shown, after obtaining the element group type to which the element group belongs, the results of further judging the degree of incompleteness through projection features, proportional relationships, etc. include: 403-Group-Dog element group is complete, 1404-Group-Dog food element group is complete, 1405-Group-"Dog is a good friend of man" element group is incomplete, 1401-Group-"Original Video 1" element group is incomplete, and 1402-Group-"Huawei HUAWE" element group is incomplete.

[0251] S7026: Calculate the element group attributes of the incomplete element group.

[0252] Element group attributes of each incomplete element group are determined based on the element group type to which the incomplete element group belongs. The element group attributes of the incomplete element group include at least one first incompleteness parameter value. The first incompleteness parameter value is used to indicate the incompleteness ratio of the element group. The element group attributes of the incomplete element group may also include a second incompleteness parameter value and a third incompleteness parameter value, etc., used to indicate the incompleteness type, duration, time period type, importance, surrounding element groups, etc.

[0253] The first incomplete parameter value can be expressed in various forms. The first incomplete parameter value can be a numerical value, a vector, a row in a matrix, a column in a matrix, etc., which is not limited here.

[0254] Exemplarily, when the first defect parameter value is a column in a matrix, the first defect parameter value is used to represent the degree of defect of any element in the element group in the image frame corresponding to the element. The defect degree calculation process includes calculating the element's defect degree using a proportional relationship, projection characteristics, and the like. Furthermore, the defect ratio of the element group can be determined using the first defect parameter.

[0255] For example, when the second video is a video of M frames, and all frames are selected when obtaining the element group, in this case, the attributes of the element group D-Group can be expressed as a matrix of M columns. Among them, at least one column of data in the matrix contains the degree of incompleteness of the elements in the incomplete element group, which is the first incompleteness parameter value of the element group D-Group. When the third column of data in the matrix represents the first incompleteness parameter value of the element group D-Group, the data in the third column and the first row represent the degree of incompleteness of the elements of the first image frame in the second video in the element group D-Group. The incompleteness ratio of D-Group is obtained according to the first incompleteness parameter value of D-Group.

[0256] Exemplarily, when the first incomplete parameter is expressed as a numerical value, the first incomplete parameter value is the incompleteness ratio of the element group.

[0257] According to the completeness of the elements in the incomplete element group on each image frame in the second video, the second incomplete parameter value in the second incomplete parameter group of each incomplete element group is determined respectively, where the second incomplete parameter value may include: duration, time period type, surrounding element group, incomplete type, importance, etc.

[0258] Taking duration as an example, duration is the shortest period of time during which the elements of the incomplete element group appear incompletely in the second video. For any incomplete element group, the code 1 can be used to indicate incompleteness, 0 to indicate completeness, and -1 to indicate non-appearance. The completeness of the element in each frame (one frame per second) 0 can be expressed as: (30 bits in total) 000001100001111100001111110-1-1-1. If the element group appears for 30 seconds and the shortest incomplete period is 21 seconds, then the duration value in the second incomplete parameter value in the first incomplete parameter group of the incomplete element group is 21 seconds. There are multiple methods for defining whether an element in an element group is incomplete, complete, or absent in a frame in which it appears. For example, when the incompleteness value of a frame in the first incompleteness parameter of the first incompleteness parameter group of the element group is greater than a preset incompleteness threshold, the element is considered incomplete in the frame; if the incompleteness value of a frame is a default value, the element is considered absent in the frame; if the incompleteness value of a frame is less than a preset incompleteness threshold, the element is considered complete in the image frame. Alternatively, in step S7022, when the electronic device performs element detection on the image frame, upon obtaining the element, it simultaneously detects whether the element appears and, if so, whether it is complete.

[0259] The importance is related to the element group type to which the element group belongs and the area occupied by the element group; the incompleteness type is related to the proportional relationship of the element groups, and the values ​​of the incompleteness type include: horizontal incompleteness and vertical incompleteness; the surrounding element groups are related to the positions of all element groups; the time period type is a preset value, and the value of the time period type can be 20 seconds after the start of the video (before the end of the video), 20 seconds after the transition point (before the transition point), etc.

[0260] S7027: Calculate the impact of the incomplete element group on the video image content.

[0261] According to the element group attributes of the incomplete element group obtained in S7026 and the preset rules for the element group affecting the integrity of the picture, the impact of the element group on the video picture content can be determined, where the results include: the incomplete element group affects the integrity of the video picture content, and the incomplete element group does not affect the integrity of the video picture.

[0262] The rules for element groups affecting picture integrity and the results of determining the impact of incomplete element groups on video picture content can be referred to the above term explanation (4) preset rules for element groups affecting picture integrity, which will not be repeated here.

[0263] Figure 17 This is an exemplary structural diagram of the rule that element groups affect the integrity of the picture in the embodiment of the present application. Figure 17 As shown, element groups of different element group types correspond to at least one element group affecting the picture integrity rule.

[0264] The rule regarding the impact of element groups on picture integrity can also be expressed in other forms. For example, the rule can be expressed through an evaluation function f1. The reference value and threshold value in the rule regarding the impact of element groups on picture integrity represent the mapping relationship between the input and output of evaluation function f1. The input of evaluation function f1 is the element group attributes of the incomplete element group, and the output is whether the element group affects the video picture content integrity or whether the element group does not affect the video picture content integrity.

[0265] Figure 18 This is an exemplary diagram of the effect of element groups on picture integrity in the embodiment of the present application. Figure 18 As shown, taking the element group "Other Post-Text 1401 - Group - "Original Video 1" as an example, its incompleteness is 0.15, its duration is 30 seconds, and its importance is 5. Therefore, the output of the integrity evaluation function f1 for the element group "Other Post-Text 1401 - Group - "Original Video 1" is: this element group affects the integrity of the original video's picture content. Similarly, the output of the integrity evaluation function f1 for the element group 1402 - Group - "Huawei HUAWE" is: this element group affects the integrity of the original video's picture content. The output of the integrity evaluation function f1 for the element group 1403 - Group - "This element group does not affect the integrity of the original video's picture content." The output of the integrity evaluation function f1 for the element group 1404 - Group - "Dog Food" is: this element group does not affect the integrity of the original video's picture content. The output of the integrity evaluation function f1 for the element group 1405 - Group - "Dogs are Man's Best Friends" is: this element group does not affect the integrity of the original video's picture content.

[0266] S703: Obtaining the original video content integrity evaluation result.

[0267] It can be understood that when step S701 is executed, the original video is cut into N first videos. At this time, step S702 will output N groups of data after execution, where each group of data includes: an incomplete element group, the impact of the incomplete element group on the video screen content, and the element group attributes of the incomplete element group.

[0268] In an embodiment of the present application, the original video picture content integrity evaluation result can be obtained based on N groups of data, or the picture content integrity evaluation result of the second video can be obtained based on one group of data, and the original video picture content integrity evaluation result can be determined based on the N groups of picture content integrity evaluation results, which is not limited here.

[0269] In the embodiment of the present application, there are many specific ways to obtain the original video content integrity evaluation result based on the impact of the incomplete element group on the video content:

[0270] (1) Determine whether the original video content is complete based on the effect of the incomplete element group on the video content;

[0271] The integrity judgment model can be used to determine whether the original video content is complete, and the result can be notified to the user or video producer.

[0272] The definition, construction and training of the integrity judgment model can be referred to the above term explanation (5) integrity judgment model, which will not be repeated here.

[0273] This application does not limit the input parameters of the integrity judgment model. Without inventive efforts by those skilled in the art, the input of the integrity judgment model can be other image features or data features, such as the combination, linear superposition, or nonlinear superposition of parameters in the element group attributes, or other parameters used to express the degree of incompleteness of the element group, or other parameters used to express the position of the element group, the area occupied by the element group, etc.

[0274] The completeness judgment model can also be expressed in the form of a function or a mapping relationship. Taking function f2 as an example, the picture completeness evaluation function f2 is a many-to-two function relationship. The input is the impact of the incomplete element group on the video picture content and the incomplete element group. The output includes: the original video picture content is complete, the original video picture content is incomplete. The mapping relationship from the input to the output of function f2 can be: if there is at least one incomplete element group in the input that affects the completeness of the video picture content, then the output is that the original video picture content is incomplete; if 50% or more (the number of incomplete element groups that affect the completeness of the video picture content / the number of element groups) of incomplete element groups in the input affect the completeness of the video picture content, then the output is that the original video picture content is incomplete.

[0275] It is understandable that the completeness judgment model can quickly determine whether the content of the original video is complete, realize automatic machine review, and improve review efficiency.

[0276] (2) Based on the impact of the incomplete element group on the video content and the classification information of the original video, the score of the completeness of the original video content is obtained;

[0277] The scoring model can be used to determine the completeness score of the original video content, and the result can be notified to the user or video producer.

[0278] The definition, construction and training of the scoring model can be referred to the above term explanation (6) scoring model, which will not be repeated here. The scoring model can be a model constructed using the BP neural network model in the field of artificial intelligence.

[0279] Figure 19This is an exemplary schematic diagram of the BP neural network model in the embodiment of this application.

[0280] like Figure 19 As shown in Figure 1, the BP neural network model includes: input layer, hidden layer, and output layer.

[0281] This application does not limit the input parameters of the scoring model. Without creative effort by those skilled in the art, the input of the scoring model can be other image features or data features, such as the combination, linear superposition, or nonlinear superposition of parameters in the element group attributes, or other parameters used to express the degree of incompleteness of the element group, or other parameters used to express the position of the element group, the area occupied by the element group, etc.

[0282] It is understandable that the scoring model can quantify the completeness of the original video content, and more effectively distinguish original videos of different completeness, which is conducive to video producers or users to have a more comprehensive understanding of the completeness of the original video.

[0283] (3) Based on the impact of the incomplete element group on the video image content and the element group attributes of the incomplete element group, modification suggestions for the integrity of the original video image content are obtained.

[0284] The definition, construction, and training of the modification suggestion generation model can be referred to in the above term explanation (7) Modification suggestion generation model, which will not be repeated here. The modification suggestion generation model can be a model constructed using a decision tree algorithm.

[0285] The selection of node attributes in the decision tree algorithm is related to the parameters in the element group attributes of the element group. Alternatively, the selection of node attributes in the decision tree algorithm is related to the threshold and / or reference value in the preset element group impacting the picture integrity rule.

[0286] The parameters in the element group attributes of the element group, the thresholds and / or reference values ​​in the element group's rules affecting picture integrity are node attributes with high gain and high purity, which can effectively improve the robustness, accuracy, precision, etc. of the decision tree algorithm.

[0287] It is understood that when a video producer or user receives a modification suggestion provided by an electronic device, they can modify the original video in a targeted manner based on the modification suggestion to improve the integrity of the original video content. The modification suggestion can be presented in the form of an audit report.

[0288] This application does not limit the input parameters or node parameters of the modification suggestion generation model. Without creative effort by those skilled in the art, the input of the modification suggestion generation model or the node parameters in the decision tree algorithm can be other image features or data features, such as the combination, linear superposition, nonlinear superposition, etc. of parameters in the element group attributes, or other parameters used to express the degree of incompleteness of the element group, or other parameters used to express the position of the element group, the area occupied by the element group, etc.

[0289] Figure 20 A schematic diagram of the architecture of the modification suggestion generation model in the embodiment of the present application.

[0290] like Figure 20 As shown, parameters such as the area occupied by the incomplete element group, duration, importance, and degree of incompleteness can be selected as node attributes, and a modification suggestion generation model can be constructed and trained through a decision tree algorithm.

[0291] Exemplarily, when any incomplete element group is input into the modification suggestion generation model constructed and trained using the decision tree algorithm: when the area of ​​the element group is less than 5% of the video screen, and the degree of incompleteness of the element group is less than 1 / 3, the output of the modification suggestion generation model for the incomplete element group is: no processing (Pass); when the area of ​​the element group is less than 5% of the video screen, and the degree of incompleteness of the element group is greater than 1 / 3, and the duration of the element group is less than 15 seconds, the output of the modification suggestion generation model for the incomplete element group is: no processing (Pass); when the area of ​​the element group is less than 5% of the video screen, and the degree of incompleteness of the element group is greater than 1 / 3, and the duration of the element group is greater than 15 seconds and less than 30 seconds, the output of the modification suggestion generation model for the incomplete element group is: crop (Cut); when the area of ​​the element group is less than 5% of the video screen, and the degree of incompleteness of the element group is greater than 1 / 3, and the duration of the element group is greater than 15 seconds and less than 30 seconds. When the area of ​​the element group is between 5% and 50% of the video screen, and the duration is less than 1 second, the output of the modification suggestion generation model for the incomplete element group is: no processing (Pass); when the area of ​​the element group is between 5% and 50% of the video screen, and the duration is less than 1 second, and the importance is less than or equal to 3, the output of the modification suggestion generation model for the incomplete element group is: Cover; when the area of ​​the element group is between 5% and 50% of the video screen, and the duration is less than 1 second, and the importance is greater than 3, the output of the modification suggestion generation model for the incomplete element group is: Cut; when the area of ​​the element group is greater than 50% of the video screen, the output of the modification suggestion generation model for the incomplete element group is: Cut.

[0292] Figure 21 A schematic diagram of modification suggestions for the original video image output by the modification suggestion generation model in an embodiment of the present application.

[0293] like Figure 21 As shown, the image modification suggestions for the original video include: element group, element group type, element group start time, element group duration, element group position, and element group modification suggestions. For example, the modification suggestion for element group 1401-Group-"Original Video 1" is Cut; the modification suggestion for element group 1402-Group-"HUAWE" is Cover; the modification suggestion for element group 1403-Group-"Dog" is Pass; the modification suggestion for element group 1404-Group-"Dog Food" is Pass; and the modification suggestion for element group 1405-Group-"Dogs are Man's Best Friends" is Cut.

[0294] It is worth noting that the video content integrity assessment method involved in this application can be run on an offline electronic device or on a cloud-based electronic device; wherein, the offline electronic device can be a local networked or unnetworked electronic device of the video producer or user, such as a computer, mobile terminal, etc.; the cloud-based electronic device can be a cloud server of the video review system.

[0295] In the embodiment of the present application, the electronic device may be a mobile electronic device or a PC, which is not limited here. For example, Figure 22 A structural diagram of an electronic device 100 provided in an embodiment of the present application.

[0296] The following embodiments are described in detail using electronic device 100 as an example. It should be understood that electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different component configuration. The various components shown in the figure may be implemented in hardware, including one or more signal processing and / or application-specific integrated circuits, software, or a combination of hardware and software.

[0297] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0298] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0299] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0300] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0301] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0302] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0303] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing the touch function of the electronic device 100.

[0304] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls through a Bluetooth headset.

[0305] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0306] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0307] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0308] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0309] The SIM interface can be used to communicate with the SIM card interface 195 to implement the function of transmitting data to the SIM card or reading data in the SIM card.

[0310] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0311] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0312] The charging management module 140 is configured to receive charging input from a charger, which may be a wireless charger or a wired charger.

[0313] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160.

[0314] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0315] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0316] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0317] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0318] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0319] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0320] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0321] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0322] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0323] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0324] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0325] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0326] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0327] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0328] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0329] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation of DDR SDRAM is generally referred to as DDR5 SDRAM), etc.

[0330] Non-volatile memory may include disk storage devices and flash memory.

[0331] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage cell potential level; universal flash storage (UFS) and embedded multi media card (eMMC) can be divided into UFS and embedded multi media card according to the storage specification.

[0332] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0333] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0334] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 via the external memory interface 120 to implement data storage. For example, files such as music and videos can be stored in the external non-volatile memory.

[0335] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0336] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0337] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0338] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0339] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0340] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0341] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be located on display screen 194. There are many types of pressure sensors 180A, such as resistive, inductive, and capacitive. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When force acts on pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the intensity of the pressure based on this change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch location based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch location but with different touch intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a first pressure threshold is applied to a short message application icon, a command to view short messages is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to a short message application icon, a command to create a new short message is executed.

[0342] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0343] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0344] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be configured.

[0345] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0346] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0347] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0348] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0349] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0350] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from shutting down abnormally due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to prevent abnormal shutdown due to low temperature.

[0351] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, in a location different from that of the display screen 194.

[0352] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0353] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0354] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0355] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and disconnected from the electronic device 100 by inserting it into or removing it from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications.

[0356] In the embodiment of the present application, the processor 110 can call the computer instructions stored in the internal memory 121 to enable the electronic device 100 to execute the video picture content integrity assessment method in the embodiment of the present application.

[0357] Figure 23 1 is a schematic block diagram of the software structure of the electronic device 100 in an embodiment of the present invention.

[0358] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, the system is divided into four layers: application layer, application framework layer, runtime and system libraries layer, and kernel layer.

[0359] The application layer can include a series of application packages.

[0360] like Figure 23 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications (also referred to as applications).

[0361] In the embodiment of the present application, the application layer may further include a video evaluation module.

[0362] The video evaluation module can be used to execute the video content integrity evaluation method in the embodiment of the present application.

[0363] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0364] like Figure 23 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, a local profile assistant (LPA), and the like.

[0365] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0366] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0367] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0368] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0369] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0370] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically, without requiring user interaction. For example, the Notification Manager can be used to notify users of completed downloads, message reminders, and so on. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog interfaces on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0371] The runtime includes the core library and the virtual machine. The runtime is responsible for scheduling and management of the Android system.

[0372] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the core library.

[0373] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0374] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0375] The surface manager is used to manage the display subsystem and provide the fusion of two-dimensional (2D) and three-dimensional (3D) layers for multiple applications.

[0376] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0377] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0378] A 2D graphics engine is a drawing engine for 2D drawings.

[0379] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, sensor driver, and virtual card driver.

[0380] For example, Figure 24 A hardware structure diagram of the electronic device 200 provided in an embodiment of the present application.

[0381] The electronic device 200 includes:

[0382] Input device 201, output device 202, processor 203 and memory 204 (wherein the number of processor 203 in electronic device 200 can be one or more, Figure 24In some embodiments of the present application, the input device 201, the output device 202, the processor 203 and the memory 204 may be connected via a bus or other means, wherein: Figure 24 The bus connection is taken as an example.

[0383] The processor 203 calls the operating instructions stored in the memory 204 to enable the electronic device 200 to execute the video picture content integrity assessment method in the embodiment of the present application.

[0384] As used in the above embodiments, the term “when…” may be interpreted to mean “if…” or “after…” or “in response to determining…” or “in response to detecting…”, depending on the context. Similarly, the phrases “upon determining…” or “if (stated condition or event) is detected” may be interpreted to mean “if determining…” or “in response to determining…” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.

[0385] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).

[0386] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for evaluating video content, characterized in that: include: The electronic device identifies elements in at least two image frames in a video to be detected, the at least two image frames including a first image frame and a second image frame, the elements being basic elements in the image frames; The electronic device determines an element group in the video to be detected based on the identified elements, wherein the same or identical elements in multiple image frames belong to one element group; the element group includes a first element group, which includes a first element in the first image frame and a first element in the second image frame; The electronic device determines an element group attribute of the first element group, where the element group attribute of the first element group includes a first incompleteness parameter value, and the first incompleteness parameter value is used to indicate a degree of incompleteness of the first element group; The electronic device determines a picture content evaluation result of the video to be detected based at least on the element group attribute of the first element group.

2. The method according to claim 1, characterized in that The electronic device determines, at least according to the element group attribute of the first element group, a picture content evaluation result of the video to be detected, specifically including: The electronic device determines, based on the element group attributes of the first element group, an influence result of the first element group on the content of the video picture to be detected; The electronic device determines a picture content evaluation result of the video to be detected based on at least an impact result of the first element group on the picture content of the video to be detected.

3. The method according to claim 2, characterized in that The electronic device determines, according to the element group attribute of the first element group, an influence result of the first element group on the content of the video picture to be detected, specifically including: The electronic device determines the impact of the first element group on the content of the video picture to be detected according to the element group attribute of the first element group and a preset rule of the element group affecting the picture integrity.

4. The method according to claim 2, characterized in that The electronic device determines, based at least on the effect of the first element group on the picture content of the video to be detected, a picture content evaluation result of the video to be detected, specifically including: The electronic device determines whether the video to be detected is complete based on at least the effect of the first element group on the content of the video to be detected; and / or, the electronic device determines a score of the video to be detected based at least on the impact of the first element group on the content of the video to be detected and classification information of the video to be detected; And / or, the electronic device determines modification suggestions for the video to be detected, including modification suggestions for the first element group, based at least on the impact of the first element group on the content of the video to be detected and the element group attributes of the first element group.

5. The method according to claim 1, wherein The electronic device determines, at least according to the element group attribute of the first element group, a picture content evaluation result of the video to be detected, specifically including: The electronic device determines whether the video to be detected is complete based on at least the element group attribute of the first element group; and / or, the electronic device determines a score of the video to be detected based at least on the element group attributes of the first element group and classification information of the video to be detected; And / or, the electronic device determines, based at least on the element group attributes of the first element group, modification suggestions for the video to be detected, including modification suggestions for the first element group.

6. The method according to claim 4, characterized in that The modification suggestions for the first element group include at least one of: pass, no processing, cropping, smearing, frame deletion, and blurring.

7. The method according to any one of claims 1 to 6, characterized in that The electronic device determines, based on the identified elements, an element group in the video to be detected, including a first element group, specifically including: The electronic device forms an element group with the elements representing the same basic element in the at least two image frames that are identified.

8. The method according to claim 7, characterized in that The electronic device forms an element group with elements representing the same basic element in the at least two image frames, including a first element group, which specifically includes: The electronic device determines a first element similarity between a first element in the first image frame and a first element in the second image frame, where the element similarity is used to indicate a degree of similarity between element images; The electronic device determines that the similarity of the first elements is greater than a preset similarity threshold, and forms a first element group with the first element in the first image frame and the first element in the second image frame.

9. The method according to claim 8, characterized in that Before the step of determining the first element similarity between the first element in the first image frame and the first element in the second image frame, the method further includes: The electronic device determines that the distance between the first position and the second position is less than a preset first distance threshold; the first position is the position of the first element in the first image frame in the first image frame, and the second position is the position of the first element in the second image frame in the second image frame.

10. The method according to any one of claims 1 to 6, 8 to 9, characterized in that After the electronic device determines, based on the identified elements, an element group in the video to be detected, including a first element group, the method further includes: The electronic device calculates the completeness of the element group in the video to be detected, where the completeness is used to indicate whether the element group is complete; The electronic device filters out an incomplete element group from the element groups in the video to be detected, including the first element group, and the completeness of the incomplete element group is incomplete.

11. The method according to any one of claims 1 to 6, 8 to 9, characterized in that After the electronic device determines, based on the identified elements, an element group in the video to be detected, including a first element group, the method further includes: The electronic device filters out a suspected incomplete element group from the element group in the video to be detected, including the first element group, and the suspected incomplete element group is an element group whose position is less than a preset second distance threshold from the edge of the picture.

12. The method according to claim 11, characterized in that After the electronic device screens out suspected incomplete element groups from the element groups in the video to be detected, the method further includes: The electronic device calculates the completeness of the suspected incomplete element group in the video to be detected, where the completeness is used to indicate whether the element group is complete; The electronic device filters out an incomplete element group from the suspected incomplete element groups in the video to be detected, including the first element group, and the completeness of the incomplete element group is incomplete.

13. The method according to any one of claims 1 to 6, 8 to 9 and 12, characterized in that The element group attributes of the first element group further include a second incomplete parameter value, which is used to represent at least one of the incomplete type, duration, importance, surrounding element group, and time period type of the first element group.

14. An electronic device, characterized in that: The electronic device includes: one or more processors and memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, where the computer program code includes computer instructions. The one or more processors call the computer instructions to cause the electronic device to execute: Identifying elements in at least two image frames in a video to be detected, the at least two image frames including a first image frame and a second image frame, the elements being basic elements in the image frames; Determining an element group in the video to be detected based on the identified elements, wherein the same or identical elements in multiple image frames belong to one element group; the element group includes a first element group, the first element group including a first element in the first image frame and a first element in the second image frame; determining an element group attribute of the first element group, where the element group attribute of the first element group includes a first incompleteness parameter value, where the first incompleteness parameter value is used to indicate a degree of incompleteness of the first element group; A picture content evaluation result of the video to be detected is determined based on at least the element group attribute of the first element group.

15. The electronic device according to claim 14, characterized in that The one or more processors are specifically configured to call the computer instructions to enable the electronic device to execute: determining, based on the element group attributes of the first element group, an influence result of the first element group on the content of the video picture to be detected; An evaluation result of the picture content of the video to be detected is determined based on at least the impact of the first element group on the picture content of the video to be detected.

16. The electronic device according to claim 15, characterized in that The one or more processors are specifically configured to call the computer instructions to enable the electronic device to execute: According to the element group attributes of the first element group and the preset element group impacting picture integrity rule, the impact result of the first element group on the content of the to-be-detected video picture is determined.

17. The electronic device according to claim 15, characterized in that The one or more processors are specifically configured to call the computer instructions to enable the electronic device to execute: determining whether the video to be detected is complete based on at least the effect of the first element group on the content of the video to be detected; and / or, determining a score of the video to be detected based at least on an impact of the first element group on the content of the video to be detected and classification information of the video to be detected; And / or, determining modification suggestions for the video to be detected, including modification suggestions for the first element group, at least based on the impact of the first element group on the content of the video to be detected and the element group attributes of the first element group.

18. The electronic device according to claim 14, wherein: The one or more processors are specifically configured to call the computer instructions to enable the electronic device to execute: determining whether the video to be detected is complete based on at least an element group attribute of the first element group; and / or, determining a score of the video to be detected based at least on the element group attributes of the first element group and classification information of the video to be detected; And / or, determining modification suggestions for the video to be detected at least based on the element group attributes of the first element group, including modification suggestions for the first element group.

19. The electronic device according to claim 17, wherein: The modification suggestions for the first element group include at least one of: pass, no processing, cropping, smearing, frame deletion, and blurring.

20. The electronic device according to any one of claims 14 to 19, characterized in that: The one or more processors are specifically configured to call the computer instructions to enable the electronic device to execute: The elements representing the same basic element in the at least two identified image frames are formed into an element group.

21. The electronic device according to claim 20, characterized in that The one or more processors are specifically configured to call the computer instructions to enable the electronic device to execute: determining a first element similarity between a first element in the first image frame and a first element in the second image frame, where the element similarity is used to indicate a degree of similarity between element images; It is determined that the similarity of the first elements is greater than a preset similarity threshold, and the first element in the first image frame and the first element in the second image frame are combined into a first element group.

22. The electronic device according to claim 21, wherein: The one or more processors are further configured to call the computer instructions to cause the electronic device to execute: Determine that the distance between the first position and the second position is less than a preset first distance threshold; the first position is the position of the first element in the first image frame in the first image frame, and the second position is the position of the first element in the second image frame in the second image frame.

23. The electronic device according to any one of claims 14 to 19, 21 to 22, characterized in that: The one or more processors are further configured to call the computer instructions to cause the electronic device to execute: Calculating the completeness of the element group in the video to be detected, where the completeness is used to indicate whether the element group is complete; An incomplete element group is screened out from the element groups in the video to be detected, including the first element group, and the completeness of the incomplete element group is incomplete.

24. The electronic device according to any one of claims 14 to 19, 21 to 22, characterized in that: The one or more processors are further configured to call the computer instructions to cause the electronic device to execute: A suspected incomplete element group is screened out from the element groups in the video to be detected, including the first element group. The suspected incomplete element group is an element group whose distance from the edge of the picture is less than a preset second distance threshold.

25. The electronic device according to claim 24, characterized in that The one or more processors are further configured to call the computer instructions to cause the electronic device to execute: Calculating the completeness of the suspected incomplete element group in the video to be detected, where the completeness is used to indicate whether the element group is complete; An incomplete element group is screened out from the suspected incomplete element groups in the video to be detected, including the first element group, and the completeness of the incomplete element group is incomplete.

26. The electronic device according to any one of claims 14 to 19, 21 to 22, and 25, characterized in that: The element group attributes of the first element group further include a second incomplete parameter value, which is used to represent at least one of the incomplete type, duration, importance, surrounding element group, and time period type of the first element group.

27. A chip system, applied to an electronic device, the chip system comprising one or more processors, the processors being configured to call computer instructions so that the electronic device executes the method as claimed in any one of claims 1 to 13.

28. A computer program product comprising instructions, characterized in that When the computer program product is run on an electronic device, the electronic device is enabled to perform the method according to any one of claims 1 to 13.

29. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 13.

Images