Artificial Intelligence-Based Content Review Method and System

Abstract

The present application provides a content review method and system based on artificial intelligence. The method includes: a computer device receives the content to be reviewed, and the content can be a short video; the computer device detects the short video to determine whether it is a picture-in-picture video. If it is a picture-in-picture video, the picture-in-picture video is split into two videos, and each of the two videos is used as a single video to perform content review; if it is not a picture-in-picture video, the short video is used as a single video to perform content review; when the computer device determines that each video has passed the review, it determines that the content review of the picture-in-picture video has passed, and when any one of the multiple videos fails the review, it determines that the content review of the picture-in-picture video fails. The technical solution provided by the present application has the advantage of high review accuracy.

Description

Technical Field

[0001] The present invention relates to the Internet and the field of images, and particularly to a content review method and system based on artificial intelligence. Background Art

[0002] Content review is a review method that many Internet platforms must carry out. With the rise of short videos, the number of short video reviews is increasing. The method of manual review can no longer meet the requirements of content review. Especially for short videos, they can be presented as a whole video through splicing (such as picture-in-picture). Such content cannot obtain results through a single review. Therefore, an artificial intelligence review method is needed to implement content review and improve the accuracy of review. Summary of the Invention

[0003] Embodiments of the present invention provide a content review method and system based on artificial intelligence, which perform artificial intelligence review on the splicing of short videos, improve the accuracy of review, and have the advantage of improving the user experience.

[0004] In a first aspect, embodiments of the present invention provide a content review method based on artificial intelligence. The method includes the following steps:

[0005] A computer device receives content to be reviewed, and the content may be: a short video;

[0006] The computer device detects the short video to determine whether it is a picture-in-picture video. If it is a picture-in-picture video, the picture-in-picture video is split into two videos, and each video in the two videos is used as a single video to perform content review; if it is not a picture-in-picture video, the short video is used as a single video to perform content review;

[0007] When the computer device determines that each video has passed the review, it determines that the content review of the picture-in-picture video has passed. When any one of the multiple videos fails the review, it determines that the content review of the picture-in-picture video fails.

[0008] In a second aspect, a content review system based on artificial intelligence is provided. The system includes:

[0009] A communication unit for receiving content to be reviewed, and the content may be: a short video;

[0010] An auditing unit is configured to detect a short video to determine whether it is a picture-in-picture video. If it is a picture-in-picture video, the picture-in-picture video is split into two videos, and each of the two videos is used as a single video to perform content auditing. If it is not a picture-in-picture video, the short video is used as a single video to perform content auditing. When it is determined that each video passes the audit, it is determined that the content audit of the picture-in-picture video passes. When it is determined that any one of the multiple videos fails the audit, it is determined that the content audit of the picture-in-picture video fails.

[0011] In a third aspect, a computer-readable storage medium is provided, which stores a program for electronic data exchange. Wherein, the program causes the terminal to execute the method provided in the first aspect.

[0012] Implementing the embodiments of the present invention has the following beneficial effects: The technical solution provided in this application enables a computer device to receive content to be audited, and the content can be: a short video; the computer device detects the short video to determine whether it is a picture-in-picture video. If it is a picture-in-picture video, the picture-in-picture video is split into multiple videos, and each of the multiple videos is used as a single video to perform content auditing. If it is not a picture-in-picture video, the short video is used as a single video to perform content auditing. When the computer device determines that each video passes the audit, it is determined that the content audit of the picture-in-picture video passes. When it is determined that any one of the multiple videos fails the audit, it is determined that the content audit of the picture-in-picture video fails. In this way, by performing different strategies on the short video for content auditing, the accuracy of the audit is improved, and the user experience is enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0014] Figure 1 is a schematic structural diagram of a computer device

[0015] Figure 2 is a schematic flowchart of a content auditing method based on artificial intelligence;

[0016] Figure 3 is a schematic structural diagram of a content auditing system based on artificial intelligence. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0018] The terms "first", "second", "third", "fourth", etc. in the description and claims of the present invention and the accompanying drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0019] Referring to

[0020] Referring to Figure 1 , Figure 1 A computer device is provided. Specifically, the above computer device can be a computer device with operating systems such as IOS, Android, HarmonyOS, etc. Of course, it can also be a computer device with other systems, and the present application does not limit the specific operating system above. The above computer device can be connected to other devices wirelessly. Of course, it can also be connected to other devices by wire. The computer device can specifically be: a personal computer, a server, a smart phone, a tablet computer, etc.

[0021] For short videos, since there are many ways to cut them, for Internet platforms, it is allowed to quote some known short videos. For example, video A + original video B is allowed. However, for the spliced video C of video A + non-original video B, video C is a separate video, and it is not similar to video A and video B during comparison. However, this type of video belongs to simple splicing, which is not allowed on Internet video platforms. However, since the content of video C is independent content, it cannot be effectively audited through existing artificial intelligence audits. Therefore, a new artificial intelligence content audit method is needed to implement the audit of these videos.

[0022] Referring toFigure 2 , Figure 2 is a schematic flowchart of a content review method based on artificial intelligence provided by this application. This method is executed by a computer device as shown in Figure 1 and includes the following steps as shown in Figure 2 :

[0023] Step S201: The computer device receives the content to be reviewed, and the content can be: short video;

[0024] The above short video can be short videos from different platforms, and this application does not limit the specific forms of the above platforms.

[0025] Step S202: The computer device detects the short video to determine whether it is a picture-in-picture video. If it is a picture-in-picture video, the picture-in-picture video is split into 2 videos, and each video in the 2 videos is used as a single video to perform content review; if it is not a picture-in-picture video, the short video is used as a single video to perform content review;

[0026] Step S203: When the computer device determines that each video has passed the review, it determines that the content review of the picture-in-picture video has passed. When any one of the multiple videos fails the review, it determines that the content review of the picture-in-picture video fails.

[0027] The technical solution provided by this application is that the computer device receives the content to be reviewed, and the content can be: short video; the computer device detects the short video to determine whether it is a picture-in-picture video. If it is a picture-in-picture video, the picture-in-picture video is split into multiple videos, and each video in the multiple videos is used as a single video to perform content review; if it is not a picture-in-picture video, the short video is used as a single video to perform content review; when the computer device determines that each video has passed the review, it determines that the content review of the picture-in-picture video has passed. When any one of the multiple videos fails the review, it determines that the content review of the picture-in-picture video fails. In this way, by performing different strategies on the short video for content review, the accuracy of the review is improved, and the user experience is improved.

[0028] Exemplarily, the computer device detecting the short video to determine whether it is a picture-in-picture video can specifically include:

[0029] The computer device divides the short video into n equal parts vertically, extracts x frame sub-images from the first equal part among the n equal parts, forms x matrices with the RGB values of the pixel points of each frame sub-image of the x frame sub-images, calculates the differences between the matrices corresponding to adjacent frame sub-images to determine x difference matrices, obtains the pixel point regions corresponding to the non-zero elements from the x difference matrices and determines them as the sub-regions to be determined, traverses the n equal parts to obtain n determined sub-regions, connects the n determined sub-regions in the order from left to right to obtain the short video region, extracts the first sub-video of the first equal part among the n equal parts in the short video region, equally divides the first frame image of the first sub-video into 2 parts vertically, forms the first sub-matrix and the second sub-matrix with the RGB values of the pixel points of the 2 parts respectively, calculates the difference between the first sub-matrix and the second sub-matrix to obtain the first difference sub-matrix, calculates the average value of all the element values of the first difference sub-matrix to obtain the first average value, traverses the x frames of the first sub-video to obtain the average values of the x frames of the first sub-video, calculates the variance of all the average values and the average value of all the average values, i.e., the total average value. If the variance is less than the variance threshold and the total average value is greater than the average threshold, it is determined that the short video has a picture-in-picture video. The above n can be an integer greater than or equal to 10.

[0030] Exemplarily, the above splitting the picture-in-picture video into 2 videos can specifically include:

[0031] The short video region of the picture-in-picture video is divided into 2 equal parts vertically, and each equal part corresponds to a single video.

[0032] The above method for determining the picture-in-picture is obtained by confirming the picture-in-picture video. For the picture-in-picture video, it is generally distributed vertically or horizontally. The above mainly solves the vertical distribution method. First, determine the attention region of the short video (because there may be non-video regions, i.e., non-attention regions, such as black regions, on the top and bottom of the short video), so it is necessary to determine the attention region, and then divide the attention region into upper and lower equal parts to obtain two sub-matrices. For the same frame video, since the outermost regions are taken here, the changes are relatively small, because it is probably the background region of the attention region. After equally dividing the two parts like this, if it is a completely identical video (i.e., not a picture-in-picture video), due to the correlation of the video, the difference between the upper and lower matrices is generally not too large. However, for the picture-in-picture video, since the upper and lower videos are completely different videos, the upper and lower changes will be very large, so the difference will generally be relatively large, but the difference fluctuation will be relatively small, that is, the variance will be small. Through this kind of detection, the detection of the picture-in-picture can be realized. The technical solution for the detection of the left-right structure is described below.

[0033] Exemplarily, the above computer device detecting the short video to determine whether it is a picture-in-picture video can specifically include:

[0034] The computer device divides the short video into n equal parts horizontally, extracts the x-frame sub-images of the first equal part among the n equal parts, forms x matrices with the RGB values of the pixel points of each frame of the x-frame sub-images, calculates the differences between the matrices corresponding to adjacent frame sub-images to determine x difference matrices, obtains the pixel point regions corresponding to the non-zero elements from the x difference matrices and determines them as the sub-regions to be determined, traverses the n equal parts to obtain n determined sub-regions, connects the n determined sub-regions in the order from top to bottom to obtain the short video region, extracts the first sub-video of the first equal part in the short video region among the n equal parts, equally divides the first frame image of the first sub-video into 2 parts horizontally, forms the first sub-matrix and the second sub-matrix with the RGB values of the 2 parts respectively, calculates the difference between the first sub-matrix and the second sub-matrix to obtain the first difference sub-matrix, calculates the average value of all the element values of the first difference sub-matrix to obtain the first average value, traverses the x frames of the first sub-video to obtain the average values of the x frames of the first sub-video, calculates the variance of all the average values of the first sub-video and the average value of all the average values, i.e., the total average value. If the variance is less than the variance threshold and the total average value is greater than the average threshold, it is determined that the short video has a picture-in-picture video.

[0035] Exemplarily, the splitting of the picture-in-picture video into 2 videos specifically may include:

[0036] Dividing the short video region of the picture-in-picture video into 2 equal parts horizontally, and each equal part corresponds to a single video.

[0037] Exemplarily, the specific manner of performing content review on the short video as a single video may adopt the review manner of the platform, and the review manner of the single video is not limited here.

[0038] Refer to Figure 3 , Figure 3 For a content review system based on artificial intelligence provided by the present application, the system includes:

[0039] A communication unit 301, configured to receive the content to be reviewed, and the content may be: a short video;

[0040] A review unit 302, configured to detect the short video to determine whether it is a picture-in-picture video. If it is a picture-in-picture video, split the picture-in-picture video into 2 videos, and perform content review on each of the 2 videos as a single video; if it is not a picture-in-picture video, perform content review on the short video as a single video; when it is determined that each video passes the review, it is determined that the content review of the picture-in-picture video passes, and when it is determined that any one of the multiple videos fails the review, it is determined that the content review of the picture-in-picture video fails.

[0041] Exemplarily,

[0042] The auditing unit is specifically configured to divide the short video into n equal parts in the vertical direction, extract x frame sub-images of the first equal part among the n equal parts, form x matrices with the RGB values of the pixel points of each frame sub-image of the x frame sub-images, calculate the differences between the matrices corresponding to adjacent frame sub-images to determine x difference matrices, obtain the pixel point regions corresponding to the non-zero elements from the x difference matrices and determine them as the sub-regions to be determined, traverse the n equal parts to obtain n determined sub-regions, connect the n determined sub-regions in the order from left to right to obtain the short video region, extract the first sub-video of the first equal part among the n equal parts in the short video region, divide the first frame image of the first sub-video into 2 equal parts in the vertical direction, form the first sub-matrix and the second sub-matrix with the RGB values of the pixel points of the 2 equal parts respectively, calculate the difference between the first sub-matrix and the second sub-matrix to obtain the first difference sub-matrix, calculate the average value of all the element values of the first difference sub-matrix to obtain the first average value, traverse the x frames of the first sub-video to obtain the average values of the x frames of the first sub-video, calculate the variance of all the average values of the first sub-video and the average value of all the average values, i.e., the total average value. If the variance is less than the variance threshold and the total average value is greater than the average threshold, it is determined that the short video has a picture-in-picture video; the above n is an integer greater than or equal to 10.

[0043] Exemplarily,

[0044] The auditing unit is specifically configured to divide the short video region of the picture-in-picture video into 2 equal parts in the vertical direction, and each equal part corresponds to a single video.

[0045] Exemplarily,

[0046] The auditing unit is specifically configured to divide the short video into n equal parts in the horizontal direction, extract x frame sub-images of the first equal part among the n equal parts, form x matrices with the RGB values of the pixel points of each frame sub-image of the x frame sub-images, calculate the differences between the matrices corresponding to adjacent frame sub-images to determine x difference matrices, obtain the pixel point regions corresponding to the non-zero elements from the x difference matrices and determine them as the sub-regions to be determined, traverse the n equal parts to obtain n determined sub-regions, connect the n determined sub-regions in the order from top to bottom to obtain the short video region, extract the first sub-video of the first equal part among the n equal parts in the short video region, divide the first frame image of the first sub-video into 2 equal parts in the horizontal direction, form the first sub-matrix and the second sub-matrix with the RGB values of the pixel points of the 2 equal parts respectively, calculate the difference between the first sub-matrix and the second sub-matrix to obtain the first difference sub-matrix, calculate the average value of all the element values of the first difference sub-matrix to obtain the first average value, traverse the x frames of the first sub-video to obtain the average values of the x frames of the first sub-video, calculate the variance of all the average values of the first sub-video and the average value of all the average values, i.e., the total average value. If the variance is less than the variance threshold and the total average value is greater than the average threshold, it is determined that the short video has a picture-in-picture video; the n is an integer greater than or equal to 10.

[0047] Exemplarily,

[0048] The auditing unit is specifically configured to divide the short - video area of the picture - in - picture video into two equal parts in the horizontal direction, and each equal part corresponds to a single video.

[0049] Exemplarily, the auditing unit in the embodiments of the present application can be used to execute Figure 2 the refinement solutions, alternative solutions, etc. of the embodiments shown, which will not be elaborated here.

[0050] The embodiments of the present invention also provide a computer storage medium. Among them, the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute some or all of the steps of any one of the content auditing methods based on artificial intelligence described in the above - mentioned method embodiments.

[0051] The embodiments of the present invention also provide a computer program product. The computer program product includes a non - transitory computer - readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute some or all of the steps of any one of the content auditing methods based on artificial intelligence described in the above - mentioned method embodiments.

[0052] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, some steps can be received in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0053] In the above - mentioned embodiments, the descriptions of each embodiment have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0054] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the coupling or direct coupling or communication connection shown or discussed with each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical or other form.

[0055] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable memory, which can include: a flash drive, a read-only memory (abbreviation: ROM), a random access memory (abbreviation: RAM), a magnetic disk, an optical disc, etc.

[0056] The above embodiments of the present invention have been introduced in detail. Specific examples are used herein to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A content review method based on artificial intelligence, the method comprising the following steps: A computer device receives the content to be reviewed, and the content is: a short video; The computer device detects the short video to determine whether it is a pip video. If it is a pip video, the pip video is split into two videos, and each of the two videos is used as a single video to perform content review; If it is not a pip video, the short video is used as a single video to perform content review; When the computer device determines that each video has passed the review, it determines that the pip video content has passed the review. When any one of the multiple videos fails the review, it determines that the pip video content fails the review; The computer device detecting whether the short video is a pip video specifically includes: The computer device divides the short video into n equal parts in the vertical direction, extracts x frame sub-images of the first equal part among the n equal parts, forms x matrices with the RGB values of the pixel points of each frame sub-image of the x frame sub-images, calculates the differences between the matrices corresponding to the adjacent frame sub-images to determine x difference matrices, obtains the pixel point regions corresponding to the non-zero elements from the x difference matrices to determine the sub-regions to be determined, traverses the n equal parts to obtain n determined sub-regions, connects the n determined sub-regions in the order from left to right to obtain the short video region, extracts the first sub-video of the first equal part in the short video region among the n equal parts, equally divides the first frame image of the first sub-video into 2 parts in the vertical direction, forms the first sub-matrix and the second sub-matrix with the RGB values of the pixel points of the 2 parts respectively, calculates the difference between the first sub-matrix and the second sub-matrix to obtain the first difference sub-matrix, calculates the average value of all the element values of the first difference sub-matrix to obtain the first average value, traverses the x frames of the first sub-video to obtain the average values of the x frames of the first sub-video, calculates the variance of all the average values of the first sub-video and the average value of all the average values, i.e., the total average value. If the variance is less than the variance threshold and the total average value is greater than the average threshold, it is determined that the short video has a pip video; where n is an integer greater than or equal to 10; The splitting of the pip video into 2 videos specifically includes: dividing the short video region of the pip video into 2 equal parts in the vertical direction, and each equal part corresponds to a single video; The computer device detecting whether the short video is a pip video specifically includes: The computer device divides the short video into n equal parts horizontally, extracts x frame sub-images from the first part among the n parts, forms x matrices with the RGB values of the pixel points of each frame sub-image of the x frame sub-images, calculates the differences between the matrices corresponding to adjacent frame sub-images to determine x difference matrices, obtains the pixel point regions corresponding to the non-zero elements from the x difference matrices and determines them as the sub-regions to be determined, traverses the n parts to obtain n determined sub-regions, connects the n determined sub-regions in the order from top to bottom to obtain the short video region, extracts the first sub-video of the first part among the n parts in the short video region, equally divides the first frame image of the first sub-video into 2 parts horizontally, forms the first sub-matrix and the second sub-matrix with the RGB values of the pixel points of the 2 parts respectively, calculates the difference between the first sub-matrix and the second sub-matrix to obtain the first difference sub-matrix, calculates the average value of all the element values of the first difference sub-matrix to obtain the first average value, traverses the x frames of the first sub-video to obtain the average values of the x frames of the first sub-video, calculates the variance of all the average values and the average value of all the average values, i.e., the total average value. If the variance is less than the variance threshold and the total average value is greater than the average threshold, it is determined that the short video has a picture-in-picture video; the n is an integer greater than or equal to 10; The splitting of the picture-in-picture video into 2 videos specifically includes: dividing the short video region of the picture-in-picture video into 2 equal parts horizontally, and each part corresponds to a single video.

2. An artificial intelligence-based content review system, the system includes: A communication unit for receiving the content to be reviewed, and the content is: a short video; A review unit for detecting the short video to determine whether it is a picture-in-picture video. If it is a picture-in-picture video, split the picture-in-picture video into 2 videos, and perform content review on each video as a single video; If it is a non-picture-in-picture video, perform content review on the short video as a single video; when it is determined that each video passes the review, it is determined that the content review of the picture-in-picture video passes. When it is determined that any one of the multiple videos fails the review, it is determined that the content review of the picture-in-picture video fails; The review unit is specifically configured to divide the short video into n equal parts in the vertical direction, extract x frame sub-images of the first equal part among the n equal parts, form x matrices with the RGB values of the pixel points of each frame sub-image of the x frame sub-images, calculate the differences between the matrices corresponding to adjacent frame sub-images to determine x difference matrices, obtain the pixel point regions corresponding to the non-zero elements from the x difference matrices and determine them as the sub-regions to be determined, traverse the n equal parts to obtain n determined sub-regions, connect the n determined sub-regions in the order from left to right to obtain the short video region, extract the first sub-video of the first equal part among the n equal parts in the short video region, divide the first frame image of the first sub-video into 2 equal parts in the vertical direction, form the first sub-matrix and the second sub-matrix with the RGB values of the pixel points of the 2 equal parts respectively, calculate the difference between the first sub-matrix and the second sub-matrix to obtain the first difference sub-matrix, calculate the average value of all the element values of the first difference sub-matrix to obtain the first average value, traverse the x frames of the first sub-video to obtain the average values of the x frames of the first sub-video, calculate the variance of all the average values of the first sub-video and the average value of all the average values, that is, the total average value. If the variance is less than the variance threshold and the total average value is greater than the average threshold, it is determined that the short video has a picture-in-picture video; the above n is an integer greater than or equal to 10; The review unit is specifically configured to divide the short video region of the picture-in-picture video into 2 equal parts in the vertical direction, and each equal part corresponds to a single video; The review unit is specifically configured to divide the short video into n equal parts in the horizontal direction, extract x frame sub-images of the first equal part among the n equal parts, form x matrices with the RGB values of the pixel points of each frame sub-image of the x frame sub-images, calculate the differences between the matrices corresponding to adjacent frame sub-images to determine x difference matrices, obtain the pixel point regions corresponding to the non-zero elements from the x difference matrices and determine them as the sub-regions to be determined, traverse the n equal parts to obtain n determined sub-regions, connect the n determined sub-regions in the order from top to bottom to obtain the short video region, extract the first sub-video of the first equal part among the n equal parts in the short video region, divide the first frame image of the first sub-video into 2 equal parts in the horizontal direction, form the first sub-matrix and the second sub-matrix with the RGB values of the pixel points of the 2 equal parts respectively, calculate the difference between the first sub-matrix and the second sub-matrix to obtain the first difference sub-matrix, calculate the average value of all the element values of the first difference sub-matrix to obtain the first average value, traverse the x frames of the first sub-video to obtain the average values of the x frames of the first sub-video, calculate the variance of all the average values of the first sub-video and the average value of all the average values, that is, the total average value. If the variance is less than the variance threshold and the total average value is greater than the average threshold, it is determined that the short video has a picture-in-picture video; the n is an integer greater than or equal to 10; The review unit is specifically configured to divide the short video region of the picture-in-picture video into 2 equal parts in the horizontal direction, and each equal part corresponds to a single video.

Images