Method and system for detecting digital human rendered videos

Abstract

This application proposes a method and system for detecting digital human rendered videos. The method includes: receiving a digital human rendered video to be detected and converting it into a frame format; asynchronously performing various types of anomaly detection on the converted digital human rendered video, including: anomaly detection based on deep learning and convolutional neural networks, boundary anomaly detection based on adjacent image comparison, image segmentation anomaly detection, and video stuttering anomaly detection; obtaining the detection result for each anomaly detection, multiplying the detection score in each detection result by the corresponding weight to obtain the final target score of the digital human rendered video; determining whether the target score is less than a preset score threshold, and re-rendering the digital human rendered video if the target score is less than the score threshold. This method performs multi-layer image quality detection on computer-rendered digital human videos, which can eliminate anomalies and ensure the quality of digital human rendered videos.

Description

Technical Field

[0001] This application relates to the field of digital human technology, and in particular to a method and system for detecting digital human rendered videos. Background Technology

[0002] Currently, the adoption rate of digital humans is gradually increasing, and they are widely used in various technological fields. Digital humans are a product of the integration of information science and life science. They utilize information science methods to virtually simulate the human body at different levels of form and function, achieving accurate simulation of the human body from the microscopic to the macroscopic level by establishing multi-layered digital models. By generating videos containing rendered digital humans, various functions such as providing information and voice interaction can be achieved. However, currently rendered digital human videos occasionally exhibit abnormal visual phenomena, therefore, anomaly detection is needed for the rendered digital human videos.

[0003] In related technologies, anomaly detection for digital human rendered videos is typically performed manually, with staff checking each generated video sequentially. However, this manual detection method is inefficient, costly, and consumes significant human resources. Furthermore, it is prone to missed detections and has a high error rate. Summary of the Invention

[0004] This application aims to at least partially address one of the technical problems in the related art.

[0005] Therefore, the first objective of this application is to propose a detection method for digital human rendered videos. This method performs image quality detection on the video stream / frames of a digital human image automatically rendered by a computer, solves the problem of abnormal images mixed in with the automatically rendered video, and significantly improves the quality and stability of the digital human rendering results.

[0006] The second objective of this application is to propose a detection system for digital human rendered videos.

[0007] The third objective of this application is to provide a non-transitory computer-readable storage medium.

[0008] To achieve the above objectives, a first aspect of this application proposes a method for detecting digital human rendered videos, comprising the following steps:

[0009] Receive the digital human rendered video to be detected, and convert the digital human rendered video into a frame format;

[0010] Multiple types of anomaly detection are asynchronously performed on the converted digital human rendering video. These anomaly detection types include: anomaly detection based on deep learning and convolutional neural networks, boundary anomaly detection based on adjacent image comparison, image segmentation anomaly detection, and video stuttering anomaly detection.

[0011] Obtain the detection result for each of the above anomalies, and multiply the detection score in each detection result by the corresponding weight to obtain the final target score of the digital human rendered video;

[0012] Determine whether the target score is less than a preset score threshold, and if the target score is less than the score threshold, re-render the digital human rendering video.

[0013] Optionally, in one embodiment of this application, before converting the digital human rendered video into a frame format, the method further includes: analyzing the digital human rendered video and configuring corresponding virtual memory for the detection task of the digital human rendered video based on the analysis results.

[0014] Optionally, in one embodiment of this application, anomaly detection based on deep learning and convolutional neural networks includes: obtaining a YOLOv5 network model pre-trained with a large number of labeled anomaly images; inputting all the frame images into the trained YOLOv5 network model, and obtaining the detection results for each frame image output by the YOLOv5 network model.

[0015] Optionally, in one embodiment of this application, image segmentation anomaly detection includes: segmenting each image frame according to different resolutions to obtain multiple target regions in each image frame, wherein the multiple target regions are the portion above the chest of the digital human excluding the face region; comparing adjacent slices to obtain the detection result of each image frame.

[0016] Optionally, in one embodiment of this application, before asynchronously performing multiple types of anomaly detection on the converted digital human rendered video, the method further includes: determining the type of the digital human rendered video based on the magnitude of the digital human's movements; and assigning the corresponding weight to each type of anomaly detection based on the type of the digital human rendered video.

[0017] Optionally, in one embodiment of this application, after re-rendering the digital human rendering video, the method further includes: detecting the re-rendered digital human rendering video repeatedly, and if the detection fails, recording the number of times the re-rendered digital human rendering video is re-rendered according to the video number; determining whether the number of re-rendered times is greater than a preset repetition threshold, and if the number of re-rendered times is greater than the repetition threshold, ending the detection process and sending the re-rendered digital human rendering video to the abnormal task labeling system.

[0018] Optionally, in one embodiment of this application, the detection result further includes: anomaly type, anomaly location coordinates, and anomaly orientation. The re-rendering of the digital human rendering video includes: re-rendering based on the detected anomaly type, anomaly location coordinates, and anomaly orientation with the goal of eliminating the anomaly.

[0019] To achieve the above objectives, a second aspect of this application provides a detection system for digital human rendered videos, comprising the following modules:

[0020] The conversion module is used to receive the digital human rendered video to be detected and convert the digital human rendered video into a frame format.

[0021] An anomaly detection module is used to asynchronously perform various types of anomaly detection on the converted digital human rendering video. The various types of anomaly detection include: anomaly detection based on deep learning and convolutional neural networks, boundary anomaly detection based on adjacent image comparison, image segmentation anomaly detection, and video stuttering anomaly detection.

[0022] The calculation module is used to obtain the detection result of each of the anomalies, and multiply the detection score in each detection result by the corresponding weight to obtain the final target score of the digital human rendered video.

[0023] The judgment module is used to determine whether the target score is less than a preset score threshold, and if the target score is less than the score threshold, the digital human rendering video is re-rendered.

[0024] Optionally, in one embodiment of this application, the system further includes a configuration module, specifically configured to: analyze the digital human rendering video and configure corresponding virtual memory for the detection task of the digital human rendering video based on the analysis results.

[0025] To implement the above embodiments, a third aspect of this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method for detecting digital human rendered video in the above embodiments.

[0026] The technical solution provided by the embodiments of this application brings at least the following beneficial effects: This application integrates multiple detection methods such as detection based on deep learning and convolutional neural network algorithms, image comparison detection, transparency channel detection, line detection, and segmentation detection. It combines the order of these detection methods and assigns corresponding weights to each method, thereby performing multi-level anomaly detection on digital human rendered videos. Through multiple detections and comprehensive processing of various detection results, it achieves more accurate detection of possible anomalies and stuttering in each frame of the digital human rendered video. It can be applied to detect various types of anomalies in digital human rendered videos, improving the accuracy of detection results, reducing the false recognition rate, and enriching the types of anomalies that can be accurately detected. Furthermore, videos with anomalies are re-rendered. Therefore, this application ensures the quality of the final output digital human rendered video and improves the detection efficiency and accuracy of abnormal video frames.

[0027] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0028] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein

[0029] Figure 1 This is a flowchart illustrating a method for detecting digital human rendered video according to an embodiment of this application;

[0030] Figure 2 This is a schematic diagram illustrating a specific frame segmentation method proposed in an embodiment of this application.

[0031] Figure 3 This is a schematic diagram illustrating a specific detection process for digital human rendered video according to an embodiment of this application;

[0032] Figure 4 This is a schematic diagram of the structure of a digital human rendering video detection system proposed in an embodiment of this application. Detailed Implementation

[0033] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0034] The following description, with reference to the accompanying drawings, illustrates a method and system for detecting digital human rendered videos according to embodiments of the present invention.

[0035] Figure 1 This is a flowchart of a method for detecting digital human rendered video according to an embodiment of this application, as shown below. Figure 1 The method includes the following steps:

[0036] Step S101: Receive the digital human rendering video to be detected and convert the digital human rendering video into a frame format.

[0037] The digital human rendering video to be detected is a video containing a digital human image that is automatically rendered by a computer. The digital human rendering video can be a video in various formats, such as a video stream or a frame-by-frame format.

[0038] Specifically, the system receives the video rendering results output by the computer's digital human rendering system and converts the acquired digital human rendering video to be detected into a unified frame format. That is, it converts the received video, which may be in video stream format or frame format, into frame format. The converted digital human rendering video consists of multiple consecutive image frames.

[0039] It should be noted that, in practical applications, in order to reduce the requirements on the system performing the video detection task, in one embodiment of this application, before converting the digital human rendered video into a frame format, the method further includes: analyzing the digital human rendered video and configuring corresponding virtual memory for the detection task of the digital human rendered video based on the analysis results.

[0040] Virtual memory is a computer system memory management technique that allows video detection programs to believe they have contiguous available memory. In this embodiment, the application performs pre-configuration before detection, allocating virtual memory for the video detection application to compensate for insufficient video memory when detecting digital human rendered videos on systems with low-configuration graphics cards.

[0041] In practice, when configuring virtual memory, the rendering video of the digital human to be detected is first analyzed. This includes analyzing the types of digital humans rendered in the video and estimating the memory required to execute the current detection task. Based on the analysis results, the corresponding virtual memory is configured for the current video task.

[0042] Understandably, the memory space required for different video detection tasks—for example, whether the digital human in the video to be detected is 2D, 3D, or hyper-realistic—will vary. To ensure that the virtual memory configured in this application can meet the needs of detecting different types of digital human rendered videos, this application configures different virtual memory spaces for different digital human rendered video detection tasks. Specifically, the virtual memory capacity corresponding to the current analysis result can be determined based on historical running data and expert knowledge.

[0043] Step S102 involves asynchronously performing various types of anomaly detection on the converted digital human rendering video. These anomaly detection methods include: anomaly detection based on deep learning and convolutional neural networks, boundary anomaly detection based on adjacent image comparison, image segmentation anomaly detection, and video stuttering anomaly detection.

[0044] Specifically, the digital human rendered video, converted to frame format, undergoes asynchronous anomaly detection using multiple methods. Detection focuses on defects and anomalies that may appear in each frame of the digital human rendered video, such as abnormal character design, abnormal hair, and unexpected straight lines, as well as stuttering issues between consecutive frames. The multi-layered anomaly detection in this application includes: anomaly detection based on deep learning and convolutional neural networks, boundary anomaly detection based on adjacent image comparison, image segmentation anomaly detection, and video stuttering anomaly detection. That is, this application includes anomaly point detection for each frame based on deep learning and convolutional neural networks, and detection of image differences based on image comparison between adjacent frames (including grayscale contrast and line detection).

[0045] In practical implementation, one possible approach is to asynchronously pass the rendered video of the digital human in frame format to four anomaly detection modules for asynchronous detection. The following provides a detailed explanation of these four anomaly detection methods.

[0046] As a first example, anomaly detection based on deep learning and convolutional neural networks includes: obtaining a YOLOv5 network model pre-trained with a large number of labeled anomaly images; inputting all the frame images into the completed YOLOv5 network model, and obtaining the detection results for each frame image output by the YOLOv5 network model.

[0047] In this example, the YOLOv5 network model is chosen as the neural network model for detection. YOLO is an object detection algorithm that redefines object detection as a regression problem, applying a single convolutional neural network (CNN) to the entire image, dividing the image into a grid, and predicting the class probability and bounding box for each grid. The YOLOv5 network is an object detection network based on the YOLO algorithm, characterized by its fast detection speed and relatively simple structure.

[0048] In this embodiment, the YOLOv5 network model can be trained in advance using a large number of labeled abnormal images during the offline stage. The specific training process can refer to the training methods in related technologies. For example, during the training process, the anchor boxes with initially set length and width are first obtained. The network outputs the predicted boxes based on the initial anchor boxes. The predicted boxes are then compared with the real boxes, the difference between the two is calculated, and then the network parameters are updated in reverse and iterated. These steps will not be elaborated here.

[0049] Furthermore, each frame of the digital human rendering video to be detected is input into the trained YOLOv5 network model for anomaly detection, and the detection results for each frame are obtained.

[0050] As a second example, we perform boundary anomaly detection based on comparison of adjacent images. In this example, boundary anomaly detection is performed by comparing adjacent image frames. In practice, boundary anomaly detection can be performed in various ways.

[0051] For example, alpha channel detection can be performed. The alpha channel represents the transparency information of pixels in a digital image. White alpha pixels define opaque colored pixels, while black alpha pixels define transparent pixels. Gray levels between black and white represent the semi-transparent parts of a color image. Since the same boundary may have the same transparency, boundary detection can be performed using the alpha channel. In this embodiment, frames with transparent backgrounds are input into a video frame comparison module to detect boundary anomalies between adjacent frames in the video.

[0052] For example, line detection can also be performed. In this example, various line detection algorithms can be used, such as the traditional Hough transform or the LSD fast line detection algorithm. Alternatively, deep learning algorithms can be used to detect lines, such as training a detection network like LCNN on a wireframe dataset and then using the trained line detection network for detection.

[0053] As a third example, image segmentation anomaly detection is performed. This detection includes segmenting each frame according to different resolutions to obtain multiple target regions in each frame. These target regions are the area above the chest of the digital human, excluding the face area. Adjacent slices are compared to obtain the detection results for each frame.

[0054] In this example, to avoid the influence of facial expressions and improve the appearance of enlarged images, the image frames are segmented according to different resolutions. The segmentation focuses on the area above the person's chest, excluding the face, and then adjacent slices are compared. For example... Figure 2 As shown, the image frame is segmented into target regions corresponding to three bounding boxes. This eliminates the influence of facial expressions on anomaly detection, focusing on the target regions within the three bounding boxes, specifically the hair of the person who is prone to problems. The pixels of different gray levels behind the digital human figure in the image represent transparency information. Adjacent image frames are then segmented in the same way, and the adjacent slices are compared to obtain the detection results. The segmentation is based on the resolution of each image frame, making the image in the segmented target regions easier to detect, and ensuring that the comparison of adjacent slices has the same basic conditions.

[0055] As a fourth example, video stuttering detection is performed. In this example, each frame is sent to the stuttering detection module, and the detection result for each frame is obtained by detecting the similarity between adjacent video frames and frames in between.

[0056] In practice, the comparison between adjacent video frames and frames in between includes detecting whether the similarity between frames is too high or too low. It can be understood that when the similarity is too high, it indicates that adjacent video frames may be duplicate images, thus indicating that the duplication is caused by video stuttering. When the similarity is too low, it indicates that video stuttering is causing missing frames between adjacent frames, resulting in discontinuous video content.

[0057] Step S103: Obtain the detection results for each type of anomaly detection, and multiply the detection score in each detection result by the corresponding weight to obtain the final target score of the digital human rendered video.

[0058] In this embodiment, the anomaly detection result includes anomaly detection score, anomaly type, anomaly location coordinates, and anomaly orientation. After each anomaly detection, the anomaly detection module outputs the detection result for that round, and the specific content of the detection result corresponds to the type of anomaly detection.

[0059] As an example, for anomaly detection based on deep learning and convolutional neural networks, the detection results include: the detection score for each frame, the anomaly type of the anomaly point, and the anomaly location coordinates. For boundary anomaly detection based on adjacent image comparison, the detection results include: the anomaly detection score for each frame, the anomaly type of the anomaly point, and the anomaly location coordinates. For image segmentation anomaly detection, the detection results include: the detection score for each frame, the anomaly type, and the anomaly orientation, where the anomaly orientation corresponds to the corresponding segmented target region, such as... Figure 2 The example shown only provides three directions: top left, top, and top right, but not specific coordinates. For video stuttering anomaly detection, the results include: the detection score and anomaly type for each frame.

[0060] For example, anomaly detection based on deep learning and convolutional neural networks can detect abnormal pixel locations, including those that failed to render, those with abnormal colors, and those where rendered content appears in unexpected locations. This anomaly detection has high accuracy and can detect things like differences between the ends of a person's hair. Boundary anomaly detection based on adjacent image comparison can detect anomalies such as unexpected straight lines appearing in a frame. For example, due to the need for coordinate axes during rendering or rendering failures, the rendered video may contain straight lines resembling coordinate axes or occlusion areas composed of straight lines. Boundary anomaly detection can detect these lines that should not appear. For image segmentation anomaly detection, anomaly types include detecting anomalies in a person's hairstyle within the target area.

[0061] Furthermore, it is understandable that, due to the different detection accuracies and target anomaly types of the aforementioned anomaly detection methods, the reliability of the detection results varies when different digital human rendering videos are input in practical applications. Consequently, the weight of each detection result also differs when generating the final detection result. Therefore, to further improve the accuracy and precision of the generated final detection result, this application multiplies the detection score of each detection result by its corresponding weight to obtain the final target score of the digital human rendering video.

[0062] The weight of each detection result can be set according to actual needs. As one possible implementation, before asynchronously performing various types of anomaly detection on the converted digital human rendered video, the type of digital human rendered video is determined based on the range of the digital human's movements, and then a corresponding weight is assigned to each type of anomaly detection based on the type of digital human rendered video.

[0063] Specifically, the range of motion of the digital human in the rendered video affects several detection methods. For example, the digital human in the rendered video might be performing small movements like lecturing or explaining at the front of the stage, or larger movements like dancing. When the digital human's movements are large, it significantly impacts the detection methods for comparing adjacent frames. In such cases, the accuracy and reliability of boundary anomaly detection and image segmentation anomaly detection based on adjacent image comparison are low. Therefore, this embodiment determines the video type as large or small movements based on the range of motion of the digital human, and then adjusts the weights corresponding to each type of anomaly detection based on the video type.

[0064] For example, since anomaly detection based on deep learning and convolutional neural networks is largely unaffected by the digital human's movements, when the digital human's movements are large, a higher weight is assigned to anomaly detection based on deep learning and convolutional neural networks, while the weights of boundary anomaly detection and image segmentation anomaly detection based on adjacent image comparison are correspondingly reduced. Conversely, when the digital human's movements in the current input video are small, the weight of adjacent image comparison detection is adaptively increased.

[0065] Step S104: Determine whether the target score is less than a preset score threshold. If the target score is less than the score threshold, re-render the digital human rendering video.

[0066] The preset score threshold was obtained through extensive data testing before detection, indicating that the rendering result of the digital human rendered video is a normal threshold.

[0067] Specifically, if the final detected target score of the current digital human rendered video is greater than the score threshold, it indicates that the digital human rendered video is a normal image; otherwise, it is an image with anomalies. If the target score is less than the score threshold, the digital human rendered video is re-rendered.

[0068] In one embodiment of this application, the abnormal digital human rendering view is returned to the digital human rendering system for re-rendering. During re-rendering, the re-rendering is performed with the goal of eliminating the abnormality, based on the detected abnormality type, abnormal location coordinates, and abnormal orientation. For example, based on the coordinates of an abnormal point that has not been rendered with color, the abnormal point is re-rendered so that the expected color is rendered at that coordinate point.

[0069] Furthermore, in one embodiment of this application, after re-rendering the digital human rendering video, the method further includes: detecting the re-rendered digital human rendering video repeatedly, and if the detection fails, recording the number of times the re-rendered digital human rendering video has been re-rendered according to the video label. Then, it is determined whether the number of re-rendered times exceeds a preset repetition threshold. If the number of re-rendered times exceeds the repetition threshold, the detection process ends and the re-rendered digital human rendering video is sent to the abnormal task labeling system.

[0070] Specifically, based on video ID and other labeling information, the number of times a video is re-rendered is counted. If the number of retries exceeds a threshold, the retries are skipped, the video is released directly, and sent to the anomaly task annotation system for annotation by annotators and for collecting training materials for the model. This also includes analyzing the causes and manifestations of the anomaly in the video to adjust the rendering parameters of the rendering system. Furthermore, based on manually annotated cases with detection errors, the parameters of the anomaly detection model based on deep learning and convolutional neural networks are adjusted to improve the detection accuracy of the model by combining actual detection data.

[0071] It should be noted that, in one embodiment of this application, to save computer storage resources, the virtual memory configured for this task can be deleted after the current digital human rendering video detection task is completed, thereby saving storage resources for other tasks in the system. The corresponding virtual memory is reconfigured before each video detection task is executed.

[0072] In summary, the digital human rendering video detection method of this application integrates multiple detection methods, including detection based on deep learning and convolutional neural network algorithms, image comparison detection, alpha channel detection, line detection, and segmentation detection. It combines these methods in a specific order and assigns corresponding weights to each method, thereby performing multi-level anomaly detection on the digital human rendering video. Through multiple detections and comprehensive processing of various detection results, this method achieves more accurate detection of potential anomalies and stuttering in each frame of the digital human rendering video. It is applicable to detecting various types of anomalies in digital human rendering videos, improving the accuracy of detection results, reducing the false recognition rate, and enriching the types of anomalies that can be accurately detected. Furthermore, videos with anomalies are re-rendered. Therefore, this method ensures the quality of the final output digital human rendering video and improves the efficiency and accuracy of detecting abnormal video frames.

[0073] Based on the above embodiments, in order to more clearly describe the specific processing flow of the digital human rendered video detection method of this application, a specific detection flow is also proposed in one embodiment of this application. Figure 3This diagram illustrates a specific detection process for digital human rendered video according to an embodiment of this application. The steps in this detection process, from receiving the video to be detected in the digital human rendering system to obtaining the detection result, are as follows: Figure 3 As shown, it will not be elaborated further here.

[0074] For example, based on the detection process of this digital human rendered video, taking the business scenario of video output detection as an example, after rendering a video of a digital human saying "hello", the detection process is as follows:

[0075] First, the system input is a video with no background (the background is the alpha channel). Upon receiving the video, the system begins splitting it into frames and asynchronously inputting these frames to four detection modules. Each module provides its own detection score, anomaly type for outlier points, and other information. The system then performs a weighted evaluation of the returned results to determine whether the video is acceptable. If acceptable, it is output normally; otherwise, it needs to be sent to the rendering system for regeneration, with a regeneration count kept. For videos that still fail after exceeding the retry count, an annotation task is sent to the annotation system for annotation personnel to collect materials for model training.

[0076] It should be noted that the specific implementation of each step in this embodiment can be referred to the relevant description in the above embodiment, and will not be repeated here.

[0077] To achieve the above embodiments, this application also proposes a detection system for digital human rendered videos. Figure 4 This is a schematic diagram of the structure of a digital human rendering video detection system proposed in an embodiment of this application.

[0078] like Figure 4 As shown, the system includes a conversion module 100, an anomaly detection module 200, a calculation module 300, and a judgment module 400.

[0079] The conversion module 100 is used to receive the digital human rendering video to be detected and convert the digital human rendering video into a frame format.

[0080] The anomaly detection module 200 is used to asynchronously perform various types of anomaly detection on the converted digital human rendering video. These anomaly detection types include: anomaly detection based on deep learning and convolutional neural networks, boundary anomaly detection based on comparison of adjacent images, image segmentation anomaly detection, and video stuttering anomaly detection.

[0081] The calculation module 300 is used to obtain the detection results of each anomaly detection, and multiply the detection score in each detection result by the corresponding weight to obtain the final target score of the digital human-rendered video.

[0082] The judgment module 400 is used to determine whether the target score is less than a preset score threshold. If the target score is less than the score threshold, the digital human rendering video is re-rendered.

[0083] Optionally, in one embodiment of this application, the system further includes a configuration module, which is specifically used to: analyze the digital human rendering video and configure corresponding virtual memory for the detection task of the digital human rendering video based on the analysis results.

[0084] Optionally, in one embodiment of this application, the anomaly detection module 200 is specifically used to: obtain a YOLOv5 network model that has been pre-trained using a large number of labeled anomaly images; input all the frame images into the trained YOLOv5 network model, and obtain the detection results for each frame image output by the YOLOv5 network model.

[0085] Optionally, in one embodiment of this application, the anomaly detection module 200 is specifically used to: segment each frame according to different resolutions, obtain multiple target regions in each frame, wherein the multiple target regions are the part above the chest of the digital human and excluding the face region; compare adjacent slices, and obtain the detection result of each frame.

[0086] Optionally, in one embodiment of this application, the calculation module 300 is further configured to: determine the type of digital human rendered video based on the range of motion of the digital human; and assign corresponding weights to each type of anomaly detection based on the type of digital human rendered video.

[0087] Optionally, in one embodiment of this application, the judgment module 400 is further configured to: detect the re-rendered digital human rendering video repeatedly, and if the detection fails, record the number of times the re-rendered digital human rendering video is re-rendered according to the video number; determine whether the number of re-rendered times is greater than a preset re-rendered times threshold, and if the number of re-rendered times is greater than the re-rendered times threshold, end the detection process and send the re-rendered digital human rendering video to the abnormal task labeling system.

[0088] Optionally, in one embodiment of this application, the detection result further includes: anomaly type, anomaly location coordinates, and anomaly orientation. The judgment module 400 is further configured to: re-render based on the detected anomaly type, anomaly location coordinates, and anomaly orientation with the goal of eliminating the anomaly.

[0089] It should be noted that the description of the aforementioned embodiment of the detection method for digital human rendered video is also applicable to the system of this embodiment, and the implementation principle is the same, so it will not be repeated here.

[0090] In summary, the digital human rendering video detection system of this application integrates multiple detection methods, including detection based on deep learning and convolutional neural network algorithms, image comparison detection, alpha channel detection, line detection, and segmentation detection. It combines these methods in a specific order and assigns corresponding weights to each method, thereby performing multi-level anomaly detection on the digital human rendering video. Through multiple detection processes and comprehensive processing of various detection results, the system achieves more accurate detection of potential anomalies and stuttering in each frame of the digital human rendering video. It is applicable to detecting various types of anomalies in digital human rendering videos, improving the accuracy of detection results, reducing the false recognition rate, and enriching the types of anomalies that can be accurately detected. Furthermore, videos with anomalies are re-rendered. Therefore, this system ensures the quality of the final output digital human rendering video and improves the efficiency and accuracy of detecting abnormal video frames.

[0091] To implement the above embodiments, the present invention also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method for detecting digital human rendered video as described in the first aspect of the present application.

[0092] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the use of illustrative expressions for the above terms in multiple embodiments or examples does not imply that these embodiments or examples are identical. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0093] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0094] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0095] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0096] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0097] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0098] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0099] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for detecting digital human rendered videos, characterized in that, Includes the following steps: Receive the digital human rendered video to be detected, and convert the digital human rendered video into a frame format; Multiple types of anomaly detection are asynchronously performed on the converted digital human rendering video. These anomaly detection types include: anomaly detection based on deep learning and convolutional neural networks, boundary anomaly detection based on adjacent image comparison, image segmentation anomaly detection, and video stuttering anomaly detection. Obtain the detection result for each of the above anomalies, and multiply the detection score in each detection result by the corresponding weight to obtain the final target score of the digital human rendered video; Determine whether the target score is less than a preset score threshold, and if the target score is less than the score threshold, re-render the digital human rendering video; The image segmentation anomaly detection includes: Each frame is segmented according to different resolutions to obtain multiple target regions in each frame. The multiple target regions are the part above the chest of the digital human, excluding the face area. The adjacent slices are compared to obtain the detection results for each frame of the image.

2. The method for detecting digital human rendered video according to claim 1, characterized in that, Before converting the digital human rendered video into a frame format, the method further includes: Analyze the digital human rendered video and configure corresponding virtual memory for the detection task of the digital human rendered video based on the analysis results.

3. The method for detecting digital human rendered video according to claim 1 or 2, characterized in that, The anomaly detection based on deep learning and convolutional neural networks includes: Obtain a YOLOv5 network model that has been pre-trained using a large number of labeled anomalous images; All the frame images are input into the trained YOLOv5 network model to obtain the detection results of each frame image output by the YOLOv5 network model.

4. The method for detecting digital human rendered video according to claim 1, characterized in that, Prior to asynchronously performing various types of anomaly detection on the converted digital human rendered video, the method also includes: The type of video rendered by the digital human is determined based on the range of motion of the digital human. The corresponding weight is assigned to each type of anomaly detection based on the type of the digital human rendered video.

5. The method for detecting digital human rendered video according to claim 1, characterized in that, After re-rendering the digital human rendered video, the process further includes: The re-rendered digital human rendering video is repeatedly detected, and if the detection fails, the number of times the re-rendered digital human rendering video is repeatedly rendered is recorded according to the video number; Determine whether the number of repeated renderings is greater than a preset repeat count threshold. If the number of repeated renderings is greater than the repeat count threshold, end the detection process and send the re-rendered digital human rendering video to the abnormal task labeling system.

6. The method for detecting digital human rendered video according to claim 1, characterized in that, The detection results also include: anomaly type, anomaly location coordinates, and anomaly orientation. The re-rendering of the digital human rendered video includes: Based on the detected anomaly type, anomaly location coordinates, and anomaly orientation, a re-render is performed with the goal of eliminating the anomaly.

7. A detection system for digital human rendered video, characterized in that, Includes the following modules: The conversion module is used to receive the digital human rendered video to be detected and convert the digital human rendered video into a frame format. An anomaly detection module is used to asynchronously perform various types of anomaly detection on the converted digital human rendered video. The various types of anomaly detection include: anomaly detection based on deep learning and convolutional neural networks, boundary anomaly detection based on adjacent image comparison, image segmentation anomaly detection, and video stuttering anomaly detection. The calculation module is used to obtain the detection result of each of the anomalies, and multiply the detection score in each detection result by the corresponding weight to obtain the final target score of the digital human rendered video. The judgment module is used to determine whether the target score is less than a preset score threshold, and if the target score is less than the score threshold, to re-render the digital human rendering video; The image segmentation anomaly detection includes: Each frame is segmented according to different resolutions to obtain multiple target regions in each frame. The multiple target regions are the part above the chest of the digital human, excluding the face area. The adjacent slices are compared to obtain the detection results for each frame of the image.

8. The detection system for digital human rendered video according to claim 7, characterized in that, Also includes: The configuration module is specifically used for: Analyze the digital human rendered video and configure the corresponding virtual memory for the detection task of the digital human rendered video based on the analysis results.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for detecting digital human rendered videos as described in any one of claims 1-6.

Images