A frame skipping detection model training method and device, equipment and readable storage medium

Abstract

The application provides a frame skipping detection model training method and device, equipment and a readable storage medium, relates to the technical field of computers, and comprises obtaining a face image data material set, preprocessing the face image data material set to obtain processed training data; based on a depth separable convolution, feature extraction is performed on the training data to obtain extracted feature data; according to a loss function, the feature data is calculated to obtain a frame skipping detection model. The beneficial effects of the application are that a lightweight depth convolutional neural network is designed to extract high-quality video frame features, and whether there is a large pre-set semantic change between two frames of a video is distinguished by using the same, thereby efficiently solving the problem caused by video splicing in automatic video editing. Through frame skipping detection, the automatic video editing software can more intelligently process the problem, thereby improving the user experience and satisfaction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a frame skipping detection model training method and device, equipment and readable storage medium. BACKGROUND

[0002] At present, small video has become an essential part of people's daily life, and automatic video editing has become a necessity for users. In automatic video editing, there are great problems in processing spliced videos. For example, for portrait beautification, there are several problems with the spliced video: there may be a large displacement of the portrait position between the front and back frames, which causes the portrait tracking algorithm to be abnormal; the portrait ID changes between the front and back frames (i.e., the front and back portraits do not belong to the same person), which causes some specific portrait effects to be incorrect.

[0003] To solve the above two main problems, a common approach is to perform face detection and face recognition on each frame of the video sequence, but this will result in a long video processing time, which users in the current mobile device scenario are unlikely to accept. Therefore, how to detect frame skipping in a video sequence has become an important problem. SUMMARY

[0004] The present application relates to the technical field of computer, in particular to a frame skipping detection model training method and device, equipment and readable storage medium.

[0005] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0006] obtaining a set of face image data materials, preprocessing the set of face image data materials to obtain processed training data;

[0007] performing feature extraction on the training data based on a depth separable convolution to obtain extracted feature data;

[0008] calculating the feature data according to a loss function to obtain a frame skipping detection model

[0009] Preferably, the set of face image data materials is obtained, and the set of face image data materials is preprocessed to obtain processed training data, which includes:

[0010] creating face image training data, the face image training data being generated in the form of a k-tuple;

[0011] adjusting the face image training data in brightness, contrast, scale and distance to obtain adjusted face image training data;

[0012] According to the similarity, the adjusted face image training data is recombined in pairs to obtain processed training data.

[0013] Preferably, the deep separable convolution is used to extract features from the training data to obtain extracted feature data, which includes:

[0014] Obtain the training data, and obtain a plurality of candidate video jump frames from the training data, wherein the video jump frame detection result corresponding to the training data is a video without abnormality.

[0015] Perform target object detection on each candidate video jump frame to obtain a target object region corresponding to each candidate video jump frame.

[0016] Based on the target object region, a video jump frame sequence is obtained, and the video jump frame sequence is screened to obtain a trained video jump frame sequence.

[0017] Based on the deep separable convolution, features are extracted from the video jump frame sequence to obtain the extracted feature data.

[0018] Preferably, the loss function is used to calculate the feature data to obtain a jump frame detection model, including:

[0019] Based on the loss function, a first data is obtained, which includes the weight of each gain intensity, the difference interval between two gain intensities, and the weight of feature difference.

[0020] Obtain a second data, which includes a preset probability that the i+1th frame and the 0th frame exist in the model prediction.

[0021] According to the first data and the second data, a jump frame detection model is calculated.

[0022] Based on the jump frame detection model and real-time network, parameter updating is performed until the convergence condition is met to obtain a target jump frame detection model.

[0023] In a second aspect, the application also provides a jump frame detection model training device, which comprises an acquisition module, an extraction module and a calculation module, wherein:

[0024] The acquisition module is used to acquire a face image data material set, and pre-process the face image data material set to obtain processed training data.

[0025] The extraction module is used to extract features from the training data based on a deep separable convolution to obtain extracted feature data.

[0026] The computing module is configured to calculate the feature data according to a loss function to obtain the frame skipping detection model.

[0027] Preferably, the obtaining module comprises a creating unit, an adjusting unit and a combining unit, wherein:

[0028] The creating unit is configured to create face image training data, which is generated in the form of a k-tuple.

[0029] The adjusting unit is configured to adjust the face image training data in brightness, contrast, scale and distance to obtain adjusted face image training data.

[0030] The combining unit is configured to recombine the adjusted face image training data in pairs according to similarity to obtain processed training data.

[0031] Preferably, the extracting module comprises a first obtaining unit, a detecting unit, a screening unit and an extracting unit, wherein:

[0032] The first obtaining unit is configured to obtain the training data and obtain a plurality of candidate video frames from the training data, wherein the video frame detection result corresponding to the training data is a video without abnormality.

[0033] The detecting unit is configured to detect target objects in each candidate video frame to obtain a target object region corresponding to each candidate video frame.

[0034] The screening unit is configured to obtain a video frame sequence based on the target object region and screen the video frame sequence to obtain a trained video frame sequence.

[0035] The extracting unit is configured to extract features of the video frame sequence based on the depth separable convolution to obtain extracted feature data.

[0036] Preferably, the computing module comprises a second obtaining unit, a third obtaining unit, a computing unit and an updating unit, wherein:

[0037] The second obtaining unit is configured to obtain first data based on a loss function, wherein the first data comprises a weight of each gain intensity, a difference interval between two gain intensities and a weight of feature difference.

[0038] The third obtaining unit is configured to obtain second data, wherein the second data comprises a preset probability that the i+1th frame and the 0th frame exist frame skipping in model prediction.

[0039] The computing unit is configured to calculate the frame skipping detection model according to the first data and the second data.

[0040] an updating unit configured to perform parameter updating based on the frame skipping detection model and real-time network until a convergence condition is met, to obtain a target frame skipping detection model.

[0041] In a third aspect, the present application also provides a frame skipping detection model training device, comprising:

[0042] a memory configured to store a computer program;

[0043] a processor configured to implement the steps of the frame skipping detection model training method when the computer program is executed.

[0044] In a fourth aspect, the present application also provides a readable storage medium having a computer program stored thereon, wherein the computer program is configured to implement the steps of the frame skipping detection model training method when executed by a processor.

[0045] The present application has the following beneficial effects: the present application extracts high-quality video frame features through a lightweight deep convolutional neural network, and uses the features to distinguish whether there is a large pre-set semantic change between two frames of a video, thereby efficiently solving the problem caused by video splicing in automated video editing. By detecting video frame skipping, the automated video editing software can more intelligently process the problem, thereby improving the user's experience and satisfaction.

[0046] Other features and advantages of the present application will be described in the following description, and some will become apparent from the description, or will be learned through implementation of the embodiments of the present application. The purposes and other advantages of the present application can be achieved and obtained through the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be considered as limiting the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0048] Figure 1 a frame skipping detection model training method flowchart described in the embodiments of the present application;

[0049] Figure 2 a frame skipping detection model training device structure diagram described in the embodiments of the present application;

[0050] Figure 3 a frame skipping detection model training device structure diagram described in the embodiments of the present application;

[0051] In the figure: 701, an acquisition module; 7011, a creation unit; 7012, an adjustment unit; 7013, a combination unit; 702, an extraction module; 7021, a first acquisition unit; 7022, a detection unit; 7023, a screening unit; 7024, an extraction unit; 703, a calculation module; 7031, a second acquisition unit; 7032, a third acquisition unit; 7033, a calculation unit; 7034, an update unit; 800, a frame skipping detection model training device; 801, a processor; 802, a memory; 803, a multimedia assembly; 804, an I / O interface; 805, a communication assembly. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings of the embodiments of the present application to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application but not all the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present application belong to the scope of protection of the present application.

[0053] It should be noted that: similar reference numbers and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, the terms “first”, “second”, etc. are only used for differentiation in description and cannot be understood as indicating or implying relative importance.

[0054] Embodiment 1

[0055] The embodiment provides a frame skipping detection model training method.

[0056] Referring to Figure 1 , the method includes steps S100, S200, and S300.

[0057] S100, acquiring a face image data material set, pre-processing the face image data material set to obtain processed training data.

[0058] It can be understood that the step S100 includes S101, S102, and S103, wherein:

[0059] S101, creating face image training data, the face image training data is generated in the form of a k-tuple;

[0060] It should be noted that in the training stage, firstly, we need to make training data, our training data is generated in the form of k-tuple, that is, (I0, I1, …, Ik-1) is generated as training data once. The k-tuple satisfies the following characteristics: diff (Il+1, I0)>diff (Il, I0), wherein diff is the difference degree of two images, l∈(1, 2, …, k-2).

[0061] S102, adjust the brightness, contrast, scale and distance of the face image training data to obtain the adjusted face image training data;

[0062] S103, recombine the adjusted face image training data according to the similarity to obtain the processed training data.

[0063] It should be noted that the deep convolutional neural network is designed, and generally, large-scale training data is needed to train the deep neural network to obtain a better feature representation, but the cost of labeling such data is high, therefore, starting from image data, relevant processing is performed to obtain different types and degrees of adjustment data:

[0064] 1. The face region is cut out from the face image data to form a material set Faces.

[0065] 2. Randomly select a picture I, and select a main face0 from Faces.

[0066] 2.1, paste face0 randomly in I to obtain I0

[0067] 2.2, randomly select one of the following operations, and perform k different scale gains to obtain I1, …, Ik

[0068] 2.2.1, brightness adjustment;

[0069] 2.2.2, contrast adjustment;

[0070] 2.2.3, scale adjustment of face in image;

[0071] 2.2.4, distance adjustment of face0 in image and I0;

[0072] 2.2.5, select k people face1, …, facek most similar to face0 from Faces according to the similarity;

[0073] 3. Output the image pair (I0, I1, …, Ik) as training data.

[0074] S200, performing feature extraction on the training data based on a depth separable convolution, to obtain extracted feature data.

[0075] It can be understood that the step S200 includes S201, S202, S203 and S204.

[0076] S201, obtaining the training data, and obtaining a plurality of candidate video jump frames from the training data, wherein the training data corresponds to a video jump frame detection result of no video anomaly.

[0077] Specifically, the computer device can obtain a training video, extract a plurality of video frames from the training video as candidate video jump frames, and obtain a plurality of candidate video jump frames. Each candidate video jump frame can be a continuous video jump frame or a non-continuous video jump frame. For example, the training video includes video jump frames 1-20, the computer device can extract video jump frames 5-10 from the training video as candidate video jump frames, and the computer device can extract odd frames or even frames from the training video as candidate video jump frames.

[0078] S202, performing target object detection on each of the candidate video jump frames to obtain a target object region corresponding to each of the candidate video jump frames.

[0079] S203, obtaining a video jump frame sequence based on the target object region, and screening the video jump frame sequence to obtain a trained video jump frame sequence.

[0080] The video jump frame sequence is a training sample corresponding to the video anomaly detection model. The video jump frame sequence is obtained by filtering and training the screening video frames from the initial video jump frame sequence, that is, the video jump frame sequence is an incomplete video frame sequence relative to the initial video jump frame sequence. The initial video jump frame sequence includes a plurality of video frames, and each video jump frame can be a continuous video jump frame, for example, the initial video jump frame sequence includes five continuous video jump frames, specifically video jump frames 1-5. Of course, each video jump frame can also be a non-continuous video jump frame, for example, the initial video jump frame sequence includes video jump frames 1, 3, 5, 7 and 10. It can be understood that each video jump frame in the initial video jump frame sequence can only include a foreground image region, or can include a foreground image region and a background image region. The foreground image region can specifically refer to an image region where the target object region is located.

[0081] S204, performing feature extraction on the video jump frame sequence based on the depth separable convolution, to obtain the extracted feature data.

[0082] It should be noted that in the model design part, the depth separable convolution is used to control the amount of calculation, and when training, the model width is expanded to enhance the ability of the model to learn features, and in the inference stage, the reparameterization technique is used to reduce the model calculation amount, so as to meet the real-time requirements of the mobile terminal.

[0083] S300, according to the loss function, the feature data is calculated, and the frame skipping detection model is obtained.

[0084] It can be understood that the step S300 includes S301, S302, S303 and S304.

[0085] S301, based on the loss function, obtaining first data, the first data includes the weight of each gain intensity, the difference interval between two gain intensities and the weight of feature difference degree;

[0086] S302, obtaining second data, the second data includes the probability of the existence of frame skipping between the i+1th frame and the 0th frame in the preset model prediction;

[0087] S303, according to the first data and the second data, the frame skipping detection model is calculated;

[0088] S304, based on the frame skipping detection model and real-time network, the parameter is updated until the convergence condition is met, and the target frame skipping detection model is obtained.

[0089] It should be noted that the input image pair (I0,…,Ik-1) is extracted by the network to obtain the feature (F0,…,F-1), and the Diff i (f(Ii,θ),f(I0,θ))(where i=0,…,k-1) of I0 and Ii is obtained, and the loss calculation is performed:

[0090]

[0091] Where Diff ij (f(I i ,θ),f(I j ,θ))=||f(I i ,θ)-f(I j ,θ)| p ,α o represents the weight of each gain intensity, m i represents the difference interval between two gain intensities; Pred(i+1,0) represents the probability of the existence of frame skipping between the i+1th frame and the 0th frame in the model prediction; βi is the weight of the feature difference degree; ||x||p represents the p norm distance, and the p norm distance is used to measure the feature difference degree in the patent.

[0092] In summary, a real-time frame skipping detection model on a mobile terminal can be obtained, and the unnatural problem of video automatic editing can be solved. After training, a model that can predict whether a video frame is skipped in real time is obtained.

[0093] It should be noted that in actual business, we add a video frame skipping detection model in the face tracking process to determine whether the current frame has a jump. If there is a jump, we reset the tracker and re-detect and track the object that is tracked incorrectly. In this way, users will not feel the error effect caused by incorrect tracking, and video automatic editing will be more natural.

[0094] Embodiment 2

[0095] As shown in Figure 2 The embodiment provides a frame skipping detection model training device, which is shown in Figure 2 The device comprises an acquisition module 701, an extraction module 702 and a calculation module 703, wherein:

[0096] The acquisition module 701 is configured to acquire a face image data material set, pre-process the face image data material set, and obtain processed training data.

[0097] The extraction module 702 is configured to perform feature extraction on the training data based on a deep separable convolution, and obtain extracted feature data.

[0098] The calculation module 703 is configured to calculate the feature data according to a loss function, and obtain a frame skipping detection model.

[0099] Specifically, the acquisition module 701 comprises a creating unit 7011, an adjusting unit 7012 and a combining unit 7013, wherein:

[0100] The creating unit 7011 is configured to create face image training data, and the face image training data is generated in the form of a k-tuple.

[0101] The adjusting unit 7012 is configured to adjust the brightness, contrast, scale and distance of the face image training data, and obtain adjusted face image training data.

[0102] The combining unit 7013 is configured to re-combine the adjusted face image training data in pairs according to the similarity, and obtain processed training data.

[0103] Specifically, the extraction module 702 comprises a first acquisition unit 7021, a detection unit 7022, a screening unit 7023 and an extraction unit 7024, wherein:

[0104] The first acquisition unit 7021 is configured to acquire the training data, and acquire a plurality of candidate video jump frames from the training data, wherein the training data corresponds to a video jump frame detection result of no video abnormality.

[0105] The detection unit 7022 is configured to perform target object detection on each of the candidate video jump frames to obtain a target object region corresponding to each of the candidate video jump frames.

[0106] The screening unit 7023 is configured to obtain a video jump frame sequence based on the target object region, and screen the video jump frame sequence to obtain a trained video jump frame sequence.

[0107] The extraction unit 7024 is configured to perform feature extraction on the video jump frame sequence based on the depth separable convolution to obtain extracted feature data.

[0108] Specifically, the calculation module 703 includes a second acquisition unit 7031, a third acquisition unit 7032, a calculation unit 7033, and an update unit 7034.

[0109] The second acquisition unit 7031 is configured to acquire first data based on a loss function, wherein the first data includes a weight of each gain intensity, a difference interval between two gain intensities, and a weight of a feature difference degree.

[0110] The third acquisition unit 7032 is configured to acquire second data, wherein the second data includes a preset probability that the i+1th frame and the 0th frame exist jump frames in model prediction.

[0111] The calculation unit 7033 is configured to calculate a jump frame detection model according to the first data and the second data.

[0112] The update unit 7034 is configured to perform parameter update based on the jump frame detection model and a real-time network until a convergence condition is met to obtain a target jump frame detection model.

[0113] It should be noted that, as to the apparatus in the above-described embodiments, the specific manners in which various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described here in detail.

[0114] Embodiment 3

[0115] Corresponding to the above method embodiments, the present embodiment also provides a jump frame detection model training device. The jump frame detection model training device described below can be mutually corresponding to the jump frame detection model training method described above.

[0116] Figure 3FIG. 8 is a block diagram of a frame skipping detection model training device 800 according to an exemplary embodiment. As shown in FIG. 8, the frame skipping detection model training device 800 can include a processor 801, a memory 802. The frame skipping detection model training device 800 can also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805. Figure 3

[0117] ​The processor 801 is configured to control overall operations of the frame skipping detection model training device 800 to complete all or part of the steps of the frame skipping detection model training method described above. The memory 802 is configured to store various types of data to support the operations of the frame skipping detection model training device 800. For example, the data can include instructions for any application or method operating on the frame skipping detection model training device 800, and application-related data, such as contact data, sent and received messages, pictures, audio, video, and the like. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk. The multimedia component 803 can include a screen and an audio component. The screen can be a touch screen, for example. The audio component is configured to output and / or input audio signals. For example, the audio component can include a microphone configured to receive external audio signals. The received audio signals can be further stored in the memory 802 or transmitted through the communication component 805. The audio component also includes at least one speaker configured to output audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules, which can be a keyboard, a mouse, a button, and the like. The buttons can be virtual buttons or physical buttons. The communication component 805 is configured to perform wired or wireless communication between the frame skipping detection model training device 800 and other devices. The wireless communication, such as Wi-Fi, Bluetooth, near field communication (NFC), 2G, 3G, or 4G, or a combination of one or more of them, so the corresponding communication component 805 can include a Wi-Fi module, a Bluetooth module, and an NFC module.

[0118] In an example embodiment, the frame skipping detection model training device 800 can be implemented by one or more Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor or other electronic elements for performing the frame skipping detection model training method described above.

[0119] In another example embodiment, a computer readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the frame skipping detection model training method described above. For example, the computer readable storage medium can be the memory 802 described above including program instructions, which can be executed by the processor 801 of the frame skipping detection model training device 800 to complete the frame skipping detection model training method described above.

[0120] Embodiment 4:

[0121] Corresponding to the method embodiments above, in this embodiment, a readable storage medium is also provided, which can be referred to each other below and above in the description of the frame skipping detection model training method.

[0122] A readable storage medium, on which a computer program is stored, the computer program, when executed by a processor, implements the steps of the frame skipping detection model training method of the method embodiments described above.

[0123] The readable storage medium can be specifically a U disk, a mobile hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various readable storage media that can store program codes.

[0124] In summary, automated video editing can greatly improve the editing efficiency of users, but the existing automated video has many problems in splicing video scenes, resulting in inconsistent editing effects with user expectations. The present application helps the automated video editing software to more intelligently process this problem by detecting video frame skipping, thereby improving the user experience and satisfaction.

[0125] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0126] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for training a frame skip detection model, characterized in that, include: Obtain a set of face image data materials, preprocess the set of face image data materials, and obtain processed training data; Based on depthwise separable convolution, feature extraction is performed on the training data to obtain the extracted feature data. Based on the loss function, the feature data is calculated to obtain the frame skipping detection model; The step of calculating the frame skip detection model based on the feature data according to the loss function includes: Based on the loss function, first data is obtained, which includes the weights of each gain intensity, the difference interval between two gain intensities, and the weights of feature difference. Obtain second data, which includes the probability of frame skipping between frame i+1 and frame 0 in the preset model prediction; A frame skipping detection model is calculated based on the first and second data. Based on the frame skipping detection model and the real-time network, the parameters are updated until the convergence condition is met, thus obtaining the target frame skipping detection model.

2. The frame skipping detection model training method according to claim 1, characterized in that... The process involves acquiring a set of facial image data, preprocessing the set of facial image data to obtain processed training data, including: Create face image training data, which is generated in the form of k-tuples; The brightness, contrast, scale, and distance of the face image training data are adjusted to obtain the adjusted face image training data. Based on the similarity difference, the adjusted face image training data is recombined in pairs to obtain the processed training data.

3. The frame skipping detection model training method according to claim 1, characterized in that... The feature extraction based on depthwise separable convolution is performed on the training data to obtain extracted feature data, including: The training data is obtained, and multiple candidate video skip frames are obtained from the training data. The video skip frame detection result corresponding to the training data is that the video has no abnormalities. Target object detection is performed on each of the candidate video skip frames to obtain the target object region corresponding to each candidate video skip frame; Based on the target object region, a video frame skipping sequence is obtained, and the video frame skipping sequence is filtered to obtain the trained video frame skipping sequence. Based on the depthwise separable convolution, feature extraction is performed on the video frame skipping sequence to obtain the extracted feature data.

4. A frame skip detection model training device, characterized in that, include: Acquisition module: used to acquire a set of face image data materials, preprocess the set of face image data materials, and obtain processed training data; Extraction module: used to extract features from the training data based on depthwise separable convolution to obtain extracted feature data; Calculation module: used to calculate the feature data according to the loss function to obtain the frame skipping detection model; The computing module includes: The second acquisition unit is used to acquire first data based on the loss function. The first data includes the weights of each gain intensity, the difference interval between two gain intensities, and the weights of feature difference. The third acquisition unit is used to acquire the second data, which includes the frame skipping probability between frame_i and frame_0 in the preset model prediction. Calculation unit: used to calculate the frame skipping detection model based on the first data and the second data; Update unit: used to update parameters based on the frame skip detection model and the real-time network until the convergence condition is met, and obtain the target frame skip detection model.

5. The frame skipping detection model training device according to claim 4, characterized in that, The acquisition module includes: Creation unit: used to create face image training data, which is generated in the form of k-tuples; Adjustment unit: used to adjust the brightness, contrast, scale and distance of the face image training data to obtain the adjusted face image training data; Combination unit: used to sort the adjusted face image training data according to the degree of difference, so as to obtain the processed training data.

6. The frame skipping detection model training device according to claim 4, characterized in that, The extraction module includes: First acquisition unit: used to acquire the training data, acquire multiple candidate video skip frames from the training data, and the video skip frame detection result corresponding to the training data is that the video has no abnormalities; Detection unit: used to perform target object detection on each of the candidate video skip frames to obtain the target object region corresponding to each candidate video skip frame; Filtering unit: used to obtain a video skipping frame sequence based on the target object region, and to filter the video skipping frame sequence to obtain the trained video skipping frame sequence; Extraction unit: used to extract features from the video frame skip sequence based on the depthwise separable convolution, and obtain the extracted feature data.

7. A frame skip detection model training device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the frame skip detection model training method as described in any one of claims 1 to 3 when executing the computer program.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program that, when executed by a processor, implements the steps of the frame skip detection model training method as described in any one of claims 1 to 3.

Images