Detection method and device and storage medium

Abstract

The invention discloses a detection method and device and a storage medium, and the method comprises the steps: obtaining video data comprising a to-be-detected target, and determining an image set comprising a preset frame number of video images; inputting the image set into at least one first model, and determining a preset feature sequence corresponding to each first model; and determining a detection result of the to-be-detected target by adopting a second model according to the at least one preset feature sequence. Therefore, the to-be-detected target can be comprehensively detected through the first model and the second model, and the accuracy of the detection result is improved.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a detection method, device and storage medium. Background Art

[0002] With the continuous development of technology, the complexity of images is getting higher and higher. Traditional detection methods can no longer meet the increasingly complex image detection needs, resulting in insufficient accuracy of existing image detection results. Therefore, how to improve the detection accuracy of the detection method is a problem to be solved. Summary of the invention

[0003] In order to solve the existing technical problems, the embodiments of the present invention provide a detection method, a device and a storage medium.

[0004] The technical solution of the present invention is achieved in this way:

[0005] An embodiment of the present invention provides a detection method, the method comprising:

[0006] Acquire video data including the target to be detected, and determine an image set including video images with a preset number of frames;

[0007] Inputting the image set into at least one first model, and determining a preset feature sequence corresponding to each of the first models;

[0008] According to the at least one preset feature sequence, a second model is used to determine a detection result of the target to be detected.

[0009] In the above scheme, the preset feature sequence includes at least one of the following: a face feature sequence, a background environment feature sequence, an illegal item feature sequence, a light and dark change feature sequence, an eye feature sequence, an action feature sequence and a sound feature sequence.

[0010] In the above solution, the step of inputting the image set into at least one first model and determining a preset feature sequence corresponding to each of the first models comprises:

[0011] Determining, according to the third model, characteristic images of the targets to be detected corresponding to the video images respectively;

[0012] The characteristic image of the target to be detected is input into the first model to determine the light-dark change characteristic sequence characterizing the light-dark change of the target to be detected.

[0013] In the above solution, the step of inputting the characteristic image of the target to be detected into the first model to determine the light-dark change characteristic sequence characterizing the light-dark change of the target to be detected includes:

[0014] Using the first model, performing preset processing on the feature image of the target to be detected to determine a brightness image of a preset number of frames;

[0015] Determining a brightness change gradient according to time based on the brightness image of the preset number of frames;

[0016] The brightness change feature sequence is determined according to the brightness change gradient.

[0017] In the above solution, the step of determining the detection result of the target to be detected by using the second model according to the at least one preset feature sequence includes:

[0018] Using the second model, determining the probability weight corresponding to each of the preset feature sequences;

[0019] According to the probability weight, a detection result of the target to be detected is determined.

[0020] In the above solution, determining the detection result of the target to be detected according to the probability weight includes:

[0021] According to the probability weight, a binary classification task is performed to determine the detection result of the target to be detected.

[0022] In the above scheme, the method further comprises:

[0023] According to the image set including the preset number of frame images, using a preset detection model, determining whether the target to be detected is a predetermined target;

[0024] The step of acquiring video data of the target to be detected and determining an image set including video images of a preset number of frames includes:

[0025] The target to be detected is determined to be a predetermined target, video data including the target to be detected is acquired, and an image set including video images with a preset number of frames is determined.

[0026] The embodiment of the present invention provides a detection device, which includes: a first determination module, a second determination module and a third determination module; wherein:

[0027] The first determination module is used to obtain video data including the target to be detected and determine an image set including video images with a preset number of frames;

[0028] The second determination module is used to input the image set into at least one first model to determine a preset feature sequence corresponding to each of the first models;

[0029] The third determination module is used to determine the detection result of the target to be detected using the second model according to the at least one preset feature sequence.

[0030] In the above scheme, the preset feature sequence includes at least one of the following: a face feature sequence, a background environment feature sequence, an illegal item feature sequence, a light and dark change feature sequence, an eye feature sequence, an action feature sequence and a sound feature sequence.

[0031] In the above solution, the second determination module is used to determine the characteristic images of the target to be detected corresponding to the video images respectively according to the third model;

[0032] The characteristic image of the target to be detected is input into the first model to determine the light-dark change characteristic sequence characterizing the light-dark change of the target to be detected.

[0033] In the above solution, the second determination module is used to use the first model to perform preset processing on the feature image of the target to be detected to determine a brightness image of a preset number of frames;

[0034] Determining a brightness change gradient according to time based on the brightness image of the preset number of frames;

[0035] The brightness change feature sequence is determined according to the brightness change gradient.

[0036] In the above solution, the third determination module is used to use the second model to determine the probability weight corresponding to each of the preset feature sequences;

[0037] According to the probability weight, a detection result of the target to be detected is determined.

[0038] In the above solution, the third determination module is used to perform a binary classification task according to the probability weight to determine the detection result of the target to be detected.

[0039] In the above solution, the device further includes: a fourth determining module; wherein,

[0040] The fourth determination module is used to determine whether the target to be detected is a predetermined target by using a preset detection model according to the image set including the preset number of frames;

[0041] The second determination module is used to determine that the target to be detected is a predetermined target, obtain video data including the target to be detected, and determine an image set including video images with a preset number of frames.

[0042] In a third aspect, an embodiment of the present invention further provides a detection device, comprising a memory, a processor, and an executable program stored in the memory and capable of being run by the processor, wherein the processor executes the steps of any one of the detection methods when running the executable program.

[0043] In a fourth aspect, an embodiment of the present invention further provides a storage medium having an executable program stored thereon, wherein the executable program implements the steps of any one of the detection methods when executed by a processor.

[0044] The detection method, device and storage medium provided in the embodiments of the present invention obtain video data including the target to be detected, determine an image set including a preset number of video frames; input the image set into at least one first model, determine a preset feature sequence corresponding to each of the first models; and use a second model to determine the detection result of the target to be detected based on the at least one preset feature sequence. In the scheme of the embodiments of the present invention, the accuracy of the detection result can be improved by performing comprehensive detection of the target to be detected by the first model and the second model. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of a detection method provided by an embodiment of the present invention;

[0046] Figure 2 A video image schematic diagram of a detection method provided by an application embodiment of the present invention

[0047] Figure 3 A second model schematic diagram of a detection method provided by an embodiment of the present invention;

[0048] Figure 4 A schematic flow chart of a detection method provided by an application embodiment of the present invention;

[0049] Figure 5 A schematic diagram of the structure of a detection device provided by an embodiment of the present invention;

[0050] Figure 6 A schematic structural diagram of another detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] With the rapid development of Internet technology, online video examinations have become a new type of examination method that is not restricted by time and space, greatly facilitating candidates and educational institutions. However, this examination method also brings serious cheating problems, such as using mobile phones to assist cheating tools and consult materials. Traditional anti-cheating technology cannot effectively deal with these problems, so more advanced technical means are needed. The current cheating detection method is usually: a method based on single-frame image single model processing, that is, using image processing technology and deep learning to analyze and judge the information on the candidate's screen, using a single model to analyze the image, and dividing the image into non-cheating and several other corresponding cheating categories and outputting the results.

[0052] However, the above method has the following problems:

[0053] First, using a single model for cheating detection is easily interfered by non-factual factors such as environment and lighting, making this detection method unable to adapt to different scenarios and needs.

[0054] Second, the test results obtained by using a single-frame image as the test material are only instantaneous results, and cheating behavior usually lasts for a period of time, so a single-frame image cannot accurately represent the behavior over a period of time. For example, a squint in a very short period of time, or an instantaneous change in the background caused by a person passing behind the examinee in a very short period of time, cannot be truly counted as cheating behavior. Therefore, the method of detecting cheating by analyzing abnormal results from a single-frame image has low accuracy and poor robustness.

[0055] To solve at least one of the above problems, various embodiments of the present invention provide a detection method, including: acquiring video data including a target to be detected, and determining an image set containing video images with a preset number of frames; inputting the image set into at least one first model, and determining a preset feature sequence corresponding to each of the first models; and using a second model to determine a detection result of the target to be detected based on the at least one preset feature sequence.

[0056] The present invention will be further described in detail below in conjunction with the embodiments.

[0057] Figure 1 A flow chart of a detection method provided by an embodiment of the present invention; Figure 1 As shown, the method includes:

[0058] Step 101, obtaining video data including a target to be detected, and determining an image set including video images with a preset number of frames;

[0059] Step 102: input the image set into at least one first model, and determine a preset feature sequence corresponding to each of the first models;

[0060] Step 103: According to the at least one preset feature sequence, a second model is used to determine a detection result of the target to be detected.

[0061] In some possible implementations, the target to be detected may include an examinee, and the video data containing the target to be detected may be video picture data containing the examinee.

[0062] Furthermore, the acquiring of video data including the target to be detected may include:

[0063] The camera of the examinee's mobile terminal is called in real time to obtain the corresponding video data of the examinee who is taking the exam.

[0064] Here, the mobile terminal may include but is not limited to: a computer, a mobile phone, and a tablet computer.

[0065] In a possible implementation, the preset number of frames may be m frames, where m is a preset frame value and may be set according to actual needs, and is not specifically limited here.

[0066] Determining an image set including video images with a preset number of frames may include:

[0067] The video data is subjected to frame extraction processing at a preset time interval, and m frames of video images are extracted as an image set; the preset time interval can also be set according to actual needs, and is not specifically limited here.

[0068] Furthermore, the live sound in the m frames of video images can be synchronously collected.

[0069] In a possible implementation, the first model may be used to extract a preset feature sequence from m frames of video images.

[0070] Here, a feature may refer to an attribute associated with a detection result in a video image. For example, a feature may include but is not limited to: actions, expressions, and object categories. A preset feature sequence may refer to a data structure determined by sorting features in a preset manner. For example, the form of a preset feature sequence may include but is not limited to: an array or a list.

[0071] Furthermore, at least one preset feature sequence may be used to indicate at least one cheating method. For example, the preset feature sequence is an action feature sequence of the target to be detected, which may be used to indicate that the target to be detected is cheating by flipping through a book.

[0072] In a possible implementation, the second model can be used to perform a comprehensive detection on at least one preset feature sequence determined by the first model to determine a detection result of the target to be detected.

[0073] Here, the detection results of different targets to be detected by at least one first model may refer to whether the examinee has cheated within the time of m frames; comprehensive detection may refer to calculating the cheating method represented by at least one preset feature sequence through the second model, determining the comprehensive cheating detection results of the targets to be detected, and comprehensively determining whether the examinee has cheated.

[0074] Specifically, the camera of the mobile terminal used by the examinee to take the online test is called in real time to obtain the examinee's video data, and the video data is subjected to preset frame extraction processing to determine an image set containing m frames of video images; the image set is input into a first model to determine at least one preset feature sequence corresponding to each frame of the m frames of video images; each preset feature sequence is input into a second model to determine whether the examinee cheats within the time of m frames.

[0075] In a possible implementation manner, at least one image set including video images with a preset number of frames may be determined.

[0076] Exemplarily, a set of k images including m frames of video images is determined, and the first model and the second model are used to determine whether cheating occurs in the test within the time of the k m frames of video images, thereby improving the robustness of the cheating detection correlation time.

[0077] Through the above method, firstly, by using at least one first model and a second model to determine the cheating detection result of the target to be detected, the application scope of the detection method can be increased and the accuracy of the detection result can be improved; secondly, by performing cheating detection through an image set of video images with a preset frame number, the detection result has time correlation, which can effectively analyze the cheating behavior within a period of time, thereby improving the accuracy of detection.

[0078] In some embodiments, the preset feature sequence includes at least one of the following: a facial feature sequence, a background environment feature sequence, an illegal item feature sequence, a light and dark change feature sequence, an eye feature sequence, an action feature sequence, and a sound feature sequence.

[0079] In one possible implementation, by inputting an image set into at least one first model, a facial feature sequence, a background environment feature sequence, a violation item feature sequence, a light and dark change feature sequence, an eye feature sequence, an action feature sequence, and a sound feature sequence can be determined by the first model.

[0080] In one possible implementation, the first model may include but is not limited to: a facial feature extraction model, a background environment feature extraction model, an illegal item feature extraction model, a light and dark change feature extraction model, an eye feature extraction model, an action feature extraction model, and a sound feature extraction model.

[0081] Among them, the face feature extraction model can be used to extract the facial feature values ​​of the target to be detected in the video image, the background environment feature extraction model can be used to extract the background environment feature values ​​in the video image, the illegal item feature extraction model can be used to extract the number and category of illegal items in the video image, the light and dark change feature extraction model can be used to extract the light and dark change gradient of the face and body part of the target to be detected in the video image, the eyeball feature extraction model can be used to extract the eyeball movement direction characteristics of the target to be detected in the video image, the action feature extraction model can be used to extract the action and posture characteristics of the target to be detected in the video image, and the sound feature extraction model can be used to extract the audio features in the video image. Specifically, the face feature sequence is determined by the face feature extraction model, the background environment feature sequence is determined by the background environment feature extraction model, the illegal item feature sequence is determined by the illegal item feature extraction model, the light and dark change feature sequence is determined by the light and dark change feature extraction model, the eyeball feature sequence is determined by the eyeball feature extraction model, the action feature sequence is determined by the action feature extraction model, and the sound feature sequence is determined by the sound feature extraction model.

[0082] In a possible implementation, the first model may be a facial feature extraction model, and the preset feature sequence may be a facial feature sequence; and determining the facial feature sequence by using the facial feature extraction model may include:

[0083] Determine that the face recognition model is a face feature extraction model, wherein the face recognition model is used to determine whether each frame of the m frames of video images is the person of the target to be detected;

[0084] If it is determined that the nth video image in the m-frame video image is the person to be detected, the corresponding sequence value of the frame video image is 0; if it is determined that the nth video image in the m-frame video image is not the person to be detected, the corresponding sequence value of the frame video image is 1;

[0085] The sequence values ​​corresponding to each frame of the m-frame video image are sorted in time order to determine the facial feature sequence of the m-frame video image.

[0086] In a possible implementation, the first model may be a background environment feature extraction model, and the preset feature sequence may be a background environment feature sequence; and determining the background environment feature sequence by using the background environment feature extraction model may include:

[0087] Determine a preset image segmentation model, perform image segmentation on each of the m frames of video images, and determine the video image of the background part and the video image of the target part to be detected;

[0088] A background environment feature extraction model is determined, and a video image of a background portion is input into the background environment feature extraction model to determine a background environment feature sequence.

[0089] Here, the preset image segmentation model can be used to detect the background environment part and the person part in the image, and segment the image into a video image of the background part and a video image of the target part to be detected.

[0090] The background part of the video image may refer to the background environment image behind the target to be detected; the target part of the video image may refer to the person image that only includes the target to be detected.

[0091] For example, Figure 2 As shown, Figure 2 A video image schematic diagram of a detection method provided by an application embodiment of the present invention; the image includes two parts: a target part to be detected and a background part; a preset image segmentation model can be Figure 2 The image shown is segmented to determine the character image and the background environment image.

[0092] In a possible implementation, the preset image segmentation model may be trained in a first predetermined manner by determining training samples.

[0093] Exemplarily, the method of determining training samples may include: determining historical video data, extracting at least one frame of video image from the historical video data, and segmenting the characters and environmental background in the video image; determining historical character images and historical environmental background images, and using the historical character images and historical environmental background images as training samples; training a preset image segmentation model in a first predetermined manner may refer to inputting the training samples into a u-net-based segmentation model to obtain a preset image segmentation model.

[0094] Here, the n-net-based segmentation model may refer to a convolutional neural network model for semantic segmentation, which performs accurate segmentation by determining the category of each pixel in the image; by training the u-net-based segmentation model as a preset image segmentation model, the model may be trained based on a small number of training samples, and by classifying each pixel in the video image, a higher segmentation accuracy may be obtained.

[0095] In a possible implementation, determining a background environment feature extraction model, inputting a video image of a background portion into the background environment feature extraction model, and determining a background environment feature sequence may include:

[0096] Determine a training sample, and train the background environment feature extraction model in a second predetermined manner;

[0097] The background part of the video image is input into the background environment feature extraction model, the score corresponding to each frame of the video image is determined, and the scores corresponding to each frame of the background part of the video image are sorted in chronological order as the background environment feature sequence.

[0098] Exemplarily, a method for determining training samples may include: scoring the historical environment background image corresponding to the historical video data according to the degree of compliance with preset regulations, and using the historical environment background image and the corresponding scoring value as training samples; training the background environment feature extraction model in a second predetermined manner may refer to: inputting the historical environment background image into a convolutional neural network model for training to determine the background environment feature extraction model.

[0099] Here, the preset regulations may include but are not limited to: the ambient background brightness is greater than the preset brightness threshold, and the number of people in the ambient background is less than the preset number threshold; the preset brightness threshold and the preset number threshold can be set according to specific needs.

[0100] The historical environment background image is scored according to the degree to which it complies with preset regulations. The score is proportional to the degree of compliance with the regulations. The score range can be within the preset range of 0 to 1, and no specific limitation is made here in actual application.

[0101] Furthermore, the background environment feature extraction model may refer to a background environment scoring model, and the background environment feature may refer to a scoring value of the degree to which the background environment image of the target to be detected complies with preset regulations.

[0102] In a possible implementation, determining the illegal item feature sequence by using the illegal item feature extraction model may include:

[0103] Determine a training sample and train a model for extracting features of illegal items in a third predetermined manner;

[0104] Input m frames of video images into the illegal item feature extraction model, and output the prohibited item location and prohibited item category table as the illegal item feature sequence.

[0105] Exemplarily, the method of determining training samples may include: determining pictures containing prohibited items, marking the locations of prohibited items and the categories of prohibited items in the pictures as labels; training a feature extraction model for prohibited items in a third predetermined manner may include: inputting pictures and labels into a network based on yolov5 for training, and determining a feature extraction model for prohibited items.

[0106] Here, prohibited items may refer to items that should not appear in the examination scene, such as mobile phones, notes, books, materials, etc.

[0107] Furthermore, yolov5 can be a target detection and recognition model that can identify the location and category of training objects in images. By using yolov5 to train a feature extraction model for illegal objects, the speed and accuracy of illegal object recognition can be improved.

[0108] In a possible implementation, determining the eye feature sequence by using an eye feature extraction model may include:

[0109] Extract m+1 frames of video images from the video data, input the m+1 frames of video images into the face feature extraction model, and determine the eyeball position of the target to be detected;

[0110] According to the m+1 eyeball positions, m vectors are determined as eyeball feature sequences.

[0111] Specifically, the eyeball feature extraction model may refer to a face feature extraction model, which obtains the eyeball position of each frame in the m+1 frames of video images by detecting the facial key points of the target to be detected; for the m+1 eyeball positions, starting from the first position i=1, the eyeball vector obtained by subtracting the i-th position from the i+1-th position is calculated in sequence, and the m+1 eyeball positions can calculate and determine m eyeball vectors, and the m eyeball vectors are used as the eyeball feature sequence.

[0112] Here, the eye vector can represent the eye movement direction and distance of the target to be detected. By determining the eye vector of the target to be detected as the eye feature sequence, it can be detected whether the target to be detected has a large range of eye changes during the examination.

[0113] In a possible implementation, determining the action feature sequence by using the action feature extraction model may include:

[0114] Extract m frames of video images from the video data, input the m frames of video images into the motion feature extraction model, determine the position list of each human skeleton key point in each frame of video image, and use the m human skeleton key point position list as the motion feature sequence.

[0115] Here, the motion feature extraction model may refer to a pose estimation model, such as an openpose model, a densepose model, an AlphaPose model, and the like.

[0116] In a possible implementation, determining the sound feature sequence by using a sound feature extraction model may include:

[0117] While extracting m frames of video images from the video data, the sound clips of the target to be detected within the m frames are obtained, and the sound feature sequence is determined according to the sound clips.

[0118] Specifically, a sound segment within m frame time is converted into an audio signal, m audio segments of length n are uniformly determined in the audio signal as sampling points, Mel frequency cepstrum coefficients (MFCC) are extracted from the sampling points, and an MFCC feature sequence is determined as a sound feature sequence.

[0119] Here, if the target to be detected speaks during the test, that is, there are sound clips, then the MFCC feature extraction can retain the semantically relevant information of the target to be detected and filter out irrelevant information such as background noise; the MFCC feature extraction process may include but is not limited to: preprocessing, fast Fourier transform, Mei filter group, logarithmic operation, discrete cosine transform, dynamic feature extraction and other steps; using MFCC feature extraction to determine the sound feature sequence can save equipment costs, and there is no special requirement for the sound input device, which can be completed through the computer of the target to be detected.

[0120] Through the above method, by using at least one first model to analyze and judge various aspects of information of the target to be detected, at least one preset feature sequence is determined. On the one hand, the accuracy and robustness of cheating behavior judgment are improved, and the analysis dimension of cheating detection is increased, thereby improving the detection effect of online examinations. On the other hand, relatively professional models are used to extract features for different cheating methods, further improving the accuracy of cheating detection.

[0121] In some embodiments, inputting the image set into at least one first model and determining a preset feature sequence corresponding to each of the first models comprises:

[0122] Determining, according to the third model, characteristic images of the targets to be detected corresponding to the video images respectively;

[0123] The characteristic image of the target to be detected is input into the first model to determine the light-dark change characteristic sequence characterizing the light-dark change of the target to be detected.

[0124] In a possible implementation, the third model may refer to a preset image segmentation model, and determining the characteristic images of the targets to be detected corresponding to the video images may refer to segmenting the video images through the preset image segmentation model to determine the video images of the target parts to be detected.

[0125] In a possible implementation, the first model may refer to a light-dark change feature extraction model.

[0126] Specifically, the video image of the examinee's part is input into the light and dark change feature extraction model to determine the light and dark change feature sequence representing the light and dark changes of the target to be detected.

[0127] Through the above method, a preset image segmentation model is used to segment the video image, determine the characteristic image of the target part to be detected in the video image, and determine the light and dark change characteristic sequence based on the characteristic image, which can reduce the error value when detecting the light and dark changes of the target to be detected itself and improve the accuracy of cheating detection.

[0128] In some embodiments, inputting the feature image of the target to be detected into the first model to determine the light-dark change feature sequence characterizing the light-dark change of the target to be detected includes:

[0129] Using the first model, performing preset processing on the feature image of the target to be detected to determine a brightness image of a preset number of frames;

[0130] Determining a brightness change gradient according to time based on the brightness image of the preset number of frames;

[0131] The brightness change feature sequence is determined according to the brightness change gradient.

[0132] In a possible implementation, the preset processing may refer to a hue saturation value (HSV) transformation, and the HSV transformation may refer to converting an image from a red green blue (RGB) model to an HSV model, thereby making the image more consistent with the intuitive vision of the human eye.

[0133] Here, by performing HSV transformation on the feature image of the target to be detected, the brightness, hue and vividness of the color can be expressed intuitively, which facilitates the comparison between colors and thus determines the brightness image.

[0134] In a possible implementation, brightness may refer to the lightness and darkness distribution of a color.

[0135] Specifically, a brightness change feature extraction model is used to perform HSV transformation on the video image of the examinee's part, determine the brightness image of m frames, determine the brightness change gradient between the current frame and the previous frame in the brightness image of m frames frame by frame, and determine the brightness change gradient matrix as the brightness change feature sequence.

[0136] Through the above method, the light and dark change feature sequence corresponding to the target to be detected is determined according to the light and dark change feature extraction model. According to the light and dark changes of the target to be detected itself, cheating behaviors related to electronic screens with obscure movements of candidates (such as the mobile phone screen is outside the camera) can be effectively detected, further improving the analysis dimension of cheating detection.

[0137] In some embodiments, the determining the detection result of the target to be detected by using the second model according to the at least one preset feature sequence includes:

[0138] Using the second model, determining the probability weight corresponding to each of the preset feature sequences;

[0139] According to the probability weight, a detection result of the target to be detected is determined.

[0140] In a possible implementation, the probability weight may refer to an intermediate result determined by inputting each preset feature sequence into the second model.

[0141] Here, the intermediate result can be an intermediate variable relative to the detection result output by the second model, and the probability weight, as an intermediate variable, participates in the calculation of the final result.

[0142] Specifically, determine the second model, input the facial feature sequence, background environment feature sequence, illegal item feature sequence, light and dark change feature sequence, eye feature sequence, action feature sequence and sound feature sequence into the second model, determine the intermediate results corresponding to each preset feature sequence in the second model, and determine the cheating detection result of the examinee based on the intermediate results.

[0143] In one possible implementation, historical video data is determined, k m-frame video images are extracted from the historical video data, and facial feature sequences, background environment feature sequences, illegal item feature sequences, light and dark change feature sequences, eye feature sequences, motion feature sequences, and sound feature sequences corresponding to each of the k m-frame video images are determined respectively.

[0144] Mark k m-frame video images to determine whether there is cheating behavior in each m-frame video image; if there is cheating behavior in the m-frame video image, mark the m-frame video image as 1; if there is no cheating behavior in the m-frame video image, mark the m-frame video image as 0.

[0145] In a possible implementation, k preset feature sequences are preprocessed and used as training data to determine the second model.

[0146] Exemplarily, preprocessing may refer to: setting a preset feature sequence of a non-single input as a single input, and not processing the preset feature sequence of the single input.

[0147] For example, the preset feature sequence is converted from a variable-length list to a fixed-length list, and the preset feature sequence in the fixed-length list is not processed.

[0148] A variable-length list may refer to a list whose length and elements are not fixed, and a fixed-length list may refer to a list whose length and elements are fixed.

[0149] Since the number and category of prohibited items appearing in each frame of video image may be different, the prohibited item feature sequence is a variable length list; the method of setting the prohibited item feature sequence of the variable length list as a single input may include: setting the maximum number of detected prohibited items to p, if the length of the prohibited item feature sequence is greater than p (that is, the number of detected prohibited items is greater than p), only taking the first p elements in the prohibited item feature sequence, if the length of the prohibited item feature sequence is less than p, setting the positions less than p in the prohibited item feature sequence to 0, so that the length of the prohibited item feature sequence becomes a fixed length p; each element in the prohibited item feature sequence is input into a dense convolutional neural network and the output number is set to 1, so that the prohibited item feature sequence becomes a single input.

[0150] Here, each element in the prohibited item feature sequence is a single input, so the prohibited item feature sequence as a whole is also a single input; a single input may refer to an input value whose quantity as the second model input is 1, and a non-single input may refer to an input value whose quantity as the second model input is not 1.

[0151] By using a dense convolutional neural network, the computational complexity of the second model can be reduced and the detection performance can be improved.

[0152] Furthermore, since the number and category of actions of the target to be detected in each frame of video image may be different, the action feature sequence is also a variable-length list; the method of setting the action feature sequence of the variable-length list as a single input may include: setting the maximum number of human skeleton key points to q, if the length of the action feature sequence is greater than q, only taking the first q elements in the action feature sequence, if the length of the action feature sequence is less than q, setting the positions less than q in the action feature sequence to 0, so that the length of the action feature sequence becomes a fixed length q; each element in the action feature sequence is input into a dense convolutional neural network and the output number is set to 1, so that the action feature sequence becomes a single input.

[0153] Furthermore, the matrix size of the brightness change gradient matrix as the brightness change feature sequence can be x*y, and after preprocessing, the output size of the brightness change gradient matrix is ​​set to 1*1, thereby setting the non-single input brightness change feature sequence to a single input.

[0154] In a possible implementation, the facial feature sequence, background environment feature sequence, eye feature sequence and voice feature sequence may be fixed-length lists, and the facial feature sequence, background environment feature sequence, eye feature sequence and voice feature sequence are not processed.

[0155] In a possible implementation, the fixed-length lists corresponding to the seven preprocessed preset feature sequences may be normalized.

[0156] Exemplarily, the data sizes in the fixed-length lists corresponding to the seven preset feature sequences are processed, and the size intervals of all data are set to be between 0 and 1, which can avoid the problem of errors caused by excessively large values ​​in the data.

[0157] Here, the size range of the data can be set according to actual needs and is not specifically limited here.

[0158] In a possible implementation, the fixed-length list corresponding to the 7 preset feature sequences after normalization is evaluated in time series, and each evaluation can determine a 7-dimensional vector; m 7-dimensional vectors are used as training data for training in time series to determine the second model.

[0159] Exemplarily, the second model may be a long short-term memory (LSTM) model.

[0160] Here, the LSTM model is a neural network that has the ability to memorize long-term and short-term information. The use of the LSTM model can solve the problem of gradient disappearance during training of the recurrent neural network (RNN), which makes the model no longer converge.

[0161] Through the above method, the preset feature sequence is preprocessed and normalized, and the preprocessed preset feature sequence is used as training data to determine the second model, so that the second model can comprehensively detect whether the exam is cheating from different cheating methods, effectively improving the accuracy of cheating detection.

[0162] In some embodiments, determining the detection result of the target to be detected according to the probability weight includes:

[0163] According to the probability weight, a binary classification task is performed to determine the detection result of the target to be detected.

[0164] In one possible implementation, Figure 3 As shown, Figure 3 A schematic diagram of a second model of a detection method provided for an embodiment of the present invention; wherein, the number of neurons in the fully connected layer in the second model is the same as the number of preset feature sequences, and the number of neurons in the output layer is 1.

[0165] For example, Figure 3 As shown, the number of preset feature sequences is 7, so a fully connected layer of 7 neurons and an output layer of 1 neuron can be set in the second model.

[0166] Among them, each fully connected layer of 7 neurons outputs an intermediate result corresponding to a preset feature sequence; the output layer of 1 neuron performs a binary classification task on the 7 intermediate results corresponding to the 7 fully connected layers, and the output value can be 0 or 1, 1 can indicate that the candidate has cheated, and 0 can indicate that the candidate has not cheated.

[0167] Here, the fully connected layer may refer to a common layer type in a neural network, which can play a role in classification; the output layer may refer to the last layer of neurons in a neural network model, which is used to provide output values; a binary classification task may refer to a classification task with two type labels, and the two type labels may refer to cheating and no cheating.

[0168] Further, Figure 3 The input corresponding to the fully connected layer can be input[0] to input[m-1], where input[0] can be a 7-dimensional vector corresponding to the 7 preset feature sequences of the first frame of the m-frame video image, and the m-frame video image can correspond to m 7-dimensional vectors.

[0169] In a possible implementation, the 7 intermediate results output by the 7 fully connected layers may correspond to the cheating probabilities of the 7 cheating behaviors.

[0170] For example, if the intermediate result of the fully connected layer output corresponding to the eye feature sequence is 0.8, it can be represented that there is an 80% probability that the examinee cheated by peeking; if the intermediate result of the fully connected layer output corresponding to the prohibited item feature sequence is 0.1, it can be represented that there is a 10% probability that the examinee cheated by using prohibited items.

[0171] Through the above method, by setting at least one fully connected layer and an output layer in the second model, the fully connected layer outputs the intermediate result, and the output layer performs a binary classification task to output whether the examinee cheats. At least one fully connected layer analyzes and judges the overall scenario of almost all behaviors and information features of the examinee over a period of time, thereby improving the accuracy and robustness of anti-cheating and solving the problem of some cheating behaviors being misjudged.

[0172] In some embodiments, the method further comprises:

[0173] According to the image set including the preset number of frame images, using a preset detection model, determining whether the target to be detected is a predetermined target;

[0174] The step of acquiring video data of the target to be detected and determining an image set including video images of a preset number of frames includes:

[0175] The target to be detected is determined to be a predetermined target, video data including the target to be detected is acquired, and an image set including video images with a preset number of frames is determined.

[0176] In a possible implementation, the preset detection model may refer to a living face detection model, and the preset target may refer to a real person.

[0177] Specifically, a live face detection model is determined, and the live face detection model is used to detect whether the examinee is a real person; if the examinee is a real person, the first model and the second model are used to determine whether the examinee cheats; if the examinee is not a real person, it is directly determined that the examinee cheats.

[0178] Exemplarily, live face training data is determined to train a live face detection model; the candidate's registration information is input into the live face detection model to determine whether the candidate is a real person. If the candidate is a real person, True can be output, and if not, False can be output.

[0179] Here, the living face detection model can be an LSTM model.

[0180] The method provided by the embodiment of the present invention extracts m frames of video images from video data of a candidate taking an exam, uses a first model and a second model to detect the m frames of video images, determines whether the candidate cheats and the cheating probabilities corresponding to different cheating methods, can perform comprehensive detection on multiple frames of video images through multiple models, and analyze and judge the overall scene by comprehensively analyzing the characteristics of almost all behaviors and information of the candidate within a period of time, which can improve the accuracy and robustness of cheating detection; uses light and dark change detection as one of the comprehensive detections, can effectively detect the overall scene analysis and judgment by comprehensively analyzing the characteristics of almost all behaviors and information of the candidate within a period of time, and further improves the analysis dimension of cheating detection.

[0181] Figure 3 A video image schematic diagram of a detection method provided by an application embodiment of the present invention; Figure 3 As shown, the image includes: a video image of the target part to be detected and a video image of the background part.

[0182] Figure 4 A flow chart of a detection method provided by an application embodiment of the present invention; Figure 4 As shown, the method includes:

[0183] Step 401: extract the video image of the examinee.

[0184] Here, the camera of the examinee's mobile terminal is called to obtain the video data corresponding to the examinee taking the exam, and m frames of video images are extracted from the video data.

[0185] Step 402: Determine whether the examinee is a real person; if the examinee is a real person, execute step 403; if the examinee is not a real person, execute step 405.

[0186] Here, a live face detection model is trained, and the candidate’s face registration information is input to determine whether the candidate is a real person.

[0187] Step 403: Determine a plurality of preset feature sequences of the video image.

[0188] Here, the facial feature sequence can be determined by the facial feature extraction model, the background environment feature sequence can be determined by the background environment feature extraction model, the illegal item feature sequence can be determined by the illegal item feature extraction model, the light and dark change feature sequence can be determined by the light and dark change feature extraction model, the eye feature sequence can be determined by the eye feature extraction model, the action feature sequence can be determined by the action feature extraction model, and the sound feature sequence can be determined by the sound feature extraction model.

[0189] Step 404: Determine the test result of the examinee.

[0190] Here, a second model can be determined, and the facial feature sequence, background environment feature sequence, illegal item feature sequence, light and dark change feature sequence, eye feature sequence, action feature sequence and sound feature sequence can be input into the second model to determine the intermediate results corresponding to each preset feature sequence in the second model, and the test results of the examinee can be determined based on the intermediate results.

[0191] Specifically, based on the probability weights, a binary classification task is performed to determine the test results of the examinees.

[0192] Step 405: Determine whether the examinee has cheated.

[0193] Here, when the live face detection model determines that the examinee in the video image is not a real person, it can be directly judged that the examinee is cheating.

[0194] The following provides multiple specific examples in combination with any of the above embodiments:

[0195] The main technical routes of the present invention are as follows:

[0196] It is divided into nine parts: video stream acquisition, identity authentication, environmental detection, illegal item detection, light and dark change detection, eye tracking, motion detection, speech detection, and comprehensive judgment mechanism.

[0197] 1. Video stream acquisition (equivalent to acquiring video data including the target to be detected).

[0198] The algorithm of this method pulls the video stream of the computer camera of the candidate who is currently taking the test in real time, and extracts frames from it at a certain interval (equivalent to determining an image set containing a preset number of video images), and then sends the extracted images (such as Figure 3The parameters m are set to the number of frames for subsequent joint processing of multiple frames, that is, m frames of images or the corresponding video segments of each frame in the m frames of images are input as a whole into the subsequent algorithm according to the time sequence to judge the cheating behavior for a whole period of time. At the same time, the continuous on-site sound in the video within the m-frame time is collected.

[0199] 2. Identity verification.

[0200] (1) Prepare a live face set, and use an LSTM-based neural network to train the live faces in each time series in the face set to obtain a live face detection network for the time series. At the same time, prepare a face recognition model and register the candidate information into the model (equivalent to determining the live face detection model).

[0201] (2) Every time m frames are extracted from the video stream, the m frames are sent to the face recognition model in sequence to determine whether each frame is of the candidate. If it is not the candidate himself, 0 is output, and if it is the candidate himself, 1 is output. The result is then converted into a sequence face_list of length m in chronological order (equivalent to determining the face feature sequence through the face feature extraction model).

[0202] (3) Send the m frames of images into the live face detection network in time series to detect whether the current m frames are alive, and output the result as the live variable. If it is, output True, otherwise output False (equivalent to inputting the candidate's registration information into the live face detection model to determine whether the candidate is a real person).

[0203] 3. Background environment detection.

[0204] The specific training and use process of background environment detection is as follows:

[0205] (1) Prepare the video of the video test, extract a large number of pictures from them, manually segment and mark the people and the background environment, and input them as training samples into the u-net-based image segmentation model for training to obtain an image segmentation model that can segment the people and the background environment into two parts (equivalent to determining the preset image segmentation model, performing image segmentation on each frame of the m frames of video images, and determining the video image of the background part and the video image of the target part to be detected).

[0206] (2) The different background environment images after segmentation are scored according to their compliance with the regulations, with scores ranging from 0 to 1. The images are then input into a convolutional neural network based on CNN for training to obtain a background environment scoring model (equivalent to a training background environment feature extraction model).

[0207] (3) Every time m frames are extracted from the video stream, the m frames are first sent to the image segmentation model in sequence, and then the segmented background environment images are sent to the environment scoring model to obtain the score of each frame, and then the score sequence environment_list of the background environment compliance degree of the current m frames is output (equivalent to determining the background environment feature sequence through the background environment feature extraction model).

[0208] 4. Detection of illegal items.

[0209] During the exam, cheating usually occurs when using or looking at prohibited items for a long time. However, it is possible that prohibited items (such as mobile phones) flash in front of the camera. This situation does not necessarily constitute cheating. Therefore, we first obtain the list of prohibited items for each frame and then proceed to the subsequent steps. The specific acquisition process is as follows:

[0210] (1) Prepare pictures containing prohibited items (such as mobile phones, notes, tablets, books), and mark the types and locations of various prohibited items as labels. Then input the pictures and labels into a network based on yolov5 (the yolov5 series model is a target detection and recognition model that can identify the location and category of training objects in the image) for training to obtain a prohibited item detection and recognition model (i.e., a training illegal item feature extraction model).

[0211] (2) Every time m frames are extracted from the video stream, the m frames are sent to the prohibited item detection and recognition model in sequence to obtain the location and category list of prohibited items in each frame as obj, and then the obj of m frames is output to obj_list in time series (equivalent to determining the feature sequence of illegal items through the illegal item feature extraction model).

[0212] 5. Detection of light and dark changes.

[0213] Usually, during the video test, candidates may cheat by switching the test interface (such as switching to a search engine), using tablets or mobile phones, etc. Sometimes these methods will not be captured by the camera, but in the process of switching screens and turning the mobile phone (tablet) on and off, it can be found by observing frame by frame that the part of the face and part of the body facing the screen will have obvious changes in brightness, while the overall environment brightness usually does not change. Therefore, estimating the brightness changes of the face and part of the body facing the screen is an effective cheating detection method. The specific process is as follows:

[0214] Every time m frames are extracted from the video stream, the m frames are first sent to the image segmentation model (i.e. the preset image segmentation model) in sequence. After segmentation, the human part is subjected to HSV transformation to obtain the brightness image. Then, for each pixel of the brightness image, the brightness change gradient of the human part for the previous frame is calculated frame by frame to obtain the brightness gradient matrix (size is x*y) light; the light of m frames is output to light_list according to the time series (equivalent to determining the brightness change feature sequence through the brightness change feature extraction model).

[0215] 6. Eye tracking

[0216] During the test, the normal direction of the examinee's eyes should be within the screen range, and the overall direction should not change much. If the eye direction deviates greatly or changes frequently and over a large range, it is possible that the examinee has repeatedly looked at areas outside the screen, which may be cheating. Therefore, it is necessary to collect the changes in the eye movement direction. The specific process is as follows:

[0217] (1) Every time m+1 frames are extracted from the video stream, the m+1 frames are sequentially sent to the face recognition model (i.e., face feature extraction model). Usually, the face recognition model includes a face key point detection function, through which the accurate positions of the two eyeballs in the current frame image can be obtained.

[0218] (2) For the m+1 positions obtained in the previous step, we start from the first position i=1 and calculate the vector v obtained by subtracting the i-th position from the i+1-th position. This vector represents the direction and distance of the eye movement from the i-th frame to the i+1-th frame. A total of m similar vectors eye can be obtained from the m+1 positions. The m similar vectors eye are output to eye_list in sequence (equivalent to determining the eye feature sequence through the eye feature extraction model).

[0219] 7. Motion detection.

[0220] During the exam, cheating behavior can also be judged from the movements and postures of the head, upper torso, and hands. Therefore, human postures also need to be collected. The specific process is as follows:

[0221] Every time m frames are extracted from the video stream, the m frames are sent to openpose or a similar posture estimation model in sequence to obtain the pose list of the positions of the key points of the human skeleton in each frame. This list is taken as a whole to obtain a total of m poses, and the m poses are output to pose_list in sequence (equivalent to determining the action feature sequence through the action feature extraction model).

[0222] 8. Speech detection.

[0223] Judging from the sounds collected during the test, it is also possible to determine whether the sounds have human voices or other characteristics related to cheating. Therefore, it is necessary to collect the sounds during the test. The specific process is as follows:

[0224] Every time m frames are extracted from the video stream, a sound clip of the time length within m frames is obtained, the sound clip is converted into an audio signal, m points are uniformly sampled from it, and the audio signal of each point is n in length. MFCC features are extracted to obtain the MFCC feature sequence mfcc_list (equivalent to determining the sound feature sequence through the sound feature extraction model).

[0225] 9. Comprehensive judgment mechanism.

[0226] Since the judgment of cheating behavior within a period of time is based on the time series judgment of 7 individual behaviors, a neural network based on LSTM is constructed to train the 7 time series (equivalent to training the second model). The specific training steps are as follows:

[0227] (1) Prepare the candidate's video, extract k m-frame video streams from it, and obtain the time series of the seven individual behavioral features of each m-frame video stream respectively according to the above seven methods (determine the historical video data, extract k m-frame video images from the historical video data, and determine the facial feature sequence, background environment feature sequence, illegal item feature sequence, light and dark change feature sequence, eye feature sequence, action feature sequence and sound feature sequence corresponding to each m-frame video image in the k m-frame video images).

[0228] (2) Manually review the k video streams and mark whether they contain cheating behavior during the entire video process. If it does contain cheating behavior, the video stream is marked as 1, and if it does not contain cheating behavior, it is marked as 0 (equivalent to marking k m-frame video images and determining whether there is cheating behavior in each m-frame video image).

[0229] (3) Preprocess some time series, including:

[0230] (a), face_list and environment_list are single inputs and remain unchanged;

[0231] (b) Each obj in obj_list is a variable-length list. The maximum number of prohibited items detected is set to p. If the length of obj is greater than p, only the first p elements are taken; if the length of obj is less than p, the part less than p is set to 0; obj is made into a fixed length, and each obj in obj_list is input into DenseNet (i.e., dense convolutional neural network) and the output is set to 1, so that the output of obj_list is obj_list_1 as a single input (equivalent to converting the preset feature sequence from the form of a variable-length list to the form of a fixed-length list).

[0232] (c) Each element in light_list is a brightness gradient matrix. Each brightness gradient matrix is ​​input into the CNN convolutional neural network to extract the features of each matrix and set the output size to 1*1, so that the output of light_list light_list_1 is a single input.

[0233] (d), each element in eye_list and mfcc_list is a fixed-length list. The processing method is the same as the second half of (b). Each element is input into DenseNet and the output is set to 1, so that eye_list and mfcc_list are output as eye_list_1 and mfcc_list_1 respectively, both of which are single input.

[0234] (e) Each element in pose_list is a list of indefinite length. Since the number and type of skeletal key points that may appear in each frame of the image are not necessarily the same, the processing method is the same as (b). The maximum number of key points q is set. If the length of each element is greater than q, only the first q elements are taken; if the length of each element is less than q, the part less than q is set to 0; each element is made fixed-length, and each element in pose_list is input into DenseNet and the output is set to 1, so that the output of pose_list pose_list_1 is a single input (equivalent to converting the preset feature sequence from a variable-length list to a fixed-length list).

[0235] (4) The 7 lists obtained after processing are normalized, and then values ​​are taken according to the time series. Each time a value is taken, a 7-dimensional vector is obtained. M 7-dimensional vectors are input into the LSTM according to the time series. After each output, a fully connected layer of 7 neurons and an output layer of one neuron are added. The output layer after the last LSTM outputs a value of 1 or 0 indicating whether cheating occurs. The 7 intermediate results output by the fully connected layer of 7 neurons represent the cheating probabilities corresponding to the 7 items respectively (equivalent to the 7 intermediate results output by the 7 fully connected layers corresponding to the cheating probabilities of the 7 cheating behaviors).

[0236] (5) Input all k 7-dimensional vectors of length m into LSTM and complete a round of training. After multiple rounds of training and convergence, a comprehensive judgment model of cheating behavior based on 7 items is obtained (equivalent to training m 7-dimensional vectors as training data according to time series to determine the second model).

[0237] The following is the cheating behavior judgment process:

[0238] In the preamble, the video stream with a time length of m frames is processed, and a total of 7 time series with a length of m and a live value for identifying living faces are obtained. The comprehensive judgment mechanism is as follows:

[0239] (1) If the live value of the live face recognition is False, the current non-live face is directly output and identified as cheating (equivalent to using a live face detection model to detect whether the candidate is a real person. If the candidate is not a real person, it is directly determined that the candidate is cheating).

[0240] (2) When live is True, the seven output lists are input into the trained LSTM-based comprehensive judgment model in time series to obtain the cheating probabilities of the seven sub-items and the final comprehensive judgment results (equivalent to using the first model and the second model to determine whether the examinee cheats if the examinee is a real person).

[0241] Figure 5 A schematic diagram of a detection device provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the device 50 includes: a first determination module 501, a second determination module 502 and a third determination module 503;

[0242] The first determination module 501 is used to obtain video data including the target to be detected, and determine an image set including video images with a preset number of frames;

[0243] The second determination module 502 is used to input the image set into at least one first model to determine a preset feature sequence corresponding to each of the first models;

[0244] The third determination module 503 is used to determine the detection result of the target to be detected by using the second model according to the at least one preset feature sequence.

[0245] Specifically, the preset feature sequence includes at least one of the following: a face feature sequence, a background environment feature sequence, an illegal item feature sequence, a light and dark change feature sequence, an eye feature sequence, an action feature sequence, and a sound feature sequence.

[0246] Specifically, the second determination module 502 is used to determine the feature images of the to-be-detected targets respectively corresponding to the video images according to the third model;

[0247] The characteristic image of the target to be detected is input into the first model to determine the light-dark change characteristic sequence characterizing the light-dark change of the target to be detected.

[0248] Specifically, the second determination module 502 is used to use the first model to perform a preset process on the feature image of the target to be detected to determine a brightness image of a preset number of frames;

[0249] Determining a brightness change gradient according to time based on the brightness image of the preset number of frames;

[0250] The brightness change feature sequence is determined according to the brightness change gradient.

[0251] Specifically, the third determination module 503 is used to determine the probability weight corresponding to each of the preset feature sequences by using the second model;

[0252] According to the probability weight, a detection result of the target to be detected is determined.

[0253] Specifically, the third determination module 503 is used to perform a binary classification task according to the probability weight to determine the detection result of the target to be detected.

[0254] Specifically, the device further includes: a fourth determining module 504; wherein,

[0255] The fourth determination module 504 is used to determine whether the target to be detected is a predetermined target by using a preset detection model according to the image set including the preset number of frames;

[0256] The second determination module 502 is used to determine that the target to be detected is a predetermined target, obtain video data including the target to be detected, and determine an image set including video images with a preset number of frames.

[0257] It should be noted that: when the detection device provided in the above embodiment implements the corresponding detection method, only the division of the above program modules is used as an example. In actual application, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. Figure 1 The embodiments of the method shown belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0258] To implement the method of the embodiment of the present invention, the embodiment of the present invention provides a detection device, such as Figure 6As shown, the detection device 60 includes: a processor 601 and a memory 602 for storing a computer program that can be run on the processor; wherein,

[0259] When the processor 601 is used to run the computer program, it executes: obtaining video data including the target to be detected, determining an image set including a preset number of video frames; inputting the image set into at least one first model, determining a preset feature sequence corresponding to each of the first models; and using a second model to determine the detection result of the target to be detected based on the at least one preset feature sequence. Specifically, the first device can execute the following Figure 1 The method shown, and Figure 1 The detection method embodiments shown belong to the same concept, and their specific implementation processes are detailed in the method embodiments, which will not be repeated here.

[0260] In practical application, such as Figure 6 As shown, the detection device 60 may also include: at least one network interface 603. The various components in the detection device 60 are coupled together via a bus system 604. It is understood that the bus system 604 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 6 In the figure, various buses are marked as bus system 604. There may be at least one processor 601. The network interface 603 is used for wired or wireless communication between the detection device 60 and other devices.

[0261] The memory 602 in the embodiment of the present invention is used to store various types of data to support the operation of the detection device 60 .

[0262] The method disclosed in the above embodiment of the present invention can be applied to the processor 601, or implemented by the processor 601. The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 601 or the instruction in the form of software. The above processor 601 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 601 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 602. The processor 601 reads the information in the memory 602 and completes the steps of the above method in combination with its hardware.

[0263] In an exemplary embodiment, the detection device 60 can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0264] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored; when the computer program is executed by a processor, the computer program executes: obtaining video data including a target to be detected, determining an image set including a preset number of video frames; inputting the image set into at least one first model, determining a preset feature sequence corresponding to each of the first models; and using a second model according to the at least one preset feature sequence to determine the detection result of the target to be detected. Specifically, the computer program can also execute the following Figure 1 The method shown, and Figure 1 The method embodiments shown belong to the same concept, and their specific implementation processes are detailed in the method embodiments, which will not be repeated here.

[0265] In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0266] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0267] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0268] A person skilled in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0269] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0270] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0271] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0272] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A detection method, characterized in that: The method comprises: Acquire video data including the target to be detected, and determine an image set including video images with a preset number of frames; Inputting the image set into at least one first model, and determining a preset feature sequence corresponding to each of the first models; According to the at least one preset feature sequence, a second model is used to determine a detection result of the target to be detected.

2. The method according to claim 1, characterized in that The preset feature sequence includes at least one of the following: a face feature sequence, a background environment feature sequence, an illegal item feature sequence, a light and dark change feature sequence, an eye feature sequence, an action feature sequence and a sound feature sequence.

3. The method according to claim 2, characterized in that The step of inputting the image set into at least one first model and determining a preset feature sequence corresponding to each of the first models comprises: Determining, according to the third model, characteristic images of the targets to be detected corresponding to the video images respectively; The characteristic image of the target to be detected is input into the first model to determine the light-dark change characteristic sequence characterizing the light-dark change of the target to be detected.

4. The method according to claim 3, characterized in that The step of inputting the characteristic image of the target to be detected into the first model to determine the light-dark change characteristic sequence characterizing the light-dark change of the target to be detected comprises: Using the first model, performing preset processing on the feature image of the target to be detected to determine a brightness image of a preset number of frames; Determining a brightness change gradient according to time based on the brightness image of the preset number of frames; The brightness change feature sequence is determined according to the brightness change gradient.

5. The method according to claim 2, characterized in that: Determining the detection result of the target to be detected by using the second model according to the at least one preset feature sequence includes: Using the second model, determining the probability weight corresponding to each of the preset feature sequences; According to the probability weight, a detection result of the target to be detected is determined.

6. The method according to claim 5, characterized in that Determining the detection result of the target to be detected according to the probability weight includes: According to the probability weight, a binary classification task is performed to determine the detection result of the target to be detected.

7. The method according to claim 1, characterized in that The method further comprises: According to the image set including the preset number of frame images, using a preset detection model, determining whether the target to be detected is a predetermined target; The step of acquiring video data of the target to be detected and determining an image set including video images of a preset number of frames includes: The target to be detected is determined to be a predetermined target, video data including the target to be detected is acquired, and an image set including video images with a preset number of frames is determined.

8. A detection device, characterized in that: The device comprises: a first determining module, a second determining module and a third determining module; wherein, The first determination module is used to obtain video data including the target to be detected and determine an image set including video images with a preset number of frames; The second determination module is used to input the image set into at least one first model to determine a preset feature sequence corresponding to each of the first models; The third determination module is used to determine the detection result of the target to be detected using the second model according to the at least one preset feature sequence.

9. A detection device, characterized in that: The device includes a network interface, a memory and a processor; wherein the network interface is used to realize connection communication between components; the memory is used to store a computer program that can be run on the processor; and the processor is used to execute the method described in any one of claims 1 to 7 when running the computer program.

10. A computer storage medium, characterized in that: The computer storage medium stores a computer program, and when the computer program is executed by at least one processor, the method according to any one of claims 1 to 7 is implemented.

Images