Method and device for identifying target object in video, equipment and storage medium

By calculating the scene transformative score of the video frame and extracting keyframes for feature matching, the problem of insufficient recognition efficiency and accuracy of target objects in the video is solved, and efficient identification of target objects in the video is achieved, and the supervision personnel are assisted in safe inspection.

CN118968372APending Publication Date: 2024-11-15BEIJING WONDERSOFT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410237279.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-03-01
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

It is difficult for the prior art to efficiently and accurately detect and identify target objects in videos, especially in the fields of information exchange, where identification efficiency and accuracy are insufficient.

Method used

By obtaining the video file to be detected and the object image library, calculate the scene transformability score of the video frame, cut the lens, extract the keyframes, and feature extraction and matching them, obtaining the target feature vector and information, and realizing the target object in the video Identification.

Benefits of technology

It realizes efficient and accurate detection and identification of target objects in the video, which can assist supervisors in video content detection and improve security.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968372A_ABST
    Figure CN118968372A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a method and device for identifying a target object in a video, equipment and a storage medium. The method comprises the following steps: acquiring a to-be-detected video file and a target object image library; calculating a scene change score of a video frame in the to-be-detected video file; when the scene change score is higher than a preset scene change score, taking the video frame with the scene change score higher than the preset scene change score as a target video frame; cutting the to-be-detected video file according to the target video frame to obtain shots; key frames are respectively determined from the shot of the video file to be detected; performing feature extraction on the key frames of the shot to obtain feature vectors to be detected; and determining a feature vector matched with the to-be-detected feature vector from a target object image library as a target feature vector, and obtaining target object information of a target object corresponding to the target feature vector. According to the embodiment of the invention, whether the target object exists in the to-be-detected video file can be efficiently and accurately determined.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of video technology, and in particular to a method and device for identifying a target object in a video, an electronic device, and a storage medium. Background Art

[0002] In specific implementation, a series of still images are continuously projected onto the screen at a certain rate. Such continuous images are called videos. Through videos, people can understand the entire process of an event more intuitively and comprehensively, so as to make better decisions and judgments. As a carrier of information transmission, videos contain rich image content.

[0003] Therefore, if there is a technology that can efficiently and accurately detect and identify target objects (such as target persons) in videos, then this technology will play an important role in areas such as information exchange. Summary of the invention

[0004] The embodiment of the present invention provides a method for identifying a target object in a video, so as to solve the problem of how to efficiently and accurately detect and identify the target object in a video in the field of information exchange.

[0005] Correspondingly, an embodiment of the present invention further provides a device for identifying a target object in a video, an electronic device, and a storage medium to ensure the implementation and application of the above method.

[0006] In order to solve the above problem, an embodiment of the present invention discloses a method for identifying a target object in a video, the method comprising:

[0007] Acquire a video file to be detected and a target object image library; wherein the target object image library includes target object information and feature vectors corresponding to the target object;

[0008] Calculating the scene changeability score of the video frame in the video file to be detected;

[0009] When the scene changeability score of the video frame is higher than a preset scene changeability score, taking the video frame having the scene changeability score higher than the preset scene changeability score as a target video frame;

[0010] The video file to be detected is cut according to the target video frame to obtain a plurality of shots; wherein the shots are video frames in which the background of the video file to be detected does not change and the movements of the object are continuous;

[0011] Determine key frames respectively from the shots of the video file to be detected;

[0012] Extracting features from the key frames of the shots respectively to obtain feature vectors to be detected;

[0013] A feature vector matching the feature vector to be detected is determined from the target object image library as a target feature vector, and target object information of the target object corresponding to the target feature vector is acquired.

[0014] Optionally, the calculating the scene changeability score of the video frame in the video file to be detected includes:

[0015] Obtaining image size information of the current video frame in the video file to be detected;

[0016] Acquire the color gamut image information and contour image information of the current video frame to form a four-dimensional matrix of the current video frame, and acquire the color gamut image information and contour image information of a video frame previous to the current video frame to form a four-dimensional matrix of the previous video frame;

[0017] The scene changeability score of the current video frame is calculated according to the image size information of the current video frame, the four-dimensional matrix of the current video frame and the four-dimensional matrix of the previous video frame.

[0018] Optionally, the determining key frames from the shots of the video file to be detected respectively includes:

[0019] Calculating the difference scores between the video frames in the shot of the video file to be detected;

[0020] Using the difference score of the shot as the ordinate and the serial number of the video frame of the shot as the abscissa to draw a difference curve of the shot;

[0021] Performing smoothing on the difference curve;

[0022] A valley value is determined from the smoothed difference curve, and a video frame corresponding to a sequence number corresponding to the valley value is used as a key frame of the shot.

[0023] Optionally, the calculating the difference scores between the video frames in the shots of the video file to be detected includes:

[0024] Acquire a first video frame and a second video frame adjacent to each other in the shot of the video file to be detected;

[0025] Calculating an average value and a variance of the first video frame, an average value and a variance of the second video frame, and a covariance of the first video frame and the second video frame;

[0026] A dissimilarity score between video frames in the shot is calculated based on a mean value and a variance of the first video frame, a mean value and a variance of the second video frame, and a covariance between the first video frame and the second video frame.

[0027] Optionally, the target object information includes a target object image and identity information corresponding to the target object, and determining a feature vector matching the feature vector to be detected from the target object image library as a target feature vector, and acquiring target object information of the target object corresponding to the target feature vector, includes:

[0028] Calculate the similarity between the feature vector to be detected and the feature vector in the target object image library, and use the feature vector ranked before a preset number of bits in similarity as the target feature vector;

[0029] Acquire a target object image corresponding to the target feature vector from the target object image library as an initial target object image;

[0030] Classifying the initial target object image to obtain a classification result, and calculating an inter-class variance value corresponding to the initial target object image according to the classification result, the similarity corresponding to the initial target object image, and the preset number of bits;

[0031] Determining a maximum inter-class variance value from among the inter-class variance values;

[0032] When the maximum inter-class variance exceeds a preset threshold, the initial target object image is used as a final target object image, identity information corresponding to the final target object image is acquired, and the final target object image and the identity information are output.

[0033] Optionally, classifying the initial target object image to obtain a classification result, and calculating the inter-class variance value corresponding to the initial target object image according to the classification result, the similarity corresponding to the initial target object image, and the preset number of bits, includes:

[0034] Classifying the initial target object image to obtain a classification result; wherein the classification result includes a true class and a false class;

[0035] Calculate the true class ratio according to the number of the initial target object images of the true class and the preset number of bits, and calculate the false class ratio according to the number of the initial target object images of the false class and the preset number of bits;

[0036] Calculating a true class mean value according to the similarity corresponding to the initial target object image of the true class, and calculating a false class mean value according to the similarity corresponding to the initial target object image of the false class;

[0037] The inter-class variance value is calculated according to the true class proportion, the false class proportion, the true class mean and the false class mean.

[0038] Optionally, the target object at least includes a target person.

[0039] The embodiment of the present invention further discloses a device for identifying a target object in a video, the device comprising:

[0040] A data acquisition module, used to acquire a video file to be detected and a target object image library; wherein the target object image library includes target object information and feature vectors corresponding to the target object;

[0041] A scene changeability score calculation module, used to calculate the scene changeability score of the video frame in the video file to be detected;

[0042] a target video frame determining module, configured to, when the scene changeability score of the video frame is higher than a preset scene changeability score, use the video frame having the scene changeability score higher than the preset scene changeability score as a target video frame;

[0043] A shot cutting module is used to cut the video file to be detected according to the target video frame to obtain a plurality of shots; wherein the shots are video frames in the video file to be detected where the background does not change and the action of the object is continuous;

[0044] A key frame determination module, used to determine key frames from the shots of the video file to be detected;

[0045] A feature vector extraction module, used for extracting features from the key frames of the shots to obtain feature vectors to be detected;

[0046] The target object information acquisition module is used to determine a feature vector matching the feature vector to be detected from the target object image library as a target feature vector, and acquire target object information of the target object corresponding to the target feature vector.

[0047] Optionally, the scene changeability score calculation module is used to:

[0048] Obtaining image size information of the current video frame in the video file to be detected;

[0049] Acquire the color gamut image information and contour image information of the current video frame to form a four-dimensional matrix of the current video frame, and acquire the color gamut image information and contour image information of a video frame previous to the current video frame to form a four-dimensional matrix of the previous video frame;

[0050] The scene changeability score of the current video frame is calculated according to the image size information of the current video frame, the four-dimensional matrix of the current video frame and the four-dimensional matrix of the previous video frame.

[0051] Optionally, the key frame determination module is used to:

[0052] Calculating the difference scores between the video frames in the shot of the video file to be detected;

[0053] Using the difference score of the shot as the ordinate and the serial number of the video frame of the shot as the abscissa to draw a difference curve of the shot;

[0054] Performing smoothing on the difference curve;

[0055] A valley value is determined from the smoothed difference curve, and a video frame corresponding to a sequence number corresponding to the valley value is used as a key frame of the shot.

[0056] Optionally, the key frame determination module is used to:

[0057] Acquire a first video frame and a second video frame adjacent to each other in the shot of the video file to be detected;

[0058] Calculating an average value and a variance of the first video frame, an average value and a variance of the second video frame, and a covariance of the first video frame and the second video frame;

[0059] A dissimilarity score between video frames in the shot is calculated based on a mean value and a variance of the first video frame, a mean value and a variance of the second video frame, and a covariance between the first video frame and the second video frame.

[0060] Optionally, the target object information acquisition module is used to:

[0061] Calculate the similarity between the feature vector to be detected and the feature vector in the target object image library, and use the feature vector ranked before a preset number of bits in similarity as the target feature vector;

[0062] Acquire a target object image corresponding to the target feature vector from the target object image library as an initial target object image;

[0063] Classifying the initial target object image to obtain a classification result, and calculating an inter-class variance value corresponding to the initial target object image according to the classification result, the similarity corresponding to the initial target object image, and the preset number of bits;

[0064] Determining a maximum inter-class variance value from among the inter-class variance values;

[0065] When the maximum inter-class variance exceeds a preset threshold, the initial target object image is used as a final target object image, identity information corresponding to the final target object image is acquired, and the final target object image and the identity information are output.

[0066] Optionally, the target object information acquisition module is used to:

[0067] Classifying the initial target object image to obtain a classification result; wherein the classification result includes a true class and a false class;

[0068] Calculate the true class ratio according to the number of the initial target object images of the true class and the preset number of bits, and calculate the false class ratio according to the number of the initial target object images of the false class and the preset number of bits;

[0069] Calculating a true class mean value according to the similarity corresponding to the initial target object image of the true class, and calculating a false class mean value according to the similarity corresponding to the initial target object image of the false class;

[0070] The inter-class variance value is calculated according to the true class proportion, the false class proportion, the true class mean and the false class mean.

[0071] Optionally, the target object at least includes a target person.

[0072] An embodiment of the present invention further discloses an electronic device, comprising: a processor; and a memory, on which executable code is stored. When the executable code is executed, the processor executes a method for identifying a target object in a video as described in one or more of the embodiments of the present invention.

[0073] The embodiments of the present invention further disclose one or more machine-readable media on which executable codes are stored. When the executable codes are executed, a processor executes the method for identifying a target object in a video as described in one or more of the embodiments of the present invention.

[0074] Compared with the prior art, the embodiments of the present invention have the following advantages:

[0075] In an embodiment of the present invention, a video file to be detected and a target object image library are obtained, wherein the target object image library may include target object information and feature vectors corresponding to the target object, the scene variability score of the video frame in the video file to be detected is calculated, when the scene variability score of the video frame is higher than a preset scene variability score, the video frame with a scene variability score higher than the preset scene variability score is used as a target video frame, the video file to be detected is cut according to the target video frame to obtain a plurality of shots, wherein the shots are video frames in the video file to be detected in which the background does not change and the movement of the object is continuous, key frames are respectively determined from the shots of the video file to be detected, and features are extracted from the key frames of the shots to obtain feature vectors to be detected, and then, a feature vector matching the feature vector to be detected can be determined from the target object image library as a target feature vector, and then the target object information of the target object corresponding to the target feature vector can be obtained, thereby realizing the detection and recognition of the target object in the video. In the fields of information exchange, etc., the embodiments of the present invention can efficiently and accurately determine whether there is a target object in the video file to be detected, such as a target person or important item, for detection and identification, and then obtain the target object information corresponding to the target object, and then perform corresponding processing based on the results of detection and identification. For example, it can assist supervisors in performing video content detection on the video file to be detected, thereby playing a role in maintaining safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 is a flowchart of a method for identifying a target object in a video according to an embodiment of the present invention;

[0077] Figure 2 is a video level schematic diagram of a video of the present invention;

[0078] Figure 3 It is a schematic diagram of a module for identifying a video file to be detected according to the present invention;

[0079] Figure 4 It is a structural block diagram of an embodiment of a device for identifying a target object in a video according to the present invention;

[0080] Figure 5 It is a schematic diagram of the structure of a device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0081] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0082] Reference Figure 1 , is a flowchart of an embodiment of a method for identifying a target object in a video of the present invention, comprising the following steps:

[0083] Step 101: Obtain a video file to be detected and a target object image library; wherein the target object image library includes target object information and feature vectors corresponding to the target object.

[0084] In a specific implementation, the target object may include a target person (for example, a person who is sensitive to appearing in certain places). Of course, the target object may also include key items (for example, valuable items). The video file to be detected may be a real-time video or an existing video that needs to be detected. The method of the embodiment of the present invention may be applied to fields such as the information exchange field. In the field of information exchange: Video is a mainstream information transmission method. The identity and portrait information of sensitive persons are easily contained in the video. Specifically, the video file to be detected may be detected and compared with the target object image library for verification, so that it is possible to quickly detect whether the video can be transmitted externally. Of course, in addition to being applicable to the field of information exchange, the embodiment of the present invention may also be applied to other fields that require detection and identification of the target object, and the embodiment of the present invention does not need to be limited to this. For ease of explanation, the following mainly uses the target object as a target person as an example for explanation.

[0085] In an embodiment of the present invention, a target person image of an input target person (usually an image containing the face of the target person) can be pre-extracted using a convolutional neural network or other method to obtain a high-dimensional feature vector, and then the target object information of the target person can be obtained. Specifically, the target object information of the target person may at least include the target object image (target person image) and identity information of the target person, and a vector index library (target object image library) is constructed based on the identity information, feature vectors, etc., so as to facilitate the subsequent determination of whether there is target object information of the target person that matches the target object image library in the video file to be detected.

[0086] Specifically, when it is necessary to detect and identify the target object in the video, the video file to be detected and the target object image library can be obtained, and then it can be determined whether the video file to be detected matches the target object information of the target person in the target object image library. If they match, the corresponding target object information can be obtained and provided to relevant personnel, such as the supervisor of the venue, to assist the relevant personnel in making corresponding processing.

[0087] It should be noted that the embodiments of the present invention may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described herein within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).

[0088] Step 102: Calculate the scene changeability scores of the video frames in the video file to be detected.

[0089] Step 103: When the scene variability score of the video frame is higher than a preset scene variability score, the video frame having the scene variability score higher than the preset scene variability score is used as a target video frame.

[0090] Step 104: cutting the video file to be detected according to the target video frame to obtain a plurality of shots; wherein the shots are video frames in which the background of the video file to be detected does not change and the movements of the object are continuous.

[0091] Reference Figure 2 , is a schematic diagram of the video level of a video of the present invention, such as Figure 2 As shown, the video (video file to be detected) is composed of frames (video frames, i.e., images), shots, and scenes, which are continuously stacked and spliced ​​in chronological order. Specifically, the part of the video whose background does not change is called a scene, and a continuous action in the scene can be called a shot. A video frame is the smallest component unit of a video, and each video frame is an independent image. In other words, the shot of the embodiment of the present invention can be a number of continuous video frames in which the background of the video file to be detected does not change and the action of the object is continuous. The target video frame refers to a video frame that can reflect the change of the background of the object in the video to be detected or the discontinuous action of the object. For example, in a surveillance video of a place, the video frame corresponding to the moment when a person enters a building from the street can be regarded as a key frame.

[0092] In an embodiment of the present invention, for a video frame in a video file to be detected, its scene variability score can be calculated respectively. If the scene variability score of a video frame is higher than a preset scene variability score, the video frame can be used as a target video frame. Based on the target video frame, the video file to be detected can be shot cut, so that one or more shots can be obtained. If the scene variability score of the video frame is lower than or equal to the preset scene variability score, other video frames in the video file to be detected can be calculated continuously until all video frames in the video file to be detected are traversed to complete the shot cutting of the video file to be detected. After the video file to be detected in the embodiment of the present invention is cut, each scene may be divided into different shots, but a single shot only contains one scene. The embodiment of the present invention can effectively improve the running speed and accuracy of key frame extraction by cutting the video file to be detected into multiple shots with scene information, and then extracting key frames based on shots that are smaller than the video file to be detected.

[0093] Step 105: determine key frames from the shots of the video file to be detected.

[0094] Step 106: extract features from the key frames of the shot to obtain feature vectors to be detected.

[0095] Step 107: Determine a feature vector that matches the feature vector to be detected from the target object image library as a target feature vector, and obtain target object information of the target object corresponding to the target feature vector.

[0096] In a specific implementation, the video file to be detected may also include video frames that are key frames. Specifically, the key frames are certain video frames in the entire video file to be detected or the shot that best reflect the main video content of the video. For example, the key frames may be video frames containing the front face of a person object in the shot.

[0097] In an embodiment of the present invention, after the video file to be detected is cut into several shots, key frames can be determined from the shots respectively, and then feature extraction can be performed based on the key frames to obtain feature vectors to be detected corresponding to the key frames, and then the vector to be detected can be matched with feature vectors in the target object image library, and the matched feature vectors can be used as target feature vectors, and then the target object information of the target object corresponding to the target feature vector can be obtained from the target object image library, such as target object images and identity information, and provided to relevant personnel. For example, the similarity between the vector to be detected and the feature vector in the target object image library (such as cosine similarity) can be calculated, and then the feature vector ranked before a preset number of bits N in similarity is used as the target feature vector, and the target object information corresponding to the target feature vector, i.e., the facial image and identity information of the target person, can be obtained from the target object image library, where N is an integer, and usually N is greater than or equal to 2.

[0098] In the above-mentioned method for identifying target objects in videos, a video file to be detected and a target object image library are obtained, wherein the target object image library may include target object information and feature vectors corresponding to the target object, and the scene changeability score of the video frame in the video file to be detected is calculated. When the scene changeability score of the video frame is higher than a preset scene changeability score, the video frame with a scene changeability score higher than the preset scene changeability score is used as a target video frame. The video file to be detected is cut according to the target video frame to obtain a plurality of shots, wherein the shots are video frames in the video file to be detected in which the background does not change and the movement of the object is continuous, key frames are determined from the shots of the video file to be detected, and features are extracted from the key frames of the shots to obtain feature vectors to be detected. Then, a feature vector matching the feature vector to be detected can be determined from the target object image library as a target feature vector, and then the target object information of the target object corresponding to the target feature vector can be obtained, thereby realizing detection and identification of the target object in the video. In the fields of information exchange, etc., the embodiments of the present invention can efficiently and accurately determine whether there is a target object in the video file to be detected, such as a target person or important item, for detection and identification, and then obtain the target object information corresponding to the target object, and then perform corresponding processing based on the results of detection and identification. For example, it can assist supervisors in performing video content detection on the video file to be detected, thereby playing a role in maintaining safety.

[0099] In an embodiment of the present invention, the step 102 of calculating the scene changeability score of the video frame in the video file to be detected may include:

[0100] Obtaining image size information of the current video frame in the video file to be detected;

[0101] Acquire the color gamut image information and contour image information of the current video frame to form a four-dimensional matrix of the current video frame, and acquire the color gamut image information and contour image information of a video frame previous to the current video frame to form a four-dimensional matrix of the previous video frame;

[0102] The scene changeability score of the current video frame is calculated according to the image size information of the current video frame, the four-dimensional matrix of the current video frame and the four-dimensional matrix of the previous video frame.

[0103] In the embodiment of the present invention, HSLE represents a four-dimensional matrix consisting of HSL (Hue, Saturation, Lightness) color domain image information and contour image information (edge) of a video frame.

[0104] As an example, in the embodiment of the present invention, for a video frame in a video file to be detected, the scene changeability score Score can be calculated according to the following formula 1:

[0105] Formula 1:

[0106]

[0107] Among them, HSLE Previous_Frame HSLE is the color gamut image information and contour image information of the current video frame. Current_Frame M*N represents the color gamut image information and contour image information of the previous video frame of the current video frame. M*N represents the image size information of the current video frame (image). Specifically, M and N represent the horizontal and vertical sizes of the current video frame, respectively.

[0108] In an embodiment of the present invention, the step 105 of respectively determining key frames from the shots of the video file to be detected includes:

[0109] Calculating the difference scores between the video frames in the shot of the video file to be detected;

[0110] Using the difference score of the shot as the ordinate and the serial number of the video frame of the shot as the abscissa to draw a difference curve of the shot;

[0111] Performing smoothing on the difference curve;

[0112] A valley value is determined from the smoothed difference curve, and a video frame corresponding to a sequence number corresponding to the valley value is used as a key frame of the shot.

[0113] The difference score may be a structural similarity (SSIM). Specifically, SSIM is based on the strong correlation between pixels of an image (video frame), assuming that the human visual system (HSV, i.e., Hue, Saturation, and Value) mainly obtains structural information from the visible area. Therefore, by detecting whether the structural information has changed, approximate information of image distortion can be perceived. The higher the structural similarity of two images (video frames), the more similar the two images look.

[0114] In an embodiment of the present invention, for each shot obtained after the video file to be detected is cut, all video frames in the shot may be calculated. In an embodiment of the present invention, the calculation of the difference scores between the video frames in the shot of the video file to be detected may include:

[0115] Acquire a first video frame and a second video frame adjacent to each other in the shot of the video file to be detected;

[0116] Calculating an average value and a variance of the first video frame, an average value and a variance of the second video frame, and a covariance of the first video frame and the second video frame;

[0117] A dissimilarity score between video frames in the shot is calculated based on a mean value and a variance of the first video frame, a mean value and a variance of the second video frame, and a covariance between the first video frame and the second video frame.

[0118] In one example, the difference score SSIM between video frames in a shot may be calculated frame by frame according to the following formula 2:

[0119] Formula 2:

[0120]

[0121] Where x and y represent two adjacent video frames in a certain shot, namely the first video frame and the second video frame, μ x The average value, μ y represents the mean value of y, represents the variance of x, represents the variance of y, σ xy represents the covariance of x and y, and c1 and c2 represent constants.

[0122] In an embodiment of the present invention, after obtaining the difference score SSIM of the lens, the difference score SSIM of the entire lens is used as the ordinate and the serial number of the video frame is used as the abscissa to draw the difference curve of the lens, and the difference curve is smoothed, and then for the difference curve after smoothing, the valley value of the difference curve change is found. At this time, the ordinate, i.e., the serial number of the video frame, is the position of the key frame, and the video frame is extracted from the lens according to the serial number and marked as the key frame of the lens. The embodiment of the present invention performs smoothing on the difference curve to reduce the impact of oscillation or noise of the difference curve, and can effectively reduce the extraction of similar key frames, thereby speeding up the calculation speed for the video file to be detected.

[0123] In an embodiment of the present invention, the target object information may include a target object image and identity information corresponding to the target object, and the step 107 of determining a feature vector matching the feature vector to be detected from the target object image library as a target feature vector, and obtaining the target object information of the target object corresponding to the target feature vector, includes:

[0124] Calculate the similarity between the feature vector to be detected and the feature vector in the target object image library, and use the feature vector ranked before a preset number of bits in similarity as the target feature vector;

[0125] Acquire a target object image corresponding to the target feature vector from the target object image library as an initial target object image;

[0126] Classifying the initial target object image to obtain a classification result, and calculating an inter-class variance value corresponding to the initial target object image according to the classification result, the similarity corresponding to the initial target object image, and the preset number of bits;

[0127] Determining a maximum inter-class variance value from among the inter-class variance values;

[0128] When the maximum inter-class variance exceeds a preset threshold, the initial target object image is used as a final target object image, identity information corresponding to the final target object image is acquired, and the final target object image and the identity information are output.

[0129] In a specific implementation, target object information corresponding to the target person is stored in the target object image library, wherein the target object information may include a target object image and identity information corresponding to the target person, for example, a portrait image (face image) of the target person, and identity information such as the target person's name, height, gender, and weight.

[0130] In an embodiment of the present invention, after obtaining the key frame of the shot, the same feature extraction is performed on the portrait information in the key frame and recorded as the feature vector to be detected. Then, the feature vector to be detected is compared with the feature vector in the target object image library, and the feature vectors of the Top N (N>2) with the highest similarity to the feature vector to be detected are used as the target feature vector. Then, the target object information (including the target object image) of the target person corresponding to the target feature vector is obtained, and the target object image is used as the initial target object image. Then, the inter-class variance value (maximum inter-class variance) of the initial target object image can be determined by methods such as maximum inter-class variance. Specifically, the maximum inter-class variance directly divides the given one-dimensional sequence according to each interval, and calculates the mean variance of the two classes after being divided into two classes.

[0131] After obtaining the inter-class variance value, it can be determined whether the initial target object image is the final target object image to be output based on the inter-class variance value and the preset threshold. Specifically, if the inter-class variance value is greater than the preset threshold, it can be determined that the initial target object image and other true class target object images are the final target object images to be output, wherein the final target object image can be one or more images. Finally, the final target object image and the corresponding identity information can be output to provide to relevant personnel. The preset threshold is a constant obtained by reverse training based on the labeled face recognition result.

[0132] In an embodiment of the present invention, classifying the initial target object image to obtain a classification result, and calculating the inter-class variance value corresponding to the initial target object image according to the classification result, the similarity corresponding to the initial target object image, and the preset number of bits, includes:

[0133] Classifying the initial target object image to obtain a classification result; wherein the classification result includes a true class and a false class;

[0134] Calculate the true class ratio according to the number of the initial target object images of the true class and the preset number of bits, and calculate the false class ratio according to the number of the initial target object images of the false class and the preset number of bits;

[0135] Calculating a true class mean value according to the similarity corresponding to the initial target object image of the true class, and calculating a false class mean value according to the similarity corresponding to the initial target object image of the false class;

[0136] The inter-class variance value is calculated according to the true class proportion, the false class proportion, the true class mean and the false class mean.

[0137] In an example, the inter-class variance value variance corresponding to the initial target object image may be calculated according to the following formula 3.

[0138] Formula 3:

[0139]

[0140] in, It represents the true class ratio, which is the ratio of the number of the original target object images of the true class to the preset number of digits N. True_mean represents the true class mean, which is calculated based on the similarity corresponding to the original target object images of the true class. It represents the proportion of false classes, which is the ratio of the number of initial target object images of the false class to the preset number of digits N. False_mean represents the mean of the false class, which is calculated based on the similarity corresponding to the initial target object images of the false class. N represents the preset number of digits, which is greater than or equal to 2. maxinum represents the maximum inter-class variance value.

[0141] In an embodiment of the present invention, after determining the initial target object images whose similarity ranking is before the preset number of bits N, the initial target object images are classified to obtain classification results, wherein the classification results include true classes and false classes. For example, assuming that the preset number of bits N=5, 1 represents the true class, and 0 represents the false class, then there are 5 initial target object images, and then the initial target object images are classified. When the initial target object images are classified for the first time, it can be assumed that the initial target object image 0 is the true initial target object image, and the initial target object images 1, 2, 3, and 4 are false classes. Then, the calculation process of obtaining the inter-class variance value variance when the initial target object images are classified for the first time is:

[0142]

[0143] Then, when the initial target object images are classified for the second time, it can be assumed that the initial target object images 0 and 1 are true initial target object images, and the initial target object images 2, 3, and 4 are false images, and so on, until the inter-class variance values ​​after all classification situations are calculated. The results are shown in the variance column in Table 1.

[0144] Table 1:

[0145]

[0146] Subsequently, the maximum inter-class variance value maxinum=0.11350 is determined from the inter-class variance value variance, and then the maximum inter-class variance value maxinum is compared with the preset threshold. If the maximum inter-class variance value maxinum> the preset threshold, the initial target object image before the maximum position is a hit, and the subsequent position is marked as a miss, as shown in the Verified (classification result) column in Table 1. If the maximum inter-class variance value maxinum< the preset threshold, all the initial target object images are marked as misses.

[0147] Among them, the maximum position is the maximum inter-class variance value maxinum in the specific data table obtained for classifying the initial target object image, and the position of the initial target object image corresponding to the maximum inter-class variance value maxinum. Assuming that the maximum inter-class variance value maxinum is the inter-class variance value variance corresponding to Image 1 in Table 2 below, the maximum position is the row corresponding to Image 1, and Image 0 and Image 1 in Table 2 can be regarded as being hit.

[0148] Table 2:

[0149]

[0150] It should be noted that, in the related technical solutions, the method of performing interval frame extraction for the video file to be detected will result in too many similar frames being extracted, thereby increasing the computational cost and reducing the processing speed, while the embodiment of the present invention can cut the video file to be detected into multiple shots and then extract key frames from the shots respectively, so the number of images and the number of detections of the video file to be detected can be reduced, thereby speeding up the detection speed and reducing the computational cost in the detection process. In the related technical solutions, the target face in the key frame is compared with the target person image library, and the returned result is the similarity between the target face and each target person, and the highest similarity does not necessarily hit, therefore, the embodiment of the present invention can first obtain the target object image whose similarity is ranked before the preset number of bits, and then calculate the inter-class variance value of the target object image to determine whether the target object image is the target object image that is finally output.

[0151] In order to enable those skilled in the art to better understand the embodiments of the present invention, specific examples are used below for illustration. Figure 3 , is a schematic diagram of a module for identifying a video file to be detected in the present invention, which is mainly divided into three main modules: video analysis, target person feature library construction, and face detection. The main work content of each module is:

[0152] The video parsing module consists of two parts: shot cutting and key frame extraction. Shot cutting cuts the input video file (video file to be detected) into shots with scene significance; key frame extraction calculates the difference scores between video frames in the shot and extracts key frames after smoothing the difference score curve.

[0153] The target person feature library module first uses a convolutional neural network to extract features from the input target person image to obtain a high-dimensional feature vector, and constructs a target person feature library (target object image library) with the identity information corresponding to the target person, so that the face recognition module can realize vector recall.

[0154] The face recognition module searches for faces in key frames and extracts features from faces to obtain feature vectors to be detected. Then, based on the similarity between the feature vectors to be detected and the feature vectors in the target person feature library, it searches for the top N (N>2) similar results before the preset number of digits from the target person index library, i.e., the target object information (i.e., similar objects, which may specifically include target object images and identity information) of the top N target persons. Finally, the inter-class variance of the returned target object information is calculated to determine whether the similar objects are hit. If hit, the location information of the target person appearing in the video (xx minutes and xx seconds of the video) and the identity information of the target person are output.

[0155] Specifically, the embodiment of the present invention is directed to the detection and recognition technology of the target person in the video, and the overall solution process can be divided into:

[0156] Step 1: Input the video file to be detected and the target person image library (target object image library);

[0157] Step 2: For the video frame in the video to be detected, calculate its scene changeability score according to the aforementioned formula 1;

[0158] Step 3: If the scene changeability score is higher than the preset scene changeability score, the video file to be detected is cut, otherwise the calculation continues;

[0159] Step 4: For all video frames in the shot, calculate the difference scores between video frames frame by frame according to Formula 2;

[0160] Step 5: Draw the difference curve of the shot using the difference score of the entire shot as the ordinate and the serial number of the video frame as the abscissa, and perform curve smoothing on the difference curve;

[0161] Step 6: Find the valley value of the difference curve. At this time, the vertical coordinate, i.e. the sequence number of the frame, is the position of the key frame. According to the sequence, the video frame in the video can be extracted and recorded as the key frame;

[0162] Step 7: Extract features from the portrait image in the target person image library to obtain feature vectors and save them;

[0163] Step 8: Perform the same feature extraction on the portrait information in the key frame to obtain the feature data to be detected;

[0164] Step 9: Compare the feature data to be detected with the feature vectors in the target person image library, and return the target object information (including the initial target object image) corresponding to the top N (N>2) feature vectors with the closest similarity to the feature data to be detected;

[0165] Step 10: Calculate the inter-class variance value based on the aforementioned formula 3, and determine whether the initial target object image is hit by judging whether the inter-class variance value is greater than a preset threshold;

[0166] Step 11: If the initial target object image hits, that is, the inter-class variance value is greater than the preset threshold, the initial target object image can be used as the final target object image, and the final target object image and the corresponding identity information, as well as the location information of the target person in the video are output.

[0167] Among them, after step 3, each scene may be divided into different shots, but a single shot only contains one scene. Step 3 divides the video into shots with scene information, which can effectively improve the running speed and accuracy of key frame extraction. Step 5 smoothes the difference curve to reduce the impact of difference curve oscillation or noise, which can effectively reduce the extraction of similar key frames, thereby speeding up the calculation speed.

[0168] In summary, the embodiment of the present invention proposes a technology for detecting target persons in videos. The technology first extracts key frames from the video file to be detected, then extracts features from the portrait information in the key frames, and then compares the portrait features (feature vectors) in the target person image library with the portrait features (feature vectors) in the key frames, and finally returns the time position and spatial position of the target person in the video, as well as the identity information of the target person, which can assist supervisors in content detection of the video and play a role in maintaining safety. The embodiment of the present invention can realize rapid detection of whether there is a target person in the video, and reduce the number of images and the number of detections of the video file to be detected by methods such as lens cutting, thereby speeding up the detection speed and reducing the computational cost in the detection process.

[0169] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0170] On the basis of the above embodiments, this embodiment further provides a device for identifying a target object in a video, which is applied to electronic devices such as terminal devices and servers.

[0171] Reference Figure 4 , shows a structural block diagram of an embodiment of a device for identifying a target object in a video of the present invention, which may specifically include the following modules:

[0172] The data acquisition module 401 is used to acquire the video file to be detected and the target object image library; wherein the target object image library includes the target object information and feature vector corresponding to the target object;

[0173] A scene changeability score calculation module 402, used to calculate the scene changeability score of the video frame in the video file to be detected;

[0174] a target video frame determining module 403, configured to, when the scene variability score of the video frame is higher than a preset scene variability score, use the video frame having the scene variability score higher than the preset scene variability score as a target video frame;

[0175] The shot cutting module 404 is used to cut the video file to be detected according to the target video frame to obtain a plurality of shots; wherein the shots are video frames in the video file to be detected where the background does not change and the action of the object is continuous;

[0176] A key frame determination module 405, configured to respectively determine key frames from the shots of the video file to be detected;

[0177] A feature vector extraction module 406 is used to extract features from the key frames of the shots to obtain feature vectors to be detected;

[0178] The target object information acquisition module 407 is used to determine a feature vector matching the feature vector to be detected from the target object image library as a target feature vector, and acquire target object information of the target object corresponding to the target feature vector.

[0179] In one embodiment of the present invention, the scene changeability score calculation module 402 is used to:

[0180] Obtaining image size information of the current video frame in the video file to be detected;

[0181] Acquire the color gamut image information and contour image information of the current video frame to form a four-dimensional matrix of the current video frame, and acquire the color gamut image information and contour image information of a video frame previous to the current video frame to form a four-dimensional matrix of the previous video frame;

[0182] The scene changeability score of the current video frame is calculated according to the image size information of the current video frame, the four-dimensional matrix of the current video frame and the four-dimensional matrix of the previous video frame.

[0183] In one embodiment of the present invention, the key frame determination module 405 is used to:

[0184] Calculating the difference scores between the video frames in the shot of the video file to be detected;

[0185] Using the difference score of the shot as the ordinate and the serial number of the video frame of the shot as the abscissa to draw a difference curve of the shot;

[0186] Performing smoothing on the difference curve;

[0187] A valley value is determined from the smoothed difference curve, and a video frame corresponding to a sequence number corresponding to the valley value is used as a key frame of the shot.

[0188] In one embodiment of the present invention, the key frame determination module 405 is used to:

[0189] Acquire a first video frame and a second video frame adjacent to each other in the shot of the video file to be detected;

[0190] Calculating an average value and a variance of the first video frame, an average value and a variance of the second video frame, and a covariance of the first video frame and the second video frame;

[0191] A dissimilarity score between video frames in the shot is calculated based on a mean value and a variance of the first video frame, a mean value and a variance of the second video frame, and a covariance between the first video frame and the second video frame.

[0192] In an embodiment of the present invention, the target object information acquisition module 407 is used to:

[0193] Calculate the similarity between the feature vector to be detected and the feature vector in the target object image library, and use the feature vector ranked before a preset number of bits in similarity as the target feature vector;

[0194] Acquire a target object image corresponding to the target feature vector from the target object image library as an initial target object image;

[0195] Classifying the initial target object image to obtain a classification result, and calculating an inter-class variance value corresponding to the initial target object image according to the classification result, the similarity corresponding to the initial target object image, and the preset number of bits;

[0196] Determining a maximum inter-class variance value from among the inter-class variance values;

[0197] When the maximum inter-class variance exceeds a preset threshold, the initial target object image is used as a final target object image, identity information corresponding to the final target object image is acquired, and the final target object image and the identity information are output.

[0198] In an embodiment of the present invention, the target object information acquisition module 407 is used to:

[0199] Classifying the initial target object image to obtain a classification result; wherein the classification result includes a true class and a false class;

[0200] Calculate the true class ratio according to the number of the initial target object images of the true class and the preset number of bits, and calculate the false class ratio according to the number of the initial target object images of the false class and the preset number of bits;

[0201] Calculating a true class mean value according to the similarity corresponding to the initial target object image of the true class, and calculating a false class mean value according to the similarity corresponding to the initial target object image of the false class;

[0202] The inter-class variance value is calculated according to the true class proportion, the false class proportion, the true class mean and the false class mean.

[0203] In an embodiment of the present invention, the target object at least includes a target person.

[0204] In an embodiment of the present invention, a video file to be detected and a target object image library are obtained, wherein the target object image library may include target object information and feature vectors corresponding to the target object, the scene variability score of the video frame in the video file to be detected is calculated, when the scene variability score of the video frame is higher than a preset scene variability score, the video frame with a scene variability score higher than the preset scene variability score is used as a target video frame, the video file to be detected is cut according to the target video frame to obtain a plurality of shots, wherein the shots are video frames in the video file to be detected in which the background does not change and the movement of the object is continuous, key frames are respectively determined from the shots of the video file to be detected, and features are extracted from the key frames of the shots to obtain feature vectors to be detected, and then, a feature vector matching the feature vector to be detected can be determined from the target object image library as a target feature vector, and then the target object information of the target object corresponding to the target feature vector can be obtained, thereby realizing the detection and recognition of the target object in the video. In the fields of information exchange, etc., the embodiments of the present invention can efficiently and accurately determine whether there is a target object in the video file to be detected, such as a target person or important item, for detection and identification, and then obtain the target object information corresponding to the target object, and then perform corresponding processing based on the results of detection and identification. For example, it can assist supervisors in performing video content detection on the video file to be detected, thereby playing a role in maintaining safety.

[0205] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0206] An embodiment of the present invention further provides a non-volatile readable storage medium, in which one or more modules (programs) are stored. When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present invention.

[0207] The embodiments of the present invention provide one or more machine-readable media on which instructions are stored, and when executed by one or more processors, an electronic device executes one or more of the methods described in the above embodiments. In the embodiments of the present invention, the electronic device includes various types of devices such as terminal devices and servers (clusters).

[0208] The embodiments of the present disclosure may be implemented as a device configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the device may include electronic devices such as terminal devices and servers (clusters). Figure 5 An exemplary apparatus 1100 that can be used to implement various embodiments described in the present invention is schematically shown.

[0209] For one embodiment, Figure 5 An exemplary apparatus 1100 is shown having one or more processors 1102, a control module (chip set) 1104 coupled to at least one of the (one or more) processors 1102, a memory 1106 coupled to the control module 1104, a non-volatile memory (NVM) / storage device 1108 coupled to the control module 1104, one or more input / output devices 1110 coupled to the control module 1104, and a network interface 1112 coupled to the control module 1104.

[0210] The processor 1102 may include one or more single-core or multi-core processors, and the processor 1102 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 1100 can be used as a terminal device, server (cluster), etc. described in the embodiments of the present invention.

[0211] In some embodiments, the apparatus 1100 may include one or more computer-readable media (e.g., memory 1106 or NVM / storage device 1108) having instructions 1114 and one or more processors 1102 combined with the one or more computer-readable media and configured to execute the instructions 1114 to implement a module to perform the actions described in the present disclosure.

[0212] For one embodiment, the control module 1104 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 1102 and / or any suitable device or component in communication with the control module 1104 .

[0213] The control module 1104 may include a memory controller module to provide an interface to the memory 1106. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0214] The memory 1106 may be used, for example, to load and store data and / or instructions 1114 for the device 1100. For one embodiment, the memory 1106 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 1106 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0215] For one embodiment, control module 1104 may include one or more input / output controllers to provide an interface to NVM / storage device 1108 and input / output device(s) 1110 .

[0216] For example, NVM / storage 1108 may be used to store data and / or instructions 1114. NVM / storage 1108 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0217] NVM / storage device 1108 may include storage resources that are physically part of the device on which apparatus 1100 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 1108 may be accessed via input / output device(s) 1110 over a network.

[0218] (One or more) input / output devices 1110 may provide an interface for the apparatus 1100 to communicate with any other appropriate device, and the input / output device 1110 may include a communication component, an audio component, a sensor component, etc. The network interface 1112 may provide an interface for the apparatus 1100 to communicate through one or more networks, and the apparatus 1100 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0219] For one embodiment, at least one of the processor(s) 1102 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 1104. For one embodiment, at least one of the processor(s) 1102 may be packaged together with the logic of one or more controllers of the control module 1104 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 1102 may be integrated on the same die with the logic of one or more controllers of the control module 1104. For one embodiment, at least one of the processor(s) 1102 may be integrated on the same die with the logic of one or more controllers of the control module 1104 to form a system-on-chip (SoC).

[0220] In various embodiments, the device 1100 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the device 1100 may have more or fewer components and / or a different architecture. For example, in some embodiments, the device 1100 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0221] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information, etc. are stored in a memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.

[0222] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0223] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable video target object recognition terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable video target object recognition terminal device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0224] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable video target object recognition terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0225] These computer program instructions can also be loaded onto a computer or other programmable terminal device for identifying a target object in a video, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions executed on the computer or other programmable terminal device for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0226] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0227] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0228] The above is a detailed introduction to a method and device for identifying a target object in a video, an electronic device and a storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a person skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for identifying a target object in a video, characterized in that: The method comprises: Acquire a video file to be detected and a target object image library; wherein the target object image library includes target object information and feature vectors corresponding to the target object; Calculating the scene changeability score of the video frame in the video file to be detected; When the scene changeability score of the video frame is higher than a preset scene changeability score, taking the video frame having the scene changeability score higher than the preset scene changeability score as a target video frame; The video file to be detected is cut according to the target video frame to obtain a plurality of shots; wherein the shots are video frames in which the background of the video file to be detected does not change and the movements of the object are continuous; Determine key frames respectively from the shots of the video file to be detected; Extracting features from the key frames of the shots respectively to obtain feature vectors to be detected; A feature vector matching the feature vector to be detected is determined from the target object image library as a target feature vector, and target object information of the target object corresponding to the target feature vector is acquired.

2. The method according to claim 1, characterized in that The calculating the scene changeability score of the video frame in the video file to be detected includes: Obtaining image size information of the current video frame in the video file to be detected; Acquire the color gamut image information and contour image information of the current video frame to form a four-dimensional matrix of the current video frame, and acquire the color gamut image information and contour image information of a video frame previous to the current video frame to form a four-dimensional matrix of the previous video frame; The scene changeability score of the current video frame is calculated according to the image size information of the current video frame, the four-dimensional matrix of the current video frame and the four-dimensional matrix of the previous video frame.

3. The method according to claim 1, characterized in that: The step of respectively determining key frames from the shots of the video file to be detected includes: Calculating the difference scores between the video frames in the shot of the video file to be detected; Using the difference score of the shot as the ordinate and the serial number of the video frame of the shot as the abscissa to draw a difference curve of the shot; Performing smoothing on the difference curve; A valley value is determined from the smoothed difference curve, and a video frame corresponding to a sequence number corresponding to the valley value is used as a key frame of the shot.

4. The method according to claim 3, characterized in that The calculating the difference scores between the video frames in the shots of the video file to be detected includes: Acquire a first video frame and a second video frame adjacent to each other in the shot of the video file to be detected; Calculating an average value and a variance of the first video frame, an average value and a variance of the second video frame, and a covariance of the first video frame and the second video frame; A dissimilarity score between video frames in the shot is calculated based on a mean value and a variance of the first video frame, a mean value and a variance of the second video frame, and a covariance between the first video frame and the second video frame.

5. The method according to claim 1, characterized in that The target object information includes a target object image and identity information corresponding to the target object, and determining a feature vector matching the feature vector to be detected from the target object image library as a target feature vector, and obtaining target object information of the target object corresponding to the target feature vector, includes: Calculate the similarity between the feature vector to be detected and the feature vector in the target object image library, and use the feature vector ranked before a preset number of bits in similarity as the target feature vector; Acquire a target object image corresponding to the target feature vector from the target object image library as an initial target object image; Classifying the initial target object image to obtain a classification result, and calculating an inter-class variance value corresponding to the initial target object image according to the classification result, the similarity corresponding to the initial target object image, and the preset number of bits; Determining a maximum inter-class variance value from among the inter-class variance values; When the maximum inter-class variance exceeds a preset threshold, the initial target object image is used as a final target object image, identity information corresponding to the final target object image is acquired, and the final target object image and the identity information are output.

6. The method according to claim 5, characterized in that The step of classifying the initial target object image to obtain a classification result, and calculating the inter-class variance value corresponding to the initial target object image according to the classification result, the similarity corresponding to the initial target object image, and the preset number of bits, includes: Classifying the initial target object image to obtain a classification result; wherein the classification result includes a true class and a false class; Calculate the true class ratio according to the number of the initial target object images of the true class and the preset number of bits, and calculate the false class ratio according to the number of the initial target object images of the false class and the preset number of bits; Calculating a true class mean value according to the similarity corresponding to the initial target object image of the true class, and calculating a false class mean value according to the similarity corresponding to the initial target object image of the false class; The inter-class variance value is calculated according to the true class proportion, the false class proportion, the true class mean and the false class mean.

7. The method according to any one of claims 1 to 6, characterized in that: The target objects at least include target persons.

8. A device for identifying a target object in a video, characterized in that: The device comprises: A data acquisition module, used to acquire a video file to be detected and a target object image library; wherein the target object image library includes target object information and feature vectors corresponding to the target object; A scene changeability score calculation module, used to calculate the scene changeability score of the video frame in the video file to be detected; a target video frame determining module, configured to, when the scene changeability score of the video frame is higher than a preset scene changeability score, use the video frame having the scene changeability score higher than the preset scene changeability score as a target video frame; A shot cutting module is used to cut the video file to be detected according to the target video frame to obtain a plurality of shots; wherein the shots are video frames in which the background of the video file to be detected does not change and the action of the object is continuous; A key frame determination module, used to determine key frames from the shots of the video file to be detected; A feature vector extraction module, used for extracting features from the key frames of the shots to obtain feature vectors to be detected; The target object information acquisition module is used to determine a feature vector matching the feature vector to be detected from the target object image library as a target feature vector, and acquire target object information of the target object corresponding to the target feature vector.

9. An electronic device, characterized in that: include: processor; and A memory having executable codes stored thereon, which, when executed, enables the processor to execute the method for identifying a target object in a video as described in any one of claims 1 to 7.

10. One or more machine-readable media having executable codes stored thereon, which, when executed, enable a processor to execute the method for identifying a target object in a video according to any one of claims 1 to 7.