Video identifier detection method and device, electronic equipment and computer storage medium
By performing disframing and timing feature analysis on video data, the confidence of the video identification detection results is calculated, and the problem of low detection accuracy caused by single-frame analysis in the prior art is solved, and the accuracy of video identification detection is improved.
Patent Information
- Application Number
- CN202411050567.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-12
AI Technical Summary
The existing video identification detection methods mainly rely on single-frame image analysis, resulting in low accuracy of detection results and lack of effective solutions.
By deframed video data, obtaining image sequences, and identifying and detecting each frame of images, calculating the confidence identified on the timing feature sequence, and using timing information to assist in verifying the detection results, increasing the accuracy of the detection results.
By analyzing the changes in the identification timing characteristics in the video, the accuracy and certainty of the identification detection results are improved, and the accuracy of the video identification detection is enhanced.
Smart Images

Figure CN118942011A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of image data processing, and in particular to a method, device, electronic device and computer storage medium for detecting a logo of a video. Background Art
[0002] In the field of video processing and analysis, the detection of logos such as station logos, watermarks or logos is a critical task. Video logo detection helps in many applications such as copyright protection, brand recognition and content review.
[0003] Current video identification detection methods usually only analyze a single frame of image and rely on the detection results of a single frame of image, resulting in a problem that the detection results are not accurate enough.
[0004] With respect to the above-mentioned problems existing in the related technologies, no effective solutions have been found so far. Summary of the invention
[0005] The present application provides a video identification detection method, device, electronic device and computer storage medium to solve the above-mentioned technical problems existing in the related art.
[0006] According to an embodiment of the present application, a method for detecting identifiers in a video is provided, comprising: obtaining video data to be processed, de-framing the video data to obtain an image sequence; performing identifier detection on each frame of the image sequence to obtain a plurality of identifier detection frame sequences, wherein each identifier detection frame sequence corresponds to an identifier; obtaining a temporal feature sequence corresponding to the identifier detection frame sequence; and calculating, based on the temporal feature sequence, a confidence level of the identifier on the corresponding identifier detection frame sequence.
[0007] Furthermore, performing logo detection on each frame image in the image sequence to obtain a plurality of logo detection frame sequences includes: identifying logos in each frame image in the image sequence to obtain logo detection frames and logo categories corresponding to the logo detection frames; classifying the logo detection frames according to the logo categories to obtain logo detection frame sequences corresponding to each category of logos.
[0008] Furthermore, after obtaining the identification detection frame sequences corresponding to each type of identification, the method also includes: obtaining the image frame number corresponding to the current detection frame sequence corresponding to the current identification; dividing the current detection frame sequence according to the continuity of the image frame number to obtain a sub-detection frame sequence; and using the sub-detection frame sequence as the identification detection frame sequence corresponding to the current identification.
[0009] Furthermore, obtaining the timing feature sequence corresponding to the identification detection frame sequence includes: for each identification detection frame in the identification detection frame sequence, obtaining the vertex coordinates of the identification detection frame; calculating the center position of the identification detection frame according to the vertex coordinates to form a position sequence corresponding to the identification detection frame sequence, and the timing feature sequence includes the position sequence.
[0010] Furthermore, after forming the position sequence corresponding to the identification detection frame sequence, the method also includes: for each identification detection frame in the identification detection frame sequence, obtaining the first center position of the current detection frame from the position sequence, and obtaining the second center position of the next detection frame; calculating the position difference between the second center position and the first center position; using the position difference as the moving speed of the current detection frame; recording the moving speed of each identification detection frame to form a speed sequence corresponding to the identification detection frame sequence, and the timing feature sequence includes the position sequence and the speed sequence.
[0011] Furthermore, the time series feature sequence includes a position sequence, a velocity sequence and an acceleration sequence. According to the time series feature sequence, calculating the confidence of the mark on the corresponding mark detection frame sequence includes: respectively calculating the first variance of the position sequence, the second variance of the velocity sequence, and the third variance of the acceleration sequence; using the first variance, the second variance and the third variance to calculate the confidence of the mark on the corresponding mark detection frame sequence.
[0012] Furthermore, using the first variance, the second variance and the third variance to calculate the confidence of the mark on the corresponding mark detection frame sequence includes: normalizing the first variance, the second variance and the third variance respectively to obtain a first normalized variance, a second normalized variance and a third normalized variance; and performing weighted summation of the first normalized variance, the second normalized variance and the third normalized variance to obtain the confidence of the mark on the corresponding mark detection frame sequence.
[0013] According to another embodiment of the present application, a video identification detection device is provided, including a deframing module, which is used to obtain video data to be processed, defram the video data, and obtain an image sequence; a detection module, which is used to perform identification detection on each frame image in the image sequence, and obtain a plurality of identification detection frame sequences, wherein each identification detection frame sequence corresponds to an identification; an acquisition module, which is used to obtain a time series feature sequence corresponding to the identification detection frame sequence; and a calculation module, which is used to calculate the confidence of the identification on the corresponding identification detection frame sequence based on the time series feature sequence.
[0014] Furthermore, the detection module includes an identification unit, which is used to identify the logo in each frame image in the image sequence to obtain a logo detection frame and a logo category corresponding to the logo detection frame; and a classification unit, which is used to classify the logo detection frame according to the logo category to obtain a logo detection frame sequence corresponding to each category of logo.
[0015] Furthermore, the detection module also includes a segmentation unit, which is used to obtain the image frame number corresponding to the current detection frame sequence corresponding to the current identifier; segment the current detection frame sequence according to the continuity of the image frame number to obtain a sub-detection frame sequence; and use the sub-detection frame sequence as the identifier detection frame sequence corresponding to the current identifier.
[0016] Furthermore, the acquisition module includes a first acquisition unit, which is used to acquire the vertex coordinates of the identification detection frame for each identification detection frame in the identification detection frame sequence; a first calculation unit, which is used to calculate the center position of the identification detection frame according to the vertex coordinates to form a position sequence corresponding to the identification detection frame sequence, and the time series feature sequence includes the position sequence.
[0017] Furthermore, the acquisition module includes a second acquisition unit, which is used to obtain the first center position of the current detection frame from the position sequence and the second center position of the next detection frame for each identification detection frame in the identification detection frame sequence; a second calculation unit, which is used to calculate the position difference between the second center position and the first center position; use the position difference as the moving speed of the current detection frame; record the moving speed of each identification detection frame to form a speed sequence corresponding to the identification detection frame sequence, and the time series feature sequence includes the position sequence and the speed sequence.
[0018] Furthermore, the calculation module includes a third calculation unit, which is used to respectively calculate the first variance of the position sequence, the second variance of the velocity sequence, and the third variance of the acceleration sequence; and use the first variance, the second variance and the third variance to calculate the confidence of the mark on the corresponding mark detection box sequence.
[0019] Furthermore, the third calculation unit includes a normalization processing subunit, which is used to normalize the first variance, the second variance and the third variance respectively to obtain a first normalized variance, a second normalized variance and a third normalized variance; and perform weighted summation on the first normalized variance, the second normalized variance and the third normalized variance to obtain the confidence of the mark on the corresponding mark detection box sequence.
[0020] According to another embodiment of the present application, a computer storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned device embodiments when running.
[0021] According to another embodiment of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: the memory is used to store computer programs; the processor is used to execute the steps in the above method by running the program stored in the memory.
[0022] According to yet another embodiment of the present application, a computer program product comprising instructions is also provided, which enables the computer to execute the steps in the above method when the computer program product is executed on the computer.
[0023] Through the embodiments of the present application, video data to be processed is obtained, and the video data is deframed to obtain an image sequence; identification detection is performed on each frame image in the image sequence to obtain a plurality of identification detection frame sequences, wherein each identification detection frame sequence corresponds to an identification; a timing feature sequence corresponding to the identification detection frame sequence is obtained; based on the timing feature sequence, the confidence of the identification on the corresponding identification detection frame sequence is calculated, and by analyzing the characteristic changes of the identification in the video in the timing, the timing information of the video is fully utilized, and the confidence of the identification detection result in the timing is calculated, and the identification detection result is auxiliary verified to increase the certainty in the timing, thereby increasing the accuracy of the identification detection result in the video. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0025] Figure 1 is a hardware structure block diagram of a computer according to an embodiment of the present application;
[0026] Figure 2 is a flow chart of a method for detecting a video identifier in an embodiment of the present application;
[0027] Figure 3 is a schematic diagram of the structure after the image sequence is classified and segmented in an embodiment of the present application;
[0028] Figure 4 It is a structural block diagram of a video identification detection device according to an embodiment of the present application. DETAILED DESCRIPTION
[0029] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.
[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0031] Example 1
[0032] The method embodiment provided in the first embodiment of the present application can be executed in a mobile phone, a computer, a tablet or a similar computing device. Taking running on a computer as an example, Figure 1 is a hardware structure block diagram of a computer in an embodiment of the present application. Figure 1 As shown, the computer may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the computer may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.
[0033] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a video identification detection method in an embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0034] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0035] In this embodiment, a method for detecting a video mark is provided. Figure 2 is a flow chart of a video identification detection method according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:
[0036] Step S10, obtaining the video data to be processed, de-framing the video data, and obtaining an image sequence;
[0037] Step S20, performing identification detection on each frame of the image sequence to obtain a plurality of identification detection frame sequences, wherein each identification detection frame sequence corresponds to one identification;
[0038] In this embodiment, a target detection model can be used to detect the logo of the image, wherein the logo includes logos such as station logos, watermarks or logos. Select a suitable target detection model architecture, such as YOLO, Faster R-CNN, SSD (SingleShot MultiBox Detector), etc. Use the labeled data set to train the selected model. The training goal is to enable the model to recognize the logo in the image and give the logo category C, the logo category confidence Pc and the detection box of the logo on the image, specifically the four vertex coordinates of the detection box.
[0039] Each frame image I in the image sequence is input into the trained target detection model, and the target detection model outputs the identification category C, the identification category confidence Pc and the detection box of the identification on the image. The detection results of each frame image are recorded to obtain several identification detection box sequences, where each identification detection box sequence corresponds to one identification.
[0040] Step S30, obtaining a temporal feature sequence corresponding to the identification detection frame sequence;
[0041] Step S40, calculating the confidence of the mark on the corresponding mark detection frame sequence according to the temporal feature sequence.
[0042] For each marker detection frame sequence, a time series feature sequence corresponding to the marker detection frame sequence is obtained. In this embodiment, the time series feature sequence can reflect the change of the time series feature of the marker on the image sequence. According to the time series feature sequence, the confidence of the marker on the corresponding marker detection frame sequence is calculated.
[0043] Through the above steps, the video data to be processed is obtained, the video data is deframed, and an image sequence is obtained; the identification detection is performed on each frame image in the image sequence to obtain a plurality of identification detection frame sequences, wherein each identification detection frame sequence corresponds to an identification; the time series feature sequence corresponding to the identification detection frame sequence is obtained; according to the time series feature sequence, the confidence of the identification on the corresponding identification detection frame sequence is calculated, and by analyzing the characteristic changes of the identification in the video in the time series, the time series information of the video is fully utilized, and the confidence of the identification detection result in the time series is calculated, and the identification detection result is auxiliary verified to increase the certainty in the time series, thereby increasing the accuracy of the identification detection result in the video.
[0044] In an example of the present embodiment, identification detection is performed on an image sequence of 100 frames of images to obtain two identification detection frame sequences corresponding to identification A: identification detection frame sequence 1 corresponds to identification A, including frames 1 to 40, and identification detection frame sequence 2 corresponds to identification A, including frames 51 to 100, and identification A is not detected in frames 41 to 50; or a identification detection frame sequence corresponding to identification A is obtained: identification detection frame sequence 3 corresponds to identification A, including frames 1 to 100, and so on.
[0045] In this embodiment, one frame of image may also include multiple logos. For example, in a scenario where a certain station logo broadcasts the content of another station logo, one frame of image includes two station logo logos. In the above example, logo A corresponds to two logo detection frame sequences (frames 1 to 40, and frames 51 to 100, respectively), and logo B corresponds to one logo detection frame sequence (for example, frame 1 to frame 50), that is, there are two station logo logos in the images from frames 1 to 40.
[0046] In one implementation of this embodiment, the identification detection is performed on each frame of the image sequence to obtain a plurality of identification detection frame sequences including:
[0047] S21, identifying a logo in each frame of the image sequence, and obtaining a logo detection frame and a logo category corresponding to the logo detection frame;
[0048] S22, classifying the logo detection frames according to the logo categories to obtain logo detection frame sequences corresponding to the various logo categories.
[0049] In this embodiment, for a scene in which a video data may include multiple logos, for example, a mixed edited video data, the front part is the source of station logo A, the middle part is the network source without station logo, and the back part is the source of station logo B, then a set of logo detection frames of different types of logos will be detected, and the detection results will be classified according to the logo category to obtain a logo detection frame sequence corresponding to each category of logo. Subsequently, confidence calculation is performed according to the category, and the confidence of each category of logo on the corresponding logo detection frame sequence can be accurately calculated.
[0050] Perform target detection on each frame in the image sequence, identify the identification detection frame of the identification in the image, the identification category and the identification category confidence corresponding to each identification detection frame, classify the identification detection frame according to the identification category, and obtain the identification detection frame sequence corresponding to each type of identification. In another example, the image sequence of 100 frames is subjected to identification detection, and the identification detection frame sequence a1 corresponding to the identification A from the 1st frame to the 50th frame is obtained; the identification B corresponds to the identification detection frame sequence b1 from the 51st frame to the 100th frame. Then, for identification A, the temporal feature sequence corresponding to the identification detection frame sequence a1 is obtained, and the confidence of identification A on the identification detection frame sequence a1 is calculated, that is, the confidence of identification A on the image frame from the 1st frame to the 50th frame is obtained. For identification B, the temporal feature sequence corresponding to the identification detection frame sequence b1 is obtained, and the confidence of identification B on the identification detection frame sequence b1 is calculated, that is, the confidence of identification B on the image frame from the 51st frame to the 100th frame is obtained.
[0051] In this embodiment, for a scene in which a video data includes multiple identifiers, the identifier detection frames are classified according to the identifier category to obtain a sequence of identifier detection frames corresponding to each category of identifiers. When analyzing the changes in the temporal features in the sequence, the consistency of the categories is ensured, and the temporal features are compared for the identifiers of the same category to avoid mixed feature comparisons between identifiers of different categories, which may lead to inaccurate comparison results.
[0052] In this implementation, after obtaining the marker detection frame sequences corresponding to the various markers, the method further includes:
[0053] S23, for a current detection frame sequence corresponding to the current identifier, obtaining an image frame sequence number corresponding to the current detection frame sequence;
[0054] S24, dividing the current detection frame sequence according to the continuity of the image frame sequence numbers to obtain sub-detection frame sequences;
[0055] S25, using the sub-detection frame sequence as the identification detection frame sequence corresponding to the current identification.
[0056] In this embodiment, for the situation that there may be a certain time period in the middle where no mark is detected on the video, refer to Figure 3 , for the logo detection frame sequence of station logo A (frames 1 to 13 followed by frames 20 to 37), that is, the image frame numbers corresponding to the logo detection frame sequence of station logo A are not continuous, then the current detection frame sequence is segmented according to the continuity of the image frame numbers to obtain sub-detection frame sequences (that is, as shown in Figure 3 The segment after segmentation: the 1st frame detection frame to the 13th frame detection frame, and the 20th frame detection frame to the 37th frame detection frame), then the station logo A includes two identification detection frame sequences, namely the two sub-detection frame sequences mentioned above, wherein the image frame numbers corresponding to the sub-detection frame sequences are continuous.
[0057] In this embodiment, a sub-detection frame sequence is obtained by segmentation. For each sub-detection frame sequence, the confidence of the identification on different segments in the video can be calculated separately, which helps to distinguish the confidence differences of the identification on different segments in the video, thereby improving the overall confidence assessment accuracy.
[0058] In another implementation of this embodiment, obtaining the temporal feature sequence corresponding to the identification detection frame sequence includes:
[0059] S31, for each marker detection frame in the marker detection frame sequence, obtaining vertex coordinates of the marker detection frame;
[0060] S32, calculating the center position of the marker detection frame according to the vertex coordinates to form a position sequence corresponding to the marker detection frame sequence, wherein the temporal feature sequence includes the position sequence.
[0061] In this embodiment, refer to Figure 3After performing logo detection on the image sequence and obtaining three logo detection frame sequences (the first detection frame sequence (1st frame detection frame to 13th frame detection frame) and the second detection frame sequence (20th frame detection frame to 37th frame detection frame) corresponding to the station logo A, and the third detection frame sequence (51st frame detection frame to 84th frame detection frame) corresponding to the station logo B), for each logo detection frame sequence, obtain the four vertex coordinates of each logo detection frame in the logo detection frame sequence, calculate the center point coordinates of each logo detection frame according to the vertex coordinates, as the center position of the logo detection frame, record the center position x = (x, y) of each logo detection frame, and form a position sequence {x 1 ,x 2 ,…,x n}, where the start and end frame numbers 1 and n are calculated according to the segmented fragments, that is, each identification detection box sequence starts from 1, and n is the sequence length.
[0062] In this embodiment, the position change of the logo on the sequence can be obtained by using the position sequence corresponding to the identification detection frame sequence. Taking the station logo as an example, the actual station logo is usually fixed at a fixed position. The confidence of the specific movement law of the logo on the corresponding logo detection frame sequence can be calculated through the position sequence, which is a time-series feature sequence. Based on the confidence, it can be analyzed whether the station logo position in the position sequence is concentrated in a fixed position. If the station logo position in the position sequence is concentrated in a fixed position, it means that the station logo detection results of the multiple frames of images are normal; if the station logo position in the position sequence is relatively scattered, it means that there are misdetected station logos in the station logo detection results of the multiple frames of images. For example, other graphics similar to the station logo in the image are mistakenly detected as the station logo. Further screening can be performed to improve the accuracy of the detection results.
[0063] In an application scenario of this embodiment, after forming the position sequence corresponding to the identification detection frame sequence, the method further includes:
[0064] S33, for each identified detection frame in the identified detection frame sequence, obtaining a first center position of the current detection frame from the position sequence, and obtaining a second center position of the next detection frame;
[0065] S34, calculating a position difference between the second center position and the first center position;
[0066] S35, using the position difference as the moving speed of the current detection frame;
[0067] S36, recording the moving speed of each marker detection frame to form a speed sequence corresponding to the marker detection frame sequence, wherein the time series feature sequence includes the position sequence and the speed sequence.
[0068] In this embodiment, in addition to the position sequence, the speed sequence is also considered, that is, the time series feature sequence includes the position sequence and the speed sequence.
[0069] For each marker detection frame in the marker detection frame sequence, the moving speed of the marker detection frame is calculated by the position difference of the marker detection frame between adjacent frames, that is, the first center position of the current marker detection frame is obtained from the position sequence, and the second center position of the next marker detection frame is obtained, and the position difference between the second center position and the first center position is calculated, and the position difference is used as the moving speed of the current detection frame. The moving speed of each marker detection frame is recorded to form a speed sequence corresponding to the marker detection frame sequence {v 1 ,v 2 ,…,v n-1}. Among them, v i =x i+1 -x i .
[0070] In this embodiment, by identifying the speed sequence corresponding to the detection frame sequence, the change in the moving speed of the marker on the sequence can be obtained. If the speed tends to 0, the confidence tends to 1, that is, the more accurate the marker detection result is, the more normal it is that the marker is fixed at a fixed position.
[0071] In another application scenario of this embodiment, the time series feature sequence includes a position sequence, a velocity sequence and an acceleration sequence.
[0072] For each marker detection frame in the marker detection frame sequence, the acceleration of the marker detection frame is calculated by the speed difference of the marker detection frame between adjacent frames, that is, the first moving speed of the current marker detection frame is obtained from the speed sequence, and the second moving speed of the next marker detection frame is obtained, and the speed difference between the second moving speed and the first moving speed is calculated, and the speed difference is used as the acceleration of the current detection frame. The acceleration of each marker detection frame is recorded to form an acceleration sequence corresponding to the marker detection frame sequence {a 1 ,a 2 ,…,a n-2}. Among them, a i =v i+1 -v i .
[0073] In this embodiment, by identifying the acceleration sequence corresponding to the detection frame sequence, the change in the moving speed of the marker on the sequence can be obtained. If the acceleration tends to 0, the confidence tends to 1, that is, the more accurate the marker detection result is, the more normal it is that the marker is fixed in a fixed position.
[0074] In another implementation of this embodiment, the time series feature sequence includes a position sequence, a speed sequence, and an acceleration sequence, and according to the time series feature sequence, calculating the confidence of the mark on the corresponding mark detection frame sequence includes:
[0075] S41, respectively calculating a first variance of the position sequence, a second variance of the velocity sequence, and a third variance of the acceleration sequence;
[0076] S42: Calculate the confidence of the mark on the corresponding mark detection box sequence by using the first variance, the second variance and the third variance.
[0077] Calculate the position sequence {x 1 ,x 2 ,…,x n The first variance of Speed sequence {v 1 ,v 2 ,…,v n-1 The second variance of And the acceleration sequence {a 1 ,a 2 ,…,a n-2} third party difference First Variance Second variance Difference with third party The confidence of the logo on the corresponding logo detection box sequence is calculated.
[0078] The variance can reflect the degree of dispersion or concentration of the time series feature sequence. In this embodiment, the more concentrated the position, speed and acceleration are, the closer the confidence is to 1, which means that the more fixed the position of the mark detected in the video or video clip, the more constant the speed and acceleration are, and the higher the accuracy of the mark detection result.
[0079] In this implementation, using the first variance, the second variance, and the third variance to calculate the confidence of the marker on the corresponding marker detection frame sequence includes:
[0080] S421, normalizing the first variance, the second variance, and the third variance respectively to obtain a first normalized variance, a second normalized variance, and a third normalized variance;
[0081] S422, performing weighted summation on the first normalized variance, the second normalized variance, and the third normalized variance to obtain a confidence level of the marker on the corresponding marker detection box sequence.
[0082] In this embodiment, the normalized variance can be calculated by the following normalization formula: Among them, σ 2is the variance of the corresponding sequence, σ 2 T-max is a preset maximum threshold, exemplarily σ 2 T-max is set to 4.
[0083] Calculate the first normalized variance of the position series Calculate the second normalized variance of the velocity series Calculate the third normalized variance of the acceleration series
[0084] Perform weighted summation of the above normalized variances Obtain the confidence p of the corresponding identification detection sequence in time sequence t . Wherein, wx, wv and wa correspond to the weights of the first normalized variance, the second normalized variance and the third normalized variance respectively. Optionally, the sum of wx, wv and wa is 1.
[0085] In this embodiment, since the variance values of the three features of position, velocity and acceleration may differ greatly, the variance is normalized to eliminate the scale difference, so as to facilitate the subsequent weighted summation.
[0086] This embodiment improves the accuracy and reliability of logo detection by fusing the position, speed and acceleration information of the logo in the video and calculating the temporal confidence of logo detection in the video. Specifically, by analyzing the image of each frame in the video, the center position of the logo is detected, and then the position, speed and acceleration sequence between consecutive frames is calculated. Finally, the confidence of the logo in time is calculated by weighted summation of the statistical features of these sequences.
[0087] The solution of this embodiment provides an implementation process of a method for detecting a video identifier, including:
[0088] 1. Video preprocessing: Import the video file to be processed and deframe the video to obtain an image sequence.
[0089] 2. Logo detection: Perform logo detection on each frame after decoding, and identify the logo category, category confidence, and location. Divide the recognition results by category, and then process them by category. Output the detection box sequence in each category of icon.
[0090] 3. Position sequence generation: Split the detection frame sequence of each category, because there may be a situation where the logo cannot be detected in the video at a certain time period in the middle; calculate the center point coordinates of each sequence detection frame of each category logo as the position sequence.
[0091] 4. Speed sequence generation: The speed sequence is calculated by the position difference between adjacent frames.
[0092] 5. Acceleration sequence generation: The acceleration sequence is calculated by the difference in speed between adjacent frames.
[0093] 6. Statistical feature calculation: Calculate the normalized variance of the position sequence. Calculate the normalized variance of the velocity sequence. Calculate the normalized variance of the acceleration sequence.
[0094] 7. Confidence calculation: Perform weighted summation based on statistical characteristics to obtain the confidence of the corresponding station logo detection sequence in time sequence.
[0095] 8. Output the results.
[0096] This embodiment makes full use of the temporal characteristics of the video logo. First, a pre-trained logo detection model is used to detect the decoded video frame by frame to obtain the temporal sequence of the logo detection frame. The logo's position, speed, and acceleration are calculated and analyzed to obtain the confidence level of the logo in the timing, thereby improving the accuracy of detection.
[0097] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0098] Example 2
[0099] In this embodiment, a video identification detection device is also provided to implement the above-mentioned embodiments and preferred implementation modes, which have been described and will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0100] Figure 4 is a structural block diagram of a video identification detection device according to an embodiment of the present application, such as Figure 4 As shown, the device comprises:
[0101] The de-framing module 41 is used to obtain the video data to be processed, de-frame the video data, and obtain an image sequence;
[0102] A detection module 42, configured to perform a mark detection on each frame of the image sequence to obtain a plurality of mark detection frame sequences, wherein each mark detection frame sequence corresponds to one mark;
[0103] An acquisition module 43 is used to acquire a temporal feature sequence corresponding to the identification detection frame sequence;
[0104] The calculation module 44 is used to calculate the confidence of the mark on the corresponding mark detection box sequence according to the temporal feature sequence.
[0105] Optionally, the detection module includes an identification unit, which is used to identify the logo in each frame image in the image sequence to obtain a logo detection frame and a logo category corresponding to the logo detection frame; and a classification unit, which is used to classify the logo detection frame according to the logo category to obtain a logo detection frame sequence corresponding to each category of logo.
[0106] Optionally, the detection module also includes a segmentation unit, which is used to obtain the image frame number corresponding to the current detection frame sequence corresponding to the current identifier; segment the current detection frame sequence according to the continuity of the image frame number to obtain a sub-detection frame sequence; and use the sub-detection frame sequence as the identifier detection frame sequence corresponding to the current identifier.
[0107] Optionally, the acquisition module includes a first acquisition unit, used to acquire the vertex coordinates of the identification detection frame for each identification detection frame in the identification detection frame sequence; a first calculation unit, used to calculate the center position of the identification detection frame according to the vertex coordinates to form a position sequence corresponding to the identification detection frame sequence, and the timing feature sequence includes the position sequence.
[0108] Optionally, the acquisition module includes a second acquisition unit, which is used to obtain, for each identification detection frame in the identification detection frame sequence, the first center position of the current detection frame from the position sequence, and the second center position of the next detection frame; a second calculation unit, which is used to calculate the position difference between the second center position and the first center position; use the position difference as the moving speed of the current detection frame; record the moving speed of each identification detection frame to form a speed sequence corresponding to the identification detection frame sequence, and the timing feature sequence includes the position sequence and the speed sequence.
[0109] Optionally, the calculation module includes a third calculation unit, which is used to respectively calculate the first variance of the position sequence, the second variance of the velocity sequence, and the third variance of the acceleration sequence; and use the first variance, the second variance and the third variance to calculate the confidence of the mark on the corresponding mark detection box sequence.
[0110] Optionally, the third calculation unit includes a normalization processing subunit, which is used to normalize the first variance, the second variance and the third variance respectively to obtain a first normalized variance, a second normalized variance and a third normalized variance; and perform weighted summation of the first normalized variance, the second normalized variance and the third normalized variance to obtain the confidence of the mark on the corresponding mark detection box sequence.
[0111] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.
[0112] Example 3
[0113] An embodiment of the present application further provides a computer storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
[0114] Optionally, in this embodiment, the computer storage medium may be configured to store a computer program for performing the following steps:
[0115] S1, obtaining video data to be processed, de-framing the video data, and obtaining an image sequence;
[0116] S2, performing identification detection on each frame of the image sequence to obtain a plurality of identification detection frame sequences, wherein each identification detection frame sequence corresponds to one identification;
[0117] S3, obtaining a temporal feature sequence corresponding to the identification detection frame sequence;
[0118] S4, calculating the confidence of the mark on the corresponding mark detection frame sequence according to the temporal feature sequence.
[0119] Optionally, in this embodiment, the above-mentioned computer storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.
[0120] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0121] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0122] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:
[0123] S1, obtaining video data to be processed, de-framing the video data, and obtaining an image sequence;
[0124] S2, performing identification detection on each frame of the image sequence to obtain a plurality of identification detection frame sequences, wherein each identification detection frame sequence corresponds to one identification;
[0125] S3, obtaining a temporal feature sequence corresponding to the identification detection frame sequence;
[0126] S4, calculating the confidence of the mark on the corresponding mark detection frame sequence according to the temporal feature sequence.
[0127] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.
[0128] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0129] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0130] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0132] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0133] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a computer storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned computer storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.
[0134] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A video identification detection method, characterized in that: The method comprises: Acquire video data to be processed, and deframe the video data to obtain an image sequence; Performing identification detection on each frame of the image sequence to obtain a plurality of identification detection frame sequences, wherein each identification detection frame sequence corresponds to one identification; Obtaining a temporal feature sequence corresponding to the identification detection frame sequence; According to the temporal feature sequence, the confidence of the mark on the corresponding mark detection frame sequence is calculated.
2. The method according to claim 1, characterized in that Performing logo detection on each frame of the image sequence to obtain a plurality of logo detection frame sequences including: Identify the logo in each frame of the image sequence to obtain a logo detection frame and a logo category corresponding to the logo detection frame; The identification detection frames are classified according to the identification categories to obtain identification detection frame sequences corresponding to the various identification categories.
3. The method according to claim 2, characterized in that After obtaining the marker detection frame sequences corresponding to the various markers, the method further includes: For a current detection frame sequence corresponding to a current identifier, obtaining an image frame sequence number corresponding to the current detection frame sequence; The current detection frame sequence is divided according to the continuity of the image frame sequence numbers to obtain a sub-detection frame sequence; The sub-detection frame sequence is used as the identification detection frame sequence corresponding to the current identification.
4. The method according to claim 1, characterized in that Acquiring the temporal feature sequence corresponding to the identification detection frame sequence includes: For each marker detection frame in the marker detection frame sequence, obtaining vertex coordinates of the marker detection frame; The center position of the marker detection frame is calculated according to the vertex coordinates to form a position sequence corresponding to the marker detection frame sequence, and the temporal feature sequence includes the position sequence.
5. The method according to claim 4, characterized in that After forming the position sequence corresponding to the identification detection frame sequence, the method further includes: For each identified detection frame in the identified detection frame sequence, obtaining a first center position of the current detection frame from the position sequence, and obtaining a second center position of the next detection frame; calculating a position difference between the second center position and the first center position; Using the position difference as the moving speed of the current detection frame; The moving speed of each mark detection frame is recorded to form a speed sequence corresponding to the mark detection frame sequence, and the time series feature sequence includes the position sequence and the speed sequence.
6. The method according to claim 1, characterized in that The time series feature sequence includes a position sequence, a speed sequence, and an acceleration sequence. According to the time series feature sequence, calculating the confidence of the mark on the corresponding mark detection frame sequence includes: respectively calculating a first variance of the position sequence, a second variance of the velocity sequence, and a third variance of the acceleration sequence; The first variance, the second variance and the third variance are used to calculate the confidence of the mark on the corresponding mark detection box sequence.
7. The method according to claim 6, characterized in that Using the first variance, the second variance, and the third variance to calculate the confidence of the mark on the corresponding mark detection frame sequence includes: Normalizing the first variance, the second variance, and the third variance respectively to obtain a first normalized variance, a second normalized variance, and a third normalized variance; A weighted sum is performed on the first normalized variance, the second normalized variance, and the third normalized variance to obtain a confidence level of the marker on the corresponding marker detection box sequence.
8. A video identification detection device, characterized in that: include: A de-framing module, used to obtain the video data to be processed, de-frame the video data, and obtain an image sequence; A detection module, used to perform identification detection on each frame of the image sequence to obtain a plurality of identification detection frame sequences, wherein each identification detection frame sequence corresponds to one identification; An acquisition module, used to acquire a temporal feature sequence corresponding to the identification detection frame sequence; A calculation module is used to calculate the confidence of the mark on the corresponding mark detection box sequence according to the temporal feature sequence.
9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; wherein: Memory, used to store computer programs; A processor, configured to execute the video identification detection method according to any one of claims 1 to 7 by running a program stored in a memory.
10. A computer storage medium, characterized in that: The computer storage medium includes a stored program, wherein the program, when executed, executes the video identification detection method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Method and device for detecting video false alarm target
CN101739691A
Subtitle extraction method and device, computer equipment, readable storage medium and product
CN114648774A
Identification detection method and device, electronic equipment and storage medium
CN114842379A
Station caption detection method and related equipment
CN115471829A
Method and apparatus for determining interaction gesture, and electronic device
WO2023169282A1
