Identification code detection method and device, electronic device and storage medium

Abstract

The disclosed embodiments provide a method and device for detecting an identification code, an electronic device, and a storage medium. The method includes: obtaining a key frame to be identified in a target video segment of a video to be identified; obtaining a target contour and a fitting polygon of each key frame to be identified; locating a target identification point of the target contour of each key frame to be identified based on the target contour and the fitting polygon of each key frame to be identified; projecting the target identification point of the target contour of each key frame to be identified to a target common plane; and detecting a target identification code in the video to be identified based on the target identification point on the target common plane. Computer vision technology in the field of artificial intelligence can be used, and can be applied to video file storage scenarios.

Description

Technical Field

[0001] The present disclosure relates to the field of computer and communication technology, and in particular to an identification code detection method and device, an electronic device, and a computer-readable storage medium. Background Art

[0002] In the related art, there are two methods for detecting identification codes: feature point-based detection methods and deep neural network-based detection methods. However, the feature point-based detection methods in the related art have low accuracy, and the deep neural network-based detection methods require a large number of training samples, so they are slow and require more computing and storage resources. Summary of the invention

[0003] The embodiments of the present disclosure provide an identification code detection method and device, an electronic device, and a computer-readable storage medium, which can realize rapid and accurate detection of a target identification code in a video to be identified.

[0004] An embodiment of the present disclosure provides an identification code detection method, which includes: obtaining key frames to be identified in a target video segment of a video to be identified; obtaining a target contour and a fitting polygon of each key frame to be identified; locating a target identification point of the target contour of each key frame to be identified based on the target contour and the fitting polygon of each key frame to be identified; projecting the target identification point of the target contour of each key frame to be identified onto a target common plane; and detecting a target identification code in the video to be identified based on the target identification point on the target common plane.

[0005] The embodiment of the present disclosure provides an identification code detection device, which includes: a key frame acquisition unit to be identified, used to acquire the key frame to be identified in the target video segment of the video to be identified; a target contour polygon fitting unit, used to obtain the target contour and the fitting polygon of each key frame to be identified; a target identification point positioning unit, used to locate the target identification point of the target contour of each key frame to be identified according to the target contour and the fitting polygon of each key frame to be identified; a target identification point projection unit, used to project the target identification point of the target contour of each key frame to be identified to a target common plane; and a target identification code detection unit, used to detect the target identification code in the video to be identified according to the target identification point on the target common plane.

[0006] In some exemplary embodiments of the present disclosure, the target contour polygon fitting unit includes: a contour detection unit, used to detect the contour in each key frame to be identified; a contour fitting unit, used to fit the contour in each key frame to be identified, and obtain the fitting polygon of the contour in each key frame; a target contour determination unit, used to filter the contour in each key frame to be identified according to the contour in each key frame to be identified and its fitting polygon, and determine the target contour in each key frame to be identified from the contours of each key frame to be identified.

[0007] In some exemplary embodiments of the present disclosure, the contour detection unit includes: a color space conversion unit, used to perform color space conversion on each key frame to be identified to obtain a grayscale image of each key frame to be identified; an edge feature extraction unit, used to perform edge feature extraction on the grayscale image of each key frame to be identified to obtain an edge feature image of each key frame to be identified; and a contour extraction unit, used to detect contours in the edge feature image of each key frame to be identified.

[0008] In some exemplary embodiments of the present disclosure, the target contour determination unit includes: a target contour filtering unit, used to determine a contour that meets the following conditions as the target contour: the area of ​​the contour is greater than or equal to the contour threshold; the rotation angle of the minimum circumscribed rectangle of the contour is a right angle; the aspect ratio of the minimum circumscribed rectangle of the contour is a predetermined aspect ratio value; the number of sides of the fitting polygon of the contour is a set number of sides; the sides formed by three adjacent points of the fitting polygon of the contour are right angles; and the fitting polygon of the contour is a convex hull.

[0009] In some exemplary embodiments of the present disclosure, the target identification point positioning unit includes: a contour average perimeter calculation unit, used to calculate the average perimeter of the target contour in each key frame to be identified; a central moment calculation unit, used to calculate the specified order central moment of the fitting polygon of the target contour in each key frame to be identified as the identification point of the corresponding target contour; a target identification point detection unit, used to determine the target identification point of the target contour of each key frame to be identified from the identification points in each key frame to be identified according to the average perimeter of the target contour in each key frame to be identified.

[0010] In some exemplary embodiments of the present disclosure, the target recognition point detection unit includes: a reference point selection unit, used to randomly select a recognition point from the recognition points in each key frame to be recognized as a reference point; a recognition point elimination unit, used to eliminate recognition points whose distance from the reference point is less than or equal to a first distance threshold, wherein the first distance threshold is equal to a first multiple of the average circumference of the target contour in the corresponding key frame to be recognized; and a target recognition point determination unit, used to determine the target recognition point of the target contour of each key frame to be recognized from the recognition points retained after elimination.

[0011] In some exemplary embodiments of the present disclosure, the target recognition point determination unit includes: a center point determination unit, which is used to determine the center point of each key frame to be identified according to the recognition point of the target contour of each key frame to be identified; and a target recognition point determination unit, which is used to use the recognition point whose distance from the center point is less than or equal to a second distance threshold as the target recognition point of the target contour of the corresponding key frame to be identified. The second distance threshold is equal to the second multiple of the average perimeter of the target contour in the corresponding key frame to be identified, and the second multiple is greater than the first multiple.

[0012] In some exemplary embodiments of the present disclosure, the target identification code detection unit includes: a basic point determination unit, used to take the target identification point on the target common plane that is closest to the coordinate origin as the basic point; a basic point connection unit, used to connect the basic point with two target identification points adjacent to the basic point on the target common plane respectively; an identification point connection unit, used to connect two target identification points adjacent to the basic point; a first target identification code judgment unit, used to determine that the target identification code exists in the video to be identified if the basic point and the two target identification points adjacent to the basic point are connected to form an isosceles right triangle.

[0013] In some exemplary embodiments of the present disclosure, the target identification code detection unit includes: a basic point determination unit, which is used to take the target identification point on the target common plane that is closest to the coordinate origin as the basic point; a basic point connection unit, which is used to connect the basic point with two target identification points adjacent to the basic point on the target common plane respectively; an identification point connection unit, which is used to connect two target identification points adjacent to the basic point; an identification point fitting unit, which is used to fit the shape formed by the basic point and the two target identification points adjacent to the basic point to obtain a fitted quadrilateral on the target common plane; and a second target identification code judgment unit, which is used to determine that the target identification code exists in the video to be identified if the rotation angle of the fitted quadrilateral is a right angle.

[0014] In some exemplary embodiments of the present disclosure, the key frame to be identified acquisition unit includes: a target video segment extraction unit, used to extract a specified time period of the video to be identified as the target video segment; an image group acquisition unit, used to obtain an image group in the target video segment; and a key frame to be identified determination unit, used to take the first frame in each image group as the key frame to be identified.

[0015] The embodiments of the present disclosure provide a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the identification code detection method in the above embodiments is implemented.

[0016] An embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a storage device configured to store at least one program, so that when the at least one program is executed by the at least one processor, the at least one processor implements the identification code detection method in the above embodiment.

[0017] According to one aspect of the present disclosure, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the identification code detection method provided in various optional implementations of the above-mentioned embodiments.

[0018] In the technical solutions provided in some embodiments of the present disclosure, on the one hand, in the process of detecting the target identification code of the video to be identified, the characteristic that the target identification code has a fixed position in the time domain in the video to be identified is utilized to extract the key frames to be identified from the target video segment of the video to be identified, thereby improving the detection speed of the target identification code and reducing the occupied computing resources and storage resources; on the other hand, combined with the spatial position invariance and temporal position invariance of the target identification code in the video to be identified, the target identification points of the target contour of each key frame to be identified are projected onto the same target common plane, so that the target identification points of multiple key frames to be identified in the video to be identified can be combined to detect the target identification code, thereby improving the recognition accuracy of the target identification code, thereby achieving a better balance between the detection time and performance of the target identification code. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The flowchart of the identification code detection method according to an embodiment of the present disclosure is schematically shown.

[0020] Figure 2 The schematic diagram schematically shows that the target identification code in the video to be identified is a square two-dimensional code according to an embodiment of the present disclosure.

[0021] Figure 3 The schematic diagram schematically shows a case where a target identification code in a video to be identified is a circular QR code according to an embodiment of the present disclosure.

[0022] Figure 4 The flowchart of the identification code detection method according to an embodiment of the present disclosure is schematically shown.

[0023] Figure 5 The flowchart of video frame extraction according to an embodiment of the present disclosure is schematically shown.

[0024] Figure 6The schematic diagram schematically shows an edge feature map extracted when the target identification code in the video to be identified is a square two-dimensional code according to an embodiment of the present disclosure.

[0025] Figure 7 The schematic diagram schematically shows an edge feature map extracted when the target identification code in the video to be identified is a circular two-dimensional code according to an embodiment of the present disclosure.

[0026] Figure 8 The schematic diagram shows the Figure 6 Schematic diagram of the detection results of target recognition points in the edge feature map shown.

[0027] Fig. 9 The schematic diagram shows the Figure 7 Schematic diagram of the detection results of target recognition points in the edge feature map shown.

[0028] Fig.10 The block diagram of an identification code detection device according to an embodiment of the present disclosure is schematically shown.

[0029] Fig.11 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0031] The features, structures or characteristics described in the present disclosure may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0032] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and steps, nor must they be executed in the order described. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0033] In this specification, the terms "a", "an", "the", "said" and "at least one" are used to indicate the presence of at least one element / component / etc.; the terms "comprising", "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first", "second" and "third" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0034] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0035] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0036] Computer vision technology (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further perform image processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D (3D) technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0037] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0038] The solution provided in the embodiments of the present application involves technologies such as artificial intelligence computer vision technology, which is specifically illustrated by the following embodiments.

[0039] The identification code detection method provided in each embodiment of the present disclosure can be executed by any electronic device, which can be a server or a user equipment (UE), or can be implemented interactively between a server and a user equipment.

[0040] The server in the embodiments of the present disclosure may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. UE can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. UE and server can be directly or indirectly connected via wired or wireless communication, which is not limited in the present disclosure.

[0041] The exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0042] Figure 1 The flowchart of the identification code detection method according to an embodiment of the present disclosure is schematically shown.

[0043] In the embodiments of the present disclosure, the identification code generally refers to an identifier generated for purposes such as copyright protection, brand promotion, and URL (Uniform Resource Locator) identification, and the target identification code may be any identification code to be detected, and its form may include a QR code. The method provided in the embodiments of the present disclosure may be used to detect whether a video to be identified contains a specific target identification code.

[0044] Among them, the two-dimensional bar code uses certain specific geometric figures, black and white figures distributed in a plane (two-dimensional direction) according to a certain rule to record data symbol information; in the code compilation, the concept of "0" and "1" bit stream that constitutes the internal logic basis of the computer is used, and several geometric shapes corresponding to the binary system are used to represent text and numerical information, which is automatically recognized by image input devices or photoelectric scanning devices to realize automatic information processing.

[0045] The following description will be made by taking the target identification code as a two-dimensional code as an example, but the present disclosure is not limited thereto. The two-dimensional code may include a bar-shaped two-dimensional code, a square two-dimensional code, a circular two-dimensional code, and the like.

[0046] like Figure 1 As shown, the method provided by the embodiment of the present disclosure may include the following steps.

[0047] In step S110, a key frame to be identified in a target video segment of a video to be identified is obtained.

[0048] In video compression, each frame represents a still image. In actual compression, the redundancy between video frames is used to design corresponding algorithms to reduce the data capacity. For example, IPB frames can be used, where I frames represent key frames, which belong to intra-frame compression and retain the complete video picture. Only the data of this frame is needed for decoding. P frame represents the difference between this frame and the previous key frame (or P frame). When decoding, the difference defined by this frame needs to be superimposed with the previously cached picture to generate the final picture. P frame is also a difference frame. P frame does not have complete picture data, but only the picture difference data with the previous frame. B frame is a bidirectional difference frame, that is, B frame records the difference between this frame and the previous and next frames. To decode B frame, not only the previous cached picture needs to be obtained, but also the picture after decoding, and the final picture is obtained by superimposing the previous and next pictures with the data of this frame. In the disclosed embodiment, the key frame to be identified refers to the key frame extracted from the target video segment in the video to be identified and is used to detect whether there is a target identification code therein.

[0049] In an exemplary embodiment, obtaining a key frame to be identified in a target video segment of a video to be identified may include: extracting a specified time period of the video to be identified as the target video segment; obtaining a group of pictures (GOP) in the target video segment; and taking the first frame in each group of pictures as the key frame to be identified.

[0050] Among them, GOP represents the distance between two I frames and includes a group of continuous pictures. When the pictures in the video to be identified change dramatically, the value of GOP will become smaller, thereby ensuring the picture quality of the video to be identified.

[0051] In the embodiment of the present disclosure, the target video clip is part of the video content in the video to be identified, wherein the specified time period can be set according to actual needs. In the following example, the content of the video to be identified within 10 seconds before and after is taken as the target video clip. That is, the frame extraction (extraction of key frames to be identified) of the present disclosure is the key frames within 10 seconds before and after the video to be identified as the key frames to be identified, rather than extracting the key frames in the entire video to be identified. This can reduce the number of extracted key frames to be identified and speed up the detection speed of the target identification code. At the same time, due to the characteristic that the position of the target identification code in the time domain of the video to be identified remains unchanged (having position invariance), that is, during the playback of the video to be identified, although the video content of each frame played will change, the target identification code usually does not change with the change of the played video content, but is fixed at a certain position of the display interface, such as the lower right corner of the display interface. Therefore, in the embodiment of the present disclosure, the frame sequence in the target video clip in the video to be identified is sampled at a certain interval to obtain the key frames to be identified. Compared with extracting the key frames in the entire video to be identified, it does not affect the accuracy of the detection.

[0052] In other embodiments, the first 10 seconds of the video to be identified may be used as the target video segment, or the last 10 seconds of the video to be identified may be used as the target video segment, which is not limited in the present disclosure.

[0053] In an exemplary embodiment, a maximum number k of key frames to be identified extracted from the target video segment may be limited, where k is a positive integer greater than 1, for example, the maximum number k=7, but the present disclosure is not limited thereto.

[0054] In step S120, the target contour and its fitting polygon of each key frame to be identified are obtained.

[0055] In an exemplary embodiment, obtaining the target contour and its fitting polygon of each key frame to be identified may include: detecting the contour in each key frame to be identified; fitting the contour in each key frame to be identified to obtain the fitting polygon of the contour in each key frame; filtering the contour in each key frame to be identified according to the contour in each key frame to be identified and its fitting polygon, and determining the target contour in each key frame to be identified from the contours of each key frame to be identified.

[0056] In an exemplary embodiment, detecting the contour in each key frame to be identified may include: performing color space conversion on each key frame to be identified to obtain a grayscale image of each key frame to be identified; performing edge feature extraction on the grayscale image of each key frame to be identified to obtain an edge feature image of each key frame to be identified; and detecting the contour in the edge feature image of each key frame to be identified.

[0057] In the disclosed embodiment, if the key frame to be identified is a color image, such as an RGB (red green blue) image, the key frame to be identified can be converted into a grayscale image first. Then, an edge detection algorithm can be used to extract edge features from the grayscale image to obtain an edge feature map. After obtaining the edge feature map, all contours contained in the edge feature map can be detected by a contour detection algorithm.

[0058] Among them, edge detection is a basic problem in computer vision and a method of image feature extraction. Through edge detection, pixels in the image where the brightness changes dramatically can be obtained. These pixels are usually on the contour.

[0059] In the embodiments of the present disclosure, any edge detection algorithm can be used to extract edge features of grayscale images. In the following examples, the Canny edge detection algorithm is used as an example, but in other embodiments, any one of the first-order Roberts Cross operator, Prewitt operator, Sobel operator, Kirsch operator, compass operator, etc., or any one of the second-order Marr-Hildreth, the second-order derivative zero crossing in the gradient direction, Laplacian operator, etc. can also be used.

[0060] Contour detection refers to the process of extracting contours in a digital image containing a target and background, ignoring the influence of the texture of the background and the internal of the target and the noise interference, and using certain techniques and methods. In the embodiment of the present disclosure, any contour detection algorithm can be used to perform contour detection of an edge feature map. In the following examples, the contour extraction function findContours() in OpenCV is used as an example for illustration, but the present disclosure is not limited to this.

[0061] In an exemplary embodiment, the contours in each key frame to be identified are filtered according to the contours in each key frame to be identified and their fitting polygons, and the target contour in each key frame to be identified is determined from the contours of each key frame to be identified, which can include: determining the contour that meets the following conditions as the target contour: the area of ​​the contour is greater than or equal to the contour threshold; the rotation angle of the minimum circumscribed rectangle of the contour is a right angle; the aspect ratio of the minimum circumscribed rectangle of the contour is a predetermined aspect ratio value; the number of sides of the fitting polygon of the contour is a set number of sides; the side formed by three adjacent points of the fitting polygon of the contour is a right angle; the fitting polygon of the contour is a convex hull.

[0062] In the disclosed embodiment, contours that simultaneously satisfy the above filtering conditions are determined as target contours, and the corresponding target contours are retained in the key frame to be identified; contours that do not satisfy any one or more of the above filtering conditions are determined as noise contours and are removed.

[0063] In step S130, the target recognition point of the target contour of each key frame to be recognized is located according to the target contour of each key frame to be recognized and its fitting polygon.

[0064] In the disclosed embodiment, the identification point may also be referred to as a detection point or a positioning point. The identification point is a feature point in the target identification code, which can be used to assist in detecting whether the target identification code exists in the video to be identified, and to determine the location and size of the target identification code according to the identification point. The target identification point is an identification point that meets certain conditions and is selected from each key frame to be identified.

[0065] The identification code detection method provided by the embodiment of the present disclosure is a general identification code detection method based on identification point analysis, which extracts and analyzes identification points of key frames to be identified in a video to be identified, so as to finally determine whether the video to be identified includes a target identification code.

[0066] In an exemplary embodiment, locating the target identification point of the target contour of each key frame to be identified based on the target contour of each key frame to be identified and its fitting polygon can include: calculating the average perimeter of the target contour in each key frame to be identified; calculating the central moment of the specified order of the fitting polygon of the target contour in each key frame to be identified as the identification point of the corresponding target contour; and determining the target identification point of the target contour of each key frame to be identified from the identification points in each key frame to be identified based on the average perimeter of the target contour in each key frame to be identified.

[0067] In the disclosed embodiment, the perimeters of the target contours in each key frame to be identified may be summed, and the summed result may be divided by the number of target contours included in the key frame to be identified to obtain the average perimeter of the target contours corresponding to the key frame to be identified.

[0068] In the following examples, the specified order central moment is described by taking the 1st order central moment as an example, but the present disclosure is not limited to this.

[0069] In an exemplary embodiment, determining the target identification point of the target contour of each key frame to be identified from the identification points in each key frame to be identified based on the average perimeter of the target contour in each key frame to be identified can include: randomly selecting an identification point from the identification points in each key frame to be identified as a reference point; eliminating identification points whose distance from the reference point is less than or equal to a first distance threshold, wherein the first distance threshold is equal to a first multiple of the average perimeter of the target contour in the corresponding key frame to be identified; and determining the target identification point of the target contour of each key frame to be identified from the identification points retained after elimination.

[0070] In the embodiment of the present disclosure, the first multiple n is a positive integer greater than or equal to 1. In the following examples, n=3 is taken as an example, but the present disclosure is not limited thereto.

[0071] In an exemplary embodiment, determining the target identification point of the target contour of each key frame to be identified from the identification points retained after elimination may include: determining the center point of each key frame to be identified based on the identification point of the target contour of each key frame to be identified; using the identification point whose distance from the center point is less than or equal to a second distance threshold as the target identification point of the target contour of the corresponding key frame to be identified; wherein the second distance threshold is equal to the second multiple of the average circumference of the target contour in the corresponding key frame to be identified, and the second multiple is greater than the first multiple.

[0072] In the embodiment of the present disclosure, the second multiple m is a positive integer greater than or equal to 1. In the following examples, m=40 is used as an example, but the present disclosure is not limited thereto.

[0073] In step S140 , the target recognition points of the target contour of each key frame to be recognized are projected onto the target common plane.

[0074] In the embodiment of the present disclosure, the target common plane can be any common plane, as long as the target identification points of all the key frames to be identified can be projected onto the same target common plane. For example, the target common plane can be the first frame in the key frames to be identified, or a common plane other than the key frames to be identified can be selected as the target common plane. The present disclosure does not limit this. In the following examples, the first frame in the key frames to be identified is used as the target common plane for example.

[0075] In step S150, the target identification code in the to-be-identified video is detected according to the target identification points on the target common plane.

[0076] In the related art, a separate judgment is made on whether a target identification code exists for each extracted key frame. In the embodiment of the present disclosure, by projecting the target identification points in all the key frames to be identified onto the same target common plane, the target identification code can be detected by combining multiple frames in the time domain of the video to be identified (i.e., multiple key frames to be identified), thereby improving the detection accuracy.

[0077] In an exemplary embodiment, detecting the target identification code in the video to be identified based on the target identification points on the target common plane may include: taking the target identification point on the target common plane that is closest to the coordinate origin as a basic point; connecting the basic point with two target identification points on the target common plane that are adjacent to the basic point respectively; connecting two target identification points adjacent to the basic point; if the basic point and the two target identification points adjacent to the basic point are connected to form an isosceles right triangle, it is determined that the target identification code exists in the video to be identified.

[0078] In an exemplary embodiment, detecting the target identification code in the video to be identified based on the target identification points on the target common plane may include: taking the target identification point on the target common plane that is closest to the coordinate origin as a basic point; connecting the basic point with two target identification points adjacent to the basic point on the target common plane respectively; connecting two target identification points adjacent to the basic point; fitting the shape formed by the basic point and the two target identification points adjacent to the basic point to obtain a fitted quadrilateral on the target common plane; if the rotation angle of the fitted quadrilateral is a right angle, it is determined that the target identification code exists in the video to be identified.

[0079] The identification code detection method provided by the embodiment of the present disclosure, on the one hand, in the process of detecting the target identification code of the video to be identified, utilizes the characteristic that the target identification code has a fixed position in the time domain in the video to be identified, and extracts the key frames to be identified from the target video segment of the video to be identified, thereby improving the detection speed of the target identification code and reducing the occupied computing resources and storage resources; on the other hand, combined with the spatial position invariance and temporal position invariance of the target identification code in the video to be identified, the target identification points of the target contour of each key frame to be identified are projected onto the same target common plane, so that the target identification points of multiple key frames to be identified in the video to be identified can be combined to detect the target identification code, thereby improving the recognition accuracy of the target identification code, thereby achieving a better balance between the detection time and performance of the target identification code.

[0080] The identification code detection method provided by the embodiment of the present disclosure has a good balance between performance and accuracy. The identification code is usually for the purpose of brand promotion or personal identity identification, and can be applied to filter pictures or videos containing identification codes on the Internet. In the Internet video platform, it is usually necessary to filter and screen videos containing identification codes in order to maintain high-quality video content and improve the visibility of the promoted individuals. This type of identification code usually has the following characteristics. First, the spatial position of the identification code is usually fixed, that is, position invariance; second, the temporal position of the identification code is also fixed, that is, temporal position invariance.

[0081] The video to be identified containing the target identification code is as follows Figure 2 and Figure 3 shown.

[0082] Figure 2 The schematic diagram schematically shows that the target identification code in the video to be identified is a square two-dimensional code according to an embodiment of the present disclosure.

[0083] like Figure 2 As shown, assuming that a video 220 to be identified is being played on the display screen 210 of a desktop computer, a target identification code 230 is displayed at a fixed position in the lower left corner of the video 220 to be identified. The target identification code 230 is a square two-dimensional code. The target identification code 230 includes a positioning pattern (Position Detection Pattern) 231 in the upper right corner, a positioning pattern 232 in the upper left corner, and a positioning pattern 233 in the lower right corner.

[0084] Figure 3 The schematic diagram schematically shows a case where a target identification code in a video to be identified is a circular QR code according to an embodiment of the present disclosure.

[0085] like Figure 3 As shown, assuming that a video 320 to be identified is being played on the display screen 310 of a smart phone, a target identification code 330 is displayed in the upper middle position of the video 320 to be identified. The target identification code 330 is a circular two-dimensional code, and the target identification code 330 includes a positioning pattern 331 in the upper left corner, a positioning pattern 332 in the upper right corner, and a positioning pattern 333 in the lower left corner.

[0086] Figure 2 and Figure 3 In the examples, the target identification code is set to include positioning patterns at three different corners, because a square or circular two-dimensional code can be identified by using positioning patterns at three different corners. By finding the target contour of the positioning pattern, the target identification code can be detected and positioned, but the present disclosure is not limited to this. In other embodiments, the target identification code can also set more or fewer positioning patterns at different positions.

[0087] Figure 4 The flowchart of the identification code detection method according to an embodiment of the present disclosure is schematically shown. The identification code detection method provided by the embodiment of the present disclosure may include the design of a video frame extraction strategy, target identification code feature extraction, target identification code identification point detection, target identification code target identification point determination and time domain joint determination steps. Figure 4 As shown, the method provided by the embodiment of the present disclosure may include the following steps.

[0088] In step S410, based on the position invariance and time domain invariance of the target video code in the video to be identified, key frames to be identified within a specific time period are extracted for detection.

[0089] For the target identification code detection task in the video to be identified, the frame extraction strategy is relatively important. If a frame-by-frame detection method is adopted, extracting each frame will increase the redundancy of the detection. The repeated detection caused by the redundant detection increases the time-consuming performance of the algorithm. The disclosed embodiment makes full use of the position invariance and time domain invariance of the target identification code in the video to be identified. By extracting the key frames within the specified time period of the video to be identified as the key frames to be identified for detection, and performing joint verification of multiple frames in the time domain of the video to be identified, a fast and accurate detection effect can be achieved.

[0090] Figure 5 The flowchart of video frame extraction according to an embodiment of the present disclosure is schematically shown.

[0091] like Figure 5 As shown, a video to be identified is obtained from a video source 501, and a decoder 502 is used to decode the video to be identified to obtain a video frame data stream 503, and a key frame extraction step 504 is performed to obtain a video key frame 505 as a key frame to be identified.

[0092] Among them, the key frame extraction step 504 may specifically include: by controlling the frame extraction parameters of FFmpeg (FastForward Moving Picture Experts Group, a multimedia video processing tool), for example, setting the specified time period to 10 seconds before and after the video to be identified, a key frame sequence of the target video segment is obtained, the frame extraction type is I frame as the key frame to be identified, and the maximum number of extracted frames is 7. FFmpeg is used to determine the GOP, and then the first frame of each GOP is extracted as the key frame to be identified. There are two reasons why the key frames within the specified time period are used for detection. The first point is that the image quality of the extracted key frame to be identified is relatively high, which is conducive to improving the accuracy of detection. The second point is that the extracted key frame to be identified is the first frame of each GOP. The identification code detection based on GOP greatly reduces redundant detection. There is no drastic change in the image of the video to be identified within a GOP, that is, the position of the target identification code has a certain stability within the GOP.

[0093] In step S420, the spatial domain features of the target identification code are processed and extracted.

[0094] In combination with the characteristics of the target identification code, the spatial domain features of the target identification code are processed and extracted, and the process of extracting the target identification code features may further include steps S421 and S422.

[0095] In step S421, the key frame to be identified is converted into a color space, mapped from a color space to a grayscale space, and a grayscale image is obtained.

[0096] First, the key frame to be identified is converted into a color space, that is, the key frame to be identified is mapped from the color space to the gray space, that is, grayscaled.

[0097] The disclosed embodiment may use any color space conversion to grayscale the color key frame to be identified, such as component method, maximum method, average method, etc. Here, the weighted average method is used as an example for illustration.

[0098] Among them, the weighted average method can perform weighted average of the R, G, and B components of each pixel of the key frame to be identified through the following formula to obtain the gray value Gray of each pixel:

[0099] Gray=0.2989R+0.5870G+0.1140B (1)

[0100] In step S422, edge features are extracted from the grayscale image using an edge detection algorithm to obtain an edge feature map.

[0101] For example, the Canny edge detection algorithm is used to extract edge features from a grayscale image to obtain an edge feature map.

[0102] Figure 6 The schematic diagram schematically shows an edge feature map extracted when the target identification code in the video to be identified is a square two-dimensional code according to an embodiment of the present disclosure.

[0103] Figure 7 The schematic diagram schematically shows an edge feature map extracted when the target identification code in the video to be identified is a circular two-dimensional code according to an embodiment of the present disclosure.

[0104] In step S430, identification points of the target identification code are detected.

[0105] The identification point detection step may specifically include the following steps S431 - S433 .

[0106] In step S431, all contours included in the edge feature map are detected using a contour detection algorithm.

[0107] In the disclosed embodiment, there is a regular square or circle around the identification point of the target identification code, and the edge feature map is used to detect all contours contained in the edge feature map through a contour detection algorithm.

[0108] In step S432, each contour is fitted to obtain a fitting polygon.

[0109] In the disclosed embodiment, since the area around the identification point of the target identification code has a relatively regular shape (square or circle), each contour is fitted to obtain a fitting polygon, for example, the approxPloyDP function in OpenCV may be used.

[0110] In step S433, the filter condition is designed to remove the noise contour by using the recognition point being a regular shape to obtain the target contour.

[0111] In the embodiment of the present disclosure, the condition that the target identification code has a regular shape around the identification point is used to design a filtering condition to remove the noise contour. The designed filtering conditions include the following points, and each detected contour is judged to determine whether it meets the following filtering conditions at the same time:

[0112] a. The area of ​​the contour is greater than or equal to the contour threshold.

[0113] For example, the contour threshold may be set to an area of ​​100 pixels, such as 10 pixels x 10 pixels = 100 pixels.

[0114] b. The rotation angle of the minimum circumscribed rectangle of the contour is a right angle.

[0115] In the disclosed embodiment, the four corners of the minimum circumscribed rectangle are all 90 degrees regardless of their angles. Here, the filtering condition that the rotation angle of the minimum circumscribed rectangle of the contour is a right angle needs to be satisfied.

[0116] b. The aspect ratio of the minimum circumscribed rectangle of the outline is a predetermined aspect ratio value.

[0117] The theoretical value of the predetermined value of the aspect ratio is 1, and it is set to 0.95 in the embodiment of the present disclosure.

[0118] d. The number of sides of the fitting polygon of the contour is 4.

[0119] The number of sides of the fitted polygon is 4, that is, each contour is fitted as a quadrilateral.

[0120] e. The sides formed by three adjacent points of the contour fitting polygon are right angles.

[0121] In the embodiment of the present disclosure, since the identification point of the target identification code is a rectangle, the fitting polygon is judged to be a rectangle, but the rectangle cannot be judged under the polygon condition, and it can only be judged as a rectangle in combination with the above d filtering condition. Specifically, the sin value of the side composed of three adjacent points of the fitting polygon can be calculated, and the theoretical value is 1, which is set to 0.98 in the embodiment of the present disclosure.

[0122] f. The fitting polygon of the contour is a convex hull, which meets the requirements of square and circular contours.

[0123] The final target contour is obtained by filtering the above filtering conditions.

[0124] In step S440, the target identification point of the target identification code is determined.

[0125] The target recognition point determination step may further include the following steps S441-S443.

[0126] In step S441 , the average perimeter of the target contour and the first-order central moment of each fitting polygon are calculated, and the first-order central moment is used as the identification point of the target contour.

[0127] After all target contours are obtained, the average perimeter of the target contour and the first-order central moment of the fitting polygon of each target contour are first calculated as the centroid of the target contour, and the centroid of the target contour is considered as the identification point of the target contour.

[0128] In step S442, similar identification points are removed by calculating the Euclidean distance between the identification points and setting the distance to be greater than n times (eg, n=3) the average perimeter of the target contour.

[0129] Since there may be multiple target contours containing the identification point around the identification point, which will cause duplication of the identification point, a random identification point is selected as the reference point. The Euclidean distance between other identification points except the reference point and the reference point is calculated, and the identification points with Euclidean distance greater than 3 times the average perimeter of the target contour are retained, and the identification points with Euclidean distance less than 3 times the average perimeter of the target contour are eliminated to remove similar identification points.

[0130] In step S443, distant outliers are removed by calculating the Euclidean distance between the identification points and setting the distance to be greater than m times (eg, m=40) the average perimeter of the target contour.

[0131] At the same time, there may be outliers among the recognition points. Determine a center point of all the recognition points, and then judge whether the Euclidean distance between each recognition point and the center point of all the recognition points is greater than 40 times the average circumference of the target contour. If the Euclidean distance between a certain recognition point and the center point of all the recognition points is greater than 40 times the average circumference of the target contour, the recognition point is considered to be an outlier and is removed to remove farther outliers; if the Euclidean distance between a certain recognition point and the center point of all the recognition points is less than or equal to 40 times the average circumference of the target contour, the recognition point is retained.

[0132] The final retained recognition point is taken as the target recognition point.

[0133] Figure 8 The schematic diagram shows the Figure 6 Schematic diagram of the detection results of target recognition points in the edge feature map shown.

[0134] like Figure 8 The detection results of the target identification points include the upper left corner target outline 811 and its target identification point 812, the lower left corner target outline 821 and its target identification point 822, the upper right corner target outline 841 and its target identification point 842, and the lower right corner target outline 831 and its target identification point 832.

[0135] Fig. 9 The schematic diagram shows the Figure 7 Schematic diagram of the detection results of target recognition points in the edge feature map shown.

[0136] like Fig. 9 The detection result of the target recognition point includes the upper left corner target outline 911 and its target recognition point 912 , the lower left corner target recognition point 922 , and the upper right corner target recognition point 932 .

[0137] In step S450, based on the position invariance and time domain invariance of the target identification code in the video to be identified, the detected time domain frame sequence is projected onto the same target common plane, and a joint verification of multiple frames is performed in the time domain of the video to be identified.

[0138] In the disclosed embodiments, considering that the target identification code in the video to be identified is usually fixed at a certain position, the spatial position of the target identification code is invariant in the time domain. By utilizing this characteristic, the detected time domain frame sequence (i.e., a frame sequence formed by arranging each key frame to be identified in chronological order in the video to be identified) is projected onto the same target common plane for joint judgment. On the one hand, it can avoid the situation where the target identification point cannot be detected due to the missing contour of a single frame. On the other hand, it can increase the fault tolerance of the target identification code detection.

[0139] Specifically, all key frames to be identified can be projected onto the first frame or onto a common plane as their target common plane, so that the detection results are judged within the same plane. Because not all key frames to be identified will detect target identification points, the detection results of all key frames to be identified are put together through projection. Since the position of the target identification code is unchanged in the time domain, the projection positions of corresponding target identification points in different key frames to be identified are the same.

[0140] In an exemplary embodiment, a coordinate system can be established on the target common plane, for example, the upper left corner of the target common plane is used as the coordinate origin, the vertical downward direction is used as the X-axis, and the horizontal rightward direction is used as the Y-axis, and the coordinate values ​​of each target identification point projected on the target common plane are determined, and the target identification point with the smallest coordinate value (i.e., the closest distance to the coordinate origin) is used as the basic point, and two target identification points adjacent to the basic point are connected, and the two target identification points are connected. For example, assuming that Figure 8 In the embodiment, each target recognition point is projected to the first frame, and the target recognition point 812 can be used as the basic point, and the target recognition point 812 is connected to the adjacent target recognition point 822, and the target recognition point 812 is connected to the adjacent target recognition point 842. At this time, the target recognition point 832 in the lower right corner can be ignored, and then it is directly determined whether the target recognition point 812, the target recognition point 822, and the target recognition point 842 are connected to form an isosceles right triangle. If it is an isosceles right triangle, it is determined that there is a target recognition code in the video to be recognized, and the size, position, etc. of the target recognition code can be determined according to the coordinate values ​​of the target recognition point 812, the target recognition point 822, and the target recognition point 842. If it is not an isosceles right triangle, it is determined that there is no target recognition code in the video to be recognized.

[0141] In other embodiments, Figure 8 In the embodiment, the projection of each target recognition point to the first frame is taken as an example, and the target recognition point 812 can be used as the basic point, and the target recognition point 812 is connected with the adjacent target recognition point 822, and the target recognition point 812 is connected with the adjacent target recognition point 842. At this time, the target recognition point 832 in the lower right corner can be ignored, and then the shape formed by the connection of the target recognition point 812, the target recognition point 822, and the target recognition point 842 is fitted to obtain a fitting quadrilateral, and it is determined whether the rotation angle of the fitting quadrilateral is a right angle. If it is a right angle, it can be determined that there is a target recognition code in the video to be recognized, and the size, position, etc. of the target recognition code can be determined according to the coordinate values ​​of the target recognition point 812, the target recognition point 822, and the target recognition point 842. If it is not a right angle, it is determined that there is no target recognition code in the video to be recognized.

[0142] In the embodiment of the present disclosure, the identification code data of 2 types of video platforms were collected as a test set and manually annotated, wherein 500 pieces of data of square QR codes and circular QR codes were used to verify the effect of the identification code detection method provided by the embodiment of the present disclosure: wherein, when the performance is 720p, it takes 16.67ms / frame (frame); when the performance is 1080p, it takes 46.34ms / frame. wherein, the accuracy rate is 77.2% for circular QR codes and 69.04% for square QR codes.

[0143] The identification code detection method provided by the embodiment of the present disclosure can detect a video to be identified that contains a target identification code. By designing a video frame extraction strategy, feature extraction of the target identification code, identification point detection of the target identification code, target identification point determination of the target identification code and time domain joint discrimination and other steps, the spatial position invariance and time position invariance of the target identification code are combined to reduce redundant time consumption. A filtering condition for the contour of the target identification code is specially designed, and the detection accuracy is further improved in combination with time domain joint discrimination to achieve a higher detection performance. Compared with the frame-by-frame detection or single-frame detection algorithm, a better balance between accuracy and performance is obtained, and the method has great practical application value.

[0144] The identification code detection method provided by the embodiments of the present disclosure can be applied to a wide range of scenarios.

[0145] For example, videos containing identification codes are commonly spread on the Internet, and the videos uploaded by users on the platform are usually diverse. Among them, videos containing identification codes may inadvertently expose the platform to the risk of hidden platform brand supervision. The method provided in the embodiments of the present disclosure can reduce the losses caused by the lack of brand supervision.

[0146] For another example, the identification code detection method provided in the embodiment of the present disclosure can be applied to optimize cloud video storage. The identification code detection method provided in the embodiment of the present disclosure can be used to implement cloud filtering of videos, thereby filtering out videos containing identification codes and storing and distributing more valuable videos.

[0147] For another example, the identification code detection method provided by the embodiment of the present disclosure can be applied to video editing. The identification code can be detected on the content of the video, thereby providing prompts to the editing user, improving the user experience and raising the user's copyright awareness, thereby reducing the legal risks of the platform.

[0148] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing.

[0149] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The backend services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the rapid development and application of the Internet industry, in the future, each item may have its own identification mark, which needs to be transmitted to the backend system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, which can only be achieved through cloud computing.

[0150] Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.

[0151] At present, the storage method of the storage system is: create a logical volume, and when creating a logical volume, allocate physical storage space for each logical volume. The physical storage space may be composed of disks of a storage device or several storage devices. The client stores data on a logical volume, that is, stores the data on the file system. The file system divides the data into many parts, each of which is an object. The object contains not only data but also additional information such as data identification (ID, ID entity). The file system writes each object into the physical storage space of the logical volume, and the file system records the storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.

[0152] The process of the storage system allocating physical storage space to a logical volume is as follows: based on the estimated capacity of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of independent redundant disk arrays (RAID, Redundant Array of Independent Disks), the physical storage space is pre-divided into stripes. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0153] Fig.10 The block diagram of the identification code detection device according to an embodiment of the present disclosure is schematically shown. Fig.10 As shown, the identification code detection device 1000 provided in the embodiment of the present disclosure may include a key frame acquisition unit 1010 to be identified, a target contour polygon fitting unit 1020, a target identification point positioning unit 1030, a target identification point projection unit 1040 and a target identification point detection unit 1050.

[0154] In the disclosed embodiment, the key frame acquisition unit 1010 to be identified can be used to acquire the key frame to be identified in the target video segment of the video to be identified. The target contour polygon fitting unit 1020 can be used to obtain the target contour and its fitting polygon of each key frame to be identified. The target identification point positioning unit 1030 can be used to locate the target identification point of the target contour of each key frame to be identified based on the target contour and its fitting polygon of each key frame to be identified. The target identification point projection unit 1040 can be used to project the target identification point of the target contour of each key frame to be identified to the target common plane. The target identification code detection unit 1050 can be used to detect the target identification code in the video to be identified based on the target identification point on the target common plane.

[0155] In an exemplary embodiment, the target contour polygon fitting unit 1020 may include: a contour detection unit, which can be used to detect the contour in each key frame to be identified; a contour fitting unit, which can be used to fit the contour in each key frame to be identified to obtain the fitting polygon of the contour in each key frame; a target contour determination unit, which can be used to filter the contour in each key frame to be identified according to the contour in each key frame to be identified and its fitting polygon, and determine the target contour in each key frame to be identified from the contours of each key frame to be identified.

[0156] In an exemplary embodiment, the contour detection unit may include: a color space conversion unit, which can be used to perform color space conversion on each key frame to be identified to obtain a grayscale image of each key frame to be identified; an edge feature extraction unit, which can be used to perform edge feature extraction on the grayscale image of each key frame to be identified to obtain an edge feature image of each key frame to be identified; and a contour extraction unit, which can be used to detect contours in the edge feature image of each key frame to be identified.

[0157] In an exemplary embodiment, the target contour determination unit may include: a target contour filtering unit, which can be used to determine a contour that meets the following conditions as the target contour: the area of ​​the contour is greater than or equal to the contour threshold; the rotation angle of the minimum circumscribed rectangle of the contour is a right angle; the aspect ratio of the minimum circumscribed rectangle of the contour is a predetermined aspect ratio value; the number of sides of the fitting polygon of the contour is a set number of sides; the sides formed by three adjacent points of the fitting polygon of the contour are right angles; and the fitting polygon of the contour is a convex hull.

[0158] In an exemplary embodiment, the target identification point positioning unit 1030 may include: a contour average perimeter calculation unit, which can be used to calculate the average perimeter of the target contour in each key frame to be identified; a central moment calculation unit, which can be used to calculate the specified order central moment of the fitting polygon of the target contour in each key frame to be identified as the identification point of the corresponding target contour; a target identification point detection unit, which can be used to determine the target identification point of the target contour of each key frame to be identified from the identification points in each key frame to be identified based on the average perimeter of the target contour in each key frame to be identified.

[0159] In an exemplary embodiment, the target recognition point detection unit may include: a reference point selection unit, which can be used to randomly select a recognition point from the recognition points in each key frame to be recognized as a reference point; a recognition point elimination unit, which can be used to eliminate recognition points whose distance from the reference point is less than or equal to a first distance threshold, wherein the first distance threshold is equal to the first multiple of the average circumference of the target contour in the corresponding key frame to be recognized; a target recognition point determination unit, which can be used to determine the target recognition point of the target contour of each key frame to be recognized from the recognition points retained after elimination.

[0160] In an exemplary embodiment, the target recognition point determination unit may include: a center point determination unit, which may be used to determine the center point of each key frame to be identified according to the recognition point of the target contour of each key frame to be identified; and a target recognition point determination unit, which may be used to use the recognition point whose distance from the center point is less than or equal to a second distance threshold as the target recognition point of the target contour of the corresponding key frame to be identified. The second distance threshold is equal to the second multiple of the average perimeter of the target contour in the corresponding key frame to be identified, and the second multiple is greater than the first multiple.

[0161] In an exemplary embodiment, the target identification code detection unit 1050 may include: a basic point determination unit, which can be used to take the target identification point on the target common plane that is closest to the coordinate origin as the basic point; a basic point connection unit, which can be used to connect the basic point with two target identification points adjacent to the basic point on the target common plane respectively; an identification point connection unit, which can be used to connect two target identification points adjacent to the basic point; a first target identification code judgment unit, which can be used to determine that the target identification code exists in the video to be identified if the basic point and the two target identification points adjacent to the basic point are connected to form an isosceles right triangle.

[0162] In an exemplary embodiment, the target identification code detection unit 1050 may include: a basic point determination unit, which can be used to take the target identification point on the target common plane that is closest to the coordinate origin as the basic point; a basic point connection unit, which can be used to connect the basic point with two target identification points adjacent to the basic point on the target common plane respectively; an identification point connection unit, which can be used to connect two target identification points adjacent to the basic point; an identification point fitting unit, which can be used to fit the shape formed by the basic point and the two target identification points adjacent to the basic point to obtain a fitted quadrilateral on the target common plane; a second target identification code judgment unit, which can be used to determine that the target identification code exists in the video to be identified if the rotation angle of the fitted quadrilateral is a right angle.

[0163] In an exemplary embodiment, the key frame to be identified acquisition unit 1010 may include: a target video segment extraction unit, which can be used to extract a specified time period of the video to be identified as the target video segment; an image group acquisition unit, which can be used to obtain an image group in the target video segment; and a key frame to be identified determination unit, which can be used to take the first frame in each image group as the key frame to be identified.

[0164] Other contents of the identification code detection device of the embodiment of the present disclosure may refer to the above embodiment.

[0165] It should be noted that although several units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0166] Reference below Fig.11 , which shows a schematic diagram of the structure of an electronic device suitable for implementing an embodiment of the present application. Fig.11 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0167] Reference Fig.11 The electronic device provided by the embodiment of the present disclosure may include: a processor 1101, a communication interface 1102, a memory 1103 and a communication bus 1104.

[0168] The processor 1101 , the communication interface 1102 and the memory 1103 communicate with each other via the communication bus 1104 .

[0169] Optionally, the communication interface 1102 may be an interface of a communication module, such as an interface of a GSM (Global System for Mobile communications) module. The processor 1101 is used to execute a program. The memory 1103 is used to store a program. The program may include a computer program, which includes computer operating instructions. Among them, the program may include: a game client program.

[0170] The processor 1101 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present disclosure.

[0171] The memory 1103 may include a high-speed RAM (random access memory) memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0172] Among them, the program can be specifically used to: obtain the key frames to be identified in the target video segment of the video to be identified; obtain the target contour and its fitting polygon of each key frame to be identified; locate the target identification point of the target contour of each key frame to be identified according to the target contour and its fitting polygon of each key frame to be identified; project the target identification point of the target contour of each key frame to be identified to the target common plane; detect the target identification code in the video to be identified according to the target identification point on the target common plane.

[0173] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional implementations of the above-mentioned embodiments.

[0174] It should be understood that any number of elements in the drawings of the present disclosure is for illustration rather than limitation, and any naming is only for distinction rather than having any limiting meaning.

[0175] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0176] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for detecting an identification code, characterized in that: include: Obtaining key frames to be identified in a target video segment of a video to be identified; Obtain the target contour and its fitting polygon of each key frame to be identified; According to the target contour of each key frame to be identified and its fitting polygon, the target identification point of the target contour of each key frame to be identified is located, which includes: calculating the average perimeter of the target contour in each key frame to be identified; calculating the central moment of the specified order of the fitting polygon of the target contour in each key frame to be identified as the identification point of the corresponding target contour; randomly selecting an identification point from the identification points in each key frame to be identified as a reference point; eliminating identification points whose distance from the reference point is less than or equal to a first distance threshold, wherein the first distance threshold is equal to a first multiple of the average perimeter of the target contour in the corresponding key frame to be identified; determining the target identification point of the target contour of each key frame to be identified from the identification points retained after elimination; Projecting the target identification points of the target contour of each key frame to be identified onto the target common plane; The target identification code in the to-be-identified video is detected according to the target identification point on the target common plane.

2. The method according to claim 1, characterized in that Obtain the target contour and its fitting polygon of each key frame to be identified, including: Detecting contours in each key frame to be identified; Fitting the contour in each key frame to be identified to obtain a fitting polygon of the contour in each key frame; The contour in each key frame to be identified is filtered according to the contour in each key frame to be identified and the fitting polygon thereof, and the target contour in each key frame to be identified is determined from the contours of each key frame to be identified.

3. The method according to claim 2, characterized in that Detect the contours in each key frame to be identified, including: Performing color space conversion on each key frame to be identified to obtain a grayscale image of each key frame to be identified; Extract edge features from the grayscale image of each key frame to be identified, and obtain an edge feature image of each key frame to be identified; Detect contours in the edge feature map of each key frame to be identified.

4. The method according to claim 2, characterized in that: Filtering the contour in each key frame to be identified according to the contour in each key frame to be identified and the fitting polygon thereof, and determining the target contour in each key frame to be identified from the contours of each key frame to be identified, including: The contour that meets the following conditions is determined as the target contour: The area of ​​the contour is greater than or equal to a contour threshold; The rotation angle of the minimum circumscribed rectangle of the outline is a right angle; The aspect ratio of the minimum circumscribed rectangle of the outline is a predetermined aspect ratio value; The number of sides of the fitting polygon of the contour is the set number of sides; The sides formed by three adjacent points of the fitting polygon of the contour are right angles; The fitted polygon of the contour is the convex hull.

5. The method according to claim 1, characterized in that Determine the target identification points of the target contour of each key frame to be identified from the identification points retained after elimination, including: Determine the center point of each key frame to be identified according to the identification point of the target contour of each key frame to be identified; Taking the identification point whose distance from the center point is less than or equal to the second distance threshold as the target identification point of the target contour corresponding to the key frame to be identified; The second distance threshold is equal to a second multiple of an average perimeter of the target contour in the corresponding key frame to be identified, and the second multiple is greater than the first multiple.

6. The method according to claim 1, characterized in that Detecting a target identification code in the to-be-identified video according to a target identification point on a target common plane includes: Taking the target identification point on the target common plane closest to the coordinate origin as the basic point; Connecting the basic point to two target identification points adjacent to the basic point on the target common plane respectively; connecting two target identification points adjacent to the basic point; If the basic point and two target identification points adjacent to the basic point are connected to form an isosceles right triangle, it is determined that the target identification code exists in the video to be identified.

7. The method according to claim 1, characterized in that Detecting a target identification code in the to-be-identified video according to a target identification point on a target common plane includes: Taking the target identification point on the target common plane closest to the coordinate origin as the basic point; Connecting the basic point to two target identification points adjacent to the basic point on the target common plane respectively; connecting two target identification points adjacent to the basic point; Fitting the basic point and the shape formed by connecting two target identification points adjacent to the basic point to obtain a fitting quadrilateral on the target common plane; If the rotation angle of the fitted quadrilateral is a right angle, it is determined that the target identification code exists in the video to be identified.

8. The method according to claim 1, characterized in that: Obtain the key frames to be identified in the target video segment of the video to be identified, including: Extracting a specified time period of the video to be identified as the target video segment; Obtaining a group of images in the target video segment; The first frame in each image group is used as the key frame to be identified.

9. An identification code detection device, characterized in that: include: A key frame acquisition unit to be identified, used to acquire the key frame to be identified in the target video segment of the video to be identified; A target contour polygon fitting unit, used to obtain the target contour and its fitting polygon of each key frame to be identified; A target recognition point positioning unit, used to locate the target recognition point of the target contour of each key frame to be recognized according to the target contour of each key frame to be recognized and its fitting polygon; A target identification point projection unit, used for projecting the target identification points of the target contour of each key frame to be identified onto a target common plane; A target identification code detection unit, used for detecting a target identification code in the to-be-identified video according to a target identification point on a target common plane; The target recognition point positioning unit includes: a contour average perimeter calculation unit, used to calculate the average perimeter of the target contour in each key frame to be recognized; a central moment calculation unit, used to calculate the specified order central moment of the fitting polygon of the target contour in each key frame to be recognized as the recognition point of the corresponding target contour; a target recognition point detection unit, used to determine the target recognition point of the target contour of each key frame to be recognized from the recognition points in each key frame to be recognized according to the average perimeter of the target contour in each key frame to be recognized; The target recognition point detection unit comprises: A reference point selection unit, used for randomly selecting an identification point from the identification points in each key frame to be identified as a reference point; An identification point elimination unit, used to eliminate identification points whose distance from the reference point is less than or equal to a first distance threshold, wherein the first distance threshold is equal to a first multiple of an average perimeter of a target contour in a key frame to be identified; The target recognition point determination unit is used to determine the target recognition point of the target contour of each key frame to be recognized from the recognition points retained after elimination.

10. The device according to claim 9, characterized in that The target contour polygon fitting unit comprises: A contour detection unit, used for detecting contours in each key frame to be identified; A contour fitting unit, used for fitting the contour in each key frame to be identified, and obtaining a fitting polygon of the contour in each key frame; The target contour determining unit is used to filter the contour in each key frame to be identified according to the contour in each key frame to be identified and its fitting polygon, and determine the target contour in each key frame to be identified from the contours of each key frame to be identified.

11. The device according to claim 10, characterized in that The contour detection unit comprises: A color space conversion unit, used for performing color space conversion on each key frame to be identified to obtain a grayscale image of each key frame to be identified; An edge feature extraction unit is used to extract edge features from the grayscale image of each key frame to be identified, so as to obtain an edge feature image of each key frame to be identified; The contour extraction unit is used to detect the contour in the edge feature map of each key frame to be identified.

12. The device according to claim 10, characterized in that The target contour determination unit comprises: The target contour filtering unit is used to determine the contour that meets the following conditions as the target contour: The area of ​​the contour is greater than or equal to a contour threshold; The rotation angle of the minimum circumscribed rectangle of the outline is a right angle; The aspect ratio of the minimum circumscribed rectangle of the outline is a predetermined aspect ratio value; The number of sides of the fitting polygon of the contour is the set number of sides; The sides formed by three adjacent points of the fitting polygon of the contour are right angles; The fitted polygon of the contour is the convex hull.

13. The device according to claim 9, characterized in that The target identification point determination unit comprises: A center point determination unit, used to determine the center point of each key frame to be identified according to the identification point of the target contour of each key frame to be identified; A target recognition point determination unit, configured to use a recognition point whose distance from the center point is less than or equal to a second distance threshold as a target recognition point corresponding to a target contour of a key frame to be recognized; The second distance threshold is equal to a second multiple of an average perimeter of the target contour in the corresponding key frame to be identified, and the second multiple is greater than the first multiple.

14. The device according to claim 9, characterized in that The target identification code detection unit comprises: A basic point determination unit, used to take the target identification point on the target common plane that is closest to the coordinate origin as a basic point; A basic point connection unit, used to connect the basic point with two target identification points adjacent to the basic point on the target common plane respectively; An identification point connecting unit, used to connect two target identification points adjacent to the basic point; The first target identification code determination unit is used to determine that the target identification code exists in the video to be identified if the basic point and two target identification points adjacent to the basic point are connected to form an isosceles right triangle.

15. The device according to claim 9, characterized in that The target identification code detection unit comprises: A basic point determination unit, used to take the target identification point on the target common plane that is closest to the coordinate origin as a basic point; A basic point connection unit, used to connect the basic point with two target identification points adjacent to the basic point on the target common plane respectively; An identification point connecting unit, used to connect two target identification points adjacent to the basic point; A recognition point fitting unit, used for fitting the shape formed by connecting the basic point and two target recognition points adjacent to the basic point, to obtain a fitting quadrilateral on the target common plane; The second target identification code determination unit is used to determine that the target identification code exists in the video to be identified if the rotation angle of the fitted quadrilateral is a right angle.

16. The device according to claim 9, characterized in that The key frame acquisition unit to be identified includes: A target video segment extraction unit, used to extract a specified time period of the video to be identified as the target video segment; An image group obtaining unit, used to obtain an image group in the target video segment; The key frame to be identified determining unit is used to take the first frame in each image group as the key frame to be identified.

17. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

18. An electronic device, comprising: at least one processor; A storage device configured to store at least one program, when the at least one program is executed by the at least one processor, causes the at least one processor to implement the method according to any one of claims 1 to 8.

19. A computer program product, comprising computer instructions, wherein a processor executes the computer instructions to implement the method according to any one of claims 1 to 8.

Images