Icon Recognition Method, Device, and Computer-Readable Storage Medium

Abstract

Embodiments of the present application relate to the field of artificial intelligence technology, and disclose an icon recognition method, device, and computer-readable storage medium; embodiments of the present application obtain an image to be recognized from a video to be detected; cut a preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area; input the image sub-block into a trained detection model to obtain the confidence of the image sub-block containing the target icon; when it is detected that the confidence is within a preset confidence range, perform optical character recognition processing on the image sub-block within the preset confidence range to obtain an optical character recognition result; determine the target recognition result of the video to be detected according to the result of the optical character recognition processing. In this way, the image to be recognized is cropped to obtain an image sub-block, the confidence of the image sub-block is determined by the detection model, and the image sub-block with the confidence within the preset confidence range is further subjected to optical character recognition processing, improving the accuracy and efficiency of icon recognition.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and particularly to an icon recognition method, apparatus, and computer-readable storage medium. Background Art

[0002] With the rise of short video creation and sharing platforms, the development of short videos has brought rich experiences to people. However, some users, for the sake of popularity, plagiarize the original authors' labor achievements by copying a large number of popular videos, which violates the rights and interests of the original authors. In order to ensure that each short video published on the platform is not of low originality, it is necessary to review each short video to be published. It should be noted that each published short video will carry a watermark mark of the corresponding platform, such as an icon. By reviewing whether each short video carries an icon, it is determined whether each short video to be published is original. In the related art, when reviewing a short video, each video frame corresponding to the short video is scanned to determine whether the short video carries an icon according to the scanning result.

[0003] In the process of researching and practicing the existing technology, the inventors of this application found that when recognizing an icon in a short video in the prior art, each video frame needs to be scanned, and then it is determined whether an icon is carried according to the scanning result. This short icon recognition method has a cumbersome process, low efficiency, and low accuracy. Currently, there is a lack of a technology for icon recognition of videos that combines high efficiency and high accuracy. Summary of the Invention

[0004] Embodiments of this application provide an icon recognition method, apparatus, and computer-readable storage medium, which can improve the efficiency and accuracy of icon recognition.

[0005] Embodiments of this application provide an icon recognition method, including:

[0006] Obtain a to-be-recognized image from a to-be-detected video;

[0007] Clip a preset area of the to-be-recognized image to obtain an image sub-block corresponding to the preset area;

[0008] Input the image sub-block into a trained detection model to obtain the confidence level that the image sub-block contains a target icon;

[0009] When it is detected that the confidence level is within a preset confidence range, perform optical character recognition processing on the image sub-blocks within the preset confidence range to obtain an optical character recognition result;

[0010] Determine the target recognition result of the to-be-detected video according to the result of the optical character recognition processing.

[0011] Correspondingly, embodiments of this application provide an icon recognition apparatus, including:

[0012] An acquisition unit, configured to acquire an image to be identified from a video to be detected;

[0013] a pre-processing unit, configured to cut a preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area;

[0014] An input unit, configured to input the image sub-block into a trained detection model to obtain a confidence level that the image sub-block contains a target icon;

[0015] a recognition unit configured to, when detecting that the confidence level is within a preset confidence level range, perform optical character recognition processing on the image sub-block within the preset confidence level range to obtain an optical character recognition result;

[0016] A determination unit is used to determine the target recognition result of the video to be detected according to the result of the optical character recognition processing.

[0017] In some embodiments, the determining unit is further configured to:

[0018] Recognize the optical character recognition result according to the part-of-speech information to obtain a part-of-speech result;

[0019] When it is detected that the part-of-speech result includes target word information in the preset word information list, determining that the target word information is a target icon, and obtaining an identifier of the target icon;

[0020] Obtaining mark position information corresponding to the target icon in the image sub-block;

[0021] The identification and mark position information of the target icon are determined as the target recognition result.

[0022] In some embodiments, the determining unit is further configured to:

[0023] When it is detected that the part-of-speech result does not include the target word information in the preset word information list, performing optical character recognition processing on the image to be recognized to obtain an optical character recognition result corresponding to the image to be recognized;

[0024] Recognize the optical character recognition result corresponding to the image to be recognized according to the part-of-speech information to obtain the word information result corresponding to the image to be recognized;

[0025] Matching the word information result with the target word information in the preset word information list;

[0026] When the number of matched target word information is greater than or equal to a preset word information number threshold, determining the matched target word information as the target icon in the image to be recognized;

[0027] Obtain the identifier of the target icon and obtain the marked position information of the target icon in the to-be-recognized image;

[0028] Determine the identifier of the target icon and the marked position information as the target recognition result.

[0029] In some embodiments, the input unit is further configured to:

[0030] Input the image sub-block into the trained detection model;

[0031] Extract features from the image sub-block through the detection model to obtain image sub-features;

[0032] Perform convolution processing on the image sub-features through the detection model to obtain a predicted feature map;

[0033] Decode the dimension of the predicted feature map through the classification layer in the detection model to obtain the confidence that the image sub-block contains the target icon.

[0034] In some embodiments, the determination unit is further configured to:

[0035] When it is detected that the confidence is greater than the preset confidence range, determine that the image sub-block corresponding to the confidence contains the target icon;

[0036] Obtain the marked position information of the image sub-block corresponding to the confidence and obtain the identifier of the target icon;

[0037] Determine the identifier of the target icon and the marked position information as the target recognition result of the to-be-detected video.

[0038] In some embodiments, the determination unit is further configured to:

[0039] When it is detected that the confidence is less than the preset confidence range, determine that the to-be-recognized image does not carry the target icon;

[0040] Determine that not carrying the target icon is the target recognition result of the to-be-detected video.

[0041] In some embodiments, the icon recognition device further includes a training unit, configured to:

[0042] Obtain a sample image, where a preset area in the sample image carries a sample icon;

[0043] Clip the sample image to obtain a sample image sub-block carrying the sample icon, and obtain the sample confidence that the sample image sub-block contains the sample icon;

[0044] Input the sample image sub - block into a preset model to obtain the predicted confidence that the sample image sub - block contains the sample icon;

[0045] Obtain the confidence difference between the sample confidence and the predicted confidence;

[0046] Iteratively train the network parameters of the preset model according to the confidence difference until the confidence difference converges, and obtain the trained detection model.

[0047] In some embodiments, the obtaining unit is further configured to:

[0048] Receive a video to be detected;

[0049] Extract the target image frames from the video to be detected;

[0050] Determine the target image frame as the image to be recognized.

[0051] In addition, an embodiment of the present application further provides a computer device, including a processor and a memory. The memory stores an application program, and the processor is configured to run the application program in the memory to implement the icon recognition method provided by the embodiment of the present application.

[0052] In addition, an embodiment of the present application further provides a computer - readable storage medium. The computer - readable storage medium stores multiple instructions, and the instructions are suitable for being loaded by a processor to execute the steps in any of the icon recognition methods provided by the embodiment of the present application.

[0053] In addition, an embodiment of the present application further provides a computer program. The computer program includes computer instructions, and the computer instructions are stored in a computer - readable storage medium. The processor of the computer device reads the computer instructions from the computer - readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in any of the icon recognition methods provided by the embodiment of the present application.

[0054] Embodiments of the present application can obtain an image to be recognized from a video to be detected; cut a preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area; input the image sub-block into a trained detection model to obtain the confidence level of the image sub-block containing the target icon; when it is detected that the confidence level is within a preset confidence range, perform optical character recognition processing on the image sub-block within the preset confidence range to obtain an optical character recognition result; determine the target recognition result of the video to be detected according to the result of the optical character recognition processing. Thus, when detecting the video to be detected, embodiments of the present application obtain the image to be recognized corresponding to the video to be detected, cut it to obtain an image sub-block, and perform target icon recognition on the image sub-block through a trained detection model, avoiding full-image recognition of the image to be recognized, reducing the recognition amount of the area of the image to be recognized, and improving the icon recognition efficiency; after obtaining the confidence level of the target icon contained in the image sub-block, determine whether to perform optical character recognition processing on the image to be recognized according to the confidence level, improving the accuracy when recognizing icons in the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can also be obtained according to these drawings.

[0056] Figure 1 It is a schematic diagram of the scenario of the icon recognition system provided by the embodiments of the present application;

[0057] Figure 2 It is a schematic diagram of the step flow of the icon recognition method provided by the embodiments of the present application;

[0058] Figure 3 It is another schematic diagram of the step flow of the icon recognition method provided by the embodiments of the present application;

[0059] Figure 4 It is a block diagram of the icon recognition method provided by the embodiments of the present application;

[0060] Figure 5 It is a schematic diagram of a sample image carrying a sample icon provided by the embodiments of the present application;

[0061] Figure 6 It is a schematic diagram of the scenario of the icon recognition method provided by the embodiments of the present application;

[0062] Figure 7 It is a schematic diagram of the structure of the icon recognition device provided by the embodiments of the present application;

[0063] Figure 8It is a schematic structural diagram of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0064] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0065] The embodiments of the present application provide an icon recognition method, device, and computer-readable storage medium. Specifically, the embodiments of the present application will be described from the perspective of an icon recognition device, which can be specifically integrated in a computer device. The computer device can be a server or a terminal device, etc. Among them, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication methods, and the present application does not limit this.

[0066] Artificial Intelligence (AI) is a theory, method, technology, and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling the machines to have the functions of perception, reasoning, and decision-making.

[0067] The solution provided by the embodiments of the present application relates to technologies such as icon recognition in artificial intelligence, and will be specifically described through the following embodiments:

[0068] For example, refer to Figure 1 , which is a schematic diagram of the scenario of the icon recognition system provided by the embodiments of the present application. This scenario includes a terminal 10 and a server 20, and the terminal 10 and the server 20 are wirelessly communicatively connected to implement data interaction.

[0069] The user selects the video to be uploaded through the terminal 10 and uploads the video (the video to be detected) to the server 20 corresponding to the corresponding platform through the terminal 10, so that the server 20 audits the video to be detected, such as the recognition of the target icon in the video.

[0070] The server 20 can obtain the image to be recognized from the video to be detected; cut the preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area; input the image sub-block into the trained detection model to obtain the confidence level that the image sub-block contains the target icon; when it is detected that the confidence level is within the preset confidence range, perform optical character recognition processing on the image sub-block within the preset confidence range to obtain the optical character recognition result; determine the target recognition result of the video to be detected according to the result of the optical character recognition processing.

[0071] Among them, icon recognition can include processing methods such as obtaining the image to be recognized corresponding to the video to be detected, determining the confidence level, and optical character recognition processing.

[0072] The following will be described in detail respectively. It should be noted that the order of the following embodiments does not limit the preferred order of the embodiments.

[0073] See Figure 2 , Figure 2 which is a schematic diagram of the step flow of an icon recognition method provided by an embodiment of the present application. The specific process is as follows:

[0074] 101. Obtain the image to be recognized from the video to be detected.

[0075] Among them, the video to be detected can be a video to be detected and audited that the user requests to publish to the target sharing platform, and the target allocation platform is responsible for auditing and detecting the video to be detected. Such as the videos to be published on target sharing platforms such as short video platforms and material platforms. If the video to be detected is downloaded by the requesting user through other platforms, that is, it is not the original creation of the requesting user, then the video to be detected may carry watermarks, logos, tags or marks of the corresponding platform. Therefore, it is necessary to audit the video to be published.

[0076] Among them, the image to be recognized can be the image corresponding to a certain video frame in the video to be recognized. For example, when the user requests to publish a short video to the target sharing platform, such as a video platform, when the target sharing platform detects the short video to be detected, by obtaining the target video frame in the video and determining the target video frame as the image to be recognized, so as to facilitate subsequent icon recognition of the image to be recognized. Such as the icon can be a watermark, mark, tag, etc.

[0077] In some embodiments, taking a video to be released as an example, it is necessary to review or identify the images corresponding to the video frames in the video. The step of "obtaining the image to be recognized from the video to be detected" may include:

[0078] (1) Receive the video to be detected;

[0079] (2) Extract the target image frames from the video to be detected;

[0080] (3) Determine the target image frame as the image to be recognized.

[0081] Among them, the video to be detected can be a video requested by a user to be released on a target platform. For example, platform A needs to review the video to be detected.

[0082] In order to review the video to be detected, the embodiments of the present application identify the images corresponding to the video frames in the video to be detected. Since the video to be detected contains multiple video frames, in order to reduce the workload in the process of detecting the video, in this embodiment, a small number of video frames among multiple video frames are selected as the images to be detected. Specifically, when receiving the video to be detected, the method of extracting the target image frames from the video to be detected can be: extracting all the video frames in the video to be detected to obtain a video frame set corresponding to the video to be detected; sorting the video frames in the video frame set according to the temporal relationship between the video frames in the video to be detected to obtain a video frame sequence; obtaining the target image frames corresponding to the target timings in the video frame sequence. Among them, the target image frames corresponding to the target timings can be the target image frames corresponding to the first timing, the middle timing, or the end timing in the video frame sequence, or can also be the target image frames corresponding to the first, middle, and last three video frames respectively, which is not limited here. In this way, it is convenient to improve the efficiency in subsequent icon recognition.

[0083] Further, after extracting the target image frames of the video to be detected, the target image frames are determined as the images to be recognized to be reviewed by the platform.

[0084] 102. Cut a preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area.

[0085] Among them, the preset area can be a certain area or position in the image to be recognized, such as image areas such as the upper left, lower left, upper right, lower right, and middle in the image to be recognized.

[0086] Since after a video is published on a certain platform, the watermark mark (icon) it carries is usually located in the corner area of the video screen, such as the upper left or upper right corner area of the video screen, etc. In order to improve the detection efficiency of the watermark mark (icon), the watermark mark (icon) can be detected by performing detection on a specific area of the image to be recognized. Specifically, after obtaining the corresponding image to be recognized from the video to be detected in an embodiment of the present application, preprocessing is performed on a preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area.

[0087] It should be noted that the method for preprocessing the image to be recognized can be: cutting, capturing or intercepting a preset area of the image to be recognized to obtain a corresponding image sub-block. Specifically, the preprocessing process can be: obtaining the image size of the image to be recognized; based on the size of the image to be recognized, determining the size of the preset area of the image to be recognized according to the preprocessing rule; intercepting the preset area in the image to be recognized according to the preset area size to obtain an image sub-block. Among them, the preprocessing rule can include the position of the preset area of the image to be processed, the size ratio of the preset area in the image to be recognized, etc. For example, in the preprocessing rule, the preset area includes areas such as the upper left, lower left, upper right, lower right, and middle of the image to be recognized, and the size ratio of this area in the image to be recognized is 1 / 5. Then, according to this size ratio, the upper left, lower left, upper right, lower right, and middle areas in the image to be recognized are intercepted respectively to obtain image sub-blocks corresponding to these areas. The above is only an example and is not limited here.

[0088] It should be noted that when preprocessing the preset area of the image to be recognized, only one area of the image to be recognized can be preprocessed. In addition, multiple preset areas in the image to be recognized can also be preprocessed simultaneously. For example, the upper left, lower left, upper right, lower right, and middle areas in the image to be recognized are preprocessed simultaneously to obtain image sub-blocks corresponding to each area, that is, multiple image sub-blocks, so as to facilitate subsequent icon recognition of multiple image sub-blocks, avoid the phenomenon of missed detection of icons (watermark marks), and improve the accuracy when detecting icons in the video.

[0089] Through the above method, the image to be recognized is cut according to the preprocessing rule to improve the accuracy of the image sub-block corresponding to the preset area. While ensuring that the target icon (watermark mark) to be detected is located within the image sub-block as much as possible, at the same time, the size of the image sub-block is reduced as much as possible to reduce the image size during subsequent icon recognition of the image sub-block and improve the efficiency during subsequent icon recognition of the image sub-block.

[0090] Through the above method, so that when the subsequent detection model performs icon recognition, it only recognizes the image sub-block, avoids full-image recognition of the image to be recognized, reduces the amount of recognition of the area of the image to be recognized, and improves the icon recognition efficiency.

[0091] 103. Input the image sub - block into the trained detection model to obtain the confidence that the image sub - block contains the target icon.

[0092] Among them, the detection model can be a model including a feature extraction layer (Darknet - 53), a convolutional network layer (Convolutional Set), a classification layer (sigmoid), etc. Among them, the feature extraction layer is used to perform basic feature extraction on the image; the convolutional network layer is used to perform convolutional processing on the extracted features to obtain a prediction feature map; the classification layer is used to convert the prediction feature map into a prediction result.

[0093] Among them, the detection model can be an icon recognition model that is iteratively trained until convergence, used to recognize or detect the target icon at a specific position in the image to be recognized, so as to determine the confidence of the target icon at the specific position. Specifically, the trained detection model is jointly trained by the sample image sub - blocks in the preset area of the sample image and the sample confidence that the sample image sub - blocks contain sample icons.

[0094] Among them, the confidence can be the credible value corresponding to the icon event contained in a certain area of the image to be recognized, used to reflect the credibility or probability that a certain area or position in the image to be recognized contains the target icon.

[0095] In order to detect icons in the image to be recognized, the embodiment of the present application inputs the image sub - block into the trained detection model to obtain the confidence value that the image sub - block contains the target icon.

[0096] In some embodiments, the step of "inputting the image sub - block into the trained detection model to obtain the confidence that the image sub - block contains the target icon" includes:

[0097] (1) Input the image sub - block into the trained detection model;

[0098] (2) Perform feature extraction on the image sub - block through the feature extraction layer in the detection model to obtain image sub - features;

[0099] (3) Perform convolutional processing on the image sub - features through the convolutional layer in the detection model to obtain a prediction feature map;

[0100] (4) Decode the dimension of the prediction feature map through the classification layer in the detection model to obtain the confidence that the image sub - block contains the target icon.

[0101] Among them, when the detection model receives an image sub-block, it extracts features from the image sub-block through the feature extraction layer (Darknet-53) in the detection model to obtain image sub-features corresponding to the image sub-block; it performs convolution processing on the image sub-features through the convolutional network layer (Convolutional Set) in the detection model to obtain a predicted feature map; it obtains the dimension of the predicted feature map through the classification layer (Sigmoid) and decodes the dimension of the predicted feature map to obtain the confidence of the image sub-block containing the target icon.

[0102] In some embodiments, the convolutional network layer includes a first convolutional network layer, a second convolutional network layer, and a third convolutional network layer. The step of "performing convolution processing on the image sub-features through the convolutional layer in the detection model to obtain a predicted feature map" includes:

[0103] (3.1) Perform convolution processing on the image sub-features through the first convolutional network layer in the convolutional network layer to obtain a first predicted feature map.

[0104] Specifically, the first convolutional network layer includes a 3*3 convolutional layer and a 1*1 convolutional layer. Among them, convolution processing is performed on the image sub-features through the 3*3 convolutional layer and the 1*1 convolutional layer in the first convolutional network layer respectively to obtain a first predicted feature map.

[0105] (3.2) Perform convolution processing on the image sub-features through the second convolutional network layer in the convolutional network layer to obtain a second predicted feature layer.

[0106] Specifically, the second convolutional network layer includes a 1*1 convolutional layer, an upsampling layer, a fusion layer, a 3*3 convolutional layer, and a 1*1 convolutional layer. Among them, convolution processing is performed on the image sub-features through the 1*1 convolutional layer in the second convolutional network layer to obtain a first convolutional feature result; the first convolutional feature result is upsampled through the upsampling layer to obtain an upsampled first feature result; the upsampled first feature result is fused with the 26*26 feature result output by the feature extraction layer (Darknet-53) through the fusion layer to obtain a first fused feature; the first fused feature is subjected to convolution processing on the image sub-features through the 3*3 convolutional layer and the 1*1 convolutional layer respectively to obtain a second predicted feature layer.

[0107] (3.3) Perform convolution processing on the image sub-features through the third convolutional network layer in the convolutional network layer to obtain a third predicted feature layer.

[0108] Specifically, the third convolutional network layer includes a 1×1 convolutional layer, an upsampling layer, a fusion layer, a 3×3 convolutional layer, and a 1×1 convolutional layer. Among them, the image sub-features are convolved through the 1×1 convolutional layer in the third convolutional network layer to obtain the first convolutional feature result; the convolutional feature result is upsampled through the upsampling layer to obtain the first feature result after upsampling; the first feature result after upsampling is fused with the 26×26 feature result output by the feature extraction layer (Darknet-53) through the fusion layer to obtain the first fusion feature.

[0109] Further, the first fusion feature is convolved through a 1×1 convolutional layer to obtain a second convolutional feature result; the second convolutional feature result is upsampled through the upsampling layer to obtain the second feature result after upsampling; the second feature result after upsampling is fused with the 52×52 feature result output by the feature extraction layer (Darknet-53) through the fusion layer to obtain the second fusion feature.

[0110] The second fusion feature is sequentially passed through a 3×3 convolutional layer and a 1×1 convolutional layer to respectively perform convolutional processing on the image sub-features, obtaining the third prediction feature layer.

[0111] (3.4) Determine the first prediction feature map, the second prediction feature map, and the third prediction feature map as the prediction feature map.

[0112] In some embodiments, before the step of "inputting the image sub-block into the trained detection model", it further includes:

[0113] A. Obtain a sample image, where a preset area in the sample image carries a sample icon.

[0114] Among them, the sample image can be an image with a sample icon, the sample image can be a directly obtained image, or can also be an image corresponding to a video frame in a sample video, which is not limited here. It should be noted that a preset area in the sample image carries a sample icon.

[0115] Among them, the preset area can be a certain area or position in the sample image, such as image areas in the upper left, lower left, upper right, lower right, middle, etc. of the sample image.

[0116] It should be noted that the sample image can be an artificially synthesized image. Since it is rather cumbersome to obtain the sample data of the watermark video, in the embodiments of this application, a data synthesis method is adopted to fit the real data, that is, the sample image containing the sample icon. During data synthesis, the randomness of the size and position of the sample icon (watermark mark) can be ensured. For example, the icon can be set in any preset area in the sample image, such as any one or multiple image areas in the upper left, lower left, upper right, lower right, or middle of the sample image, and the icon size is random, but the size of the icon is less than the preset area, which is not limited here. For example, when synthesizing the sample image by the data synthesis method, the corresponding sample icon can be obtained from the sample icons or watermark marks of existing sharing platforms or enterprises, and then synthesized with the preselected image to ensure the randomness of the size and position of the sample icon (watermark mark), and perform random cropping or occlusion processing on the sample icon (watermark mark) to obtain the sample image. In addition, data enhancement processing such as blurring, occlusion, brightness, color change, and adding noise can be performed on the preselected image before synthesizing the sample image or the synthesized sample image to further obtain the corresponding sample image for use in model training.

[0117] B. Shear the sample image to obtain a sample image sub-block carrying the sample icon, and obtain the sample confidence of the sample icon contained in the sample image sub-block.

[0118] In order to improve the efficiency in subsequent model training, in the embodiments of this application, the preset area in the sample image is sheared to obtain sample image sub-blocks of corresponding sizes. In this way, when subsequently training the preset model, the speed of the preset model for identifying the icon (watermark mark) in the image is reduced, thereby improving the speed of subsequent iterative training and accelerating the model training efficiency.

[0119] In order to train the preset model, it is necessary to obtain the sample confidence of the sample icon contained in the preset area of the sample image. This sample confidence can be set by the way of human perception, representing the credibility or probability value of the sample icon contained in the preset area of the sample image.

[0120] C. Input the sample image sub-block into the preset model to obtain the predicted confidence of the sample icon contained in the sample image sub-block.

[0121] D. Obtain the confidence difference between the sample confidence and the predicted confidence.

[0122] E. Iteratively train the network parameters of the preset model according to the confidence difference until the confidence difference converges to obtain the trained detection model.

[0123] Specifically, the preset model can be a model including a feature extraction layer (Darknet-53), a convolutional network layer (Convolutional Set), and a classification layer (Sigmoid). Among them, the convolutional network layer in the preset model performs convolutional processing on the preset area of the sample image to obtain sub-features of the sample image; the convolutional network layer in the detection model performs convolutional processing on the sub-features of the sample image to obtain a sample of the predicted feature map; the classification layer obtains the dimension of the predicted feature map and decodes the dimension of the predicted feature map sample to obtain the confidence of the sample image sub-block containing the target icon.

[0124] Among them, during the model training process, the predicted confidence is the credibility or probability obtained by the preset model based on the input sample image. The predicted confidence reflects the credibility or probability of the preset model in recognizing that the preset area of the sample image contains the sample icon, and there may be a certain difference between it and the actually set sample confidence.

[0125] In order to make the confidence output by the preset model more accurate when recognizing a video, or closer to the detection or recognition of human perception, the embodiment of this application calculates the confidence difference between the sample confidence and the predicted confidence, and determines the gap between the current preset model and the ideal target detection model through the confidence difference. Thus, by adjusting the network parameters of the preset model and performing iterative training after adjustment until there is no gap or the two are very close between the predicted confidence and the sample confidence, the trained detection model is obtained by default. In this way, the preset model is jointly trained through the sample image sub-block and the sample confidence to obtain the trained detection model.

[0126] The model is trained in the above manner to obtain a trained target model, that is, a detection model, and based on the detection model, the image sub-blocks of the image to be recognized are recognized to improve the recognition efficiency and accuracy of the graphic identification in the video.

[0127] 104. When it is detected that the confidence is within the preset confidence range, perform optical character recognition processing on the image sub-block within the preset confidence range to obtain the optical character recognition result.

[0128] Among them, the preset confidence range can be the preset value range corresponding to the confidence, which is used to distinguish the situation of the image to be recognized containing an icon (watermark), and limit the image sub-block corresponding to the confidence within this value range to perform optical character recognition processing.

[0129] Among them, Optical Character Recognition (OCR) can be a process of determining the shape of corresponding characters by detecting dark and bright patterns when recognizing the characters, and then translating the shape into computer text using character recognition methods; that is, converting image sub-blocks or icons in an image into corresponding image files in an optical manner, and converting the text in the image into a text format through recognition software for further editing, processing, or other operations by a word processing software.

[0130] To improve the accuracy during icon (watermark) detection, after obtaining the confidence level of an image sub-block containing a target icon in an embodiment of the present application, it can be determined whether to perform optical character recognition on the corresponding image sub-block according to the magnitude of the confidence level. Specifically, in an embodiment of the present application, a confidence level value range is set, that is, a preset confidence range. Since the credibility of the image sub-block corresponding to the confidence level within this range containing the target icon is relatively high, but it cannot be directly determined whether it contains an icon or a watermark. To improve the accuracy of icon (watermark) detection, optical character recognition can be performed on the image sub-block corresponding to the confidence level within this preset confidence range to convert the character information contained in the icon in the image sub-block into text information, obtaining an optical character recognition result.

[0131] For example, assuming the value range of the confidence level is [0, 1], and the preset confidence range set in an embodiment of the present application is [0.5, 0.9]. The image sub-blocks input into the detection model include the image sub-blocks corresponding to the upper left, lower left, upper right, lower right, and middle image regions of the image to be recognized. Among them, the image sub-block in the upper left image region of the image to be recognized has the highest confidence level of containing the target icon. For example, the confidence level of the upper left image region is 0.8 (the confidence levels of other image sub-blocks may be 0 or values less than 0.5). Then, since the confidence level of the image sub-block corresponding to the upper left image region containing the target icon is within the preset confidence range [0.5, 0.9], optical character recognition is performed on the image sub-block corresponding to the upper left region to convert the character information contained in the icon in the image sub-block into text information, obtaining an optical character recognition result.

[0132] It should be noted that the optical character recognition result contains word information or text. Subsequently, it is determined whether the image sub-block contains the target icon based on the word information or text contained in the optical character recognition result, so as to determine whether the video to be detected contains the target icon (watermark).

[0133] 105. Determine the target recognition result of the video to be detected according to the result of the optical character recognition process.

[0134] The target recognition result is the result of icon recognition for the video to be detected. Specifically, when it is detected that an image sub-block contains the target icon, the target recognition result may include: the marked position information, identification, etc. of the target icon in the image or video frame to be recognized; when it is detected that the image sub-block does not contain the target icon, the target recognition result may be that the video to be detected does not carry the target icon, or other expressions, which are not limited here.

[0135] In some embodiments, the step of "determining the target recognition result of the video to be detected according to the result of optical character recognition processing" includes:

[0136] (1) Recognize the optical character recognition result according to the word property information to obtain a word property result.

[0137] The word property information may be information based on which word classes are divided, and is used to divide relevant texts into multiple word features or word information. For example, after obtaining the optical character recognition result, the words or characters in the optical character recognition result are divided according to the word property information to obtain a word property result, and the word property result may include word information.

[0138] (2) When it is detected that the word property result contains the target word information in the preset word information list, determine the target word information as the target icon and obtain the identification of the target icon.

[0139] The preset word information list may be a list containing the text information or word information corresponding to existing icons (watermarks). For example, the preset word information list includes "A", "A icon", "A game", "A video", "B", "B icon", "B video", "B headline", etc.

[0140] In order to identify whether an image sub-block with a confidence level within the preset confidence range contains a target icon, in the embodiment of the present application, after obtaining the word property result, it is detected whether the word property result contains word information matching the target word information in the preset word information list. When the word property result contains one or more target word information in the preset word information list, the target word information is determined as the target icon, and the identification of the target icon is obtained. For example, assuming that the word property result contains the word information of "A video", the word property result is matched with the word features in the preset word information list. When the target word information in the preset word information list matches the word information in the word property result, such as "A video" in the preset word information list matches "A video" in the word property result, then "A video" is determined as the target icon, and the identification corresponding to the target icon obtained may be "A video", "A", etc.

[0141] (3) Obtain the marked position information corresponding to the target icon in the image sub-block.

[0142] Among them, the mark position information can be the position information of the image sub-block carrying the target icon in the image to be identified (the video frame of the video to be detected). The position information of the image sub-block in the image to be identified can reflect the position of the target icon (watermark) in the video to be identified.

[0143] To obtain the position information of the target icon in the image to be identified, the embodiment of the present application determines the position information of the corresponding image sub-block in the image to be identified. Specifically, by identifying the position information of the image sub-block corresponding to the confidence level in the image to be identified, this position information is determined as the marker position information of the target icon.

[0144] (4) The identification and marking position information of the target icon are determined as the target recognition result.

[0145] In an embodiment of the present application, after obtaining the mark position information of the image sub-block corresponding to the confidence level, that is, the mark position information of the target icon and the identifier of the target icon, the corresponding target recognition result is generated based on the mark position information and identifier of the target icon.

[0146] In an embodiment of the present application, since the confidence that the corresponding image sub-block contains the target icon is within a preset confidence range, this may be because the model recognizes the icon features of a similar icon, but does not completely match the actual icon, or because the clarity of the target icon is low, resulting in the confidence output by the detection model when detecting the target icon being within the preset confidence range.

[0147] Due to the large size of the target icon, the complete icon is not covered in the image sub-block, resulting in the confidence that the corresponding image sub-block contains the target icon being within the preset confidence range. However, the part-of-speech result corresponding to the optical character recognition result does not include the target word information in the preset word information list. Therefore, it is necessary to perform optical character recognition on the entire area of the image to be recognized, that is, full-image recognition, to determine whether the image to be recognized contains the target icon.

[0148] Therefore, in some embodiments, the present invention further includes:

[0149] A. When it is detected that the part-of-speech result does not contain the target word information in the preset word information list, optical character recognition processing is performed on the image to be recognized to obtain an optical character recognition result corresponding to the image to be recognized, and the optical character recognition result corresponding to the image to be recognized is recognized according to the part-of-speech information to obtain a word information result corresponding to the image to be recognized.

[0150] The word information result may include multiple word information corresponding to all texts in the image to be recognized.

[0151] Since the image sub-blocks with confidence levels within the preset confidence range belong to specific regions (preset regions, such as the upper left, lower left, upper right, lower right, middle, etc.) in the image to be recognized, the target icon information they reflect is one-sided. To improve the accuracy of icon recognition, when the image sub-blocks with confidence levels within the preset confidence range are detected not to contain the target word information in the preset word information list, the embodiments of the present application can perform optical character recognition on the image to be recognized corresponding to one or more video frames in the video to be detected, that is, perform optical character recognition on the entire region of the image to be recognized, obtain the optical character recognition result corresponding to the image to be recognized, and perform conversion of word information on the optical character recognition result corresponding to the image to be recognized to obtain the word information result of the entire region of the image to be recognized, so as to facilitate determining whether the target icon (watermark) is included according to the word information result in the subsequent process.

[0152] It should be noted that optical character recognition processing is performed on the entire region of the image to be recognized, where the entire region refers to all regions in the image to be recognized, that is, any region in the entire image. For example, an entire image to be recognized is recognized to obtain all the optical character recognition results corresponding to the image to be recognized.

[0153] B. Match the word information result with the target word information in the preset word information list.

[0154] C. When the number of the matched target word information is greater than or equal to the preset word information quantity threshold, determine the matched target word information as the target icon in the image to be recognized.

[0155] Among them, the preset word information quantity threshold can be a value for determining whether the image to be recognized contains the target icon. For example, when the word information included in the image to be recognized is matched with the preset word information list to obtain the number of the matched target word information, when the number of the target word information is greater than or equal to the threshold, it is determined that the image to be recognized contains the target icon.

[0156] D. Obtain the identifier of the target icon and obtain the marked position information of the target icon in the image to be recognized.

[0157] E. Determine the identifier and marked position information of the target icon as the target recognition result.

[0158] In order to improve the accuracy of identifying icons (watermarks), the embodiment of the present application can perform optical character recognition processing on the entire image or the entire area of the image to be identified corresponding to the video frame in the video to be detected, and obtain the optical character recognition result corresponding to the entire area; perform part-of-speech information recognition on the optical character recognition result to obtain the word information result of the entire area in the image to be identified; match the word information result with the preset word information list to determine the number of matched target word information; when the number of target word information contained in the word information result is greater than or equal to the preset word information number threshold, the matched target word information is determined as the target icon in the image to be identified, or it is determined that the image to be identified contains the target icon (watermark).

[0159] For example, the preset word information list may also include "Come to A (application) to pay attention to hot current events", "Come to A (application) to pay attention to the development of events", "Use the latest version of A (application) to scan the code and add me as a friend", etc. Since the last frame in the video to be detected may contain a relatively large amount of relevant information, such as a relatively large amount of word information associated with the target icon, when the part-of-speech result of the image sub-block whose confidence is within a preset confidence range does not include the target word information in the preset word information list, optical character recognition is performed on the entire region of the image corresponding to the last frame in the video to be detected to obtain an optical character recognition result corresponding to the image corresponding to the last frame; the optical character recognition result is converted into a word information result containing one or more word information. When the word information result matches multiple target word information in the preset word information list, such as the matched target word information is "A (application)", "Come to A (application) and pay attention to hot current events", or "Use the latest version of A (application) to scan the code and add me as a friend", the number of matched target word information is 3, and the preset word information number threshold is 2. Since the number of matched target word information is greater than the preset word information number threshold, it is determined that the image to be identified contains the target icon. Specifically, all matched target word information can be determined as the target icon, or word information with a higher frequency of occurrence among the multiple matched target word information can be determined as the target icon.

[0160] Furthermore, in order to obtain a target recognition result that includes the identification and mark position information of the target icon, the embodiment of the present application obtains the mark position information of the image sub-block corresponding to the confidence level, that is, the mark position information of the target icon, and obtains the identification of the target icon after determining that the image sub-block corresponding to the confidence level contains the target icon, thereby generating a corresponding target recognition result based on the mark position information and identification of the target icon.

[0161] In addition, in an embodiment of the present application, when the confidence that the detection model output image sub-block contains the target icon is greater than the preset confidence range, since the confidence is relatively high, it can be directly determined that the image sub-block corresponding to the confidence contains the target icon, that is, the video to be detected carries the target image (watermark mark).

[0162] Among some embodiments, the embodiments of the present application further include:

[0163] (1) When it is detected that the confidence level is greater than the preset confidence range, it is determined that the image sub-block corresponding to the confidence level contains the target icon;

[0164] (2) Obtain the marked position information of the image sub-block corresponding to the confidence level, and obtain the identifier of the target icon;

[0165] (3) Determine the identifier of the target icon and the marked position information as the target recognition result of the video to be detected.

[0166] Among them, the identifier of the target icon can be the identifier or mark of a certain platform, indicating the mark given after the video to be detected was previously successfully published on the corresponding platform. For example, when Video 1 was previously published on Sharing Platform A, Sharing Platform A will attach an A icon (watermark mark) to a certain area in the picture of this Video 1, then the icon (watermark) carried by Video 1 is the A icon.

[0167] Among them, the marked position information can be the position information of the image sub-block carrying the target icon in the image to be recognized (the video frame picture of the video to be detected). The position of the target icon (watermark) in the video to be recognized can be reflected through the position information of the image sub-block in the image to be recognized.

[0168] It should be noted that when the confidence level output by the detection model is greater than the preset confidence range, due to the relatively high confidence level, it indicates that the corresponding image sub-block contains the target icon (watermark mark). Therefore, it can be directly determined that the image sub-block corresponding to this confidence level contains the target icon. For example, when Video 1 was previously published on Sharing Platform A, Sharing Platform A will attach an A icon (watermark mark) to a certain area in the picture of this Video 1; when the user downloads this Video 1 from Sharing Platform A and publishes it to Sharing Platform B to move the Video 1 that has been published on Sharing Platform A to Sharing Platform B; since this Video 1 contains the A icon, when Sharing Platform B detects Video 1, including detecting whether Video 1 contains the target icon (watermark mark), specifically, it is detected through the trained model, and the confidence level that the image sub-block corresponding to Video 1 contains the A icon is output as 0.98, while the preset confidence range is [0.5, 0.9], and the embodiments of the present application can determine that the corresponding image sub-block contains the target icon.

[0169] Further, in order to obtain the target recognition result including the identifier and marked position information of the target icon, after the embodiments of the present application determine that the image sub-block corresponding to the confidence level contains the target icon, they obtain the marked position information of the image sub-block corresponding to the confidence level, that is, the marked position information of the target icon, and obtain the identifier of the target icon, so as to generate the corresponding target recognition result according to the marked position information and identifier of the target icon.

[0170] In addition, in the embodiments of the present application, when the confidence level of the image sub-block output by the detection model containing the target icon is less than the preset confidence range, due to the relatively low confidence level, it can be directly determined that the image sub-block corresponding to the confidence level does not contain the target icon, that is, the video to be detected does not carry the target image (watermark).

[0171] In some embodiments, the embodiments of the present application further include:

[0172] A. When it is detected that the confidence level is less than the preset confidence range, it is determined that the image to be recognized does not carry the target icon;

[0173] B. Determining that the absence of the target icon is the target recognition result of the video to be detected.

[0174] It should be noted that when the confidence level output by the detection model is less than the preset confidence range, due to the relatively low confidence level, it indicates that the corresponding image sub-block does not contain the target icon (watermark). Therefore, it can be directly determined that the image sub-block corresponding to this confidence level does not contain the target icon, that is, the video to be detected does not carry the target icon.

[0175] For example, when Video 2 has not been previously published on any other sharing platform, any area in the picture of this Video 2 will not contain the target icon (watermark); when the user publishes this Video 2 to Sharing Platform B, since this Video 2 does not contain any target icons, when Sharing Platform B detects Video 1, it specifically performs icon detection on the image sub-block corresponding to Video 2 through the trained model, and no target icon (watermark) is detected in the image sub-block, then a relatively low confidence level, such as 0.2, is output. Since this confidence level is less than the preset confidence range of [0.5, 0.9], it indicates that the credibility of the image sub-block corresponding to this confidence level containing the target icon is relatively low. Therefore, the embodiments of the present application can determine that the corresponding image sub-block does not contain the target icon, that is, it is determined that Video 1 does not carry the target icon.

[0176] Furthermore, when it is determined that the image to be recognized does not carry the target icon, the target recognition result of the video to be detected is that the video to be detected does not carry the target icon.

[0177] As can be seen from the above, the embodiments of the present application can obtain the image to be recognized from the video to be detected; clip the preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area; input the image sub-block into the trained detection model to obtain the confidence level of the target icon included in the image sub-block; when it is detected that the confidence level is within the preset confidence range, perform optical character recognition processing on the image sub-block within the preset confidence range to obtain the optical character recognition result; determine the target recognition result of the video to be detected according to the result of the optical character recognition processing. It can be seen that when detecting the video to be detected, the embodiments of the present application obtain the image to be recognized corresponding to the video to be detected, preprocess it to obtain image sub-blocks, and use the trained detection model to recognize the target icon in the image sub-blocks, avoiding full-image recognition of the image to be recognized, reducing the recognition amount of the area of the image to be recognized, and improving the icon recognition efficiency; after obtaining the confidence level of the target icon included in the image sub-block, determine whether to perform optical character recognition processing on the image to be recognized according to the confidence level, improving the accuracy when recognizing the icon in the video.

[0178] According to the method described in the above embodiments, the following will be further described in detail with examples.

[0179] Taking icon recognition as an example, the embodiments of the present application further describe the icon recognition method provided by the embodiments of the present application.

[0180] See Figure 3 , Figure 3 which is another step flow schematic diagram of the icon recognition method provided by the embodiments of the present application, Figure 4 which is a flow block diagram of the icon recognition method provided by the embodiments of the present application, Figure 5 which is a schematic diagram of a sample image carrying a sample icon provided by the embodiments of the present application, Figure 6 which is a schematic diagram of the scenario of the icon recognition method provided by the embodiments of the present application; for ease of understanding, please also refer to Figure 3 、 Figure 4 、 Figure 5 and Figure 6 to describe the embodiments of the present application.

[0181] In the embodiments of the present application, the description will be made from the perspective of an icon recognition device, which can be specifically integrated in computer devices such as terminals or servers. When the processor on the terminal or server executes the program corresponding to the icon recognition method, the specific process of the icon recognition method is as follows:

[0182] 201. Obtain the image to be recognized from the video to be detected.

[0183] The video to be detected can be a video to be detected and audited that the user requests to publish to a target sharing platform. The target allocation platform is responsible for auditing and detecting the video to be detected. For example, videos to be published on target sharing platforms such as short video platforms and material platforms. If the video to be detected is downloaded by the requesting user through other platforms and is not the original creation of the requesting user, the video to be detected may carry watermarks, logos, tags, or marks of the corresponding platform. Therefore, it is necessary to audit the video to be distributed.

[0184] Among them, the image to be recognized can be an image corresponding to a certain video frame in the video to be recognized. For example, when the user requests to publish a short video to a target sharing platform, such as a video platform, when the target sharing platform detects the short video to be detected, by obtaining the target video frame in the video, the target video frame is determined as the image to be recognized, so as to facilitate subsequent icon recognition of the image to be recognized. For example, the icon can be a watermark, a mark, a tag, etc.

[0185] In order to audit the video to be detected, the embodiments of the present application identify the image corresponding to the video frame in the video to be detected. Since the video to be detected contains multiple video frames, in order to reduce the workload in the process of detecting the video, in this embodiment, a small number of video frames among the multiple video frames are selected as the images to be detected. Specifically, when receiving the video to be detected, the method of extracting the target image frame in the video to be detected can be: extracting all the video frames in the video to be detected to obtain a video frame set corresponding to the video to be detected; sorting the video frames in the video frame set according to the temporal sequence relationship between the video frames in the video to be detected to obtain a video frame sequence; obtaining the target image frame corresponding to the target temporal sequence. Among them, the target image frame corresponding to the target temporal sequence can be the target image frame corresponding to the first temporal sequence, the middle temporal sequence, or the last temporal sequence in the video frame sequence, or can also be the target image frames corresponding to the first, middle, and last three video frames respectively, which is not limited here.

[0186] 202. Cut the preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area.

[0187] Among them, the preset area can be a certain area or position in the image to be recognized, such as image areas such as the upper left, lower left, upper right, lower right, and middle in the image to be recognized.

[0188] After a video is published on a certain platform, the watermark mark (icon) it carries is usually located in the corner area of the video screen, such as the upper left or upper right corner area of the video screen, etc. In order to improve the detection efficiency of the watermark mark (icon), the watermark mark (icon) can be detected by performing detection on a specific area of the image to be recognized. Specifically, after obtaining the corresponding image to be recognized from the video to be detected in an embodiment of the present application, preprocessing is performed on a preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area.

[0189] It should be noted that the method for preprocessing the image to be recognized can be: cutting, capturing or intercepting the preset area of the image to be recognized to obtain the corresponding image sub-block. Specifically, the preprocessing process can be: obtaining the image size of the image to be recognized; based on the size of the image to be recognized, determining the size of the preset area of the image to be recognized according to the preprocessing rule; intercepting the preset area in the image to be recognized according to the size of the preset area to obtain the image sub-block. Among them, the preprocessing rule can include the position of the preset area of the image to be processed, the size ratio of the preset area in the image to be recognized, etc. For example, in the preprocessing rule, the preset area includes the upper left, lower left, upper right, lower right, middle and other areas of the image to be recognized, and the size ratio of this area in the image to be recognized is 1 / 5. Then, according to this size ratio, the upper left, lower left, upper right, lower right, middle areas in the image to be recognized are intercepted respectively to obtain the image sub-blocks corresponding to these areas. The above is only an example and is not limited here.

[0190] Through the above method, the image to be recognized is preprocessed according to the preprocessing rule to improve the accuracy of the image sub-block corresponding to the preset area, ensuring that the target icon (watermark mark) to be detected is located within the image sub-block as much as possible. At the same time, the size of the image sub-block is reduced as much as possible to reduce the image size during subsequent icon recognition of the image sub-block and improve the efficiency during subsequent icon recognition of the image sub-block.

[0191] 203. Input the image sub-block into the trained detection model to obtain the confidence that the image sub-block contains the target icon.

[0192] Among them, the trained detection model is jointly trained by sample image sub-blocks in a preset area of the sample image and the sample confidence of the sample image sub-blocks containing sample icons. Specifically, the process of obtaining the trained detection model by training with a preset model can be as follows: Obtain a sample image, where the preset area in the sample image carries a sample icon; Cut the sample image to obtain sample image sub-blocks carrying sample icons, and obtain the sample confidence of the sample image sub-blocks containing sample icons; Input the sample image sub-blocks into the preset model to obtain the predicted confidence of the sample image sub-blocks containing sample icons; Obtain the confidence difference between the sample confidence and the predicted confidence; Iteratively train the network parameters of the preset model according to the confidence difference until the confidence difference converges to obtain the trained detection model.

[0193] It should be noted that the sample image can be an artificially synthesized image. Since it is rather cumbersome to obtain sample data of the watermark video, in the embodiments of this application, a data synthesis method is adopted to fit real data, that is, a sample image containing a sample icon. Among them, during data synthesis, the randomness of the size and position of the sample icon (watermark mark) can be ensured. For example, an icon can be set in any preset area of the sample image, such as in any one or multiple image areas in the upper left, lower left, upper right, lower right, or middle of the sample image, and the icon size is random, but the size of the icon is smaller than the preset area, which is not limited here.

[0194] For example, referring to Figure 5 , when using the data synthesis method to synthesize a sample image, the corresponding sample icon can be obtained from existing sharing platforms or enterprises' sample icons or watermark marks, and then synthesized with a preselected image to ensure the randomness of the size and position of the sample icon (watermark mark), and perform random cropping or occlusion on the sample icon (watermark mark) and other processing to obtain the sample image. As Figure 5 shown, each sample image contains one sample icon, the positions of the sample icons in each sample image are different, and the sample icons are different; for example, the sample icon "A" in the two "scenery pictures" at the top is located in the upper left area and the upper right area of the sample image respectively; in the two video frame pictures corresponding to the game pictures in the middle, the sample icon "B" is located in the middle area of the sample image; in the sample images corresponding to the two "award ceremony" video frames, the icon "C" is located in the lower left area and the lower right area of the sample image respectively. It should be noted that Figure 5 the positions of the icons shown in the sample images are only examples, and in the actual training process, the positions of the sample images in the synthesized sample images can be determined according to the actual situation.

[0195] In addition, data augmentation processing such as blurring, occlusion, brightness, color change, adding noise, etc. can be performed on the preselected image before sample image synthesis or the synthesized sample image to further obtain the corresponding sample image for model training.

[0196] Among them, the detection model can be a model including a feature extraction layer (Darknet-53), a convolutional network layer (Convolutional Set), a classification layer (sigmoid), etc. When the detection model receives an image sub-block, the feature extraction layer (Darknet-53) in the detection model extracts features from the image sub-block to obtain image sub-features corresponding to the image sub-block; the convolutional network layer (Convolutional Set) in the detection model performs convolutional processing on the image sub-features to obtain a prediction feature map; the classification layer (Sigmoid) obtains the dimension of the prediction feature map and decodes the dimension of the prediction feature map to obtain the confidence of the image sub-block containing the target icon.

[0197] Specifically, the convolutional network layer (Darknet-53) includes a first convolutional network layer, a second convolutional network layer, and a third convolutional network layer.

[0198] Among them, the first convolutional network layer includes a 3*3 convolutional layer and a 1*1 convolutional layer. The 3*3 convolutional layer and the 1*1 convolutional layer in the first convolutional network layer perform convolutional processing on the image sub-features respectively to obtain a first prediction feature map.

[0199] Among them, the second convolutional network layer includes a 1*1 convolutional layer, an upsampling layer, a fusion layer, a 3*3 convolutional layer, and a 1*1 convolutional layer. The 1*1 convolutional layer in the second convolutional network layer performs convolutional processing on the image sub-features to obtain a first convolutional feature result; the upsampling layer upsamples the first convolutional feature result to obtain an upsampled first feature result; the fusion layer fuses the upsampled first feature result with the 26*26 feature result output by the feature extraction layer (Darknet-53) to obtain a first fusion feature; the first fusion feature is passed through the 3*3 convolutional layer and the 1*1 convolutional layer to perform convolutional processing on the image sub-features respectively to obtain a second prediction feature layer.

[0200] Among them, the third convolutional network layer includes a 1*1 convolutional layer, an upsampling layer, a fusion layer, a 3*3 convolutional layer, and a 1*1 convolutional layer. The first fusion feature is convolved through the 1*1 convolutional layer to obtain a second convolutional feature result; the second convolutional feature result is upsampled through the upsampling layer to obtain an upsampled second feature result; the upsampled second feature result is fused with the 52*52 feature result output by the feature extraction layer (Darknet-53) through the fusion layer to obtain a second fusion feature; the second fusion feature is successively convolved through the 3*3 convolutional layer and the 1*1 convolutional layer to perform convolutional processing on the image sub-features, respectively, to obtain a third prediction feature layer.

[0201] Furthermore, the first prediction feature map, the second prediction feature map, and the third prediction feature map are determined as the prediction feature map.

[0202] Specifically, the process of the classification layer (Sigmoid) generating the confidence value is as follows:

[0203] Among them, the dimension of the prediction feature map is obtained, and the dimension of the prediction feature map is N*N*[3*(4 + 1 + C)], where N*N is the size of the output feature map; there are a total of 3 Anchor boxes, and each box has 4-dimensional prediction box values, namely Tx, Ty, Tw, Th; and it includes 1-dimensional prediction box confidence and C-dimensional object category numbers. Among them, the small-size feature map is used to detect large-size objects, and the large-size feature map is used to detect small-size objects, improving the detection performance of the model for different-size targets.

[0204] Among them, the dimension of the prediction feature map is decoded to obtain the confidence of the image sub-block containing the target icon, and the process can be as follows:

[0205] The prediction feature map is decoded through the Sigmoid activation function to obtain the confidence. The value of the confidence after decoding is between (0, 1), such as 0.1, 0.3, 0.5, 0.8, 0.95, etc.

[0206] In the embodiment of the present application, this confidence can be the credible value corresponding to the icon event contained in a certain area of the image to be recognized, and is used to reflect the credibility or probability that a certain area or position in the image to be recognized contains the target icon.

[0207] See Figure 6, which reflects the processing strategy for the target icon (watermark) when the confidence level is in different confidence ranges. In the embodiments of the present application, for the preset confidence range, two thresholds are set as thresh1 and thresh2 respectively. Among them, when the confidence level output by the detection model is in [thresh1, thresh2], optical character recognition (OCR) needs to be performed on the image sub-block or the image to be recognized; when the confidence level output by the detection model is in [thresh2, 1.0], it is determined that the video to be detected contains the target icon (watermark); when the confidence level output by the detection model is in [0, thresh1], it is determined that the video to be detected does not contain the target icon (watermark). Specifically, when the confidence level output by the detection model is within the preset confidence range, steps 204-205 are executed; when the confidence level output by the detection model is greater than the preset confidence range, steps 206-208 are executed; when the confidence level output by the detection model is less than the preset confidence range, steps 209-210 are executed.

[0208] 204. When it is detected that the confidence level is within the preset confidence range, perform optical character recognition processing on the image sub-block within the preset confidence range to obtain the optical character recognition result.

[0209] Among them, the preset confidence range can be the preset value range corresponding to the confidence level, which is used to distinguish the situation of the image to be recognized containing the icon (watermark), and limit the image sub-block corresponding to the confidence level within this value range to perform optical character recognition processing.

[0210] Among them, optical character recognition (OCR) can be a process of determining its shape by detecting dark and bright patterns when recognizing corresponding characters, and then translating the shape into computer text using character recognition methods; that is, using an optical method to convert the icon in the image sub-block or the image into a corresponding image file, and converting the text in the image into a text format through recognition software for further editing and processing or other processing by a word processing software.

[0211] In order to improve the accuracy of icon (watermark) detection. For example, assuming that the confidence value range is [0, 1], the embodiment of the present application sets the preset confidence range to [0.5, 0.9], and the image sub-blocks input to the detection model include image sub-blocks corresponding to the upper left, lower left, upper right, lower right and middle image areas of the image to be identified, wherein the confidence that the image sub-block in the upper left image area of the image to be identified contains the target icon is the highest, such as the confidence of the upper left image area is 0.8 (the confidence of other image sub-blocks may be 0 or less than 0.5), then since the confidence that the image sub-block corresponding to the upper left image area contains the target icon is in the preset confidence range [0.5, 0.9], optical character recognition is performed on the image sub-block corresponding to the upper left area to convert the character information contained in the icon in the image sub-block into text information to obtain an optical character recognition result.

[0212] It should be noted that the optical character recognition result contains word information or text, and subsequently the image sub-block is determined to contain the target icon based on the optical character recognition result, thereby determining whether the video to be detected contains the target icon (watermark mark).

[0213] 205. Determine a target recognition result of the video to be detected according to a result of the optical character recognition processing.

[0214] The target recognition result is the result of icon recognition on the video to be detected. When it is detected that the image sub-block contains the target icon, the target recognition result may include: the mark position information and identification of the target icon in the image or video frame to be recognized.

[0215] In some embodiments, a target recognition result of a video to be detected is determined based on a result of an optical character recognition process, and the method may be: identifying the optical character recognition result based on part-of-speech information to obtain a part-of-speech result; when it is detected that the part-of-speech result contains target word information in a preset word information list, determining the target word information as a target icon, and obtaining an identifier of the target icon; obtaining mark position information corresponding to the target icon in the image sub-block; and determining the identifier and mark position information of the target icon as the target recognition result.

[0216] Exemplarily, the preset word information list includes "A", "A icon", "A game", "A video", "B", "B icon", "B video", "B headline", etc.; assuming that when the word information of "A video" is included in the word nature result, the word nature result is matched with the word features in the preset word information list. When the target word information in the preset word information list matches the word information in the word nature result, such as "A video" in the preset word information list matches "A video" in the word nature result, then "A video" is determined as the target icon, and the identifier corresponding to the target icon obtained can be "A video", "A", etc. Further, by identifying the position information of the image sub-block corresponding to the confidence level in the image to be recognized, this position information is determined as the marking position information of the target icon.

[0217] In addition, since the last frame video frame in the video to be detected may contain relatively more word information associated with the target icon. For example, it shows icons (watermarks) of other platforms, user IDs, promotional slogans, etc. Exemplarily, the preset word information list can also include "Come to A (application), follow hot current events", "Come to A (application), follow the development of events", "Use the latest version of A (application) to scan the code and add me as a friend", etc.; in the embodiment of the present application, when the word nature result of the image sub-block with the confidence level within the preset confidence range does not contain the target word information in the preset word information list, optical character recognition is performed on the entire area of the image corresponding to the last frame video frame in the video to be detected, and the optical character recognition result corresponding to the image of the last frame video frame is obtained; the optical character recognition result is converted into a word information result containing one or more word information. When the word information result matches multiple target word information in the preset word information list, such as the target word information being "A (application)", "Come to A (application), follow hot current events", "Use the latest version of A (application) to scan the code and add me as a friend", that is, the number of target word information matched is 3, and the preset word information quantity threshold is 2. Since the number of target word information matched is greater than the preset word information quantity threshold, therefore, it is determined that the image to be recognized contains the target icon. Specifically, all the target word information matched can be determined as the target icon, or the word information with a higher frequency of occurrence among the multiple target word information matched can be determined as the target icon. Further, the marking position information of the image sub-block corresponding to the confidence level is obtained, that is, the marking position information of the target icon, and the identifier of the target icon is obtained, such as "A (application)", so as to generate the corresponding target recognition result according to the marking position information and identifier of the target icon.

[0218] It should be noted that when it is detected that the part-of-speech result does not contain the target word information in the preset word information list, full-area optical character recognition is performed on the image to be identified corresponding to the last frame of the video to be detected to obtain the optical character recognition result corresponding to the whole area; and the word information in the optical character recognition result corresponding to the whole area is identified to determine the number of corresponding word information; when the number of word information is greater than or equal to the preset word information number threshold, it is determined that the target icon is included in the image to be identified, and the target recognition result is determined based on the mark position information and identification of the target icon. In the above manner, it can be avoided that the video sharer removes the target icon (watermark mark) by cropping the video frames other than the last frame of the video. By identifying the image corresponding to the last frame, the accuracy of identifying the icon (watermark mark) in the video can be improved.

[0219] 206. When it is detected that the confidence level is greater than a preset confidence level range, determine that the image sub-block corresponding to the confidence level contains the target icon.

[0220] The target icon can be a platform logo or mark, indicating a mark assigned to the video to be tested after it was successfully published on the corresponding platform. For example, when Video 1 was previously published on Sharing Platform A, Sharing Platform A would attach an A icon (watermark mark) to a certain area of the video 1 screen. In this case, the icon (watermark) carried by Video 1 is Icon A.

[0221] 207. Obtain the mark position information of the image sub-block corresponding to the confidence level, and obtain the identifier of the target icon.

[0222] 208. Determine the target icon's identifier and mark position information as the target recognition result of the video to be detected.

[0223] By executing steps 206 - 208 , when the confidence level detected is greater than a preset confidence level range, an object recognition result of the video to be detected is generated.

[0224] For example, when the confidence level output by the detection model is greater than the preset confidence range, due to the relatively high confidence level, it indicates that the corresponding image sub-block contains the target icon (watermark), so it can be directly determined that the image sub-block corresponding to this confidence level contains the target icon. For example, when Video 1 was previously published on Sharing Platform A, Sharing Platform A will attach an A icon (watermark) to a certain area in the picture of this Video 1; when the user downloads this Video 1 from Sharing Platform A and publishes it to Sharing Platform B to move the Video 1 that has been published on Sharing Platform A to Sharing Platform B; since this Video 1 contains an A icon, when Sharing Platform B detects Video 1, it includes detecting whether Video 1 contains the target icon (watermark), specifically through the trained model for detection, and the confidence level that the image sub-block corresponding to Video 1 contains the A icon is output as 0.98, while the preset confidence range is [0.5, 0.9], and the embodiments of the present application can determine that the corresponding image sub-block contains the target icon.

[0225] Furthermore, by obtaining the marking position information and identification of the target icon, the target recognition result of the video to be detected is generated.

[0226] 209. When it is detected that the confidence level is less than the preset confidence range, it is determined that the image to be recognized does not carry the target icon.

[0227] It should be noted that when the confidence level output by the detection model is less than the preset confidence range, due to the relatively low confidence level, it indicates that the corresponding image sub-block does not contain the target icon (watermark), so it can be directly determined that the image sub-block corresponding to this confidence level does not contain the target icon, that is, the video to be detected does not carry the target icon.

[0228] 210. Determine that the video to be detected does not carry the target icon as the target recognition result.

[0229] For example, when Video 2 was not previously published on any other sharing platform, no area in the picture of this Video 2 will contain the target icon (watermark); when the user publishes this Video 2 to Sharing Platform B, since this Video 2 does not contain any target icon, when Sharing Platform B detects Video 2, specifically through the trained model to detect the icons in the image sub-block corresponding to Video 2, no target icon (watermark) is detected in the image sub-block, and a relatively low confidence level is output, such as 0.2. Since this confidence level is less than the preset confidence range of [0.5, 0.9], it indicates that the credibility that the image sub-block corresponding to this confidence level contains the target icon is relatively low. Therefore, the embodiments of the present application can determine that the corresponding image sub-block does not contain the target icon, that is, it is determined that Video 1 does not carry the target icon.

[0230] By executing steps 201 - 210, the process as in Figure 4 is implemented. Specifically, Figure 4The specific process of the icon recognition method shown is as follows:

[0231] 301. Extract the first, middle, and last three video frames from the video to be detected as the images to be recognized.

[0232] 302. Cut the preset area of the image to be recognized to obtain the corresponding image sub-blocks.

[0233] 303. Input the image sub-blocks into the detection model, and obtain the confidence level of the image sub-blocks containing the target icon through the detection model.

[0234] 304. Determine whether the confidence level is within the preset confidence range; if the confidence level is within the preset confidence range, execute process 305, if the confidence level is not within the preset confidence range, execute process 310.

[0235] 305. Perform optical character recognition on the image sub-blocks within the preset confidence range to obtain the optical character recognition results corresponding to the image sub-blocks.

[0236] 306. Detect whether the optical character recognition results corresponding to the image sub-blocks contain the word information corresponding to the target icon; if the corresponding word information is detected, it is determined that the video contains the target icon (watermark), if the corresponding word information is not detected, execute process 307.

[0237] 307. Perform full-image optical character recognition on the image to be recognized corresponding to the last frame to obtain the optical character recognition results corresponding to the full image of the image to be recognized.

[0238] 308. Detect whether the optical character recognition results corresponding to the full image of the image to be recognized contain the word information corresponding to the target icon; among them, if the word information corresponding to the target icon is detected, execute process 309, if the word information corresponding to the target icon is not detected, it is determined that the video does not contain the target icon (watermark).

[0239] 309. Determine whether the video contains the target icon (watermark) according to the number of detected word information; among them, if the number of detected corresponding word information is greater than or equal to the preset word information number threshold, it is determined that the video contains the target icon (watermark), if the number of detected corresponding word information is less than the preset word information number threshold, it is determined that the video does not contain the target icon (watermark).

[0240] It should be noted that for the specific implementation of processes 301 - 309, reference can be made to the previous embodiments, which will not be elaborated here.

[0241] As can be seen from the above, the embodiments of the present application can obtain the image to be recognized from the video to be detected; cut the preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area; input the image sub-block into the trained detection model to obtain the confidence of the image sub-block containing the target icon; when it is detected that the confidence is within the preset confidence range, perform optical character recognition processing on the image sub-block within the preset confidence range to obtain the optical character recognition result; determine the target recognition result of the video to be detected according to the result of the optical character recognition processing. It can be seen that when detecting the video to be detected, the embodiments of the present application obtain the image to be recognized corresponding to the video to be detected, preprocess it to obtain image sub-blocks, and perform target icon recognition on the image sub-blocks through the trained detection model, avoiding full-image recognition of the image to be recognized, reducing the recognition amount of the area of the image to be recognized, and improving the icon recognition efficiency; after obtaining the confidence of the target icon in the image sub-block, determine whether to perform optical character recognition processing on the image to be recognized according to the confidence, improving the accuracy when recognizing the icon in the video.

[0242] To better implement the above method, the embodiments of the present application further provide an icon recognition device, which can be integrated in a computer device, such as a server or a terminal, etc. The terminal can include a tablet computer, a notebook computer, and / or a personal computer, etc.

[0243] For example, as Figure 7 shown, the icon recognition device may include an acquisition unit 401, a preprocessing unit 402, an input unit 403, a recognition unit 404, and a determination unit 405.

[0244] The acquisition unit 401 is configured to obtain the image to be recognized from the video to be detected;

[0245] The preprocessing unit 402 is configured to cut the preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area;

[0246] The input unit 403 is configured to input the image sub-block into the trained detection model to obtain the confidence of the image sub-block containing the target icon;

[0247] The recognition unit 404 is configured to perform optical character recognition processing on the image sub-block within the preset confidence range to obtain the optical character recognition result when it is detected that the confidence is within the preset confidence range;

[0248] The determination unit 405 is configured to determine the target recognition result of the video to be detected according to the result of the optical character recognition processing.

[0249] In some embodiments, the determination unit 405 is further specifically used to identify the optical character recognition result based on the part-of-speech information to obtain a part-of-speech result; when it is detected that the part-of-speech result contains the target word information in the preset word information list, the target word information is determined to be a target icon, and the identifier of the target icon is obtained; the mark position information corresponding to the target icon in the image sub-block is obtained; and the identifier and mark position information of the target icon are determined as the target recognition result.

[0250] In some embodiments, the determining unit 405 is further configured to:

[0251] When it is detected that the part-of-speech result does not contain the target word information in the preset word information list, performing optical character recognition processing on the entire area of the image to be recognized to obtain an optical character recognition result corresponding to the entire area, and recognizing the optical character recognition result corresponding to the entire area according to the part-of-speech information to obtain a word information result;

[0252] Match the word information result with the target word information in the preset word information list;

[0253] When the number of matched target word information is greater than or equal to a preset word information number threshold, the matched target word information is determined as the target icon in the image to be recognized;

[0254] Obtaining the identifier of the target icon and the marking position information of the target icon in the image to be identified;

[0255] The identification and marking position information of the target icon are determined as the target recognition result.

[0256] In some embodiments, the input unit 403 is further specifically used to input the image sub-block into the trained detection model; perform feature extraction on the image sub-block through the feature extraction layer in the detection model to obtain image sub-features; perform convolution processing on the image sub-features through the convolution layer in the detection model to obtain a predicted feature map; decode the dimension of the predicted feature map through the classification layer in the detection model to obtain the confidence that the image sub-block contains the target icon.

[0257] In some embodiments, the determination unit 405 is further specifically used to determine that the image sub-block corresponding to the confidence level contains a target icon when it is detected that the confidence level is greater than a preset confidence level range; obtain the marking position information of the image sub-block corresponding to the confidence level, and obtain the identifier of the target icon; and determine the identifier and marking position information of the target icon as the target recognition result of the video to be detected.

[0258] In some embodiments, the determination unit 405 is further configured to: when it is detected that the confidence level is less than a preset confidence range, determine that the image to be identified does not carry the target icon; and determine that the image to be identified does not carry the target icon as the target recognition result of the video to be detected.

[0259] In some embodiments, the icon recognition device further includes a training unit, and specifically, the training unit is configured to:

[0260] Obtain a sample image, wherein a preset area in the sample image carries a sample icon;

[0261] Cut the sample image to obtain a sample image sub-block carrying the sample icon, and obtain the sample confidence of the sample icon included in the sample image sub-block;

[0262] Input the sample image sub-block into a preset model to obtain the predicted confidence of the sample icon included in the sample image sub-block;

[0263] Obtain the confidence difference between the sample confidence and the predicted confidence;

[0264] Iteratively train the network parameters of the preset model according to the confidence difference until the confidence difference converges, and obtain a trained detection model.

[0265] In some embodiments, the obtaining unit 401 is further configured to receive a video to be detected; extract a target image frame from the video to be detected; and determine the target image frame as an image to be recognized.

[0266] As can be seen from the above, in the embodiment of the present application, the obtaining unit 401 can obtain an image to be recognized from the video to be detected; the preprocessing unit 402 cuts a preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area; the input unit 403 inputs the image sub-block into the trained detection model to obtain the confidence of the image sub-block including the target icon; the recognition unit 404 is configured to perform optical character recognition processing on the image sub-block within the preset confidence range when detecting that the confidence is within the preset confidence range to obtain an optical character recognition result; and the determination unit 405 determines the target recognition result of the video to be detected according to the result of the optical character recognition processing. Therefore, when detecting the video to be detected, the embodiment of the present application obtains an image to be recognized corresponding to the video to be detected, preprocesses it to obtain an image sub-block, and performs target icon recognition on the image sub-block through the trained detection model, avoiding full-image recognition of the image to be recognized, reducing the recognition amount of the area of the image to be recognized, and improving the icon recognition efficiency; after obtaining the confidence of the target icon included in the image sub-block, it determines whether to perform optical character recognition processing on the image to be recognized according to the confidence, improving the accuracy when recognizing the icon in the video.

[0267] The embodiment of the present application further provides a computer device, as Figure 8 shown, which shows a schematic structural diagram of the computer device involved in the embodiment of the present application. Specifically:

[0268] The computer device may include components such as a processor 501 with one or more processing cores, a memory 502 of one or more computer-readable storage media, a power supply 503, and an input unit 504. Those skilled in the art can understand that Figure 8 the computer device structure shown in

[0269]

[0270]

[0271]

[0272] does not limit the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements. Among them:

[0270] The processor 501 is the control center of the computer device, connecting various parts of the entire computer device through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 502, and by calling data stored in the memory 502, it executes various functions of the computer device and processes data, thereby performing overall detection of the computer device. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above modem processor may not be integrated into the processor 501. The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, image playback function, etc.), etc.; the data storage area can store data created according to the use of the computer device. In addition, the memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502. The computer device also includes a power supply 503 that powers each component. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 503 may also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.The computer device may further include an input unit 504, which may be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0273] Although not shown, the computer device may further include a display unit, etc., which will not be elaborated here. Specifically, in the embodiments of the present application, the processor 501 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 502 according to the following instructions, and the processor 501 will run the application programs stored in the memory 502 to implement various functions as follows:

[0274] Obtain a to-be-recognized image from the to-be-detected video; cut a preset area of the to-be-recognized image to obtain an image sub-block corresponding to the preset area; input the image sub-block into the trained detection model to obtain the confidence level that the image sub-block contains a target icon; when it is detected that the confidence level is within a preset confidence range, perform optical character recognition processing on the image sub-block within the preset confidence range to obtain an optical character recognition result; determine the target recognition result of the to-be-detected video according to the result of the optical character recognition processing.

[0275] For the specific implementation of each of the above operations, reference may be made to the previous embodiments and will not be elaborated here.

[0276] As can be seen from the above, the embodiments of the present application can obtain a to-be-recognized image from the to-be-detected video; cut a preset area of the to-be-recognized image to obtain an image sub-block corresponding to the preset area; input the image sub-block into the trained detection model to obtain the confidence level that the image sub-block contains a target icon; when it is detected that the confidence level is within a preset confidence range, perform optical character recognition processing on the image sub-block within the preset confidence range to obtain an optical character recognition result; determine the target recognition result of the to-be-detected video according to the result of the optical character recognition processing. Thus, when detecting the to-be-detected video, the embodiments of the present application obtain the to-be-recognized image corresponding to the to-be-detected video, cut it to obtain an image sub-block, and perform target icon recognition on the image sub-block through the trained detection model, avoiding full-image recognition of the to-be-recognized image, reducing the recognition amount of the to-be-recognized image area, and improving the icon recognition efficiency; after obtaining the confidence level that the image sub-block contains a target icon, determine whether to perform optical character recognition processing on the to-be-recognized image according to the confidence level, improving the accuracy when recognizing icons in the video.

[0277] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0278] To this end, an embodiment of the present application provides a computer-readable storage medium, which stores multiple instructions that can be loaded by a processor to execute the steps in any of the icon recognition methods provided by the embodiments of the present application. For example, the instructions can execute the following steps:

[0279] Obtain a to-be-recognized image from a to-be-detected video; clip a preset area of the to-be-recognized image to obtain an image sub-block corresponding to the preset area; input the image sub-block into a trained detection model to obtain the confidence level that the image sub-block contains a target icon; when it is detected that the confidence level is within a preset confidence range, perform optical character recognition processing on the image sub-block within the preset confidence range to obtain an optical character recognition result; determine the target recognition result of the to-be-detected video according to the result of the optical character recognition processing.

[0280] For the specific implementation of each of the above operations, reference can be made to the previous embodiments, which will not be elaborated here.

[0281] Among them, the computer-readable storage medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), a magnetic disk or an optical disc, etc.

[0282] Since the instructions stored in the computer-readable storage medium can execute the steps in any of the icon recognition methods provided by the embodiments of the present application, the beneficial effects that can be achieved by any of the icon recognition methods provided by the embodiments of the present application can be realized. For details, reference can be made to the previous embodiments, which will not be elaborated here.

[0283] The above has introduced in detail an icon recognition method, device, and computer-readable storage medium provided by the embodiments of the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, based on the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. An icon recognition method, characterized in that: include: Obtain the image to be identified from the video to be detected; Cutting a preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area; Inputting the image sub-block into the trained detection model to obtain the confidence that the image sub-block contains the target icon; When it is detected that the confidence level is within a preset confidence level range, performing optical character recognition processing on the image sub-block within the preset confidence level range to obtain an optical character recognition result; Determining a target recognition result of the video to be detected based on the result of the optical character recognition processing includes: recognizing the optical character recognition result based on the part-of-speech information to obtain a part-of-speech result; when it is detected that the part-of-speech result contains target word information in a preset word information list, determining that the target word information is a target icon and obtaining an identifier of the target icon; obtaining mark position information corresponding to the target icon in the image sub-block; and determining the identifier and mark position information of the target icon as the target recognition result.

2. The method according to claim 1, characterized in that Also includes: When it is detected that the part-of-speech result does not include the target word information in the preset word information list, performing optical character recognition processing on the image to be recognized to obtain an optical character recognition result corresponding to the image to be recognized; Recognize the optical character recognition result corresponding to the image to be recognized according to the part-of-speech information to obtain the word information result corresponding to the image to be recognized; Matching the word information result with the target word information in the preset word information list; When the number of matched target word information is greater than or equal to a preset word information number threshold, determining the matched target word information as the target icon in the image to be recognized; Obtaining the identifier of the target icon and obtaining the marking position information of the target icon in the image to be identified; The identification and mark position information of the target icon are determined as the target recognition result.

3. The method according to claim 1, characterized in that Inputting the image sub-block into the trained detection model to obtain the confidence that the image sub-block contains the target icon includes: Inputting the image sub-block into the trained detection model; Performing feature extraction on the image sub-block using the detection model to obtain image sub-features; Performing convolution processing on the image sub-features through the detection model to obtain a predicted feature map; The dimensions of the predicted feature map are decoded by the classification layer in the detection model to obtain the confidence that the image sub-block contains the target icon.

4. The method according to claim 1, wherein Before inputting the image sub-block into the trained detection model, the method further includes: Acquire a sample image, wherein a preset area in the sample image carries a sample icon; Cutting the sample image to obtain a sample image sub-block carrying the sample icon, and obtaining a sample confidence of the sample icon in the sample image sub-block; Inputting the sample image sub-block into a preset model to obtain a prediction confidence that the sample image sub-block contains the sample icon; Obtaining a confidence difference between the sample confidence and the prediction confidence; The network parameters of the preset model are iteratively trained according to the confidence difference until the confidence difference converges to obtain a trained detection model.

5. The method according to claim 1, wherein Also includes: When it is detected that the confidence level is greater than the preset confidence level range, determining that the image sub-block corresponding to the confidence level includes a target icon; Obtaining the mark position information of the image sub-block corresponding to the confidence level, and obtaining the identifier of the target icon; The identification and mark position information of the target icon are determined as the target recognition result of the video to be detected.

6. The method according to claim 1, characterized in that Also includes: When it is detected that the confidence level is less than the preset confidence level range, determining that the image to be identified does not carry the target icon; The video that does not carry the target icon is determined as the target recognition result of the video to be detected.

7. The method according to claim 1, characterized in that The step of obtaining an image to be identified from a video to be detected includes: Receive the video to be detected; Extracting a target image frame from the video to be detected; The target image frame is determined as an image to be recognized.

8. An icon recognition device, characterized in that: include: An acquisition unit, configured to acquire an image to be identified from a video to be detected; a pre-processing unit, configured to cut a preset area of the image to be recognized to obtain an image sub-block corresponding to the preset area; An input unit, configured to input the image sub-block into a trained detection model to obtain a confidence level that the image sub-block contains a target icon; a recognition unit configured to, when detecting that the confidence level is within a preset confidence level range, perform optical character recognition processing on the image sub-block within the preset confidence level range to obtain an optical character recognition result; A determination unit is used to determine a target recognition result of the video to be detected based on the result of the optical character recognition processing, including: recognizing the optical character recognition result based on the part-of-speech information to obtain a part-of-speech result; when it is detected that the part-of-speech result contains target word information in a preset word information list, determining that the target word information is a target icon and obtaining an identifier of the target icon; obtaining mark position information corresponding to the target icon in the image sub-block; and determining the identifier and mark position information of the target icon as the target recognition result.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium is computer-readable and stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor to execute the steps of the icon recognition method according to any one of claims 1 to 7.

10. A computer device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to execute the steps of the icon recognition method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the icon recognition method according to any one of claims 1 to 7.

Images