Image processing method, device, image processing equipment and storage medium

The recognition of incomplete subtitle images through the object detection model is solved, and the problem of low recognition accuracy and recall of incomplete subtitle images in the prior art is achieved, achieving higher recognition accuracy and user experience.

CN113569613BActive Publication Date: 2024-11-26TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110195326.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-02-20
Publication Date
2024-11-26
Estimated Expiration
2041-02-20

AI Technical Summary

Technical Problem

The prior art is difficult to effectively identify and process incomplete subtitle images, resulting in a decline in image quality and affecting the user's viewing experience.

Method used

By calling the trained object detection model, the prediction category of the text box to which each text content in the image to be detected belongs, and whether the image is an incomplete subtitle image is determined based on the prediction category and confidence level, improving the recognition accuracy and recall rate.

Benefits of technology

It improves the recognition accuracy and recall rate of incomplete subtitles images, reduces the misidentification of incomplete subtitles images, and improves the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113569613B_ABST
    Figure CN113569613B_ABST
Patent Text Reader

Abstract

The present application discloses an image processing method, apparatus, image processing device and storage medium, wherein the method comprises: obtaining an image to be detected; calling a trained target detection model to determine the predicted category of a text box to which each text content in the image to be detected belongs, the predicted category being used to indicate the probability that the text content contained in the text box is incomplete; if there is at least one text box whose predicted category is a target category, obtaining the confidence of each target text box whose predicted category is the target category, the target category being used to indicate that the probability that the text content contained in the text box is incomplete is greater than a first preset threshold; if there is at least one target text box whose confidence is greater than a second preset threshold, outputting the predicted type of the image to be detected as a first type, the first type being used to indicate that the image to be detected is an incomplete subtitle image, which can improve the accuracy and recall rate of recognizing incomplete subtitle images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, an image processing device, an image processing equipment and a computer storage medium. Background Art

[0002] With the popularity and development of information flow and short videos, more and more platforms are available for users (including self-media and ordinary users) to create and publish images or videos. Images or videos uploaded and published by users may have incomplete subtitles, resulting in reduced image or video quality, which seriously affects the user's viewing experience. Therefore, how to identify incomplete subtitled images is an important research topic in image processing technology. Summary of the invention

[0003] The embodiments of the present invention provide an image processing method, an apparatus, an image processing device and a storage medium, which can determine the predicted category of each text box in the image to be detected through a target detection model, identify the image to be detected as an incomplete subtitle image based on the predicted category of the text box, and improve the accuracy and recall rate of identifying incomplete images.

[0004] On the one hand, an embodiment of the present invention provides an image processing method, the image processing method comprising:

[0005] Acquire the image to be detected;

[0006] Calling the trained object detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs, wherein the predicted category is used to indicate the probability that the text content contained in the text box is incomplete content;

[0007] If there is at least one text box whose predicted category is the target category, obtaining the confidence of each target text box whose predicted category is the target category, wherein the target category is used to indicate that the probability that the text content contained in the text box is incomplete content is greater than a first preset threshold;

[0008] If there is at least one target text box whose confidence is greater than a second preset threshold, the prediction type of the image to be detected is output as a first type, where the first type is used to indicate that the image to be detected is an incomplete subtitle image.

[0009] On the other hand, an embodiment of the present invention provides an image processing device, the image processing device comprising:

[0010] An acquisition unit, used for acquiring an image to be detected;

[0011] A determination unit, used to call a trained target detection model to determine a predicted category of a text box to which each text content in the image to be detected belongs, wherein the predicted category is used to indicate a probability that the text content contained in the text box is incomplete content;

[0012] The acquisition unit is further configured to acquire the confidence of each target text box whose predicted category is the target category if there is at least one text box whose predicted category is the target category, wherein the target category is used to indicate that the probability that the text content contained in the text box is incomplete content is greater than a first preset threshold;

[0013] The output unit is used to output the predicted type of the image to be detected as a first type if there is at least one target text box whose confidence is greater than a second preset threshold, wherein the first type is used to indicate that the image to be detected is an incomplete subtitle image.

[0014] In another aspect, an embodiment of the present invention provides an image processing device, the image processing device comprising an input interface and an output interface, and the image processing device further comprising:

[0015] a processor adapted to implement one or more instructions; and,

[0016] A computer storage medium storing one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the following steps:

[0017] Acquire the image to be detected;

[0018] Calling the trained object detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs, wherein the predicted category is used to indicate the probability that the text content contained in the text box is incomplete content;

[0019] If there is at least one text box whose predicted category is the target category, obtaining the confidence of each target text box whose predicted category is the target category, wherein the target category is used to indicate that the probability that the text content contained in the text box is incomplete content is greater than a first preset threshold;

[0020] If there is at least one target text box whose confidence is greater than a second preset threshold, the prediction type of the image to be detected is output as a first type, where the first type is used to indicate that the image to be detected is an incomplete subtitle image.

[0021] In another aspect, an embodiment of the present invention provides a computer storage medium, wherein the computer storage medium stores one or more instructions, wherein the one or more instructions are suitable for being loaded by a processor and executing the following steps:

[0022] Acquire the image to be detected;

[0023] Calling the trained object detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs, wherein the predicted category is used to indicate the probability that the text content contained in the text box is incomplete content;

[0024] If there is at least one text box whose predicted category is the target category, obtaining the confidence of each target text box whose predicted category is the target category, wherein the target category is used to indicate that the probability that the text content contained in the text box is incomplete content is greater than a first preset threshold;

[0025] If there is at least one target text box whose confidence is greater than a second preset threshold, the prediction type of the image to be detected is output as a first type, where the first type is used to indicate that the image to be detected is an incomplete subtitle image.

[0026] When identifying whether the image to be detected is an incomplete subtitle image, the embodiment of the present invention can first call the trained target detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs, and then determine whether there is a target text box with a predicted category of the target category in the image to be detected. If there is a target text box with a predicted category of the target category, the image to be detected is determined to be an incomplete subtitle image according to the confidence of the target text box and the second preset threshold. Considering that the image to be detected may contain text boxes with complete subtitle content and text boxes with incomplete subtitle content at the same time, compared with the incomplete subtitle image recognition method based on image classification, calling the trained target detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs can distinguish each text box in the image to be detected, can pay attention to the local area information in the image to be detected, and improve the accuracy and recall rate of incomplete subtitle image recognition. In addition, the incomplete subtitle image can be identified based on the comparison result of the confidence of the target text box containing the incomplete subtitle content and the second preset threshold, which can further improve the accuracy of incomplete subtitle image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 is a schematic diagram of an incomplete subtitle provided by an embodiment of the present invention;

[0029] Figure 2 is a schematic diagram of an incomplete subtitle provided by an embodiment of the present invention;

[0030] Figure 3a is a schematic diagram of a scenario of the image processing method provided by an embodiment of the present invention;

[0031] Figure 3b is another scenario schematic diagram of the image processing method provided by an embodiment of the present invention;

[0032] Figure 4 is a flowchart of an image processing method provided by an embodiment of the present invention;

[0033] Figure 5 is a flowchart of another image processing method provided by an embodiment of the present invention;

[0034] Figure 6 is a structural schematic diagram of an image processing device provided by an embodiment of the present invention;

[0035] Figure 7 It is a structural schematic diagram of an image processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technology mainly includes several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. Among them, computer vision technology (Computer Vision, CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and image processing equipment to replace human eyes to identify, track, and measure targets, and further perform image processing so that the image processing equipment processes the image into an image that is more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, video processing, and optical character recognition (Optical Character Recognition, OCR) and other technologies.

[0038] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. In addition, artificial intelligence technology can also be applied in other fields. For example, computer vision technology in artificial intelligence technology can be used to realize the recognition of incomplete subtitle images.

[0039] Among them, an incomplete subtitle image refers to an image with incomplete text content. In one embodiment, when the video or image posted by the user is not directly acquired by the image acquisition device, but is obtained by the user editing the image material, the image or video posted by the user may have incomplete subtitles. For example, due to factors such as different size ratios of different platforms, the user may need to crop the image material, which may cause the text box to which the subtitles belong in the image material to be cropped, thereby making the subtitle content in the cropped text box incomplete, and then an image containing incomplete subtitle content can be obtained, that is, an incomplete subtitle image. Different text directions have different subtitle types, and the subtitles can be divided into horizontal subtitles (such as vertical subtitles) according to the text direction. Figure 1 ) and vertical subtitles (as shown Figure 2 The following examples illustrate an image containing horizontal subtitles and an image containing incomplete subtitles containing vertical subtitles.

[0040] See also Figure 1 , Figure 1The image 100 and the normal horizontal subtitle 101 in the image 100 are included. Optionally, during the editing process, the image 100 may be cropped horizontally to obtain the image 110. Then correspondingly, the normal horizontal subtitle 101 in the image 100 may also be cropped horizontally during the editing process to obtain an incomplete horizontal subtitle, that is, a horizontally cropped horizontal subtitle 111, and the image 110 containing the horizontally cropped horizontal subtitle 111 is an incomplete subtitle image. Optionally, during the editing process, the image 100 may be cropped vertically to obtain the image 120. Then correspondingly, the normal horizontal subtitle 101 in the image 100 may also be cropped vertically during the editing process to obtain an incomplete horizontal subtitle, that is, a vertically cropped horizontal subtitle 121, and the image 120 containing the vertically cropped horizontal subtitle 121 is an incomplete subtitle image. Optionally, during the editing process, the image 100 may be cropped horizontally and vertically at the same time to obtain the image 130. Accordingly, the normal horizontal subtitle 101 in the image 100 may also be cropped horizontally and vertically at the same time, obtaining an incomplete horizontal subtitle, namely, a horizontally cropped horizontal subtitle 131. Among them, for the horizontal subtitle, the impact of horizontal cropping is greater than that of vertical cropping, so for the horizontal subtitle that is cropped horizontally and vertically at the same time, the cropped horizontal subtitle can be called a horizontally cropped horizontal subtitle.

[0041] See also Figure 2 , Figure 2 The image 200 and the normal vertical subtitle 201 in the image 200 are included. Optionally, during the editing process, the image 200 may be cropped horizontally to obtain the image 210. Then correspondingly, the normal vertical subtitle 201 in the image 200 may also be cropped horizontally during the editing process to obtain an incomplete vertical subtitle, that is, a horizontally cropped vertical subtitle 211, and the image 210 containing the horizontally cropped vertical subtitle 211 is an incomplete subtitle image. Optionally, during the editing process, the image 200 may be cropped vertically to obtain the image 220. Then correspondingly, the normal vertical subtitle 201 in the image 200 may also be cropped vertically during the editing process to obtain an incomplete vertical subtitle, that is, a vertically cropped vertical subtitle 221, and the image 220 containing the vertically cropped vertical subtitle 221 is an incomplete subtitle image. Optionally, during the editing process, the image 200 may be cropped horizontally and vertically at the same time to obtain the image 230. Accordingly, the normal vertical subtitle 201 in the image 200 may also be cropped horizontally and vertically at the same time, obtaining an incomplete vertical subtitle, namely a vertically cropped vertical subtitle 231. For the vertical subtitle, the impact of vertical cropping is greater than that of horizontal cropping, so for the vertical subtitle that is cropped horizontally and vertically at the same time, the cropped vertical subtitle can be called a vertically cropped vertical subtitle.

[0042] The incomplete subtitle images described in the above examples are either incomplete subtitle images including incomplete horizontal subtitles or incomplete vertical subtitles. It should be understood that the incomplete subtitle images described in the present invention may also include incomplete horizontal subtitles and incomplete vertical subtitles.

[0043] In one embodiment, the embodiment of the present invention proposes an image processing method applied to the field of image processing technology based on the machine learning algorithm mentioned above. In the image processing method, the trained target detection model can be called to determine the predicted category of the text box to which each text content in the image to be detected belongs. When the target text box containing incomplete text content in the image to be detected is determined according to the predicted category of the text box, it can be determined whether the image to be detected is an incomplete subtitle image according to the confidence of the target text box. Compared with the recognition method of incomplete subtitle images based on image classification, local features in the image to be detected can be effectively focused on, and the text content of each text box in the image to be detected can be distinguished by the target detection model, and the text box containing complete subtitle content and the text box containing incomplete subtitle content can be detected from the image to be detected at the same time, and the incomplete subtitle image can be identified according to the text box with incomplete subtitle content, which can improve the accuracy and recall rate of incomplete subtitle image recognition. Under the same recognition accuracy, the recall rate is improved by 21 percentage points.

[0044] In a specific implementation, the image processing method can be executed by an image processing device, and the image processing device mentioned here can refer to any device with data computing function, such as a terminal device or a server. The terminal device can include but is not limited to: smart phones, tablet computers, laptops, wearable devices, desktop computers, etc.; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, content delivery networks (CDN), middleware services, domain name services, security services, and big data and artificial intelligence platforms, etc.

[0045] Among them, the image processing method has a wide range of applicable scenarios and can be applied to various platforms that need to conduct content review or cover image selection. For example, news information platforms, browser platforms, short video platforms, etc. When the image processing method of the present invention is applied to the above-mentioned information flow platform, the content or cover can be effectively reviewed, and the incomplete subtitle images appearing in the platform are significantly reduced, which improves the user experience.

[0046] In one embodiment, the image processing method provided by the embodiment of the present invention can be applied to various platforms that require content review. Before a user uploads and publishes an image or video, the image processing device needs to use the image processing method of the present invention to process the image to be uploaded, the image to be published, the video to be uploaded, or the video to be published, and identify whether there is an incomplete subtitle image in the image to be uploaded, the image to be published, the video to be uploaded, or the video to be published. If there is an incomplete subtitle image, it will be intercepted or downgraded, for example, the image to be uploaded or the video to be uploaded will be intercepted; for example, the image to be published or the video to be published will be downgraded. If there is no incomplete subtitle image, it will be published or uploaded normally, for example, entering the recommendation pool, such as Figure 3a As shown. In one embodiment, the image processing method provided by the embodiment of the present invention can be applied to various platforms that require cover image selection. In information flow platforms and short video platforms, in order to attract users to read, the image processing device needs to process the candidate cover images in the images to be uploaded, the images to be published, the videos to be uploaded or the videos to be published. If there are incomplete subtitles in the candidate cover images, that is, the candidate cover images are incomplete subtitle images, then the candidate cover images cannot be used as the final cover images, the candidate cover images are removed, and the next step of the cover image selection process cannot be triggered. If there are no incomplete subtitle images in the candidate cover images, that is, the cover images are complete subtitle images, then the next step of the cover image selection process can be triggered, and the cover image processing module is triggered to execute the next step, such as Figure 3b shown.

[0047] Based on the above description, an embodiment of the present invention proposes an image processing method; the image processing method can be executed by the above-mentioned image processing device. Figure 4 As shown, the image processing method may include the following S401-S404:

[0048] S401, obtaining an image to be detected.

[0049] In an embodiment of the present invention, the resolution of the image to be detected can be any value; for example, the resolution of the image to be detected can be 600*800, the resolution of the image to be detected can also be 3000*4000, and so on. Specifically, the image to be detected can be an independent frame of image, or it can be any frame of image in an image sequence. Among them, the image sequence refers to a set of images consisting of multiple frames arranged in chronological order; for example, the image sequence can be a video, and the video can be a film and television video, a short video, and so on. Among them, short videos can also be called short videos, which are generally videos with a playback time of N minutes (for example, 4 minutes, 5 minutes, etc.) circulated on new media on the Internet.

[0050] S402: Calling the trained object detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs, where the predicted category is used to indicate the probability that the text content contained in the text box is incomplete content.

[0051] Among them, the predicted category of the text box is any one of a plurality of categories to be selected. The image processing device can determine a plurality of categories to be selected of the text box according to the text content in the text box. In one embodiment, it can be known from the foregoing that the subtitles may include the following 6 cases: normal horizontal subtitles, horizontally cut horizontal subtitles, vertically cut horizontal subtitles, normal vertical subtitles, horizontally cut vertical subtitles, and vertically cut vertical subtitles. Then the plurality of categories to be selected of the text box may include one or more of the following: a text box containing normal horizontal subtitles, a text box containing horizontally cut horizontal subtitles, a text box containing vertically cut horizontal subtitles, a text box containing normal vertical subtitles, a text box containing horizontally cut vertical subtitles, and a text box containing vertically cut vertical subtitles.

[0052] The prediction category is used to indicate the probability that the text content contained in the text box is incomplete content, that is, the prediction category is used to indicate the probability that the subtitle contained in the text box is incomplete subtitle.

[0053] Among them, the target detection model may include any one of the following: a target detection model constructed based on the EfficientDet algorithm, a target detection model constructed based on the YOLO algorithm, and a target detection model constructed based on the RCNN algorithm.

[0054] In one embodiment, for the target detection model constructed by the EfficientDet algorithm, the target detection model can be composed of a neural network EfficientNet, a bidirectional feature pyramid network (BiFPN) and a target detection prediction head network. Among them, EfficientNet is an image feature extraction network based on a convolutional neural network (CNN) constructed by network automatic search (NAS), which is used to extract a basic feature set of the image to be detected, and the basic feature set includes basic features of the image to be detected at multiple sizes. BiFPN is an image fusion network, which is used to fuse the basic features of multiple sizes in the basic feature set to obtain fused features, so that the fused features can be combined with the fused features at multiple sizes, thereby improving the accuracy of the target detection model. The target detection prediction head network is used to predict the predicted category of the text box to which each text content in the image to be detected belongs based on the fused features.

[0055] In one embodiment, for a target detection model constructed based on the YOLO algorithm, the target detection model first divides the image to be detected into fixed grids (for example, 7*7). If the center of a sample object falls on the corresponding grid, the grid is responsible for the regression of the object position; each grid predicts the object position and confidence information, and encodes this information into a vector, and outputs the result of the grid prediction. The prediction results of all grids are combined to obtain the predicted category of the text box to which each text content in the image to be detected belongs.

[0056] In one embodiment, for a target detection model constructed based on the RCNN algorithm, the target detection model first uses the Region Proposal algorithm to extract candidate regions in the image to be detected, and normalizes the extracted candidate regions to obtain the input value of CNN. Then, CNN is used to perform convolution and / or pooling operations on the input value to obtain a feature vector with a fixed dimension. Finally, the feature vector is classified using a feature training classifier to obtain the predicted category of the text box to which each text content in the image to be detected belongs.

[0057] S403: If there is at least one text box whose predicted category is the target category, obtain the confidence of each target text box whose predicted category is the target category, where the target category is used to indicate that the probability that the text content contained in the text box is incomplete content is greater than a first preset threshold.

[0058] Specifically, if there is at least one text box in the image to be detected whose predicted category is the target category, the confidence of each target text box whose predicted category is the target category is obtained. If there is no target text box in the image to be detected whose predicted category is the target category, the predicted type of the image to be detected is output as the second type, and the second type is used to indicate that the image to be detected is a complete subtitle image. Among them, the target text box may include one or more of the following: a text box containing horizontal subtitles cut horizontally, a text box containing vertically cut horizontal subtitles, a text box containing horizontally cut vertical subtitles, and a text box containing vertically cut vertical subtitles. The probability that the subtitle content contained in each target text box is an incomplete subtitle is greater than a first preset threshold.

[0059] For example. Assume that there are two text boxes in the image to be detected, namely text box A, text box B and text box C. The predicted category A of text box A, the predicted category B of text box B and the predicted category C of text box C are determined by the trained target detection model. When the probability that the predicted category A indicates that the text content contained in text box A is incomplete is less than or equal to the first preset threshold, the probability that the predicted category B indicates that the text content contained in text box B is incomplete is less than or equal to the first preset threshold, and the probability that the predicted category C indicates that the text content contained in text box C is incomplete is less than or equal to the first preset threshold, there is no target text box whose preset category is the target category in the image to be detected. Then the image processing device can output that the type of the image to be detected is the second type. When the probability that the predicted category A indicates that the text content contained in the text box A is incomplete is less than or equal to the first preset threshold, the probability that the predicted category B indicates that the text content contained in the text box B is incomplete is greater than the first preset threshold, and the probability that the predicted category C indicates that the text content contained in the text box C is incomplete is greater than the first preset threshold, there is a target text box whose preset category is the target category in the image to be detected. Then the image processing device can obtain the confidence of the text box B and the confidence of the text box C.

[0060] S404: If there is at least one target text box whose confidence is greater than a second preset threshold, output the prediction type of the image to be detected as a first type, where the first type is used to indicate that the image to be detected is an incomplete subtitle image.

[0061] Specifically, the image processing device can compare the target text box with a second preset threshold. If there is at least one target text box whose confidence is greater than the second preset threshold, the predicted type of the image to be detected is output as the first type; if the confidence of each target text box is less than or equal to the second preset threshold, the predicted type of the image to be detected is output as the second type.

[0062] Continuing with the above example, after obtaining the confidence of text box B and the confidence of text box C, if the confidence of text box B and the confidence of text box C are both less than or equal to the second preset threshold, the type of the image to be detected is output as the second type; if the confidence of text box B is greater than the second preset threshold or the confidence of text box C is greater than the second preset threshold, then the image processing device can output the predicted type of the image to be detected as the first type.

[0063] Among them, the second preset thresholds corresponding to the prediction categories of each target text box may be the same or different. In one embodiment, when the second preset thresholds corresponding to the prediction categories of each target text box are the same, the confidence of each target text box can be compared with the second preset threshold, and the prediction type of the image to be detected is output according to the comparison result. Continuing the above example, the confidence of text box B and the confidence of text box C are compared with the same second preset threshold, and the prediction type of the image to be detected is output according to the comparison result. In another embodiment, the second preset thresholds corresponding to the prediction categories of each target text box may be different. For example, when the target text box is a text box containing horizontal subtitles cut horizontally, the second preset threshold is the second preset threshold 1; when the target text box is a text box containing vertically cut horizontal subtitles, the second preset threshold is the second preset threshold 2; when the target text box is a text box containing horizontally cut vertical subtitles, the second preset threshold is the second preset threshold 3; when the target text box is a text box containing vertically cut vertical subtitles, the second preset threshold is the second preset threshold 4. Continuing with the above example, if text box B is a text box containing horizontal subtitles cut horizontally, and text box C is a text box containing horizontal subtitles cut vertically, then the image processing device compares the confidence of text box B with the second preset threshold 1, and compares the confidence of text box C with the second preset threshold 2, and outputs the predicted type of the image to be detected according to the comparison result. For another example, when the target text box is a text box containing horizontal subtitles cut horizontally or when the target text box is a text box containing horizontal subtitles cut vertically, the second preset threshold is the second preset threshold 1; when the target text box is a text box containing horizontal subtitles cut horizontally or when the target text box is a text box containing vertical subtitles cut vertically, the second preset threshold is the second preset threshold 2. And so on.

[0064] When identifying whether the image to be detected is an incomplete subtitle image, the embodiment of the present invention can first call the trained target detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs, and then determine whether there is a target text box with a predicted category of the target category in the image to be detected. If there is a target text box with a predicted category of the target category, the image to be detected is determined to be an incomplete subtitle image according to the confidence of the target text box and the second preset threshold. Considering that the image to be detected may contain text boxes with complete subtitle content and text boxes with incomplete subtitle content at the same time, compared with the incomplete subtitle image recognition method based on image classification, calling the trained target detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs can distinguish each text box in the image to be detected, can pay attention to the local area information in the image to be detected, and improve the accuracy and recall rate of incomplete subtitle image recognition. In addition, the incomplete subtitle image can be identified based on the comparison result of the confidence of the target text box containing the incomplete subtitle content and the second preset threshold, which can further improve the accuracy of incomplete subtitle image recognition.

[0065] See above Figure 4 It can be seen from the relevant description of the method embodiment shown that Figure 4 The image processing method shown can call the trained object detection model to realize the recognition of incomplete caption images. Therefore, before calling the trained object detection model, the object detection model needs to be trained. Based on this, see Figure 5 , Figure 5 A schematic flow chart of another image processing method is shown, and the image processing method may include S501-S506:

[0066] S501: Perform text recognition on the initial image using an optical character recognition algorithm to determine each candidate text box in the initial image and the text content contained in each candidate text box.

[0067] Among them, the optical character recognition (OCR) algorithm refers to the process in which an image processing device determines candidate text boxes and the text content contained in each candidate text box by detecting dark and light patterns, and then translates them into computer text using a character recognition method.

[0068] In one embodiment, the image processing device may include an OCR module, which uses the OCR recognition service provided by the data platform unit to perform text recognition on the initial image through an OCR algorithm to determine each candidate text box in the initial image and the text content contained in each candidate text box. The initial image may refer to an image material collected by an image acquisition device, or may be an image obtained through editing.

[0069] S502: Invoke an incoherent recognition model to process the text content contained in each candidate text box, and determine a reference text box with incoherent text content among all the candidate text boxes contained in the initial image.

[0070] Among them, since the OCR algorithm will detect and recognize all text content in the initial image, it is impossible to effectively distinguish whether the text content in the initial image is artificially added subtitle content or background text in the image captured by the image acquisition device, such as text content included in the captured image material. Moreover, since the background text of the image is usually located in the bottom layer of the initial image, the background text of the image may be covered by the layer above the bottom layer in the image, making the text content of the background text of the image incoherent. Therefore, the incoherent recognition model can be called to process the text content contained in each candidate text box, and determine the reference text box with incoherent text content among all the candidate text boxes contained in the initial image.

[0071] Specifically, the image processing device can call the incoherence recognition model to perform incoherence detection on each line of text content of each candidate text box to obtain the incoherence evaluation value of each line of text content of each candidate text box; then perform context splicing on each line of text content of each candidate text box to obtain the spliced ​​text content of each candidate text box, and call the incoherence recognition model to perform incoherence detection on the spliced ​​text content of each candidate text box to obtain the incoherence evaluation value of the spliced ​​text content of each candidate text box; finally, based on the incoherence evaluation value of each line of text content of each candidate text box and the incoherence evaluation value of the spliced ​​text content of each candidate text box, determine the incoherence evaluation value of each candidate text box; if the incoherence evaluation value of any one of the candidate text boxes is greater than the incoherence evaluation threshold, then determine any one of the candidate text boxes as a reference text box with incoherent text content.

[0072] The incoherence recognition model may be a text classification model based on Bert, and the semantic coherence of the text content may be recognized to obtain an incoherence evaluation value of the text content.

[0073] Among them, in order to avoid misjudging some subtitle contents with line breaks as incoherent background text, it is necessary to determine the incoherence evaluation value of the candidate text box based on the incoherence evaluation value of each line of text content in the candidate text box and the incoherence evaluation value of the concatenated text content of the candidate text box. Optionally, the minimum value of the incoherence evaluation value of each line of text content in the candidate text box and the incoherence evaluation value of the concatenated text content of the candidate text box can be used as the incoherence evaluation value of the candidate text box. Optionally, the incoherence evaluation value of each line of text content in the candidate text box and the incoherence evaluation value of the concatenated text content of the candidate text box can also be weightedly summed to obtain the incoherence evaluation value of the candidate text box.

[0074] Among them, if the incoherence evaluation value of any candidate text box is greater than the incoherence evaluation threshold, then the any candidate text box is determined to be a reference text box with incoherent text content. Among them, there may be multiple types of incoherence evaluation values, for example, the incoherence evaluation value is an incoherence score, the incoherence evaluation value is an incoherence level, and so on. For example, when the incoherence evaluation value is an incoherence score, the incoherence evaluation threshold is an incoherence score threshold, and if the incoherence score of the candidate text box is greater than the incoherence score threshold, then the candidate text box is a reference text box. For example, when the incoherence evaluation value is an incoherence level, the incoherence evaluation threshold is an incoherence level threshold, and if the incoherence level of the candidate text box is higher than the incoherence level threshold, then the candidate text box is a reference text box.

[0075] S503: Filter the text content contained in the reference text box in the initial image to obtain a target image.

[0076] The initial image may be a single image or an image set including multiple images. Correspondingly, the target image obtained from the initial image may be a single image or an image set including multiple images.

[0077] S504: Crop the target image according to a preset cropping method to obtain a sample image.

[0078] Optionally, when the target image is a single image, the single image may be cropped multiple times to obtain a sample image set including multiple sample images. Optionally, when the target image is an image set including multiple images, each image in the image set may be cropped once in a preset cropping manner to obtain a sample image corresponding to each image, and then a sample image set may be obtained based on the sample images corresponding to all images; each image in the image set may also be cropped multiple times in a preset cropping manner to obtain multiple sample images corresponding to each image, and then a sample image set may be obtained based on all sample images. The preset cropping manner may include random cropping and / or fixed cropping. The up, down, left, and right cropping range indicated by the preset cropping manner may not exceed one third of the height and width of the target image.

[0079] S505, identifying each text box in the sample image according to the position and size of each text box in the target image and a preset cropping method; comparing each text box in the sample image with the corresponding text box in the target image to determine the baseline type of the sample image.

[0080] Specifically, each text box in the sample image is identified according to its position and size in the target image and a preset cropping method; each text box in the sample image is compared with the corresponding text box in the target image to determine the category of each text box in the sample image, and then the base type of the sample image is determined.

[0081] In order to better illustrate the embodiments of the present invention, Figure 1 The example below illustrates this. Figure 1In the example, if image 100 is the target image, the text box containing the normal horizontal subtitle 101 is named as text box 102. Assume that a sample image 110 is obtained by a preset cropping method. In the sample image 110, a text box 112 can be found. By comparing the text box 112 with the text box 102, it can be determined that the text box 112 is horizontally cropped, so it can be determined that the normal horizontal subtitle 101 in the text box 112 is horizontally cropped, and a horizontally cropped horizontal subtitle 111 is obtained. The category of each text box in each sample image of the sample image set can be determined by a method similar to the above method of determining the text box containing horizontally cropped horizontal subtitles in the sample image. Each sample image may include one or more of the following categories of text boxes: a text box containing normal horizontal subtitles, a text box containing horizontally cropped horizontal subtitles, a text box containing vertically cropped horizontal subtitles, a text box containing normal vertical subtitles, a text box containing horizontally cropped vertical subtitles, and a text box containing vertically cropped vertical subtitles. Based on this, each sample image can determine the basic type of the sample image according to the category of each text box in the sample image.

[0082] S506, constructing a training sample including a sample image and a benchmark type of the sample image, calling a target detection model to process the sample image to obtain a predicted type of the sample image; training the target detection model according to the predicted type and the benchmark type to obtain a trained target detection model.

[0083] After obtaining the trained target detection model through the above S501-S506, the image processing device can use the trained target detection model to predict the image to be detected and output the detection result of the image to be detected, that is, output the predicted type of the image to be detected as an incomplete subtitle image or a complete subtitle image.

[0084] When constructing training samples, the embodiment of the present invention performs text recognition on the initial image through the OCR algorithm, and calls the incoherent recognition model to determine the reference text box with incoherent text content from all candidate text boxes in the initial image, and then filters the text content contained in the reference text box in the initial image to obtain the target image, and finally crops the target image according to the preset cropping method to obtain the sample image and determines the reference type of the sample image according to each text box in the sample image and the corresponding text box in the target image, so as to obtain a training sample set containing a large number of sample images and the reference type of the sample image, thereby saving the human cost and time cost of data annotation. At the same time, compared with the recognition method of incomplete subtitle images based on OCR, the present invention can determine the reference text box containing background text through the incoherent recognition model, can accurately distinguish the incomplete subtitles and background text detected by OCR, avoid misidentifying the incoherent background text as incomplete subtitles, improve the accuracy of constructing training samples, and then improve the accuracy of the trained target detection model, and improve the accuracy of identifying the incomplete subtitle quasi-image.

[0085] Based on the description of the above-mentioned image processing method embodiment, the embodiment of the present invention further discloses an image processing device, which may be a computer program (including program code) running in the above-mentioned image processing device. Figure 4 or Figure 5 See the method shown in Figure 6 , the image processing device can run the following units:

[0086] An acquisition unit 601 is used to acquire an image to be detected;

[0087] A determination unit 602 is used to call a trained object detection model to determine a predicted category of a text box to which each text content in the image to be detected belongs, wherein the predicted category is used to indicate a probability that the text content contained in the text box is incomplete content;

[0088] The acquisition unit 601 is further configured to acquire the confidence of each target text box whose predicted category is the target category if there is at least one text box whose predicted category is the target category, wherein the target category is used to indicate that the probability that the text content contained in the text box is incomplete content is greater than a first preset threshold;

[0089] The output unit 603 is used to output the predicted type of the image to be detected as a first type if there is at least one target text box whose confidence is greater than a second preset threshold, wherein the first type is used to indicate that the image to be detected is an incomplete subtitle image.

[0090] In one implementation, after obtaining the confidence of each target text box whose predicted category is the target category, the output unit 603 is further used to:

[0091] If the confidences of the target text boxes are all less than or equal to the second preset threshold, the type of the image to be detected is output as the second type, where the second type is used to indicate that the image to be detected is a complete subtitle image.

[0092] In another embodiment, the output unit 603 is used to output the predicted type of the image to be detected as the first type if there is at least one target text box whose confidence is greater than the second preset threshold, including:

[0093] Comparing the confidence of each target text box with a second preset threshold corresponding to the predicted category of each target text box;

[0094] If there is at least one target text box whose confidence is greater than a second preset threshold corresponding to the predicted category of the target text box, the type of the image to be detected is output as the first type.

[0095] In another embodiment, before calling the trained object detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs, the acquisition unit 601 is further used to acquire a training sample, wherein the training sample includes a sample image and a reference type of the sample image;

[0096] Calling the target detection model to process the sample image to obtain a predicted type of the sample image;

[0097] The target detection model is trained according to the prediction type and the benchmark type to obtain the trained target detection model.

[0098] In another implementation manner, the acquiring unit 601 acquires the training sample, including:

[0099] The target image is cropped according to a preset cropping method to obtain a sample image;

[0100] Identify each text box in the sample image according to the position and size of each text box in the target image and the preset cropping method;

[0101] Comparing each text box in the sample image with a corresponding text box in the target image to determine a reference type of the sample image;

[0102] A training sample including the sample image and a benchmark type of the sample image is constructed.

[0103] In another embodiment, before the acquisition unit 601 crops the target image according to a preset cropping method to obtain a sample image, it is further configured to:

[0104] Performing text recognition on the initial image by using an optical character recognition algorithm to determine each candidate text box in the initial image and the text content contained in each candidate text box;

[0105] Invoking an incoherent recognition model to process text content contained in each candidate text box, and determining a reference text box with incoherent text content among all candidate text boxes contained in the initial image;

[0106] The text content contained in the reference text box in the initial image is filtered to obtain the target image.

[0107] In another embodiment, the acquisition unit 601 calls the incoherent recognition model to process the text content contained in each candidate text box, and determines a reference text box with incoherent text content among all the candidate text boxes contained in each initial image, including:

[0108] Calling the incoherence recognition model to perform incoherence detection on each line of text content in each candidate text box to obtain an incoherence evaluation value of each line of text content in each candidate text box;

[0109] Perform context splicing on each line of text content of each candidate text box to obtain spliced ​​text content of each candidate text box, and call the incoherence recognition model to perform incoherence detection on the spliced ​​text content of each candidate text box to obtain incoherence evaluation values ​​of the spliced ​​text content of each candidate text box;

[0110] Determining the incoherence evaluation value of each candidate text box based on the incoherence evaluation value of each line of text content of each candidate text box and the incoherence evaluation value of the spliced ​​text content of each candidate text box;

[0111] If the incoherence evaluation value of any one of the candidate text boxes is greater than the incoherence evaluation threshold, the any one of the candidate text boxes is determined as the reference text box with incoherent text content.

[0112] According to one embodiment of the present invention, Figure 4 or Figure 5 Each step involved in the method shown can be performed by Figure 6 The various units in the image processing device shown in FIG. Figure 4 Steps S401 and S403 shown are Figure 6 The acquisition unit 601 shown in FIG. 4 is executed, and step S402 is performed by Figure 6 The determination unit 602 shown in FIG. 4 is executed, and step S404 is performed by Figure 6 The output unit 604 shown in FIG.

[0113] According to another embodiment of the present invention, Figure 6 The various units in the image processing device shown can be separately or completely combined into one or several other units to form, or one (some) of the units can be further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present invention, other units can also be included based on the image processing device. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0114] According to another embodiment of the present invention, the processing elements and storage elements including a central processing unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM) and the like can be executed on a general computing device such as a computer. Figure 4 or Figure 5 A computer program (including program code) for each step involved in the corresponding method shown in Figure 6 The image processing apparatus shown in the figure can realize the image processing method of the embodiment of the present invention. The computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above-mentioned image processing device through the computer-readable recording medium and run therein.

[0115] When identifying whether the image to be detected is an incomplete subtitle image, the embodiment of the present invention can first call the trained target detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs, and then determine whether there is a target text box with a predicted category of the target category in the image to be detected. If there is a target text box with a predicted category of the target category, the image to be detected is determined to be an incomplete subtitle image according to the confidence of the target text box and the second preset threshold. Considering that the image to be detected may contain text boxes with complete subtitle content and text boxes with incomplete subtitle content at the same time, compared with the incomplete subtitle image recognition method based on image classification, calling the trained target detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs can distinguish each text box in the image to be detected, can pay attention to the local area information in the image to be detected, and improve the accuracy and recall rate of incomplete subtitle image recognition. In addition, the incomplete subtitle image can be identified based on the comparison result of the confidence of the target text box containing the incomplete subtitle content and the second preset threshold, which can further improve the accuracy of incomplete subtitle image recognition.

[0116] Based on the description of the above-mentioned image processing method embodiment, the present invention also discloses an image processing device. Figure 7 The image processing device at least includes a processor 701, an input interface 702, an output interface 703, and a computer storage medium 704 which can be connected via a bus or other means.

[0117] The computer storage medium 704 is a memory device in the image processing device, which is used to store programs and data. It is understandable that the computer storage medium 704 here can include both the built-in storage medium of the image processing device and the extended storage medium supported by the image processing device. The computer storage medium 704 provides a storage space, which stores the operating system of the image processing device. In addition, one or more instructions suitable for being loaded and executed by the processor 701 are also stored in the storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed RAM memory; optionally, it can also be at least one computer storage medium away from the aforementioned processor. The processor can be called a central processing unit (CPU), which is the core and control center of the image processing device, suitable for implementing one or more instructions, specifically loading and executing one or more instructions to achieve the corresponding method flow or function.

[0118] In one embodiment, the processor 701 may load and execute one or more instructions stored in the computer storage medium 704 to implement the following operations: Figure 4 or Figure 5 In the specific implementation, one or more instructions in the computer storage medium 704 are loaded by the processor 701 and the following steps are executed:

[0119] Acquire the image to be detected;

[0120] Calling the trained object detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs, wherein the predicted category is used to indicate the probability that the text content contained in the text box is incomplete content;

[0121] If there is at least one text box whose predicted category is the target category, obtaining the confidence of each target text box whose predicted category is the target category, wherein the target category is used to indicate that the probability that the text content contained in the text box is incomplete content is greater than a first preset threshold;

[0122] If there is at least one target text box whose confidence is greater than a second preset threshold, the prediction type of the image to be detected is output as a first type, where the first type is used to indicate that the image to be detected is an incomplete subtitle image.

[0123] In one implementation, after obtaining the confidence of each target text box whose predicted category is the target category, the processor 701 is further configured to:

[0124] If the confidences of the target text boxes are all less than or equal to the second preset threshold, the type of the image to be detected is output as the second type, where the second type is used to indicate that the image to be detected is a complete subtitle image.

[0125] In another embodiment, the processor 701 is configured to output the predicted type of the image to be detected as the first type if there is at least one target text box whose confidence is greater than a second preset threshold, including:

[0126] Comparing the confidence of each target text box with a second preset threshold corresponding to the predicted category of each target text box;

[0127] If there is at least one target text box whose confidence is greater than a second preset threshold corresponding to the predicted category of the target text box, the type of the image to be detected is output as the first type.

[0128] In another embodiment, before calling the trained object detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs, the processor 701 is further used to obtain a training sample, wherein the training sample includes a sample image and a reference type of the sample image;

[0129] Calling the target detection model to process the sample image to obtain a predicted type of the sample image;

[0130] The target detection model is trained according to the prediction type and the benchmark type to obtain the trained target detection model.

[0131] In another implementation manner, the processor 701 obtains the training sample, including:

[0132] The target image is cropped according to a preset cropping method to obtain a sample image;

[0133] Identify each text box in the sample image according to the position and size of each text box in the target image and the preset cropping method;

[0134] Comparing each text box in the sample image with a corresponding text box in the target image to determine a reference type of the sample image;

[0135] A training sample including the sample image and a benchmark type of the sample image is constructed.

[0136] In another implementation manner, before the processor 701 crops the target image according to a preset cropping method to obtain a sample image, the processor 701 is further configured to:

[0137] Performing text recognition on the initial image by using an optical character recognition algorithm to determine each candidate text box in the initial image and the text content contained in each candidate text box;

[0138] Invoking an incoherent recognition model to process text content contained in each candidate text box, and determining a reference text box with incoherent text content among all candidate text boxes contained in the initial image;

[0139] The text content contained in the reference text box in the initial image is filtered to obtain the target image.

[0140] In another embodiment, the processor 701 calls the incoherent recognition model to process the text content contained in each candidate text box, and determines a reference text box with incoherent text content in all the candidate text boxes contained in each initial image, including:

[0141] Calling the incoherence recognition model to perform incoherence detection on each line of text content in each candidate text box to obtain an incoherence evaluation value of each line of text content in each candidate text box;

[0142] Perform context splicing on each line of text content of each candidate text box to obtain spliced ​​text content of each candidate text box, and call the incoherence recognition model to perform incoherence detection on the spliced ​​text content of each candidate text box to obtain incoherence evaluation values ​​of the spliced ​​text content of each candidate text box;

[0143] Determining the incoherence evaluation value of each candidate text box based on the incoherence evaluation value of each line of text content of each candidate text box and the incoherence evaluation value of the spliced ​​text content of each candidate text box;

[0144] If the incoherence evaluation value of any one of the candidate text boxes is greater than the incoherence evaluation threshold, the any one of the candidate text boxes is determined as the reference text box with incoherent text content.

[0145] When identifying whether the image to be detected is an incomplete subtitle image, the embodiment of the present invention can first call the trained target detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs, and then determine whether there is a target text box with a predicted category of the target category in the image to be detected. If there is a target text box with a predicted category of the target category, the image to be detected is determined to be an incomplete subtitle image according to the confidence of the target text box and the second preset threshold. Considering that the image to be detected may contain text boxes with complete subtitle content and text boxes with incomplete subtitle content at the same time, compared with the incomplete subtitle image recognition method based on image classification, calling the trained target detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs can distinguish each text box in the image to be detected, can pay attention to the local area information in the image to be detected, and improve the accuracy and recall rate of incomplete subtitle image recognition. In addition, the incomplete subtitle image can be identified based on the comparison result of the confidence of the target text box containing the incomplete subtitle content and the second preset threshold, which can further improve the accuracy of incomplete subtitle image recognition.

[0146] It should be noted that the embodiment of the present invention further provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the image processing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the image processing device performs the above-mentioned image processing method embodiment. Figure 4 or Figure 5 The steps performed in

[0147] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. An image processing method, characterized in that: include: Acquire the image to be detected; Calling the trained object detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs, wherein the predicted category is used to indicate the probability that the text content contained in the text box is incomplete content; If there is at least one text box whose predicted category is the target category, obtaining the confidence of each target text box whose predicted category is the target category, wherein the target category is used to indicate that the probability that the text content contained in the text box is incomplete content is greater than a first preset threshold; If there is at least one target text box whose confidence is greater than a second preset threshold, outputting the prediction type of the image to be detected as a first type, where the first type is used to indicate that the image to be detected is an incomplete subtitle image; The target text box includes one or more of the following: The probability that the subtitle content contained in each target text box including the horizontally cut horizontal subtitle, the vertically cut horizontal subtitle, the horizontally cut vertical subtitle and the vertically cut vertical subtitle is incomplete subtitle is greater than a first preset threshold.

2. The method according to claim 1, characterized in that After obtaining the confidence of each target text box whose predicted category is the target category, the method further includes: If the confidences of the target text boxes are all less than or equal to the second preset threshold, the type of the image to be detected is output as the second type, where the second type is used to indicate that the image to be detected is a complete subtitle image.

3. The method according to claim 1, characterized in that If there is at least one target text box whose confidence is greater than a second preset threshold, outputting the predicted type of the image to be detected as the first type includes: Compare the confidence of each target text box with a second preset threshold corresponding to the predicted category of each target text box; If there is at least one target text box whose confidence is greater than a second preset threshold corresponding to the predicted category of the target text box, the type of the image to be detected is output as the first type.

4. The method according to claim 1, characterized in that Before calling the trained object detection model to determine the predicted category of the text box to which each text content in the image to be detected belongs, the method further includes: Acquire a training sample, wherein the training sample includes a sample image and a reference type of the sample image; Calling the target detection model to process the sample image to obtain a predicted type of the sample image; The target detection model is trained according to the prediction type and the benchmark type to obtain the trained target detection model.

5. The method according to claim 4, characterized in that The obtaining of training samples comprises: The target image is cropped according to a preset cropping method to obtain a sample image; Identify each text box in the sample image according to the position and size of each text box in the target image and the preset cropping method; Comparing each text box in the sample image with a corresponding text box in the target image to determine a reference type of the sample image; A training sample including the sample image and a benchmark type of the sample image is constructed.

6. The method according to claim 5, characterized in that Before cropping the target image according to the preset cropping method to obtain the sample image, the method further includes: Performing text recognition on the initial image by using an optical character recognition algorithm to determine each candidate text box in the initial image and the text content contained in each candidate text box; Invoking an incoherent recognition model to process text content contained in each candidate text box, and determining a reference text box with incoherent text content among all candidate text boxes contained in the initial image; The text content contained in the reference text box in the initial image is filtered to obtain the target image.

7. The method according to claim 6, characterized in that The incoherent recognition model is called to process the text content contained in each candidate text box, and a reference text box with incoherent text content is determined in all candidate text boxes contained in the initial image, including: Calling the incoherence recognition model to perform incoherence detection on each line of text content in each candidate text box to obtain an incoherence evaluation value of each line of text content in each candidate text box; Perform context splicing on each line of text content of each candidate text box to obtain spliced ​​text content of each candidate text box, and call the incoherence recognition model to perform incoherence detection on the spliced ​​text content of each candidate text box to obtain incoherence evaluation values ​​of the spliced ​​text content of each candidate text box; Determining the incoherence evaluation value of each candidate text box based on the incoherence evaluation value of each line of text content of each candidate text box and the incoherence evaluation value of the spliced ​​text content of each candidate text box; If the incoherence evaluation value of any one of the candidate text boxes is greater than the incoherence evaluation threshold, the any one of the candidate text boxes is determined as the reference text box with incoherent text content.

8. An image processing device, characterized in that: include: An acquisition unit, used for acquiring an image to be detected; A determination unit, used to call a trained target detection model to determine a predicted category of a text box to which each text content in the image to be detected belongs, wherein the predicted category is used to indicate a probability that the text content contained in the text box is incomplete content; The acquisition unit is further configured to acquire the confidence of each target text box whose predicted category is the target category if there is at least one text box whose predicted category is the target category, wherein the target category is used to indicate that the probability that the text content contained in the text box is incomplete content is greater than a first preset threshold; an output unit, configured to output the predicted type of the image to be detected as a first type if there is at least one target text box whose confidence is greater than a second preset threshold, wherein the first type is used to indicate that the image to be detected is an incomplete subtitle image; The target text box includes one or more of the following: The probability that the subtitle content contained in each target text box including the horizontally cut horizontal subtitle, the vertically cut horizontal subtitle, the horizontally cut vertical subtitle and the vertically cut vertical subtitle is incomplete subtitle is greater than a first preset threshold.

9. An image processing device, comprising an input interface and an output interface, characterized in that: Also includes: a processor adapted to implement one or more instructions; as well as, A computer storage medium storing one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the image processing method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the image processing method according to any one of claims 1 to 7.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the image processing method according to any one of claims 1 to 7 are implemented.