Picture processing method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides a picture processing method and device, electronic equipment and a computer readable storage medium. The picture processing method comprises the steps that an initial picture is acquired; in response to determining that the initial picture is a long screenshot, dividing the initial picture into a plurality of segmented pictures, performing region type detection on each segmented picture by using a target detection technology, and integrating detection results of each segmented picture to obtain a detection result of the initial picture; determining a to-be-shielded target area according to the detection result of the initial picture by using an optical character recognition technology and a natural language processing technology; and shielding the target area to obtain a target picture. Therefore, the picture is divided, the divided picture is detected based on the target detection technology, and the detection results are integrated, so that the accuracy and integrity of picture privacy information identification are improved. Meanwhile, the picture processing method can be executed by using an artificial intelligence model.

Description

Technical Field

[0001] The present disclosure generally relates to the field of computers, and more specifically, to a picture processing method, device, electronic device, and computer-readable storage medium. Background Art

[0002] With the development of technology, mobile terminals are becoming more and more popular. At present, more and more users like to share various files with people around them, such as pictures, videos, documents, recordings, etc. When sharing files, some content may involve user privacy. Therefore, it is necessary to block the part of the shared content that involves user privacy, thereby ensuring the privacy security of users.

[0003] In the prior art, in order to protect the user's personal privacy data, possible privacy information (including names, school names, bank account numbers, or ID card numbers, etc.) can be automatically detected and automatically shielded / blocked / covered / coded. However, when the shared file is a long screenshot, the privacy information of the long screenshot cannot be accurately identified, and the accuracy of privacy information identification is poor. Summary of the invention

[0004] Embodiments of the present disclosure provide a method, device, electronic device, and computer-readable storage medium for image processing to at least solve the problem of poor accuracy in identifying privacy information in long screenshots in the above-mentioned related technologies.

[0005] According to a first aspect of an embodiment of the present disclosure, a picture processing method is provided, the picture processing method comprising: obtaining an initial picture; in response to determining that the initial picture is a long screenshot, dividing the initial picture into a plurality of segmented pictures, performing region type detection on each segmented picture using target detection technology, and integrating the detection results of each segmented picture to obtain a detection result of the initial picture; determining a target area to be occluded according to the detection result of the initial picture using optical character recognition technology and natural language processing technology; and occluding the target area to obtain a target picture.

[0006] Optionally, in response to a first ratio being greater than or equal to a first preset threshold, the initial image is determined to be the long screenshot, wherein the first ratio is determined based on an aspect ratio of the initial image and a preset aspect ratio.

[0007] Optionally, dividing the initial picture into a plurality of segmented pictures includes: dividing the initial picture into a plurality of segmented pictures according to a preset length, or equally dividing the initial picture into a plurality of segmented pictures based on the length of the initial picture.

[0008] Optionally, the target detection technology is used to perform region type detection on each segmented picture, and the detection results of each segmented picture are integrated to obtain the detection results of the initial picture, including: for each segmented picture, an intermediate picture is obtained based on the intermediate area between the segmented picture and the next adjacent segmented picture; the target detection technology is used to perform region type detection on each segmented picture and each intermediate picture to obtain the detection results of each segmented picture and each intermediate picture, wherein the detection results include the entity area, the entity type of the entity area and the location coordinates of the entity area; the detection results of all segmented pictures and all intermediate pictures are integrated into the initial picture to obtain the detection results of the initial picture.

[0009] Optionally, the detection results of all segmented pictures and the detection results of all intermediate pictures are integrated into the initial picture to obtain the detection results of the initial picture, including: converting the local position coordinates of each entity area of ​​each segmented picture in the current picture into the global position coordinates in the initial picture; converting the local position coordinates of each entity area of ​​each intermediate picture in the current picture into the global position coordinates in the initial picture; based on the global position coordinates of each entity area of ​​each segmented picture and each intermediate picture in the initial picture, reflecting each entity area and its entity type of all segmented pictures and all intermediate pictures into the initial picture to obtain the detection results of the initial picture.

[0010] Optionally, reflecting each entity region of all segmented images and all intermediate images into the initial image includes: when two entity regions have overlapping parts, replacing the two entity regions with a second entity region covering the two entity regions.

[0011] Optionally, the use of target detection technology to perform region type detection on each segmented image, and integrating the detection results of each segmented image to obtain the detection result of the initial image, includes: for each segmented image, performing the following steps: using the target detection technology to perform region type detection on the segmented image to obtain the detection result of the segmented image, the detection result includes the entity area, the entity type of the entity area and the position coordinates of the entity area; sorting the entity areas from top to bottom based on the position coordinates of the entity areas in the segmented image to establish an entity array; determining the entity type of the last entity area in the entity array; updating the bottom position of the segmented image according to the entity type and position coordinates of the last entity area, and using the updated bottom position of the segmented image as the top position of the next segmented image; integrating the detection results of all segmented images into the initial image to obtain the detection result of the initial image.

[0012] Optionally, updating the bottom position of the segmented image according to the entity type and position coordinates of the last entity area includes: when the entity type of the last entity area is the first entity type, and the distance between the position coordinates of the last entity area and the bottom position of the segmented image is within a preset range, searching the entity array from back to front for entity areas of the second entity type or the third entity type, and updating the bottom position of the segmented image based on the position coordinates of the found entity areas; when the entity type of the last entity area is the second entity type, and the distance between the position coordinates of the last entity area and the bottom position of the segmented image is within the preset range, searching the entity array from back to front for entity areas of the second entity type or the third entity type other than the last entity area. And based on the position coordinates of the found entity area, the bottom position of the segmented picture is updated; when the entity type of the last entity area is the third entity type, and the distance between the position coordinates of the last entity area and the bottom position of the segmented picture is within the preset range, the entity area of ​​the second entity type or the third entity type other than the last entity is searched from back to front in the entity array, and the bottom position of the segmented picture is updated based on the position coordinates of the found entity area, wherein, when the entity area of ​​the second entity type or the third entity type is not found in the entity array, the bottom position of the segmented picture is kept unchanged; when the distance between the position coordinates of the last entity area and the bottom position of the segmented picture exceeds the preset range, the bottom position of the segmented picture is kept unchanged.

[0013] Optionally, the detection results of all segmented pictures are integrated into the initial picture to obtain the detection results of the initial picture, including: converting the local position coordinates of each entity area of ​​each segmented picture in the current picture into the global position coordinates in the initial picture; based on the global position coordinates of each entity area of ​​each segmented picture in the initial picture, reflecting each entity area and its entity type of all segmented pictures into the initial picture to obtain the detection results of the initial picture.

[0014] Optionally, the processing method further includes: when the entity types of adjacent entity areas in the initial image are all third entity types, and the distance between adjacent entity areas is less than or equal to a second preset threshold, merging adjacent entity areas.

[0015] Optionally, the use of optical character recognition technology and natural language processing technology to determine the target area to be occluded according to the detection result of the initial image includes: when the entity type of the entity area of ​​the initial image is a first entity type, determining the entity area as the target area to be occluded; when the entity type of the entity area of ​​the initial image is a second entity type or a third entity type, intercepting the entity area as a temporary image; using optical character recognition technology to recognize text information in the temporary image; and using natural language processing technology to determine the target area to be occluded in the temporary image from the text information.

[0016] Optionally, the use of optical character recognition technology and natural language processing technology to determine the target area to be occluded according to the detection result of the initial image includes: when the entity type of the entity area of ​​the initial image is a first entity type, determining the entity area as the target area to be occluded; when the entity type of the entity area of ​​the initial image is a second entity type or a third entity type, intercepting the entity area as a temporary image; using optical character recognition technology to recognize text information in the temporary image; and using natural language processing technology to determine the target area to be occluded in the temporary image from the text information.

[0017] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing device, comprising: an image acquisition unit, configured to acquire an initial image; an image division unit, configured to, in response to determining that the initial image is a long screenshot, divide the initial image into a plurality of segmented images, perform region type detection on each segmented image using target detection technology, and integrate the detection results of each segmented image to obtain a detection result of the initial image; a target region determination unit, configured to determine a target region to be occluded according to the detection result of the initial image using optical character recognition technology and natural language processing technology; and an occlusion unit, configured to occlude the target region to obtain a target image.

[0018] Optionally, the image acquisition unit is further configured to: in response to a first ratio being greater than or equal to a first preset threshold, determine that the initial image is the long screenshot, wherein the first ratio is determined based on an aspect ratio of the initial image and a preset aspect ratio.

[0019] Optionally, the picture dividing unit is further configured to: divide the initial picture into a plurality of divided pictures according to a preset length, or divide the initial picture into a plurality of divided pictures on an even basis based on the length of the initial picture.

[0020] Optionally, the image division unit is further configured to: for each segmented image, obtain an intermediate image based on the intermediate area between the segmented image and the next adjacent segmented image; use target detection technology to perform region type detection on each segmented image and each intermediate image to obtain detection results for each segmented image and each intermediate image, wherein the detection results include entity areas, entity types of entity areas, and location coordinates of entity areas; integrate the detection results of all segmented images and all intermediate images into the initial image to obtain the detection results of the initial image.

[0021] Optionally, the image division unit is further configured to: convert the local position coordinates of each entity area of ​​each segmented image in the current image into global position coordinates in the initial image; convert the local position coordinates of each entity area of ​​each intermediate image in the current image into global position coordinates in the initial image; based on the global position coordinates of each entity area of ​​each segmented image and each intermediate image in the initial image, reflect each entity area and its entity type of all segmented images and all intermediate images into the initial image to obtain the detection result of the initial image.

[0022] Optionally, the picture dividing unit is further configured to: when there is an overlapping portion between two physical areas, replace the two physical areas with a second physical area covering the two physical areas.

[0023] Optionally, the image division unit is further configured to: for each segmented image, perform the following steps: use target detection technology to perform region type detection on the segmented image to obtain a detection result of the segmented image, wherein the detection result includes an entity region, an entity type of the entity region, and a position coordinate of the entity region; sort the entity regions from top to bottom based on the position coordinates of the entity regions in the segmented image to establish an entity array; determine the entity type of the last entity region in the entity array; update the bottom position of the segmented image according to the entity type and position coordinates of the last entity region, and use the updated bottom position of the segmented image as the top position of the next segmented image; integrate the detection results of all segmented images into the initial image to obtain the detection result of the initial image.

[0024] Optionally, the image division unit is further configured to: when the entity type of the last entity area is the first entity type, and the distance between the position coordinates of the last entity area and the bottom position of the segmented image is within a preset range, search for entity areas of the second entity type or the third entity type from back to front in the entity array, and update the bottom position of the segmented image based on the position coordinates of the found entity area; when the entity type of the last entity area is the second entity type, and the distance between the position coordinates of the last entity area and the bottom position of the segmented image is within the preset range, search for entity areas of the second entity type or the third entity type other than the last entity area from back to front in the entity array, and update the bottom position of the segmented image based on the position coordinates of the found entity area The method comprises the steps of: updating the bottom position of the segmented picture according to the coordinates of the entity area; when the entity type of the last entity area is the third entity type, and the distance between the position coordinates of the last entity area and the bottom position of the segmented picture is within the preset range, searching the entity array from back to front for an entity area of ​​the second entity type or the third entity type other than the last entity, and updating the bottom position of the segmented picture based on the position coordinates of the found entity area, wherein when the entity area of ​​the second entity type or the third entity type is not found in the entity array, the bottom position of the segmented picture is kept unchanged; when the distance between the position coordinates of the last entity area and the bottom position of the segmented picture exceeds the preset range, the bottom position of the segmented picture is kept unchanged.

[0025] Optionally, the image division unit is further configured to: convert the local position coordinates of each entity area of ​​each segmented image in the current image into global position coordinates in the initial image; based on the global position coordinates of each entity area of ​​each segmented image in the initial image, reflect each entity area and its entity type of all segmented images into the initial image to obtain the detection result of the initial image.

[0026] Optionally, the image processing device also includes: an entity area merging unit, configured to merge adjacent entity areas when the entity types of adjacent entity areas in the initial image are all third entity types and the distance between adjacent entity areas is less than or equal to a second preset threshold.

[0027] Optionally, the target area determination unit is further configured to: when the entity type of the entity area of ​​the initial image is the first entity type, determine the entity area as the target area to be occluded; when the entity type of the entity area of ​​the initial image is the second entity type or the third entity type, capture the entity area as a temporary image; use optical character recognition technology to recognize text information in the temporary image; and use natural language processing technology to determine the target area to be occluded in the temporary image from the text information.

[0028] Optionally, the target area determination unit is further configured to: when the entity type of the entity area of ​​the initial image is the first entity type, determine the entity area as the target area to be occluded; when the entity type of the entity area of ​​the initial image is the second entity type or the third entity type, capture the entity area as a temporary image; use optical character recognition technology to recognize text information in the temporary image; and use natural language processing technology to determine the target area to be occluded in the temporary image from the text information.

[0029] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement any one of the image processing methods described above.

[0030] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium storing a computer program is provided. When the computer program is executed by a processor, the image processing method as described above is implemented.

[0031] According to the image processing method, device, electronic device and computer-readable storage medium disclosed in the present invention, it is possible to divide the image, perform region type detection on the divided image through target detection technology and integrate the detection results, and then use optical character recognition technology and natural language processing technology to determine the target area of ​​the image to be blocked, thereby reducing the phenomenon of incorrect recognition or missed recognition of privacy information in the image and improving the accuracy and completeness of privacy information recognition.

[0032] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0034] Figure 1 is a diagram showing an example of preprocessing a picture in the prior art.

[0035] Figure 2 is a flowchart illustrating a picture processing method according to an embodiment of the present disclosure.

[0036] Figure 3 is a diagram illustrating an example of obtaining an intermediate picture based on adjacent segmented pictures according to an embodiment of the present disclosure.

[0037] Figure 4 is a diagram illustrating an example of a result of region type detection according to an embodiment of the present disclosure.

[0038] Figure 5 : is a diagram illustrating an example of a manner of processing entity regions having overlapping portions according to an embodiment of the present disclosure.

[0039] Figure 6 is a diagram illustrating an example of updating a bottom position of a divided picture according to an embodiment of the present disclosure.

[0040] Figure 7 is a diagram showing the overall design of an NLP module that performs NLP technology according to an embodiment of the present disclosure.

[0041] Figure 8 is a diagram illustrating an example of multi-line text processing within an entity area according to an embodiment of the present disclosure.

[0042] Fig. 9 2 is a diagram illustrating an example of an implementation scheme of a picture processing method according to an embodiment of the present disclosure.

[0043] Fig.10 is a flowchart illustrating an example of identifying and shielding private information by inserting an intermediate picture according to an embodiment of the present disclosure.

[0044] Fig.11 is a flowchart showing an example of identifying and shielding private information by dynamically adjusting the bottom of a picture according to an embodiment of the present disclosure.

[0045] Fig.12 is a block diagram illustrating a picture processing apparatus according to an embodiment of the present disclosure.

[0046] Fig.13 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0047] The following specific embodiments are provided to help the reader gain a comprehensive understanding of the methods, devices and / or systems described herein. However, various changes, modifications and equivalents of the methods, devices and / or systems described herein will be clear after understanding the disclosure of the present disclosure. For example, the order of operations described herein is only an example and is not limited to those orders set forth herein, but can be changed as will be clear after understanding the disclosure of the present disclosure, except for operations that must occur in a specific order. In addition, for greater clarity and conciseness, descriptions of features known in the art may be omitted.

[0048] The features described herein may be implemented in different forms and should not be construed as being limited to the examples described herein. Rather, the examples described herein have been provided to illustrate only some of the many possible ways to implement the methods, devices, and / or systems described herein, many of which will be clear after understanding the disclosure of the present disclosure.

[0049] As used herein, the term "and / or" includes any one of the associated listed items and any combination of any two or more.

[0050] Although terms such as "first", "second", and "third" may be used herein to describe various members, components, regions, layers, or portions, these members, components, regions, layers, or portions should not be limited by these terms. Instead, these terms are only used to distinguish one member, component, region, layer, or portion from another member, component, region, layer, or portion. Therefore, without departing from the teachings of the examples described herein, the first member, first component, first region, first layer, or first portion referred to in the examples may also be referred to as the second member, second component, second region, second layer, or second portion.

[0051] In the specification, when an element (such as a layer, a region, or a substrate) is described as being “on”, “connected to”, or “coupled to” another element, the element may be directly “on”, “connected to”, or “coupled to” another element, or one or more other elements may be present therebetween. Conversely, when an element is described as being “directly on”, “directly connected to”, or “directly coupled to” another element, there may be no other elements present therebetween.

[0052] The terms used herein are only used to describe various examples and are not intended to limit the disclosure. Unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. The terms "comprise", "include" and "have" indicate the presence of the described features, quantities, operations, components, elements and / or combinations thereof, but do not exclude the presence or addition of one or more other features, quantities, operations, components, elements and / or combinations thereof.

[0053] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by a person of ordinary skill in the art to which the present disclosure belongs after understanding the present disclosure. Unless explicitly defined as such herein, terms (such as those defined in a general dictionary) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and should not be interpreted in an idealized or overly formal manner.

[0054] Furthermore, in the description of examples, when it is considered that a detailed description of a well-known related structure or function would cause vague interpretation of the present disclosure, such a detailed description will be omitted.

[0055] Figure 1 is a diagram showing an example of preprocessing a picture in the prior art.

[0056] Reference Figure 1 In the prior art, before using the object detection (OD) technology to identify the region type of the image, the image needs to be preprocessed. Figure 1 As shown in (a) in the figure, the rectangular image is first converted into a square. There are two ways to convert a rectangular image into a square. One is to fill the rectangle with a grayscale image to convert the rectangular image into a square image. The other is to directly stretch the rectangular image into a square image, but this method will distort the content in the image, resulting in low image recognition accuracy. Then, the converted square image is scaled to a preset size to reduce the amount of calculation of the target detection technology. Again, as shown in Figure 1 As shown in (a) of Figure 2, the effective content (i.e., non-grayscale content) of the ordinary screenshot is effectively preserved, so the private information in the image can be accurately and effectively identified. Figure 1 As shown in (b) in the figure, when the image is a long screenshot, after preprocessing the long screenshot, the recognition effect of the image is poor due to the high proportion of grayscale part in the square image converted from the long screenshot, and a large amount of privacy information cannot be correctly recognized, resulting in the failure of the privacy protection function of the long screenshot.

[0057] In order to solve the above problems, the image processing method, device, electronic device and computer-readable storage medium provided by the present disclosure can improve the accuracy and completeness of privacy information identification in long screenshots by dividing long screenshots, detecting the divided long screenshots based on target detection technology, and then determining the target area to be blocked in the long screenshots through optical character recognition technology and natural language processing technology.

[0058] Figure 2 is a flowchart illustrating a picture processing method according to an embodiment of the present disclosure.

[0059] Reference Figure 2 In step S201, an initial picture may be obtained. Exemplarily, when the user obtains the initial picture by taking a screenshot, it may be determined whether the initial picture is a chat picture by using an image binary classification model. For example, a square picture may be captured according to the width of the initial picture, and the image binary classification technology and the square picture may be used to determine whether the initial picture is a chat picture. When it is determined that the initial picture is a chat picture, a privacy information shielding function may be triggered for the chat picture.

[0060] It should be noted that the "privacy information" mentioned in this disclosure may include but is not limited to the following items: name, profile picture, ID number, title of instant messaging group, telephone number, bank card account number, license plate number, address, e-mail, courier number, flight number, etc.

[0061] In step S202, in response to determining that the initial image is a long screenshot, the initial image is divided into multiple segmented images, each segmented image is subjected to region type detection using target detection technology, and the detection results of each segmented image are integrated to obtain the detection result of the initial image.

[0062] According to an embodiment of the present disclosure, when the initial image is a chat image, in response to the first ratio being greater than or equal to the first preset threshold, the initial image is determined to be a long screenshot. Here, the first ratio may be determined according to the aspect ratio of the initial image and the preset aspect ratio. Alternatively, the preset aspect ratio may be determined according to the length and width of the screen of the mobile terminal. For example, the length h of the screen of the mobile terminal may be obtained. 1 and width 1 , according to the ratio of the length and width of the screen, determine the preset aspect ratio r 1 =h 1 / w 1 . Assume that the height of the initial image is h 2 , width is w 2 , then the aspect ratio of the initial image is r 2 =h 2 / w 2 The first ratio is the ratio of the aspect ratio of the initial image to the preset aspect ratio. The first preset threshold can be set according to actual needs. For example, the value range of the first preset threshold can be 1.3-1.5. Generally, the first preset threshold is set to 1.3, but the present disclosure is not limited thereto. When the first ratio of the aspect ratio of the initial image to the preset aspect ratio is greater than or equal to 1.3 (i.e., r 2 / r1>=1.3), it can be determined that the initial image is a long screenshot.

[0063] On the other hand, when it is determined that the initial image is a long screenshot, the number of the multiple segmented images that the initial image needs to be divided into can be determined according to the first ratio. For example, when the first ratio is 2.5, the number of segmented pictures that need to be divided is 3.

[0064] Furthermore, when the initial picture is divided, two different methods can be used. The initial picture can be divided into multiple segmented pictures according to a preset length, or the initial picture can be evenly divided into multiple segmented pictures based on the length of the initial picture. For example, for the first method, the initial picture can be divided into multiple segmented pictures according to the length of the screen of the mobile terminal. When the number of segmented pictures to be divided is 3, two segmented pictures with the same length and width as the screen and one segmented picture with a length less than or equal to the length of the screen can be obtained. On the other hand, for the second method, when the number of segmented pictures to be divided is 3, the initial picture can be evenly divided into 3 segmented pictures of the same length.

[0065] According to an embodiment of the present disclosure, in order to ensure the recognition accuracy of the object located at the bottom of the segmented picture, different processing methods can be adopted to use target detection technology to perform region type detection on each segmented picture, and integrate the detection results of each segmented picture to obtain the detection result of the initial picture.

[0066] Specifically, in one processing method, for each segmented picture, an intermediate picture can be obtained based on the intermediate area between the segmented picture and the next adjacent segmented picture; target detection technology is used to perform region type detection on each segmented picture and each intermediate picture to obtain detection results for each segmented picture and each intermediate picture, wherein the detection results include entity areas, entity types of entity areas, and location coordinates of entity areas; the detection results of all segmented pictures and all intermediate pictures are integrated into the initial picture to obtain the detection results of the initial picture.

[0067] Figure 3 is a diagram illustrating an example of obtaining an intermediate picture based on adjacent segmented pictures according to an embodiment of the present disclosure.

[0068] Reference Figure 3 , assuming that there are three segmented pictures, namely, segmented picture 1, segmented picture 2 and segmented picture 3. The middle area of ​​the adjacent segmented picture 1 and segmented picture 2 (namely, the lower half of segmented picture 1 and the upper half of segmented picture 2) can be selected to obtain the middle picture 1.5, and the middle area of ​​the adjacent segmented picture 2 and segmented picture 3 (namely, the lower half of segmented picture 2 and the upper half of segmented picture 3) can be selected to obtain the middle picture 2.5.

[0069] As described above, the detection results of the segmented images and the intermediate images may include the entity area, the entity type of the entity area, and the location coordinates of the entity area. Using target detection technology to perform region type detection, a total of 5 entity types can be obtained, namely "chat content (content)", "other content (other)", "chat title (title)", "account nickname (username)", and "account avatar (avatar)". Here, "chat title (title)", "account nickname (username)", and "account avatar (avatar)" can be collectively referred to as the first entity type, "other content (other)" as the second entity type, and "chat content (content)" as the third entity type. However, the specific classification of entity types is not limited to the above examples, but can be classified differently as needed.

[0070] Figure 4 is a diagram illustrating an example of a result of region type detection according to an embodiment of the present disclosure.

[0071] Reference Figure 4 , Figure 4 (a) and (b) show the results of region type detection of the two images. Through the target detection technology, the above five entity types are detected in both images. In addition, the entity regions detected in the two images can be rectangular regions, where:

[0072] "title", in Figure 4 Abbreviated as "t" in the text box: the title of the chat screenshot, i.e., the chat title. When the picture is a screenshot of a private chat interface of an instant messaging application, "title" is the username of the other party's account; when the picture is a screenshot of a group chat interface of an instant messaging application, "title" is the name of the instant messaging group.

[0073] "avatar", in Figure 4 Abbreviated as "a" in Chinese: The account's avatar, including the other party's and your own avatars, is usually a square, rounded square or circle.

[0074] "username", in Figure 4 Abbreviated as "u" in Chinese: Nickname of the account, usually appears in group chats.

[0075] "content", in Figure 4 Abbreviated as "c": chat content, which can be text, pictures, emoticons, web page reference information, resource information, voice information, text reference information, etc. Among them, "web page reference information" can be a jump link, that is, if the user clicks on the jump link, he can jump to a new page; "text reference information" can be text information that quotes the speech of a certain account.

[0076] "other", in Figure 4 Abbreviated as "o": other content, which can be time information, message withdrawal information, information on inviting new accounts to join the group, prompt information for a certain account, modification of the title of the group chat, administrator management information for the instant messaging group, etc.

[0077] The object detection technology used in the present disclosure can be performed by an OD module. The OD module can be a multi-classification object detection model, and exemplarily, the OD module can be a You Only Look Once (YOLO), a Single Shot Multibox Detector (SSD), a RetinaNet, or other models.

[0078] According to an embodiment of the present disclosure, the detection results of all segmented pictures and the detection results of all intermediate pictures can be integrated into the initial picture in the following manner. Specifically, the position coordinates of the entity area of ​​each segmented picture and the entity area of ​​each intermediate picture can be converted. That is, the local position coordinates of each entity area of ​​each segmented picture in the current picture are converted to the global position coordinates in the initial picture; the local position coordinates of each entity area of ​​each intermediate picture in the current picture are converted to the global position coordinates in the initial picture. Then, according to the global position coordinates of each entity area of ​​each segmented picture and each intermediate picture in the initial picture, each entity area and its entity type of all segmented pictures and all intermediate pictures are reflected in the initial picture to obtain the detection result of the initial picture.

[0079] Optionally, when each entity region of all divided pictures and all intermediate pictures is reflected in the initial picture, if two entity regions have overlapping parts, the two entity regions are replaced by a second entity region covering the two entity regions. Figure 5 is a diagram showing an example of a method of processing entity regions having overlapping portions according to an embodiment of the present disclosure. Figure 5 , there must be duplicate detection results between the two adjacent segmented images and the intermediate image between them. For example, both the segmented image 1 and the intermediate image 1.5 identify the entity region of the first entity type, thereby generating two overlapping entity regions. At this time, the two overlapping entity regions can be replaced by a second entity region that can cover the two overlapping entity regions.

[0080] According to the embodiments of the present disclosure, by adding an intermediate picture between two adjacent segmented pictures, the lack or omission of privacy information identification of the initial picture can be reduced, thereby improving the accuracy and completeness of the privacy information identification of the picture.

[0081] Alternatively, in another processing method, first, for each segmented picture, the following steps can be performed: using the target detection technology to perform region type detection on the segmented picture to obtain the detection result of the segmented picture, wherein, as described above, the detection result includes the entity region, the entity type of the entity region, and the position coordinates of the entity region; sorting the entity regions from top to bottom based on the position coordinates of the entity regions in the segmented picture to establish an entity array; determining the entity type of the last entity region in the entity array; updating the bottom position of the segmented picture according to the entity type and position coordinates of the last entity region, and using the updated bottom position of the segmented picture as the top position of the next segmented picture. After updating the bottom position of the segmented picture for each segmented picture, the detection results of all segmented pictures can be integrated into the initial picture to obtain the detection result of the initial picture.

[0082] Figure 6 is a diagram illustrating an example of updating a bottom position of a divided picture according to an embodiment of the present disclosure.

[0083] Reference Figure 6 First, the target detection technology is used to detect the region type of the segmented image 1, and the detection result of the segmented image 1 is obtained. Figure 6 As shown in (a), when the entity type of the last entity area is the first entity type, i.e., one of "chat title (title)", "account nickname (username)", and "account avatar (avatar)", and the distance between the position coordinates of the last entity area and the bottom position of the split image is within a preset range, the entity area of ​​the second entity type (i.e., "other content (other)") or the third entity type (i.e., "chat content (content)") is searched from back to front in the entity array, and the bottom position of the split image is updated based on the position coordinates of the found entity area. Here, the preset range can be determined according to actual needs, and is used to determine whether the entity area is near the bottom of the split image. In this way, if Figure 6 As shown in (a), when the entity type of the last entity area in the entity array is the account's avatar (a), entity areas whose entity types are other content (o) or chat content (c) in the entity array are selected from the back to the front to update the bottom of Figure 1.

[0084] like Figure 6 As shown in (b) in the figure, the bottom of segmented image 1 is updated to the bottom of the entity area whose entity type is chat content (c), and correspondingly, the top of segmented image 2 is updated to the updated bottom of segmented image 1. Subsequently, the region type detection is performed on segmented image 2 by the target detection technology to obtain the detection result of segmented image 2. Figure 6As shown in (b) in FIG. 1 , when the entity type of the last entity area is the second entity type, i.e., "other content (other)", and the distance between the position coordinates of the last entity area and the bottom position of the segmented image is within a preset range, the entity area of ​​the second entity type (i.e., "other content (other)") or the third entity type (i.e., "chat content (content)") other than the last entity area is searched from the back to the front in the entity array, and the bottom position of the segmented image 2 is updated based on the position coordinates of the found entity area. In this way, Figure 6 As shown in (b), when the entity type of the last entity area in the entity array is other content (o), entity areas whose entity type is other content (o) or chat content (c) in the entity array are selected from back to front to update the bottom of the segmented image 2.

[0085] like Figure 6 As shown in (c) in FIG. 1 , the bottom of segmented image 2 is updated to the bottom of the entity region whose entity type is other content (o), and correspondingly, the top of segmented image 3 is updated to the updated bottom of segmented image 2. Subsequently, the region type detection is performed on segmented image 3 by using the object detection technology to obtain the detection result of segmented image 3.

[0086] Optionally, when the entity type of the last entity area is the third entity type, i.e., "chat content", and the distance between the position coordinates of the last entity area and the bottom position of the segmented image is within a preset range, the entity area of ​​the second entity type or the third entity type other than the last entity is searched from the back to the front in the entity array, and the bottom position of the segmented image is updated based on the position coordinates of the found entity area. Further, when the entity area of ​​the second entity type or the third entity type is not found in the entity array, the bottom position of the segmented image is kept unchanged.

[0087] Optionally, when the distance between the position coordinates of the last entity area and the bottom position of the segmented image exceeds a preset range, the bottom position of the segmented image is kept unchanged. At this time, the position coordinates of the last entity area are at a certain distance from the bottom of the segmented image, and the entity area will not be incomplete or truncated.

[0088] In this way, by dynamically adjusting the bottom positions of multiple segmented images, the entity area of ​​each segmented image input to the OD module is achieved to be complete or non-truncated.

[0089] Next, the local position coordinates of each entity area of ​​each segmented image in the current image are converted to the global position coordinates in the initial image; and based on the global position coordinates of each entity area of ​​each segmented image in the initial image, each entity area and its entity type of all segmented images are reflected in the initial image to obtain the detection domain result of the initial image. In this way, the detection results of all segmented images are integrated into the initial image, and the detection results of the initial image are obtained.

[0090] According to an embodiment of the present disclosure, when the entity types of adjacent entity areas in the initial image are both the third entity type, and the distance between adjacent entity areas is less than or equal to the second preset threshold, the adjacent entity areas are merged. Specifically, when the entity types of two adjacent entity areas in the initial image are both the third entity type, that is, both are chat content, and the distance between the two entity areas is less than or equal to the second preset threshold, the two adjacent entity areas can be merged into one entity area. In this way, it can be determined that the two adjacent entity areas belong to the same entity area and are the same chat content, thereby realizing the merging of entity areas across pages.

[0091] According to the embodiments of the present disclosure, by using the OD module to dynamically adjust the bottom position of the segmented image, it is possible to avoid truncation or incompleteness of the privacy information of the initial image, and the processing speed is fast, thereby improving the efficiency of identifying the privacy information of the image, and improving the accuracy and completeness of identifying the privacy information of the image.

[0092] Return to reference Figure 2 In step S203, optical character recognition technology (OCR) and natural language processing technology (NLP) can be used to determine the target area to be blocked according to the detection result of the initial image.

[0093] Figure 7 is a diagram showing the overall design of an NLP module that performs NLP technology according to an embodiment of the present disclosure. Figure 7 ,The NLP module can include three submodules, namely "rule submodule", "dictionary submodule" and "deep learning submodule". After the text information is input into the NLP module, the NLP module can perform named entity recognition on the input text information.

[0094] More specifically, in step S203, when the entity type of the entity area of ​​the initial image is the first entity type, the entity area is determined as the target area to be occluded; when the entity type of the entity area of ​​the initial image is the second entity type or the third entity type, the entity area is captured as a temporary image; optical character recognition technology is used to identify text information in the temporary image; and natural language processing technology is used to determine the target area to be occluded in the temporary image from the text information.

[0095] In this way, when the entity type of the entity area is any one of the first entity type, i.e., "chat title (title)", "account nickname (username)", and "account avatar (avatar)", the entity area is identified as the area to be blocked. When the entity type of the entity area is the second entity type, i.e., "other content (other)" or the third entity type "chat content (content)", the corresponding entity area is intercepted to obtain a temporary image, and the text information in the temporary image is recognized by optical character recognition technology, and the recognized text information is spliced ​​when necessary, and then the target area to be blocked in the text information is determined by natural language processing technology.

[0096] Figure 8 is a diagram showing an example of multi-line text processing in an entity area according to an embodiment of the present disclosure. Figure 8 , the characters of each line can be split and combined to determine the range of each target area group and whether it should be blocked. Specifically, the text in the target area can be split, truncated and recombined to form different target area groups based on the OCR results (including the line number and the characters of each line) and the NLP results. Figure 8 Among them, area group 1 is the account avatar, which should be blocked, area groups 2, 4 and 5 are chat content that does not include private information and does not need to be blocked, and area group 2 is chat content that includes private information and should be blocked.

[0097] Return to reference Figure 2 In step S204, the target area is blocked to obtain a target image. For example, the "blocking" method can be Gaussian blur processing or coding using color blocks, but the present disclosure is not limited thereto. Figure 8 The situation after occlusion is shown.

[0098] According to an embodiment of the present disclosure, an artificial intelligence model can be used to perform the image processing method as described above. For example, the above-mentioned OD module, OCR module, and NLP module can all be implemented by an artificial intelligence model, but the present disclosure is not limited thereto. For example, specifically, the initial image can be used as the input of the artificial intelligence model, and the artificial intelligence model can be used to perform the determination of long screenshots, the division of images, the detection of area types, the integration of detection results, optical character recognition, natural language processing, the determination of target areas, and the occlusion of target areas, thereby obtaining the output of the artificial intelligence model as the target image.

[0099] The artificial intelligence model can be obtained through training. Here, "obtained through training" means training a basic artificial intelligence model with multiple training data through a training algorithm to obtain a predefined operating rule or artificial intelligence model, which is configured to perform the desired features (or purposes).

[0100] As an example, the artificial intelligence model may include multiple neural network layers. Each of the multiple neural network layers includes multiple weight values, and the neural network calculation is performed by calculating between the calculation result of the previous layer and the multiple weight values.

[0101] Fig. 9 2 is a diagram showing an example of an implementation scheme of a picture processing method according to an embodiment of the present disclosure. Fig. 9 The input is a long screenshot. The OD module, OCR module and NLP module perform region type detection, optical character recognition and natural language processing on the output long screenshot, and output the target area to be occluded in the long screenshot.

[0102] Fig.10 is a flowchart illustrating an example of identifying and shielding private information by inserting an intermediate picture according to an embodiment of the present disclosure.

[0103] Reference Fig.10 In step S1001, a long screenshot operation is performed on the chat interface of the instant messaging application to obtain a chat interface image.

[0104] In step S1002, the long screenshot is divided into multiple pictures. The division method may be to divide the long screenshot according to the screen length of the mobile terminal, or to divide the long screenshot into multiple pictures evenly according to the length of the long screenshot. Each divided picture is numbered.

[0105] In step S1003, the middle area of ​​two adjacent pictures is selected to obtain the middle picture. Further, the label of the middle picture is determined according to the average value of the labels of the two adjacent pictures.

[0106] In step S1004, the OD module is used to perform region type detection on all images to obtain detection results of all images. The detection results of all images are integrated into a long screenshot. The detection results include entity regions, entity types of entity regions, and location coordinates of entity regions. The entity types of entity regions include first entity types: title, username, and avatar, second entity types: other, and third entity types: content.

[0107] In step S1005, the detection results of all pictures are integrated into the long screenshot to obtain the detection results of the long screenshot. Specifically, the position coordinates of the entity areas in all pictures are converted from the local coordinates of the current picture to the global coordinates of the long screenshot. The detection results of the long screenshot are determined according to the global coordinates of all pictures.

[0108] In step S1006 , a large entity area that can cover the overlapped entity area in the detection result of the long screenshot is selected to replace the overlapped entity area.

[0109] In step S1007, for the entity type in the detection result of the long screenshot being the first entity type, the first type area: title, username and avatar may be directly determined as the privacy area that needs to be blocked.

[0110] In step S1008 , for the entity type in the detection result of the long screenshot being the second entity type or the third entity type, an image of each entity area is captured.

[0111] In step S1009, the OCR module is used to perform character recognition on each captured image to obtain a character region, wherein the character region may contain at least one character, each character may correspond to a respective line number, and the “line number” is used to indicate which line the corresponding character is located in the character region.

[0112] In step S1010, the NLP module is used to perform entity recognition on the character area to determine the privacy area that needs to be blocked.

[0113] Fig.11 is a flowchart showing an example of identifying and shielding private information by dynamically adjusting the bottom of a picture according to an embodiment of the present disclosure.

[0114] Reference Fig.11 In step S1101, a long screenshot operation is performed on the chat interface of the instant messaging application to obtain a chat interface image.

[0115] In step S1102, the long screenshot is divided into multiple pictures. The division method may be to divide the long screenshot according to the screen length of the mobile terminal, or to divide the long screenshot into multiple pictures evenly according to the length of the long screenshot. Each divided picture is numbered.

[0116] In step S1103, the image is input into the OD module in sequence according to the label to perform region type detection, and the detection result of the image is obtained. The detection result includes the entity region, the entity type of the entity region, and the location coordinates of the entity region. The entity type of the entity region includes the first entity type: title, username, and avatar, the second entity type: other, and the third entity type: content.

[0117] In step S1104, the position coordinates of the entity areas of the picture are sorted from top to bottom to obtain an entity array, and the entity type of the last entity area in the entity array is determined.

[0118] In step S1105, when the entity type of the last entity area is the first entity type and its position coordinates are close to the bottom of the picture, the entity array is searched from back to front for an entity area whose entity type is the second entity type or the third entity type, and the bottom position coordinates of the entity area are used as the new bottom of the picture; when the entity type of the last entity area is the second entity type and its position coordinates are close to the bottom of the picture, the entity array is searched from back to front for an entity area whose entity type is the second entity type or the third entity type other than the last entity area, and the bottom position coordinates of the entity area are used as the new bottom of the picture; when the entity type of the last entity area is the third entity type and its position coordinates are close to the bottom of the picture, the entity array is searched from back to front for an entity area whose entity type is the second entity type or the third entity type other than the last entity area, and the bottom position coordinates of the entity area are used as the new bottom of the picture; when the entity area of ​​the second entity type or the third entity type is not found in the entity array, the bottom position of the picture is kept unchanged; when the position coordinates of the last entity area are far away from the bottom position of the picture, the bottom position of the picture is kept unchanged.

[0119] In step S1106, the detection results of all pictures are integrated into the long screenshot to obtain the detection results of the long screenshot. Specifically, the position coordinates of the entity areas in all pictures are converted from the local coordinates of the current picture to the global coordinates of the long screenshot. The detection results of the long screenshot are determined according to the global coordinates of all pictures.

[0120] In step S1107, when the distance between two entity regions of the third entity type in the detection result of the long screenshot is less than a certain threshold, the two entity regions are merged.

[0121] In step S1108, for the entity type in the detection result of the long screenshot being the first entity type, the first type area: title, username and avatar may be directly determined as the privacy area that needs to be blocked.

[0122] In step S1109, for the entity type in the detection result of the long screenshot being the second entity type or the third entity type, an image of each entity area is captured.

[0123] In step S1110, the OCR module is used to perform character recognition on each captured image to obtain a character region, wherein the character region may contain at least one character, each character may correspond to a respective line number, and the “line number” is used to indicate which line the corresponding character is located in the character region.

[0124] In step S1111, the NLP module is used to perform entity recognition on the character area to determine the privacy area that needs to be blocked.

[0125] Fig.12 is a block diagram illustrating a picture processing apparatus according to an embodiment of the present disclosure.

[0126] Reference Fig.12 The device 1200 may include a picture acquisition unit 1201, a picture division unit 1202, a target area determination unit 1203 and an occlusion unit 1204.

[0127] The image acquisition unit 1201 can acquire the initial image; the image division unit 1202 can divide the initial image into multiple segmented images in response to determining that the initial image is a long screenshot, use target detection technology to perform region type detection on each segmented image, and integrate the detection results of each segmented image to obtain the detection result of the initial image; the target area determination unit 1203 can use optical character recognition technology and natural language processing technology to determine the target area to be occluded according to the detection result of the initial image; the occlusion unit 1204 can occlude the target area to obtain the target image.

[0128] According to an embodiment of the present disclosure, the picture acquisition unit 1201 may determine that the initial picture is a long screenshot in response to the first ratio being greater than or equal to a first preset threshold, wherein the first ratio may be determined based on the aspect ratio of the initial picture and a preset aspect ratio.

[0129] According to an embodiment of the present disclosure, the picture dividing unit 1202 may divide the initial picture into a plurality of divided pictures according to a preset length, or evenly divide the initial picture into a plurality of divided pictures based on the length of the initial picture.

[0130] On the one hand, the picture division unit 1202 can obtain an intermediate picture for each segmented picture based on the intermediate area between the segmented picture and the next adjacent segmented picture; use the target detection technology to perform region type detection on each segmented picture and each intermediate picture to obtain the detection results of each segmented picture and each intermediate picture, wherein the detection results include the entity area, the entity type of the entity area, and the location coordinates of the entity area; integrate the detection results of all segmented pictures and the detection results of all intermediate pictures into the initial picture to obtain the detection results of the initial picture.

[0131] More specifically, the picture division unit 1202 can convert the local position coordinates of each entity area of ​​each segmented picture in the current picture into the global position coordinates in the initial picture; convert the local position coordinates of each entity area of ​​each intermediate picture in the current picture into the global position coordinates in the initial picture; based on the global position coordinates of each entity area of ​​each segmented picture and each intermediate picture in the initial picture, each entity area and its entity type of all segmented pictures and all intermediate pictures are reflected in the initial picture to obtain the detection result of the initial picture.

[0132] Optionally, when there is an overlapping portion between two physical regions, the picture dividing unit 1202 may use a second physical region covering the two physical regions to replace the two physical regions.

[0133] On the other hand, the image division unit 1202 can perform the following steps for each segmented image: use the target detection technology to detect the region type of the segmented image to obtain the detection result of the segmented image, and the detection result includes the entity region, the entity type of the entity region and the position coordinates of the entity region; sort the entity regions from top to bottom based on the position coordinates of the entity regions in the segmented image to establish an entity array; determine the entity type of the last entity region in the entity array; update the bottom position of the segmented image according to the entity type and position coordinates of the last entity region, and use the updated bottom position of the segmented image as the top position of the next segmented image; integrate the detection results of all segmented images into the initial image to obtain the detection result of the initial image.

[0134] More specifically, when the entity type of the last entity area is the first entity type, and the distance between the position coordinates of the last entity area and the bottom position of the segmented picture is within a preset range, the picture division unit 1202 may search for entity areas of the second entity type or the third entity type from back to front in the entity array, and update the bottom position of the segmented picture based on the position coordinates of the found entity areas; when the entity type of the last entity area is the second entity type, and the distance between the position coordinates of the last entity area and the bottom position of the segmented picture is within a preset range, the picture division unit 1202 may search for entity areas of the second entity type or the third entity type other than the last entity area from back to front in the entity array, and update the bottom position of the segmented picture based on the position coordinates of the found entity areas. The bottom position of the segmented picture is updated according to the mark; when the entity type of the last entity area is the third entity type, and the distance between the position coordinate of the last entity area and the bottom position of the segmented picture is within a preset range, the picture division unit 1202 can search for entity areas of the second entity type or the third entity type other than the last entity from the back to the front in the entity array, and update the bottom position of the segmented picture based on the position coordinates of the found entity areas, wherein when the entity area of ​​the second entity type or the third entity type is not found in the entity array, the bottom position of the segmented picture is kept unchanged; when the distance between the position coordinate of the last entity area and the bottom position of the segmented picture exceeds the preset range, the picture division unit 1202 can keep the bottom position of the segmented picture unchanged. The picture division unit 1202 can convert the local position coordinates of each entity area of ​​each segmented picture in the current picture into the global position coordinates in the initial picture; based on the global position coordinates of each entity area of ​​each segmented picture in the initial picture, each entity area and its entity type of all segmented pictures are reflected in the initial picture to obtain the detection result of the initial picture.

[0135] According to an embodiment of the present disclosure, the image processing device 1200 further includes an entity region merging unit (not shown). When the entity types of adjacent entity regions in the initial image are all the third entity type, and the distance between the adjacent entity regions is less than or equal to the second preset threshold, the entity region merging unit may merge the adjacent entity regions.

[0136] According to an embodiment of the present disclosure, when the entity type of the entity area of ​​the initial image is the first entity type, the target area determination unit 1203 may determine the entity area as the target area to be occluded; when the entity type of the entity area of ​​the initial image is the second entity type or the third entity type, the target area determination unit 1203 may capture the entity area as a temporary image, use optical character recognition technology to recognize text information in the temporary image, and use natural language processing technology to determine the target area to be occluded in the temporary image from the text information. Fig.13is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0137] Reference Fig.13 The electronic device 1300 includes at least one memory 1301 and at least one processor 1302. The at least one memory 1301 stores instructions. When the instructions are executed by the at least one processor 1302, the image processing method according to the embodiment of the present disclosure is executed.

[0138] As an example, the electronic device 1300 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instructions. Here, the electronic device 1300 is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above instructions (or instruction sets) individually or in combination. The electronic device 1300 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0139] In the electronic device 1300, the processor 1302 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0140] The processor 1302 may execute instructions or codes stored in the memory 1301, wherein the memory 1301 may also store data. Instructions and data may also be sent and received over a network via a network interface device, wherein the network interface device may employ any known transmission protocol.

[0141] The memory 1301 may be integrated with the processor 1302, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 1301 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 1301 and the processor 1302 may be operationally coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 1302 can read files stored in the memory.

[0142] In addition, the electronic device 1300 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 1300 may be connected to each other via a bus and / or a network.

[0143] The image processing method according to the embodiment of the present disclosure may be written as a computer program and stored on a computer-readable storage medium. When the computer program is executed by a processor, the image processing method as described above may be implemented. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, nonvolatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer program and any associated data, data files and data structures in a non-transitory manner and provide computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. In one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0144] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0145] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for processing an image, characterized in that: The image processing method comprises: Get the initial image; In response to determining that the initial image is a long screenshot, dividing the initial image into a plurality of segmented images, performing region type detection on each segmented image using an object detection technology, and integrating the detection results of each segmented image to obtain a detection result of the initial image; Using optical character recognition technology and natural language processing technology, determining the target area to be blocked according to the detection result of the initial image; The target area is blocked to obtain a target image.

2. The image processing method according to claim 1, characterized in that: The determining that the initial image is a long screenshot includes: In response to the first ratio being greater than or equal to a first preset threshold, the initial image is determined to be the long screenshot, wherein the first ratio is determined based on an aspect ratio of the initial image and a preset aspect ratio.

3. The image processing method according to claim 1, characterized in that: The step of dividing the initial picture into a plurality of segmented pictures comprises: The initial picture is divided into a plurality of segmented pictures according to a preset length, or the initial picture is evenly divided into a plurality of segmented pictures based on the length of the initial picture.

4. The image processing method according to any one of claims 1 to 3, characterized in that: The target detection technology is used to detect the region type of each segmented image, and the detection results of each segmented image are integrated to obtain the detection results of the initial image, including: For each segmented picture, an intermediate picture is obtained based on the intermediate area between the segmented picture and the next adjacent segmented picture; Performing region type detection on each segmented image and each intermediate image using target detection technology to obtain detection results of each segmented image and each intermediate image, wherein the detection results include entity regions, entity types of entity regions, and location coordinates of entity regions; The detection results of all segmented pictures and the detection results of all intermediate pictures are integrated into the initial picture to obtain the detection result of the initial picture.

5. The image processing method according to claim 4, characterized in that: The step of integrating the detection results of all segmented images and the detection results of all intermediate images into the initial image to obtain the detection result of the initial image includes: Convert the local position coordinates of each entity area of ​​each segmented image in the current image to the global position coordinates in the initial image; Convert the local position coordinates of each entity area of ​​each intermediate image in the current image to the global position coordinates in the initial image; Based on the global position coordinates of each entity region of each segmented image and each intermediate image in the initial image, each entity region and its entity type of all segmented images and all intermediate images are reflected in the initial image to obtain the detection result of the initial image.

6. The image processing method according to claim 5, characterized in that: The step of reflecting each entity area of ​​all segmented pictures and all intermediate pictures into the initial picture comprises: When there is an overlapped portion between two physical regions, the two physical regions are replaced by a second physical region covering the two physical regions.

7. The image processing method according to any one of claims 1 to 3, characterized in that: The target detection technology is used to detect the region type of each segmented image, and the detection results of each segmented image are integrated to obtain the detection results of the initial image, including: For each segmented image, perform the following steps: Performing region type detection on the segmented image using target detection technology to obtain a detection result of the segmented image, wherein the detection result includes an entity region, an entity type of the entity region, and a position coordinate of the entity region; Sort the entity regions from top to bottom based on the position coordinates of the entity regions in the segmented picture to establish an entity array; Determining the entity type of the last entity region in the entity array; updating the bottom position of the segmented picture according to the entity type and position coordinates of the last entity area, and using the updated bottom position of the segmented picture as the top position of the next segmented picture; The detection results of all segmented images are integrated into the initial image to obtain the detection result of the initial image.

8. The image processing method according to claim 7, characterized in that: The updating of the bottom position of the segmented image according to the entity type and position coordinates of the last entity area includes: When the entity type of the last entity area is the first entity type, and the distance between the position coordinates of the last entity area and the bottom position of the segmented picture is within a preset range, searching for entity areas of the second entity type or the third entity type from the back to the front in the entity array, and updating the bottom position of the segmented picture based on the position coordinates of the found entity areas; When the entity type of the last entity area is the second entity type, and the distance between the position coordinates of the last entity area and the bottom position of the segmented picture is within the preset range, searching the entity array from back to front for entity areas of the second entity type or the third entity type other than the last entity area, and updating the bottom position of the segmented picture based on the position coordinates of the found entity areas; When the entity type of the last entity area is the third entity type, and the distance between the position coordinates of the last entity area and the bottom position of the segmented picture is within the preset range, searching the entity array from back to front for an entity area of ​​the second entity type or the third entity type other than the last entity, and updating the bottom position of the segmented picture based on the position coordinates of the found entity area, wherein when the entity area of ​​the second entity type or the third entity type is not found in the entity array, the bottom position of the segmented picture is kept unchanged; When the distance between the position coordinates of the last entity area and the bottom position of the segmented image exceeds the preset range, the bottom position of the segmented image is kept unchanged.

9. The image processing method according to claim 7, characterized in that: The step of integrating the detection results of all segmented images into the initial image to obtain the detection result of the initial image includes: Convert the local position coordinates of each entity area of ​​each segmented image in the current image to the global position coordinates in the initial image; Based on the global position coordinates of each entity region of each segmented image in the initial image, each entity region and its entity type of all segmented images are reflected in the initial image to obtain the detection result of the initial image.

10. The image processing method according to claim 9, characterized in that: The processing method also includes: When the entity types of adjacent entity regions in the initial image are all the third entity type, and the distance between adjacent entity regions is less than or equal to a second preset threshold, the adjacent entity regions are merged.

11. The image processing method according to claim 4, characterized in that: The method of using the optical character recognition technology and the natural language processing technology to determine the target area to be blocked according to the detection result of the initial image includes: When the entity type of the entity area of ​​the initial image is the first entity type, determining the entity area as a target area to be blocked; When the entity type of the entity area of ​​the initial image is the second entity type or the third entity type, intercepting the entity area as a temporary image; Recognize text information in the temporary image using optical character recognition technology; Natural language processing technology is used to determine the target area to be blocked in the temporary image from the text information.

12. The image processing method according to claim 7, characterized in that: The method of using the optical character recognition technology and the natural language processing technology to determine the target area to be blocked according to the detection result of the initial image includes: When the entity type of the entity area of ​​the initial image is the first entity type, determining the entity area as a target area to be blocked; When the entity type of the entity area of ​​the initial image is the second entity type or the third entity type, intercepting the entity area as a temporary image; Recognize text information in the temporary image using optical character recognition technology; Natural language processing technology is used to determine the target area to be blocked in the temporary image from the text information.

13. A picture processing device, characterized in that: The image processing device comprises: A picture acquisition unit, configured to acquire an initial picture; a picture division unit, configured to, in response to determining that the initial picture is a long screenshot, divide the initial picture into a plurality of segmented pictures, perform region type detection on each segmented picture using a target detection technology, and integrate the detection results of each segmented picture to obtain a detection result of the initial picture; a target area determination unit configured to determine the target area to be blocked according to the detection result of the initial image by using optical character recognition technology and natural language processing technology; The blocking unit is configured to block the target area to obtain a target image.

14. An electronic device, characterized in that: The electronic device includes a processor and a memory, the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the image processing method as described in any one of claims 1 to 12.

15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 12 is implemented.

Images