Image region division method and device, equipment and medium

Abstract

The embodiment of the invention discloses an image region division method and device, equipment and a medium, and relates to the technical field of image processing. The method comprises the following steps: performing object detection on a specified image to obtain a detection frame area of each to-be-detected object in the specified image; the specified image comprises at least one to-be-detected object; performing image segmentation on the specified image to obtain an initial mask region of each to-be-detected object in the specified image; and based on the detection frame area of each to-be-detected object, expanding the initial mask area of each to-be-detected object to obtain a target mask area of each to-be-detected object. According to the embodiment of the invention, the mask division of the to-be-detected object in the specified image can be optimized, so that the target mask area of each to-be-detected object is reasonably expanded. And furthermore, based on the optimized target mask region, the consistency of the generated image and the role information can be improved when the multi-role image is generated.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, device, equipment and medium for dividing an image region. Background Art

[0002] In computer image processing, mask is a technique used to select, filter or hide specific areas in an image. In the image processing process, a specific image, graphic or object can be used to cover or block all or part of the image to be processed to control the image processing area or process. Among them, the specific image or object used to cover and block is called a mask or template.

[0003] For example, given a layout diagram containing a person, the purpose of image processing is to combine the person information in the layout diagram with the specified person information to generate an image containing the specified person information and the person information in the layout diagram. During the image processing, the layout diagram can be subjected to human body detection to obtain the rectangular area of ​​each human body in the layout diagram. However, using the rectangular area to replace the person and generate a new image will result in crosstalk of the person information in the overlapping area. Especially when multiple person images are involved, the image processing effect is not ideal. Summary of the invention

[0004] In view of the above problems in the prior art, the embodiments of the present disclosure provide a method, device, equipment and medium for dividing an image region. The embodiments of the present disclosure re-divide the region of each object to be detected in a specified image through object detection and image segmentation, and optimize the mask division of the object to be detected in the specified image, so that the target mask region of each object to be detected is reasonably expanded on the basis of the initial mask region, and then based on the optimized target mask region, when generating an image containing multiple roles, the consistency of the generated image and the role information can be improved.

[0005] To achieve the above objective, the present disclosure provides, in a first aspect, a method for dividing an image region, comprising:

[0006] Performing object detection on a specified image to obtain a detection frame area of ​​each object to be detected in the specified image; wherein the specified image includes at least one object to be detected;

[0007] Perform image segmentation on the specified image to obtain an initial mask area for each object to be detected in the specified image;

[0008] Based on the detection frame area of ​​each object to be detected, the initial mask area of ​​each object to be detected is expanded respectively to obtain the target mask area of ​​each object to be detected.

[0009] As a possible implementation of the first aspect, based on the detection frame area of ​​each object to be detected, the initial mask area of ​​each object to be detected is expanded respectively to obtain a target mask area of ​​each object to be detected, including:

[0010] For each pixel point of the detection frame area of ​​at least one object to be detected, determining the distance between the pixel point and the object identifier of each object to be detected;

[0011] According to the distance, the initial mask area of ​​the object to be detected to which the pixel point corresponding to the distance belongs is determined, so as to expand the initial mask area of ​​each object to be detected to obtain the target mask area of ​​each object to be detected.

[0012] As a possible implementation manner of the first aspect, for each pixel point of a detection frame area of ​​at least one object to be detected, determining a distance between the pixel point and an object identifier of each object to be detected includes:

[0013] For each object to be detected, marking the pixel points within the initial mask area of ​​the object to be detected as the object identifier of the object to be detected;

[0014] For each pixel point of the detection frame area of ​​at least one object to be detected, a distance between the pixel point and an object identifier of each object to be detected is determined.

[0015] As a possible implementation manner of the first aspect, determining, according to the distance, an initial mask area of ​​the object to be detected to which a pixel point corresponding to the distance belongs includes:

[0016] For each pixel point of the detection frame area of ​​at least one object to be detected, determine the object identifier closest to the pixel point, and determine the initial mask area of ​​the object to be detected corresponding to the nearest object identifier, which is the initial mask area of ​​the object to be detected to which the pixel point belongs.

[0017] As a possible implementation manner of the first aspect, for each object to be detected, marking a pixel point in an initial mask area of ​​the object to be detected as an object identifier of the object to be detected includes:

[0018] Sequentially mark the pixel points in the initial mask area of ​​at least one object to be detected as the object identifier of the object to be detected corresponding to it; wherein,

[0019] In response to the overlap of the initial mask area of ​​at least one object to be detected, the object identifier marked later covers the object identifier marked earlier.

[0020] As a possible implementation of the first aspect, based on the detection frame area of ​​each object to be detected, the initial mask area of ​​each object to be detected is expanded respectively to obtain a target mask area of ​​each object to be detected, including:

[0021] In response to the overlap of the detection frame area of ​​at least one object to be detected, determining a union of the detection frame area of ​​at least one object to be detected as the area to be marked;

[0022] Based on the area to be marked, the initial mask area of ​​each object to be detected is expanded respectively to obtain the target mask area of ​​each object to be detected.

[0023] As a possible implementation of the first aspect, based on the detection frame area of ​​each object to be detected, the initial mask area of ​​each object to be detected is expanded respectively to obtain a target mask area of ​​each object to be detected, including:

[0024] Constructing a label map according to the specified image and the detection frame area and the initial mask area of ​​each object to be detected, wherein the size of the label map is consistent with the size of the specified image, and the label map includes the detection frame area and the initial mask area of ​​each object to be detected;

[0025] Based on the detection frame area of ​​each object to be detected in the label image, the initial mask area of ​​the object to be detected in the label image is expanded respectively to obtain the target mask area of ​​each object to be detected.

[0026] As a possible implementation manner of the first aspect, a pixel value of a pixel point in the marking image is a first preset value.

[0027] A second aspect of the present disclosure provides a device for dividing an image region, comprising:

[0028] A detection unit, configured to perform object detection on a specified image to obtain a detection frame area of ​​each object to be detected in the specified image; wherein the specified image includes at least one object to be detected;

[0029] A segmentation unit, used to perform image segmentation on a specified image to obtain an initial mask region of each object to be detected in the specified image;

[0030] The expansion unit is used to expand the initial mask area of ​​each object to be detected based on the detection frame area of ​​each object to be detected, so as to obtain the target mask area of ​​each object to be detected.

[0031] As a possible implementation manner of the second aspect, the expansion unit includes:

[0032] A first determination subunit is used to determine, for each pixel point of a detection frame area of ​​at least one object to be detected, a distance between the pixel point and an object identifier of each object to be detected;

[0033] The second determining subunit is used to determine, according to the distance, the initial mask area of ​​the object to be detected to which the pixel point corresponding to the distance belongs, so as to expand the initial mask area of ​​each object to be detected to obtain the target mask area of ​​each object to be detected.

[0034] As a possible implementation manner of the second aspect, the first determining subunit is configured to:

[0035] For each object to be detected, marking the pixel points within the initial mask area of ​​the object to be detected as the object identifier of the object to be detected;

[0036] For each pixel point of the detection frame area of ​​at least one object to be detected, a distance between the pixel point and an object identifier of each object to be detected is determined.

[0037] As a possible implementation manner of the second aspect, the second determining subunit is used to:

[0038] For each pixel point of the detection frame area of ​​at least one object to be detected, determine the object identifier closest to the pixel point, and determine the initial mask area of ​​the object to be detected corresponding to the nearest object identifier, which is the initial mask area of ​​the object to be detected to which the pixel point belongs.

[0039] As a possible implementation manner of the second aspect, the first determining subunit is configured to:

[0040] Sequentially mark the pixel points in the initial mask area of ​​at least one object to be detected as the object identifier of the object to be detected corresponding to it; wherein,

[0041] In response to the overlap of the initial mask area of ​​at least one object to be detected, the object identifier marked later covers the object identifier marked earlier.

[0042] As a possible implementation manner of the second aspect, the expansion unit is used to:

[0043] In response to the overlap of the detection frame area of ​​at least one object to be detected, determining a union of the detection frame area of ​​at least one object to be detected as the area to be marked;

[0044] Based on the area to be marked, the initial mask area of ​​each object to be detected is expanded respectively to obtain the target mask area of ​​each object to be detected.

[0045] As a possible implementation manner of the second aspect, the expansion unit is used to:

[0046] Constructing a label map according to the specified image and the detection frame area and the initial mask area of ​​each object to be detected, wherein the size of the label map is consistent with the size of the specified image, and the label map includes the detection frame area and the initial mask area of ​​each object to be detected;

[0047] Based on the detection frame area of ​​each object to be detected in the label image, the initial mask area of ​​the object to be detected in the label image is expanded respectively to obtain the target mask area of ​​each object to be detected.

[0048] As a possible implementation manner of the second aspect, a pixel value of a pixel point in the marking image is a first preset value.

[0049] A third aspect of the present disclosure provides an electronic device, including:

[0050] a memory for storing a computer program product;

[0051] The processor is used to execute the computer program product stored in the memory, and when the computer program product is executed, the method described in any one of the first aspects above is implemented.

[0052] A fourth aspect of the present disclosure provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the method described in any one of the first aspects above is implemented.

[0053] The fifth aspect of the present disclosure provides a computer program product, including computer program instructions, which implement any of the methods described in the first aspect when executed by a processor. The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0055] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0056] Figure 1 A flowchart of an embodiment of the method for dividing an image area disclosed in the present invention;

[0057] Figure 2 A schematic diagram of a human body segmentation mask according to an embodiment of the image region division method disclosed herein;

[0058] Figure 3 A schematic diagram of mask division of an embodiment of the image area division method disclosed in the present invention;

[0059] Figure 4A flowchart of an embodiment of the method for dividing an image area disclosed in the present invention;

[0060] Figure 5 A flowchart of an embodiment of the method for dividing an image area disclosed in the present invention;

[0061] Figure 6 A flowchart of an embodiment of the method for dividing an image area disclosed in the present invention;

[0062] Figure 7 A flowchart of an embodiment of the method for dividing an image area disclosed in the present invention;

[0063] Figure 8 A flowchart of an embodiment of the method for dividing an image area disclosed in the present invention;

[0064] Fig. 9 A flowchart of an embodiment of the method for dividing an image area disclosed in the present invention;

[0065] Fig.10 A flowchart of an embodiment of the method for dividing an image area disclosed in the present invention;

[0066] Fig.11 A schematic diagram of the generation effect of an embodiment of the image area division method disclosed in the present invention;

[0067] Fig.12 It is a structural schematic diagram of an embodiment of the image area division device disclosed in the present invention;

[0068] Fig.13 It is a structural schematic diagram of an embodiment of the image area division device disclosed in the present invention;

[0069] Fig.14 A block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0070] Although the embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0071] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0072] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0073] The following first introduces the existing method, and then introduces the technical solution of the present invention in detail.

[0074] In an example of image processing, given an arbitrary image containing a person, for example, the image is a layout diagram for guiding the action posture of the person. The purpose of image processing is to combine the person information in the layout diagram with the specified person information to generate an image containing the specified person information and the person information in the layout diagram. In the process of generating images containing the specified person information multiple times, in order to ensure that the image of the same person is consistent and the effect is realistic, it is necessary to accurately identify the area in the layout diagram corresponding to the person information.

[0075] In one solution of the prior art, a rectangular human body detection frame is used as a mask area for each character. Specifically, a human body detection is performed on a given layout diagram using an image object detection algorithm to obtain a rectangular human body detection frame for each character in the layout diagram, and the obtained rectangular human body detection frame is used as a mask area for each character.

[0076] In the above scheme, if the layout diagram to be processed involves multiple person images, the interference between the multiple person information is small when the rectangular detection frames of multiple persons do not overlap or have very small overlaps. However, if there is a large overlap in the rectangular area, then the overlapping area belongs to multiple persons at the same time, so the person information in the overlapping area will interfere with each other, resulting in image distortion, which seriously affects the consistency of the person in the generated image.

[0077] In another solution of the prior art, the characters in the layout diagram are segmented using an image segmentation algorithm to obtain a human body segmentation mask for each character in the layout diagram, and the obtained human body segmentation mask is used as the mask area for each character. With this solution, even if two characters in the layout diagram are very close, they will not overlap, thereby avoiding crosstalk of character information and improving the character consistency of the generated image to a certain extent.

[0078] However, the mask area obtained in the above scheme is strictly limited to the character area in the layout diagram. If the character in the layout diagram and the character in the specified role information are very different in shape, clothing, hairstyle, etc., the generated image will keep the outline of the character in the layout diagram. In this case, the layout diagram has too many restrictions on the generated image, and the character image cannot be changed according to the role needs in the generated image, so that the character in the generated image does not conform to the character image in the specified role information, thereby reducing the character consistency of the image.

[0079] In summary, the prior art has the following defects: when multiple character images are involved, it causes the problem of character information crosstalk and reduced character consistency.

[0080] Based on the technical problems existing in the above-mentioned prior art, the present disclosure provides a method, device, equipment and medium for dividing an image area. When it comes to multi-character image processing, on the one hand, when there is overlap in the detection frame areas between the characters, there is a problem of mutual crosstalk of the character information in the overlapping area, which in turn affects the character consistency of the generated image. On the other hand, if the segmentation mask is directly used as the target mask area for image replacement, the final image will be strictly restricted by the character features, resulting in the inability to change the hairstyle, body shape, etc., which will also lead to a decrease in the consistency between the generated image and the character information. In summary, how to reasonably divide the image mask area is crucial to the character consistency effect of the generated image.

[0081] In view of this, the embodiment of the present disclosure re-divides the region of each object to be detected in the specified image through object detection and image segmentation, and optimizes the mask division of the object to be detected in the specified image, so that the target mask area of ​​each object to be detected is reasonably expanded on the basis of the initial mask area. Based on the optimized target mask area, when generating an image containing multiple roles, the consistency between the generated image and the role information can be improved, thereby solving the technical problems of role information crosstalk and reduced role consistency mentioned in the prior art.

[0082] Figure 1 FIG. 1 is a flow chart of an embodiment of the method for dividing an image region disclosed in the present invention. Figure 1 As shown, it may specifically include:

[0083] Step S110, performing object detection on the designated image to obtain a detection frame area of ​​each object to be detected in the designated image, wherein the designated image includes at least one object to be detected, wherein the object to be detected may include a target object such as a human body, an animal, or other objects.

[0084] Generally, a specified image may include multiple characters. The processing tasks for the specified image may include character replacement, information filtering or shielding, feature extraction, etc. of multiple characters. For example, when replacing the character in the specified image with pre-specified character information, a reasonable division of the image mask area for the specified image can make the image processing effect better and the generated image more realistic. Among them, the character information may include the character's figure, body proportions, clothing style, and can indicate the character's appearance characteristics, personality characteristics, etc.

[0085] In step S110, firstly, an image object detection algorithm is used to perform object detection on the specified image. Image object detection refers to the use of theories and methods in fields such as image processing and pattern recognition to locate objects of interest from an image and give a bounding box for each object. The area surrounded by the bounding box is the detection box area. The detection box is a rectangular box used to calibrate the position and size of each object in the image. Image object detection algorithms may include RCNN (Regions with Convolutional Neural Networks), YOLO (You Only Look Once) algorithm, etc.

[0086] In one example, an image object detection algorithm includes the following steps:

[0087] (1) Preprocessing: Perform operations such as image denoising, image enhancement, and color space conversion on the image to be detected.

[0088] (2) Window sliding: Slide a window of fixed size in the image to be detected and use the sub-image in the window as the candidate area.

[0089] (3) Feature extraction: Use a specific algorithm to extract features from the candidate area.

[0090] (4) Feature selection: Select representative features from the feature vector to reduce the dimension of the extracted features.

[0091] (5) Feature classification: Use a specific classifier to classify features and determine whether the candidate area contains the target and its category.

[0092] (6) Post-processing: Merge the intersecting candidate areas that are determined to be of the same category, calculate the bounding box of each target, and complete target detection.

[0093] like Figure 1As shown, step S120, image segmentation is performed on the specified image to obtain an initial mask area of ​​each object to be detected in the specified image. Wherein, the specified image is firstly segmented to obtain a segmentation mask of each object to be detected in the specified image, and then the initial mask area of ​​the object to be detected is determined according to the segmentation mask of the object to be detected.

[0094] Image segmentation is the process of separating the object from the background in an image. The result of segmentation is the boundary contour between the object and the background extracted from the image. The difference between image segmentation and image object detection is that the result of image segmentation is the contour line that fits the boundary of the object. Figure 2 As shown in , the contour line is usually a curve where the boundary between the object and the background is located. The result of image object detection is usually a rectangular box.

[0095] In step S120, the specified image is segmented using an image segmentation algorithm. The image segmentation algorithm may include Mask R-CNN (Mask Region-based Convolutional Neural Network), DeepLab, etc. DeepLab is a series of semantic segmentation algorithms based on convolutional neural networks, which achieves accurate recognition and segmentation of different objects and backgrounds by classifying and labeling images at the pixel level.

[0096] In one example, an image segmentation algorithm includes the following steps:

[0097] (1) Image preprocessing: This step involves grayscale conversion, filtering, binarization and other operations on the input image to improve the accuracy of image segmentation. These operations help reduce image noise, enhance the contrast between the target and the background, and prepare for the subsequent segmentation step.

[0098] (2) Edge detection: Use edge detection algorithms (such as Roberts algorithm, Prewitt algorithm, Sobel algorithm, etc.) to detect the edge of the image and obtain an edge map. These algorithms can identify the edge information in the image and provide a basis for subsequent connected domain analysis.

[0099] (3) Connected domain analysis: Connected domain analysis is performed on the edge map, and the connected domain is used as the region to divide the image. The purpose of this step is to divide the image into several specific regions with unique properties.

[0100] (4) Region merging: Merge the segmented regions to eliminate small regions and improve segmentation accuracy. This step helps to further optimize the segmentation results and ensure that the segmented regions better meet actual needs.

[0101] Through the above steps, the image segmentation algorithm can divide the image into several specific areas with unique properties, thereby extracting the target object of interest.

[0102] See also Figure 2 After performing image segmentation on a specified image, segmentation masks of multiple objects can be obtained. The segmentation mask includes multiple pixel points belonging to the corresponding object in the specified image. The edge of the segmentation mask can be a contour curve that includes the object in the specified image.

[0103] After performing image segmentation on the specified image and obtaining the segmentation mask of each object to be detected, the initial mask area of ​​the object to be detected is determined based on the segmentation mask of the object to be detected. For each object to be detected, the area range of the segmentation mask of each object to be detected is marked as the corresponding initial mask area of ​​each object to be detected. Figure 3 For example, Figure 3 The objects to be detected in the image are two human bodies. According to the specified image, the human body detection frame areas of the two characters are first obtained, namely Figure 3 The box area of ​​the person in . The union of the box areas of the two people is the total box area of ​​the person. Then, according to the specified image, the segmentation masks of the two people are obtained respectively, that is, Figure 3 The seg area of ​​the characters in the figure, where the seg area is the initial mask area of ​​each human body.

[0104] Specifically, for each object to be detected in the specified image, the set of pixels in the segmentation mask of the object to be detected is used as the initial mask area of ​​the object to be detected. Among them, the contour line of the segmentation mask is aligned with the boundary of the object to be detected, so the pixels in the segmentation mask belong to the image range corresponding to the object to be detected. Based on this, the initial mask area is determined according to the segmentation mask, and the range is further expanded on the basis of the initial mask area, and finally a reasonable target mask area is obtained. In the subsequent steps, the target mask area of ​​the object to be detected is finally determined by marking the pixels. The above pixel-based image processing method achieves high-precision image processing by directly operating each pixel of the image.

[0105] like Figure 1 As shown, in step S130, based on the detection frame area of ​​each object to be detected, the initial mask area of ​​each object to be detected is expanded to obtain the target mask area of ​​each object to be detected. Specifically, within the range of the detection frame area, the initial mask area of ​​the object to be detected is expanded to obtain the target mask area of ​​the object to be detected.

[0106] In this step, the mask area of ​​each object to be detected in the specified image is re-divided using the detection frame area obtained in step S110 and the initial mask area obtained in step S120. For example, the initial mask area is appropriately expanded, and during the expansion process, the range of the detection frame area can be referred to and the expansion is performed within the range of the detection frame area.

[0107] In one example, for the overlapping area of ​​the detection frame areas between multiple characters, the pixels in the overlapping area are simultaneously in the detection frames of character A and character B. For the pixels in the overlapping area, the distance from the pixel to the initial mask area of ​​the above two characters can be compared, or the pixel value of the pixel can be compared with the pixel value and / or other image feature values ​​of the edge line of the initial mask area of ​​the above two characters. If the distance from the pixel to the initial mask area of ​​character A is short, or the pixel value and / or other image feature values ​​of the pixel are close to those of character A, the pixel in the overlapping area is expanded to the initial mask area of ​​character A.

[0108] In summary, the embodiments of the present disclosure utilize the detection frame and the segmentation mask to re-divide the region of each object to be detected in the specified image, and can optimize the mask division of the objects to be detected in the specified image, so that the target mask area of ​​each object to be detected is reasonably expanded on the basis of the segmentation mask, thereby improving the consistency of the generated image and the character information without introducing character crosstalk, and then based on the optimized mask area, when generating an image containing multiple characters, the consistency of the generated image and the character information can be improved.

[0109] like Figure 4 As shown, in one embodiment, Figure 1 Step S130 in the method of expanding the initial mask area of ​​each object to be detected based on the detection frame area of ​​each object to be detected to obtain a target mask area of ​​each object to be detected includes:

[0110] Step S410, for each pixel point of the detection frame area of ​​at least one object to be detected, determining the distance between the pixel point and the object identifier of each object to be detected;

[0111] Step S420 , determining, according to the distance, an initial mask region of the object to be detected to which the pixel point corresponding to the distance belongs, so as to expand the initial mask region of each object to be detected to obtain a target mask region of each object to be detected.

[0112] In one example, the objects to be detected include character A and character B. For the overlapping area of ​​the detection frame areas between multiple characters, the pixels in the overlapping area are simultaneously in the detection frames of character A and character B. For the pixels in the overlapping area, the first distance from the pixel to each object identifier of character A is calculated, wherein the minimum value in the first distance is LA; and the second distance from the pixel to each object identifier of character B is calculated, wherein the minimum value in the second distance is LB. If LA is less than LB, it is determined that the object to be detected to which the pixel belongs is character A, and the pixel is expanded into the initial mask area of ​​character A to obtain the target mask area of ​​character A.

[0113] In another example, the objects to be detected include character A and character B. For the pixel points in the overlapping area, the distance LA from the pixel point to the initial mask area of ​​character A is calculated, and the distance LB from the pixel point to the initial mask area of ​​character B is calculated. If LA is less than LB, it is determined that the object to be detected to which the pixel point belongs is character A, and the pixel point is expanded into the initial mask area of ​​character A to obtain the target mask area of ​​character A. Among them, the shortest distance from the pixel point to the edge line of the initial mask area can be used as the distance from the pixel point to the initial mask area.

[0114] like Figure 5 As shown, in one embodiment, Figure 4 Step S410 in the embodiment of the present invention is to determine, for each pixel point of the detection frame area of ​​at least one object to be detected, a distance between the pixel point and the object identifier of each object to be detected, including:

[0115] Step S510, for each object to be detected, marking the pixel points in the initial mask area of ​​the object to be detected as the object identifier of the object to be detected;

[0116] Step S520: for each pixel point in the detection frame area of ​​at least one object to be detected, determine the distance between the pixel point and the object identifier of each object to be detected.

[0117] In one example, the object identifier corresponding to the first object to be detected in the specified image is set to 1, the object identifier corresponding to the second object to be detected is set to 2, and the object identifier corresponding to the Nth object to be detected is set to N. Then all pixels in the initial mask area of ​​the first object to be detected are marked as 1, all pixels in the initial mask area of ​​the second object to be detected are marked as 2, and so on until the initial mask areas of all objects to be detected in the specified image are marked. Figure 3 For example, in the total box area of ​​the characters, the marks of the pixels in the seg areas of the two characters can be marked as 1 and 2 respectively.

[0118] See also Figure 3, for each pixel point in the total box area of ​​the person, that is, for each pixel point in the detection box area of ​​at least one object to be detected, determine the distance between the pixel point and the object identifier of each object to be detected. Figure 3 , based on the marked object identifiers 1 and 2, for each pixel in the total area of ​​the box, the distance between the pixel and each object identifier is determined respectively, that is, the distance value between the pixel and all the pixels with object identifiers 1 and 2 is calculated respectively.

[0119] In one embodiment, Figure 4 Step S420 in the embodiment of the present invention determines, according to the distance, an initial mask area of ​​the object to be detected to which the pixel point corresponding to the distance belongs, including:

[0120] For each pixel point of the detection frame area of ​​at least one object to be detected, determine the object identifier closest to the pixel point, and determine the initial mask area of ​​the object to be detected corresponding to the nearest object identifier, which is the initial mask area of ​​the object to be detected to which the pixel point belongs.

[0121] Specifically, if the pixel point itself is located in the initial mask area of ​​a certain object to be detected, the pixel point itself has been marked as the object identifier of the object to be detected. In this case, the object identifier closest to the pixel point is itself, and the distance is zero. Therefore, after being processed by step S420, the pixel point still belongs to the initial mask area of ​​the original object to be detected. In other words, after the pixel point with the original object identifier of 1 is processed by step S420, it still belongs to the initial mask area of ​​the original object to be detected, and the identifier is still 1; similarly, after the pixel point with the original object identifier of 2 is processed by step S420, it still belongs to the initial mask area of ​​the original object to be detected, and the identifier is still 2.

[0122] Specifically, if the position of the pixel is not within the initial mask area of ​​a certain object to be detected, the object identifier closest to the pixel is calculated; the initial mask area corresponding to the nearest object identifier is the initial mask area to which the pixel belongs. Assuming that there are only two objects to be detected in the specified image, whose object identifiers are 1 and 2 respectively, the distance values ​​between the pixel and all the pixels with object identifiers 1 and 2 have been calculated in step S410. In step S420, the minimum value of these distance values ​​is taken. If the distance value between the pixel and a pixel with object identifier 1 is the minimum, the initial mask area corresponding to object identifier 1 is the initial mask area to which the pixel belongs, and the pixel is expanded to the initial mask area corresponding to object identifier 1.

[0123] In one embodiment, Figure 5Step S510 in the method of marking, for each object to be detected, pixels within the initial mask area of ​​the object to be detected as an object identifier of the object to be detected includes:

[0124] Sequentially mark the pixel points in the initial mask area of ​​at least one object to be detected as the object identifier of the object to be detected corresponding to it; wherein,

[0125] In response to the overlap of the initial mask area of ​​at least one object to be detected, the object identifier marked later covers the object identifier marked earlier.

[0126] In one example, all pixels in the initial mask area of ​​the first object to be detected are first marked as 1, and then all pixels in the initial mask area of ​​the second object to be detected are marked as 2. If the initial mask areas of the two objects to be detected overlap, then the mark 2 covers the mark 1, that is, the object ID marked later covers the object ID marked earlier. This process is repeated until all initial mask areas of the objects to be detected in the specified image are marked.

[0127] like Figure 6 As shown, in one embodiment, Figure 1 Step S130 in the method of expanding the initial mask area of ​​each object to be detected based on the detection frame area of ​​each object to be detected to obtain a target mask area of ​​each object to be detected includes:

[0128] Step S210, in response to the overlap of the detection frame area of ​​at least one object to be detected, determining a union of the detection frame area of ​​at least one object to be detected as the area to be marked;

[0129] Step S215 , based on the area to be marked, the initial mask area of ​​each object to be detected is expanded respectively to obtain a target mask area of ​​each object to be detected.

[0130] Taking the role replacement of multiple characters as an example, in order to improve the consistency between the generated image and the specified character information, the segmentation mask of the human body can be appropriately expanded. And in order to avoid information crosstalk between multiple characters, the expansion range should not be too large. The detection frame of the object to be detected in the specified image is the human body detection frame. During the expansion process, the area range of the human body detection frame can be referred to, and the maximum area range of the expansion does not exceed the area range of the human body detection frame. The union of all human body detection frames in the specified image is the maximum expansion range of the mask area corresponding to each human body. In step S210, the union of all human body detection frames in the specified image is used as the area to be marked. In subsequent steps, each pixel point in the area to be marked is expanded to the target mask area corresponding to each human body.

[0131] See also Figure 3For example, according to the specified image, we first obtain the human body detection frames of the two characters, namely Figure 3 The union of the box areas of the two characters is the total box area of ​​the character, that is, the area to be marked.

[0132] like Figure 7 As shown, in one embodiment, Figure 6 Step S215 in the embodiment may specifically include:

[0133] Step S220, determining a re-marked area according to the area to be marked and the segmentation mask of at least one object to be detected in the specified image; wherein the union of the detection frame areas of all the objects to be detected is the area to be marked, and the re-marked area is the area of ​​the area to be marked except the mask areas of all the objects to be detected, that is, the area to be marked includes the re-marked area and the initial mask area of ​​each object to be detected;

[0134] In the area to be marked, the area outside the initial mask area of ​​each object to be detected is determined as the re-marked area. Specifically, the pixels in the initial mask area can be marked in the area to be marked first, and then the unmarked area is determined as the re-marked area.

[0135] Step S230, based on the re-marked area, the initial mask area of ​​the object to be detected is expanded to obtain the target mask area corresponding to the object to be detected. Specifically, the distance from each pixel point in the re-marked area to the object identifier of each object to be detected is calculated, and then the initial mask area of ​​the object to be detected to which the pixel point in the re-marked area belongs is determined according to the distance corresponding to each pixel point in the re-marked area.

[0136] Each pixel in the re-marked area needs to be expanded into the mask area corresponding to each object to be detected. The expansion can be performed according to the distance between the pixel and each object to be detected, and each pixel in the re-marked area is expanded into the mask area corresponding to the object to be detected that is closest to it.

[0137] In the above embodiments, the maximum area range of the expansion of the mask area of ​​the object to be detected in the specified image is determined based on the area to be marked, and the pixels in the heavy-marked area are expanded to the mask area of ​​the object based on the segmentation mask. Based on the above scheme, the mask area division is more reasonable, and the mask based on the reasonable division can achieve better image processing effect.

[0138] In one embodiment, Figure 7 Step S220 in the step of determining the re-marking region according to the region to be marked and the segmentation mask of at least one object to be detected in the specified image includes:

[0139] Mark the pixels in the initial mask area as mask identifiers corresponding to the object to be detected;

[0140] The pixel points in the to-be-marked area that are not marked as mask marks are taken as to-be-marked points; and the set of all to-be-marked points is determined as the re-marked area.

[0141] Still taking the role replacement of multiple characters as an example, according to the segmentation mask of each human body in the specified image, the initial mask area of ​​each human body is marked in the area to be marked. Specifically, it may include: for each human body in the specified image, the set of pixel points in the segmentation mask of each human body is used as the initial mask area of ​​each human body; and the pixel points in the segmentation mask of each human body are marked as the mask identifier corresponding to each human body.

[0142] Specifically, the union of all human detection frames in the specified image is used as the maximum area range of the human body. In one example, all pixels in all human detection frame areas in the specified image are marked as 255. All pixels marked as 255 constitute the union of all human detection frames, that is, the area to be marked. Figure 3 In the example above, all pixels in the total area of ​​the character's box are marked as 255.

[0143] Mark the marking map in turn according to the human segmentation masks in the specified image, and mark the pixels in the segmentation mask of each human body as the mask identifier corresponding to each human body. For example, the preset mask identifier value can be used as the mask identifier. In one example, the mask identifier value corresponding to the first human body is set to 1, the mask identifier value corresponding to the second human body is set to 2, and so on until the mask identifier value corresponding to the Nth human body is N. Then all pixels in the first human segmentation mask are marked as 1, and all pixels in the second human segmentation mask are marked as 2, and so on until all human segmentation masks in the specified image are marked. The set of pixels with the same mask identifier value constitutes the initial mask area corresponding to the human body. See Figure 3 For example, in the total box area of ​​the characters, the marks of the pixels in the seg areas of the two characters can be marked as 1 and 2 respectively.

[0144] After the pixel marking in the initial mask area is completed, continue to mark the unmarked pixel points in the area to be marked. As mentioned above, in the specified image, first mark all the pixel points in the human body detection frame area as 255, that is, all the pixel points marked as 255 constitute the area to be marked. Then, mark the pixel points in each human body segmentation mask in the area to be marked with the corresponding mask mark value in turn. Then the pixel points in the area to be marked that are not marked with the mask mark value are the points to be marked, that is, all the pixel points marked as 255 are the points to be marked. After that, re-mark all the points marked as 255 in the specified image, and determine the set of all the points to be marked as the re-marked area.

[0145] In the subsequent steps, by re-marking the pixels in the re-marked area and marking them as the mask mark value of a certain human body, the mask area of ​​the human body is reasonably expanded and there is no intersection between the mask areas of each human body, so as to achieve the effect of improving the consistency of the characters and eliminating crosstalk.

[0146] In one embodiment, Figure 7 In step S230, based on the re-marked area, the initial mask area of ​​the object to be detected is expanded to obtain a target mask area corresponding to the object to be detected, including:

[0147] According to the distance between each pixel point in the re-marked area and the initial mask area of ​​each object to be detected, the initial mask area of ​​each object to be detected is expanded to obtain the expanded mask area of ​​each object to be detected;

[0148] The expanded mask area of ​​the object to be detected is used as the target mask area corresponding to the object to be detected.

[0149] For each pixel in the re-labeled area, it is necessary to expand it to the mask area corresponding to each object to be detected. Specifically, the distance from each pixel in the re-labeled area to the initial mask area of ​​each object to be detected can be calculated, and the pixel can be expanded to the initial mask area with the closest distance. Among them, the shortest distance from the pixel to the edge line of the initial mask area can be used as the distance from the pixel to the initial mask area. After all the pixels in the re-labeled area are expanded, the expanded mask area of ​​each object to be detected is finally obtained. See Figure 3 In the marked area, all the pixels in the entire area to be marked have been marked. In the marked area, the dark image area and the light image area represent the expanded mask areas of the two human bodies respectively.

[0150] See also Figure 3, extract the expanded mask areas of the two characters from the marked area respectively, and obtain the final mask area of ​​the characters. Based on the distance from the pixel point to the initial mask area of ​​each object to be detected, the final mask area reasonably expands the initial mask area, which can prevent the crosstalk of character information in the overlapping area of ​​the human body detection frame, and avoid excessive restrictions on the human body segmentation mask that lead to reduced character consistency, so as to achieve realistic and natural image effects.

[0151] Figure 8 FIG. 1 is a flow chart of an embodiment of the method for dividing the image mask area of ​​the present disclosure. Figure 8 As shown, in one embodiment, the initial mask area of ​​each object to be detected is expanded according to the distance between each pixel point in the re-marked area and the initial mask area of ​​each object to be detected, so as to obtain the expanded mask area of ​​each object to be detected, including:

[0152] Step S310, for all the points to be marked, respectively calculating the initial mask area closest to the points to be marked in the area to be marked;

[0153] Step S320, marking the point to be marked as the mask mark corresponding to the nearest initial mask area; marking the pixel points in the initial mask area as the mask mark corresponding to the object to be detected;

[0154] Step S330 , obtaining an expanded mask area of ​​each object to be detected according to the mask identifiers of all pixels in the area to be marked.

[0155] For all the points to be marked, calculate the distance between the point to be marked and the initial mask area of ​​each object to be detected in the specified image. For example, there are N human bodies in the specified image. If the shortest distance from the point to be marked to the edge line of the initial mask area of ​​the Mth human body is the smallest, then the Mth human body is determined as the initial mask area closest to the point to be marked. Mark the point to be marked with the mask mark value M corresponding to the Mth human body, that is, expand the point to be marked to the mask area of ​​the Mth human body. After all the pixels in the area to be marked are re-marked, the set of pixels with the same mask mark value constitutes the expanded mask area corresponding to the human body. See Figure 3 In the example, the pixels in the seg area of ​​the two characters are marked as 1 and 2 respectively, and then the other points in the specified image that are still marked as 255 are re-marked to obtain the expanded mask area. Figure 3 The marked area.

[0156] The above embodiments reasonably divide the points to be marked into the corresponding mask area of ​​each object to be detected by calculating the distance of each point to be marked and performing marking processing, thereby improving the image processing effect based on the reasonable mask area.

[0157] In one embodiment, Figure 8 In step S310 of the method, calculating the initial mask area closest to the point to be marked in the area to be marked includes:

[0158] Calculate the distance values ​​from the to-be-marked point to all the pixel points in the initial mask area of ​​each to-be-detected object in the to-be-marked area respectively;

[0159] The initial mask area corresponding to the minimum value among all distance values ​​is determined as the initial mask area closest to the point to be marked.

[0160] In the example of human replacement, for all points to be marked, the distance values ​​from all pixels in the initial mask area of ​​each human body to the point to be marked are calculated respectively. The pixel corresponding to the minimum value among all distance values ​​is the pixel closest to the point to be marked in all initial mask areas. The initial mask area of ​​the human body where the pixel is located is the initial mask area of ​​the human body closest to the point to be marked. And, the point to be marked is re-marked with the mask mark value of the pixel.

[0161] The above embodiment calculates the distance by traversing all the initial mask areas, so that the points to be marked are finally divided into the mask area with the closest distance, thereby improving the image processing effect based on the reasonably divided mask areas.

[0162] In one implementation, based on the detection frame area of ​​each object to be detected, the initial mask area of ​​each object to be detected is expanded to obtain the target mask area of ​​each object to be detected, including:

[0163] Constructing a label map according to the specified image and the detection frame area and the initial mask area of ​​each object to be detected, wherein the size of the label map is consistent with the size of the specified image, and the label map includes the detection frame area and the initial mask area of ​​each object to be detected;

[0164] Based on the detection frame area of ​​each object to be detected in the label image, the initial mask area of ​​the object to be detected in the label image is expanded respectively to obtain the target mask area of ​​each object to be detected.

[0165] In one implementation, the pixel value of a pixel point in the marker image is a first preset value. For example, the first preset value may be set to 0.

[0166] Specifically, still taking the role replacement of multiple characters as an example, the union of all human detection frames in the specified image is used as the maximum area range that the character can expand. An image of the same size as the specified image and with all pixel marks of 0 can be constructed as a labeling map. First, all pixels in all human detection frame areas in the labeling map are marked as 255. All pixels marked as 255 constitute the union of all human detection frames, that is, the area to be marked. See Figure 3 In the example of FIG. 1 , all pixels in the total box area of ​​the person are marked as 255. According to this embodiment, there is no interference information in the label map and the pixel values ​​of the pixels in the label map are uniformly marked, so as to quickly calculate the target mask area of ​​each object to be detected.

[0167] Mark the marking map in turn according to the human segmentation masks in the specified image, and mark the pixels in the segmentation mask of each human body as the mask mark corresponding to each human body. The preset mask mark value can be used as the mask mark. For example, the mask mark value corresponding to the first human body is set to 1, the mask mark value corresponding to the second human body is set to 2, and so on until the mask mark value corresponding to the Nth human body is N. Then all the pixels in the first human segmentation mask are marked as 1, and all the pixels in the second human segmentation mask are marked as 2, and so on until all the human segmentation masks in the specified image are marked. The set of pixels with the same mask mark value constitutes the initial mask area corresponding to the human body. See Figure 3 For example, in the total box area of ​​the characters, the marks of the pixels in the seg areas of the two characters can be marked as 1 and 2 respectively.

[0168] After the pixel marking in the initial mask area is completed, continue to mark the pixels that are not marked in the area to be marked. As mentioned above, in the above-mentioned marking diagram, all pixels in all human body detection frame areas are first marked as 255, that is, all pixels marked as 255 constitute the area to be marked. Then the pixels in each human body segmentation mask in the area to be marked are marked as corresponding mask mark values ​​in turn, and the pixels in the area to be marked that are not marked as mask mark values ​​are points to be marked, that is, all pixels marked as 255 are points to be marked. After that, all points marked as 255 in the marking diagram are re-marked, and the set of all points to be marked is determined as the re-marked area. Finally, according to the distance from each pixel in the re-marked area to the initial mask area of ​​each object to be detected, the initial mask area of ​​each object to be detected is expanded to obtain the target mask area corresponding to each object to be detected. The specific implementation steps for implementing expansion in the marking diagram are similar to the way of implementing expansion on a specified image. The specific steps and beneficial effects or technical problems solved can be referred to the description in the above implementations, or the description in the content of the invention, which will not be repeated here.

[0169] In summary, taking the role replacement of multiple characters as an example, an embodiment of the method for dividing an image region disclosed in the present invention may include the following steps:

[0170] S1: Perform human body detection and human body segmentation on the specified image to obtain the human body detection frame and human body segmentation mask.

[0171] S2: Construct an image with the same size as the specified image and all the labels are 0 as the label map, and mark the pixels in the human detection box in the label map as 255.

[0172] S3: According to the coordinates of the human body segmentation mask, mark the pixels in the human body segmentation mask area in sequence and mark them as corresponding human body marking points. The first human body is marked as 1, the second human body is marked as 2, and so on. The marked number is the mask identifier.

[0173] S4: Take each pixel in the marking image marked as 255 as a point to be marked, and re-mark the point to be marked. That is, calculate the distance from each human marker point to the point to be marked, find the human marker point closest to the point to be marked, and re-mark the current point to be marked with the mask mark of this human marker point.

[0174] S5: When all the points to be marked are marked, the mask area of ​​each human body is taken out from the marked image in sequence starting from 1 according to the marking order, and used as the mask area for the final person replacement.

[0175] By using the mask area obtained by the division of the embodiment of the present disclosure, the image can be further processed to generate an image that meets the requirements. Fig. 9 As shown, an example of image generation may specifically include the following steps:

[0176] Step S100, receiving a specified image and an image generation prompt, wherein the image generation prompt includes role information corresponding to each object to be detected in the specified image;

[0177] Steps S110 to S130, using an image region division method to obtain a target mask region of each object to be detected in a specified image;

[0178] Step S150 , using the role information corresponding to each object to be detected, the target mask area of ​​each object to be detected is replaced respectively to generate an object replacement image corresponding to the designated image.

[0179] Fig.10 A flowchart of generating a character replacement image is shown. In an image generation example, the input information and output information of the character replacement task are:

[0180] Input information: a layout diagram and the specified character information of each character, as well as the corresponding text. The layout diagram is used to guide the character's movements and postures. The specified character information includes information that can represent the character's appearance, personality and other characteristics, which can specifically include the character's character modeling, body proportions, clothing style, etc. The character information can be displayed using a character diagram. The corresponding text is used to indicate the character's movements and postures or clothing style, etc.

[0181] Output information: A picture in which multiple characters are generated according to the postures of the layout diagram and the instructions of the text.

[0182] See also Fig. 9 and Fig.10 First, the layout image is used as the designated image, and human body detection and human body segmentation are performed on the layout image to obtain a human body detection frame and a human body segmentation mask, respectively. According to the image area division method provided by the present disclosure, a labeling image is constructed based on the human body detection frame and the human body segmentation mask. The human body area is automatically divided in the labeling image to obtain the mask area corresponding to each human body in the layout image. Then, the mask area of ​​the person is extracted, and the person is replaced using the mask area, and finally a person replacement effect image is generated. For the implementation details of the image area division, please refer to Figures 1 to 8 The relevant descriptions of the various embodiments in the present invention will not be repeated here one by one.

[0183] In the above example, character information can be input during the character replacement stage. The input character information can include the character's image, character description, etc. These character information and the character's Mask area are one-to-one corresponding, that is, for each Mask there is a corresponding character information input. There are many ways to replace characters, for example, you can use the IPAdapter tool to replace characters.

[0184] In the above examples, the IPAdapter tool itself does not use the restrictions of the mask to generate the image. In the generated image, the expected layout structure in the layout diagram needs to be maintained. Based on the embodiments of the present disclosure, the mask can be used as guidance information when generating the image to replace the mask area corresponding to the human body in the layout diagram instead of replacing the entire image. Specifically, the information of the mask area corresponding to the human body divided in the layout diagram can be added to the input information of the character replacement task. This information serves as guidance information and plays a supervisory role, indicating that the image should be replaced within a reasonable divided area to avoid information crosstalk and reduced character consistency due to the area being too large or too small.

[0185] Fig.11 FIG. 1 is a schematic diagram showing the generation effect of an embodiment of the image area division method disclosed in the present invention. Fig.11In the example, refer to the characters in the two character images given to replace the corresponding two characters in the layout image. Among them, one of the characters in the character image wears red clothes and has long curly hair. Fig.11 The effects of the generated images of the prior art solution and the disclosed solution are compared in . It can be clearly seen that the hairstyle of the person wearing red clothes in the generated image of the prior art solution is short hair. It can be seen that the generated image of the prior art solution cannot change the character image according to the needs of the role, the character in the generated image does not conform to the character image of the role diagram, and the character consistency of the image is poor. In contrast, the hairstyle of the person wearing red clothes in the generated image of the disclosed solution is long curly hair. It can be seen that the disclosed solution reasonably optimizes the mask division of the human body area in the layout diagram, so that the mask area corresponding to each human body is reasonably expanded, thereby improving the character consistency between the generated image and the role diagram.

[0186] like Fig.12 As shown, the present disclosure also provides an embodiment of a corresponding image region division device. For the beneficial effects or technical problems solved by the device, please refer to the description of the method corresponding to each device, or refer to the description in the invention content, which will not be repeated here.

[0187] In an embodiment of the device for dividing the image area, the device comprises:

[0188] The detection unit 100 is used to: perform object detection on a specified image to obtain a detection frame area of ​​each object to be detected in the specified image; wherein the specified image includes at least one object to be detected;

[0189] The segmentation unit 200 is used to: perform image segmentation on the specified image to obtain an initial mask area of ​​each object to be detected in the specified image;

[0190] The expansion unit 300 is used to expand the initial mask area of ​​each object to be detected based on the detection frame area of ​​each object to be detected, so as to obtain the target mask area of ​​each object to be detected.

[0191] like Fig.13 As shown, in one embodiment, the expansion unit 300 includes:

[0192] A first determining subunit 310 is used to determine, for each pixel point of a detection frame area of ​​at least one object to be detected, a distance between the pixel point and an object identifier of each object to be detected;

[0193] The second determining subunit 320 is used to determine, according to the distance, the initial mask area of ​​the object to be detected to which the pixel point corresponding to the distance belongs, so as to expand the initial mask area of ​​each object to be detected to obtain the target mask area of ​​each object to be detected.

[0194] In one implementation, the first determining subunit 310 is configured to:

[0195] For each object to be detected, marking the pixel points within the initial mask area of ​​the object to be detected as the object identifier of the object to be detected;

[0196] For each pixel point of the detection frame area of ​​at least one object to be detected, a distance between the pixel point and an object identifier of each object to be detected is determined.

[0197] In one implementation, the second determining subunit 320 is configured to:

[0198] For each pixel point of the detection frame area of ​​at least one object to be detected, determine the object identifier closest to the pixel point, and determine the initial mask area of ​​the object to be detected corresponding to the nearest object identifier, which is the initial mask area of ​​the object to be detected to which the pixel point belongs.

[0199] In one implementation, the first determining subunit 310 is configured to:

[0200] Sequentially mark the pixel points in the initial mask area of ​​at least one object to be detected as the object identifier of the object to be detected corresponding to it; wherein,

[0201] In response to the overlap of the initial mask area of ​​at least one object to be detected, the object identifier marked later covers the object identifier marked earlier.

[0202] In one embodiment, the expansion unit 300 is used to:

[0203] In response to the overlap of the detection frame area of ​​at least one object to be detected, determining a union of the detection frame area of ​​at least one object to be detected as the area to be marked;

[0204] Based on the area to be marked, the initial mask area of ​​each object to be detected is expanded respectively to obtain the target mask area of ​​each object to be detected.

[0205] In one embodiment, the expansion unit 300 is used to:

[0206] Constructing a label map according to the specified image and the detection frame area and the initial mask area of ​​each object to be detected, wherein the size of the label map is consistent with the size of the specified image, and the label map includes the detection frame area and the initial mask area of ​​each object to be detected;

[0207] Based on the detection frame area of ​​each object to be detected in the label image, the initial mask area of ​​the object to be detected in the label image is expanded respectively to obtain the target mask area of ​​each object to be detected.

[0208] In one implementation, the pixel value of a pixel point in the marker image is a first preset value.

[0209] Below, reference Fig.14 The electronic device according to the embodiment of the present disclosure is described. The electronic device may be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signals from them.

[0210] Fig.14 A block diagram of an electronic device according to an embodiment of the present disclosure is illustrated.

[0211] like Fig.14 As shown, the electronic device includes one or more processors and memory.

[0212] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0213] The memory may store one or more computer program products, and the memory may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program products may be stored on the computer-readable storage medium, and the processor may run the computer program product to implement the various embodiments of the present disclosure described above and / or other desired functions.

[0214] In one example, the electronic device may further include: an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0215] In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

[0216] The output device can output various information to the outside, including the determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0217] Of course, to simplify, Fig.14 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.

[0218] In addition to the above methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above part of this specification.

[0219] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0220] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above part of this specification.

[0221] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0222] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0223] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0224] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0225] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0226] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0227] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0228] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for dividing an image region, characterized in that: include: Performing object detection on a specified image to obtain a detection frame area of ​​each object to be detected in the specified image; wherein the specified image includes at least one object to be detected; Performing image segmentation on the designated image to obtain an initial mask area of ​​each object to be detected in the designated image; Based on the detection frame area of ​​each object to be detected, the initial mask area of ​​each object to be detected is expanded respectively to obtain the target mask area of ​​each object to be detected.

2. The method according to claim 1, characterized in that The method of expanding the initial mask area of ​​each object to be detected based on the detection frame area of ​​each object to be detected to obtain a target mask area of ​​each object to be detected includes: For each pixel point of the detection frame area of ​​the at least one object to be detected, determining the distance between the pixel point and the object identifier of each object to be detected; According to the distance, the initial mask area of ​​the object to be detected to which the pixel point corresponding to the distance belongs is determined, so as to expand the initial mask area of ​​each object to be detected to obtain the target mask area of ​​each object to be detected.

3. The method according to claim 2, characterized in that The step of determining, for each pixel point of the detection frame area of ​​the at least one object to be detected, a distance between the pixel point and the object identifier of each object to be detected comprises: For each object to be detected, marking the pixel points within the initial mask area of ​​the object to be detected as the object identifier of the object to be detected; For each pixel point of the detection frame area of ​​the at least one object to be detected, the distance between the pixel point and the object identifier of each object to be detected is determined.

4. The method according to claim 3, characterized in that The step of determining, based on the distance, an initial mask area of ​​the object to be detected to which the pixel point corresponding to the distance belongs includes: For each pixel point of the detection frame area of ​​the at least one object to be detected, determine the object identifier closest to the pixel point, and determine the initial mask area of ​​the object to be detected corresponding to the nearest object identifier, which is the initial mask area of ​​the object to be detected to which the pixel point belongs.

5. The method according to claim 3, characterized in that: The step of marking, for each object to be detected, the pixel points within the initial mask area of ​​the object to be detected as the object identifier of the object to be detected includes: Sequentially marking the pixel points in the initial mask area of ​​the at least one object to be detected with the object identifier of the object to be detected corresponding to it; wherein, In response to the overlap of the initial mask area of ​​the at least one object to be detected, the object identifier marked later covers the object identifier marked earlier.

6. The method according to any one of claims 1 to 5, characterized in that: The method of expanding the initial mask area of ​​each object to be detected based on the detection frame area of ​​each object to be detected to obtain a target mask area of ​​each object to be detected includes: In response to the overlap of the detection frame areas of the at least one object to be detected, determining a union of the detection frame areas of the at least one object to be detected as the area to be marked; Based on the area to be marked, the initial mask area of ​​each of the objects to be detected is expanded respectively to obtain a target mask area of ​​each of the objects to be detected.

7. The method according to any one of claims 1 to 5, characterized in that: The method of expanding the initial mask area of ​​each object to be detected based on the detection frame area of ​​each object to be detected to obtain a target mask area of ​​each object to be detected includes: Constructing a label map according to the designated image and the detection frame area and the initial mask area of ​​each object to be detected, wherein the size of the label map is consistent with the size of the designated image, and the label map includes the detection frame area and the initial mask area of ​​each object to be detected; Based on the detection frame area of ​​each object to be detected in the label image, the initial mask area of ​​the object to be detected in the label image is expanded respectively to obtain the target mask area of ​​each object to be detected.

8. The method according to claim 7, characterized in that The pixel values ​​of the pixels in the label map are first preset values.

9. A device for dividing an image region, characterized in that: include: A detection unit, configured to perform object detection on a specified image to obtain a detection frame area of ​​each object to be detected in the specified image; wherein the specified image includes at least one object to be detected; A segmentation unit, configured to perform image segmentation on the designated image to obtain an initial mask region of each object to be detected in the designated image; The expansion unit is used to expand the initial mask area of ​​each object to be detected based on the detection frame area of ​​each object to be detected, so as to obtain the target mask area of ​​each object to be detected.

10. An electronic device, characterized in that: include: A memory for storing a computer program product; A processor is used to execute the computer program product stored in the memory, and when the computer program product is executed, it implements the method described in any one of claims 1 to 8.

11. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method described in any one of claims 1 to 8 is implemented.

Images