Cutout methods, equipment, media and products

Abstract

The present application provides a cutout method, device, medium and product, and relates to the field of image extraction technology. The cutout method includes obtaining an image matrix output by a cutout model corresponding to an input first image, the image matrix includes indication information indicating a target object in the first image, and the first image is obtained by processing the original image based on the information of the cutout model; the image matrix is ​​expanded according to the original image to obtain the first matrix; the first matrix is ​​converted according to the image matrix and a threshold to obtain a target matrix, and the threshold is determined based on the cutout information of the cutout model; the target object in the original image is obtained based on the target matrix and the original image. After the cutout model is cutout processed, the embodiment of the present application further adjusts the cutout processing result, thereby improving the accuracy of the cutout result, effectively improving the cutout effect, and enhancing the user experience.

Description

Technical Field

[0001] The present application relates to the field of image extraction technology, and more specifically, to a cutout method, device, medium and product. Background Art

[0002] With the development of science and technology, the application of image processing is becoming more and more extensive. For example, in order to protect user privacy, a video or image is cut out to replace the background or blur the background. However, in practice, when using traditional cutout solutions, it is difficult to achieve the purpose of cutout very accurately. For example, the cutout image still has a virtual image of a semi-transparent transition zone, or a jagged effect, resulting in poor cutout effect. Summary of the invention

[0003] The present application provides a method, device, medium and product for cutting out images, which can solve the problem of poor cutting out effect of existing cutting out methods. To achieve this purpose, the present application provides the following solutions.

[0004] According to one aspect of an embodiment of the present application, a cutout method is provided, comprising:

[0005] Acquire an image matrix output by a cutout model corresponding to an input first image, wherein the image matrix includes indication information indicating a target object in the first image, and the first image is obtained by processing an original image based on information of the cutout model;

[0006] Expanding the image matrix according to the original image to obtain a first matrix;

[0007] Converting the first matrix according to the image matrix and a threshold value to obtain a target matrix, wherein the threshold value is determined based on the cutout information of the cutout model;

[0008] A target object in the original image is acquired based on the target matrix and the original image.

[0009] In a possible implementation, the information includes image input information, and processing the original image based on the information of the cutout model includes:

[0010] Determine the length of the longest side of the original image after scaling according to the image input information, and determine the scaling ratio based on the length and the width and height of the original image;

[0011] The original image is scaled proportionally according to the scaling ratio.

[0012] In a possible implementation, the information includes sampling information, and processing the original image based on the information of the cutout model includes:

[0013] Acquire side length information of the input image according to the sampling information;

[0014] The image to be input corresponding to the original image is filled based on the side length information to obtain the first image, where the image to be input is determined based on the image input information.

[0015] In a possible implementation, filling the image to be input corresponding to the original image based on the side length information includes:

[0016] Acquire filling information of the image to be input according to the side length information, and create a gray image based on the filling information;

[0017] The pixel values ​​of the area in the gray image corresponding to the image to be input are determined as the pixel values ​​of the image to be input.

[0018] In a possible implementation, the step of expanding the image matrix according to the original image to obtain a first matrix includes:

[0019] If it is determined that the first image includes a filling area, a portion of the image matrix corresponding to the filling area is removed, and the removed image matrix is ​​expanded according to the size of the original image to obtain a first matrix.

[0020] In a possible implementation, converting the first matrix according to the image matrix and a threshold to obtain a target matrix includes:

[0021] Convert the first matrix based on formula (1) to obtain a target matrix;

[0022]

[0023] Among them, Threshold is the threshold, α is the first matrix, A is the target matrix, and n represents the degree of transformation.

[0024] In a possible implementation manner, acquiring the target object in the original image based on the target matrix and the original image includes:

[0025] Obtaining a pixel matrix of the original image;

[0026] The pixel matrix is ​​dot-multiplied by the target matrix to obtain an image of the target object.

[0027] According to one aspect of an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described above.

[0028] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described above are implemented.

[0029] According to one aspect of an embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0030] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0031] The cutout method provided in the present application obtains an image matrix corresponding to the first input image output by the cutout model; expands the image matrix according to the original image corresponding to the first image to obtain the first matrix; converts the first matrix according to the image matrix and the threshold obtained based on the cutout information of the cutout model to obtain the target matrix; obtains the target object in the original image based on the target matrix and the original image. The embodiment of the present application can further adjust the cutout processing result after the cutout model performs cutout processing, thereby improving the accuracy of the cutout result, effectively improving the cutout effect, and enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following is a brief introduction to the drawings required for use in describing the embodiments of the present application.

[0033] Figure 1 A flowchart of a cutout method provided in an embodiment of the present application;

[0034] Figure 2 A specific workflow diagram of the cutout method provided in the embodiment of the present application;

[0035] Figure 3 This is a structural diagram of the electronic device of this application. DETAILED DESCRIPTION

[0036] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0037] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation of other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the present technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" indicates that it is implemented as "A", or is implemented as "A", or is implemented as "A and B".

[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0039] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0040] The cutout method, device, medium and product provided in this application are intended to solve at least one technical problem existing in the prior art.

[0041] The present application provides a method for cutting out a picture, such as Figure 1 , Figure 2 As shown, the cutout method can be executed by a mobile phone, a computer, a server, a camera, or other devices that can load a cutout model and use the cutout model to perform a cutout operation.

[0042] Optionally, the device that executes the cutout method may be a device end in edge computing or an edge device used to connect the device end and the cloud, and the image of the target object is extracted using the device end or the edge device.

[0043] Optionally, the target object may be a person, an animal, a car, or other objects that can be recognized and extracted by the cutout model.

[0044] Optionally, the cutout method includes:

[0045] S101: Obtain an image matrix output by a matting model corresponding to an input first image.

[0046] Optionally, the image matrix includes indication information indicating a target object in a first image, and the first image is obtained by processing an original image based on information of a matting model.

[0047] Optionally, the data in the image matrix can be used to represent the probability that the pixel corresponding to the data in the first image belongs to the target object. Specifically, the probability value range is [0, 1]. If the data is 1, it means that the pixel corresponds to the target object, and if it is 0, it means that the pixel corresponds to the background area.

[0048] Optionally, the information of the cutout model may include at least one of image input information and sampling information. The original image is adjusted to an image matching the cutout model according to the information.

[0049] Optionally, when the information of the cutout model is image input information, the image input information includes image input size information of the cutout model, and the scaling ratio of the first image is calculated based on the image input information, so as to obtain a scaled image related to the image input requirement of the cutout model. Wherein, processing the original image based on the information of the cutout model includes: determining the length of the longest side of the original image after scaling based on the image input information, determining the scaling ratio based on the length and the width and height of the original image; scaling the original image proportionally according to the scaling ratio. By scaling proportionally, it is ensured that the image is not distorted.

[0050] In one embodiment, the original image and the image input to the cutout model are both rectangular images, and the original image is an RGB image with a size of 1920×1080×3. The image input information of the cutout model is obtained, and the image input size of the cutout model is determined to be 288*480 according to the image input information, that is, the length of the longest side is 480, and the ratio of the longest side to the longest side in the original image is determined as the scaling ratio. The formula for calculating the scaling ratio can be: s is the scaling ratio, L long represents the length of the long side determined based on the image input size, w represents the width of the original image, and h represents the height of the original image. The width and height of the scaled original image can be respectively calculated by R w =w·s,R h = h·s calculation, R w Represents the width after scaling, R h Indicates the height after scaling.

[0051] The scaling ratio of the original image is obtained by the above formula, and the original image is scaled according to the scaling ratio to obtain an image with a size of 480×270×3.

[0052] Optionally, the length of the minimum side can be obtained according to the image input size, the scaling ratio is calculated based on the length, and the image is scaled according to the scaling ratio. The formula for calculating the scaling ratio can be: L short Represents the minimum side determined based on the image input size.

[0053] Optionally, since the size of the image obtained by proportional scaling may not match the image required by the cutout model, the scaled image can be filled according to the sampling information of the cutout model. Therefore, the information of the cutout model may include sampling information, and the original image is processed based on the information of the cutout model, including: obtaining the side length information of the input image according to the sampling information; filling the image to be input corresponding to the original image based on the side length information to obtain a first image, and the image to be input is determined based on the image input information.

[0054] Optionally, the sampling information may include downsampling information, and the side length information may include a multiple by which the scaled image needs to be reduced. Information about the multiple by which the scaled image needs to be reduced may be acquired based on the downsampling information.

[0055] In one embodiment, it is determined based on the sampling information that when the cutout model performs image processing, it needs to perform 5 times of 2 downsampling, that is, the input image to be input will eventually be reduced by 2 to the fifth power, so that it can be determined that the image will eventually be reduced to one-thirtieth of the input image in the cutout model. That is, the width and height of the image to be input need to be integer multiples of 32 for the model to process. If not, padding is required. The remainder after dividing the width and height by 32 can be calculated by the following formula:

[0056] q w =R w mod 32

[0057] q h =R h mod 32

[0058] Among them, q w Represents the remainder after the width of the original image is divided by 32 after scaling, q h Indicates the remainder after the height of the original image after scaling is divided by 32. The number of pixels to be filled corresponding to the width and height is determined based on the remainder.

[0059] Optionally, filling the image to be input corresponding to the original image based on the side length information includes: obtaining filling information of the image to be input according to the side length information, and creating a gray image based on the filling information; and determining the pixel value of the area in the gray image corresponding to the image to be input as the pixel value of the image to be input.

[0060] In one embodiment, it is determined according to the side length information that the width and height of the image to be input need to be an integer multiple of 32. The number of pixels that need to be increased in the width and height of the image to be input is calculated by the following formula.

[0061] add w =32-q w

[0062] add h =32-q h

[0063] Among them, add w Indicates the number of pixels corresponding to the width of the input image, add h Indicates the number of pixels corresponding to the height of the input image.

[0064] Optionally, the created gray image may be an RGB image, and the pixel values ​​of the image may be set according to actual requirements. The pixel values ​​may be different from the pixel values ​​corresponding to the boundaries of the image to be input, so as to distinguish the added pixels. The size of the gray image is consistent with the size of the first image obtained after filling.

[0065] Optionally, the position of the region in the gray image corresponding to the image to be input can be set according to actual needs, as long as the entire region is located in the gray image.

[0066] In one embodiment, the width and height of the gray image are (R w +add w ,R h +add h ), the width and height of the scaled input image are (R w ,R h ), the width and height of the input image is less than or equal to the width and height of the gray image, that is, the coordinates of the upper left corner of the image after scaling (x 1 ,y 1 ) can be in the x coordinates of the gray image 1 ∈[0,add w ],y 1 ∈[0,add h ] move, so that the gray image [x 1 :x 1 +R w ,y 1 :y 1 +R h ] area is equal to the pixel value of the image to be input (that is, the pixel value of the pixel point in the area corresponds one to one to the pixel value of the pixel point in the image to be input), and the first image is obtained.

[0067] Optionally, the acquired first image is input into a cutout model, and the cutout model recognizes the target object in the first image and outputs an image matrix corresponding to the first image according to the recognition result.

[0068] In one embodiment, the target object is a portrait, and the image matrix is ​​an Alpha matrix α of the portrait. The size of the matrix is ​​consistent with the size of the first image. The values ​​of the elements in the matrix α are between 0 and 1 (including 0 and 1), the part corresponding to the portrait is 1, and the part corresponding to the background is 0. The Alpha matrix can be regarded as the probability that each pixel point of the first image corresponds to a person, the maximum is 1, that is, a person, 0 is the background, and the pixel point connecting the person and the background is a little fuzzy, so it is the part to be determined, which is between 0 and 1.

[0069] S102: Expanding the image matrix according to the original image to obtain a first matrix.

[0070] Optionally, the image matrix is ​​expanded according to the original image to obtain the first matrix, including: if it is determined that the first image includes a filling area, a portion of the image matrix corresponding to the filling area is eliminated, and the eliminated image matrix is ​​expanded according to the size of the original image to obtain the first matrix.

[0071] Optionally, the position information of the area in the first image corresponding to the image to be input may be pre-stored, and the portion of the image matrix corresponding to the filling area may be removed according to the position information and the correspondence between the first image and the image matrix.

[0072] Optionally, when the first image does not include a filling area, the image matrix may be directly enlarged according to the size of the original image to obtain the first matrix.

[0073] Optionally, the size of the first matrix is ​​consistent with the size of the pixel matrix corresponding to the original image.

[0074] Optionally, the image matrix may be enlarged by a bilinear interpolation method, or may be interpolated using a cubic spline interpolation method, a nearest neighbor interpolation method, a bicubic interpolation method, a Lanczos interpolation method, or other interpolation methods to enlarge the image matrix.

[0075] S103: Convert the first matrix according to the image matrix and the threshold value to obtain a target matrix.

[0076] Optionally, converting the first matrix according to the image matrix and the threshold to obtain a target matrix includes:

[0077] Based on formula (1), the first matrix is ​​transformed to obtain the target matrix;

[0078]

[0079] Among them, Threshold is the threshold, α is the first matrix, A is the target matrix, and n represents the degree of transformation.

[0080] Optionally, the formula used to transform the first matrix may also be other formulas that can effectively determine the boundary between the target object and the background and achieve edge smoothing between the target object and the background.

[0081] Optionally, the threshold is determined based on the cutout information of the cutout model, and the threshold can be determined according to the type of the cutout model and the type of the target object. Alternatively, after obtaining the image matrix, the cutout result corresponding to the image matrix is ​​fed back to a preset object, threshold adjustment information is obtained based on the feedback information of the preset object, and the threshold is adjusted according to the threshold adjustment information. The proportion of elements in the image matrix that cannot accurately indicate the connection between the target object and the background can also be obtained, the deviation between the proportion and the preset proportion is obtained, and the threshold is adjusted based on the deviation.

[0082] In one embodiment, the threshold value ranges from 0 to 1, and can be 0.5. n represents the degree of transformation. The function of this formula is to move the value of α closer to 0 and 1. When the value of α (here α refers to the value of the matrix α above) is less than Threshold, A approaches 0; when the value of α is greater than Threshold, A approaches 1. This formula can make the blurred boundary between the target object and the background clearer.

[0083] Optionally, both Threshold and n can be adjusted according to the cutout effect, or can be a preset value. Among them, the adjustment of the Threshold value can determine the boundary between the target object and the background, and the value is between [0,1]. If the Threshold is too large, part of the edge of the target object will be segmented. If the Threshold is too small, there will be a circle of background around the edge of the target object, resulting in a poor effect. For n, the value should be greater than or equal to 1. The role of n is to smooth the edge of the target object and the background. The larger n is (too convergent), the worse the smoothing effect is, and the effect is closer to the effect of binarization (either 1 or 0). If n is too small (not convergent enough), the transformation effect of α will be poor or even invisible. After outputting the image matrix, the cutout image corresponding to the image matrix can also be output, and feedback information from the user based on the cutout image can be obtained, and Threshold and n can be adjusted according to the feedback information.

[0084] S104: Acquire a target object in the original image based on the target matrix and the original image.

[0085] Optionally, acquiring the target object in the original image based on the target matrix and the original image includes: acquiring a pixel matrix of the original image; and multiplying the pixel matrix by the target matrix to obtain an image of the target object.

[0086] In one embodiment, the target object is a portrait, a pixel matrix corresponding to the original image is obtained, an element of the pixel matrix includes pixel value information of a pixel point in the original image, the pixel matrix is ​​multiplied by the target matrix to obtain an image of the portrait, and the image of the portrait is output.

[0087] In an optional embodiment, an electronic device is provided, such as Figure 3 As shown, Figure 3 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0088] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0089] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0090] Memory 4003 can be ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or it can be EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disc, optical disk, digital versatile disc, Blu-ray disc, etc.), magnetic disk storage medium, other magnetic storage devices or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation here.

[0091] The memory 4003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the above method embodiment.

[0092] Among them, the electronic device can be any electronic product that can interact with an object, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0093] The electronic device may also include a network device and / or an object device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers for cloud computing.

[0094] The network where the electronic device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (Virtual Private Network, VPN), etc.

[0095] An embodiment of the present application provides a computer-readable storage medium, which includes a computer program. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0096] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0097] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described in the drawings.

[0098] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0099] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. A method for cutting out a picture, characterized in that: include: Acquire an image matrix output by a cutout model corresponding to an input first image, wherein the image matrix includes indication information indicating a target object in the first image, and the first image is obtained by processing an original image based on information of the cutout model; The information of the cutout model includes image input information and sampling information. Processing the original image based on the information of the cutout model includes: Determine the length of the longest side of the original image after scaling according to the image input information, and determine the scaling ratio based on the length and the width and height of the original image; scaling the original image in proportion to the scaling ratio; Acquire side length information of the input image according to the sampling information; Filling the image to be input corresponding to the original image based on the side length information to obtain the first image, where the image to be input is determined based on the image input information; Expanding the image matrix according to the original image to obtain a first matrix includes: If it is determined that the first image includes a filling area, a portion of the image matrix corresponding to the filling area of ​​the first image is removed, and the image matrix after the filling area is removed is expanded according to the size of the original image to obtain a first matrix, where the size of the first matrix is ​​consistent with the size of the pixel matrix corresponding to the original image; If it is determined that the first image does not include a filling area, expanding the image matrix according to the size of the original image to obtain a first matrix; Converting the first matrix according to the image matrix and a threshold value to obtain a target matrix, wherein the threshold value is determined based on the cutout information of the cutout model; The step of converting the first matrix according to the image matrix and a threshold value to obtain a target matrix includes: Convert the first matrix based on formula (1) to obtain a target matrix; （1） in, is the threshold, α is the first matrix, A is the target matrix, and n represents the degree of transformation; A target object in the original image is acquired based on the target matrix and the original image.

2. The cutout method according to claim 1, characterized in that: The filling the image to be input corresponding to the original image based on the side length information includes: Acquire filling information of the image to be input according to the side length information, and create a gray image based on the filling information; The pixel values ​​of the area in the gray image corresponding to the image to be input are determined as the pixel values ​​of the image to be input.

3. The cutout method according to claim 1, characterized in that: The acquiring the target object in the original image based on the target matrix and the original image comprises: Obtaining a pixel matrix of the original image; The pixel matrix is ​​dot-multiplied by the target matrix to obtain an image of the target object.

4. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 3.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

6. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images