An image processing method, apparatus, electronic device, and storage medium

Abstract

The application provides an image processing method and device, electronic equipment and storage medium, the method comprises: obtaining an accessory image and a human body image; determining a transformation matrix according to a plurality of coordinate points of the accessory image and a plurality of coordinate points of the human body image; superimposing the accessory image on the human body image using the transformation matrix. In the implementation process of the above scheme, the transformation matrix is determined according to the plurality of coordinate points of the accessory image and the plurality of coordinate points of the human body image, and the accessory image is superimposed on the human body image using the transformation matrix, thereby improving the low efficiency of manually producing the accessory image and improving the efficiency of producing the accessory image.

Description

Technical Field

[0001] This application relates to the technical fields of image processing, target detection, and target recognition, and more specifically, to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, the creation of accessory images (such as handbag images) is mostly done manually. For example, customers might wear new accessories and take photos in front of a mirror, or family or friends might take photos of customers wearing new accessories. Another example is staff photographing or editing mannequins wearing new accessories, then using the edited images as product detail images on the accessory's webpage. However, in practice, when a large number of accessory images need to be produced, it's difficult to create a large number of product detail images in a timely manner. In other words, creating accessory images manually is inefficient. Summary of the Invention

[0003] The purpose of this application is to provide an image processing method, apparatus, electronic device, and storage medium to improve the low efficiency of creating accessory images.

[0004] This application provides an image processing method, including: acquiring an accessory image and a human body image; determining a transformation matrix based on multiple coordinate points of the accessory image and multiple coordinate points of the human body image; and superimposing the accessory image onto the human body image using the transformation matrix. In the implementation of the above scheme, by determining the transformation matrix based on multiple coordinate points of the accessory image and multiple coordinate points of the human body image, and using the transformation matrix to superimpose the accessory image onto the human body image, the low efficiency of manually creating accessory images is improved, thereby increasing the efficiency of accessory image creation.

[0005] Optionally, in this embodiment, the accessory image is a handbag image; obtaining the accessory image and the human body image includes: obtaining the image to be processed, using a neural network model to perform target detection on the image to be processed, and obtaining the handbag image and / or the human body image. In the implementation of the above scheme, by using a neural network model to perform target detection on the image to be processed to obtain the handbag image and / or the human body image, the problem of background in the image to be processed affecting the overlay effect of the accessory image is improved, effectively improving the accuracy of creating the accessory image.

[0006] Optionally, in this embodiment, the multiple coordinate points of the handbag image include: a first coordinate point, a second coordinate point, and a third coordinate point that are not collinear; the multiple coordinate points of the human body image include: a fourth coordinate point, a fifth coordinate point, and a sixth coordinate point that are not collinear; before determining the transformation matrix based on the multiple coordinate points of the accessory image and the multiple coordinate points of the human body image, the method further includes: identifying the shoulder strap region and the bag body region from the handbag image, and identifying the shoulder point in a first direction and the waist point in a second direction from the human body image, wherein the first direction and the second direction are opposite; determining the shoulder strap vertex of the shoulder strap region as the first coordinate point, and determining the connection point between the shoulder strap region and the bag body region as the second coordinate point; then, selecting the third coordinate point from the handbag image; determining the shoulder point in the first direction as the fourth coordinate point, and determining the waist point in the second direction as the fifth coordinate point; then, selecting the sixth coordinate point from the human body image.

[0007] Optionally, in this embodiment, selecting a third coordinate point from the handbag image includes: if the origin of the coordinate axis of the handbag image is not collinear with the first and second coordinate points, then the origin of the coordinate axis is used as the third coordinate point. In the implementation of the above scheme, by using the origin of the coordinate axis as the third coordinate point, the calculation process is simplified and the efficiency of creating accessory images is improved because the calculation of the origin of the coordinate axis is very simple.

[0008] Optionally, in this embodiment of the application, selecting a third coordinate point from the bag image includes: using the abscissa value of the first coordinate point as the abscissa value of the third coordinate point, and using the ordinate value of the second coordinate point as the ordinate value of the third coordinate point to determine the third coordinate point; or, using the abscissa value of the second coordinate point as the abscissa value of the third coordinate point, and using the ordinate value of the first coordinate point as the ordinate value of the third coordinate point to determine the third coordinate point.

[0009] Optionally, in this embodiment, the transformation matrix is ​​an affine matrix; superimposing the accessory image onto the human body image using the transformation matrix includes: performing an affine transformation on the shoulder strap region using the affine matrix to obtain a shoulder strap affine image, and performing an affine transformation on the bag body region using the affine matrix to obtain a bag body affine image; stitching the shoulder strap affine image and the bag body affine image together to obtain a shoulder bag affine image; and superimposing the shoulder bag affine image onto the human body image. In the implementation of the above solution, by stitching the shoulder strap affine image and the bag body affine image together to obtain a shoulder bag affine image, and then superimposing the shoulder bag affine image onto the human body image, the inefficiency of manually creating accessory images is improved, thereby increasing the efficiency of accessory image creation.

[0010] Optionally, in this embodiment, the transformation matrix is ​​a perspective matrix; using the transformation matrix to overlay the accessory image onto the human body image includes: using the perspective matrix to perform perspective transformation on the shoulder strap area to obtain a shoulder strap perspective image, and using the perspective matrix to perform perspective transformation on the bag body area to obtain a bag body perspective image; stitching the shoulder strap perspective image and the bag body perspective image together to obtain a shoulder bag perspective image; and overlaying the shoulder bag perspective image onto the human body image.

[0011] Optionally, in embodiments of this application, the human body image includes: a virtual model image or a real customer image.

[0012] This application also provides an image processing apparatus, including: an accessory image acquisition module for acquiring an accessory image and a human body image; a transformation matrix determination module for determining a transformation matrix based on multiple coordinate points of the accessory image and multiple coordinate points of the human body image; and an accessory image overlay module for overlaying the accessory image onto the human body image using the transformation matrix.

[0013] Optionally, in this embodiment of the application, the accessory image is a handbag image; the accessory image acquisition module includes: an image target detection submodule, used to acquire the image to be processed, use a neural network model to perform target detection on the image to be processed, and obtain the handbag image and / or human body image.

[0014] Optionally, in this embodiment, the multiple coordinate points of the bag image in the image processing device include: a first coordinate point, a second coordinate point, and a third coordinate point that are not collinear; the multiple coordinate points of the human body image include: a fourth coordinate point, a fifth coordinate point, and a sixth coordinate point that are not collinear; the image processing device further includes: a bag image recognition module, used to identify the shoulder strap area and the bag body area from the bag image, and to identify the shoulder point in a first direction and the waist point in a second direction from the human body image, wherein the first direction and the second direction are opposite; a first coordinate filtering module, used to determine the shoulder strap vertex of the shoulder strap area as the first coordinate point, and to determine the connection point between the shoulder strap area and the bag body area as the second coordinate point, and then to filter out the third coordinate point from the bag image; a second coordinate filtering module, used to determine the shoulder point in the first direction as the fourth coordinate point, and to determine the waist point in the second direction as the fifth coordinate point, and then to filter out the sixth coordinate point from the human body image.

[0015] Optionally, in an embodiment of this application, the first coordinate filtering module includes: a first coordinate determination submodule, used to take the origin of the coordinate axis as the third coordinate point if the origin of the coordinate axis of the bag image is not collinear with the first coordinate point and the second coordinate point.

[0016] Optionally, in this embodiment of the application, the first coordinate filtering module includes: a second coordinate determination submodule, used to determine a third coordinate point by using the abscissa value of the first coordinate point as the abscissa value of the third coordinate point and the ordinate value of the second coordinate point as the ordinate value of the third coordinate point; and a third coordinate determination submodule, used to determine a third coordinate point by either using the abscissa value of the second coordinate point as the abscissa value of the third coordinate point and the ordinate value of the first coordinate point as the ordinate value of the third coordinate point.

[0017] Optionally, in this embodiment, the transformation matrix is ​​an affine matrix; the accessory image overlay module includes: a region affine transformation submodule, used to perform an affine transformation on the shoulder strap region using an affine matrix to obtain a shoulder strap affine image, and to perform an affine transformation on the bag body region using an affine matrix to obtain a bag body affine image; an affine image acquisition submodule, used to stitch the shoulder strap affine image and the bag body affine image together to obtain a shoulder bag affine image; and an affine image overlay submodule, used to overlay the shoulder bag affine image onto the human body image.

[0018] Optionally, in this embodiment, the transformation matrix is ​​a perspective matrix; the accessory image overlay module includes: a region perspective transformation submodule, used to perform perspective transformation on the shoulder strap region using the perspective matrix to obtain a shoulder strap perspective image, and to perform perspective transformation on the bag body region using the perspective matrix to obtain a bag body perspective image; a perspective image acquisition submodule, used to stitch the shoulder strap perspective image and the bag body perspective image together to obtain a shoulder bag perspective image; and a perspective image overlay submodule, used to overlay the shoulder bag perspective image onto the human body image.

[0019] Optionally, in embodiments of this application, the human body image includes: a virtual model image or a real customer image.

[0020] This application also provides an electronic device, including a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions, when executed by the processor, perform the method described above.

[0021] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor to perform the methods described above.

[0022] Other features and advantages of embodiments of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of this application. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 The illustration shown is a schematic flowchart of the image processing method provided in an embodiment of this application;

[0025] Figure 2 The illustration shows a schematic diagram of the overlay process of accessory images and human body images provided in an embodiment of this application;

[0026] Figure 3 The diagram shown is a schematic representation of the target detection process provided in an embodiment of this application.

[0027] Figure 4 The diagram shows three implementation schemes for filtering coordinate points provided in the embodiments of this application;

[0028] Figure 5 The diagram shown is a structural schematic of the image processing apparatus provided in an embodiment of this application;

[0029] Figure 6 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in the embodiments of this application are for illustrative and descriptive purposes only and are not intended to limit the protection scope of the embodiments of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in the embodiments of this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of the embodiments of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0031] Furthermore, the described embodiments are merely a part of the embodiments of this application, and not all of them. The components of the embodiments of this application described and illustrated herein can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed embodiments of this application, but merely to illustrate selected embodiments of this application.

[0032] It is understood that the terms "first" and "second" in the embodiments of this application are used to distinguish similar objects. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" are not necessarily different. In the description of the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. The term "multiple" refers to two or more (including two), and similarly, "multiple groups" refers to two or more groups (including two groups).

[0033] Before introducing the image processing method provided in the embodiments of this application, let's first introduce some concepts involved in the embodiments of this application:

[0034] Image processing refers to the techniques of analyzing, processing, and manipulating images to meet visual, psychological, or other requirements. Image processing is an application of signal processing in the field of images.

[0035] It should be noted that the image processing method provided in this application embodiment can be executed by an electronic device. Here, an electronic device refers to a device terminal or server with the function of executing computer programs. Device terminals include, for example, smartphones, personal computers, tablets, personal digital assistants, or mobile internet devices. A server refers to a device that provides computing services through a network. Servers include, for example, x86 servers and non-x86 servers. Non-x86 servers include, for example, mainframes, minicomputers, and UNIX servers.

[0036] The following describes the applicable scenarios for this image processing method, including but not limited to: product webpage creation for online accessory stores, new accessory product demonstrations in shopping malls, and video editing for mobile live streams. In the application scenario of creating product webpages for online accessory stores, this image processing method can be used to create images of live or virtual models wearing new accessories. In the application scenario of demonstrating new accessory products in shopping malls, this image processing method can be used to enhance the functionality of electronic mirrors in shopping malls. For example, if a new accessory product (such as a new handbag or crossbody accessory) is about to be launched, but the mall does not have a ready-made product for customers to try on, this image processing method can be used to create images of customers trying on the accessory. Specifically, at least one camera can be installed on the electronic mirror. With the customer's consent, the camera can acquire the customer's human image. Then, the image processing method is used to overlay the accessory image onto the human image, and the overlaid human image is output on the electronic mirror, allowing the customer to see themselves trying on the accessory. In the application scenarios of mobile live video editing, this image processing method can be used to modify the host or guests in the mobile live broadcast, such as overlaying an accessory image (such as a new handbag or sash, etc.) on the host or guests.

[0037] Please see Figure 1 The illustrated diagram shows a flowchart of an image processing method provided in an embodiment of this application. The idea behind this image processing method is to determine a transformation matrix based on multiple coordinate points of an accessory image and multiple coordinate points of a human body image, and then use the transformation matrix to superimpose the accessory image onto the human body image, thereby improving the efficiency of creating accessory images. Implementations of the above-described image processing method may include:

[0038] Step S110: Obtain the accessory image and the human body image.

[0039] Accessory images refer to images of items used to accessorize, wear, or adorn the human body. These accessories include, but are not limited to, crossbody bags, laptop bags, and wallets, as well as embroidered ribbons or sashes. It is understood that the aforementioned human body images can include images of virtual models or real customers.

[0040] Step S120: Determine the transformation matrix based on multiple coordinate points of the accessory image and multiple coordinate points of the human body image.

[0041] Understandably, the transformation parameters can be calculated using three coordinate point pairs. Then, by filling the transformation matrix with these parameters, the transformation matrix described above can be obtained. Similarly, multiple transformation parameters in the transformation formula can be calculated using more coordinate point pairs, and the average of these multiple transformation parameters can be filled into the transformation matrix to obtain the transformation matrix described above. This transformation matrix can be either a perspective matrix or an affine matrix.

[0042] Step S130: Use a transformation matrix to overlay the accessory image onto the human body image.

[0043] Please see Figure 2 The illustration shows a schematic diagram of the overlay process of an accessory image and a human body image provided in an embodiment of this application; it can be understood that after using a transformation matrix to overlay the accessory image onto the human body image, the overlaid human body image can be obtained.

[0044] In the above implementation process, by determining the transformation matrix based on multiple coordinate points of the accessory image and multiple coordinate points of the human body image, and using the transformation matrix to superimpose the accessory image onto the human body image, the inefficiency of manually creating accessory images is improved, thereby increasing the efficiency of accessory image creation.

[0045] Please see Figure 3 The illustration shows a schematic diagram of the target detection process provided in the embodiment of this application; as an optional implementation of the above-mentioned step S110, the accessory image can be a handbag image, or of course, other accessory images. Here, a handbag image is used as an example for explanation; the above-mentioned implementation of obtaining the accessory image and the human body image may include:

[0046] Step S111: Obtain the image to be processed.

[0047] The methods for acquiring the image to be processed in step S111 above include: First, using a terminal device such as a camera, video recorder, or color camera to capture the target object and acquire the image to be processed; then, the terminal device sends the image to be processed to an electronic device, which receives the image sent by the terminal device and can store the image to be processed in a file system, database, or mobile storage device; Second, acquiring a pre-stored image to be processed, specifically, for example, acquiring the image from a file system, database, or mobile storage device; Third, using software such as a browser to acquire the image to be processed from the Internet, or using other applications to access the Internet to acquire the image to be processed.

[0048] Step S112: Use a neural network model to perform target detection on the image to be processed, and obtain the bag image and / or human image.

[0049] The implementation of step S112 above can be exemplified by using a single-stage or two-stage detection model of a neural network to detect objects in the image to be processed. It is understood that the image to be processed can include three types: the first type includes only a bag image; the second type includes only a human image; and the third type includes both a bag image and a human image. Similarly, the obtained bag image and / or human image can be categorized into three cases: the first type detects a bag image from an image to be processed that only includes a bag image; the second type detects a human image from an image to be processed that only includes a human image; and the third type detects both a bag image and a human image from an image to be processed that includes both. Specific examples of single-stage detection models that can be used include Feature Fusion Single Shot multi-box Detector (FSSD) and YOLO networks. Specific examples of two-stage detection models that can be used include Region Convolutional Neural Network (RCNN), Fast RCNN, and Faster RCNN series network models.

[0050] As an optional implementation of step S120 above, the multiple coordinate points of the handbag image may include: a first coordinate point, a second coordinate point, and a third coordinate point that are not collinear; the multiple coordinate points of the human body image may include: a fourth coordinate point, a fifth coordinate point, and a sixth coordinate point that are not collinear; before determining the transformation matrix based on the multiple coordinate points of the accessory image and the multiple coordinate points of the human body image, the method further includes:

[0051] Step S121: Identify the shoulder strap area and bag body area from the bag image, and identify the shoulder point in the first direction and the waist point in the second direction from the human body image. The first direction and the second direction are opposite.

[0052] For example, the above-mentioned step S121 can be implemented by using network models of RCNN, fast RCNN and faster RCNN series to identify the shoulder strap region and bag body region from the bag image, and to identify the shoulder point in the first direction and the waist point in the second direction from the human body image, wherein the first direction and the second direction are opposite.

[0053] Step S122: Determine the first coordinate point as the apex of the shoulder strap area, and determine the connection point between the shoulder strap area and the bag body area as the second coordinate point. Then, select the third coordinate point from the bag image.

[0054] Step S123: Determine the shoulder point in the first direction as the fourth coordinate point, and the waist point in the second direction as the fifth coordinate point. Then, select the sixth coordinate point from the human body image.

[0055] An example implementation of step S123 above involves using an executable program compiled or interpreted in a preset programming language to calculate a fourth coordinate point based on the shoulder point in the first direction, and a fifth coordinate point based on the waist point in the second direction. Then, a sixth coordinate point is selected from the human body image. Programming languages ​​that can be used include, for example, C, C++, Java, BASIC, JavaScript, LISP, Shell, Perl, Ruby, Python, and PHP. There are many ways to select the sixth coordinate point from the human body image; the origin of the coordinate axes can be used as the sixth coordinate point, or other coordinate points can be used. This process will be described in detail below.

[0056] Please see Figure 4 The diagram shows three implementations of the filtering coordinate points provided in this application embodiment; as a first optional implementation of step S122 above, the implementation of filtering the third coordinate point from the bag image may include:

[0057] Step S122a: If the origin of the coordinate axis of the bag image is not collinear with the first coordinate point and the second coordinate point, then the origin of the coordinate axis of the bag image is taken as the third coordinate point.

[0058] Step S122b: If the origin of the coordinate axis of the human body image is not collinear with the fourth and fifth coordinate points, then the origin of the coordinate axis of the human body image is taken as the sixth coordinate point.

[0059] It's understandable that the origin of the coordinate axis in the bag image is a known coordinate (0, 0). Therefore, by taking the origin of the coordinate axis in the bag image as the third coordinate point, we can deduce that the coordinates of the third coordinate point are (0, 0). Similarly, the origin of the coordinate axis in the human body image is also a known coordinate (0, 0). Therefore, by taking the origin of the coordinate axis in the human body image as the sixth coordinate point, we can deduce that the coordinates of the sixth coordinate point are also (0, 0).

[0060] As a second optional implementation of step S122 above, the implementation of filtering out the third coordinate point from the bag image may include:

[0061] Step S122c: Use the x-coordinate value of the first coordinate point as the x-coordinate value of the third coordinate point, and use the y-coordinate value of the second coordinate point as the y-coordinate value of the third coordinate point to determine the third coordinate point.

[0062] Step S122d: Use the x-coordinate of the fourth coordinate point as the x-coordinate of the sixth coordinate point, and use the y-coordinate of the fifth coordinate point as the y-coordinate of the sixth coordinate point to determine the sixth coordinate point.

[0063] For example, the implementation of steps S122c to S122d above can be understood as follows: The distance between the first coordinate point and the second coordinate point is approximately the shoulder width of the human body in the human body image, which can be represented as W. If the human body image is a customer image, there is a size difference between the human body image and the actual customer; the size scale here can be represented as S. Assuming the total length of the shoulder strap is L, then it can be calculated using the formula... Calculate the shoulder strap half length; where L half This indicates the half-length of the shoulder strap, L indicates the total length of the shoulder strap, T indicates the chest span in the anthropometric image, and S indicates the scale between the anthropometric image and the actual customer. Then, according to... The formula calculates the ordinate value of the second coordinate point, where B y A represents the ordinate value of the second coordinate point. y L represents the ordinate value of the first coordinate point. half W represents the shoulder strap half-length, and W represents the width of the human shoulder in the human image.

[0064] As a third optional implementation of step S122 above, the implementation of filtering the third coordinate point from the bag image may include:

[0065] Step S122e: Use the x-coordinate value of the second coordinate point as the x-coordinate value of the third coordinate point, and use the y-coordinate value of the first coordinate point as the y-coordinate value of the third coordinate point to determine the third coordinate point.

[0066] Step S122f: Use the x-coordinate of the fifth coordinate point as the x-coordinate of the sixth coordinate point, and use the y-coordinate of the fourth coordinate point as the y-coordinate of the sixth coordinate point to determine the sixth coordinate point.

[0067] The implementation principles and methods of steps S122e to S122f ​​are similar to those of steps S122c to S122d. Therefore, their implementation principles and methods will not be described here. If there is anything unclear, please refer to the description of steps S122c to S122d.

[0068] Understandably, after determining multiple coordinate points in both the accessory image and the human body image—that is, after selecting three or more coordinate points from both the accessory and human body images—then the formula for affine transformation can be applied. The affine transformation parameters can be obtained through calculation; these parameters include: cos(θ), -sin(θ), sin(θ), cos(θ), and t. x and t y θ represents the rotation angle, t x t represents the offset of the horizontal axis. y The offset of the vertical axis is represented by x and y, which represent the coordinates of the points in the bag image (including the first, second, and third coordinates mentioned above, etc.), and x' and y' represent the coordinates of the points in the human body image (including the fourth, fifth, and sixth coordinates, etc.). Then, based on these affine transformation parameters (including: cos(θ), -sin(θ), sin(θ), cos(θ), t...), the coordinates are calculated... x and t y The affine matrix described above can then be determined. The specific values ​​are then determined, and finally, an affine matrix is ​​used to overlay the accessory image onto the human body image.

[0069] As an optional implementation of step S130 above, the transformation matrix can be an affine matrix. Therefore, the implementation of superimposing the accessory image onto the human body image using the transformation matrix can include:

[0070] Step S131: Perform an affine transformation on the shoulder strap region using an affine matrix to obtain an affine image of the shoulder strap, and perform an affine transformation on the bag body region using an affine matrix to obtain an affine image of the bag body.

[0071] Step S132: Stitch together the affine image of the shoulder strap and the affine image of the bag body to obtain the affine image of the shoulder bag.

[0072] Step S133: Overlay the affine image of the handbag onto the human body image.

[0073] The implementation of steps S131 to S133 above is as follows: Using the `getAffineTransform` function interface in the Open Source Computer Vision Library (OpenCV), an affine transformation is performed on the shoulder strap region using an affine matrix as a parameter to obtain an affine image of the shoulder strap. Then, an affine transformation is performed on the bag body region using the same affine matrix to obtain an affine image of the bag body. Next, an executable program compiled or interpreted using a preset programming language is used to stitch the shoulder strap affine image and the bag body affine image together to obtain an affine image of the shoulder bag. Finally, the OpenCV library is used to overlay the shoulder bag affine image onto the human body image.

[0074] Similarly, when determining multiple coordinate points in the accessory image and the human body image, the specific criteria can be used. For example, four or more coordinate points can be selected from the accessory image (assuming the shoulder strap area is a rectangle in the image, then the upper left, upper right, lower left, and lower right coordinate points of the shoulder strap area can be selected) and four or more coordinate points can be selected from the human body image. After selecting four or more coordinate points from both the accessory and human body images, the perspective transformation formula can be applied. Calculations are performed to obtain the perspective transformation parameters, which include a0, a1, a2, a3, b0, b1, c0, and c1. x and y represent the coordinates of points in the bag image, and x' and y' represent the coordinates of points in the human body image. Then, based on these perspective transformation parameters (a0, a1, a2, a3, b0, b1, c0, and c1), the aforementioned perspective matrix can be determined. The specific values ​​are then determined, and finally, the accessory image is overlaid onto the human body image using a perspective matrix.

[0075] As another optional implementation of step S130 above, the transformation matrix can be a perspective matrix. Therefore, the implementation of superimposing the accessory image onto the human body image using the transformation matrix can include:

[0076] Step S134: Use a perspective matrix to perform perspective transformation on the shoulder strap area to obtain a perspective image of the shoulder strap, and use a perspective matrix to perform perspective transformation on the bag body area to obtain a perspective image of the bag body.

[0077] Step S135: Combine the shoulder strap perspective image and the bag body perspective image to obtain the shoulder bag perspective image.

[0078] Step S136: Overlay the perspective image of the handbag onto the human body image.

[0079] For example, steps S134 to S136 described above can be implemented as follows: The perspective matrix and the shoulder strap region are used as parameters for the `warpPerspective` function in the OpenCV open-source computer vision library. Specifically, the `warpPerspective` function interface in OpenCV uses the perspective matrix as a parameter to perform perspective transformation on the shoulder strap region, obtaining a shoulder strap perspective image. Then, the perspective matrix is ​​used to perform perspective transformation on the bag body region, obtaining a bag body perspective image. Next, an executable program compiled or interpreted using a preset programming language is used to stitch the shoulder strap perspective image and the bag body perspective image together to obtain a shoulder bag perspective image. Finally, the OpenCV library is used to overlay the shoulder bag perspective image onto the human body image.

[0080] As an optional implementation of the above image processing method, the human body image may include: virtual model image or real customer image, etc.

[0081] Please see Figure 5 The diagram shown is a structural schematic of an image processing apparatus provided in an embodiment of this application; this application provides an image processing apparatus 200, including:

[0082] Accessory image acquisition module 210 is used to acquire accessory images and human body images.

[0083] The transformation matrix determination module 220 is used to determine the transformation matrix based on multiple coordinate points of the accessory image and multiple coordinate points of the human body image.

[0084] Accessory image overlay module 230 is used to overlay an accessory image onto a human body image using a transformation matrix.

[0085] Optionally, in this embodiment of the application, the accessory image is a handbag image; the accessory image acquisition module includes:

[0086] The image target detection submodule is used to acquire the image to be processed, and use a neural network model to perform target detection on the image to obtain the bag image and / or human image.

[0087] Optionally, in this embodiment, the multiple coordinate points of the bag image in the image processing device include: a first coordinate point, a second coordinate point, and a third coordinate point that are not collinear; the multiple coordinate points of the human body image include: a fourth coordinate point, a fifth coordinate point, and a sixth coordinate point that are not collinear; the image processing device further includes:

[0088] The shoulder bag image recognition module is used to identify the shoulder strap area and bag body area from the shoulder bag image, and to identify the shoulder point in the first direction and the waist point in the second direction from the human body image. The first direction and the second direction are opposite.

[0089] The first coordinate filtering module is used to determine the first coordinate point as the vertex of the shoulder strap area, and the second coordinate point as the connection point between the shoulder strap area and the bag body area. Then, the third coordinate point is filtered out from the bag image.

[0090] The second coordinate filtering module is used to determine the shoulder point in the first direction as the fourth coordinate point, and the waist point in the second direction as the fifth coordinate point. Then, it filters out the sixth coordinate point from the human body image.

[0091] Optionally, in this embodiment of the application, the first coordinate filtering module includes:

[0092] The first coordinate determination submodule is used to take the origin of the coordinate axis as the third coordinate point if the origin of the coordinate axis of the bag image is not collinear with the first coordinate point and the second coordinate point.

[0093] Optionally, in this embodiment of the application, the first coordinate filtering module includes:

[0094] The second coordinate determination submodule is used to determine the third coordinate point by using the x-coordinate value of the first coordinate point as the x-coordinate value of the third coordinate point and the y-coordinate value of the second coordinate point as the y-coordinate value of the third coordinate point.

[0095] The third coordinate determination submodule is used to determine the third coordinate point by either using the x-coordinate value of the second coordinate point as the x-coordinate value of the third coordinate point and using the y-coordinate value of the first coordinate point as the y-coordinate value of the third coordinate point.

[0096] Optionally, in this embodiment, the transformation matrix is ​​an affine matrix; the accessory image overlay module includes:

[0097] The Region Affine Transformation submodule is used to perform affine transformation on the shoulder strap region using an affine matrix to obtain the shoulder strap affine image, and to perform affine transformation on the bag body region using an affine matrix to obtain the bag body affine image.

[0098] The affine image acquisition submodule is used to stitch together the affine images of the shoulder strap and the bag body to obtain the affine image of the shoulder bag.

[0099] The affine image overlay submodule is used to overlay the affine image of a handbag onto a human body image.

[0100] Optionally, in this embodiment, the transformation matrix is ​​a perspective matrix; the accessory image overlay module includes:

[0101] The Region Perspective Transformation submodule is used to perform perspective transformation on the shoulder strap region using a perspective matrix to obtain a perspective image of the shoulder strap, and to perform perspective transformation on the bag body region using a perspective matrix to obtain a perspective image of the bag body.

[0102] The perspective image acquisition submodule is used to stitch together the shoulder strap perspective image and the bag body perspective image to obtain the shoulder bag perspective image.

[0103] The perspective image overlay submodule is used to overlay the perspective image of a handbag onto a human body image.

[0104] Optionally, in embodiments of this application, the human body image includes: a virtual model image or a real customer image.

[0105] It should be understood that this device corresponds to the image processing method embodiments described above and is capable of performing the various steps involved in the above method embodiments. The specific functions of this device can be found in the description above, and detailed descriptions are omitted here to avoid repetition. The device includes at least one software functional module that can be stored in memory or embedded in the device's operating system (OS) in the form of software or firmware.

[0106] Please see Figure 6 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. An electronic device 300 provided in this application includes a processor 310 and a memory 320. The memory 320 stores machine-readable instructions executable by the processor 310. When the machine-readable instructions are executed by the processor 310, the method described above is performed.

[0107] This application embodiment also provides a computer-readable storage medium 330, on which a computer program is stored. This computer program is executed by a processor 310 to perform the methods described above. The computer-readable storage medium 330 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0108] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0109] It should be understood that the disclosed apparatus and methods can also be implemented in other ways, as provided in the embodiments of this application. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending primarily on the functions involved.

[0110] Furthermore, the functional modules of each embodiment in this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. In addition, in the description of this specification, the reference to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., means that the specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0111] The above description is only an optional implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application.

Claims

1. An image processing method, characterized in that, include: Acquire images of accessories and human figures; The transformation matrix is ​​determined based on multiple coordinate points of the accessory image and multiple coordinate points of the human body image; The accessory image is superimposed onto the human body image using the transformation matrix; Wherein, the accessory image is a handbag image; the acquisition of the accessory image and the human body image includes: Acquire the image to be processed, and use a neural network model to perform target detection on the image to obtain the bag image and / or the human body image; The multiple coordinate points of the handbag image include: a first coordinate point, a second coordinate point, and a third coordinate point that are not collinear; the multiple coordinate points of the human body image include: a fourth coordinate point, a fifth coordinate point, and a sixth coordinate point that are not collinear; before determining the transformation matrix based on the multiple coordinate points of the accessory image and the multiple coordinate points of the human body image, the method further includes: The shoulder strap area and bag body area are identified from the bag image, and the shoulder point in a first direction and the waist point in a second direction are identified from the human body image, wherein the first direction and the second direction are opposite. The first coordinate point is determined by identifying the apex of the shoulder strap area and the connection point between the shoulder strap area and the bag body area as the second coordinate point. Then, the third coordinate point is selected from the bag image. The shoulder point in the first direction is determined as the fourth coordinate point, and the waist point in the second direction is determined as the fifth coordinate point. Then, the sixth coordinate point is selected from the human body image.

2. The method according to claim 1, characterized in that, The step of filtering the third coordinate point from the image of the handbag includes: If the origin of the coordinate axis of the bag image is not collinear with the first coordinate point and the second coordinate point, then the origin of the coordinate axis is taken as the third coordinate point.

3. The method according to claim 1, characterized in that, The step of filtering the third coordinate point from the image of the handbag includes: The third coordinate point is determined by using the x-coordinate value of the first coordinate point as the x-coordinate value of the third coordinate point, and using the y-coordinate value of the second coordinate point as the y-coordinate value of the third coordinate point. Alternatively, the third coordinate point can be determined by using the x-coordinate value of the second coordinate point as the x-coordinate value of the third coordinate point and the y-coordinate value of the first coordinate point as the y-coordinate value of the third coordinate point.

4. The method according to claim 1, characterized in that, The transformation matrix is ​​an affine matrix; the step of using the transformation matrix to overlay the accessory image onto the human body image includes: The shoulder strap region is subjected to an affine transformation using the affine matrix to obtain an affine image of the shoulder strap, and the bag body region is subjected to an affine transformation using the affine matrix to obtain an affine image of the bag body. The affine image of the shoulder strap and the affine image of the bag body are stitched together to obtain the affine image of the shoulder bag; The affine image of the shoulder bag is superimposed onto the human body image.

5. The method according to claim 1, characterized in that, The transformation matrix is ​​a perspective matrix; the step of using the transformation matrix to overlay the accessory image onto the human body image includes: The perspective matrix is ​​used to perform perspective transformation on the shoulder strap area to obtain a perspective image of the shoulder strap, and the perspective matrix is ​​used to perform perspective transformation on the bag body area to obtain a perspective image of the bag body; The shoulder strap perspective image and the bag body perspective image are stitched together to obtain a shoulder bag perspective image; The perspective image of the handbag is superimposed onto the human body image.

6. An image processing apparatus, characterized in that, include: The accessory image acquisition module is used to acquire accessory images and human body images; The transformation matrix determination module is used to determine the transformation matrix based on multiple coordinate points of the accessory image and multiple coordinate points of the human body image. An accessory image overlay module is used to overlay the accessory image onto the human body image using the transformation matrix; The accessory image is a handbag image; the accessory image acquisition module includes: The image target detection submodule is used to acquire the image to be processed, use a neural network model to perform target detection on the image to be processed, and obtain the bag image and / or the human body image. The multiple coordinate points of the bag image include: a first coordinate point, a second coordinate point, and a third coordinate point that are not collinear; the multiple coordinate points of the human body image include: a fourth coordinate point, a fifth coordinate point, and a sixth coordinate point that are not collinear; the image processing device further includes: The shoulder bag image recognition module is used to identify the shoulder strap area and bag body area from the shoulder bag image, and to identify the shoulder point in the first direction and the waist point in the second direction from the human body image. The first direction and the second direction are opposite. The first coordinate filtering module is used to determine the first coordinate point as the apex of the shoulder strap area and the second coordinate point as the connection point between the shoulder strap area and the bag body area. Then, the third coordinate point is filtered out from the bag image. The second coordinate filtering module is used to determine the shoulder point in the first direction as the fourth coordinate point, and the waist point in the second direction as the fifth coordinate point. Then, it filters out the sixth coordinate point from the human body image.

7. An electronic device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when executed by the processor, perform the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the method as described in any one of claims 1 to 5.

Images