Data generation method and apparatus, device, and storage medium

Abstract

This application discloses a data generation method, apparatus, device, and storage medium. The data generation method includes: acquiring a first user image, a model image with target clothing, and a target pose image; determining a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image based on the first user image and the model image; generating a second user image with the target pose, a second user pose image of the second user image, and a second user segmentation image based on the first user image, the first user pose image, the first user segmentation image, and the target pose image; and determining a third user image with the target clothing based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image. This application can control the generation of virtual try-on results with different poses by inputting different target pose information, thereby improving the effect of virtual try-on.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data generation method, apparatus, device and storage medium. Background Technology

[0002] With the development of e-commerce platforms, more and more people are buying clothes online. However, because users can only see the appearance of the clothing when purchasing, and cannot understand how it will look when worn, they often find that the product does not meet their expectations after receiving it offline, requiring returns and exchanges, resulting in a poor shopping experience.

[0003] Referring to patent CN111787242A, "Method and Apparatus for Virtual Fitting," this patent discloses an online virtual fitting method to solve the above problems. Specifically, it performs human body positioning and surface coordinate analysis on model images and user photos, overlaying the pixels corresponding to the clothing in the model image onto the corresponding positions in the user photo, achieving the effect of the clothing in the model image being worn by the user in the user photo. However, this method has a drawback: when the human posture in the model image differs significantly from that in the user photo, the corresponding positional relationship between the model image and the clothing will be incorrect, resulting in a poor virtual fitting effect. Summary of the Invention

[0004] The main objective of this application is to provide a data generation method, apparatus, device, and storage medium, which aims to solve the technical problem in the prior art that the virtual try-on effect is poor when the human posture in the model's picture differs greatly from the human posture in the user's photo.

[0005] To achieve the above objectives, this application provides a data generation method, the data generation method comprising:

[0006] Acquire the first user image, the model image with the target clothing, and the target pose image;

[0007] Based on the first user image and the model image, a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image are determined.

[0008] Based on the first user image, the first user pose image, the first user segmentation image, and the target pose image, a second user image with the target pose, a second user pose image of the second user image, and a second user segmentation image are generated.

[0009] Based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image, a third user image with the target clothing is determined.

[0010] Optionally, the step of generating a second user image with the target pose, a second user pose image, and a second user segmentation image based on the first user image, the first user pose image, the first user segmentation image, and the target pose image includes:

[0011] Determine the coordinates of the first key point in the first user pose image, the coordinates of the second key point in the target pose image, and the segmentation migration information to which the first user segmentation image points;

[0012] The first user image, the segmentation migration information, the first keypoint coordinates, and the second keypoint coordinates are input into a preset pose migration model. Based on the pose migration model and the segmentation migration information, the first user image is migrated from the first keypoint coordinates to the second keypoint coordinates to obtain a second user image with the target pose and a second user pose image of the second user image. Clothing segmentation is then performed on the second user image to obtain a second user segmentation image.

[0013] Optionally, the step of determining the first user pose image, the model pose image, the first user segmentation image, and the model segmentation image based on the first user image and the model image includes:

[0014] Human body information is expanded on the first user image and the model image respectively to obtain the expanded fourth user image and the expanded model image;

[0015] Feature extraction, key point localization, and classification are performed on the fourth user image and the expanded model image to obtain the first user pose image and the model pose image.

[0016] Clothing segmentation is performed on the fourth user image and the expanded model image to obtain the first user segmentation image and the model segmentation image.

[0017] Optionally, the step of expanding the human body information of the first user image and the model image to obtain the expanded fourth user image and the expanded model image respectively includes:

[0018] Extract the first feature information from the first user image and the second feature information from the model image;

[0019] Based on the first feature information and the second feature information, locate the first human body frame of the first user image and the second human body frame of the model image;

[0020] The first and second human body frames are expanded to obtain the expanded fourth user image and the expanded model image.

[0021] Optionally, after the steps of migrating the first user image from the first keypoint coordinates to the second keypoint coordinates based on the pose transfer model and the segmentation transfer information to obtain a second user image with the target pose and a second user pose image of the second user image, and performing clothing segmentation on the second user image to obtain a second user segmentation image, the method includes:

[0022] Determine the region to be filled in the second user image after pose transfer;

[0023] Based on a preset image restoration network, the region to be filled is restored to obtain the restored fifth user image.

[0024] Optionally, the step of performing image restoration on the region to be restored based on a preset image restoration network to obtain the restored fifth user image includes:

[0025] The region to be filled is downsampled a preset number of times and a preset multiple to obtain a corresponding number of first feature data.

[0026] The first feature data is upsampled a preset number of times to obtain a second feature data with the same number of elements as the first feature data, wherein the second feature data has the same scale as the image of the region to be completed;

[0027] The second feature data is fused to obtain the fifth user image after image restoration.

[0028] Optionally, the step of determining a third user image with the target clothing based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image includes:

[0029] Determine the coordinates of the third key point of the second user pose image and the coordinates of the fourth key point of the model pose image;

[0030] Determine the key point coordinate mapping between the coordinates of the third key point and the coordinates of the fourth key point;

[0031] The second user image, the model image, the second user segmentation image, the model segmentation image, and the key point coordinate mapping are input into a preset clothing transfer model. Based on the clothing transfer model and the key point coordinate mapping, the model segmentation image on the model image is transferred to the second user segmentation image on the second user image to obtain a third user image with the target clothing.

[0032] This application also provides a data generation apparatus, the data generation apparatus comprising:

[0033] The acquisition module is used to acquire the first user image, the model image with the target clothing, and the target pose image;

[0034] The determining module is used to determine a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image based on the first user image and the model image;

[0035] The generation module is used to generate a second user image with the target pose, a second user pose image of the second user image, and a second user segmentation image based on the first user image, the first user pose image, the first user segmentation image, and the target pose image;

[0036] The migration module is used to determine a third user image with the target clothing based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image.

[0037] This application also provides a data generation device, the data generation device comprising: a memory, a processor, and a program stored in the memory for implementing the data generation method.

[0038] The memory is used to store programs that implement the data generation method;

[0039] The processor is used to execute a program that implements the data generation method, so as to implement the steps of the data generation method.

[0040] This application also provides a storage medium storing a program for implementing a data generation method, wherein the program for implementing the data generation method is executed by a processor to implement the steps of the data generation method.

[0041] This application provides a data generation method, apparatus, device, and storage medium that addresses the issue of inaccurate virtual try-on when the pose in a model image differs significantly from that in a user photo, leading to errors in the positional relationship between the model image and clothing. In this application, the method involves acquiring a first user image, a model image with the target clothing, and a target pose image. Based on the first user image and the model image, a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image are determined. Based on the first user image, the first user pose image, the first user segmentation image, and the target pose image, a second user image with the target pose, a second user pose image of the second user image, and a second user segmentation image are generated. Based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image, a third user image with the target clothing is determined. In other words, by inputting different target pose information, different poses can be generated for virtual try-on, thereby improving the effectiveness of virtual try-on. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0043] Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application;

[0044] Figure 2 This is a flowchart illustrating the first embodiment of the data generation method of this application;

[0045] Figure 3 This is a schematic diagram of the data generation device of this application;

[0046] Figure 4 This is a flowchart illustrating the key point detection process of the data generation method in this application;

[0047] Figure 5 This is a schematic diagram of the clothing segmentation process in the data generation method of this application.

[0048] Figure 6 This is a schematic diagram of the system flow of the second embodiment of the data generation method of this application;

[0049] Figure 7The image restoration network structure is provided for the data generation method of this application;

[0050] Figure 8 This is a schematic flowchart of the second embodiment of the data generation method of this application.

[0051] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0052] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0053] like Figure 1 As shown, Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of this application.

[0054] The terminal in this application embodiment can be a PC, or a smartphone, tablet computer, e-book reader, MP3 (Moving Picture Experts Group Audio Layer III) player, MP4 (Moving Picture Experts Group Audio Layer IV) player, portable computer, or other portable terminal devices with display functions.

[0055] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0056] Optionally, the terminal may also include a camera, RF (Radio Frequency) circuitry, sensors, audio circuitry, a WiFi module, and so on. Sensors may include light sensors, motion sensors, and other sensors. Specifically, light sensors may include ambient light sensors and proximity sensors. The ambient light sensor can adjust the display brightness according to the ambient light level, while the proximity sensor can turn off the display and / or backlight when the mobile terminal is moved to the ear. As a type of motion sensor, a gravity accelerometer can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity, and can be used for applications that identify the mobile terminal's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition functions (such as pedometers, taps), etc. Of course, the mobile terminal may also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, which will not be elaborated here.

[0057] Those skilled in the art will understand that Figure 1 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0058] like Figure 1 As shown, the memory 1005, which serves as a computer storage medium, may include an operating device, a network communication module, a user interface module, and a data generation program.

[0059] exist Figure 1 In the terminal shown, the network interface 1004 is mainly used to connect to the backend server and communicate with the backend server; the user interface 1003 is mainly used to connect to the client (user terminal) and communicate with the client; and the processor 1001 can be used to call the data generation program stored in the memory 1005.

[0060] Reference Figure 2 This application provides a data generation method, which includes:

[0061] Step S100: Obtain the first user image, the model image with the target clothing, and the target pose image;

[0062] Step S200: Based on the first user image and the model image, determine the first user pose image, the model pose image, the first user segmentation image, and the model segmentation image;

[0063] Step S300: Based on the first user image, the first user pose image, the first user segmentation image, and the target pose image, generate a second user image with the target pose, a second user pose image of the second user image, and a second user segmentation image;

[0064] Step S400: Based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image, determine a third user image with the target clothing.

[0065] In this embodiment, the application scenario is:

[0066] As an example, one scenario for this data generation method is when a user needs to understand how the clothing will look when purchased online. Existing technology uses human body localization and surface coordinate analysis on model images and user photos to overlay the pixels corresponding to the clothing in the model image onto the corresponding positions in the user photo, thus obtaining the effect of the clothing from the model image being worn by the user in the photo. However, this method suffers from errors in the positional relationship between the model image and the clothing when there is a significant difference between the human posture in the model image and the human posture in the user photo, resulting in poor virtual try-on results. To address this scenario, this embodiment allows for the control of generating virtual try-on results with different postures by inputting different target posture information, thereby improving the effectiveness of virtual try-on.

[0067] As an example, the application scenarios for data generation are not limited to the aforementioned scenario of users purchasing clothing online, but also include various virtual try-on scenarios, without specific limitations here.

[0068] This embodiment aims to improve the effect of virtual try-on.

[0069] In this embodiment, the data generation method is applied to the data generation device.

[0070] The specific steps are as follows:

[0071] Step S100: Obtain the first user image, the model image with the target clothing, and the target pose image;

[0072] In this embodiment, the first user image is an initial image with user portrait features, which are key human body features. The first user image is usually a full-body photo of the user. The target clothing is clothing to be changed onto the user image, including clothing, shoes, hats, socks, gloves, scarves, ties, accessories, bags, umbrellas, etc. The model image with the target clothing is an initial image with model portrait features wearing the target clothing, usually an image of a model wearing the clothing taken by the clothing seller to showcase the clothing features. The target posture image is an image with a corresponding human posture, including but not limited to sitting, standing, kneeling, and walking postures. The target posture image includes but is not limited to images of virtual animated figures and images of real people, such as a virtual stick figure image in a squatting posture. It should be noted that the target posture image, the first user image, and the model image with the target clothing all contain portrait features. Specifically, the target posture image, the first user image, and the model image with the target clothing include portrait features such as shoulders, arms, head, legs, and feet.

[0073] In this embodiment, the device can acquire the first user image and the model image with the target clothing by uploading the corresponding first user image and model image with the target clothing to the device. For example, the user may store a full-body photo of the user and the model image with the target clothing, and upload the full-body photo of the user and the model image with the target clothing to the device through the user interface of the device; alternatively, the device may receive the first user image and the model image with the target clothing from a preset camera.

[0074] In this embodiment, the device can acquire the target posture image by having the user select it from a database containing various types of postures. The user can select the target posture image by the device displaying various posture images for selection, or by the user inputting text information, and the device retrieving the corresponding posture image from the database based on the text information. For example, the user inputs a text command for a sitting posture into the device's user interface, and the device retrieves the corresponding sitting posture image from the database. Alternatively, the user can upload the corresponding target posture image to the device. For example, the user may have stored sitting posture images and upload them to the device through the user interface.

[0075] Step S200: Based on the first user image and the model image, determine the first user pose image, the model pose image, the first user segmentation image, and the model segmentation image;

[0076] In this embodiment, the device determines a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image based on the first user image and the model image. The first user pose image and the model pose image are respectively images including pose information obtained from analyzing the portrait in the first user image and the pose information obtained from analyzing the portrait in the model image. The first user segmentation image and the model segmentation image are respectively portrait segmentation maps of the first user image and the model image. The portrait segmentation map is an image that segments clothing and body parts in a portrait. It should be noted that the first user segmentation image and the model segmentation image are image masks of the first user image and the model image, describing the clothing or body part information of each region of the portrait in the first user image and the model image. For example, if the model in the model image is wearing a dark yellow hat, then the information of the hat region in the model segmentation image is the hat, and the information of the head region is the model's head.

[0077] In this embodiment, the first user pose image and the model pose image are obtained by the device performing key point feature analysis on the first user image and the model image. The analysis method includes key point feature extraction and classification. The first user segmentation image and the model segmentation image are obtained by the device performing image segmentation on the portrait of the first user image and the model image, including background, hat, hair, gloves, sunglasses, top, dress, coat, socks, pants, torso, scarf, skirt, cheek, left arm, right arm, left leg, right leg, left shoe, and right shoe, generating corresponding segmentation images.

[0078] Specifically, step S200 includes the following steps S210-S230:

[0079] Step S210: Expand the human body information of the first user image and the model image respectively to obtain the expanded fourth user image and the expanded model image.

[0080] In this embodiment, to increase the background information around the target human body and make the key point estimation more accurate, this application proposes a human body information expansion method. Specifically, before determining the pose image and segmentation map of the first user image and the model image, the device expands the human body information of the first user image and the model image to obtain an expanded fourth user image and an expanded model image, thereby increasing the background information around the target human body in the target image (i.e., the first user image and the model image) and making the key point estimation more accurate.

[0081] Specifically, step S210 includes the following steps S211-S213:

[0082] Step S211: Extract the first feature information of the first user image and the second feature information of the model image;

[0083] In this embodiment, the device extracts first feature information of the first user image and second feature information of the model image. Specifically, the device uses a preset feature extractor to extract low-level feature information from the input target human image (i.e., the first user image and the model image) through a general network to obtain first feature information and second feature information. The general network is used to extract image feature information, including but not limited to convolutional neural network VGG and residual network ResNet.

[0084] Step S212: Based on the first feature information and the second feature information, locate the first human body frame of the first user image and the second human body frame of the model image;

[0085] In this embodiment, the device locates the first human body frame of the first user image and the second human body frame of the model image based on the first feature information and the second feature information. Specifically, the device determines the location of the human body in the first user image based on the first feature information and the second feature information, and then locates the first human body frame of the first user image according to the location of the human body. Similarly, the second human body frame of the model image is located using the same method.

[0086] Step S213: Expand the first human body frame and the second human body frame to obtain the expanded fourth user image and the expanded model image.

[0087] In this embodiment, the device expands the first human body frame and the second human body frame to obtain the expanded fourth user image and the expanded model image. The expansion process is equivalent to enlarging the human body frame to obtain more background information, making the prediction of key points more accurate.

[0088] Step S220: Perform feature extraction, key point localization, and classification on the fourth user image and the expanded model image to obtain the first user pose image and the model pose image.

[0089] In this embodiment, refer to Figure 4 The device extracts features from the fourth user image and the expanded model image to obtain corresponding feature information. Then, the device uses a preset key point localization and classifier to classify the feature information position by position, judge the key point information at each position, and finally generate the corresponding pose image, namely the first user pose image and the model pose image.

[0090] Step S230: Perform clothing segmentation on the fourth user image and the expanded model image to obtain the first user segmentation image and the model segmentation image.

[0091] In this embodiment, refer to Figure 5 The device uses a preset clothing classifier to segment the clothing of the fourth user image and the expanded model image, resulting in a first user segmented image and a model segmented image. To enhance the ability to distinguish detailed human body information, the categories in the clothing classifier are expanded. The expanded content for the foreground and background categories includes, but is not limited to, background, top, pants, hair, face, and other required categories. The foreground refers to the scene closest to the viewer in the image, specifically a human figure. The expanded clothing classifier is used to obtain the classification information of each part in the target image.

[0092] Step S300: Based on the first user image, the first user pose image, the first user segmentation image, and the target pose image, generate a second user image with the target pose, a second user pose image of the second user image, and a second user segmentation image;

[0093] In this embodiment, the second user image is the user image after the first user image is transformed into the target pose. That is, the second user image is the first user image with the target pose. For example, if the first user image of user A is a standing pose image and the target pose selected by user A is a sitting pose, then the second user image is the sitting pose image of user A, which is only different from the first user image in terms of pose. The second user pose image is the pose image of the second user image. The second user segmentation image is the image after the clothing segmentation of the second user image.

[0094] In this embodiment, the second user image with the target pose is generated based on the first user image, the first user pose image, the first user segmentation image, and the target pose image by performing pose transfer. According to the position coordinates of the same part in the first user pose image and the target pose image, the pose of the target pose image is transferred to the first user pose image to generate the second user image with the target pose. The generation method of the second user pose image and the second user segmentation image is the same as the above step S200, which is obtained by performing pose analysis and clothing segmentation on the second user image.

[0095] Specifically, step S300 includes the following steps S310-S320:

[0096] Step S310: Determine the coordinates of the first key point of the first user pose image, the coordinates of the second key point of the target pose image, and the segmentation migration information pointed to by the first user segmentation image;

[0097] In this embodiment, the first key point coordinates are the coordinates of each key point on the first user pose image, including but not limited to the following key points: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle. Similarly, the second key point coordinates are the coordinates of each key point on the target pose image. The segmentation migration information is the segmentation information pointed to by the first user segmented image obtained after the device segments the clothing of the first user image. The segmentation migration information includes but is not limited to background, hat, hair, gloves, sunglasses, top, dress, coat, socks, pants, torso, scarf, skirt, cheek, left arm, right arm, left leg, right leg, left shoe, and right shoe.

[0098] Step S320: Input the first user image, the segmentation migration information, the first key point coordinates, and the second key point coordinates into a preset pose migration model. Based on the pose migration model and the segmentation migration information, migrate the first user image from the first key point coordinates to the second key point coordinates to obtain a second user image with the target pose and a second user pose image of the second user image. Then, perform clothing segmentation on the second user image to obtain a second user segmentation image.

[0099] In this embodiment, the preset pose transfer model is a pre-trained neural network model. The device inputs the first user image, the segmentation transfer information, the first keypoint coordinates, and the second keypoint coordinates into the preset pose transfer model. Based on the pose transfer model, using paired human body data with different poses (i.e., the first keypoint coordinates and the second keypoint coordinates) and the segmentation transfer information, the first user image is transferred from the first keypoint coordinates to the second keypoint coordinates to obtain a second user image with the target pose. The generation method of the second user pose image and the second user segmentation image is the same as the above step S200, which is obtained by performing pose analysis and clothing segmentation on the second user image.

[0100] In step S320, based on the pose transfer model and the segmentation transfer information, the first user image is transferred from the first keypoint coordinates to the second keypoint coordinates to obtain a second user image with the target pose and a second user pose image of the second user image. After performing clothing segmentation on the second user image to obtain a second user segmentation image, the method includes the following steps A100-A200:

[0101] Step A100: Determine the region to be completed in the second user image after pose transfer;

[0102] In this embodiment, the background area changes due to the change in posture. For example, when a person changes from standing to falling, the area where the person was standing needs to be filled in. This area is the area to be filled in for the second user image after the posture shift.

[0103] Step A200: Based on a preset image inpainting network, perform image inpainting on the region to be inpainted to obtain the inpainted fifth user image.

[0104] In this embodiment, the device performs image restoration on the region to be filled based on a preset image restoration network to obtain the restored fifth user image.

[0105] Specifically, step A200 includes the following steps A210-A220:

[0106] Step A210: The region to be completed is downsampled a preset number of times and a preset multiple to obtain a corresponding number of first feature data.

[0107] In this embodiment, the device downsamples the region to be completed a preset number of times and a preset multiple to obtain a corresponding number of first feature data. The preset number of times and preset multiple are user-defined downsampling times and multiples, typically two samplings and two times or more. The first feature data is obtained based on the preset number of times and preset multiple. For example, if the region to be completed, A, is an image with a resolution of 256*256, performing three and four times downsampling on the region to be completed will yield three first feature data. The first first feature data A is the original image A (resolution 256*256), the second first feature data B is obtained by downsampling the region to be completed A once (resolution 128*128), and the third first feature data C is obtained by downsampling the region to be completed A twice (resolution 64*64). Similarly, performing six and eight times downsampling on a 256*256 resolution region to be completed will yield four first feature data.

[0108] Step A220: Upsample the first feature data a preset number of times to obtain the same number of second feature data as the first feature data, wherein the second feature data has the same scale as the image of the region to be completed;

[0109] In this embodiment, the device upsamples the first feature data a preset number of times to obtain a second feature data with the same number of elements as the first feature data. The second feature data has the same scale as the image to be completed, meaning the downsampled first feature data is upsampled accordingly to obtain second feature data with the same scale as the original image to be completed. The same size means the same resolution. For example, if the image to be completed, A, has a resolution of 256*256, and the image to be completed is downsampled three times and four times, the first feature data A is the original image A (resolution 256*256), and the second feature data B is the image to be completed after one downsampling of the image to be completed. The obtained first feature data (resolution 128*128) is obtained by downsampling the region A to be completed twice (resolution 64*64). First feature data A, B, and C are upsampled. Since A has not been downsampled, no upsampling is required, resulting in second feature data A' (resolution 256*256) which is consistent with A. B (resolution 128*128) has been downsampled once, and then upsampled once to obtain second feature data B' (resolution 256*256). C (resolution 64*64) has been downsampled twice, and then upsampled twice to obtain second feature data C' (resolution 256*256).

[0110] Step A230: Perform feature fusion on the second feature data to obtain the fifth user image after image restoration.

[0111] In this embodiment, the device fuses multiple second feature data with the same resolution to obtain a fifth user image after image restoration.

[0112] In this embodiment, refer to Figure 7 The right branch has an H / 2*W / 2 feature layer, which introduces additional shallow features from the image to be repaired (H*W), downsamples them, and then fuses the features. The repaired image (H*W) has an additional H / 2*W / 2 deep feature layer, which is upsampled and then fused to make the repair result more realistic.

[0113] In this embodiment, the device increases the contextual information for repair by performing multiple downsampling and upsampling operations in different dimensions, thereby improving the repair effect of the area to be filled.

[0114] Step S400: Based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image, determine a third user image with the target clothing.

[0115] In this embodiment, the device determines a third user image with the target clothing based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image. That is, the clothing from the model image is transferred to the human body in the second user image, and the user image with the target clothing is displayed to the user. This achieves the effect of the clothing from the model image being worn by the user in the user photo, thereby improving the user's shopping experience.

[0116] Specifically, step S400 includes the following steps S410-S430:

[0117] Step S410: Determine the coordinates of the third key point of the second user pose image and the coordinates of the fourth key point of the model pose image;

[0118] In this embodiment, the device determines the coordinates of the third key point of the second user pose image and the coordinates of the fourth key point of the model pose image, referring to step S310 above, which will not be repeated here.

[0119] Step S420: Determine the key point coordinate mapping between the coordinates of the third key point and the coordinates of the fourth key point;

[0120] In this embodiment, the device determines the key point coordinate mapping between the third key point coordinate and the fourth key point coordinate, that is, to form a corresponding key point coordinate mapping between the key point coordinates of the human body parts in the second user pose image and the corresponding key point coordinates of the human body parts in the model pose image. For example, the left shoulder coordinate of the second user pose image and the left shoulder coordinate of the model pose image are used as a mapping.

[0121] Step S430: Input the second user image, the model image, the second user segmentation image, the model segmentation image, and the key point coordinate mapping into a preset clothing transfer model. Based on the clothing transfer model and the key point coordinate mapping, transfer the model segmentation image on the model image to the second user segmentation image on the second user image to obtain a third user image with the target clothing.

[0122] In this embodiment, the preset clothing transfer model is a pre-trained neural network model. The device inputs the second user image, the model image, the second user segmentation image, the model segmentation image, and the key point coordinate mapping into the preset clothing transfer model. Based on the clothing transfer model and the key point coordinate mapping, the model segmentation image on the model image is transferred to the second user segmentation image on the second user image to obtain a third user image with the target clothing. That is, the clothing image on the model segmentation image is transferred to the human body in the second user image to obtain a third user image with the target clothing, and the user image with the target clothing is displayed to the user.

[0123] This application provides a data generation method that addresses the issue of inaccurate positioning of model images and clothing when the poses in model photos differ significantly from those in user photos, leading to poor virtual try-on results. In this method, the following steps are taken: First, a first user image, a model image with the target clothing, and a target pose image are acquired. Based on the first user image and the model image, a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image are determined. Based on the first user image, the first user pose image, the first user segmentation image, and the target pose image, a second user image with the target pose, a second user pose image of the second user image, and a second user segmentation image are generated. Based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image, a third user image with the target clothing is determined. In other words, by inputting different target pose information, different poses can be generated for virtual try-on, thereby improving the effectiveness of virtual try-on.

[0124] Based on the first embodiment described above, this application also provides another embodiment, referring to... Figure 6 and Figure 8 The data generation method includes:

[0125] S1: The device receives images sent by the user, including Clothes, Person, and Pose.

[0126] S2: Predict key points of the human body from model and user images to generate corresponding pose diagrams.

[0127] In this embodiment, the keypoint detection method for generating the corresponding pose map (Person_Pose) from the user image (Person) is as follows: The feature extractor extracts low-level feature information (feature) from the input human image through a general network (VGG, ResNet, etc.). The human body classification regressor uses the human body location box in the low-level feature image to output the human body bounding box. To increase the background information around the target human body and make the predicted keypoints more accurate, a human body expansion module is added to expand the human body bounding box output by the human body classification regressor to obtain box2. The keypoint extraction and clothing segmentation module of this application appropriately expands the target area to increase background information, thereby improving the keypoint prediction and clothing segmentation effect. Then, the corresponding position features in the feature are extracted using box2 to obtain feature2. The keypoint localization and classifier classifies feature2 position by position to determine whether it is the background or the foreground. If it is the foreground, it is the specific keypoint, and finally the corresponding pose map (Person_Pose) is generated.

[0128] S3: Perform clothing segmentation on model and user images to generate corresponding segmentation images.

[0129] In this embodiment, the segmentation method for generating a corresponding segmentation map (Person_Seg) from a user image (Person) is as follows: the feature extractor extracts low-level feature information (feature) from the input human image through a general network (VGG, ResNet, etc.), the human body classification regressor uses the human body location box in the low-level feature image to output the human body bounding box, the human body expansion module expands the box to add information around the human body to obtain box2, and uses the corresponding position features in the required feature of box2 to obtain feature2. In order to increase the ability to distinguish human body details, the categories in the clothing classifier are expanded from the two categories of foreground and background to the required categories such as background, top, pants, hair, and face.

[0130] S4: Use the user image and its pose map, segmentation map, and target pose map to complete the target pose transfer, and generate the target pose user image and its segmentation map, as well as the region to be completed.

[0131] In this embodiment, using the user image (Person) and its corresponding pose map (Person_Pose), segmentation map (Person_Seg), and target pose map (Pose), and by transforming the human body key point coordinates between Pose and Person_Pose, a target pose user image (Person2) and its corresponding pose map (Person2_Pose), segmentation map (Person2_Seg), and a region to be filled (Mask) are generated. Due to the change in pose, the background area changes, for example, the human body changes from standing to falling. At this time, the area where the person was standing needs to be filled, as described in step S6.

[0132] The process is as follows Figure 6 As shown, the human body transformation f1 is obtained by training a feature extraction network using pairs of human body data in different poses. During inference, by inputting the target pose, the original image Person can be transferred to Person2 through the human body transformation f1, and the corresponding Person_Seg is transferred to Person2_Seg, and Person_Pose is transferred to Person2_Pose.

[0133] S5: Use target pose user images and their pose maps and segmentation maps, and model images and their pose maps and segmentation maps to perform clothing transfer.

[0134] In this embodiment, by generating a target pose user image (Person2) and its corresponding pose image (Person2_Pose), segmentation image (Person2_Seg), model image (Clothes) and its corresponding pose image (Clothes_Pose), segmentation image (Clothes_Seg), and utilizing the correspondence between Person2_Pose and Clothes_Pose, the Clothes texture is transferred to Person2 to generate the human body Person3. At this time, Person3 contains Clothes clothing texture information and is consistent with the pose, but the background area needs to be filled in.

[0135] As shown in Figure 6, the clothing transformation f2 uses paired human body data wearing different clothes, which is obtained through training a feature extraction network. During inference, by inputting the clothing Clothes, Clothes_Pose, Clothes_Seg, and the transferred human body Person2, Person2_Pose, Person2_Seg, the clothing transformation f2 is used to transfer Person2 to Person3.

[0136] S6: Use an image inpainting network to repair the area to be filled.

[0137] In this embodiment, an image inpainting network and a mask for the region to be inpainted are used to inpaint the image Person3 generated by S5, generating the final result.

[0138] Image inpainting network structure as follows Figure 7 As shown, the application adds contextual information to the conventional image inpainting network, such as the right branch H / 2*W / 2 feature layer, which introduces shallow features of the image to be inpainted (H*W), downsamples them, and then performs feature fusion; for the inpainted image (H*W), additional deep features (H / 2*W / 2) are introduced, upsampled, and then performed feature fusion. By adding large-scale or small-scale feature information in the image inpainting network, information loss is avoided, algorithm performance is improved, and the image inpainting effect is enhanced.

[0139] In this embodiment, by inputting different target posture information, the virtual fitting results of the human body in different postures can be controlled to improve the effect of virtual fitting.

[0140] This application also provides a data generation apparatus, the data generation apparatus comprising:

[0141] The acquisition module 10 is used to acquire a first user image, a model image with the target clothing, and a target pose image;

[0142] The determining module 20 is used to determine a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image based on the first user image and the model image;

[0143] The generation module 30 is used to generate a second user image with a target pose, a second user pose image of the second user image, and a second user segmentation image based on the first user image, the first user pose image, the first user segmentation image, and the target pose image;

[0144] The migration module 40 is used to determine a third user image with the target clothing based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image.

[0145] Optionally, the generation module 30 includes:

[0146] The coordinate determination module is used to determine the coordinates of the first key point of the first user pose image, the coordinates of the second key point of the target pose image, and the segmentation migration information pointed to by the first user segmentation image.

[0147] The pose transfer module is used to input the first user image, the segmentation transfer information, the first key point coordinates and the second key point coordinates into a preset pose transfer model. Based on the pose transfer model and the segmentation transfer information, the first user image is transferred from the first key point coordinates to the second key point coordinates to obtain a second user image with the target pose and a second user pose image of the second user image. The second user image is then segmented by clothing to obtain a second user segmentation image.

[0148] Optionally, the determining module 20 includes:

[0149] The expansion module is used to expand the human body information of the first user image and the model image respectively to obtain the expanded fourth user image and the expanded model image;

[0150] The key point localization and classification module is used to perform feature extraction, key point localization and classification on the fourth user image and the expanded model image to obtain the first user pose image and the model pose image.

[0151] The segmentation module is used to segment the clothing of the fourth user image and the expanded model image to obtain the first user segmented image and the model segmented image.

[0152] Optionally, the expansion module includes:

[0153] The extraction module is used to extract the first feature information of the first user image and the second feature information of the model image;

[0154] The positioning module is used to locate the first human body frame of the first user image and the second human body frame of the model image based on the first feature information and the second feature information.

[0155] The human body frame expansion module is used to expand the first human body frame and the second human body frame to obtain the expanded fourth user image and the expanded model image.

[0156] Optionally, the data generation apparatus further includes:

[0157] The region to be filled module is used to determine the region to be filled in the second user image after pose transfer;

[0158] The repair module is used to repair the region to be filled based on a preset image repair network, so as to obtain the repaired fifth user image.

[0159] Optionally, the repair module includes:

[0160] The downsampling module is used to downsample the region to be completed a preset number of times and a preset multiple to obtain a corresponding number of first feature data.

[0161] An upsampling module is used to upsample the first feature data a preset number of times to obtain a second feature data with the same number of elements as the first feature data, wherein the second feature data has the same scale as the image of the region to be completed;

[0162] The feature fusion module is used to fuse the second feature data to obtain the fifth user image after image restoration.

[0163] Optionally, the migration module 40 includes:

[0164] The key point coordinate determination module is used to determine the coordinates of the third key point of the second user pose image and the coordinates of the fourth key point of the model pose image.

[0165] A key point coordinate mapping determination module is used to determine the key point coordinate mapping between the coordinates of the third key point and the coordinates of the fourth key point.

[0166] The clothing migration module is used to input the second user image, the model image, the second user segmentation image, the model segmentation image, and the key point coordinate mapping into a preset clothing migration model. Based on the clothing migration model and the key point coordinate mapping, the model segmentation image on the model image is migrated to the second user segmentation image on the second user image to obtain a third user image with the target clothing.

[0167] The specific implementation of the data generation device in this application is basically the same as the embodiments of the data generation method described above, and will not be repeated here.

[0168] Reference Figure 1 , Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of this application.

[0169] like Figure 1As shown, the terminal may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0170] Optionally, the data generation device may also include a rectangular user interface, a network interface, a camera, RF (Radio Frequency) circuitry, sensors, audio circuitry, a WiFi module, etc. The rectangular user interface may include a display screen and an input submodule such as a keyboard. Optionally, the rectangular user interface may also include a standard wired interface or a wireless interface. The network interface may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0171] Those skilled in the art will understand that Figure 1 The data generation device structure shown does not constitute a limitation on the data generation device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0172] like Figure 1 As shown, the memory 1005, as a storage medium, may include an operating system, a network communication module, and a data generation program. The operating system is a program that manages and controls the hardware and software resources of the data generation device, supporting the operation of the data generation program and other software and / or programs. The network communication module is used to enable communication between the various components within the memory 1005, as well as communication with other hardware and software in the data generation system.

[0173] exist Figure 1 In the data generation device shown, the processor 1001 is used to execute the data generation program stored in the memory 1005 to implement the steps of the data generation method described in any of the above claims.

[0174] The specific implementation of the data generation device in this application is basically the same as the embodiments of the data generation method described above, and will not be repeated here.

[0175] This application also provides a storage medium storing a program implementing a data generation method, the program implementing the data generation method being executed by a processor to implement the data generation method as follows:

[0176] Acquire the first user image, the model image with the target clothing, and the target pose image;

[0177] Based on the first user image and the model image, a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image are determined.

[0178] Based on the first user image, the first user pose image, the first user segmentation image, and the target pose image, a second user image with the target pose, a second user pose image of the second user image, and a second user segmentation image are generated.

[0179] Based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image, a third user image with the target clothing is determined.

[0180] Optionally, the step of generating a second user image with the target pose, a second user pose image, and a second user segmentation image based on the first user image, the first user pose image, the first user segmentation image, and the target pose image includes:

[0181] Determine the coordinates of the first key point in the first user pose image, the coordinates of the second key point in the target pose image, and the segmentation migration information to which the first user segmentation image points;

[0182] The first user image, the segmentation migration information, the first keypoint coordinates, and the second keypoint coordinates are input into a preset pose migration model. Based on the pose migration model and the segmentation migration information, the first user image is migrated from the first keypoint coordinates to the second keypoint coordinates to obtain a second user image with the target pose and a second user pose image of the second user image. Clothing segmentation is then performed on the second user image to obtain a second user segmentation image.

[0183] Optionally, the step of determining the first user pose image, the model pose image, the first user segmentation image, and the model segmentation image based on the first user image and the model image includes:

[0184] Human body information is expanded on the first user image and the model image respectively to obtain the expanded fourth user image and the expanded model image;

[0185] Feature extraction, key point localization, and classification are performed on the fourth user image and the expanded model image to obtain the first user pose image and the model pose image.

[0186] Clothing segmentation is performed on the fourth user image and the expanded model image to obtain the first user segmentation image and the model segmentation image.

[0187] Optionally, the step of expanding the human body information of the first user image and the model image to obtain the expanded fourth user image and the expanded model image respectively includes:

[0188] Extract the first feature information from the first user image and the second feature information from the model image;

[0189] Based on the first feature information and the second feature information, locate the first human body frame of the first user image and the second human body frame of the model image;

[0190] The first and second human body frames are expanded to obtain the expanded fourth user image and the expanded model image.

[0191] Optionally, after the steps of migrating the first user image from the first keypoint coordinates to the second keypoint coordinates based on the pose transfer model and the segmentation transfer information to obtain a second user image with the target pose and a second user pose image of the second user image, and performing clothing segmentation on the second user image to obtain a second user segmentation image, the method includes:

[0192] Determine the region to be filled in the second user image after pose transfer;

[0193] Based on a preset image restoration network, the region to be filled is restored to obtain the restored fifth user image.

[0194] Optionally, the step of performing image restoration on the region to be restored based on a preset image restoration network to obtain the restored fifth user image includes:

[0195] The region to be filled is downsampled a preset number of times and a preset multiple to obtain a corresponding number of first feature data.

[0196] The first feature data is upsampled a preset number of times to obtain a second feature data with the same number of elements as the first feature data, wherein the second feature data has the same scale as the image of the region to be completed;

[0197] The second feature data is fused to obtain the fifth user image after image restoration.

[0198] Optionally, the step of determining a third user image with the target clothing based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image includes:

[0199] Determine the coordinates of the third key point of the second user pose image and the coordinates of the fourth key point of the model pose image;

[0200] Determine the key point coordinate mapping between the coordinates of the third key point and the coordinates of the fourth key point;

[0201] The second user image, the model image, the second user segmentation image, the model segmentation image, and the key point coordinate mapping are input into a preset clothing transfer model. Based on the clothing transfer model and the key point coordinate mapping, the model segmentation image on the model image is transferred to the second user segmentation image on the second user image to obtain a third user image with the target clothing.

[0202] The specific implementation of the storage medium in this application is basically the same as the embodiments of the data generation method described above, and will not be repeated here.

[0203] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described data generation method.

[0204] The specific implementation of the computer program product of this application is basically the same as the embodiments of the data generation method described above, and will not be repeated here.

[0205] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0206] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0207] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0208] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A data generation method, characterized in that, The data generation method includes: Acquire the first user image, the model image with the target clothing, and the target pose image; Based on the first user image and the model image, a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image are determined. Based on the first user image, the first user pose image, the first user segmentation image, and the target pose image, a second user image with the target pose, a second user pose image of the second user image, and a second user segmentation image are generated. Based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image, a third user image with the target clothing is determined; The step of determining a third user image with the target clothing based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image includes: Determine the coordinates of the third key point of the second user pose image and the coordinates of the fourth key point of the model pose image; Determine the key point coordinate mapping between the coordinates of the third key point and the coordinates of the fourth key point; The second user image, the model image, the second user segmentation image, the model segmentation image, and the key point coordinate mapping are input into a preset clothing transfer model. Based on the clothing transfer model and the key point coordinate mapping, the model segmentation image on the model image is transferred to the second user segmentation image on the second user image to obtain a third user image with the target clothing.

2. The data generation method as described in claim 1, characterized in that, The step of generating a second user image with the target pose, a second user pose image, and a second user segmentation image based on the first user image, the first user pose image, the first user segmentation image, and the target pose image includes: Determine the coordinates of the first key point in the first user pose image, the coordinates of the second key point in the target pose image, and the segmentation migration information to which the first user segmentation image points; The first user image, the segmentation migration information, the first keypoint coordinates, and the second keypoint coordinates are input into a preset pose migration model. Based on the pose migration model and the segmentation migration information, the first user image is migrated from the first keypoint coordinates to the second keypoint coordinates to obtain a second user image with the target pose and a second user pose image of the second user image. Clothing segmentation is then performed on the second user image to obtain a second user segmentation image.

3. The data generation method as described in claim 1, characterized in that, The step of determining the first user pose image, the model pose image, the first user segmentation image, and the model segmentation image based on the first user image and the model image includes: Human body information is expanded on the first user image and the model image respectively to obtain the expanded fourth user image and the expanded model image; Feature extraction, key point localization, and classification are performed on the fourth user image and the expanded model image to obtain the first user pose image and the model pose image. Clothing segmentation is performed on the fourth user image and the expanded model image to obtain the first user segmentation image and the model segmentation image.

4. The data generation method as described in claim 3, characterized in that, The step of expanding the human body information of the first user image and the model image respectively to obtain the expanded fourth user image and the expanded model image includes: Extract the first feature information from the first user image and the second feature information from the model image; Based on the first feature information and the second feature information, locate the first human body frame of the first user image and the second human body frame of the model image; The first and second human body frames are expanded to obtain the expanded fourth user image and the expanded model image.

5. The data generation method as described in claim 2, characterized in that, After the steps of migrating the first user image from the first keypoint coordinates to the second keypoint coordinates based on the pose transfer model and the segmentation transfer information to obtain a second user image with the target pose and a second user pose image of the second user image, and performing clothing segmentation on the second user image to obtain a second user segmentation image, the method includes: Determine the region to be filled in the second user image after pose transfer; Based on a preset image restoration network, the region to be filled is restored to obtain the restored fifth user image.

6. The data generation method as described in claim 5, characterized in that, The step of performing image restoration on the region to be restored based on a preset image restoration network to obtain the restored fifth user image includes: The region to be filled is downsampled a preset number of times and a preset multiple to obtain a corresponding number of first feature data. The first feature data is upsampled a preset number of times to obtain a second feature data with the same number of elements as the first feature data, wherein the second feature data has the same scale as the image of the region to be completed; The second feature data is fused to obtain the fifth user image after image restoration.

7. A data generation apparatus, characterized in that, The data generation device includes: The acquisition module is used to acquire the first user image, the model image with the target clothing, and the target pose image; The determining module is used to determine a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image based on the first user image and the model image; The generation module is used to generate a second user image with the target pose, a second user pose image of the second user image, and a second user segmentation image based on the first user image, the first user pose image, the first user segmentation image, and the target pose image; The migration module is used to determine a third user image with the target clothing based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image; The step of determining a third user image with the target clothing based on the second user image, the second user pose image, the second user segmentation image, the model image, the model pose image, and the model segmentation image includes: Determine the coordinates of the third key point of the second user pose image and the coordinates of the fourth key point of the model pose image; Determine the key point coordinate mapping between the coordinates of the third key point and the coordinates of the fourth key point; The second user image, the model image, the second user segmentation image, the model segmentation image, and the key point coordinate mapping are input into a preset clothing transfer model. Based on the clothing transfer model and the key point coordinate mapping, the model segmentation image on the model image is transferred to the second user segmentation image on the second user image to obtain a third user image with the target clothing.

8. A data generation device, characterized in that, The data generation device includes: a memory, a processor, and a program stored in the memory for implementing the data generation method. The memory is used to store programs that implement the data generation method; The processor is configured to execute a program that implements the data generation method to carry out the steps of the data generation method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores a program for implementing the data generation method, which is executed by a processor to implement the steps of the data generation method as described in any one of claims 1 to 6.

Images