Matching management method in relay service between entertainment companies and trainees, and matching management server and computer program for performing same

Abstract

The present embodiments relate to a matching management server comprising: a processor; a memory for storing a program executed by the processor; and a communication interface for transmission and reception with user devices and business operator devices, wherein the communication interface receives application image information and application information of users from the user devices, receives preferred person information including artist image information and activity information of artists affiliated with business operators from the business operator devices, and transmits matching results to each of the user devices and the business operator devices, and the processor generates matching results including a business operator-specific matching rate for each user by applying the preferred person information, the application image information, and the application information to a matching model.

Description

A method for managing matching in a brokerage service between an entertainment company and a trainee, a matching management server and a computer program for performing the same.

[0001] The present invention relates to a method for managing matching in a relay service between an entertainment company and a trainee, a matching management server for performing the same, and a computer program.

[0002] The content described in this section merely provides background information regarding the present embodiment and does not constitute prior art.

[0003] Entertainment companies (agencies, corporations, etc.) are firms that provide various activities such as managing and nurturing entertainers or artists, planning albums and performances, and producing content. They discover and train idol groups, singers, actors, and comedians to introduce them to the public, and generate revenue through image management, promotional activities, and appearances in various media.

[0004] Aspiring entertainers and artists face difficulties in building networks to connect with companies, and these connections can be hampered by a lack of information. Furthermore, the absence of intermediaries or platforms to facilitate these connections between aspiring artists and companies can hinder mutual interaction, leading to issues with ineffective matching.

[0005] Therefore, as the entertainment business possesses competitive and complex characteristics, there are instances where proper matching does not occur or aspiring talents miss opportunities; thus, it is necessary to overcome these difficulties and establish a connection between the entertainment industry and aspiring talents.

[0006] The embodiments of the present invention aim to provide an inter-industry networking tool by mediating the matching of a user (e.g., a trainee) and a business operator (e.g., entertainment).

[0007] Other unspecified objects of the present invention may be further considered to the extent that they can be easily inferred from the following detailed description and effects.

[0008] According to one aspect of the present embodiment, the present invention proposes a matching management server comprising a processor, a memory storing a program executed by the processor, and a communication interface for transmitting and receiving with a user device and a carrier device, wherein the communication interface receives user support image information and support information from the user device, receives preferred person information including artist image information and activity information of an artist affiliated with the carrier from the carrier device, transmits a matching result to each of the user device and the carrier device, and the processor applies the preferred person information, the support image information, and the support information to a matching model to generate the matching result including a carrier-specific matching rate for each user.

[0009] According to another embodiment of the present invention, the present invention proposes a matching management method by a matching management server, comprising: an information receiving step of receiving user support image information and support information from a user device, and receiving preferred person information including artist image information and activity information of an artist affiliated with the operator from the operator device; a matching step of applying the preferred person information, the support image information, and the support information received through the information collection step to a matching model to generate the matching result including a matching rate for each user by operator; and an information transmission step of transmitting the matching result to each of the user device and the operator device.

[0010] As described above, according to the embodiments of the present invention, the present invention has the effect of increasing the matching success rate by assisting in the matching between entertainment and aspiring individuals, and efficiently managing multiple entertainment and aspiring individuals to provide matching as needed.

[0011] Even if an effect is not explicitly mentioned herein, the effects and potential effects described in the following specification expected by the technical features of the present invention are treated as described in the specification of the present invention.

[0012] FIG. 1 is a diagram illustrating a matching management system including a matching management server according to one embodiment of the present invention.

[0013] FIG. 2 is a block diagram illustrating a computing environment of a management server including a computing device suitable for use in preferred embodiments of the present invention.

[0014] FIGS. 3 to 5 are drawings illustrating in detail the generation of matching results through a matching management server according to an embodiment of the present invention.

[0015] FIG. 6 is an exemplary diagram showing activity information of a business operator according to one embodiment of the present invention.

[0016] FIG. 7 is a diagram showing the results provided to each user and business operator according to one embodiment of the present invention.

[0017] FIG. 8 is a diagram illustrating the generation of converted user information through a matching management server according to an embodiment of the present invention.

[0018] FIG. 9 is a diagram showing in detail the generation of converted user information through a matching management server according to one embodiment of the present invention.

[0019] FIG. 10 is a flowchart illustrating a matching management method through a matching management server according to an embodiment of the present invention.

[0020] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. The advantages and features of the present invention, and the methods for achieving them, will become clear by referring to the embodiments described below in detail together with the attached drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0021] Unless otherwise defined, all terms used in this specification (including technical and scientific terms) may be used in a meaning that is commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0022] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to indicate the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0023] Terms including ordinal numbers, such as second, first, etc., may be used to describe various components, but said components are not limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the second component may be named the first component, and similarly, the first component may be named the second component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0024] The present invention relates to a matching management method, a matching management server for performing the same, and a computer program.

[0025] FIG. 1 is a diagram illustrating a matching management system including a matching management server according to one embodiment of the present invention.

[0026] Referring to FIG. 1, the matching management system includes a matching management server (10), a user device (20), a business device (30), and a communication network (40).

[0027] The user device (20) and the operator device (30) are electronic devices that transmit data necessary for the matching management server (10) to process and receive the processed data. The user device (20) and the operator device (30) may be implemented as computing devices, and may include, but are not limited to, a smartphone, a personal computer (PC), a tablet PC, a personal digital assistant (PDA), a laptop, etc.

[0028] The matching management server (10), user device (20), and operator device (30) are connected to a communication network (40). The communication network (40) refers to a collection of communication facilities connected for the purpose of enabling communication between the matching management server (10), user device (20), and operator device (30). The communication network (40) includes nodes, circuits, trunks, and satellites, and these are connected and linked to each other.

[0029] The user device (20) and the operator device (30) can communicate via wired or wireless means. For example, various communication protocols such as short-range wireless communication, long-range wireless communication, mobile communication, and wireless LAN communication may be used for wireless communication. Examples of wireless communication protocols include Near Field Communication (NFC), ZigBee, Bluetooth, Wi-Fi, WiMAX, GSM (Global System For Mobile Communication), 3G (Third Generation) mobile communication, LTE (Long Term Evolution), 4G, and 5G, but are not limited thereto.

[0030] The matching management server (10) includes a database. A database refers to a form of data storage that allows for the free search, extraction, deletion, editing, and addition of data. The database can be implemented to suit the purpose of this embodiment using Oracle, Informix, Sybase, a Relational Database Management System (RDBMS), Gemston, Orion, an Object Oriented Database Management System (OODBMS), a distributed database, a cloud, etc.

[0031] FIG. 2 is a block diagram illustrating a computing environment of a management server including a computing device suitable for use in preferred embodiments of the present invention.

[0032] The matching management server (10), user device (20), and operator device (30) may be implemented as computing devices, and may include, but are not limited to, a smartphone, personal computer (PC), tablet PC, personal digital assistant (PDA), laptop, etc.

[0033] The matching management server (10) includes a database. A database refers to a form of data storage that allows for the free search, extraction, deletion, editing, and addition of data. The database can be implemented to suit the purpose of this embodiment using Oracle, Informix, Sybase, a Relational Database Management System (RDBMS), Gemston, Orion, an Object Oriented Database Management System (OODBMS), a distributed database, a cloud, etc.

[0034] A matching management server (10) according to a preferred embodiment of the present invention may include a processor and a memory that stores a program executed by the processor, and the processor may perform all processes.

[0035] In the embodiment illustrated in FIG. 1, each component may have different functions and capabilities in addition to those described below, and may include additional components in addition to those not described below.

[0036] The illustrated computing environment includes a matching management server (10). In one embodiment, the matching management server (10) may be any type of computing device that transmits and receives signals to and from other terminals.

[0037] The matching management server (10) includes at least one processor (12), a computer-readable storage medium (14), and a communication bus (19). The processor (12) can cause the matching management server (10) to operate according to the exemplary embodiment described above. For example, the processor (12) can execute one or more programs stored in the computer-readable storage medium (14). The one or more programs may include one or more computer-executable instructions, and the computer-executable instructions may be configured to cause the moving target monitoring device (10) to perform operations according to the exemplary embodiment when executed by the processor (12).

[0038] A computer-readable storage medium (14) is configured to store computer-executable instructions or program code, program data and / or other suitable forms of information. A program (15) stored in the computer-readable storage medium (14) includes a set of instructions executable by a processor (12). In one embodiment, the computer-readable storage medium (14) may be memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other forms of storage media that are accessed by a matching management server (10) and capable of storing desired information, or a suitable combination thereof.

[0039] The communication bus (19) interconnects various other components of the matching management server (10), including the processor (12) and the computer-readable storage medium (14).

[0040] The matching management server (10) may also include one or more input / output interfaces (16) and one or more communication interfaces (18) that provide interfaces for one or more input / output devices (not shown). The input / output interfaces (16) and communication interfaces (18) are connected to a communication bus (19). Input / output devices (not shown) may be connected to other components of the matching management server (10) through the input / output interfaces (16). Exemplary input / output devices may include input devices such as a pointing device (mouse or trackpad, etc.), a keyboard, a touch input device (touchpad or touchscreen, etc.), a voice or sound input device, various types of sensor devices and / or imaging devices, and / or output devices such as a display device, a printer, a speaker and / or a network card. Exemplary input / output devices (not shown) may be included inside the matching management server (10) as a component constituting the matching management server (10), or may be connected to a computing device as a separate device distinct from the matching management server (10).

[0041] The matching management server (10) of the present invention may include a processor, a memory for storing a program executed by the processor, and a communication interface for transmitting and receiving with a user device and a business device.

[0042] The communication interface of the matching management server (10) receives user support image information and support information from a user device, receives preferred person information including artist image information and activity information of an artist affiliated with the operator from an operator device, and can transmit matching results to each of the user device and the operator device.

[0043] The processor of the matching management server (10) can apply preferred person information, support image information, and support information to the matching model to generate a matching result including a matching rate for each user and a matching rate for each operator.

[0044] The matching management server (10) can register support image information and support information received from the user device as user information, and register artist image information and activity information received from the business device as business information.

[0045] The matching management server (10) can apply registered support image information or artist image information to the image extraction module to extract landmark points according to the face area. At this time, the landmark points may include at least one eye, nose, mouth, and contour.

[0046] The matching management server (10) performs a normalization process that converts the face region into a matchable state by rotating it based on the extracted landmark points, and the matching management server (10) embeds the face region that has undergone the normalization process and converts it into a vector of a preset size, and can calculate the similarity between the support image information and the artist image information using the converted feature vector.

[0047] The matching management server (10) can generate user viewing information by reconstructing user information so that information about each user can be viewed through the operator's device.

[0048] The matching management server (10) provides user viewing information to a business device through a communication interface and can receive activity information from the business device including at least one viewing count, contact count, and selection status based on the user viewing information.

[0049] When audition announcement information transmitted through a business device and support information transmitted through a user device are input into a matching model, the matching management server (10) can generate a first feature vector based on audition announcement information and a second feature vector based on support information by considering the correlation between audition announcement information and support information, and when support image information and artist image information are input into the matching model, it can generate a third feature vector.

[0050] The matching management server (10) can apply the first feature vector, the second feature vector, and the third feature vector to a classification module to generate a matching result including a matching rate for each operator for each user. At this time, the matching result may include an entertainment matching result including a matching rate for each of multiple operators based on the user, and a support matching result including a matching rate for each user based on the operator.

[0051] The matching management server (10) can convert audition announcement information transmitted through a business device and support information transmitted through a user device into vectors of a preset size when they are input into an embedding module, and input each of the converted audition announcement information and support information into a convolutional neural module to extract features, and input each of the extracted features into a sham module to generate a first feature vector according to the audition announcement information and a second feature vector according to the support information.

[0052] The matching management server (10) can convert the support image information and artist image information into a vector of a preset size when input into the embedding module, input the converted support image information and artist image information into the vision transformer module to divide the image into patches and extract features according to the image, and input the extracted features into the mask sham module to generate a third feature vector according to the support image information and artist image information.

[0053] The matching management server (10) can further consider activity information including the business operator's activities regarding the user, and when the activity information is input into the matching model, it can generate a fourth feature vector according to the support information and a fifth feature vector according to the activity information by considering the relationship between the activity information and the support information.

[0054] The matching management server (10) can regenerate a matching result including a matching rate by operator for each user by further applying the fourth feature vector and the fifth feature vector to the classification model.

[0055] The matching management server (10) can generate transformed user information in which at least one procedure, makeup, and styling are applied based on landmark points extracted based on user information.

[0056] The matching management server (10) can provide conversion information to a user device through a communication interface, including at least one corresponding procedure information, makeup product information, styling product information, makeup class information, and styling class information for each landmark point based on the generated conversion user information, and can provide conversion user information to a business device through a communication interface.

[0057] The matching management server (10) can generate converted user information corresponding to each business operator by using business operator-specific features extracted through artist images and option images based on business operators linked to business operators, based on preferred person information.

[0058] FIGS. 3 to 5 are drawings illustrating in detail the generation of matching results through a matching management server according to an embodiment of the present invention.

[0059] FIG. 3 is a diagram showing the generation of a matching result through a matching management server according to an embodiment of the present invention.

[0060] Referring to FIG. 3, the matching management server may include a collection unit (310), a processing unit (320), a classification unit (330), a model unit (340), and a matching unit (350), and among the various components illustrated as examples, some components may be omitted or additional components may be included.

[0061] The collection unit (310) can collect support image information and support information from a user through a user device, and artist image information and activity information from a business operator through a business operator device. At this time, the support image information and artist image information may be implemented as a full-body image or upper-body image showing the face, but are not necessarily limited thereto.

[0062] The collection unit (310) can classify artist image information by labeling it according to the business operator.

[0063] The processing unit (320) receives support image information and artist image information and can process only the face portion. Specifically, the processing unit (320) can find and extract the face area from the support image information and artist image information collected through the collection unit (310).

[0064] The classification unit (330) can classify and label the images processed through the processing unit (320) by business operator. At this time, the labeling can be done through naming for each business operator (e.g., entertainment company), but is not necessarily limited to this.

[0065] The classification unit (330) can label each of the processed support image information by further considering activity information that indicates interest for each business operator. At this time, the activity information may include whether the business operator selects a user (Pick), but is not necessarily limited thereto.

[0066] The model unit (340) can fine-tune by inputting the image processed through the processing unit (320) and the artist image classified through the classification unit (330) into the matching model.

[0067] The model unit (340) extracts major landmark points (eyes, nose, mouth, face, contour, etc.) for each of the support image information and artist image information extracted as a face region, rotates the face region based on the extracted landmark points and changes it to a state where face matching is possible, and embeds it into the face region to represent it as an N-dimensional feature vector.

[0068] The matching unit (350) can generate matching results by calculating the recommendation / matching rate for each business operator based on the data fine-tuned through the model unit (340).

[0069] The matching unit (350) can generate a matching result using the similarity between feature vectors, but is not necessarily limited thereto.

[0070] FIGS. 4 and FIGS. 5 are drawings illustrating in detail the generation of matching results through a matching management server according to an embodiment of the present invention.

[0071] Referring to FIG. 4, when audition announcement information transmitted through a business device and support information transmitted through a user device are input into a matching model, a first feature vector according to the audition announcement information and a second feature vector according to the support information are generated by considering the correlation between the audition announcement information and the support information, and when support image information and artist image information are input into the matching model, a third feature vector is generated, and at least one of the first feature vector, the second feature vector and the third feature vector is applied to a classification module to generate a matching result including a business-specific matching rate for each user.

[0072] The first feature vector and the second feature vector can be converted into vectors of audition announcement information and user (applicant) application information through a sham module, and then the distance can be calculated.

[0073] Since appearance is one of the important evaluation factors, a third feature vector can be extracted using applicant image information and artist image information. In this process, the third feature vector can be derived by extracting company information, such as appearance preferences for each company, and extracting applicant information, then combining the two.

[0074] Referring to FIG. 5, by further considering activity information including the operator's activities regarding the user, when the activity information is input into the matching model, a fourth feature vector according to the support information and a fifth feature vector according to the activity information can be generated by considering the correlation between the activity information and the support information.

[0075] The fourth feature vector and the fifth feature vector can be generated by classifying support information and the activity information of the operator (casting manager) viewing it.

[0076] Classification is performed using the applicant's profile information and the activity information of the entertainment casting manager viewing it.

[0077] The fourth feature vector can be generated using the user's (applicant's) application information (e.g., age, height, nationality, weight, etc.).

[0078] The fifth feature vector can be generated using the entertainment manager's activity information (e.g., likes, views, number of messages, etc.).

[0079] Referring to Figures 4 and 5, a sham module is a neural network structure in which two or more neural networks share the same weights and are used to compare similarity between inputs. It is a network that primarily takes pairs as inputs and is used to measure how similar two inputs are.

[0080] Accordingly, two different inputs can be provided to the sham module simultaneously, for example, audition announcement information and application information and application information and activity information and artist image information and application image information can be input.

[0081] The two inputs pass through the same neural network to generate respective feature vectors (embeddings), and the neural network can process the two inputs using the same weights. In other words, it performs the same feature extraction for both inputs.

[0082] Similarity between pairs of inputs is measured by calculating the distance between two feature vectors (usually Euclidean distance or cosine similarity); a smaller distance value indicates that the two inputs are more similar, but this is not necessarily the only factor.

[0083] Since the Siamese module processes inputs using equal weights, it can reduce the number of training parameters and ensure consistent feature representation by undergoing the same feature extraction process. Additionally, by training primarily on pairs of similar or different data, the network can learn the similarity between two inputs.

[0084] Referring to Figure 4, the Masked Siamese Networks (MSN) can utilize a vi transformer encoder capable of grouping and recommending images through the combination of Siamese and vi transformers, and can classify images within the same group based on learned images. In this case, few-shot and one-shot learning are possible, allowing the model to be trained using a small amount of data (such as applicant profile photos).

[0085] Referring to Fig. 4, the Vision Transformer module (ViT) can utilize a vi Transformer encoder capable of recognizing and classifying images within the same group based on ViT-trained images that can utilize vision tasks and Transformer encoders. Since few-shot and one-shot learning are possible, the model can be trained using small amounts of data (such as applicant profile photos), and it can be advantageous when using large datasets such as YOLO and fine-tuning using applicant profile photos.

[0086] The Vision Transformer Module is a network that applies the Transformer model to image processing tasks. It takes image data as input and utilizes the Transformer architecture to perform vision tasks such as classification, and can process images through the Transformer's Self-Attention mechanism.

[0087] The Vision Transformer module ViT can divide an image into small patches of a fixed size instead of using global filters like CNN when processing images. For example, it can divide the image into 16x16 patches.

[0088] In addition, each image patch is converted into a vector, which can perform a role similar to the tokens used in Transformers. The patch vectors are transformed into embedding vectors in a high-dimensional space, and each patch contains information about a specific region within the image.

[0089] Since the transformer is not sensitive to order, additional positional information between patches must be provided, so positional encoding is added to the patch vector to reflect the positional information of each patch.

[0090] When a Transformer encoder structure is applied to embedded patch vectors, it learns the relationships between patches through the Transformer's Self-Attention mechanism and can extract global features. At this time, it can learn how each patch relates to others.

[0091] By adding a special token called a Classification Token (CLS) to ViT, information about the entire image can ultimately be concentrated on this token. In this case, the Class Token can be used for the final prediction.

[0092] Based on the information processed by the encoder, the output layer can finally perform image classification; the output represents class-specific probabilities and can primarily lead to a softmax layer.

[0093] Therefore, while CNNs primarily learn local features, the Vision Transformer module can learn global relationships across the entire image, allowing it to consider a broader context; furthermore, it can provide very high performance when pre-trained on large datasets and fine-tuned on small datasets.

[0094] FIG. 6 is an exemplary diagram showing activity information of a business operator according to one embodiment of the present invention.

[0095] Referring to FIG. 6, activity information may include whether a selection was made, the number of views, and the number of contacts, but is not necessarily limited thereto.

[0096] The selection status indicates the result picked by the business operator (e.g., an entertainment casting manager) for each user (applicant).

[0097] The number of views indicates the number of times a business operator (e.g., an entertainment casting manager) has viewed the application image information or application information of each user (applicant).

[0098] The number of contacts indicates the number of times a business operator (e.g., an entertainment casting manager) contacted each user (applicant).

[0099] Activity information can be used to analyze correlations to calculate the favorability of each applicant and label them as preferred users for each business operator.

[0100] FIG. 7 is a diagram showing the results provided to each user and business operator according to one embodiment of the present invention.

[0101] Referring to Fig. 7, when a user (e.g., an applicant) inputs application image information and application information, they can receive a matching result including the matching rate for each entertainment agency through the process described above by finding an agency that is a good fit for them.

[0102] When a business operator (e.g., an entertainment casting manager) inputs artist image and activity information, they can receive matching rates for each user (e.g., provided in order of highest to lowest) and users with multiple business operators or those with a pre-set matching rate or higher through a view of candidates who are a good fit for the entertainment.

[0103] FIG. 8 is a diagram illustrating the generation of converted user information through a matching management server according to an embodiment of the present invention.

[0104] Referring to FIG. 8, by using support image information, it is possible to generate transformed user information in which at least one procedure, makeup, and styling is applied.

[0105] Transformed user information represents an image to which transformation information has been applied to the user's current face. Here, the transformation information includes, but is not necessarily limited to, procedures, makeup, and styling.

[0106] The converted user information can apply the support image information to the image extraction module to extract landmark points including at least one eye, nose, mouth, and contour according to the face region.

[0107] "Procedure" refers to facial plastic surgery procedures such as eye, nose, and jaw procedures; "Makeup" refers to makeup based on pre-stored eye, nose, mouth, and skin expressions; and "Coordi" refers to the outfits of tops and bottoms, accessories, etc.

[0108] The matching management server can be implemented to allow users to select a business operator for each procedure, makeup, and styling, thereby enabling the application of the operator's preferred procedures, makeup, and styling. Specifically, the matching management server can extract appearance, makeup, and code characteristics for each operator based on artist image information for each operator, and apply these characteristics to the user to generate transformed user information featuring the image preferred by each operator. Additionally, the matching management server can analyze results from affiliated clinics or general treatment hospitals and makeup shops to extract characteristics, and by applying these extracted characteristics, provide a way to verify the image transformation effect for each store.

[0109] The matching management server may provide users with profile picture additions, treatment consultation services, beauty makeup training courses, related beauty products, styling mentoring, styling products, etc., through the generated converted user information, but is not necessarily limited to these.

[0110] FIG. 9 is a diagram showing in detail the generation of converted user information through a matching management server according to one embodiment of the present invention.

[0111] Referring to Fig. 9, face region extraction and learning parameters of input support image information can be configured.

[0112] Face region extraction can detect and separate face parts from support image information. For example, computer vision techniques such as OpenCV can be used to accurately separate face regions to generate target images for the model to learn from, but are not necessarily limited to this.

[0113] According to one embodiment of the present invention, parameters necessary for training a LoRA (Low-Rank Adaptation) model for image generation through a Stable Diffusion model can be set, for example, parameters such as a training rate, batch size, and number of training iterations can be set, but are not necessarily limited thereto.

[0114] The aforementioned training parameters can be delivered to AWS Batch via AWS Step Function. Here, a Step Function represents a workflow that automates a multi-step process and allows for the management and monitoring of each step.

[0115] AWS Batch can fine-tune the Stable Diffusion model based on the provided parameters.

[0116] After the model completes the fine-tuning process, the generated face image LoRA model can be uploaded to object storage.

[0117] Through AWS Batch, you can (1) create an EC2 instance, (2) download the Stable Diffusion inference container image from ECR to EC2, and (3) download the "face image LoRA fine-tuning model" stored in object storage to the EC2 instance, and then create a user's profile image.

[0118] AWS Batch can perform inference tasks by creating new EC2 instances. Here, EC2 represents AWS's virtual server service.

[0119] Download the Stable Diffusion container image for the inference task to an EC2 instance, and the container may include an environment capable of running Stable Diffusion inference and necessary libraries.

[0120] You can import a fine-tuned face image LoRA model stored in object storage into an EC2 instance and prepare it for use in inference.

[0121] In the environment prepared through the aforementioned process, a user's profile image can be generated using a Stable Diffusion model. At this time, an optimized image reflecting the user's face can be generated by utilizing a fine-tuned LoRA model, but is not necessarily limited to this.

[0122] The image (converted user information) generated through the above-described process can be uploaded to object storage and provided to users or business operators.

[0123] FIG. 10 is a flowchart illustrating a matching management method through a matching management server according to an embodiment of the present invention. The matching management method through a matching management server can be performed by a matching management server, and a description that overlaps therewith is omitted.

[0124] The matching management method includes an information receiving step (S1010) of receiving user support image information and support information from a user device and receiving preferred person information including artist image information and activity information of an artist affiliated with a business from a business device; a matching step (S1020) of applying the preferred person information, support image information, and support information received through the information collection step to a matching model to generate a matching result including a business-specific matching rate for each user; and an information transmission step (S1030) of transmitting the matching result to each of the user device and the business device.

[0125] Although FIG. 10 describes each process as being executed sequentially, this is merely an illustrative example, and a person skilled in the art may modify and adapt the process in various ways without departing from the essential characteristics of the embodiment of the present invention, such as changing the order described in FIG. 10, executing one or more processes in parallel, or adding other processes.

[0126] The foregoing description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications, changes, and substitutions within the scope of the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are intended to explain, not limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by such embodiments and accompanying drawings. The scope of protection of the present invention shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present invention.

Claims

1. A matching management server comprising a processor, a memory for storing a program executed by said processor, and a communication interface for transmitting and receiving with a user device and a carrier device, The communication interface receives user support image information and support information from the user device, receives preferred person information including artist image information and activity information of an artist affiliated with the operator from the operator device, and transmits a matching result to each of the user device and the operator device. A matching management server characterized by the processor applying the preferred person information, the support image information, and the support information to a matching model to generate the matching result including a matching rate by operator for each user.

2. In Paragraph 1, The above processor is, The support image information and support information received from the user device are registered as user information, and the artist image information and activity information received from the operator device are registered as operator information. The above-mentioned registered support image information or artist image information is applied to the image extraction module to extract landmark points according to the face region, and A matching management server characterized in that the above landmark point includes at least one eye, nose, mouth, and contour.

3. In Paragraph 2, The above processor is, A matching management server characterized by performing a normalization process to transform the face region into a matchable state by rotating it based on the extracted landmark points, embedding the face region that has undergone the normalization process to convert it into a vector of a preset size, and calculating the similarity between support image information and artist image information using the transformed feature vector.

4. In Paragraph 3, The above processor is, User viewing information is generated by reconstructing user information so that information for each of the above users can be viewed through the above-mentioned operator device, and A matching management server characterized by the above communication interface providing the above user viewing information to the above operator device and receiving the above activity information from the above operator device, which includes at least one viewing count, contact count, and selection status performed based on the above user viewing information.

5. In Paragraph 1, The above processor is, When the audition announcement information transmitted through the above-mentioned operator device and the above-mentioned support information transmitted through the above-mentioned user device are input into the above-mentioned matching model, a first feature vector according to the above-mentioned audition announcement information and a second feature vector according to the above-mentioned support information are generated by considering the correlation between the above-mentioned audition announcement information and the above-mentioned support information, and when the above-mentioned support image information and the above-mentioned artist image information are input into the above-mentioned matching model, a third feature vector is generated. At least one of the first feature vector, the second feature vector, and the third feature vector is applied to a classification module to generate the matching result including a matching rate by operator for each of the users, and A matching management server characterized by the above matching results including an entertainment matching result that includes a matching rate for each of a plurality of operators based on a user, and a support matching result that includes a matching rate for each user based on an operator.

6. In Paragraph 5, The above processor is, A matching management server characterized by converting audition announcement information transmitted through the above-mentioned business operator device and support information transmitted through the above-mentioned user device into vectors of a preset size when each is input into an embedding module, inputting each of the converted audition announcement information and support information into a convolutional neural module to extract features, and inputting each of the extracted features into a sham module to generate a first feature vector according to the audition announcement information and a second feature vector according to the support information.

7. In Paragraph 5, The above processor is, A matching management server characterized by converting the support image information and the artist image information into vectors of a preset size when input into an embedding module, inputting the converted support image information and artist image information into a vision transformer module to divide the image into patches and extract features according to the image, and inputting the extracted features into a mask sham module to generate a third feature vector according to the support image information and the artist image information.

8. In Paragraph 5, The above processor is, Further considering the above activity information including the operator's activities regarding the user, when the above activity information is input into the matching model, a fourth feature vector according to the support information and a fifth feature vector according to the activity information are generated by considering the correlation between the above activity information and the above support information. A matching management server characterized by further applying the above-mentioned fourth feature vector and the above-mentioned fifth feature vector to the above-mentioned classification model to regenerate the above-mentioned matching result including the operator-specific matching rate for each of the above-mentioned users.

9. In Paragraph 2, The above processor is, Based on the above user information, transformed user information is generated with at least one procedure, makeup, and styling applied based on landmark points extracted, and A matching management server characterized by providing conversion information to the user device through the communication interface, the conversion information including at least one corresponding procedure information, makeup product information, styling product information, makeup class information, and styling class information for each landmark point based on the generated conversion user information, and providing the conversion user information to the operator device through the communication interface.

10. In Paragraph 9, The above processor is, A matching management server characterized by generating the above-mentioned converted user information corresponding to each business operator using business-specific features extracted through the above-mentioned artist image and option images according to the business operator linked to the business operator, based on the above-mentioned preferred person information.

11. In a matching management method by a matching management server, Information reception step of receiving user's support image information and support information from a user device, and receiving preferred person information including artist image information and activity information of an artist affiliated with the operator from the operator device; A matching step for generating the matching result including a matching rate by operator for each user by applying the preferred person information, the support image information, and the support information received through the information collection step to a matching model; and A matching management method comprising an information transmission step of transmitting a matching result to each of the user device and the operator device.

12. In Paragraph 11, The above matching step is, When the audition announcement information transmitted through the above-mentioned operator device and the above-mentioned support information transmitted through the above-mentioned user device are input into the above-mentioned matching model, a first feature vector according to the audition announcement information and a second feature vector according to the above-mentioned support information are generated by considering the correlation between the above-mentioned audition announcement information and the above-mentioned support information; when the above-mentioned support image information and the above-mentioned artist image information are input into the above-mentioned matching model, a third feature vector is generated by further considering the above-mentioned activity information including the operator's activities regarding the user, and when the above-mentioned activity information is input into the above-mentioned matching model, a fourth feature vector according to the above-mentioned support information and a fifth feature vector according to the above-mentioned activity information are generated by considering the correlation between the above-mentioned activity information and the above-mentioned support information. At least one of the first feature vector, the second feature vector, the third feature vector, the fourth feature vector, and the fifth feature vector is applied to a classification module to generate the matching result including a matching rate by operator for each of the users, and A matching management method comprising an enter matching result including a matching rate for each of a plurality of business operators based on a user, and a support matching result including a matching rate for each user based on a business operator.

13. A computer program stored on a computer-readable recording medium to execute the matching management method described in either paragraph 11 or 12 on a computer.

Images