Information processing device and program

Abstract

To provide an information processing device and program that can accurately suggest clothing items that meet customer needs, regardless of product sales. [Solution] The clothing information suggestion device (information processing device) comprises: a text conversion unit that converts customer speech data relating to clothing items the customer desires into text data; a clothing item selection unit that, through a learning process that associates multiple keywords representing the characteristics of clothing items with panel evaluations of clothing items and the clothing items themselves, selects at least one clothing database from among multiple pre-generated clothing databases for each characteristic of clothing item, based on the characteristics extracted from the text data, to store clothing items that match the customer's requests; and a virtual image generation unit that generates a virtual image by superimposing the clothing items belonging to the clothing database selected by the clothing item selection unit onto the customer's image.

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus and a program.

Background Art

[0002] Conventionally, an invention has been disclosed that acquires a customer's utterance regarding a desired clothing item and a customer's image for a customer who has come to the store, and proposes a clothing item according to the customer (for example, Patent Document 1).

[0003] In the invention disclosed in Patent Document 1, clothing items actually purchased by past visitors to the store are used as one of the learning data. Therefore, as the learning data related to clothing items with good sales performance is further enhanced, clothing items with better sales performance are more recommended to customers. Therefore, it was not always possible to propose clothing items that met the customer's needs.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to provide an information processing apparatus and a program that can propose accurate clothing items according to the customer's needs without being affected by the sales performance of the product.

Means for Solving the Problems

[0005] The information processing device of this embodiment comprises a text conversion unit, a selection unit, and a virtual image generation unit. The text conversion unit acquires the customer's spoken information relating to the clothing items it desires and converts the spoken information into text information. The selection unit, through a learning process that associates multiple keywords representing the characteristics of clothing items with panel evaluations of the clothing items and the clothing items themselves, selects at least one clothing database from among multiple pre-generated clothing databases for each characteristic of clothing items, based on the characteristics extracted from the text information, which contains clothing items that match the customer's request. The virtual image generation unit generates a first virtual image by superimposing the clothing items belonging to the clothing database selected by the selection unit onto the customer's image. [Brief explanation of the drawing]

[0006] [Figure 1] Figure 1 is a block diagram showing an example of the schematic configuration of the clothing information suggestion system according to the embodiment. [Figure 2] Figure 2 is a hardware block diagram showing an example of the hardware configuration of the clothing information suggestion system of the embodiment. [Figure 3] Figure 3 shows an example of the data structure of a product master. [Figure 4] Figure 4 illustrates the method for generating a clothing database. [Figure 5] Figure 5 is a functional block diagram showing an example of the functional configuration of a clothing information suggestion system. [Figure 6] Figure 6 shows an example of a speech instruction screen in which a clothing information suggestion system prompts the customer to speak about the image of the clothing they want. [Figure 7] Figure 7 shows an example of a shooting instruction screen in a clothing information suggestion system that prompts customers to take their own photos. [Figure 8] Figure 8 shows an example of a screen displaying potential clothing items that a customer might want, based on their spoken responses. [Figure 9] Figure 9 shows an example of a virtual image in which the clothing items desired by the customer are superimposed onto the customer's own image. [Figure 10] Figure 10 is a flowchart showing an example of the processing flow performed by the clothing information suggestion system. [Figure 11] Figure 11 shows an example of how a modified version of the clothing information suggestion system operates. [Modes for carrying out the invention]

[0007] The following describes in detail an embodiment of the clothing information suggestion system 10 of this disclosure with reference to the drawings. However, the present invention is not limited to the embodiments described below.

[0008] (Outline of the fashion information suggestion system) The schematic configuration of the clothing information suggestion system 10 of the embodiment will be explained using Figure 1. Figure 1 is a block diagram showing an example of the schematic configuration of the clothing information suggestion device of the embodiment.

[0009] The clothing information suggestion system 10 acquires speech data indicating the image of clothing items desired by customers visiting a clothing store. The clothing information suggestion system 10 then analyzes the customer's speech data to estimate the clothing items the customer desires. Furthermore, the clothing information suggestion system 10 presents the estimated clothing items desired by the customer to the customer by superimposing them onto the customer's own image.

[0010] The clothing information suggestion system 10 has a configuration in which a customer terminal 20 and a clothing information suggestion device 30 are connected by a network 12.

[0011] The customer terminal 20 uses its built-in microphone 28 to receive spoken statements from the customer regarding the clothing items they desire. The customer terminal 20 also uses its built-in camera 29 to capture an image of the customer (either a half-body or full-body shot). Furthermore, the customer terminal 20 outputs the customer's speech and image to the clothing information suggestion device 30. The customer terminal 20 then displays on the monitor 26 the clothing items the clothing information suggestion device 30 has estimated from the customer's speech, as well as a virtual image of those items superimposed on the customer's image. The customer terminal 20 is, for example, a tablet device. To accommodate multiple customers, the store is equipped with multiple customer terminals 20. Each customer terminal 20 is assigned unique identification information (e.g., an identification number).

[0012] The clothing information suggestion device 30 converts the customer's spoken utterances regarding the clothing items they desire into text information. The clothing information suggestion device 30 then analyzes the content of the text information to select the clothing items desired by the customer from the clothing database 325 (see Figure 2). The clothing information suggestion device 30 also generates a virtual image by superimposing the clothing items estimated from the customer's utterances onto the customer's image (half-body or full-body). The clothing information suggestion device 30 is an example of an information processing device in this disclosure.

[0013] Network 12 is, for example, a LAN (Local Area Network). Network 14 may be wired or wireless.

[0014] (Hardware configuration of the fashion information suggestion system) The hardware configuration of the clothing information suggestion system 10 will be explained using Figure 2. Figure 2 is a hardware block diagram showing an example of the hardware configuration of the clothing information suggestion system in this embodiment. For the sake of simplicity, the clothing information suggestion system 10 is assumed to be equipped with only one customer terminal 20.

[0015] First, the hardware configuration of the clothing information proposal device 30 will be described. The clothing information proposal device 30 included in the clothing information proposal system 10 has a configuration in which a control unit 31, a storage unit 32, and a communication controller 34 are connected to each other by an internal bus 33.

[0016] The control unit 31 controls the overall operation of the clothing information proposal device 30. The control unit 31 includes a CPU (Central Processing Unit) 311, a ROM (Read Only Memory) 312, and a RAM (Random Access Memory) 313. The CPU 311 is connected to the ROM 312 and the RAM 313 via an internal bus such as an address bus and a data bus. The CPU 311 expands various programs stored in the ROM 312 and the storage unit 32 into the RAM 313. The CPU 311 controls the operation of the clothing information proposal device 30 by operating according to various programs expanded in the RAM 313. That is, the control unit 31 has the configuration of a general computer.

[0017] The storage unit 32 is a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). Further, the storage unit 32 may be a non-volatile memory such as a flash memory that retains stored information even when the power is turned off. The storage unit 32 stores a control program 321, a product master 322, text data 323, a large language model 324, and a clothing database 325.

[0018] The control program 321 is a program that controls the overall operation of the clothing information proposal device 30. The control program 321 may be provided in a state stored in the storage unit 32, or may be recorded and provided on a computer-readable non-temporary recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD, etc. in an installable or executable file format. Further, the control program 321 may be stored on a computer connected to a network and provided by downloading via the network. Furthermore, the control program 321 may be provided or distributed via a network such as the Internet.

[0019] The product master 322 is a master file that stores product information of the clothing items handled in the store in association with a product ID that uniquely identifies the clothing items. The product information includes at least a category, a product name, a price, and keywords. Details of the structure of the product master 322 will be described later (see FIG. 3).

[0020] The text data 323 is data obtained by converting, by the clothing information proposal device 30, the speech data input from the customer terminal 20 into text. The text data 323 includes identification information that identifies the customer terminal 20 from which the speech data was input, and data obtained by converting the input speech data into text. The text data 323 is an example of the text information in the present disclosure.

[0021] The large-scale language model 324 is a language model used to analyze the meaning of text data 323. More specifically, the large-scale language model 324 is a natural language processing (NLP) model built using learning from vast amounts of data and deep learning techniques. By fine-tuning the large-scale language model 324, it can be applied to text classification, sentiment analysis, information extraction, text summarization, text generation, question answering, etc., and is now widely used. For example, the well-known chat GPT is a fine-tuned version of the large-scale language model 324 for chat. The large-scale language model 324 in the clothing information suggestion device 30 is a model that extracts the content and characteristics of clothing items desired by customers from text data 323. That is, the large-scale language model 324 analyzes the content of text data 323 and extracts keywords related to the types of clothing items and their characteristics. The large-scale language model 324 also clarifies the relationships between the types of clothing items and their characteristics. The features of the large-scale language model 324 will be described in more detail later (see Figure 5).

[0022] The clothing database 325 is a database that stores examples of clothing items handled by the store, Panera's comments on those examples, examples of coordinating them with other clothing items, and images of Panera. Further details about the clothing database 325 will be described later (see Figure 4).

[0023] Furthermore, the control unit 31 communicates with the customer terminal 20 via the communication controller 34.

[0024] For example, the clothing information suggestion device 30 acquires the customer's speech data and image from the customer terminal 20. The clothing information suggestion device 30 also outputs to the customer terminal 20 information about clothing items estimated to suit the customer's preferences, and a virtual image in which the selected clothing items are superimposed on the customer's image.

[0025] Next, the hardware configuration of the customer terminal 20 will be described. The customer terminal 20 provided by the clothing information suggestion system 10 has a configuration in which a control unit 21, a storage unit 22, a peripheral device controller 24, and a communication controller 25 are connected to each other by an internal bus 23.

[0026] The control unit 21 controls the overall operation of the customer terminal 20. The control unit 21 comprises a CPU 211, a ROM 212, and a RAM 213. The CPU 211 connects to the ROM 212 and the RAM 213 via internal buses such as an address bus and a data bus. The CPU 211 loads various programs stored in the ROM 212 and the memory unit 22 into the RAM 213. The CPU 211 controls the operation of the customer terminal 20 by operating according to the various programs loaded into the RAM 213. In other words, the control unit 21 has the configuration of a typical computer.

[0027] The storage unit 22 is a storage device such as an HDD or SSD. Alternatively, the storage unit 22 may be a non-volatile memory such as flash memory that retains stored information even when the power is turned off. The storage unit 22 stores the control program 221, the speech data 222, and the image data 223.

[0028] The control program 221 is a program that controls the overall operation of the customer terminal 20. The control program 221 may be provided stored in the storage unit 22, or it may be provided as an installable or executable file recorded on a computer-readable non-temporary recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD. The control program 221 may also be provided by storing it on a computer connected to a network and downloading it via the network. Furthermore, the control program 221 may be provided or distributed via a network such as the Internet.

[0029] The speech data 222 is audio data of a customer's speech recorded by the microphone 28. The speech data 222 is an example of speech information in this disclosure.

[0030] Image data 223 is an image of the customer's face and body taken by camera 29. The image of the customer's face and body may be, for example, a half-body or full-body image of the customer.

[0031] The control unit 21 connects to various information input / output devices, namely the monitor 26, touch panel 27, microphone 28, and camera 29, via the peripheral device controller 24.

[0032] Monitor 26 is a display device that presents various screen information to customers. Monitor 26 is, for example, an LCD monitor or an OLED monitor.

[0033] The touch panel 27 is a device that detects customer operation instructions. The touch panel 27 is installed stacked on the monitor 26, and the various buttons displayed on the monitor 26 serve as the controls.

[0034] Microphone 28 is a sound-collecting device that captures the customer's speech. Microphone 28 is installed, for example, on the periphery of the customer terminal 20, facing the same direction as the monitor 26.

[0035] Camera 29 is a camera that captures images of customers. Camera 29 is installed, for example, on the periphery of the customer terminal 20, facing the same direction as the monitor 26.

[0036] Furthermore, the control unit 21 communicates with the clothing information suggestion device 30 via the communication controller 25.

[0037] For example, the customer terminal 20 outputs the customer's speech data and the customer's image to the clothing information suggestion device 30. The customer terminal 20 also obtains information from the clothing information suggestion device 30 regarding clothing items estimated to suit the customer's preferences, as well as a virtual image created by superimposing the selected clothing items onto the customer's image.

[0038] (Data structure of the product master) Figure 3 illustrates the data structure of the product master 322. Figure 3 shows an example of the data structure of the product master.

[0039] Product Master 322 is a master file that stores product information for clothing items handled by a store, associated with a product ID, which is unique identification information for each clothing item. Product information includes category, product name, price, and keywords.

[0040] A category is information that indicates the category of a clothing item. For example, a category represents the type of clothing item, such as a shirt, jacket, sweater, or pants.

[0041] The product name is information that indicates the specific product name of a clothing item.

[0042] The price is information indicating the cost of clothing items.

[0043] Keywords are information that describes the characteristics of clothing items. These characteristics include, for example, style, trends, sales performance, and price. Ideally, keywords should include vocabulary that reflects points customers consider when searching for clothing items.

[0044] Note that the information stored in the product master 322 is not limited to the example in Figure 3. For example, the product master 322 may include the size and color of the clothing item, the product IDs of similar clothing items, the display location of the product in the store, and the storage location of the product in the back room.

[0045] (Method for generating a clothing database) Figure 4 illustrates the method for generating the clothing database 325.

[0046] The clothing database 325 is generated by aggregating Panera's comments 62 (hereinafter simply referred to as comments 62) on the best coordination examples 61 prepared by the stores. Comments 62 are examples of comments in this disclosure.

[0047] Specifically, before selling clothing items, store sales staff create a collection of 61 best-coordinated outfit examples, referencing combinations recommended in magazines, online, and the company's own sales promotion strategies. This collection of 61 best-coordinated outfit examples is like a product catalog, including images and videos of models wearing the clothing items.

[0048] The store's sales staff will have potential buyers and a large number of panelists view the collection of best outfit examples 61 they have created and collect their feedback 62. At the same time, they will also obtain images 63 of the panelists who provided feedback 62 (hereinafter simply referred to as images), associating them with the feedback 62 and the clothing items that were the subject of the feedback 62. Images 63 are either half-body or full-body images of the panelist.

[0049] The clothing information suggestion device 30 associates the clothing item that was the subject of the feedback 62, the content of the feedback 62, and the characteristics of the paneler extracted from the image 63 (for example, the shape of the facial contour, the ratio of the size of the torso to the face, the length of the legs, etc.) and registers them in the clothing database 325. At this time, various attributes of the paneler (gender, age, etc.) may be obtained through an interview and registered in the clothing database 325 in association with the aforementioned information.

[0050] The clothing database 325 is generated by separating items based on keywords that characterize them. For example, the clothing database 325 includes multiple databases, such as a clothing database with "dandy" characteristics, a clothing database with "soft" characteristics, a clothing database with "formal" characteristics, and a clothing database with "casual" characteristics.

[0051] Each database constituting the clothing database 325 stores the product ID of a specific clothing item, images of that clothing item from various angles, images showing examples of coordination with other clothing items, and panel comments related to the clothing item and coordination examples. The product names and prices of the clothing items and coordination examples stored in the clothing database 325 are obtained by referring to the product master 322 associated with the product ID.

[0052] In this way, by storing product information related to clothing items separately in the clothing database 325 and the product master 322, it is possible to construct a database usable by the clothing information suggestion system 10 without making major modifications to the product master 322 that was previously used.

[0053] (Functional configuration of the fashion information suggestion system) The functional configuration of the clothing information suggestion system 10 will be explained using Figure 5. Figure 5 is a functional block diagram showing an example of the functional configuration of the clothing information suggestion system.

[0054] First, let's explain the functional configuration of the customer terminal 20.

[0055] The control unit 21 of the customer terminal 20 deploys the control program 221 to the RAM 213 and operates it, thereby realizing the speech acquisition unit 41, customer image capture unit 42, selection result display unit 43, and virtual image display unit 44 shown in Figure 5 as functional units. Note that all or some of these functional units may be realized by dedicated hardware.

[0056] The speech acquisition unit 41 acquires customer speech data 222 related to the clothing items it desires using the microphone 28. The speech acquisition unit 41 also associates the speech data 222 with identification information that identifies the customer terminal 20 and outputs it to the clothing information suggestion device 30.

[0057] The customer imaging unit 42 uses the camera 29 to capture an image of the customer, including their face and body. The customer imaging unit 42 also associates the image data 223 with identification information that identifies the customer terminal 20 and outputs it to the clothing information suggestion device 30.

[0058] The selection result display unit 43 obtains the selection results for clothing items from the request inquiry unit 53 of the clothing information suggestion device 30 and displays the selection result display screen 81 on the monitor 26.

[0059] The virtual image display unit 44 obtains the customer's request to try on the selected clothing item. The virtual image display unit 44 also displays a virtual image generated by the clothing information suggestion device 30 in response to the customer's request to try on the clothing item on the monitor 26. The virtual image is an image of the customer trying on the clothing item selected by the clothing information suggestion device 30, that is, an image in which the clothing item is superimposed on the customer's image.

[0060] Next, the functional configuration of the clothing information suggestion device 30 will be explained.

[0061] The control unit 31 of the clothing information suggestion device 30 deploys the control program 321 to the RAM 313 and operates it, thereby realizing the following functional units as shown in Figure 5: the text conversion unit 51, the image acquisition unit 52, the request inquiry unit 53, the clothing item selection unit 54, the virtual image generation unit 55, and the image output unit 56. Note that all or part of these functional units may be implemented using dedicated hardware.

[0062] The text conversion unit 51 acquires speech data 222 and identification information that identifies the customer terminal 20 from the customer terminal 20. The text conversion unit 51 also converts the speech data 222 acquired from the customer terminal 20 into text data 323. Specifically, the text conversion unit 51 performs speech recognition on the speech data 222. Then, the text conversion unit 51 performs dictation, which is the process of transcribing text information from the speech recognition results. The method of speech recognition and dictation is not limited, but a product master 322 or a large-scale language model 324 may be used as a dictionary to store the vocabulary used when performing these operations.

[0063] The image acquisition unit 52 acquires image data 223 and identification information that identifies the customer terminal 20 from the customer terminal 20. The image data 223 and the speech data 222 are associated with each other.

[0064] The request inquiry unit 53, based on the text data 323 generated by the text conversion unit 51, instructs the clothing item selection unit 54 to select a clothing item that matches the customer's request from the clothing database 325. The request inquiry unit 53 also retrieves the clothing item selection result from the clothing item selection unit 54 and outputs it to the selection result display unit 43 on the customer terminal 20.

[0065] The clothing item selection unit 54, through a learning process that associates multiple keywords representing the characteristics of clothing items with the panelists' evaluations of the clothing items and the clothing items themselves, selects at least one clothing database 325 from among multiple clothing databases 325 that have been pre-generated for each characteristic of clothing items. This selection unit selects a specific clothing item from the selected clothing database 325 that is presumed to match the customer's request, based on features extracted from text data 323 (text information), such as the type of clothing item the customer desires and its characteristics. The clothing item selection unit 54 is an example of a selection unit in this disclosure.

[0066] Furthermore, when the clothing item selection unit 54 selects clothing items that match the customer's requests, it may use features extracted from the customer's image data 223, which is captured by the customer photography unit 42, in addition to features extracted from the text data 323. Features extracted from the customer's image data 223 include, for example, the customer's body type, facial expression, age group, and gender. The customer's age group and gender may also be declared by the customer in advance. The clothing item selection unit 54 also outputs information related to the selected clothing items, such as images of the clothing items, to the request inquiry unit 53.

[0067] The virtual image generation unit 55 generates a virtual image by superimposing clothing items belonging to the clothing database 325 selected by the clothing item selection unit 54 onto the customer's image. The virtual image generation unit 55 uses, for example, a known morphing technique to deform and superimpose the images of the selected clothing items to match the customer's half-body or full-body image.

[0068] The image output unit 56 outputs the virtual image generated by the virtual image generation unit 55 to the customer terminal 20, which is the source of the image data 223.

[0069] (Speech instruction screen) Figure 6 illustrates the speech instruction screen 71 displayed on the customer terminal 20, which directs the customer to speak. Figure 6 shows an example of a speech instruction screen in which the clothing information suggestion system prompts the customer to speak about the image of the clothing they want.

[0070] The speech instruction screen 71 is displayed on the monitor 26 of the customer terminal 20. The speech instruction screen 71 includes a message 72, a speech icon 73, and a speech button 74.

[0071] Message 72 is information that instructs the customer to verbally describe the image of the clothing item they want.

[0072] The speech icon 73 is an icon that suggests to the customer that they should speak.

[0073] The speech button 74 is an operator that instructs the user to start speaking. When the customer presses the speech button 74, the microphone 28 of the customer terminal 20 records the customer's speech and generates speech data 222. The customer terminal 20 then stores the generated speech data 222. The customer terminal 20 then determines that the speech has ended and stops recording when the sound pressure level of the speech falls below a predetermined level for a predetermined period of time.

[0074] Furthermore, in order to ensure reliable detection of the end of speech, the customer terminal 20 may record the customer's speech while the speech button 74 is pressed. The customer then releases the speech button 74 when they have finished speaking. In this case, the customer must continue to press the speech button 74 while speaking, but the end of speech can be reliably detected.

[0075] (Shooting instruction screen) Using Figure 7, we will explain the shooting instruction screen 76 that is displayed on the customer terminal 20 and instructs the customer to take a picture of themselves. Figure 7 is a diagram showing an example of a shooting instruction screen that the clothing information suggestion system prompts the customer to take a picture of themselves.

[0076] The shooting instruction screen 76 is displayed on the monitor 26 of the customer terminal 20. The shooting instruction screen 76 includes a message 77, a message 78, and a shooting button 79.

[0077] Message 77 is information that informs the customer that a full-body or partial-body photograph of the customer will be taken.

[0078] Message 78 provides information to the customer regarding how to start shooting and when to start shooting. Specifically, message 78 includes instructions that shooting will begin 10 seconds after pressing the shutter button 79.

[0079] The capture button 79 is an operator that instructs the start of image capture. When the customer presses the capture button 79, the camera 29 of the customer terminal 20 starts capturing and displays the captured image (video) on the monitor 26 in real time (so-called live view). At this time, it is desirable for the customer terminal 20 to display a timer on the monitor 26 that counts down from 10 seconds. The customer checks the image (video) displayed on the monitor 26 and adjusts their position so that half or their whole body is in the image. Then, the customer terminal 20 displays the captured image on the monitor 26 when the timer reaches 0 seconds. The customer checks the image and, if there are no problems, presses, for example, the unillustrated OK button displayed on the monitor 26. At this time, the customer terminal 20 acquires the image displayed on the monitor 26 as image data 223. On the other hand, if the customer does not like the image displayed on the monitor 26, they can repeat the image capture by pressing the capture button 79 again.

[0080] The process for acquiring the image data 223 is not limited to the one described above. For example, the customer terminal 20 may continuously acquire images for a predetermined period of time after the capture button 79 is pressed. After the predetermined time has elapsed, the customer terminal 20 may display the acquired images on the monitor 26, and the customer may select a preferred image from among them.

[0081] (Clothing item candidate display screen) Using Figure 8, we will explain an example of how the clothing information suggestion device 30 displays clothing items selected to meet the customer's needs. Figure 8 is a diagram showing an example of a screen displaying candidate clothing items that the customer desires, estimated from the customer's speech.

[0082] The selection result display screen 81 is displayed on the monitor 26 of the customer terminal 20. The selection result display screen 81 in Figure 8 is an example of a clothing item selected based on the customer's utterance data 222, "I'd like a dapper, soft jacket." The selection result display screen 81 includes the selected item 82, the selected item 83, a back button 84, and a try-on button 85.

[0083] Selected item 82 is an example of a clothing item selected by the clothing item selection unit 54 from the clothing database 325 based on keywords that describe the characteristics of the clothing, such as "dandy" and "jacket."

[0084] Selected item 83 is an example of a clothing item selected by the clothing item selection unit 54 from the clothing database 325 based on keywords that describe the characteristics of the clothing, such as "soft" and "outerwear."

[0085] Here, two types of selection items 82 and 83 are displayed, but the number of clothing items displayed depends on the number of keywords describing the characteristics of the clothing items included in the customer's speech data 222. In other words, Figure 8 is an example where two types of keywords related to the characteristics of the clothing items (dandy, soft) are used, and clothing items corresponding to each keyword are selected.

[0086] The back button 84 is an operator used to instruct the customer to re-select clothing items if they do not like the clothing items displayed on the selection results screen 81. When the back button 84 is pressed, the customer terminal 20 displays the speech instruction screen 71 (see Figure 6) on the monitor 26, prompting the customer to speak again about the clothing items they want. Subsequently, the selection of clothing items as described above is performed again.

[0087] The try-on button 85 is an operator that instructs the user to try on the clothing items displayed on the selection result display screen 81. When the try-on button 85 is pressed, the customer terminal 20 instructs the clothing information suggestion device 30 to generate a virtual image in which the customer is trying on the clothing items displayed on the selection result display screen 81 using the image data 223 of the customer.

[0088] (Virtual image display screen) Using Figure 9, we will explain an example of displaying a virtual image generated by the clothing information suggestion device 30, showing the customer wearing the clothing items they have selected. Figure 9 is a diagram showing an example of a virtual image in which the clothing items desired by the customer are superimposed on the customer's own image.

[0089] The virtual image display unit 44 of the customer terminal 20 displays a virtual image display screen 86 on the monitor 26. The virtual image display screen 86 displays a virtual image generated by the clothing information suggestion device 30, in which clothing items selected by the clothing item selection unit 54 of the clothing information suggestion device 30 are superimposed on the customer's image.

[0090] In this embodiment, the virtual image display screen 86 displays two virtual images side by side: a virtual image 87 in which clothing items selected from a clothing database 325 having "dandy" characteristics are superimposed on the customer's image, and a virtual image 88 in which clothing items selected from a clothing database 325 having "soft" characteristics are superimposed on the customer's image. The description "dandy" is explicitly displayed for virtual image 87, and the description "soft" is explicitly displayed for virtual image 88. This allows the customer to confirm that clothing items that meet their preferences have been selected. Virtual images 87 and 88 are examples of the first virtual images in this disclosure.

[0091] Furthermore, the clothing items superimposed on the customer's image, specifically the clothing items selected by the clothing item selection unit 54 of the clothing information suggestion device 30, will clearly indicate the display location of the product in the store. This provides customers with clues when searching for clothing items in the store.

[0092] (Process flow performed by the fashion information suggestion system) Figure 10 illustrates the processing flow of the clothing information suggestion system 10 in this embodiment. Figure 10 is a flowchart showing an example of the processing flow of the clothing information suggestion system.

[0093] First, let's explain the processing flow performed by the customer terminal 20.

[0094] The speech acquisition unit 41 acquires the customer's utterances related to the clothing items it desires as speech data 222 (step S11).

[0095] The speech acquisition unit 41 associates the speech data 222 with identification information that identifies the customer terminal 20 and outputs it to the clothing information suggestion device 30 (step S12).

[0096] The customer photography unit 42 takes an image of the customer (image data 223) (step S13).

[0097] The customer photography unit 42 outputs the captured customer image (image data 223) to the clothing information suggestion device 30 (step S14).

[0098] The selection result display unit 43 acquires information related to the selected clothing item from the clothing information suggestion device 30 (step S15).

[0099] The selection result display unit 43 displays information related to the selected clothing item on the monitor 26 (step S16).

[0100] The virtual image display unit 44 determines whether it has received an instruction from the customer to try on the selected clothing item (step S17). If it is determined that an instruction to try on the clothing item has been received (step S17: Yes), the process proceeds to step S18. On the other hand, if it is not determined that an instruction to try on the clothing item has been received (step S17: No), the customer terminal 20 terminates the process.

[0101] In step S17, if it is determined that an instruction to try on clothing items has been received, the virtual image display unit 44 instructs the virtual image generation unit 55 of the clothing information suggestion device 30 to generate a virtual image (step S18).

[0102] The virtual image display unit 44 acquires the generated virtual image from the image output unit 56 of the clothing information suggestion device 30 (step S19).

[0103] The virtual image display unit 44 displays a virtual image on the monitor 26 (step S20). After that, the customer terminal 20 terminates the process.

[0104] Next, we will explain the processing flow performed by the clothing information suggestion device 30.

[0105] The text conversion unit 51 acquires speech data 222 from the customer terminal 20 (step S21).

[0106] The image acquisition unit 52 acquires an image of the customer (image data 223) from the customer terminal 20 (step S22).

[0107] The text conversion unit 51 converts the speech data 222 into text data 323 (step S23).

[0108] The clothing item selection unit 54, upon receiving instructions from the request inquiry unit 53, analyzes the text data 323 to estimate the customer's requests regarding clothing items (step S24).

[0109] The clothing item selection unit 54 selects a clothing database 325 containing clothing items that match the estimated customer's preferences (step S25).

[0110] The clothing item selection unit 54 selects clothing items that meet the customer's requirements from the selected clothing database 325. Then, the clothing item selection unit 54 outputs information related to the selected clothing items to the customer terminal 20 (step S26).

[0111] The virtual image display unit 44 determines whether there is a command from the customer terminal 20 to generate a virtual image (step S27). If it is determined that there is a command to generate a virtual image (step S27: Yes), the process proceeds to step S28. On the other hand, if it is not determined that there is a command to generate a virtual image (step S27: No), the clothing information suggestion device 30 terminates processing.

[0112] In step S27, if it is determined that there is an instruction to generate a virtual image, the virtual image generation unit 55 generates a virtual image in which the clothing items instructed by the customer are superimposed on the customer's image (step S28).

[0113] The image output unit 56 outputs the virtual image generated by the virtual image generation unit 55 to the customer terminal 20 (step S29). After that, the clothing information suggestion device 30 terminates processing.

[0114] (Modified version of the embodiment) A modified example of the embodiment will be explained using Figure 11. Figure 11 shows an example of the operation of a modified version of the clothing information suggestion system.

[0115] If a customer requests, "I'd like a dapper, soft-looking jacket," the clothing information suggestion system 10 may adjust the "dapper" and "soft-looking" aspects according to the customer's instructions to suit their preferences.

[0116] Specifically, the clothing information suggestion system 10 displays a virtual image display screen 91 on the customer terminal 20, which includes images of the customer trying on clothing items that meet the customer's requirements.

[0117] The virtual image display screen 91 includes a slider bar 92 and a virtual image 94.

[0118] The slider bar 92 is an operator that adjusts the weight of "dandy" and "soft" as requested by the customer. The customer adjusts the weight of "dandy" and "soft" by sliding the slider thumb 93 of the slider bar 92 left or right. For example, in Figure 11, when the slider thumb 93 is slid all the way to the left, the weights of "dandy" and "soft" are set to 100% for "dandy" and 0% for "soft". On the other hand, when the slider thumb 93 is slid all the way to the right, the weights of "dandy" and "soft" are set to 0% for "dandy" and 100% for "soft".

[0119] The virtual image 94 is a virtual image generated by the virtual image generation unit 55 (see Figure 5), and is a virtual image in which a clothing item having characteristics that fuse "dandy" and "soft" qualities with weights corresponding to the adjustment position of the slider thumb 93 is superimposed on the customer's image. The virtual image 94 is an example of the second virtual image in this disclosure. The clothing item selection unit 54 (see Figure 5) selects a clothing item having characteristics that fuse with weights corresponding to the adjustment position of the slider thumb 93 from a plurality of clothing databases 325.

[0120] To propose clothing items that have features formed by fusing multiple features with specified weights, for example, each clothing item stored in the clothing database 325 is assigned information indicating what percentage of features such as "dandy" and "soft" that represent the characteristics of the clothing item are reflected. The clothing item selection unit 54 then selects clothing items from the clothing database 325 that have features close to the feature distribution indicated by the adjustment position of the slider sum 93.

[0121] (Effects of the embodiment) As described above, the clothing information suggestion device 30 (information processing device) of the embodiment includes: a text conversion unit 51 that acquires the customer's speech data 222 (speech information) relating to the clothing items it desires and converts the speech data 222 into text data 323 (text information); a clothing item selection unit 54 (selection unit) that, through a learning process that associates multiple keywords representing the characteristics of clothing items with panel evaluations of clothing items and the clothing items themselves, selects at least one clothing database 325 from among multiple clothing databases 325 that have been pre-generated for each characteristic of clothing items, based on the characteristics extracted from the text data 323, to store clothing items that match the customer's requests; and a virtual image generation unit 55 that generates virtual images 87, 88 (first virtual images) by superimposing the clothing items belonging to the clothing database 325 selected by the clothing item selection unit 54 onto the customer's image. Therefore, it is possible to suggest appropriate clothing items that meet the customer's requests without being affected by the sales performance of the products.

[0122] Furthermore, in the clothing information suggestion device 30 (information processing device) of this embodiment, the clothing database 325 is generated by associating examples of clothing items, examples of coordinating them with other clothing items, Panera's comments 62 related to the clothing items and coordination examples, and images of Panera. Therefore, regardless of the sales performance of the products, the characteristics of clothing items can be collected with higher accuracy.

[0123] Furthermore, in the clothing information suggestion device 30 (information processing device) of the embodiment, the clothing item selection unit 54 (selection unit) selects the relevant clothing databases 325 when the features extracted from the text data 323 (text information) are related to the multiple clothing databases 325, and the virtual image generation unit 55 generates multiple virtual images 87, 88 (first virtual images) by superimposing the clothing items belonging to each of the multiple clothing databases 325 onto the customer's image. Therefore, even if the customer requests multiple features for clothing items, it is possible to suggest clothing items that correspond to all of the features.

[0124] Furthermore, in the clothing information suggestion device 30 (information processing device) of the embodiment, the virtual image generation unit 55 (second virtual image generation unit) further generates a virtual image 94 (second virtual image) by selecting a clothing item from the multiple clothing databases 325 that fuses clothing items belonging to each of the multiple clothing databases 325 with weights according to the customer's preferences, and superimposing it onto the customer's image. Therefore, when the customer has a wide range of requests for clothing items, it is possible to suggest clothing items that fuse multiple requests in any proportion.

[0125] Although embodiments of the present invention have been described above, these embodiments are illustrative and are not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0126] 10. Apparel Information Proposal System 12 Networks 20 Customer terminals 26 monitors 27 Touch panel 28 Mike 29 Cameras 30. Apparel Information Proposal Device (Information Processing Device) 41. Speech acquisition unit 42 Customer Photography Department 43 Selection Result Display Section 44 Virtual Image Display Unit 51 Text Conversion Unit 52 Image acquisition unit 53 Requests and Inquiries Department 54 Clothing Item Selection Section (Selection Section) 55 Virtual Image Generation Unit 56 Image output section 62 Comments 71. Voice command screen 81 Selection Results Display Screen 86,91 Virtual image display screen 76 Shooting Instruction Screen 87,88 Virtual image (First virtual image) 92 Slider Bar 94. Virtual Image (Second Virtual Image) 222 Utterance data (speech information) 223 Image Data 322 Product Master 323 Text data (text information) 324 Large-Scale Language Models 325 Clothing Database [Prior art documents] [Patent Documents]

[0127] [Patent Document 1] Japanese Patent Publication No. 2024-8995

Claims

1. A text conversion unit that acquires customer speech information related to the clothing items they desire and converts the said speech information into text information, A selection unit selects at least one clothing database from among multiple pre-generated clothing databases for each clothing item, based on the characteristics extracted from the text information, which stores clothing items that match the customer's requests, through a learning process that associates multiple keywords representing the characteristics of clothing items with the panel's evaluation of the clothing items and the clothing items themselves. A virtual image generation unit generates a first virtual image by superimposing clothing items belonging to the clothing database selected by the selection unit onto the customer's image. An information processing device equipped with the following features.

2. The aforementioned clothing database is generated by associating examples of clothing items, examples of coordinating them with other clothing items, comments from panelists related to the clothing items and the coordination examples, and images of the panelists. The information processing apparatus according to claim 1.

3. The selection unit, when the features extracted from the text information are related to multiple clothing databases, selects the corresponding multiple clothing databases. The virtual image generation unit generates a plurality of first virtual images by superimposing clothing items belonging to a plurality of clothing databases onto the customer's image. The information processing apparatus according to claim 1 or claim 2.

4. The virtual image generation unit further generates a second virtual image by selecting clothing items from the multiple clothing databases, which are obtained by fusing clothing items belonging to each of the multiple clothing databases with weights according to the customer's preferences, and superimposing them onto the customer's image. The information processing apparatus according to claim 3.

5. A computer that controls an information processing device, A text conversion unit that acquires customer speech information related to the clothing items they desire and converts the said speech information into text information, A selection unit selects at least one clothing database from among multiple pre-generated clothing databases for each clothing item, based on the characteristics extracted from the text information, which stores clothing items that match the customer's requests, through a learning process that associates multiple keywords representing the characteristics of clothing items with the panel's evaluation of the clothing items and the clothing items themselves. The virtual image generation unit operates as a unit that generates a first virtual image by superimposing clothing items belonging to the clothing database selected by the selection unit onto the customer's image. program.

Images