Information processing device, information processing method, and information processing program

Abstract

This information processing device acquires a first search result image group as a search result corresponding to an input for a search with respect to a search-target image group, presents a user interface with which it is possible to input a user's evaluation of at least one search result image included in the first search result image group, and acquires a second search result image group as a re-search result for the search-target image group based on the search result image to which the user's evaluation has been applied through the user interface.

Description

Information processing device, information processing method, and information processing program

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program.

[0002] Japanese Patent Publication No. 2020-030634 discloses a technology for extracting multiple search result data that match search conditions from multiple search target data, and accepting the assignment of a positive evaluation to search result data selected from the extracted multiple search result data.

[0003] The technology described in JP 2020-030634 A searches documents, so when images are obtained as search results, there is room for improvement in terms of allowing users to efficiently rate the images in the search results.

[0004] The present disclosure has been made in consideration of the above circumstances, and aims to provide an information processing device, an information processing method, and an information processing program that allow users to efficiently assign ratings to images in search results.

[0005] A first aspect of the information processing device is an information processing device having at least one processor, which obtains a first group of search result images as search results in response to a search input for a group of search target images, presents a user interface in which a user can input a user's rating for at least one search result image included in the first group of search result images, and obtains a second group of search result images as re-search results for the group of search target images based on the search result image to which the user's rating has been assigned via the user interface.

[0006] In a second aspect of the information processing device, in the information processing device of the first aspect, the processor obtains a second group of search result images for a group of search target images based on a search input and search result images to which a user has assigned a rating via a user interface.

[0007] The third aspect of the information processing device is an information processing device of the first or second aspect, in which the user's evaluation includes at least a positive evaluation, and the second group of search result images has a higher search ranking for images from the group of search target images that have a high similarity to the search result images that have been given a positive evaluation.

[0008] The information processing device of the fourth aspect is an information processing device of the first or second aspect, in which the user's evaluation includes at least a negative evaluation, and the second group of search result images has a lower search ranking for images from the group of search target images that have a high similarity to the search result images that have been given a negative evaluation.

[0009] An information processing device of a fifth aspect is an information processing device of any one of the first to fourth aspects, wherein the user interface includes a display area for classifying at least a portion of the first set of search result images into groups.

[0010] In a sixth aspect of the information processing device, in the information processing device of the fifth aspect, the processor performs a conversion process to convert search result images included in a first group of search result images into vectors, and selects from among a plurality of different conversion processes a conversion process that results in a relatively high degree of similarity between search result images classified into the same group.

[0011] In a seventh aspect of the information processing device, in the sixth aspect of the information processing device, the processor converts each of the search result images included in the first group of search result images into a vector using a selected conversion process, and the similarity between the search result images is based on the distance using the vector obtained by the conversion.

[0012] An information processing device of an eighth aspect is the information processing device of any one of the fifth to seventh aspects, wherein the user interface includes an interface that allows a user to input an evaluation for each group.

[0013] In the information processing device of the ninth aspect, when a positive evaluation is given to multiple search result images in the information processing device of the third aspect, the similarity to the search result images is a statistical value of the similarity to each of the multiple search result images that have been given a positive evaluation.

[0014] In the information processing device of the tenth aspect, in the information processing device of the fourth aspect, when negative ratings are given to multiple search result images, the similarity to the search result images is a statistical value of the similarity to each of the multiple search result images that have been given negative ratings.

[0015] An information processing device of an eleventh aspect is an information processing device of the third or fourth aspect, in which the user interface includes an interface that allows a user to input an evaluation of the search result images included in the first group of search result images for each group, and when the first group of search result images is classified into multiple different groups, the processor weights the similarity according to the search priority for each group.

[0016] An information processing device of a twelfth aspect is an information processing device of any one of the fifth to seventh aspects, in which the user interface includes an interface capable of classifying the first group of search result images into groups having a hierarchical structure, and the closer the hierarchy of the classified groups, the higher the similarity between the search result images.

[0017] An information processing device of a thirteenth aspect is an information processing device of any one of the first to twelfth aspects, wherein the user interface includes a display area in which search result images to which a user rating is assigned are displayed, and a display area in which search result images that are not used in the search to obtain the second group of search result images are displayed.

[0018] An information processing device of a 14th aspect is an information processing device of any one of the first to 13th aspects, wherein the user interface includes a user interface that allows the user to input a user evaluation by operating the user on at least one search result image included in the first group of search result images.

[0019] In a fifteenth aspect of the information processing method, a processor provided in an information processing device executes the following process: acquiring a first group of search result images as search results in response to a search input for a group of search target images; presenting a user interface in which a user can input a user's rating for at least one search result image included in the first group of search result images; and acquiring a second group of search result images as re-search results for the group of search target images based on the search result image to which the user's rating has been assigned via the user interface.

[0020] The information processing program of the 16th aspect causes a processor provided in an information processing device to execute a process of acquiring a first group of search result images as search results in response to a search input for a group of search target images, presenting a user interface in which a user can input a user's rating for at least one search result image included in the first group of search result images, and acquiring a second group of search result images as re-search results for the group of search target images based on the search result image to which the user's rating has been assigned via the user interface.

[0021] According to the present disclosure, users can efficiently rate images in search results.

[0022] 1 is a block diagram showing an example of the configuration of an information processing system; FIG. 2 is a block diagram showing an example of the hardware configuration of a user terminal; FIG. 3 is a block diagram showing an example of the hardware configuration of a server device; FIG. 4 is a diagram showing an example of a search result display screen; FIG. 5 is a block diagram showing an example of the functional configuration of a user terminal; FIG. 6 is a diagram showing an example of a search result display screen; FIG. 7 is a block diagram showing an example of the functional configuration of a server device; FIG. 8 is a diagram for explaining search processing; FIG. 9 is a sequence diagram showing an example of search processing; FIG. 10 is a diagram showing an example of a search result display screen according to a modified example; FIG. 11 is a diagram showing an example of a search result display screen according to a modified example.

[0023] Hereinafter, examples of embodiments for carrying out the technology of the present disclosure will be described in detail with reference to the drawings.

[0024] First, the configuration of an information processing system 10 according to this embodiment will be described with reference to Fig. 1. As shown in Fig. 1, the information processing system 10 includes a user terminal 12 used by a user and a server device 14. Examples of the user terminal 12 include a tablet computer and a personal computer. Examples of the server device 14 include a server computer and a cloud server. The user terminal 12 is an example of an information processing device according to the disclosed technology.

[0025] Next, the hardware configuration of the user terminal 12 according to this embodiment will be described with reference to Fig. 2. As shown in Fig. 2, the user terminal 12 includes a CPU (Central Processing Unit) 20, a memory 21 as a temporary storage area, and a non-volatile storage unit 22. The user terminal 12 also includes a display 23 such as a liquid crystal display, an input device 24 such as a keyboard and a mouse, and a network I / F (Interface) 25 connected to a network. The CPU 20, memory 21, storage unit 22, display 23, input device 24, and network I / F 25 are connected to a bus 27. The CPU 20 is an example of a processor according to the disclosed technology.

[0026] The storage unit 22 is realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or the like. The storage unit 22 serving as a storage medium stores an information processing program 30. The CPU 20 reads the information processing program 30 from the storage unit 22, loads it into the memory 21, and executes the loaded information processing program 30.

[0027] Next, the hardware configuration of the server device 14 according to this embodiment will be described with reference to Fig. 3. As shown in Fig. 3, the server device 14 includes a CPU 40, a memory 41 as a temporary storage area, and a non-volatile storage unit 42. The server device 14 also includes a display 43 such as a liquid crystal display, an input device 44 such as a keyboard and a mouse, and a network I / F 45 connected to a network. The CPU 40, memory 41, storage unit 42, display 43, input device 44, and network I / F 45 are connected to a bus 47.

[0028] The storage unit 42 is realized by an HDD, an SSD, a flash memory, or the like. The storage unit 42 serving as a storage medium stores a search program 50. The CPU 40 reads the search program 50 from the storage unit 42, loads it into the memory 41, and executes the loaded search program 50.

[0029] An overview of the image search performed by the information processing system 10 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of a search result display screen displayed on the display 23 of the user terminal 12. As shown in Fig. 4, the search result display screen according to this embodiment includes a display area A1 in which a group of search result images is displayed, a display area A2 for classifying at least a portion of the group of search result images into groups, a button B1 for assigning a positive rating, and a button B2 for assigning a negative rating.

[0030] At least one display area A2 is displayed on the search result display screen, and multiple search result images displayed in the same display area A2 are treated as images belonging to the same group. Furthermore, on the search result display screen, a pair of buttons B1 and B2 are displayed in association with each display area A2. Note that a user may add a new display area A2 or delete an existing display area A2 by operating the input device 24 of the user terminal 12. Furthermore, in response to the addition of a new display area A2 by a user operation, buttons B1 and B2 corresponding to the new display area A2 may be added. Furthermore, in response to the deletion of a display area A2 by a user operation, buttons B1 and B2 corresponding to the deleted display area A2 may be deleted.

[0031] The user inputs a search input corresponding to a desired image via the input device 24. For example, if the user wants to obtain a group of images of "sitting dogs," the user inputs images of "sitting dogs" as the search input. The search input may also be the text "sitting dogs." A group of search result images corresponding to the search input is displayed in the display area A1. In addition to "sitting dogs," this group of search result images may also include images that are similar to but not desired by the user, such as "sitting cats" and "dogs not sitting."

[0032] The user assigns a positive rating to a desired image among the search result images displayed in display area A1 by selecting button B1 and moving the image to display area A2, where a positive rating is assigned. The user assigns a negative rating to an undesired image among the search result images displayed in display area A1 by selecting button B2 and moving the image to display area A2, where a negative rating is assigned. The group of search result images displayed in display area A1 is updated in accordance with the user's rating. A positive rating here refers to a positive rating assigned by the user to a search result image that is similar to the desired image. A negative rating refers to a negative rating assigned by the user to a search result image that is not similar to the desired image.

[0033] Next, the functional configuration of the user terminal 12 will be described with reference to Fig. 5. As shown in Fig. 5, the user terminal 12 includes a reception unit 60, a first transmission unit 62, a first acquisition unit 64, a display control unit 66, a second transmission unit 68, and a second acquisition unit 70. When the CPU 20 executes the information processing program 30, the user terminal 12 functions as the reception unit 60, the first transmission unit 62, the first acquisition unit 64, the display control unit 66, the second transmission unit 68, and the second acquisition unit 70.

[0034] The accepting unit 60 accepts an image (hereinafter referred to as an "input image") as an example of a search input input input by the user via the input device 24. The accepting unit 60 also accepts an operation by the user to designate button B1 via the input device 24. The accepting unit 60 also accepts an operation by the user to designate button B2 via the input device 24. The accepting unit 60 also accepts an operation by the user to classify at least some of the search result images displayed in the display area A1 into groups. An example of this operation is an operation to move the search result images displayed in the display area A1 to the display area A2 by dragging and dropping them.

[0035] The first transmitting unit 62 transmits the input image accepted by the first accepting unit 60 to the server device 14 via the network I / F 25 .

[0036] The first acquisition unit 64 acquires, via the network I / F 25 , a first group of search result images as search results obtained by the server device 14 in response to the input image transmitted by the first transmission unit 62 .

[0037] The display control unit 66 presents a user interface that allows the user to input an evaluation of at least one search result image included in the first group of search result images acquired by the first acquisition unit 64. Specifically, the display control unit 66 presents the evaluation to the user by controlling the display of a search result display screen (see FIG. 4 ) as an example of a user interface on the display 23. At this time, the display control unit 66 controls the display of the first group of search result images acquired by the first acquisition unit 64 in the display area A1. As will be described in detail later, the first group of search result images is assigned a search ranking, and the display control unit 66 controls the display of the first group of search result images in the display area A1 in accordance with the search ranking. In this embodiment, the display control unit 66 controls the display of the first group of search result images in the display area A1 in ascending order of search ranking from the upper left to the lower right of the display area A1.

[0038] 6, on the search result display screen, the user classifies the search result images into groups by moving the search result images from display area A1 to display area A2. The user moves the search result images to be classified into the same group into the same display area A2.

[0039] As described above, the search result display screen displays a pair of buttons B1 and B2 associated with each display area A2. By selecting button B1 or button B2, the user assigns a rating to the search result image displayed in the corresponding display area A2. That is, the search result display screen includes an interface that allows the user to input a user rating for each group. In the example of FIG. 6 , the button B1 or button B2 designated by the user is displayed in a filled-in state. That is, in the example of FIG. 6 , the user assigns a positive rating to the search result image that has been moved to the top display area A2.

[0040] In this way, the search result display screen includes an interface that allows the user to input a user's evaluation by performing an operation on at least one search result image included in the first group of search result images displayed in display area A1. An example of the user's operation in this case is an operation of moving the search result image displayed in display area A1 to display area A2 and specifying button B1 or button B2 corresponding to that display area A2.

[0041] For example, if a user searches for "sitting dog" and an image of "sitting dog" and an image of "animal other than sitting dog" are displayed in display area A1, the user assigns a positive evaluation to the image of "sitting dog" by moving the image of "sitting dog" to display area A2 designated by button B1. In this case, the user assigns a negative evaluation to the image of "animal other than sitting dog" by moving the image of "animal other than sitting dog" to display area A2 designated by button B2.

[0042] The user may designate button B1 or button B2 and then move the search result image to the display area A2 corresponding to the designated button B1 or button B2. The user may designate button B1 or button B2 after moving the search result image to the display area A2. The user may designate button B1 or button B2 once and then switch the designation of button B1 or button B2.

[0043] Furthermore, when a second group of search result images is acquired by a second acquisition unit 70 (described later), the display control unit 66 controls the display of the second group of search result images in the display area A1, thereby updating the display in the display area A1. The second group of search result images is also assigned a search ranking, and the display control unit 66 controls the display of the second group of search result images in the display area A1 according to the search ranking, similar to the first group of search result images.

[0044] When a user rating is provided via the search result display screen, the second transmission unit 68 transmits the search result images with the user rating to the server device 14 via the network I / F 25. At this time, the second transmission unit 68 assigns identification information, such as an ID for identifying each display area A2, to the search result images as tag information. This identification information also serves to identify the group into which the search result images are classified. Therefore, by referring to the tag information of the search result images, the server device 14 can determine whether multiple search result images belong to the same group or different groups.

[0045] In the present embodiment, every time a user rating is assigned to a search result image, the second transmission unit 68 transmits the search result image with the user rating to the server device 14 via the network I / F 25. Specifically, every time the user moves the search result image displayed in the display area A1 to the display area A2 where the button B1 or B2 is specified, the search result image with the user rating and the above-mentioned identification information is transmitted to the server device 14. Then, the display in the display area A1 is updated according to the re-search result by the server device 14.

[0046] The second acquisition unit 70 acquires, via the network I / F 25, a second group of search result images as re-search results searched by the server device 14 in accordance with the search images to which user ratings have been assigned and which have been sent by the second transmission unit 68.

[0047] Next, the functional configuration of the server device 14 will be described with reference to Fig. 7. As shown in Fig. 7, the server device 14 includes an acquisition unit 80, a search unit 82, and a transmission unit 84. The CPU 40 executes the search program 50 to function as the acquisition unit 80, the search unit 82, and the transmission unit 84.

[0048] The acquisition unit 80 acquires the input image transmitted from the user terminal 12 via the network I / F 45. The acquisition unit 80 also acquires the search result image, to which the user's evaluation and identification information have been added and transmitted from the server device 14, via the network I / F 45.

[0049] The search unit 82 executes a first search process according to an input image for a group of search target images. The group of search target images may be, for example, images stored in an image database or images published on the Internet. A specific example of the first search process will be described below.

[0050] As shown in FIG. 8 , the search unit 82 inputs an input image into a trained model called an encoder. The trained model converts the input image into a multi-dimensional vector and outputs the resulting vector. In this way, the search unit 82 converts the input image into a vector. The trained model is a model obtained in advance by machine learning using a training dataset, and is configured to include, for example, a neural network. Similarly, the search unit 82 converts each search target image included in the search image target image group into a vector by inputting it into the trained model.

[0051] Next, the search unit 82 derives the similarity between the input image and each search target image based on the distance between the vector obtained by transforming the input image and each vector obtained by transforming each search target image. In the example of FIG. 8 , this distance is indicated by a dashed double-headed arrow. Examples of methods for deriving this similarity include methods using p-norm (p = 1, 2, ∞, etc.) or cosine similarity. The search unit 82 then assigns a higher search ranking to each search target image the higher the similarity between the input image and the search target image. The group of search target images assigned search rankings as search results of the first search process is the first group of search result images described above. As such, in this embodiment, the similarity between images is used as a search score for determining the search ranking.

[0052] The search unit 82 then executes a second search process on a group of search target images based on the input image and the search result images to which the user has assigned a rating via the search result display screen. This group of search target images is, for example, the same as the group of search target images in the first search process. A specific example of the second search process will be described below.

[0053] Similar to the first search process, the search unit 82 converts the input image into a vector, and derives the similarity between the input image and each of the search target images according to the distance between the vector obtained by converting the input image and each of the vectors obtained by converting each of the search target images. Also, similar to the input image, the search unit 82 converts the search result images into vectors, and derives the similarity between the search result images and each of the search target images according to the distance between the vector obtained by converting the search result images and each of the vectors obtained by converting each of the search target images.

[0054] The search unit 82 assigns a higher search ranking to each search target image in the search target image group, the higher the similarity between the search target image and the search result image that has been assigned a positive evaluation. Specifically, for example, the search unit 82 assigns a higher search ranking to each search target image, the greater the total value of the similarity between the input image and the search result image.

[0055] Furthermore, the search unit 82 assigns a lower search rank to each search target image in the search target image group, the higher the similarity between the search result image that has been assigned a negative evaluation. Specifically, for example, the search unit 82 assigns a higher search rank to each search target image, the greater the value obtained by subtracting the similarity between the search result image and the input image from the similarity between the search target image and the input image.

[0056] As described above, the group of search target images to which search rankings have been assigned as re-search results of the second search process is the second group of search result images described above. In this way, in the second group of search result images, in addition to the similarity with the input image, the similarity with search result images to which the user has assigned a positive or negative rating also affects the search ranking.

[0057] Furthermore, when there are multiple search result images moved to the same display area A1, i.e., multiple search result images classified into the same group, the search unit 82 may assign a search ranking to the group of search target images as described below. That is, in this case, the search unit 82 may select, from multiple different conversion processes, a conversion process that converts the search result images included in the first group of search result images into vectors, and that results in a relatively high degree of similarity between the search result images classified into the same group. A specific example of this conversion process selection process will be described below. Note that the multiple different conversion processes are assumed to be prepared in advance. The multiple different conversion processes may be, for example, multiple trained models obtained by machine learning using multiple different learning datasets. The multiple different conversion processes may also be a combination of a trained model and two or more known algorithms that convert images into vectors, such as a self-organizing map.

[0058] The search unit 82 derives the similarity between the search images classified in the same group based on the distance between vectors obtained by transforming each of the search result images classified in the same group. In this embodiment, the search unit 82 derives the similarity between the search images classified in the same group using each of a plurality of different transformation processes. Then, the search unit 82 selects the transformation process with the highest derived similarity from among the plurality of different transformation processes. In this case, if there are three or more search result images classified in the same group, the search unit 82 may use a statistical value of the similarity between the search images for each combination of two search result images. Examples of this statistical value include a total value, an average value, or a maximum value.

[0059] In this case, the search unit 82 performs the second search process described above using the selected conversion process, thereby obtaining a second group of search result images. By selecting the conversion process that maximizes the similarity between the search result images classified by the user into the same group, it is possible to improve the likelihood that the image desired by the user will be ranked high in the search results.

[0060] The transmission unit 84 transmits a first group of search result images as the search result by the search unit 82 to the user terminal 12 via the network I / F 45. The transmission unit 84 also transmits a second group of search result images as the re-search result by the search unit 82 to the user terminal 12 via the network I / F 45.

[0061] Next, the operation of the information processing system 10 will be described with reference to Fig. 9. Fig. 9 is a sequence diagram showing an example of search processing executed by the information processing system 10.

[0062] The user inputs an input image via the input device 24. In step S10, the accepting unit 60 accepts the input image input by the user via the input device 24. In step S12, the first transmitting unit 62 transmits the input image accepted in step S10 to the server device 14 via the network I / F 25.

[0063] In step S14, the acquisition unit 80 acquires the input image transmitted from the user terminal 12 in step S12 via the network I / F 45. In step S16, the search unit 82 executes a first search process according to the input image on the group of images to be searched, as described above. The process of step S16 results in a first group of search result images. In step S18, the transmission unit 84 transmits the first group of search result images as the search results of step S16 to the user terminal 12 via the network I / F 45.

[0064] In step S20, the first acquisition unit 64 acquires the first group of search result images transmitted from the server device 14 in step S18 via the network I / F 25. In step S22, the display control unit 66 controls the display 23 to display a search result display screen on which the user can input their evaluation of at least one search result image included in the first group of search result images acquired in step S20, as described above.

[0065] On the search result display screen, the user performs an operation to assign a rating to the search result image, such as an operation to select button B1 or button B2 or an operation to move the search result image displayed in display area A1 to display area A2. In step S24, the accepting unit 60 accepts the operation to assign a rating to the search result image performed by the user. In step S26, the second transmitting unit 68 assigns the above-mentioned identification information to the search result image to which the user's rating has been assigned by the operation accepted in step S24, and transmits the same to the server device 14 via the network I / F 25.

[0066] In step S28, the acquisition unit 80 acquires, via the network I / F 45, the search result images to which the user's ratings and identification information have been assigned and which have been transmitted from the user terminal 12 in step S26. In step S30, the search unit 82, as described above, executes a second search process on a group of images to be searched, based on the input image and the search result images acquired in step S28. The process of step S30 results in a second group of search result images. In step S32, the transmission unit 84 transmits, via the network I / F 45, the second group of search result images as the results of the re-search performed in step S30 to the user terminal 12.

[0067] In step S34, the second acquisition unit 70 acquires the second group of search result images transmitted from the server device 14 in step S32 via the network I / F 25. In step S36, the display control unit 66, as described above, updates the display in the display area A1 by controlling the display of the second group of search result images acquired in step S34 in the display area A1 of the search result display screen.

[0068] After step S36, if the user performs an operation to assign a rating to at least one search result image included in the group of search result images displayed in display area A1, or an operation to correct the rating assigned to a search result image that has been moved to display area A1, processing from step S24 onwards is executed again.

[0069] As described above, according to this embodiment, the user can efficiently assign ratings to images in the search results, and the user can efficiently collect desired images.

[0070] In the above embodiment, when positive evaluations are given to multiple search result images, the search unit 82 may use statistical values ​​of the similarities between the search target image and each of the multiple search result images as the similarity (i.e., search score) for assigning a search ranking in the second search process. Similarly, when negative evaluations are given to multiple search result images, the search unit 82 may use statistical values ​​of the similarities between the search target image and each of the multiple search result images to which negative evaluations are given as the similarity for assigning a search ranking. Examples of statistical values ​​in these cases include a total value, an average value, a maximum value, etc.

[0071] In the above embodiment, when the first group of search result images is classified into a plurality of different groups, the search unit 82 may weight the similarity with the search target image in accordance with the search priority of each group in the second search process. In this case, for example, the search unit 82 may weight the similarity so that the search target image with higher similarity to the search result image belonging to a group with higher search priority is ranked higher in the search ranking.

[0072] Furthermore, in the above embodiment, the search result display screen may include an interface that allows the first group of search result images to be classified into groups having a hierarchical structure. Specifically, as shown in FIG. 10 , display area A2 may include display areas A2-1 and A2-2. For example, a user may classify dog ​​images in display area A2, and further classify Chihuahua images in display area A2-1 and Doberman images in display area A2-2. In this case, a group of dog breeds will be present in a hierarchy below the dog group. In this case, the search unit 82 may derive a higher degree of similarity between search result images in the second search process, the closer the hierarchy of the classified groups.

[0073] In the above embodiment, the user assigns a rating to a search result image by selecting button B1 or button B2 on the search result display screen. However, the disclosed technology is not limited to this. As an example, as shown in FIG. 11 , the search result display screen may be configured to assign different ratings to search target images depending on the display position of display area A2. In this case, for example, the user moves a search result image to which a positive rating is assigned to display area A2 located on the left side, and moves a search result image to which a negative rating is assigned to display area A2 located on the right side.

[0074] 11, the search result display screen may include a display area A3 for displaying search result images that will not be used in a search to obtain a second group of search result images, in addition to a display area A2 for displaying search result images to which a user's rating is assigned. After grouping search result images for a search, the user moves the search result images that the user wants to keep as a group but will not use in the next search to the display area A3. In the above embodiment, the display area A2 in which neither the button B1 nor the button B2 is designated may be the display area A3.

[0075] In the above embodiment, the display control unit 66 may control the display area A1 to display only search result images whose search rankings are equal to or higher than a threshold value among the search result images. In this case, the selection of search result images whose search rankings are equal to or higher than the threshold value may be performed by the display control unit 66 or the server device 14.

[0076] Furthermore, in the above embodiment, the evaluation given by the user to the search result image is not limited to two levels of positive evaluation or negative evaluation, and may be expressed, for example, as a numerical value.

[0077] In the above embodiment, the user terminal 12 may include at least some of the functional units included in the server device 14, or the server device 14 may include at least some of the functional units included in the user terminal 12. Furthermore, each functional unit included in the user terminal 12 and the server device 14 may be included in a single computer.

[0078] Furthermore, in the above embodiment, the following various processors can be used as the hardware structure of a processing unit that executes various processes, such as each functional unit of the user terminal 12 and each functional unit of the server device 14. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as each processing unit, as well as dedicated electrical circuits, such as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing specific processes.

[0079] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0080] Examples of configuring multiple processing units with a single processor include: first, a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server computers, and this processor functions as multiple processing units; second, a form in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs); in this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0081] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0082] In the above embodiment, the information processing program 30 is pre-stored (installed) in the storage unit 22, but the present invention is not limited to this. The information processing program 30 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The information processing program 30 may also be downloaded from an external device via a network.

[0083] In the above embodiment, the search program 50 is pre-stored (installed) in the storage unit 42, but this is not limiting. The search program 50 may be provided in a form recorded on a recording medium such as a CD-ROM, a DVD-ROM, or a USB memory. The search program 50 may also be downloaded from an external device via a network.

[0084] The disclosure of Japanese Patent Application No. 2023-189547, filed on November 6, 2023, is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. An information processing device having at least one processor, wherein the processor: obtains a first group of search result images as search results in response to a search input for a group of search target images; presents a user interface that allows a user to input a user's rating for at least one search result image included in the first group of search result images; and obtains, via the user interface, a second group of search result images as re-search results for the group of search target images based on the search result images to which the user's rating has been assigned.

2. The information processing device according to claim 1, wherein the processor obtains the second group of search result images for the group of search target images based on the search input and the search result images to which the user has assigned a rating via the user interface.

3. An information processing device as described in claim 1 or claim 2, wherein the user's evaluations include at least positive evaluations, and the second group of search result images is arranged so that images from the group of search target images that have a high similarity to the search result images that have been given positive evaluations are given a higher search ranking.

4. An information processing device as described in claim 1 or claim 2, wherein the user's evaluation includes at least a negative evaluation, and the second group of search result images is arranged so that images from the group of search target images that have a high similarity to the search result images to which a negative evaluation has been assigned are given a lower search ranking.

5. The information processing device according to claim 1 or 2, wherein the user interface includes a display area for classifying at least a portion of the first set of search result images into groups.

6. The information processing device according to claim 5, wherein the processor performs a conversion process for converting the search result images included in the first group of search result images into vectors, and selects from among a plurality of different conversion processes the conversion process that results in a relatively high degree of similarity between the search result images classified into the same group.

7. The information processing device according to claim 6, wherein each of the search result images included in the first group of search result images is converted into a vector by the selected conversion process, and the similarity is based on the distance using the vector obtained by the conversion.

8. The information processing device according to claim 5, wherein the user interface includes an interface that allows the user to input an evaluation for each of the groups.

9. An information processing device according to claim 3, wherein, when a positive evaluation is given to a plurality of the search result images, the similarity is a statistical value of the similarity between each of the plurality of search result images that have been given a positive evaluation.

10. An information processing device as described in claim 4, wherein when negative evaluations are given to multiple search result images, the similarity is a statistical value of the similarity between each of the multiple search result images to which negative evaluations have been given.

11. The information processing device of claim 3, wherein the user interface includes an interface that allows the user to input evaluations of the search result images included in the first group of search result images for each group, and the processor weights the similarity according to the search priority of each group when the first group of search result images is classified into different groups.

12. The information processing device according to claim 5, wherein the user interface includes an interface capable of classifying the first group of search result images into groups having a hierarchical structure, and the closer the hierarchy of the classified groups is, the higher the similarity between the search result images.

13. An information processing device as described in claim 1 or claim 2, wherein the user interface includes a display area in which the search result images to which the user's rating is assigned are displayed, and a display area in which the search result images not used in the search to obtain the second group of search result images are displayed.

14. An information processing device according to claim 1 or claim 2, wherein the user interface includes a user interface that allows the user to input an evaluation by an operation by the user on at least one search result image included in the first group of search result images.

15. An information processing method performed by a processor provided in an information processing device, which includes the steps of: obtaining a first group of search result images as search results in response to a search input for a group of search target images; presenting a user interface that allows a user to input a user's rating for at least one search result image included in the first group of search result images; and obtaining a second group of search result images as re-search results for the group of search target images based on the search result images to which the user's rating has been assigned via the user interface.

16. An information processing program for causing a processor provided in an information processing device to execute the following processes: acquiring a first group of search result images as search results in response to a search input for a group of search target images; presenting a user interface that allows a user to input a user's rating for at least one search result image included in the first group of search result images; and acquiring a second group of search result images as re-search results for the group of search target images based on the search result images to which the user's rating has been assigned via the user interface.

Images