Information processing device, search method, and recording medium

Abstract

The present invention provides an information processing device comprising: an reception unit that receives an input sentence describing a search target; a search means that uses a detection model for a subject, the detection model generated through machine learning of a relationship between a feature generated from an image in which the subject is appearing and a feature generated from a sentence describing the subject and detecting a predetermined subject from an image, to detect the search target described by the input sentence from an image group forming the search target; and a presentation means that presents an image in which the search target is detected.

Description

Information processing device, search method, and recording medium

[0001] The present disclosure relates to an information processing device, a search method, and a recording medium.

[0002] A technique for detecting an object from an image using an object detection model generated by machine learning is known. For example, Patent Literature 1 discloses a training data collection device that automatically collects training data used for training the object detection model.

[0003] Japanese Patent Application Publication No. 2022 / 38941

[0004] By using the object detection model described above, it is possible to search for images that contain a specific object from among multiple images. However, even when using an object detection model, it may be difficult to present images that match the intent of a searcher performing an image search. This is because it is difficult to reflect the intent of the searcher in the image search. The present disclosure has been made in consideration of such problems, and one exemplary purpose thereof is to provide a technology that enables the presentation of images that match the intent of the searcher.

[0005] An information processing device according to an exemplary aspect of the present disclosure includes a receiving means for receiving an input sentence describing a search target, a search means for detecting the search target described in the input sentence from a group of images to be searched using a subject detection model that detects a specific subject from an image, the detection model being generated by machine learning the relationship between features generated from an image containing the subject and features generated from a sentence describing the subject, and a presentation means for presenting an image in which the search target has been detected.

[0006] In a search method according to an exemplary aspect of the present disclosure, at least one processor performs a reception process for receiving an input sentence describing a search target, a search process for detecting the search target described in the input sentence from a group of images to be searched using a subject detection model that detects a specific subject from an image, the detection model being generated by machine learning the relationship between features generated from an image containing the subject and features generated from a sentence describing the subject, and a presentation process for presenting an image in which the search target has been detected.

[0007] A recording medium according to an exemplary aspect of the present disclosure is a computer-readable recording medium having recorded thereon a search program that causes a computer to function as: a receiving means for receiving an input sentence describing a search target; a subject detection model for detecting a specific subject from an image, the subject detection model being generated by machine learning the relationship between features generated from an image containing the subject and features generated from a sentence describing the subject; a search means for detecting the search target described in the input sentence from a group of images that are the subject of the search, using the detection model; and a presentation means for presenting an image in which the search target has been detected.

[0008] According to an exemplary aspect of the present disclosure, an exemplary effect is achieved in that it becomes possible to present images that match the searcher's intention.

[0009] FIG. 1 is a block diagram showing a configuration of an information processing device according to the present disclosure. FIG. 2 is a flow diagram showing the flow of a search method according to the present disclosure. FIG. 3 is a diagram showing an overview of a search system according to the present disclosure. FIG. 4 is a block diagram showing a configuration of another information processing device according to the present disclosure. FIG. 5 is a diagram showing an example of a display screen displayed by the information processing device shown in FIG. 4. FIG. 6 is a diagram showing another example of a display screen displayed by the information processing device shown in FIG. 4. FIG. 7 is a diagram showing yet another example of a display screen displayed by the information processing device shown in FIG. 4. FIG. 8 is a flow diagram showing the flow of processing executed by the information processing device shown in FIG. 4. FIG. 9 is a block diagram showing the configuration of a computer that functions as an information processing device according to the present disclosure.

[0010] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, the effects mentioned in the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, embodiments that do not exhibit the effects mentioned in the exemplary embodiments shown below may also be included in the scope of the present invention.

[0011] [First Exemplary Embodiment] A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is a basic form for each of the exemplary embodiments described below. Note that the scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in the drawings referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0012] (Configuration of information processing device 1) The configuration of the information processing device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1. As shown in Fig. 1, the information processing device 1 includes a reception unit 101, a search unit 102, and a presentation unit 103.

[0013] The receiving unit 101 receives an input sentence describing a search target. For example, as in the example of FIG. 3 described later, the receiving unit 101 may receive, as a search condition, an input sentence describing the search target in natural language (in the example of FIG. 3 , the sentence “a person sitting is photographed”).

[0014] The search unit 102 uses a predetermined detection model to detect a search target described in the input sentence accepted by the accepting unit 101 from a group of images to be searched. The detection model is a subject detection model that detects a predetermined subject from an image. More specifically, the detection model is a detection model generated by machine learning the relationship between features generated from an image containing a subject and features generated from a sentence describing the subject. By inputting a sentence describing the subject and an image containing the subject into such a detection model, it is possible to output a detection result of the area in the image containing the subject. Such a detection model can be considered to be a conventional object detection model with the addition of a natural language processing function using a language model. For example, Grounding DINO or Grounded Language-Image Pre-training (GLIP) can be applied as the detection model.

[0015] For example, as in the example of FIG. 3 described below, when the search unit 102 receives an input of the sentence "a person sitting is photographed," it uses the detection model and the input sentence "a person sitting is photographed" to detect images "in which a person sitting is photographed." One advantage of using a detection model such as the one described above is that it is not necessary to perform machine learning of images "in which a person sitting is photographed" as training data in order to perform such detection. This is because the detection model described above learns features generated from sentences describing the subject. The image group to be searched may be stored in the information processing device 1 or in another device. It is also possible to search images stored in a distributed manner across multiple devices.

[0016] The presentation unit 103 presents an image in which the search target is detected by the search unit 102. For example, the presentation unit 103 may present the image by displaying the image in which the search target is detected (in this example, an image in which a person sitting is shown), as in the example of FIG. 3 described later. The manner in which the image is presented is not particularly limited. For example, the presentation unit 103 may present the image by displaying it on a display device, or by printing it on a printing device. Furthermore, the device that presents the image (for example, the display device or printing device described above) may be included in the information processing device 1, or may be an external device to the information processing device 1.

[0017] (Effects of information processing device 1) As described above, the information processing device 1 includes a reception unit 101 that receives an input sentence describing a search target, a search unit 102 that detects the search target described in the input sentence received by the reception unit 101 from a group of images to be searched, using a subject detection model that detects a predetermined subject from an image, the detection model being generated by machine learning the relationship between features generated from an image containing the subject and features generated from a sentence describing the subject, and a presentation unit 103 that presents an image in which the search target has been detected.

[0018] According to the above configuration, the detection model used to detect the search target is a model generated by machine learning not only features generated from an image containing the subject, but also features generated from a sentence describing the subject. By using such a detection model, it is possible to detect the search target according to the content of the description given in the input sentence. In other words, according to the above configuration, it is possible to reflect the searcher's intention in the input sentence and detect an image that matches that intention. This has the effect of making it possible to present images that match the searcher's intention.

[0019] (Recording Medium) The functions of the information processing device 1 described above can also be realized by a program. A search program according to this exemplary embodiment causes a computer to function as: a receiving unit that receives an input statement describing a search target; a search unit that detects a specific subject from an image by using a subject detection model generated by machine learning the relationship between features generated from an image containing the subject and features generated from a statement describing the subject; and a presentation unit that presents images from which the search target is detected. Furthermore, a recording medium according to this exemplary embodiment is a computer-readable recording medium on which the above-described search program is recorded. The search program according to this exemplary embodiment and the computer-readable recording medium on which the search program is recorded provide the effect of presenting images that match the searcher's intent.

[0020] (Flow of Search Method) The flow of the search method will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the search method. Note that the execution entity of each step in this response method may be a processor provided in the information processing device 1, or a processor provided in another device, or each step may be executed by a processor provided in a different device.

[0021] In S11 (reception process), at least one processor receives an input sentence that describes a search target.

[0022] In S12 (search processing), at least one processor uses a predetermined detection model to detect a search target described in the input sentence accepted in the acceptance processing of S11 from a group of images to be searched. Note that the detection model is a subject detection model that detects a predetermined subject from an image, and is a detection model generated by machine learning the relationship between features generated from an image containing the subject and features generated from a sentence describing the subject.

[0023] In S13 (presentation process), at least one processor presents the image in which the search target was detected in the search process of S12.

[0024] (Effects of the Search Method) As described above, the search method employs a configuration in which at least one processor executes: a reception process for receiving an input sentence describing a search target; a search process for detecting the search target described in the input sentence received in the reception process from a group of images to be searched using a subject detection model that detects a predetermined subject from an image, the detection model being generated by machine learning the relationship between features generated from an image containing the subject and features generated from a sentence describing the subject; and a presentation process for presenting images in which the search target has been detected. Therefore, the search method has the effect of enabling the presentation of images that match the searcher's intention.

[0025] Second Exemplary Embodiment A second exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be denoted by the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technical means shown in each drawing referenced to describe this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs.

[0026] (Configuration of Search System) The configuration of the search system 7 will be described with reference to Fig. 3. Fig. 3 is a diagram showing an overview of the search system 7. The search system 7 is a system having a function of receiving an input sentence describing a search target, a function of detecting the search target described in the input sentence from a group of images to be searched, and a function of presenting images from which the search target has been detected to a user of the search system 7.

[0027] As shown in the figure, the search system 7 includes an information processing device 1A that performs the above-described searches, an image DB (Data Base) 2 that stores a group of images to be searched, and a terminal device 3 used by a user of the search system 7. In the search system 7, the information processing device 1A can communicate with the terminal device 3 via any network. Note that the image DB 2 may be stored inside the information processing device 1A.

[0028] FIG. 3 shows an example in which the terminal device 3 is a smartphone. The terminal device 3 is not limited to a smartphone and may be any device capable of inputting search conditions and viewing search results. Although FIG. 3 shows only one terminal device 3, the search system 7 can be used by two or more users, each using their own terminal device. The terminal device 3 may also have the functions of the information processing device 1A. In this case, the terminal device 3 searches for images and presents the detected images.

[0029] FIG. 3 also shows a screen A1, which is an example of a UI (User Interface) screen that the information processing device 1A displays on the terminal device 3. The screen A1 includes an input field A11 for search conditions. A user of the terminal device 3, i.e., a searcher, uses the terminal device 3 to input an input sentence describing the search target into the input field A11. Specifically, the input sentence "there is a picture of a person sitting" is input into the input field A11 in FIG. 3. In this way, the search system 7 receives a sentence expressing the search target in natural language as the input sentence. This allows the user to easily input detailed search conditions. The input sentence may also be input by voice.

[0030] Screen A1 also displays search results obtained by searching the images stored in image DB2 based on the input sentence. Specifically, multiple images including image A12 and image A13 are displayed as search results. All of these images contain a "sitting person." The areas in these images where the "sitting person" was detected are indicated by dashed lines. In this way, by displaying the search target detection results in the image superimposed on the image, the user can recognize in which area the search target was detected.

[0031] Also displayed on screen A1 are partial images A121 and A122, which are cut-outs of the area in which the search target was detected in image A12. In this manner, information processing device 1A may display images cut-out from the area in which the search target was detected. This eliminates overlap between the search target and other subjects, and overlap between multiple search targets, thereby improving the visibility of each search target.

[0032] Furthermore, the numerical value "0.92" is displayed in association with the partial image A121, and the numerical value "0.86" is displayed in association with the partial image A122. These numerical values ​​are index values ​​indicating the likelihood of the detection result, and are also called the confidence level. By displaying the confidence level, the user can recognize the likelihood of the detection result.

[0033] As described above, the search system 7 allows a sentence expressed in natural language to be used as an input sentence. This allows the user of the search system 7 to perform a re-search as if they were interacting with the search system 7. Screen A1' shown in FIG. 3 shows an example of a screen after a re-search. In input field A11, the phrase "on a balance ball" has been added to the beginning of the sentence "There is a picture of a person sitting" that was input on screen A1. Furthermore, an image A14 showing "a person sitting on a balance ball" is displayed in the search result field on screen A1'. In this way, the search system 7 allows the user to freely add an explanation to the input sentence. The user can also freely modify or rewrite the input sentence.

[0034] (Configuration of Information Processing Device 1A) The configuration of information processing device 1A will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of information processing device 1A. As shown in the figure, information processing device 1A includes a control unit 10A that controls all the units of information processing device 1A, and a storage unit 11A that stores various data used by information processing device 1A. Information processing device 1A also includes a communication unit 12A that allows information processing device 1A to communicate with other devices (e.g., terminal device 3), an input unit 13A that receives various data input to information processing device 1A, and an output unit 14A that allows information processing device 1A to output various data.

[0035] As shown in the figure, the control unit 10A of the information processing device 1A includes a condition conformance calculation unit 104A, a detected image evaluation unit 105A, a priority determination unit 106A, a classification unit 107A, a change recommendation unit 108A, and a learning unit 109A in addition to the reception unit 101, search unit 102, and presentation unit 103 included in the information processing device 1. A detection model 111A is stored in the storage unit 11A. The classification unit 107A and the change recommendation unit 108A will be described with reference to FIGS. 7 and 8, respectively. The learning unit 109A will be described later in the section "Learning."

[0036] The receiving unit 101 receives an input statement describing a search target. The input statement is one of the search conditions. The receiving unit 101 may also receive input of other search conditions. For example, the receiving unit 101 may receive, as a search condition, a designation of a subject that is preferably not included. Note that, hereinafter, a subject that is preferably not included will be referred to as an unnecessary subject. In addition to this, for example, the receiving unit 101 may also receive, as search conditions, designation of various conditions related to images that the user wants to search for, such as image size, shooting date and time, file name, etc. Furthermore, the receiving unit 101 may also receive feedback from the user regarding the image presented to the user, as will be described in detail later.

[0037] The search unit 102 uses the detection model 111A to detect a search target described in the input sentence accepted by the accepting unit 101 from a group of images to be searched. The group of images to be searched may be, for example, a group of images stored in the image DB 2. The detection model 111A is a subject detection model that detects a specific subject from an image. More specifically, the detection model 111A is a model generated by machine learning the relationship between features generated from an image containing the subject and features generated from a sentence describing the subject. For example, the detection model 111A may accept input of a sentence describing the subject and an image, and output information indicating the area in which the subject appears in the image (e.g., the coordinates of the upper left corner of a rectangular area in which the subject appears, and the width and height of the area) and a degree of certainty that the subject appears in the area.

[0038] The presentation unit 103 presents an image for which a search target has been detected by the search unit 102. The manner in which the image is presented is not particularly limited. For example, the presentation unit 103 may present the image to a searcher, who is a user of the search system 7, by displaying the image on the terminal device 3. The presentation unit 103 may also present the image to the user by outputting the image via the output unit 14A. The presentation unit 103 may also present the image to the user by printing the image using a printing device.

[0039] The condition conformance calculation unit 104A calculates the conformance of each image in which the search target is detected. The conformance is an index value indicating the degree to which the image in which the search target is detected conforms to the search conditions input by the receiving unit 101. The method for calculating the conformance will be described later in the section "Conformance Calculation Method."

[0040] The detected image evaluation unit 105A evaluates the validity of the image based on the user's feedback on the image presented by the presentation unit 103. The evaluation method will be described later with reference to FIG.

[0041] The priority determination unit 106A determines a presentation priority, which is a priority for presenting images in which a search target is detected. Specifically, the priority determination unit 106A determines the presentation priority based on the degree of conformance calculated by the condition conformance calculation unit 104A. The presentation unit 103 presents images according to the presentation priority determined by the priority determination unit 106A. For example, the presentation unit 103 may display images in descending order of presentation priority, or may display a predetermined number of images with the highest presentation priority. Furthermore, for example, the presentation unit 103 may display a predetermined number of images with the highest presentation priority in a relatively large size, and other images in a smaller size. In this way, the presentation unit 103 may present images in a manner that relatively increases the visibility of images with a high presentation priority.

[0042] Furthermore, when evaluation is performed by the detected image evaluation unit 105A, the priority determination unit 106A determines the presentation priority based on the evaluation result of the detected image evaluation unit 105A and the degree of conformance calculated by the condition conformance calculation unit 104A. For example, the priority determination unit 106A may determine the presentation priority as the sum or product of the evaluation value indicating the evaluation result of the detected image evaluation unit 105A and the degree of conformance calculated by the condition conformance calculation unit 104A, or may determine the presentation priority as a representative value (e.g., average value) of the evaluation value and the degree of conformance.

[0043] (Method of calculating degree of conformance) The method of calculating the degree of conformance may be any method capable of calculating an index value indicating the degree to which the detected image conforms to the search criteria. For example, the condition conformance calculation unit 104A may calculate the degree of conformance based on an index value indicating the likelihood of the detection result output by the detection model 111A, i.e., the above-mentioned certainty factor. If the certainty factor of the detection result in which the search target is detected from a certain image using an input sentence describing the search target is high, it can be said that the image is likely to conform to the search criteria. Therefore, the certainty factor can be used directly as the degree of conformance, and it is also possible to calculate the degree of conformance using the certainty factor.

[0044] Furthermore, when an unnecessary subject is specified as a search condition, the condition conformance calculation unit 104A may calculate the conformance based on an index value indicating the likelihood of the detection result of the unnecessary subject (which can be rephrased as the confidence level as described above) output by the detection model 111A. In this case, the condition conformance calculation unit 104A may set the conformance lower as the confidence level of the detection result of the unnecessary subject increases.

[0045] When multiple search targets are detected from one image, the condition conformance calculation unit 104A may calculate the conformance using a representative value (e.g., maximum value or average value) of the certainty of each search target. Similarly, when multiple unnecessary subjects are detected from one image, the condition conformance calculation unit 104A may calculate the conformance using a representative value of the certainty of each unnecessary subject.

[0046] For example, two search targets are detected from image A12 displayed on screen A1 in Fig. 3, and their certainty factors are 0.92 and 0.86, respectively. Therefore, the condition suitability calculation unit 104A may determine the maximum value of these certainty factors, 0.92, or the average value of these certainty factors, 0.89, as the suitability for image A12.

[0047] The condition conformance calculation unit 104A may also calculate conformance based on the size of the area in which the search target is detected. In this case, the condition conformance calculation unit 104A may increase the conformance as the area in which the search target is detected increases. The condition conformance calculation unit 104A may also calculate conformance by taking both the confidence level and the area into consideration. For example, the condition conformance calculation unit 104A may calculate the conformance as a value obtained by multiplying the area in which the search target is detected by the confidence level as a weight.

[0048] Furthermore, for an image in which both the search target and an unnecessary subject are detected, the condition conformance calculation unit 104A may calculate the conformance using the following formula. The "weight" in the following formula may be set appropriately. For example, the "weight" may be set or changed by the user.

[0049] (Suitability) = (certainty of search target) × (area of ​​search target) - {(weight) × (certainty of unnecessary subject) × (area of ​​unnecessary subject) Note that if multiple search targets are detected, the portion of {(certainty of search target) × (area of ​​search target)} may be {(representative value of certainty of search target) × (representative value of area of ​​search target)}. This portion may also be the sum of {(certainty of the search target) × (area of ​​the search target)} calculated for each search target. The same applies when multiple unnecessary subjects are found.

[0050] As described above, the priority determination unit 106A determines the presentation priority using the suitability calculated by the condition suitability calculation unit 104A. Therefore, when the priority determination unit 106A determines the presentation priority using the suitability calculated using the certainty of the detection result of the search target or the unnecessary subject, the presentation priority is determined based on the certainty of the detection result of the search target or the unnecessary subject. Similarly, when the priority determination unit 106A determines the presentation priority using the suitability calculated based on the size of the area in which the search target or the unnecessary subject is captured, the presentation priority is determined based on the size of the area in which the search target or the unnecessary subject is captured.

[0051] In this way, the information processing device 1A may include a priority determination unit 106A that determines a presentation priority for presenting images in which the search target is detected based on at least one of an index value indicating the likelihood of the detection result of the search target and the size of the area in which the search target appears. Furthermore, the presentation unit 103 may present images in accordance with the presentation priority determined by the priority determination unit 106A. This provides, in addition to the effects achieved by the information processing device 1, an effect of being able to preferentially present to the user at least one of images that are likely to include the search target and images in which the search target is prominently displayed.

[0052] (Display Screen Example 1) An example of a display screen displayed by the information processing device 1A will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of a display screen displayed by the information processing device 1A. More specifically, Fig. 5 shows a screen A2, which is an input screen including an input item A21 for an input sentence and an input item A22 for an unwanted subject, and a screen A2' transitioned from screen A2.

[0053] The input fields for search conditions on screen A2 include an input field A21 for inputting an input sentence describing the search target, and an input field A22 for specifying an unwanted subject. The user inputs the input sentence into input field A21 using, for example, terminal device 3. The sentence "person sitting" is input into input field A21 in FIG. 5. When a search is performed by search unit 102 in this state, a list of images in which "person sitting" has been detected is displayed in the display area of ​​the search results on screen A2.

[0054] In this search, the search unit 102 may input a series of sentences, in which the user has input "person sitting" and a fixed character string such as "is in the picture," into the detection model 111A. By performing a search by adding a predetermined character string to the input sentence in this way, it is possible to reduce the input burden on the user and input sentences that are expected to produce reasonable search results into the detection model 111A. Furthermore, the presentation unit 103 may display a list of images stored in the image DB 2 until the search is performed. In this case, the presentation unit 103 may display each image in an order determined based on meta information such as the date and time of shooting and the file name.

[0055] Here, if the user inputs a sentence describing the unwanted subject into input item A22, a re-search is performed. In the re-search, the search unit 102 uses the detection model 111A to detect the unwanted subject from each image stored in the image DB2. In this search, the search unit 102 inputs the sentence "a person standing" into the detection model 111A. Note that the input item A22 may not be provided, and the unwanted subject may also be input into input item A21. In this case, the search unit 102 analyzes the input sentence input into input item A21 to identify the search target and the unwanted subject, and searches for them.

[0056] Next, the condition conformance calculation unit 104 calculates conformance based on the confidence level of the detection result of the unnecessary subject, etc. Next, the priority determination unit 106A determines the presentation priority based on the calculated conformance. As a result, the presentation priority of images containing unnecessary subjects is lowered, and the search results presented by the presentation unit 103 are updated. On screen A2' in FIG. 5, Img1 and 3, which were displayed on screen A2, are not displayed, and instead Img8 and 9 are displayed.

[0057] In this way, the information processing device 1A allows users to easily perform interactive searches by entering search criteria, receiving search results, viewing the presented search results, adding or changing search criteria, and receiving search results again.

[0058] As described above, the search unit 102 may detect an unwanted subject designated as an unwanted subject from a group of images to be searched. Furthermore, the presentation unit 103 may present images based on the results of the search target detection and the results of the unwanted subject detection. This configuration provides the effect of preferentially presenting images that do not contain unwanted subjects, in addition to the effects provided by the information processing device 1.

[0059] Furthermore, as described above, the presenting unit 103 may present an input screen including input items for an input sentence and input items for an unwanted subject. Furthermore, the accepting unit 101 may accept input of the input sentence and the unwanted subject via this input screen. This provides an effect that allows the user to easily input an input sentence and specify an unwanted subject, in addition to the effect provided by the information processing device 1.

[0060] (Display Screen Example 2) Another example of a display screen displayed by the information processing device 1A will be described with reference to Fig. 6. Fig. 6 is a diagram showing another example of a display screen displayed by the information processing device 1A. More specifically, Fig. 6 shows a screen A3 when user feedback on search results is received, and a screen A3' when the feedback is reflected in the presentation priority of images.

[0061] 5, the screen A3 displays an input item A21 for the input sentence and an input item A22 for the unwanted subject, as well as a list of images resulting from the search. Of the images displayed in the list, Img1 is displayed in association with an object A31 labeled "good," and Img4 is displayed in association with an object A32 labeled "bad."

[0062] Object A31 indicates that positive feedback has been given for the image associated with this object. On the other hand, object A32 indicates that negative feedback has been given for the image associated with this object. The user provides positive feedback for images displayed as search results that are close to the image the user was looking for, and provides negative feedback for images that are different from the image the user was looking for. For example, feedback may be received by having the user select an image for receiving positive feedback or an image for receiving negative feedback. Alternatively, for example, the user may input a value (e.g., a numerical value) indicating the degree of positivity or negativity as feedback. The receiving unit 101 receives such feedback, and the presenting unit 103 displays objects A31 and A32 according to the content of the received feedback.

[0063] Also displayed on screen A3 is an object A33 for reflecting the feedback content in the presentation priority of the image. When the reception unit 101 receives an operation to select this object A33, the detected image evaluation unit 105A evaluates the image based on the user's feedback. Then, the priority determination unit 106A updates the presentation priority based on the evaluation result.

[0064] Specifically, the detected image evaluation unit 105A calculates the similarity between each image stored in the image DB2 and an image for which positive feedback has been given and an image for which negative feedback has been given. The similarity calculation may use features extracted from the images. For example, the detected image evaluation unit 105A may use intermediate features or local features generated from each image by the search unit 102 during search. Furthermore, for example, the detected image evaluation unit 105A may extract features from each image using a feature extraction model that has been machine-learned in advance to extract image features. The algorithm of the feature extraction model is not particularly limited. For example, the feature extraction model may be a neural network model. Furthermore, if the receiving unit 101 receives a value indicating the degree of positive or negative feedback as feedback, the detected image evaluation unit 105A may reflect that value in the evaluation. For example, the detected image evaluation unit 105A may use the sum or product of the feedback value and the similarity as the evaluation result of the image for which feedback has been given.

[0065] The detected image evaluation unit 105A calculates the similarity between the feature amounts acquired as described above. The method for calculating the similarity is not particularly limited. For example, the detected image evaluation unit 105A may calculate Euclidean distance or cosine similarity.

[0066] The detected image evaluation unit 105A may give a high rating to an image that has a high similarity to an image that has received positive feedback, and give a low rating to an image that has a high similarity to an image that has received negative feedback. For example, the detected image evaluation unit 105A may calculate the evaluation value using the following formula. The "weight" in the following formula may be set appropriately. For example, the "weight" may be set or changed by the user.

[0067] (Evaluation value) = (Similarity with image for which positive feedback has been given) - (Weight) × (Similarity with image for which negative feedback has been given) Note that positive feedback may be given for multiple images. When positive feedback has been given for multiple images, the detected image evaluation unit 105A may calculate the similarity of one image with each of the images for which positive feedback has been given. Then, the detected image evaluation unit 105A may use the sum or a representative value (for example, the average or maximum value) of the calculated similarities as the similarity between the image and the images for which positive feedback has been given. The same applies to images for which negative feedback has been given.

[0068] Furthermore, the detected image evaluation unit 105A may generate a classifier that identifies whether an image is desirable in line with the user's intention, based on user feedback. In this way, the detected image evaluation unit 105A can input each image in which the search target is detected to the generated classifier, and calculate an evaluation value for the image using the output value output from the classifier.

[0069] For example, the detected image evaluation unit 105A can perform machine learning using images that have received positive feedback as training data to learn the characteristics of favorable images and generate a classifier that can identify whether an image is favorable or not. In this case, the more likely it is that an image input to the classifier is a favorable image, the larger the output value of the classifier will be, and therefore the detected image evaluation unit 105A may use the output value as the evaluation value of the image.

[0070] The detected image evaluation unit 105A can also perform machine learning using images that have received negative feedback as training data to learn the characteristics of undesirable images and generate a classifier that can identify whether an image is undesirable or not. In this case, the more likely it is that an image input to the classifier is an undesirable image, the larger the output value of the classifier will be, and therefore the detected image evaluation unit 105A may use the value obtained by subtracting the output value from the maximum output value as the evaluation value of the image.

[0071] The detected image evaluation unit 105A can also generate a classifier that distinguishes between desirable images that are in line with the user's intention and undesirable images that are not in line with the user's intention by performing machine learning using images that have received positive feedback and images that have received negative feedback as training data. Images presented to the user that have received neither positive nor negative feedback may be considered undesirable images that are not in line with the user's intention. In other words, images that have been presented but have not received feedback may be included in the training data as undesirable images. The algorithm of the classifier is not particularly limited. For example, the detected image evaluation unit 105A may generate a classifier using a support vector machine or a neural network.

[0072] The priority determination unit 106A can increase the presentation priority of an image for which positive feedback has been given, an image similar to that image, or an image identified as preferable by the above-mentioned classifier, by reflecting the evaluation value calculated by the detected image evaluation unit 105A as described above in the presentation priority. The priority determination unit 106A can also decrease the presentation priority of an image for which negative feedback has been given, an image similar to that image, or an image identified as unpreferable by the above-mentioned classifier.

[0073] The method of reflecting the evaluation value in the presentation priority is not particularly limited. For example, the priority determination unit 106A may determine the presentation priority as the sum or product of the conformance calculated by the condition conformance calculation unit 104A and the evaluation value calculated by the detected image evaluation unit 105A. Furthermore, for example, the priority determination unit 106A may determine the presentation priority as a representative value (e.g., average value or maximum value) of the conformance and the evaluation value.

[0074] As described above, the receiving unit 101 may receive feedback on the image presented by the presenting unit 103. Furthermore, the presenting unit 103 may reflect the content of the feedback in the presentation of the image. This provides, in addition to the effects provided by the information processing device 1, an effect that an image that matches the user's intention can be presented by an interactive method using feedback.

[0075] (Display Screen Example 3) Another example of a display screen displayed by information processing device 1A will be described with reference to Fig. 7. Fig. 7 is a diagram showing another example of a display screen displayed by information processing device 1A. More specifically, Fig. 7 shows a screen A4 on which a representative image is displayed as a search result, a screen A5 that is displayed when the representative image is selected, and a screen A4' on which each image of the group corresponding to the representative image is displayed.

[0076] 5, screen A4 displays input items A21 for input sentences and A22 for input items for unwanted subjects, and also displays images as search results divided into three groups A41 to A43. Here, not all images belonging to each group are displayed, but one representative image (specifically, Img1, 5, or 7) is displayed for each group.

[0077] Such grouping of images is performed by the classification unit 107A. That is, the classification unit 107A classifies images in which the search target is detected into multiple groups. By grouping the images by the classification unit 107A, the presentation unit 103 can present a representative image of each group instead of each image included in the group. This provides an effect of improving the viewability of search results in addition to the effect achieved by the information processing device 1. This configuration is particularly effective when a large number of images in which the search target is detected are present.

[0078] It should be noted that any of the images belonging to a group may be selected as the representative image. For example, the presentation unit 103 may select an image randomly from the images belonging to the group as the representative image, or may select the image with the highest presentation priority as the representative image. Furthermore, the images stored in the image DB 2 may be classified in advance. In this case, the presentation unit 103 can display the representative image of each group according to the predetermined classification of each image, and therefore the classification unit 107A may be omitted.

[0079] Here, when an operation to select one of the groups is performed, the presentation unit 103 may present each image belonging to the selected group. For example, when group A41 is selected, the presentation unit 103 may display screen A5 of FIG. 7 to present each image (specifically, Img1, 2, and 6) belonging to group A41 to the user. The presentation unit 103 may display screen A5 as a pop-up, or may display screen A5 instead of screen A4. Note that the operation to select a group may be set arbitrarily. For example, an operation to select a representative image may select a group corresponding to the representative image.

[0080] Also displayed on screen A5 are object A51 for displaying each image belonging to group A41 in the search result display area, and object A52 for ending the display of screen A5. When object A51 is selected, the presentation unit 103 ends the display of screen A5 and returns to the state in which screen A4 is displayed. On the other hand, when object A52 is selected, the presentation unit 103 ends the display of screen A5 and displays screen A4'. In the search result display area on screen A4', each image (specifically, Img1, 2, and 6) belonging to group A41 is displayed. Note that the reception unit 101 may receive an operation to return to the group-by-group presentation mode while screen A4' is displayed. When such an operation is received, the presentation unit 103 displays screen A4 again.

[0081] As described above, the presentation unit 103 may switch between a presentation mode in which a representative image is displayed and a presentation mode in which each image in the group corresponding to the representative image is displayed. This provides an effect of further improving the browsability of search results in addition to the effect provided by the information processing device 1.

[0082] (Display Screen Example 4) Another example of a display screen displayed by the information processing device 1A will be described with reference to Fig. 8. Fig. 8 is a diagram showing another example of a display screen displayed by the information processing device 1A. More specifically, Fig. 8 shows a screen A6 on which a message recommending a change in search conditions is displayed. Similar to screen A2 shown in Fig. 5, screen A6 displays an input field A21 for an input sentence and an input field A22 for an unwanted subject, as well as an image of a search result and a message A61.

[0083] Message A61 indicates that a large number of images were detected. The content of message A61 also recommends changing the search conditions. Message A61 is generated by change recommendation unit 108A. When the number of images detected as the search target is outside a predetermined range, change recommendation unit 108A generates such a message and has presentation unit 103 present the message, thereby recommending the user to change the search conditions.

[0084] The predetermined range may be determined in advance. For example, the predetermined range may be determined by setting an upper limit and a lower limit for the number of images, or by setting only an upper limit or only a lower limit. In other words, the change recommendation unit 108A may recommend a change to the search conditions when the number of images in which the search target is detected exceeds a predetermined upper limit or when the number of images in which the search target is detected is less than a predetermined lower limit.

[0085] In this way, the information processing device 1A may include a change recommendation unit 108A that recommends changing the search conditions when the number of images in which the search target is detected is outside a predetermined range. This provides the effect of encouraging the user to change the search conditions so that an appropriate number of images are presented, in addition to the effect provided by the information processing device 1.

[0086] Simply put, the change recommendation unit 108A may display a standard message encouraging the user to narrow down the search criteria if the number of images in which the search target is detected is too large, and may display a standard message encouraging the user to loosen the search criteria if the number of images in which the search target is detected is too small.

[0087] 8, the change recommendation unit 108A may generate a statement indicating how to change the search conditions and cause the statement to be presented by the presentation unit 103. This allows the user to recognize how to change the search conditions in order to present an appropriate number of images, and enables the user to smoothly change the search conditions.

[0088] The method for generating a sentence indicating how to change the search criteria is not particularly limited. For example, the change recommendation unit 108A may search a thesaurus for similar words to words included in the search criteria (which may be words included in an input sentence describing the search target or words included in a sentence describing an unwanted subject). The change recommendation unit 108A may then generate a sentence recommending using the detected similar words as search criteria. Alternatively, the change recommendation unit 108A may search a co-occurrence word dictionary for co-occurrence words with words included in the search criteria, and generate a sentence recommending adding the detected co-occurrence words to the search criteria. Sentences including the similar words or co-occurrence words searched for as described above can be generated, for example, by using a sentence template, or can be generated using a language model, etc., as described below.

[0089] The change recommendation unit 108A may generate a sentence using a language model trained by machine learning to generate a sentence corresponding to an input query. In this case, the change recommendation unit 108A may input at least one of an input sentence describing a search target and a sentence describing an unnecessary subject into the language model, and generate a sentence that paraphrases the input sentence. In this case, the change recommendation unit 108A may also input conditions such as "to reduce the number of search results" or "to increase the number of search results" as a query.

[0090] (Regarding Learning) The learning unit 109A generates or updates the detection model 111A through machine learning. The training data used for machine learning may be a combination of an input sentence accepted by the accepting unit 101 and an image in which the search unit 102 detects a search target using the input sentence. By performing machine learning using such training data, the detection accuracy of the detection model 111A can be improved.

[0091] Note that the detected images may include images that do not match the user's intention. For this reason, the learning unit 109A may use, as training data, images for which positive feedback has been given, among the images for which the search unit 102 has detected the search target.

[0092] Furthermore, the detection model 111A can be used not only for image search but also for practical systems such as biometric authentication, object authentication, object recognition, and automatic control of robots and vehicles. Conventional detection models do not provide sufficient detection accuracy, and problems such as a decrease in operation accuracy can occur in practical systems such as those described above. However, by using the detection model 111A, whose detection accuracy has been improved by machine learning as described above, it is possible to solve such problems.

[0093] As described above, the information processing device 1A may include the learning unit 109A that generates or updates the detection model 111A for detecting a predetermined target using an input sentence describing the predetermined target and an image showing the predetermined target as training data. This can improve the detection accuracy of the detection model 111A.

[0094] The information processing device 1A may also include a detection unit that detects a predetermined object from an image using the generated or updated detection model 111A, thereby enabling the predetermined object to be detected with high accuracy.

[0095] (Processing Flow) The processing flow executed by the information processing device 1A will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the processing flow executed by the information processing device 1A. Note that, before starting the processing of S21, the presentation unit 103 may display an input screen for accepting input of an input sentence and designation of an unwanted subject (for example, an input screen including input items A21 for an input sentence and input items A22 for an unwanted subject, such as screen A2 shown in Fig. 5) on a device used by the user for a search (for example, terminal device 3).

[0096] In S21, the receiving unit 101 receives input of search conditions. Specifically, the receiving unit 101 receives input of an input sentence and designation of an unwanted subject. As described above, the receiving unit 101 may receive these inputs via another device such as the terminal device 3. Alternatively, the receiving unit 101 may receive these inputs via the input unit 13A. Note that the receiving unit 101 does not necessarily have to receive the designation of an unwanted subject.

[0097] In S22, the search unit 102 uses the detection model 111A to detect a search target described in the input sentence accepted in S21 from the group of images to be searched. Also, if an unnecessary subject is specified in S21, the search unit 102 uses the detection model 111A to detect the unnecessary subject from the group of images to be searched.

[0098] In S23, the condition conformance calculation unit 104A calculates the conformance of each image detected by the search in S22 with the search conditions inputted in S21. For example, when a search target and an unnecessary subject are detected in S22, the condition conformance calculation unit 104A may calculate the conformance using the confidence of the detection result of the search target and the confidence of the detection result of the unnecessary subject output by the detection model 111A.

[0099] In S24, the priority determination unit 106A determines a priority (presentation priority) for an image in which a search target was detected in S22. In S24, the presentation priority of an image in which a search target was detected but an unnecessary subject was not detected is determined based on, for example, the certainty of the detection result of the search target in the image. Also, in S24, the presentation priority of an image in which both a search target and an unnecessary subject were detected is determined based on, for example, the certainty of the detection result of the search target and the certainty of the detection result of the unnecessary subject in the image. Simply put, the priority determination unit 106A may use the degree of suitability calculated for each image by the condition suitability calculation unit 104A in S23 as the presentation priority of each image.

[0100] In S25, the presentation unit 103 presents to the user images in which the search target was detected in the search of S22, according to the presentation priority determined in S24. For example, the presentation unit 103 may present the detected images by displaying the detected images on the terminal device 3. The presentation unit 103 may also output the images via the output unit 14A. In S25, the presentation unit 103 may display information indicating the area in which the search target was detected in the presented images (for example, information such as a dashed rectangle displayed on images A12 and A13 in FIG. 3). In S25, the presentation unit 103 may also display images (such as images A121 and A122 in FIG. 3) clipped from the area in which the search target was detected.

[0101] In S26, the change recommendation unit 108A determines whether the number of images detected in the search in S22 is outside a predetermined range. If the number of images detected is outside the range (YES in S26), the process proceeds to S27. If the number of images detected is within the range (NO in S26), the process proceeds to S28.

[0102] In S27, the change recommendation unit 108A recommends that the user change the search conditions. For example, the change recommendation unit 108A may cause the presentation unit 103 to present a message recommending that the user change the search conditions. The process of S26 may be performed before S25. In this case, the process of S27 may also be performed when the image is presented in S25.

[0103] In S28, the accepting unit 101 determines whether the search conditions (specifically, either the input sentence or the unwanted subject, or both) have been changed. If the determination in S28 is YES, the process returns to S22. In S22, to which the transition is made from S28, the searching unit 102 performs a search again according to the changed search conditions. On the other hand, if the determination in S28 is NO, the process proceeds to S29.

[0104] In S29, the reception unit 101 determines whether or not there is feedback from the user. If the determination in S29 is NO, the process proceeds to S31. On the other hand, if the determination in S29 is YES, the process proceeds to S30.

[0105] In S30, the detected image evaluation unit 105A calculates an evaluation value for each image detected in S22 based on the content of the user's feedback. For example, the detected image evaluation unit 105A may calculate the similarity between the image that was the subject of the feedback and each of the other images detected in S22. In this case, the detected image evaluation unit 105A calculates an evaluation value for each image based on the calculated similarity and the content of the feedback. Note that in S30, the detected image evaluation unit 105A may generate a classifier that distinguishes between the quality of images based on the content of the feedback and calculate an evaluation value for each image using the classifier.

[0106] When the process of S30 is completed, the process returns to the process of S24, where the priority determination unit 106A updates the presentation priority based on the evaluation value calculated in S30. For example, the priority determination unit 106A may perform a process for each image by adding the evaluation value calculated in S30 to the suitability calculated in S23. In this case, the sum of the suitability and the evaluation value becomes the updated presentation priority.

[0107] In S31, the reception unit 101 determines whether an operation to end the search has been performed. If the determination in S31 is NO, the process returns to S28. On the other hand, if the determination in S31 is YES, the process in FIG. 9 ends.

[0108] 9, the classification unit 107A may classify the images detected in S22 into a plurality of groups. In this case, in S24, the presentation unit 103 may present a representative image of each group instead of each image included in the group. Furthermore, when an operation to select a presented representative image is performed, the presentation unit 103 may present each image of the group corresponding to the selected representative image.

[0109] [Modifications] The execution entity of each process described in each of the above exemplary embodiments is arbitrary and is not limited to the above examples. In other words, the functions of the information processing devices 1 and 1A can be realized by multiple devices (which can also be called processors) that can communicate with each other. For example, the execution of each process described in the flow charts of Figures 2 and 9 can be shared among multiple processors. In other words, the execution entity of the search method in each of the above exemplary embodiments may be one processor or multiple processors.

[0110] [Example of Software Implementation] Some or all of the functions of the information processing devices 1 and 1A may be implemented by hardware such as an integrated circuit (IC chip), or may be implemented by software.

[0111] In the latter case, the information processing devices 1 and 1A are realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Fig. 10. Fig. 10 is a block diagram showing the hardware configuration of computer C that functions as information processing device 1 or 1A.

[0112] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P (search program) for operating the computer C as the information processing device 1 or 1A. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing the functions of the information processing device 1 or 1A.

[0113] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0114] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0115] The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0116] [Appendix 1] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0117] (Supplementary Note 1) An information processing device comprising: a receiving means for receiving an input sentence describing a search target; a search means for detecting the search target described in the input sentence from a group of images to be searched using a subject detection model that detects a predetermined subject from an image, the detection model being generated by machine learning the relationship between features generated from an image containing the subject and features generated from a sentence describing the subject; and a presentation means for presenting an image in which the search target has been detected.

[0118] (Supplementary Note 2) The information processing device according to Supplementary Note 1, wherein the search means detects an unnecessary subject designated as an unnecessary subject from the group of images, and the presentation means presents the image based on the detection result of the search target and the detection result of the unnecessary subject.

[0119] (Supplementary Note 3) The information processing device according to Supplementary Note 2, wherein the presentation means presents an input screen including input items for the input sentence and input items for the unnecessary subject, and the acceptance means accepts input of the input sentence and the unnecessary subject via the input screen.

[0120] (Supplementary Note 4) An information processing device described in any of Supplementary Notes 1 to A3, comprising a priority determination means for determining the priority of presenting an image in which the search target is detected based on at least one of an index value indicating the likelihood of the detection result of the search target and the size of the area in which the search target appears, and the presentation means presents the image in accordance with the priority.

[0121] (Supplementary Note 5) The information processing device according to any one of Supplementary Notes 1 to A4, wherein the accepting unit accepts feedback on the image presented by the presenting unit, and the presenting unit reflects the content of the feedback in the presentation of the image.

[0122] (Supplementary Note 6) An information processing device according to any one of Supplementary Notes 1 to A5, further comprising a classification means for classifying images in which the search target is detected into a plurality of groups, and the presentation means for presenting a representative image of each group instead of each image included in the group.

[0123] (Supplementary Note 7) The information processing device according to Supplementary Note 6, wherein the presentation means switches between a presentation mode in which the representative image is displayed and a presentation mode in which each image in a group corresponding to the representative image is presented.

[0124] (Supplementary Note 8) The information processing device according to any one of Supplementary Notes 1 to 7, further comprising a change recommendation unit that recommends changing search conditions when the number of images in which the search target is detected is outside a predetermined range.

[0125] (Supplementary Note 9) A search method in which at least one processor executes a reception process for receiving an input sentence describing a search target, a search process for detecting the search target described in the input sentence from a group of images to be searched using a subject detection model that detects a predetermined subject from an image, the detection model being generated by machine learning the relationship between features generated from an image containing the subject and features generated from a sentence describing the subject, and a presentation process for presenting an image in which the search target has been detected.

[0126] (Supplementary Note 10) A computer-readable recording medium having recorded thereon a search program that causes a computer to function as: a receiving means for receiving an input sentence describing a search target; a search means for detecting a predetermined subject from an image, the search target described in the input sentence, from a group of images to be searched, using a subject detection model generated by machine learning the relationship between features generated from an image containing the subject and features generated from a sentence describing the subject; and a presentation means for presenting an image in which the search target has been detected.

[0127] REFERENCE SIGNS LIST 1, 1A INFORMATION PROCESSING DEVICE 101 RECEIVING UNIT (RECEIVING MEANS) 102 SEARCHING UNIT (SEARCHING MEANS) 103 PRESENTING UNIT (PRESENTING MEANS) 106A PRIORITY DETERMINING UNIT (PRIORITY DETERMINING UNIT) 107A CLASSIFICATION UNIT (CLASSIFICATION MEANS) 108A CHANGE RECOMMENDING UNIT (CHANGE RECOMMENDING UNIT) 111A DETECTION MODEL

Claims

1. An information processing device comprising: a receiving means for receiving an input sentence describing a search target; a search means for detecting the search target described in the input sentence from a group of images to be searched using a subject detection model that detects a specific subject from an image, the detection model being generated by machine learning the relationship between features generated from an image containing the subject and features generated from a sentence describing the subject; and a presentation means for presenting an image in which the search target has been detected.

2. The information processing device according to claim 1, wherein the search means detects unwanted subjects designated as unnecessary subjects from the group of images, and the presentation means presents the images based on the detection results of the search targets and the detection results of the unwanted subjects.

3. The information processing device according to claim 2, wherein the presentation means presents an input screen including input items for the input sentence and input items for the unnecessary subject, and the acceptance means accepts input of the input sentence and the unnecessary subject via the input screen.

4. An information processing device as claimed in any one of claims 1 to 3, further comprising a priority determination means for determining a priority for presenting an image in which the search target is detected based on at least one of an index value indicating the likelihood of the detection result of the search target and the size of the area in which the search target appears, and wherein the presentation means presents the image in accordance with the priority.

5. An information processing device according to any one of claims 1 to 3, wherein the accepting means accepts feedback on the image presented by the presenting means, and the presenting means reflects the content of the feedback in the presentation of the image.

6. An information processing device as claimed in any one of claims 1 to 3, further comprising a classification means for classifying images from which the search target is detected into a plurality of groups, and wherein the presentation means presents a representative image of each group instead of each image contained in the group.

7. The information processing device according to claim 6, wherein said presentation means switches between a presentation mode in which the representative image is displayed and a presentation mode in which each image in the group corresponding to the representative image is presented.

8. The information processing device according to any one of claims 1 to 3, further comprising a change recommendation means for recommending a change in search conditions when the number of images detected as the search target is outside a predetermined range.

9. A search method in which at least one processor performs the following steps: a reception process for receiving an input sentence describing a search target; a search process for detecting the search target described in the input sentence from a group of images to be searched using a subject detection model that detects a specific subject from an image, the detection model being generated by machine learning the relationship between features generated from an image containing the subject and features generated from a sentence describing the subject; and a presentation process for presenting an image in which the search target has been detected.

10. A computer-readable recording medium having recorded thereon a search program that causes a computer to function as: an acceptance means for accepting an input sentence describing a search target; a search means for detecting the search target described in the input sentence from a group of images to be searched using a subject detection model that detects a specific subject from an image, the detection model being generated by machine learning the relationship between features generated from an image containing the subject and features generated from a sentence describing the subject; and a presentation means for presenting an image in which the search target has been detected.

Images