Image processing device, image processing method, and program

Abstract

This image processing device (10) comprises an image acquisition unit (110), an estimation unit (120), and an output unit (130). The image acquisition unit (110) acquires an image from a terminal (20). This image includes a commodity in some regions. The estimation unit (120) processes the image acquired by the image acquisition unit (110) and thereby generates subject estimation data that indicates the result of estimation on the type of a photographing subject. For example, the estimation unit (120) processes a region of the image surrounding the commodity and thereby generates subject estimation data. The output unit (130) performs output based on the subject estimation data.

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] In recent years, users have been able to search for information about a product by using an image including the product as a search key. For example, Patent Document 1 describes the use of an image of an actual product or a catalog, or image data cut out from a video such as a television video, as a search key. Patent Document 2 describes the use of an image of a product sourced from various advertising media such as magazines, flyers, pamphlets, posters, television commercials, advertising videos, and web advertisements as a search key. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2011 / 049209 [Patent Document 2] JP 2017-33157 A Summary of the Invention [Problem to be solved by the invention]

[0004] There are a number of media that provide consumers with images that include products. In addition, consumers may also use images of actual items for searches. Meanwhile, the degree of influence that each of the actual items and image media has on consumer behavior is an important index from the perspective of marketing. The present inventor considered that in order to estimate this degree of influence, it is necessary to estimate the subject that was photographed when a person generated an image that includes a product. One example of the objective of the present invention is to estimate the subject that was photographed when a person generated an image that includes a product. [Means for solving the problem]

[0005] According to the present invention, an image capturing means captures an image of a subject, the image including a product in a part of the image capturing means; an estimation means for generating object estimation data indicating an estimation result of a type of the photographed object by processing the image; an output means for performing an output based on the target estimation data; An image processing apparatus is provided, comprising:

[0006] According to the present invention, a computer An image acquisition process for acquiring an image of a photographed subject, the image including a product in a portion of the image; an estimation process for generating object estimation data indicating an estimation result of a type of the photographed object by processing the image; An output process for performing an output based on the target estimation data; An image processing method is provided that performs the above steps.

[0007] According to the present invention, a computer includes: An image acquisition process for acquiring an image of a photographed subject, the image including a product in a portion of the image; an estimation process for generating object estimation data indicating an estimation result of a type of the photographed object by processing the image; An output process for performing an output based on the target estimation data; A program for performing the above is provided. Effect of the Invention

[0008] According to the present invention, it is possible to estimate the subject of a photograph when a person generates an image including a product. [Brief description of the drawings]

[0009] The above objects, as well as other objects, features and advantages, will become more apparent from the following preferred embodiments and the accompanying drawings.

[0010] [Figure 1]FIG. 1 is a diagram for explaining a usage environment of an image processing device according to an embodiment. [Diagram 2] FIG. 2 illustrates an example of a functional configuration of an image processing apparatus. [Diagram 3] 4 is a diagram for explaining a first example of an estimation process performed by an estimation unit; FIG. [Figure 4] 13 is a diagram for explaining a second example of the estimation process performed by the estimation unit. FIG. [Diagram 5] 13 is a diagram for explaining a second example of the estimation process performed by the estimation unit. FIG. [Figure 6] 13 is a diagram for explaining a third example of the estimation process performed by the estimation unit. FIG. [Figure 7] FIG. 13 is a diagram for explaining a fourth example of the estimation process performed by the estimation unit. [Figure 8] FIG. 4 is a diagram illustrating an example of information stored in a person information storage unit. [Figure 9] FIG. 4 is a diagram showing a first example of information output by an output unit. [Figure 10] FIG. 11 is a diagram showing a second example of information output by the output unit. [Figure 11] FIG. 2 is a diagram illustrating an example of a hardware configuration of an image processing apparatus. [Figure 12] 11 is a flowchart showing an example of a process performed by an image processing device together with a process performed by a terminal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are given the same reference numerals and the description will be omitted as appropriate.

[0012] 1 is a diagram for explaining a usage environment of an image processing device 10 according to an embodiment. The image processing device 10 is used together with a terminal 20.

[0013] The terminal 20 is a portable terminal such as a smartphone or a tablet terminal, and is operated by a person. The terminal 20 transmits an image to the image processing device 10 according to this operation. This image is generated by photographing an object to be photographed, and includes a product in a part of the area. An example of an object to be photographed is a product (actual item) arranged in a real space such as a store, and a medium including an image of the product. Examples of the medium here are printed matter such as magazines and advertisements, a screen displayed on a display based on a broadcast, a screen displayed on a display based on a social networking service (SNS), and a screen displayed on a display based on an email. Printed matter also includes advertisements placed around town. The image transmitted by the terminal 20 to the image processing device 10 may be a still image or a video.

[0014] When the image processing device 10 acquires an image from the terminal 20, the image processing device 10 processes the image to generate data indicating an estimation result of the type of the photographed object (hereinafter, referred to as object estimation data). The image processing device 10 also performs output based on the object estimation data. This output may be the object estimation data itself. Furthermore, when there are multiple terminals 20 and multiple pieces of object estimation data are generated, the output unit 130 may output the result of statistically processing the object estimation data.

[0015] Moreover, the image processing device 10 estimates a product included in an image by processing the image acquired from the terminal 20. Then, the image processing device 10 executes at least a part of a process for allowing the user of the terminal 20 to purchase the product.

[0016] The terminal 20 may have a photographing function. In this case, the terminal 20 may generate an image to be transmitted to the image processing device 10.

[0017] 2 is a diagram showing an example of a functional configuration of the image processing device 10. The image processing device 10 includes an image acquisition unit 110, an estimation unit 120, and an output unit .

[0018] The image acquiring unit 110 acquires an image from the terminal 20. As described above, this image includes a product in a partial area.

[0019] The estimation unit 120 generates the above-mentioned object estimation data by processing the image acquired by the image acquisition unit 110. For example, the estimation unit 120 generates the object estimation data by processing an area around the product in the image. Furthermore, when the image is a moving image, the estimation unit 120 generates the object estimation data by processing a plurality of frame images included in the moving image. Details of the generation process of the object estimation data will be described later with reference to other figures.

[0020] Furthermore, the estimation unit 120 processes the image acquired by the image acquisition unit 110 to estimate the product included in the image. At this time, the estimation unit 120 may further estimate products similar to this product (hereinafter, referred to as similar products). This estimation result includes, for example, at least one of the product name and the product code (for example, JAN code). Note that this estimation process may be performed, for example, by feature matching, or may be performed using a model generated by machine learning.

[0021] Then, the estimation unit 120 performs at least a part of the process required for the user of the terminal 20 to purchase this product and / or similar products, as necessary. One example of this process is to identify an online shop and / or a physical store where this product and / or similar products can be purchased, and to transmit information identifying the online shop and / or physical store (hereinafter referred to as purchase assistance information) to the terminal 20. The purchase assistance information may include a URL or link of the online shop, or may include information indicating the location of the physical store (e.g., an address and / or a map). Furthermore, the purchase assistance information may include advertisement information or coupon information related to the product and / or similar products estimated by the estimation unit 120. When identifying the online shop and / or physical store, the estimation unit 120 uses the above estimation result, i.e., at least one of the product name and product code.

[0022] The estimation unit 120 stores information indicating the estimation result in the person information storage unit 150 in association with the user of the terminal 20. Details of the information stored in the person information storage unit 150 will be described later with reference to other drawings. The person information storage unit 150 may be a part of the image processing device 10, or may be located outside the image processing device 10.

[0023] The output unit 130 performs output based on the target estimation data. As described above, this output may be the target estimation data itself. In addition, when there are multiple terminals 20, the estimation unit 120 generates target estimation data for each of the multiple terminals 20. In this case, the output unit 130 may generate output data by statistically processing the multiple target estimation data. In this case, an example of the output data is data (hereinafter, referred to as first association data) indicating the association between the attribute information of the users of the multiple terminals 20 and the target estimation data. The attribute information of the users of the multiple terminals 20 is stored in the person information storage unit 150. By using the first association data, it is possible to identify a preferred shooting target by age group (or gender), for example. A specific example of the output performed by the output unit 130 will be described later using other figures.

[0024] The image processing device 10 further includes a purchase result acquisition unit 140. The purchase result acquisition unit 140 acquires information indicating whether or not a product included in an image acquired by the image acquisition unit 110 has been purchased (hereinafter, referred to as purchase result information). The purchase result acquisition unit 140 acquires the purchase result information from, for example, the terminal 20, but may also acquire the purchase result information from another device (for example, a server that manages an online shop or a physical store). The information acquired by the purchase result acquisition unit 140 is stored in the person information storage unit 150.

[0025] Then, the output unit 130 outputs data indicating the association between the object estimation data and the purchase result information (hereinafter, referred to as second association data). At this time, the output unit 130 uses the information stored in the person information storage unit 150. By using the second association data, the user of the image processing device 10 can recognize to what extent the photographed object is linked to a purchase.

[0026] FIG. 3 is a diagram for explaining a first example of the estimation process performed by the estimation unit 120. In the example shown in this figure, the subject to be photographed is a screen displayed on a display based on radio waves or online broadcasting. This display may be used as a television or may be used as a digital signage. When an image including a product is displayed on this display, the user of the terminal 20 causes the terminal 20 to photograph the screen of this display. The terminal 20 may generate a still image or may generate a video. In the image generated by the terminal 20, the frame of the display may be shown, or the frame may not be shown and only the screen of the display may be shown. In the former case, the estimation unit 120 detects the frame of the display to determine that the subject to be photographed is a display. In the latter case, the estimation unit 120 determines that the subject to be photographed is a display when a scanning line specific to the display is included in the image.

[0027] Furthermore, if the terminal 20 generates a video, the estimation unit 120 can process the video to identify the type of content displayed on the display (for example, an advertisement aired on a television broadcast or an advertisement aired on a digital signage). This also enables the estimation unit 120 to identify the subject of the image.

[0028] Furthermore, if the image includes the edge of a display, the estimation unit 120 detects the features of the part of the image outside the display, and by performing a matching process on these features, it is possible to identify where the display is located (for example, whether it is indoors or outdoors).

[0029] 4 is a diagram for explaining a second example of the estimation process performed by the estimation unit 120. In the example shown in this figure, the photographed object is an actual product. In this case, it is preferable that the terminal 20 generates a video. Then, the estimation unit 120 determines that the photographed object is an actual product if there is a change in at least a part of the surroundings of the product, and determines that the photographed object is a printed matter such as a magazine if there is no change in the surroundings of the product.

[0030] As shown in FIG. 5, when the subject to be photographed is a printed matter, image When the estimation unit 120 detects the edge of the magazine, it determines that the photographed subject is a printed matter.

[0031] Furthermore, if the image includes the edge of a printed matter, the estimation unit 120 can identify the location where the printed matter is located (for example, whether it is indoors or outdoors) by processing the part of the image outside the printed matter.

[0032] 6 is a diagram for explaining a third example of the estimation process performed by the estimation unit 120. In the example shown in this figure, the subject of the image capture is a screen displayed on a display based on an SNS. In this case, this screen has a screen configuration specific to that SNS. By detecting the presence or absence of this screen configuration, the estimation unit 120 can identify that the subject of the image capture is an SNS and also identify the service name of that SNS.

[0033] 7 is a diagram for explaining a fourth example of the estimation process performed by the estimation unit 120. In the example shown in this figure, the subject to be photographed is a screen displayed on a display based on an e-mail. In this case, this screen has a screen configuration specific to e-mail. The estimation unit 120 can identify that the subject to be photographed is an e-mail by detecting the presence or absence of this screen configuration.

[0034] 8 is a diagram showing an example of information stored in the person information storage unit 150 of the image processing device 10. In the example shown in this figure, the person information storage unit 150 stores, for each person, identification information (hereinafter referred to as person identification information), attribute information, and history information assigned to that person in association with each other.

[0035] The attribute information includes the person's name, gender, and age, but may also include other information.

[0036] The history information includes the results of the analysis by the estimation unit 120 of the image sent by the person from the terminal 20. The analysis results by the estimation unit 120 include the photographed subject, and the product name and / or product code. The history information also includes information indicating whether or not the product estimated by the estimation unit 120 has been purchased. Note that the history information may further include the date and time of purchase of the product, and information specifying the online shop or physical store that sold the product.

[0037] FIG. 9 is a diagram showing a first example of information output by the output unit 130 of the image processing device 10. In the example shown in this figure, the output unit 130 outputs first association data. As described above, the first association data indicates the association between the attribute information of each user of the multiple terminals 20 and the object estimation data. In the example shown in this figure, the first association data indicates, for each attribute, the number of times that a person having that attribute has used the subject to be photographed in a search. The information shown in this figure may be by product name or product code, or by product category. Here, the product category may be, for example, a major category such as clothes or food, or a minor category such as coats or shirts.

[0038] 10 is a diagram showing a second example of information output by the output unit 130 of the image processing device 10. In the example shown in this figure, the output unit 130 outputs second association data. As described above, the second association data indicates the association between the object estimation data and the purchase result information. In the example shown in this figure, the second association data indicates the number of times the photographed object was used by a person who ended up purchasing the product, by person's attributes. The information shown in this figure may also be organized by product name or product code, or by product category.

[0039] 11 is a diagram showing an example of the hardware configuration of the image processing device 10. The image processing device 10 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0040] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0041] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0042] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0043] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores program modules that realize each function of the image processing device 10 (for example, the image acquisition unit 110, the estimation unit 120, the output unit 130, and the purchase result acquisition unit 140). The processor 1020 loads each of these program modules onto the memory 1030 and executes them, thereby realizing each function corresponding to the program module. The storage device 1040 also functions as a person information storage unit 150.

[0044] The input / output interface 1050 is an interface for connecting the image processing device 10 to various input / output devices.

[0045] The network interface 1060 is an interface for connecting the image processing device 10 to a network. This network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The method for connecting the network interface 1060 to the network may be a wireless connection or a wired connection. The image processing device 10 may communicate with the terminal 20 via the network interface 1060.

[0046] FIG. 12 is a flowchart showing an example of the processing performed by the image processing device 10 together with the processing performed by the terminal 20.

[0047] When the user of the terminal 20 finds a product that interests him / her, he / she causes the terminal 20 to generate an image including the product (step S10). Here, an example of an object photographed by the terminal 20 is as described with reference to FIG. 1. The terminal 20 then transmits the photographed image to the image processing device 10. At this time, the terminal 20 also transmits the user's personal identification information to the image processing device 10 (step S20).

[0048] The image acquisition unit 110 of the image processing device 10 acquires the image transmitted from the terminal 20. Then, the estimation unit 120 processes this image to estimate the subject (step S30). An example of the processing performed here is as described with reference to Figs. 2 to 7.

[0049] The estimation unit 120 also processes the image to identify the product and / or similar products contained in the image (step S40).The estimation unit 120 then identifies online shops and / or brick-and-mortar stores where the product and / or similar products can be purchased, and generates information about the online shops and / or brick-and-mortar stores, i.e., purchase assistance information (step S50).The estimation unit 120 then transmits the purchase assistance information to the terminal 20 (step S60).

[0050] Furthermore, the estimation unit 120 links the subject estimated in step S30 and information indicating the product and / or similar products identified in step S40 (e.g., at least one of the product name and product code) to each other and stores them in the person information storage unit 150. At this time, the estimation unit 120 links these pieces of information to the person identification information transmitted in step S20 (step S70).

[0051] When the terminal 20 acquires the purchase assistance information from the image processing device 10, the terminal 20 displays the purchase assistance information on the display (step S80). When the user of the terminal 20 purchases a product, the user uses the purchase assistance information.

[0052] The terminal 20 also generates information indicating whether the product was purchased (hereinafter referred to as purchase result information) (step S90), and transmits this purchase result information together with the person identification information to the image processing device 10 (step S100). The purchase result information includes the purchase date and time, and information for identifying the online shop or physical store that sold the product, as necessary.

[0053] Then, the purchase result acquiring unit 140 of the image processing device 10 stores the purchase result information transmitted from the terminal 20 as a part of the history information in the person information storage unit 150. At this time, the purchase result acquiring unit 140 links the purchase result information to the person identification information transmitted in step S100 (step S110).

[0054] Thereafter, at the required timing, the output unit 130 of the image processing device 10 generates and outputs output data.

[0055] As described above, according to this embodiment, the image processing device 10 estimates the subject of a photograph when a person generates an image including a product. Therefore, by using this estimation result, it is possible to estimate the degree of influence that the actual product or the medium that provided the product image had on the behavior of the consumer. This degree of influence is indicated, for example, by the above-mentioned first association data and second association data.

[0056] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various configurations other than those described above can also be adopted.

[0057] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the figures can be changed to the extent that the content is not affected. In addition, the above-mentioned embodiments can be combined to the extent that the content is not contradictory.

[0058] A part or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes. 1. An image acquisition means for acquiring an image of a subject, the image including a product in a part of the image; an estimation means for generating object estimation data indicating an estimation result of a type of the photographed object by processing the image; an output means for performing an output based on the target estimation data; An image processing device comprising: 2. In the image processing device according to the above item 1, The estimation means generates the object estimation data by processing an area surrounding the product in the image. 3. In the image processing device according to the above 1 or 2, the image acquisition means acquires a moving image as the image, The estimation means generates the target estimation data by processing a plurality of frame images included in the moving image. 4. In the image processing device according to any one of the above items 1 to 3, The type of subject to be photographed by the image processing device includes at least one of a screen displayed on a display based on a broadcast, a screen displayed on a display based on a social networking service (SNS), a screen displayed on a display based on an email, a printed matter, and the product arranged in real space. 5. In the image processing device according to any one of the above items 1 to 4, the image is generated by a terminal; The image acquisition means acquires the images from a plurality of the terminals, The estimation means generates the target estimation data for each of the plurality of terminals; The output means outputs first association data indicating an association between attribute information of users of the plurality of terminals and the target estimation data. 6. In the image processing device according to any one of 1 to 5 above, The estimation means further includes an image processing device that processes the image to estimate the product and / or a similar product that is similar to the product. 7. In the image processing device according to claim 6, The system further includes a purchase result acquisition unit for acquiring purchase result information indicating whether the product and / or the similar product has been purchased, The output means outputs second association data indicating an association between the object estimation data and the purchase result information. 8. The computer An image acquisition process for acquiring an image of a photographed subject, the image including a product in a portion of the image; an estimation process for generating object estimation data indicating an estimation result of a type of the photographed object by processing the image; An output process for performing an output based on the target estimation data; An image processing method. 9. The image processing method according to claim 8, The image processing method, wherein the computer generates the object estimation data by processing an area surrounding the product in the image in the estimation process. 10. In the image processing method according to 8 or 9 above, The computer includes: In the image acquisition, a moving image is acquired as the image, An image processing method, in which the estimation process generates the target estimation data by processing a plurality of frame images included in the moving image. 11. In the image processing method according to any one of the above items 8 to 10, An image processing method, in which the type of subject to be photographed includes at least one of a screen displayed on a display based on a broadcast, a screen displayed on a display based on a social networking service (SNS), a screen displayed on a display based on an email, a printed matter, and the product arranged in real space. 12. In the image processing method according to any one of the above items 8 to 11, the image is generated by a terminal; The computer includes: In the image acquisition process, the images are acquired from a plurality of the terminals; In the estimation process, the target estimation data is generated for each of the plurality of terminals; The image processing method includes outputting, in the output process, first association data indicating an association between attribute information of users of the plurality of terminals and the target inference data. 13. In the image processing method according to any one of the above items 8 to 12, The image processing method in which the computer, in the estimation process, further processes the image to estimate the product and / or similar products to the product. 14. The image processing method according to claim 13, The computer includes: and further performing a purchase result acquisition process for acquiring purchase result information indicating whether the product and / or the similar product has been purchased; The image processing method further comprises outputting, in the output process, second association data indicating an association between the object estimation data and the purchase result information. 15. On the computer, An image acquisition process for acquiring an image of a photographed subject, the image including a product in a portion of the image; an estimation process for generating object estimation data indicating an estimation result of a type of the photographed object by processing the image; An output process for performing an output based on the target estimation data; A program that performs the following. 16. In the program according to 15 above, Said presumption process generating the object estimation data by processing a region of the image surrounding the item; Including ,program. 17. In the program according to 15 or 16 above, 1. Image Acquisition process Get the video as the image This includes , Said presumption process generates the target estimation data by processing a plurality of frame images included in the video. Including ,program. 18. In the program according to any one of the above 15 to 17, A program in which the types of subjects to be photographed include at least one of a screen displayed on a display based on a broadcast, a screen displayed on a display based on a social networking service (SNS), a screen displayed on a display based on an email, a printed matter, and the product arranged in real space. 19. In the program according to any one of the above 15 to 18, the image is generated by a terminal; 1. Image Acquisition process acquires the images from the plurality of terminals This includes , Said presumption process generating the target estimation data for each of the plurality of terminals; This includes , The output process outputting first association data indicating an association between attribute information of users of the plurality of terminals and the target estimation data; Including ,program. 20. In the program according to any one of the above 15 to 19, Said presumption process further processes the image to estimate the product and / or a similar product similar to the product. Including, program. 21. In the program according to claim 20, The computer further acquires purchase result information indicating whether the product and / or the similar product has been purchased. process of Run , The output process outputs second association data indicating an association between the target estimation data and the purchase result information. Including ,program. [Explanation of symbols]

[0059] 10 Image processing device 20 Terminals 110 Image acquisition unit 120 Estimation part 130 Output section 140 Purchase result acquisition unit 150 person information storage section

Claims

1. An image acquisition means for acquiring an image of a photographed object, the image including a product in a part of the image; an estimation means for estimating, by processing the image, at least one of a screen displayed on a display based on a broadcast, a screen displayed on a display based on a social networking service (SNS), a screen displayed on a display based on an email, a printed matter, and the product arranged in a real space as the type of the photographed subject, and generating subject estimation data indicating the estimation result of the type of the photographed subject; an output means for performing an output based on the target estimation data; An image processing device comprising:

2. 2. The image processing device according to claim 1, The estimation means generates the object estimation data by processing an area surrounding the product in the image.

3. 3. The image processing device according to claim 1, the image acquisition means acquires a moving image as the image, The estimation means generates the target estimation data by processing a plurality of frame images included in the moving image.

4. 4. The image processing device according to claim 1, the image is generated by a terminal; The image acquisition means acquires the images from a plurality of the terminals, The estimation means generates the target estimation data for each of the plurality of terminals; The output means outputs first association data indicating an association between attribute information of users of the plurality of terminals and the target estimation data.

5. 5. The image processing device according to claim 1, The estimation means further processes the image to estimate the product and / or a similar product to the product.

6. 6. The image processing device according to claim 5, The system further includes a purchase result acquisition unit for acquiring purchase result information indicating whether the product and / or the similar product has been purchased, The output means outputs second association data indicating an association between the object estimation data and the purchase result information.

7. The computer An image acquisition process for acquiring an image of a photographed subject, the image including a product in a portion of the image; an estimation process for estimating, by processing the image, at least one of a screen displayed on the display based on a broadcast, a screen displayed on the display based on a social networking service (SNS), a screen displayed on the display based on an email, a printed matter, and the product arranged in real space as the type of the photographed subject, and generating subject estimation data indicating the estimation result of the type of the photographed subject; An output process for performing an output based on the target estimation data; An image processing method.

8. On the computer, An image acquisition process for acquiring an image of a photographed subject, the image including a product in a portion of the image; an estimation process for estimating, by processing the image, at least one of a screen displayed on the display based on a broadcast, a screen displayed on the display based on a social networking service (SNS), a screen displayed on the display based on an email, a printed matter, and the product arranged in real space as the type of the photographed subject, and generating subject estimation data indicating the estimation result of the type of the photographed subject; An output process for performing an output based on the target estimation data; A program that performs the following.

Images