Image management apparatus and control method

Abstract

An image management apparatus (2000) acquires an object image (10) acquired by imaging an object (12). The image management apparatus (2000) determines a main image (24) having a high degree of similarity to the object image (10) from among main information (20). The image management apparatus (2000) compares the suitability of the object image (10) as a main image (similar main image) between the object image (10) and the determined main image (24), and updates the main information (20) based on the comparison result. Specifically, when the object image (10) has higher suitability, the image management apparatus (2000) adds the object image (10) to the main information (20) as a main image. Further, in this case, the image management apparatus (2000) deletes the similar main image from the main information (20), or assigns a higher priority to the object image (10) than a priority assigned to the similar main image.

Description

TECHNICAL FIELD

[0001] The present application relates to management of images used for object recognition. BACKGROUND

[0002] A technique for recognizing a specific object from a captured image has been developed. For example, PTL 1 discloses a technique for recognizing a product and a price tag from a captured image that captures a display site.

[0003] In this context, an image of a specific object (hereinafter referred to as a master image) is used as information for recognizing an object from a captured image. PTL 2 describes generating a master image used for detecting an object from an image including the object.

[0004] Related Literature

[0005] Patent Literature

[0006] [PTL 1] International Patent Publication No. WO2016 / 052383

[0007] [PTL 2] Japanese Patent Application Publication No. 2004-127157 SUMMARY

[0008] Technical Problem

[0009] In order to improve recognition accuracy of an object in image processing, it is preferable to prepare a plurality of master images for one object. For example, when an appearance of an object differs depending on a viewing angle, a master image is prepared for each of a plurality of angles.

[0010] However, simply preparing a large number of master images requires a large amount of time to compare an object included in a captured image with the master images, and efficiency of object recognition is reduced.

[0011] The present application is made in view of the above problems, and one of the objects of the present application is to provide a technique for appropriately managing master images used for object recognition.

[0012] [Solution to the Problem]

[0013] An image management apparatus according to the present application includes: 1) an acquisition unit that acquires an object image acquired by capturing an image of an object; 2) a determination unit that determines a master image having a high degree of similarity to the object image from among master information including one or more master images; and 3) an update unit that compares appropriateness of the master image as a master image between the object image and the determined master image, and updates the master information based on a result of the comparison.

[0014] When the object image has higher suitability than the determined main image, the update unit 1) adds the object image to the main information as a main image, and 2) deletes the determined main image from the main information, or assigns a higher priority to the object image than the priority assigned to the determined main image.

[0015] A control method according to the present application is executed by a computer. The control method includes: 1) an acquisition step of acquiring an object image acquired by imaging an object; 2) a determination step of determining a main image having high similarity to the object image from among main information including one or more main images; and 3) an update step of comparing suitability as a main image between the object image and the determined main image, and updating the main information based on a result of the comparison.

[0016] In the update step, when the object image has higher suitability than the determined main image, 1) the object image is added to the main information as a main image, and 2) the determined main image is deleted from the main information, or a higher priority is assigned to the object image than the priority assigned to the determined main image.

[0017] A program of the present application causes a computer to execute each step of the control method according to the present application.

[0018] Advantageous effects of the invention

[0019] According to the present application, there is provided a technique for appropriately managing main images used for object recognition. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a conceptual diagram illustrating an operation of an image management apparatus according to a first example embodiment.

[0021] Figure 2 is a diagram illustrating a functional configuration of the image management apparatus.

[0022] Figure 3 is a diagram illustrating a computer for implementing the image management apparatus.

[0023] Figure 4 is a flowchart illustrating a processing flow executed by the image management apparatus of the first example embodiment.

[0024] Figure 5 illustrates a case where similarity between an object image and a main image is high.

[0025] Figure 6 illustrates a case where similarity between an object image and a main image is low. DETAILED DESCRIPTION

[0026] Example embodiments of the present application will be described below with reference to the accompanying drawings. Note that like components are designated by like reference numerals throughout all the drawings, and a description thereof will be appropriately omitted. Further, unless otherwise specified, each block in each block diagram represents a functional unit component, not a hardware unit component. In the following description, unless otherwise specified, various predetermined values (threshold values, etc.) are stored in advance in storage devices that are accessible from the functional component units that use these values.

[0027] [First Example Embodiment]

[0028] <SUMMARY>

[0029] Figure 1 is a conceptual diagram illustrating the operation of the image management apparatus 2000 according to the first example embodiment. In this context, the operation of the image management apparatus 2000 described below is an example for facilitating understanding of the image management apparatus 2000, and does not limit the operation of the image management apparatus 2000. Details and variations of the operation of the image management apparatus 2000 will be described later. Figure 1 The operation of the image management apparatus 2000 described below is an example for facilitating understanding of the image management apparatus 2000, and does not limit the operation of the image management apparatus 2000. Details and variations of the operation of the image management apparatus 2000 will be described later.

[0030] The image management apparatus 2000 manages master information 20. The master information 20 is used for object recognition processing by image processing. The object recognition processing herein refers to processing that determines an object included in an image. For example, in a case where identification information is pre-assigned to each known object, the object recognition processing determines the identification information of an object included in an image.

[0031] The master information 20 associates object identification information 22 with a master image 24. The object identification information 22 is identification information of an object, and the master image 24 is an image of an object determined by the identification information. The number of master images 24 associated with the object identification information 22 can be one or more. In the object recognition processing, the master information 20 indicating the master image 24 having a high degree of similarity to an image of an object to be recognized is determined from among a plurality of master information 20. Then, the object identification information 22 indicated by the master information 20 determined herein is determined as the identification information of the object to be recognized.

[0032] The image management apparatus 2000 performs processing of registering a new master image in the master information 20. To this end, first, the image management apparatus 2000 acquires a target image 10 that is a candidate for an image to be added to the master information 20. The target image 10 is an image including a target object 12.

[0033] The image management apparatus 2000 determines, from among the master images 24 indicated by the master information 20 regarding the target object 12, a master image 24 having a degree of similarity to the target image 10 equal to or greater than a threshold value. Hereinafter, the master image 24 determined herein is referred to as a similar master image.

[0034] The image management apparatus 2000 determines which one of the similar master image and the object image 10 has higher suitability as a master image. The "higher suitability as a master image" herein means higher suitability as a master image used for the object recognition processing. For example, an image having higher resolution, an image in which more key points are detected, and the like are treated as an image having higher suitability as a master image.

[0035] When it is determined that the object image 10 has higher suitability as a master image, the image management apparatus 2000 updates the master information 20. Specifically, the image management apparatus 2000 adds the object image 10 to the master information 20 as a new master image 24. Further, the image management apparatus 2000 1) deletes the similar master image from the master information 20, or 2) assigns a higher priority to the object image 10 than the priority assigned to the similar master image.

[0036] In the case of 1), when the object image 10 has higher suitability as a master image, the similar master image is deleted from the master information 20, while the object image 10 is registered in the master information 20 as a master image 24. Thus, the master information 20 includes only an image having higher suitability as a master image among the object image 10 and the similar master image. In this way, the object recognition processing can be efficiently performed using the master information 20. Further, since this allows the number of master images to be suppressed from being excessive, the capacity of a storage apparatus for storing the master information 20 can also be reduced.

[0037] In the case of 2), it is assumed that a priority is assigned to each master image 24 in the master information 20. The priority indicates a usage priority in the object recognition processing. That is, a master image 24 assigned with a higher priority is used more preferentially in the object recognition processing using the master image 24. Thus, when it is determined that the object image 10 has higher suitability as a master image, the image management apparatus 2000 causes the object image 10 to be used preferentially to the similar master image in the object recognition processing, while the image management apparatus 2000 registers the object image 10 as a new master image 24. In this way, the object recognition processing can be efficiently performed using the master information 20.

[0038] The image management apparatus 2000 will be described in more detail below.

[0039] <Example of Functional Configuration>

[0040] Figure 2is a diagram illustrating a functional configuration of the image management apparatus 2000. The image management apparatus 2000 includes an acquisition unit 2020, a determination unit 2040, and an update unit 2060. The acquisition unit 2020 acquires the target image 10. The determination unit 2040 determines, from among the master images 24 included in the master information 20, a master image 24 (a similar master image) whose similarity to the target image 10 is equal to or greater than a threshold value. The update unit 2060 determines which of the similar master image and the target image 10 has higher suitability as a master image. The update unit 2060 updates the master information 20.

[0041] When it is determined that the similar master image has higher suitability as a master image than the target image 10, the update unit 2060 adds the target image 10 to the master information 20 as a master image 24, and performs 1) a process of deleting the similar master image from the master information 20, or 2) a process of assigning a higher priority to the target image 10 than a priority assigned to the similar master image.

[0042] <Example of hardware configuration of image management apparatus 2000>

[0043] Each functional component unit of the image management apparatus 2000 can be implemented by hardware (example: hardwired electronic circuitry or the like) that implements each functional component unit, or can be implemented by a combination of hardware and software (example: a combination of an electronic circuit and a program that controls the electronic circuit, or the like). The case where each functional component unit of the image management apparatus 2000 is implemented by a combination of hardware and software will be further described below.

[0044] Figure 3 is a diagram illustrating a computer 1000 for implementing the image management apparatus 2000. The computer 1000 is any computer. For example, the computer 1000 is a stationary computer such as a personal computer (PC) or a server machine. In addition, for example, the computer 1000 is a portable computer such as a smartphone or a tablet terminal.

[0045] The computer 1000 can be a special-purpose computer designed for implementing the image management apparatus 2000, or can be a general-purpose computer. In the latter case, each function of the image management apparatus 2000 is implemented by the computer 1000, for example, by installing a predetermined application on the computer 1000. The above-described application is configured by a program for implementing the functional component units of the image management apparatus 2000.

[0046] The computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path that allows the processor 1040, the memory 1060, the storage 1080, the input / output interface 1100, and the network interface 1120 to transmit and receive data to and from each other. However, the method of connecting the processor 1040 and the like to each other is not limited to the bus connection.

[0047] The processor 1040 is various processors such as a central processing unit (CPU), a graphics processing unit (GPU), and a field-programmable gate array (FPGA). The memory 1060 is a main storage device realized by using a random access memory (RAM) and the like. The storage 1080 is a secondary storage device realized by using a hard disk, a solid state drive (SSD), a memory card, a read only memory (ROM), and the like.

[0048] The input / output interface 1100 is an interface for connecting the computer 1000 and an input / output device. For example, an input device such as a keyboard and an output device such as a display device are connected to the input / output interface 1100.

[0049] The network interface 1120 is an interface for connecting the computer 1000 to a communication network. The communication network is, for example, a local area network (LAN) or a wide area network (WAN).

[0050] The storage 1080 stores a program (a program realizing the above-described application) that realizes each functional component unit of the image management apparatus 2000. The processor 1040 realizes each functional component unit of the image management apparatus 2000 by reading the program into the memory 1060 and executing the program.

[0051] <Process flow>

[0052] Figure 4 is a flowchart illustrating a process flow executed by the image management apparatus 2000 of the first example embodiment. The acquisition unit 2020 acquires the object image 10 (S102). The determination unit 2040 determines the master information 20 about the object object 12. The determination unit 2040 determines the master image 24 whose similarity to the object image 10 is equal to or greater than a threshold value (a similar master image) from among the master images 24 included in the master information 20 (S106). The update unit 2060 determines which of the similar master image and the object image 10 has higher suitability as a master image (S108).

[0053] When it is determined that the object image 10 has higher suitability as a master image (S108: the object image 10), the update unit 2060 updates the master information 20 (S110).

[0054] The processing performed by the image management apparatus 2000 is not limited to Figure 4 the illustrated processing. For example, even when it is determined that the target image 10 has higher suitability as a master image than a similar master image (S108: similar master image), the update unit 2060 can add the target image 10 to the master information 20 as a master image. However, in this case, the update unit 2060 sets the target image 10 to have a lower priority than the priority of the similar master image.

[0055] <About the target image 10>

[0056] The target image 10 is an image of an object acquired as a result of an image being captured by a camera. For example, a user of the image management apparatus 2000 acquires the target image 10 by capturing an image of an object using a camera provided in a mobile terminal. For example, when the object is a product, an image including the product can be acquired by capturing an image of a display site where the product is placed. Note that the target image 10 is not limited to an image generated by a camera, and can be an image that has been variously processed (size change, tone correction, and the like) after being generated by a camera.

[0057] In this document, the target image 10 is an image including only one object (for example, an image of an outer rectangle and the inside of the object). On the other hand, a captured image acquired by capturing an image of an object with a camera can include a plurality of objects, or can broadly include one object and its background. For example, as described above, when an image of a display site of a product is captured, it is considered that a captured image acquired from a camera generally includes a plurality of products. Therefore, the target image 10 is generated by extracting an image region of one object from a captured image. In other words, the processing of extracting the target image 10 from a captured image is performed.

[0058] The extraction of the target image 10 can be performed by a user using an image editing application or the like, or can be performed automatically by an apparatus (hereinafter, an object detection apparatus). In the latter case, the object detection apparatus acquires a captured image, and performs processing of detecting an image region indicating an object on the captured image (object detection processing). Thus, an image region indicating an object (object region) is detected from a captured image. The object detection apparatus outputs the image region indicating the detected object as the target image 10.

[0059] Any method is used as a method by which the object detection apparatus acquires a captured image. For example, the object detection apparatus acquires a captured image transmitted from a terminal operated by a user (hereinafter, a user terminal). In addition, for example, a user can input a captured image to the object detection apparatus by directly operating the object detection apparatus.

[0060] Note that the computer that implements the object detection apparatus can be the same as or different from the computer that implements the image management apparatus 2000. The former case corresponds to providing the image management apparatus 2000 with a function of extracting the subject image 10 from the captured image.

[0061] <Acquisition of subject image 10: S102>

[0062] The acquisition unit 2020 acquires the subject image 10 (S102). Any method is used as the method by which the acquisition unit 2020 acquires the subject image 10. For example, assume that the image management apparatus 2000 performs a series of processes in accordance with a direct operation by a user. In this case, the acquisition unit 2020 accepts a user operation that inputs the subject image 10. At this time, the acquisition unit 2020 can also accept input of identification information that specifies the object object 12.

[0063] In addition, for example, assume that the image management apparatus 2000 receives a request from a user terminal and performs a series of processes in accordance with the request. In this case, for example, the acquisition unit 2020 receives a request including the subject image 10 from the user terminal and acquires the subject image 10 included in the request. Note that at this time, the request can also indicate the identification information of the object object 12.

[0064] In addition, for example, as described above, assume that the subject image 10 is extracted from a captured image by an object detection apparatus. In this case, the acquisition unit 2020 acquires the subject image 10 extracted by the object detection apparatus. At this time, the acquisition unit 2020 can acquire only a part of the subject image 10, instead of the entire subject image 10 extracted by the object detection apparatus.

[0065] For example, the acquisition unit 2020 acquires only the subject image 10 specified by a user from the subject image 10 extracted by the object detection apparatus. In this case, for example, the object detection apparatus outputs the detected object regions in a visible manner and causes the user to select the object regions. For example, the object detection apparatus outputs a captured image to which a frame or the like is attached to each of the object regions to the user terminal. The user specifies one or more object regions intended to be used as the subject image 10 by operating the user terminal. The object detection apparatus outputs the object regions specified by the user as the subject image 10.

[0066] In this document, in addition to detecting objects, the object detection apparatus can also perform recognition of the objects (determination of identification information of the objects). In this case, the acquisition unit 2020 acquires a combination of the subject image 10 and the identification information of the object object 12 from the object detection apparatus.

[0067] <Determination of main information 20 on object object 12: S104>

[0068] The determination unit 2040 determines the master information 20 regarding the object body 12 (S104). When the identification information of the object body 12 is acquired together with the object image 10, the determination unit 2040 determines the master information 20 indicating the acquired identification information of the object body 12 as the master information 20 regarding the object body 12.

[0069] On the other hand, when the identification information of the object body 12 is not acquired, the determination unit 2040 determines the identification information of the object body 12 by performing the body recognition process on the object image 10. Then, the determination unit 2040 determines the master information 20 indicating the determined identification information of the object body 12 as the master information 20 regarding the object body 12.

[0070] However, instead of performing the body recognition process by itself, the determination unit 2040 can request another device (for example, the above-described body detection device) to perform the body recognition process and acquire the processing result (that is, the identification information of the object body 12).

[0071] Here, when the identification information of the object body 12 is determined by the body recognition process, the determination result can be presented to the user in such a manner that the user checks whether there is an error. In this way, in the configuration in which the device automatically recognizes the object body 12, it is possible to prevent an association error between the identification information and the master image. Note that when the identification result has an error, the user is allowed to manually input the identification information of the object body 12. Then, the determination unit 2040 determines the master information 20 indicating the identification information to be input by the user as the object identification information 22 as the master information 20 regarding the object body 12.

[0072] <Determination of a similar master image: S106>

[0073] The determination unit 2040 determines the master image 24 having a high degree of similarity to the object image 10 from among the master images 24 included in the master information 20 regarding the object body 12 (S106). The master image 24 determined here is treated as a similar master image.

[0074] For example, the determination unit 2040 calculates the degree of similarity between the object image 10 and each of the master images 24 included in the master information 20 regarding the object body 12, and determines whether the calculated degree of similarity is equal to or greater than a threshold value. When the degree of similarity between a certain master image 24 and the object image 10 is equal to or greater than the threshold value, the determination unit 2040 determines the master image 24 as the master image 24 having a high degree of similarity to the object image 10.

[0075] Various indexes can be used as an index of the similarity of the object image 10 and the master image 24. For example, the determination unit 2040 determines matching key points between the object image 10 and the master image 24, and determines the similarity between the object image 10 and the master image 24 based on the positional difference of the matching key points. Figure 5 and 6 are graphs for illustrating the similarity based on the positional difference of the key points. Figure 5 A case where the similarity between the object image 10 and the master image 24 is high is illustrated. On the other hand, Figure 6 A case where the similarity between the object image 10 and the master image 24 is low is illustrated. Note that, in order to improve the visibility of the drawing, the object is expressed by Figure 5 and 6 dotted lines.

[0076] In Figure 5 and 6 , the key points A to E are detected from the object image 10. The key points A to E are also detected from the master image 24. The key points having the same symbol in the object image 10 and the master image 24 indicate the same key point. For example, the key point A detected from the object image 10 and the key point A detected from the master image 24 indicate the same key point. Note that, the related art can be used as a technique of detecting the same key point in two different images.

[0077] The determination unit 2040 calculates the positional difference between the key point A in the object image 10 and the key point A in the master image 24. For example, the determination unit 2040 calculates the distance between the coordinates of the key point A in the object image 10 and the coordinates of the key point A in the master image 24 as a value indicating the difference between these positions. It can be said that the longer the distance between the same key points, the greater the positional difference of the same key points. Note that, when the sizes of the object image 10 and the master image 24 are different, it is preferable to enlarge or reduce one or both of the images in such a way as to match these sizes, and then calculate the distance.

[0078] The determination unit 2040 also calculates the positional difference between the other matching key points, and calculates the similarity between the object image 10 and the master image 24 based on the calculated positional differences. For example, the similarity is calculated based on the following equation (1).

[0079] [Equation 1]

[0080]

[0081] where s denotes the similarity. Di denotes the positional difference of the key point i detected in each of the object image 10 and the master image 24. m denotes the number of matching key points between the object image 10 and the master image 24. According to equation (1), the smaller the sum of the positional differences, the higher the similarity s.

[0082] In Figure 5 , the same key points are close to each other in the object image 10 and the master image 24. Therefore, in equation (1), the sum of the position differences becomes small, and the similarity s becomes large. On the other hand, in Figure 6 , the key point C and the key point E are close to each other to some extent, but the other key points are far from each other. Therefore, in equation (1), the sum of the position differences becomes large, and the similarity s becomes small.

[0083] Note that, in addition to the position difference between the matching key points, the number of matching key points can be further taken into account. For example, as indicated by the following equation (2), the value calculated by equation (1) is multiplied by a value f(m) that increases as the number of matching key points increases. Therefore, the more the number of matching key points, the higher the similarity. Any monotonic non-decreasing function can be used as the function f().

[0084] [Num 2]

[0085]

[0086] Note that the master information 20 can also indicate, for each master image 24, information about each key point detected from the master image 24 (coordinates of the key point, feature values of the key point, and the like). In this case, the determination unit 2040 can recognize the key points of each master image 24 by referring to the master information 20.

[0087] In addition, for example, the determination unit 2040 can use the same method for calculating the similarity between the image of the object to be recognized in the object recognition processing and the master image, in order to calculate the similarity between the object image 10 and the master image 24.

[0088] The calculation of the similarity can be performed by using a machine learning technique. Specifically, an estimation model is provided for estimating the similarity between two images from the input of the two images. The estimation model is pre-learned using a plurality of learning data configured by a combination of "two images and the similarity therebetween". Various types of estimation models such as neural networks and support vector machines (SVM) can be used. The determination unit 2040 can acquire the similarity between the object image 10 and the master image 24 by inputting the object image 10 and the master image 24 into the estimation model.

[0089] In addition, for example, a determination model for determining whether the similarity between two images is high or not from the input of the images can be provided. The determination model is pre-learned using a plurality of learning data configured by a combination of "two images and the output of the correct answer (information indicating whether the images are similar or not)". As the type of the determination model, as in the case of the estimation model type, various types can be used.

[0090] The determination unit 2040 inputs the object image 10 and the main image 24 into the determination model. Then, when the determination model determines that the similarity between these images is high, the determination unit 2040 determines the input main image 24 as the main image 24 having high similarity with the object image 10.

[0091] Herein, it is assumed that there are a plurality of main images 24 having high similarity with the object image 10. In this case, the determination unit 2040 can handle only one of the plurality of main images 24 as a similar main image, or handle each of the plurality of main images 24 as a similar main image.

[0092] In the former case, for example, the determination unit 2040 handles the main image 24 having the highest similarity with the object image 10 and the main image 24 having the lowest suitability index value as similar main images, as described later. On the other hand, in the latter case, for example, in such a manner that subsequent processing can be made, the determination unit 2040 handles the main images 24 in order one by one starting from the main image 24 having the highest similarity with the object image 10 as similar main images.

[0093] Herein, in the main information 20, the main images 24 can be classified into a plurality of groups. For example, the main images 24 are classified according to the orientation (up, right, down, left, etc. in a plan view) of the object included in the main image 24. In this case, the determination unit 2040 can determine the group to which the object image 10 belongs, and can use only the main images 24 included in the group to compare with the object image 10. For example, when the object object 12 is oriented to the right, the determination unit 2040 determines the image having high similarity with the object image 10 from among the main images 24 included in the right-oriented group. Note that the related art can be used as a technique for determining the orientation of the object included in the image.

[0094] Note that the grouping of the main images 24 is not limited to the group based on the orientation. For example, when the object is a person, it is assumed that the main images 24 included in the main information 20 about the same person are grouped according to whether there is all the object (such as the case of wearing glasses or not wearing glasses and the case of wearing a hat or not wearing a hat).

[0095] <Comparison of suitability as a main image: S108>

[0096] The update unit 2060 determines which one of the object image 10 and the similar master image has higher suitability as a master image (S108). To do so, for example, the update unit 2060 calculates a value of an index (suitability index) indicating the degree of suitability as a master image for each of the object image 10 and the similar master image, and compares the suitability index values. When the suitability index value of the object image 10 is larger than the suitability index value of the similar master image, the update unit 2060 determines that the object image 10 has higher suitability as a master image. On the other hand, when the suitability index value of the object image 10 is equal to or smaller than the suitability index value of the similar master image, the update unit 2060 determines that the similar master image has higher suitability as a master image.

[0097] Various indexes can be employed as the suitability index. For example, the suitability index is the resolution degree. For example, the higher the resolution, the higher the suitability index value. In this case, the resolution of the image can be used as it is the suitability index value of the image, or the suitability index value can be calculated by using a function that converts the resolution into the suitability index value. As the above function, for example, any monotonic non-decreasing function can be used. In addition, a conversion table that associates a plurality of numerical ranges of the resolution with suitability index values of images whose resolution belongs to the numerical range can be prepared, and the resolution can be converted into the suitability index value using the conversion table.

[0098] However, there can be a case where the resolution degree required for the master image is fixed and the master image whose resolution is higher than the required resolution is considerably disadvantageous. Therefore, for example, the suitability index value based on the resolution can be a value that "increases as the resolution increases until the resolution reaches a specific value, and decreases when the resolution is higher than the specific value". In this case, as the function that converts the resolution into the suitability index value, a convex function having the above specific value as the maximum value can be used. In this case, the above conversion table can also be used.

[0099] As another example of the suitability index, for example, the number of key points can be used. That is, the larger the number of key points of the image, the higher the suitability index value. In this case, the number of key points acquired from the image can be used as it is the suitability index value of the image, or the suitability index value can be calculated by using a function that converts the number of key points into the suitability index value. As the above function, for example, any monotonic non-decreasing function can be used. In addition, a conversion table that associates a plurality of numerical ranges of the number of key points with suitability index values of images whose number of key points belongs to the numerical range can be prepared, and the number of key points can be converted into the suitability index value using the conversion table.

[0100] The calculation of the suitability index value can be implemented by using a machine learning technique. For example, an estimation model for estimating a suitability index value of an image from an input of the image is provided. The estimation model is pre-learned using a plurality of learning data represented by a combination of "an image and a suitability index value". As the type of the estimation model, various types of estimation models can be used as in the case of the type of the estimation model for estimating the above-described similarity. The update unit 2060 can acquire the suitability index value of each of the object image 10 and the similar master image by inputting each of the object image 10 and the similar master image into the estimation model.

[0101] In this context, the master information 20 can further include a suitability index value of the master image 24 associated with the master image 24. In this case, the update unit 2060 can acquire the suitability index value of the similar master image from the master information 20. Note that, in this case, when the update unit 2060 adds the object image 10 to the master information 20 as the master image 24, the suitability index value calculated by the update unit 2060 is preferably added together with the object image 10.

[0102] The update unit 2060 can not calculate the suitability index value. For example, a determination model for determining which image has a higher suitability as a master image from an input of two images is provided. The determination model is pre-learned using a plurality of learning data that is a combination of "two images and an output of a correct answer (information indicating which image has a higher suitability as a master image)". As the type of the determination model, various types can be used as in the case of the estimation model type. The update unit 2060 can determine which of the object image 10 and the similar master image has a higher suitability as a master image by inputting the object image 10 and the similar master image into the determination model.

[0103] <Update the master information 20: S110>

[0104] When it is determined that the similar master image has a higher suitability as a master image than the object image 10 (S108: the object image 10), the update unit 2060 updates the master information 20 including the similar master image (S110). The update unit 2060 at least adds the object image 10 to the master information 20 as one of the master images 24.

[0105] Further, as described above, the update unit 2060 performs one of the following two processes.

[0106] 1) Delete the similar master image

[0107] 2) Assign a higher priority to the object image 10 than the similar master image

[0108] Whether the update unit 2060 updates the master information 20 by 1) or 2) can be determined in advance or can be selected by a user.

[0109] Various methods can be considered to determine the priority to be assigned to the object image 10. For example, the update unit 2060 uses the above-described suitability index value as the priority of the master image 24. In this way, the master image 24 having a higher degree of suitability as a master image has a higher priority.

[0110] In addition, for example, the plurality of master images 24 included in the master information 20 regarding the object object 12 can be ranked according to the degree of suitability as a master image, and the order can be used as the priority. In this case, the higher the order, the higher the priority. Hereinafter, the order assigned according to the degree of suitability as a master image is also referred to as a priority order.

[0111] Suppose that the priority order of the similar master image is the highest in the master information 20. In this case, the update unit 2060 assigns the highest priority order to the object image 10. On the other hand, when the master image 24 has a higher priority order than the similar master image in the master information 20 regarding the object object 12, the update unit 2060 determines the priority order of the object image 10 by comparing the suitability as a master image among the master images 24 having a higher priority order than the similar master image and the object image 10. The process of comparing the suitability as a master image can be implemented by using the function of the update unit 2060.

[0112] Herein, as described above, the master information 20 can group the master images 24. In this case, the priority of the master image 24 can be determined by the relative order within the group, rather than the relative order among all the master images 24 included in the master information 20 regarding the object object 12. That is, the master image 24 is assigned a priority order within the group. Therefore, the update unit 2060 assigns the object object 12 the priority indicated by the priority order in the group to which the object object 12 is added.

[0113]

[0114] An upper limit can be set to the number of master images 24 that can be included in the master information 20. In this case, when the number of master images 24 does not exceed the upper limit value, the update unit 2060 performs the process of 2), and when the number of master images 24 reaches the upper limit value, the process of 1) is performed.

[0115] ​Specifically, when the object image 10 is determined to have higher suitability as a master image, the update unit 2060 first determines whether the number of master images 24 included in the master information 20 is less than an upper limit value. When the number of master images 24 is less than the upper limit value, since the number of master images 24 is equal to or less than the upper limit value even when the object image 10 is added to the master information 20, the update unit 2060 does not delete a similar master image, assigns a higher priority to the object image 10 than a priority of the similar master image, and adds the object image 10 to the master information 20. On the other hand, when the number of master images 24 is equal to or greater than the upper limit value, because the number of master images 24 exceeds the upper limit value when the object image 10 is added to the master information 20, the update unit 2060 deletes a similar master image from the master information 20 in such a way that the number of master images 24 does not exceed the upper limit value.

[0116] Note that the upper limit value of the number of master images 24 can be set for each of the above-described groups. In this case, when the number of master images 24 included in a group exceeds the upper limit value, the update unit 2060 deletes a similar master image by adding the object image 10 to the group. On the other hand, even when the object image 10 is added to the group, the number of master images 24 included in the group does not exceed the upper limit value, the update unit 2060 does not delete a similar master image, assigns a priority greater than a priority of the similar master image to the object image 10, and adds the object image 10 to the master information 20.

[0117] <Another application example of the master information 20>

[0118] The master information 20 can be used for other purposes than the object recognition process. For example, when a product is disposed of as an object, the master images 24 included in the master information 20 can be used as images of the product referred to during thought analysis of the product. In the thought analysis of the product, since an analysis of recognizing a user's demand from a feature of a product that the user likes to purchase is performed, an image of the product in which the feature is easily recognized is necessary. In this regard, according to the image management apparatus 2000, an image of a product having higher suitability as a master image 24 is preferentially kept in the master information 20. As described above, an image having higher suitability as a master image 24 can be said to be an image in which a feature of a product is easily extracted, such as an image having high resolution or an image having a large number of key points. Therefore, an image suitable for thought analysis can be acquired from the master information 20 managed by the image management apparatus 2000.

[0119] Some or all of the above example embodiments can also be described in the following supplementary notes, but are not limited to the following.

[0120] 1. An image management apparatus comprising:

[0121] an acquisition unit that acquires an object image acquired by imaging an object;

[0122] a determination unit that determines a master image having a high degree of similarity to the object image from among master information including one or more master images; and

[0123] an update unit that compares the appropriateness of the object image as a master image with the determined master image and updates the master information based on the comparison result, wherein

[0124] when the object image has higher appropriateness than the determined master image, the update unit

[0125] adds the object image to the master information as a master image, and

[0126] deletes the determined master image from the master information or assigns a higher priority to the object image than a priority assigned to the determined master image.

[0127] 2. The image management apparatus according to Supplementary Note 1, wherein

[0128] when the number of key points included in the object image is greater than the number of key points included in the determined master image or when the resolution of the object image is higher than the resolution of the determined master image, the update unit determines that the object image has higher appropriateness.

[0129] 3. The image management apparatus according to Supplementary Note 1 or 2, wherein

[0130] the master information is used for object recognition processing for recognizing an object included in an image, and

[0131] the master image to which a higher priority is assigned is preferentially used in the object recognition processing.

[0132] 4. The image management apparatus according to any one of Supplementary Notes 1 to 3, wherein

[0133] the determination unit

[0134] determines master information indicating identification information of the object, and

[0135] determines a master image having a high degree of similarity to the acquired object image from among master images included in the determined master information.

[0136] 5. The image management apparatus according to any one of Supplementary Notes 1 to 4, wherein

[0137] The determination unit calculates a similarity between the main image and the object image based on a positional difference of the same key point detected from the main image and the object image, and determines the main image whose similarity is equal to or greater than a threshold value as a main image having a high similarity with the object image.

[0138] 6. The image management apparatus according to any one of Supplementary Note 1 to 5, wherein

[0139] The update unit calculates a fitness index value indicating a degree of fitness as a main image for the object image, and determines that the object image has a higher fitness when the fitness index value of the object image is greater than the fitness index value of the main image determined by the determination unit.

[0140] 7. A control method executed by a computer, comprising:

[0141] an acquisition step of acquiring an object image acquired by imaging an object object;

[0142] a determination step of determining a main image having a high similarity with the object image from among main information including one or more main images; and

[0143] an update step of comparing fitness as a main image between the object image and the determined main image, and updating the main information based on a result of the comparison, wherein

[0144] in the update step, when the object image has a higher fitness than the determined main image, the object image is added to the main information as a main image, and

[0145] the determined main image is deleted from the main information, or a priority higher than a priority assigned to the determined main image is assigned to the object image.

[0146] 8. The control method according to Supplementary Note 7, further comprising,

[0147] in the update step, when a number of key points included in the object image is greater than a number of key points included in the determined main image, or when a resolution of the object image is higher than a resolution of the determined main image, the object image is determined to have a higher fitness.

[0148] 9. The control method according to Supplementary Note 7 or 8, wherein

[0149] the main information is used for an object recognition process for recognizing an object included in an image, and

[0150] the main image to which a higher priority is assigned is preferentially used in the object recognition process.

[0151] 10. The control method according to any one of Supplementary Note 7 to 9, further comprising:

[0152] In the determining step,

[0153] determining main information indicating identification information of the object object; and

[0154] From among the main images included in the determined main information, determining a main image having a high degree of similarity with the acquired object image.

[0155] 11. The control method according to any one of Supplementary Note 7 to 10, further comprising,

[0156] In the determining step, based on a position difference of the same key point detected from the main image and the object image, a degree of similarity between the main image and the object image is calculated, and a main image whose degree of similarity is equal to or greater than a threshold value is determined as a main image having a high degree of similarity with the object image.

[0157] 12. The control method according to any one of Supplementary Note 7 to 11, further comprising,

[0158] In the updating step, a fitness index value indicating a degree of fitness as a main image is calculated with respect to the object image, and when the fitness index value of the object image is greater than the fitness index value of the main image determined at the determining step, the object image is determined to have a higher degree of fitness.

[0159] 13. A program causing a computer to execute the control method according to any one of Supplementary Note 7 to 12.

[0160] This application is based on Japanese Patent Application No. 2019-197069 filed October 30, 2019, and claims the benefit of priority thereto, the disclosure of which is incorporated herein by reference in its entirety.

[0161] List of Reference Signs

[0162] 10 object image

[0163] 12 object object

[0164] 20 main information

[0165] 22 object identification information

[0166] 24 main image

[0167] 26 priority

[0168] 1000 computer

[0169] 1020 bus

[0170] 1040 processor

[0171] 1060 memory

[0172] 1080 storage device

[0173] 1100 input / output interface

[0174] 1120 network interface

[0175] 2000 image management apparatus

[0176] 2020 acquisition unit

[0177] 2040 determination unit

[0178] 2060 update unit

Claims

1. An image management apparatus comprising: an acquisition unit that acquires an object image acquired by imaging an object; a determination unit that determines, from among master information including one or more master images, the master image having a high degree of similarity to the object image; and an update unit that compares the suitability as a master image between the object image and the determined master image, and updates the master information based on the comparison result, wherein when the object image has a higher suitability than the determined master image, the update unit adds the object image to the master information as a master image, and deletes the determined master image from the master information, or assigns a higher priority to the object image than the priority assigned to the determined master image, wherein the master information is used for object recognition processing for recognizing an object included in an image, and the master image assigned with a higher priority is preferentially used in the object recognition processing. 2.An image management apparatus comprising: an acquisition unit that acquires an object image acquired by imaging an object; a determination unit that determines, from among master information including one or more master images, the master image having a high degree of similarity to the object image; and an update unit that compares the suitability as a master image between the object image and the determined master image, and updates the master information based on the comparison result, wherein when the object image has a higher suitability than the determined master image, the update unit adds the object image to the master information as a master image, and deletes the determined master image from the master information, or assigns a higher priority to the object image than the priority assigned to the determined master image, wherein the determination unit calculates the degree of similarity between the master image and the object image based on the position difference of the same key point detected from the master image and the object image, and determines the master image whose degree of similarity is equal to or greater than a threshold value as the master image having a high degree of similarity to the object image. 3.An image management apparatus comprising: an acquisition unit that acquires an object image acquired by imaging an object; a determination unit that determines, from among master information including one or more master images, the master image having a high degree of similarity to the object image; and an update unit that compares the suitability as a master image between the object image and the determined master image, and updates the master information based on the comparison result, wherein when the object image has a higher suitability than the determined master image, the update unit adds the object image to the master information as a master image, and deletes the determined master image from the master information, or assigns a higher priority to the object image than the priority assigned to the determined master image, wherein The update unit calculates a fitness index value indicating a degree of fitness as a master image for the object image, and determines that the object image has a higher fitness as a master image when the fitness index value of the object image is greater than the fitness index value of the master image determined by the determination unit.

4. The image management apparatus according to any one of claims 1 to 3, wherein The update unit determines that the object image has the higher fitness when the number of key points included in the object image is greater than the number of key points included in the determined master image, or when the resolution of the object image is higher than the resolution of the determined master image.

5. The image management apparatus according to any one of claims 1 to 3, wherein The determination unit determines the master information indicating identification information of the object object, and determines the master image having a high degree of similarity to the acquired object image from among the master images included in the determined master information.

6. A control method executed by a computer, comprising: an acquisition step of acquiring an object image acquired by imaging an object object; a determination step of determining a master image having a high degree of similarity to the object image from among master information including one or more master images; and an update step of comparing fitness as a master image between the object image and the determined master image, and updating the master information based on a comparison result, wherein in the update step, when the object image has a higher fitness than the determined master image, the object image is added to the master information as a master image, and the determined master image is deleted from the master information, or a higher priority is assigned to the object image than a priority assigned to the determined master image, wherein the master information is used for object recognition processing for recognizing an object included in an image, and the master image to which a higher priority is assigned is preferentially used in the object recognition processing.

7. A control method executed by a computer, comprising: an acquisition step of acquiring an object image acquired by imaging an object object; a determination step of determining a master image having a high degree of similarity to the object image from among master information including one or more master images; and an update step of comparing fitness as a master image between the object image and the determined master image, and updating the master information based on a comparison result, wherein in the update step, when the object image has a higher fitness than the determined master image, the object image is added to the master information as a master image, and the determined master image is deleted from the master information, or a higher priority is assigned to the object image than a priority assigned to the determined master image, wherein In the determining step, similarity between the main image and the target image is calculated based on a position difference of the same key point detected from the main image and the target image, and the main image whose similarity is equal to or greater than a threshold value is determined as the main image having high similarity with the target image.

8. A control method executed by a computer, comprising: an acquisition step of acquiring a target image acquired by imaging a target object; a determining step of determining, from among main information including one or more main images, a main image having high similarity with the target image; an updating step of comparing, between the target image and the determined main image, suitability as a main image, and updating the main information based on a comparison result, wherein in the updating step, when the target image has higher suitability than the determined main image, the target image is added to the main information as a main image, and the determined main image is deleted from the main information, or a priority higher than a priority assigned to the determined main image is assigned to the target image, and in the updating step, a suitability index value indicating a height of suitability as a main image is calculated for the target image, and when the suitability index value of the target image is greater than a suitability index value of the main image determined in the determining step, the target image is determined to have higher suitability as a main image.

9. The control method according to any one of claims 6 to 8, further comprising, in the updating step, when a number of key points included in the target image is greater than a number of key points included in the determined main image, or when a resolution of the target image is higher than a resolution of the determined main image, the target image is determined to have the higher suitability.

10. The control method according to any one of claims 6 to 8, further comprising: in the determining step, the main information indicating identification information of the target object is determined; and from among the main images included in the determined main information, the main image having high similarity with the acquired target image is determined. ​ ​

Images