IMAGE MANAGEMENT DEVICE, CONTROL METHOD, AND PROGRAM

Abstract

An image management device (2000) acquires a subject image (10) obtained by imaging a subject (12). The image management device (2000) identifies, from among master information (20), a master image (24) having a high degree of similarity to the subject image (10). The image management device (2000) compares the suitability, as a master image, of the subject image (10) and the identified master image (24) (similar master image) and updates the master information (20) on the basis of the comparison result. Specifically, when the subject image (10) has higher suitability, the management device (2000) adds the subject image (10) to the master information (20) as a master image. Furthermore, in this case, the image management device (2000) deletes the similar master image from the master information (20) or assigns to the subject image (10) a higher priority than the priority assigned to the similar master image.

Description

[Technical field]

[0001] The present invention relates to management of images used for object recognition. [Background technology]

[0002] Technologies for recognizing specific objects from captured images have been developed. For example, Patent Document 1 discloses a technology for recognizing products and price tags from captured images of display locations.

[0003] Here, an image of a specific object (hereinafter, a master image) is used as information for recognizing the object from a captured image. Patent Document 2 describes generating a master image used for detecting an object from an image including the object. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2016 / 052383 [Patent Document 2] JP 2004-127157 A Summary of the Invention [Problem to be solved by the invention]

[0005] In order to improve the accuracy of object recognition through image processing, it is preferable to prepare multiple master images for one object. For example, if the appearance of an object changes depending on the viewing angle, master images are prepared for each of multiple viewing angles.

[0006] However, simply preparing a large number of master images requires a lot of time to compare objects included in captured images with the master images, resulting in poor efficiency in object recognition.

[0007] The present invention has been made in view of the above-mentioned problems, and one of its objects is to provide a technique for appropriately managing master images used for object recognition. [Means for solving the problem]

[0008] The image management device of the present invention includes: 1) an acquisition unit that acquires a target image obtained by capturing an image of a target object; and 2) an object recognition process that recognizes an object included in the image. Leave To be used An image, which is an image of the object in question. From master information that contains one or more master images a specifying unit that specifies the master image having a similarity to the target image equal to or greater than a threshold; and 3) the A match between the specified master image and the target image is determined. Aptitude index value that represents the height of and an update unit that updates the master information based on the comparison result. . Change The new section shows that the target image is more accurate than the specified master image. The above Aptitude Index value but big In this case, 1) the target image is added to the master information as a master image, and 2) The above A higher priority is assigned to the target image than the priority assigned to the identified master image. The higher the priority level assigned to the master image, the higher the priority level is used in the object recognition process. The suitability index value is a value based on the resolution of the target image and the specified master image, or a value based on the number of feature points included in the target image and the specified master image.

[0009] The control method of the present invention is executed by a computer. The control method includes: 1) an acquisition step of acquiring a target image obtained by capturing an image of a target object; and 2) an object recognition process of recognizing an object included in the image. Leave To be used An image, which is an image of the object in question. From master information that contains one or more master images a step of identifying the master image having a similarity to the target image equal to or greater than a threshold; and 3) the step of identifying the master image having a similarity to the target image equal to or greater than a threshold. A match between the specified master image and the target image is determined. Aptitude index value that represents the height of and updating the master information based on the comparison result. . Change In the new step, if the target image is more suitable than the identified master image, 1) the target image is added to the master information as a master image, and 2) The above A higher priority is assigned to the target image than the priority assigned to the identified master image. The higher the priority level assigned to the master image, the higher the priority level is used in the object recognition process. The suitability index value is a value based on the resolution of the target image and the specified master image, or a value based on the number of feature points included in the target image and the specified master image.

[0010] The program of the present invention causes a computer to execute each step of the control method of the present invention. Effect of the Invention

[0011] According to the present invention, a technique for appropriately managing master images used for object recognition is provided. [Brief description of the drawings]

[0012] [Figure 1] 3 is a diagram conceptually illustrating an example of the operation of the image management device according to the first embodiment. FIG. [Diagram 2] FIG. 2 is a diagram illustrating an example of a functional configuration of an image management device. [Diagram 3] FIG. 1 is a diagram illustrating an example of a computer for implementing an image management device. [Figure 4] 4 is a flowchart illustrating a flow of a process executed by the image management device of the first embodiment. [Diagram 5] This represents a case where the target image and the master image have a high degree of similarity. [Figure 6] This represents a case where the similarity between the target image and the master image is low. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In all drawings, similar components are given similar reference numerals, and descriptions thereof will be omitted as appropriate. In addition, unless otherwise specified, in each block diagram, each block represents a configuration in functional units, not a configuration in hardware units. In the following description, unless otherwise specified, various predetermined values ​​(such as threshold values) are stored in advance in a storage device accessible from the functional configuration unit that uses the values.

[0014] [Embodiment 1] <Summary> Fig. 1 is a diagram conceptually illustrating the operation of an image management device 2000 according to the first embodiment. Here, the operation of the image management device 2000 explained using Fig. 1 is an example for making it easier to understand the image management device 2000, and does not limit the operation of the image management device 2000. Details and variations of the operation of the image management device 2000 will be described later.

[0015] The image management device 2000 manages master information 20. The master information 20 is used for object recognition processing by image processing. The object recognition processing here refers to processing for identifying an object included in an image. For example, in a situation where identification information is given in advance to each known object, the identification information of the object included in the image is identified by the object recognition processing.

[0016] The master information 20 associates object identification information 22 with a master image 24. The object identification information 22 is identification information of an object, and the master image 24 is an image of the object identified by the identification information. The number of master images 24 associated with the object identification information 22 may be one or more. In the object recognition process, a piece of master information 20 that indicates a master image 24 having a high similarity to an image of an object to be recognized is identified from the plurality of pieces of master information 20. Then, the object identification information 22 indicated by the master information 20 identified here is identified as the identification information of the object to be recognized.

[0017] The image management device 2000 performs a process of registering a new master image to the master information 20. To this end, the image management device 2000 first acquires a target image 10 that is a candidate for an image to be added to the master information 20. The target image 10 is an image that includes a target object 12.

[0018] The image management device 2000 identifies the master images 24 indicated by the master information 20 about the target object 12, whose similarity to the target image 10 is equal to or greater than a threshold value. Hereinafter, the master images 24 identified here will be referred to as similar master images.

[0019] The image management device 2000 judges whether the similar master image or the target image 10 is more suitable as a master image. Here, "more suitable as a master image" means more suitable as a master image to be used in object recognition processing. For example, an image with a higher resolution or an image with more detectable feature points is treated as an image with higher suitability as a master image.

[0020] If it is determined that the target image 10 is more suitable as a master image, the image management device 2000 updates the master information 20. Specifically, the image management device 2000 adds the target image 10 to the master information 20 as a new master image 24. Furthermore, the image management device 2000 1) deletes the similar master image from the master information 20, or 2) gives the target image 10 a higher priority than the priority given to the similar master image.

[0021] In the case of 1), if the target image 10 is more suitable as a master image, the target image 10 is registered in the master information 20 as the master image 24, while the similar master image is deleted from the master information 20. Therefore, the master information 20 includes only the image that is more suitable as a master image out of the target image 10 and the similar master image. In this way, the object recognition process using the master information 20 can be performed efficiently. Also, since the number of master images can be prevented from increasing too much, the capacity of the storage device that stores the master information 20 can be reduced.

[0022] In the case of 2), it is assumed that a priority is assigned to each master image 24 in the master information 20. The priority indicates the priority when used in the object recognition process. In other words, the master image 24 to which a higher priority is assigned is used with higher priority in the object recognition process using the master image 24. Therefore, when it is determined that the target image 10 is more suitable as a master image, the image management device 2000 registers the target image 10 as a new master image 24, while giving priority to the use of the target image 10 over the similar master image in the object recognition process. In this way, the object recognition process using the master information 20 can be performed efficiently.

[0023] The image management device 2000 will be described in more detail below.

[0024] <Example of functional configuration> 2 is a diagram illustrating an example of the functional configuration of the image management device 2000. The image management device 2000 has an acquisition unit 2020, an identification unit 2040, and an update unit 2060. The acquisition unit 2020 acquires a target image 10. The identification unit 2040 identifies a master image 24 (similar master image) whose similarity to the target image 10 is equal to or greater than a threshold value from among the master images 24 included in the master information 20. The update unit 2060 determines which of the similar master image and the target image 10 is more suitable as a master image. The update unit 2060 updates the master information 20.

[0025] If it is determined that the target image 10 is more suitable as a master image than the similar master image, the update unit 2060 adds the target image 10 to the master information 20 as a master image 24, and also performs 1) a process of deleting the similar master image from the master information 20, or 2) a process of assigning a higher priority to the target image 10 than the priority assigned to the similar master image.

[0026] <Example of Hardware Configuration of Image Management Device 2000> Each functional component of image management device 2000 may be realized by hardware that realizes each functional component (e.g., hardwired electronic circuitry, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuitry and a program that controls it, etc.). Below, a further explanation is given of the case where each functional component of image management device 2000 is realized by a combination of hardware and software.

[0027] 3 is a diagram illustrating a computer 1000 for implementing the image management device 2000. The computer 1000 is any computer. For example, the computer 1000 is a stationary computer such as a PC (Personal Computer) or a server machine. Alternatively, the computer 1000 may be a portable computer such as a smartphone or a tablet terminal.

[0028] The computer 1000 may be a dedicated computer designed to realize the image management device 2000, or may be a general-purpose computer. In the latter case, for example, by installing a predetermined application on the computer 1000, each function of the image management device 2000 is realized by the computer 1000. The application is constituted by a program for realizing the functional components of the image management device 2000.

[0029] The computer 1000 has a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path for the processor 1040, the memory 1060, the storage device 1080, the input / output interface 1100, and the network interface 1120 to transmit and receive data to and from each other. However, the method of connecting the processor 1040 and the like to each other is not limited to bus connection.

[0030] The processor 1040 is one of various processors such as a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA). The memory 1060 is a main storage device realized using a random access memory (RAM) or the like. The storage device 1080 is an auxiliary storage device realized using a hard disk, a solid state drive (SSD), a memory card, or a read only memory (ROM) or the like.

[0031] The input / output interface 1100 is an interface for connecting the computer 1000 to an input / output device. For example, the input / output interface 1100 is connected to an input device such as a keyboard and an output device such as a display device.

[0032] The network interface 1120 is an interface for connecting the computer 1000 to a communication network. This communication network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network).

[0033] The storage device 1080 stores a program (a program that realizes the above-mentioned application) that realizes each functional component of the image management device 2000. The processor 1040 realizes each functional component of the image management device 2000 by reading this program into the memory 1060 and executing it.

[0034] <Processing flow> 4 is a flowchart illustrating a process executed by the image management device 2000 of the first embodiment. The acquisition unit 2020 acquires a target image 10 (S102). The identification unit 2040 identifies master information 20 related to the target object 12. The identification unit 2040 identifies a master image 24 (similar master image) having a similarity to the target image 10 equal to or greater than a threshold value from among the master images 24 included in the master information 20 (S106). The update unit 2060 determines whether the similar master image or the target image 10 is more suitable as a master image (S108).

[0035] If it is determined that the target image 10 is more suitable as a master image (S108: target image 10), the update unit 2060 updates the master information 20 (S110).

[0036] The process executed by the image management device 2000 is not limited to the process shown in Fig. 4. For example, even if it is determined that the similar master image is more suitable as a master image than the target image 10 (S108: similar master image), the update unit 2060 may add the target image 10 to the master information 20 as a master image. In this case, however, the update unit 2060 sets a lower priority for the target image 10 than the priority of the similar master image.

[0037] <About target image 10> The target image 10 is an image of an object obtained as a result of imaging by a camera. For example, a user of the image management device 2000 obtains the target image 10 by imaging the target object with a camera provided in a mobile terminal. For example, if the object is a product, an image including the product can be obtained by imaging the display location where the product is placed. Note that the target image 10 is not limited to only images generated by a camera, but may be an image generated by a camera that has been subjected to various processes (such as resizing and color correction).

[0038] Here, the target image 10 is an image that includes only one object (for example, a circumscribing rectangle of an object and an image of its interior). On the other hand, a captured image obtained by capturing an image of an object with a camera may include multiple objects, or may include one object and its background in a broader sense. For example, as described above, when capturing an image of a display area of ​​merchandise, the captured image obtained from the camera is likely to include multiple merchandise. Therefore, the target image 10 is generated by extracting an image area of ​​one object from the captured image. In other words, a process is performed to extract the target image 10 from the captured image.

[0039] The target image 10 may be extracted by a user using an image editing application or the like, or may be extracted automatically by a device (hereinafter, object detection device). In the latter case, the object detection device acquires a captured image and performs a process (object detection process) of detecting an image area representing an object from the captured image. As a result, an image area representing an object (object area) is detected from the captured image. The object detection device outputs the image area representing the detected object as the target image 10.

[0040] The method for the object detection device to acquire the captured image is arbitrary. For example, the object detection device acquires the captured image transmitted from a terminal operated by a user (hereinafter, referred to as a user terminal). Alternatively, for example, the user may directly operate the object detection device to input the captured image to the object detection device.

[0041] The computer that realizes the object detection device may be the same as or different from the computer that realizes the image management device 2000. The former case corresponds to providing the image management device 2000 with a function of extracting the target image 10 from the captured image.

[0042] <Acquisition of target image 10: S102> The acquisition unit 2020 acquires the target image 10 (S102). The method by which the acquisition unit 2020 acquires the target image 10 is arbitrary. For example, it is assumed that the image management device 2000 performs a series of processes in response to a direct operation by a user. In this case, the acquisition unit 2020 accepts a user operation to input the target image 10. At this time, the acquisition unit 2020 may further accept an input specifying identification information of the target object 12.

[0043] Alternatively, for example, the image management device 2000 receives a request from a user terminal and performs a series of processes in response to the request. In this case, for example, the acquisition unit 2020 receives a request including a target image 10 from the user terminal and acquires the target image 10 included in the request. Note that at this time, the request may further include identification information of the target object 12.

[0044] Alternatively, for example, as described above, it is assumed that the object detection device extracts the target image 10 from the captured image. In this case, the acquisition unit 2020 acquires the target image 10 extracted by the object detection device. At this time, the acquisition unit 2020 may acquire only some of the target images 10, rather than all of the target images 10 extracted by the object detection device.

[0045] For example, the acquisition unit 2020 acquires only the target image 10 specified by the user from among the target images 10 extracted by the object detection device. In this case, for example, the object detection device outputs the detected object regions in a visible manner and allows the user to select the object region. For example, the object detection device outputs a captured image in which each object region is given a frame or the like to a user terminal. The user operates the user terminal to specify one or more object regions to be used as the target image 10. The object detection device outputs the object region specified by the user as the target image 10.

[0046] Here, the object detection device may further recognize the object (specify the identification information of the object) in addition to detecting the object. In this case, the acquisition unit 2020 acquires a combination of the target image 10 and the identification information of the target object 12 from the object detection device.

[0047] <Identifying master information 20 related to target object 12: S104> The identification unit 2040 identifies the master information 20 related to the target object 12 (S104). When the identification information of the target object 12 is acquired together with the target image 10, the identification unit 2040 identifies the master information 20 indicating the acquired identification information of the target object 12 as the master information 20 related to the target object 12.

[0048] On the other hand, when the identification information of the target object 12 has not been acquired, the identification unit 2040 identifies the identification information of the target object 12 by performing an object recognition process on the target image 10. Then, the identification unit 2040 identifies the master information 20 indicating the identified identification information of the target object 12 as the master information 20 related to the target object 12.

[0049] However, instead of performing the object recognition process by itself, the identification unit 2040 may request another device (for example, the object detection device described above) to perform the object recognition process and obtain the processing result (i.e., the identification information of the target object 12).

[0050] Here, when identifying the identification information of the target object 12 by the object recognition process, the identifying result may be presented to the user to check whether there is an error. In this way, in a configuration in which the device automatically recognizes the target object 12, it is possible to prevent an error from occurring in the association between the identification information and the master image. If there is an error in the recognition result, it is possible to allow the user to manually input the identification information of the target object 12. Then, the identifying unit 2040 identifies the master information 20 indicating the identification information input by the user in the object identification information 22 as the master information 20 related to the target object 12.

[0051] <Identifying Similar Master Images: S106> The identification unit 2040 identifies the master image 24 having a high similarity to the target image 10 from among the master images 24 included in the master information 20 related to the target object 12 (S106). The master image 24 identified here is treated as a similar master image.

[0052] For example, for each master image 24 included in the master information 20 related to the target object 12, the identification unit 2040 calculates the similarity between the target image 10 and the master image 24, and determines whether the calculated similarity is equal to or greater than a threshold. If the similarity between a certain master image 24 and the target image 10 is equal to or greater than a threshold, the identification unit 2040 identifies the master image 24 as a master image 24 having a high similarity to the target image 10.

[0053] Various indices can be used as an index of the similarity between the target image 10 and the master image 24. For example, the identification unit 2040 identifies matching feature points between the target image 10 and the master image 24, and determines the similarity between the target image 10 and the master image 24 based on the difference in the positions of the matching feature points. FIGS. 5 and 6 are diagrams for explaining the similarity based on the difference in the positions of the feature points. FIG. 5 shows a case where the similarity between the target image 10 and the master image 24 is high. On the other hand, FIG. 6 shows a case where the similarity between the target image 10 and the master image 24 is low. Note that, in order to make the figures easier to see, objects are represented by dotted lines in FIGS. 5 and 6.

[0054] 5 and 6, feature points A to E are detected from the target image 10. Feature points A to E are also detected from the master image 24. Feature points with the same symbol in the target image 10 and the master image 24 represent the same feature point. For example, feature point A detected from the target image 10 and feature point A detected from the master image 24 represent the same feature point. Note that existing technology can be used as a technology for detecting the same feature point from two different images.

[0055] The identification unit 2040 calculates the difference in position between feature point A in the target image 10 and feature point A in the master image 24. For example, the identification unit 2040 calculates the distance between the coordinates of feature point A in the target image 10 and the coordinates of feature point A in the master image 24 as a value representing the difference in these positions. It can be said that the longer the distance between the same feature points, the greater the difference in the positions of the same feature points. Note that, if the target image 10 and the master image 24 are different sizes, it is preferable to enlarge or reduce one or both of the images to match the sizes before calculating the distance.

[0056] The identification unit 2040 also calculates the position differences between the other matching feature points, and determines the similarity between the target image 10 and the master image 24 based on the calculated position differences. calculation For example, the similarity is calculated based on the following formula (1).

number

[0057] In Fig. 5, the same feature points are located close to each other in the target image 10 and the master image 24. Therefore, in formula (1), the sum of the positional differences becomes small, and the similarity s becomes large. On the other hand, in Fig. 6, although the feature points C and E are relatively close to each other, the other feature points are far apart. Therefore, in formula (1), the sum of the positional differences becomes large, and the similarity s becomes small.

[0058] In addition to the difference in the positions of matching feature points, the number of matching feature points may also be taken into consideration. For example, as shown in the following formula (2), the value calculated by formula (1) is multiplied by a value f(m) that increases as the number of matching feature points increases. As a result, the greater the number of matching feature points, the higher the similarity. Any monotonically non-decreasing function can be used as the function f().

number

[0059] Note that the master information 20 may further indicate, for each master image 24, information on each feature point detected from the master image 24 (such as the coordinates of the feature point and the feature amount of the feature point). In this case, the identification unit 2040 can grasp the feature points of each master image 24 by referring to the master information 20.

[0060] Alternatively, for example, the specification unit 2040 may use the same method for calculating the similarity between the image of the object to be recognized and the master image in the object recognition process to calculate the similarity between the object image 10 and the master image 24.

[0061] The similarity may be calculated using a machine learning technique. Specifically, an estimation model is provided that estimates the similarity of two images in response to the input of these images. This estimation model is trained in advance using multiple learning data consisting of combinations of "two images and their similarities." Various types of estimation models can be used, such as a neural network and an SVM (support vector machine). The identification unit 2040 can obtain the similarity between the target image 10 and the master image 24 by inputting them into this estimation model.

[0062] Alternatively, for example, a judgment model may be provided that judges whether or not two images are highly similar when the images are input. This judgment model is trained in advance using multiple learning data consisting of a combination of "two images and correct output (information indicating whether or not these images are similar)". As with the types of estimation models, various types of judgment models can be used.

[0063] The identification unit 2040 inputs the target image 10 and the master image 24 to the judgment model. If the discrimination model determines that the similarity between the target image 10 and the master image 24 is high, the identification unit 2040 identifies the input master image 24 as the master image 24 having a high similarity to the target image 10.

[0064] Here, it is conceivable that there are multiple master images 24 that have a high similarity to the target image 10. In this case, the identification unit 2040 may treat only one of the multiple master images 24 as a similar master image, or may treat each of the multiple master images 24 as a similar master image.

[0065] In the former case, for example, the identification unit 2040 treats the master image 24 having the greatest similarity to the target image 10 or the one having the smallest suitability index value, which will be described later, as a similar master image. On the other hand, in the latter case, for example, the identification unit 2040 treats the master image 24 having the greatest similarity to the target image 10 one by one as a similar master image, and performs the subsequent processes.

[0066] Here, in the master information 20, the master images 24 may be classified into a plurality of groups. For example, the master images 24 are classified according to the orientation of the object contained therein (upward, rightward, downward, leftward, etc. in a planar view). In this case, the identification unit 2040 may identify the group to which the target image 10 should belong, and compare only the master images 24 included in that group with the target image 10. For example, if the target object 12 faces right, the identification unit 2040 identifies the master images 24 included in the rightward group that have a high degree of similarity with the target image 10. Note that an existing technology can be used for the technology for identifying the orientation of the object contained in the image.

[0067] Note that the groups of master images 24 are not limited to groups based on orientation. For example, if the object is a person, the master images 24 may be grouped according to the presence or absence of belongings, such as when the person has glasses and when they do not, when they have a hat and when they do not, etc., in the master information 20 for the same person.

[0068] <Comparison of suitability as a master image: S108> The update unit 2060 determines which of the target image 10 and the similar master image is more suitable as a master image (S108). To this end, for example, the update unit 2060 calculates the value of an index (suitability index) indicating the level of suitability as a master image for each of the target image 10 and the similar master image, and compares these suitability index values. If the suitability index value of the target image 10 is greater than the suitability index value of the similar master image, the update unit 2060 determines that the target image 10 is more suitable as a master image. On the other hand, if the suitability index value of the target image 10 is equal to or less than the suitability index value of the similar master image, the update unit 2060 determines that the similar master image is more suitable as a master image.

[0069] Various indices can be used as the aptitude index. For example, the aptitude index is the height of the resolution. For example, the higher the resolution, the higher the aptitude index value. In this case, the resolution of the image can be used as the aptitude index value of the image as it is, or the aptitude index value can be calculated using a function that converts the resolution into an aptitude index value. For example, any monotonically non-decreasing function can be used as the function. In addition, a conversion table that associates multiple numerical ranges of resolution with aptitude index values ​​of images having resolutions that belong to the numerical ranges may be prepared, and the resolution may be converted into an aptitude index value using this conversion table.

[0070] However, there may be cases where the resolution required for the master image is fixed, and a master image with a resolution higher than that is undesirable. Therefore, for example, the suitability index value based on the resolution may be a value that "increases with increasing resolution until the resolution reaches a specific value, and decreases when the resolution exceeds the specific value." In this case, an upwardly convex function with the specific value as the maximum value can be used as a function for converting the resolution into the suitability index value. Also, in this case, the conversion table described above can be used.

[0071] As another example of the aptitude index, for example, the number of feature points can be used. That is, the more feature points an image has, the higher the aptitude index value. In this case, the number of feature points obtained from an image can be used as the aptitude index value of the image as is, or the aptitude index value can be calculated using a function that converts the number of feature points into an aptitude index value. As the function, for example, any monotonically non-decreasing function can be used. In addition, a conversion table that associates multiple numerical ranges for the number of feature points with aptitude index values ​​of images having a number of feature points that belong to the numerical ranges may be prepared, and the number of feature points may be converted into an aptitude index value using this conversion table.

[0072] The calculation of the aptitude index value may be realized by using a machine learning technique. For example, an estimation model is provided that estimates the aptitude index value of an image in response to an input of the image. This estimation model is trained in advance using multiple pieces of learning data represented by a combination of "image, aptitude index value". As with the above-mentioned estimation model for estimating similarity, various types of estimation models can be used. The update unit 2060 inputs the target image 10 and the similar master image into this estimation model, thereby obtaining the aptitude index value for each of them.

[0073] Here, the master information 20 may further include a suitability index value of the master image 24 in association with the master image 24. In this case, the updating unit 2060 can obtain the suitability index value of the similar master image from the master information 20. In this case, when the updating unit 2060 adds the target image 10 to the master information 20 as the master image 24, it is preferable to add the suitability index value calculated by the updating unit 2060 together with the target image 10.

[0074] The update unit 2060 does not need to calculate the suitability index value. For example, a judgment model that judges which image is more suitable as a master image in response to two input images is used. R This judgment model is trained in advance using multiple learning data that are combinations of "two images and correct output (information indicating which image is more suitable as a master image)". As with the types of estimation models, various types of judgment models can be used. The update unit 2060 inputs the target image 10 and the similar master image to the judgment model, thereby being able to identify which is more suitable as a master image.

[0075] <Update of master information 20: S110> If it is determined that the target image 10 is more suitable as a master image than the similar master image (S108: target image 10), the update unit 2060 updates the master information 20 including the similar master image (S110). At least, the update unit 2060 adds the target image 10 to the master information 20 as one of the master images 24.

[0076] Furthermore, as described above, the update unit 2060 performs one of the following two processes. 1) Deletion of similar master images 2) Giving the target image 10 a higher priority than similar master images

[0077] The method of either 1) or 2) that the update unit 2060 uses to update the master information 20 may be determined in advance or may be selectable by the user.

[0078] There are various methods for determining the priority to be assigned to the target image 10. For example, the update unit 2060 uses the above-mentioned suitability index value as the priority of the master image 24. In this way, the master image 24 with higher suitability as a master image has a higher priority.

[0079] Alternatively, for example, the multiple master images 24 included in the master information 20 regarding the target object 12 may be ranked according to their suitability as master images, and the rankings may be used as priorities. In this case, a higher ranking indicates a higher priority. Hereinafter, the rankings according to the suitability as master images may also be referred to as priorities.

[0080] Assume that the similar master image has the highest priority in the master information 20. In this case, the update unit 2060 gives the highest priority to the target image 10. On the other hand, if there is a master image 24 with a higher priority than the similar master image in the master information 20 related to the target object 12, the update unit 2060 compares the target image 10 with the master image 24 for suitability as a master image, thereby determining the priority of the target image 10. The process of comparing the suitability as a master image can be realized by utilizing the functions of the update unit 2060.

[0081] As described above, the master information 20 may include the master images 24 divided into groups. In this case, the priority of the master images 24 may be determined based on the relative order within the group, rather than on all the master images 24 included in the master information 20 related to the target object 12. In other words, the master images 24 are given a priority order within the group. For this reason, the update unit 2060 assigns to the target object 12 a priority order represented by the priority order in the group to which the target object 12 is added.

[0082] <<Case of setting an upper limit on the number of master images 24>> An upper limit may be set for the number of master images 24 that can be included in the master information 20. In this case, the update unit 2060 performs the process in 2) while the number of master images 24 does not exceed the upper limit, and performs the process in 1) when the number of master images 24 reaches the upper limit.

[0083] Specifically, when it is determined that the target image 10 is more suitable as a master image, the updating unit 2060 first determines whether the number of master images 24 included in the master information 20 is less than an upper limit. If the number of master images 24 is less than the upper limit, the number of master images 24 will be equal to or less than the upper limit even if the target image 10 is added to the master information 20, so the updating unit 2060 does not delete the similar master images, but assigns a higher priority to the target image 10 than the similar master images and adds it to the master information 20. On the other hand, if the number of master images 24 is equal to or more than the upper limit, the number of master images 24 will exceed the upper limit if the target image 10 is added to the master information 20, so the updating unit 20 deletes the similar master images from the master information 20 so that the number of master images 24 does not exceed the upper limit.

[0084] The upper limit of the number of master images 24 may be set for each group. In this case, if adding the target image 10 to a group would cause the number of master images 24 included in the group to exceed the upper limit, the update unit 2060 deletes the similar master image. On the other hand, if adding the target image 10 to a group would not cause the number of master images 24 included in the group to exceed the upper limit, the update unit 2060 does not delete the similar master image, but adds the target image 10 to the master information 20 with a higher priority than the similar master image.

[0085] <Other examples of using Master Information 20> The master information 20 may be utilized for purposes other than object recognition processing. For example, when a product is handled as an object, the master image 24 included in the master information 20 may be used as an image of the product to be referred to in the thought analysis of the product. In the thought analysis of the product, an analysis is performed to grasp the needs of the user from the characteristics of the product that the user prefers to purchase, so an image of the product whose characteristics are easy to grasp is required. In this regard, according to the image management device 2000, the image of the product that is more suitable as the master image 24 is preferentially left in the master information 20. As described above, an image that is more suitable as the master image 24 can be said to be an image that is easy to extract the characteristics of the product, such as having a high resolution or a large number of feature points. Therefore, an image suitable for the thought analysis can be obtained from the master information 20 managed by the image management device 2000.

[0086] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. 1. An acquisition unit that acquires a target image obtained by capturing an image of a target object; an identification unit that identifies a master image having a high similarity to the target image from master information including one or more master images; an update unit that compares the suitability of the target image and the specified master image as a master image and updates the master information based on the comparison result; When the suitability of the target image is higher than that of the specified master image, the update unit adding the target image to the master information as a master image; The image management device deletes the specified master image from the master information, or gives the target image a priority higher than the priority given to the specified master image. 2. The image management device described in 1., wherein the update unit determines that the target image has a higher suitability when the number of feature points contained in the target image is greater than the number of feature points contained in the identified master image, or when the resolution of the target image is higher than the resolution of the identified master image. 3. The master information is used in an object recognition process for recognizing an object included in an image; 3. The image management device according to claim 1 or 2, wherein the master image having a higher priority is used preferentially in the object recognition process. 4. The identification unit: Identifying the master information indicating identification information of the target object; Among the master images included in the specified master information, subject 4. The image management device according to any one of 1 to 3, which identifies an image having a high similarity to the image. 5. An image management device described in any one of 1 to 4, wherein the identification unit calculates a similarity between the master image and the target image based on the difference in the positions of identical feature points detected from the master image and the target image, and identifies the master image whose similarity is equal to or greater than a threshold as the master image having a high similarity to the target image. 6. An image management device described in any one of 1 to 5, wherein the update unit calculates a suitability index value for the target image indicating the degree of suitability as a master image, and if the suitability index value of the target image is greater than the suitability index value of the master image identified by the identification unit, determines that the target image has a higher suitability. 7. A computer-implemented control method comprising: An acquisition step of acquiring a target image obtained by capturing an image of a target object; A step of identifying a master image having a high similarity to the target image from among master information including one or more master images; an updating step of comparing the target image and the specified master image for suitability as a master image, and updating the master information based on the comparison result; In the updating step, when the suitability of the target image is higher than that of the specified master image, adding the target image to the master information as a master image; A control method comprising: deleting the specified master image from the master information; or assigning a higher priority to the target image than the priority assigned to the specified master image. 8. The control method described in 7., wherein in the update step, if the number of feature points contained in the target image is greater than the number of feature points contained in the identified master image, or if the resolution of the target image is higher than the resolution of the identified master image, the target image is determined to have a higher suitability. 9. The master information is used in an object recognition process for recognizing an object included in an image; 9. The control method according to claim 7 or 8, wherein the master image having a higher priority is used preferentially in the object recognition process. 10. In the specifying step, Identifying the master information indicating identification information of the target object; Among the master images included in the specified master information, subject The control method according to any one of 7 to 9, further comprising identifying an image having a high similarity to the image. 11. A control method described in any one of 7 to 10, wherein in the identification step, the similarity between the master image and the target image is calculated based on the difference in the positions of identical feature points detected from the master image and the target image, and the master image whose similarity is equal to or greater than a threshold is identified as the master image having a high similarity to the target image. 12. A control method described in any one of 7 to 11, wherein in the update step, a suitability index value representing the suitability of the target image as a master image is calculated, and if the suitability index value of the target image is greater than the suitability index value of the master image identified in the identification step, the target image is determined to have a higher suitability. 13. A program for causing a computer to execute the control method described in any one of 7 to 12.

[0087] This application claims priority based on Japanese Patent Application No. 2019-197069, filed on October 30, 2019, the disclosure of which is incorporated herein in its entirety. [Explanation of symbols]

[0088] 10 Target image 12 Target Object 20 Master Information 22 Object Identification Information 24 Master Image 26 Priority 1000 calculator 1020 Bus 1040 Processor 1060 Memory 1080 Storage Device 1100 Input / Output Interface 1120 Network Interface 2000 Image management device 2020 Acquisition Department 2040 Specific section 2060 Update Department

Claims

1. an acquisition unit that acquires a target image obtained by capturing an image of a target object; an identification unit that identifies a master image having a similarity to the target image equal to or greater than a threshold value from master information including one or more master images that are images of an object included in the image and are used in an object recognition process to recognize the object; an update unit that compares a suitability index value indicating a level of suitability as a master image between the specified master image and the target image, and updates the master information based on the comparison result; When the suitability index value of the target image is greater than the suitability index value of the specified master image, adding the target image to the master information as a master image; assigning a higher priority to the target image than the priority assigned to the identified master image; The master image having a higher priority is used preferentially in the object recognition process. An image management device, wherein the suitability index value is a value based on the resolution of the target image and the identified master image, or a value based on the number of feature points contained in the target image and the identified master image.

2. the master information is information in which the master image is associated with identification information assigned to each object, The acquisition unit further acquires the identification information of the target object, The identification unit is Identifying the master information indicating identification information of the target object; The image management device according to claim 1 , further comprising: identifying, from among the master images included in the identified master information, an image having a similarity to the acquired target image equal to or greater than the threshold value.

3. The image management device of claim 1 or 2, wherein the identification unit calculates the similarity between the master image and the target image based on the difference in the positions of identical feature points detected from the master image and the target image, and identifies the master image whose similarity is greater than or equal to the threshold value.

4. The image management device according to claim 1 , wherein the update unit calculates the suitability index value for the target image.

5. 1. A computer-implemented control method comprising: An acquisition step of acquiring a target image obtained by capturing an image of a target object; a step of identifying a master image having a similarity to the target image equal to or greater than a threshold value from among master information including one or more master images that are images of objects and are used in an object recognition process to recognize an object included in the image; an updating step of comparing suitability index values ​​indicating the suitability of the specified master image and the target image as a master image, and updating the master information based on the comparison result; In the updating step, when the suitability index value of the target image is larger than that of the specified master image, adding the target image to the master information as a master image; assigning a higher priority to the target image than the priority assigned to the identified master image; The master image having a higher priority is used preferentially in the object recognition process. A control method, wherein the suitability index value is a value based on the resolution of the target image and the identified master image, or a value based on the number of feature points contained in the target image and the identified master image.

6. A program for causing a computer to execute the control method according to claim 5.

Images