Information processing apparatus, control method, program, and storage medium

Abstract

To provide an information processing apparatus that can easily select desired dictionary data from existing dictionary data.SOLUTION: An information processing apparatus according to the present disclosure has: information acquisition means that acquires an image used for learning, and photographic information that is information related to photographing of the image; generation means that learns a machine learning model by using the acquired image to generate dictionary data that is information for specifying an operation of the learned machine learning model; management means that, for every generated dictionary data, manages the photographic information of the image used for learning the machine learning model related to the dictionary data, in association with the dictionary data; and selection means that, when new photographic information for selecting the dictionary data is acquired, selects one or more pieces of dictionary data from the plurality of pieces of dictionary data on the basis of the new photographic information and the photographic information for every dictionary data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, a control method, a program, and a storage medium. [Background technology]

[0002] In recent years, imaging devices have become known that detect subjects in images using machine learning models such as neural networks, etc. Such imaging devices detect specific subjects using trained machine learning models that have been trained to detect specific subjects such as people, animals, or vehicles that are in high demand as an autofocus function.

[0003] The program describing the processing of the machine learning model and the learned coefficient parameters may also be called dictionary data. Since the trained machine learning model may have different detectable subjects and detection accuracy depending on the training data, when subject detection is performed using dictionary data, if desired dictionary data according to the shooting target and purpose can be used, the accuracy and convenience of subject detection may be improved. On the other hand, when a huge amount of dictionary data is stored in a server and the dictionary data to be used is obtained from the server, it is not easy to select appropriate dictionary data from the stored dictionary data. In response to the problem of obtaining appropriate dictionary data, Patent Document 1 describes a technology for identifying a sport based on schedule information and time information of the sport in competitive sports photography and selecting dictionary data for the identified sport. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-22081 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology described in Patent Document 1 is configured to identify a sport and select dictionary data based on the identified sport, but does not take into consideration the selection of dictionary data without using the sport.

[0006] The present invention has been made in view of the above-mentioned problems, and has as its object to realize a technique that enables a user to easily select desired dictionary data from existing dictionary data. [Means for solving the problem]

[0007] In order to solve this problem, for example, an information processing device of the present invention has the following configuration: That is, the information processing device has an information acquisition means for acquiring an image used for learning and shooting information that is information related to shooting the image, a generation means for generating dictionary data that is information for identifying the operation of a trained machine learning model by training a machine learning model using the acquired image, a management means for managing, for each generated dictionary data, shooting information of an image used for training a machine learning model related to the dictionary data in association with the dictionary data, and a selection means for selecting one or more dictionary data from a plurality of dictionary data based on the new shooting information and the shooting information for each dictionary data when new shooting information for selecting dictionary data is acquired. Effect of the Invention

[0008] According to the present invention, it becomes possible to easily select desired dictionary data from existing dictionary data. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an information processing system according to an embodiment; [Diagram 2] FIG. 1 is a block diagram showing an example of the configuration of a camera according to an embodiment; [Diagram 3] FIG. 2 is a block diagram showing a schematic configuration example of a neural network processing unit according to the embodiment; [Figure 4]FIG. 1 is a block diagram showing an example of the configuration of a server according to an embodiment; [Diagram 5] A data flow diagram showing a use case according to an embodiment. [Figure 6] A flowchart showing a series of operations in a process for associating shooting information with an image according to an embodiment. [Figure 7] A flowchart showing a series of operations related to generation and transmission of teacher data according to an embodiment. [Figure 8] FIG. 13 is a diagram showing an example of an input screen for selecting teacher data according to the embodiment; [Figure 9] A flowchart showing a series of operations in a process for generating and managing dictionary data according to an embodiment. [Figure 10] FIG. 1 is a diagram showing an example of data for managing shooting information according to an embodiment; [Figure 11] A flowchart showing a series of operations in a dictionary data priority determination process according to an embodiment. [Figure 12] FIG. 1 is a diagram for explaining a similarity of shooting information according to an embodiment; [Figure 13] FIG. 13 is a diagram illustrating an example of a dictionary selection screen according to an embodiment. [Figure 14] A flowchart showing a series of operations in a process using dictionary data according to an embodiment. [Figure 15] FIG. 13 is a diagram illustrating an example of a dictionary selection screen according to another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Embodiment 1> Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0011] (Configuration of Information Processing System) An information processing system according to the first embodiment will be described below with reference to FIG. 1. The information processing system includes, for example, a camera 100 as an example of an imaging device and a server 110 as an example of an information processing device. In the following description, a server capable of storing and providing dictionary data will be used as an example of an information processing device, but the present embodiment is not limited to a server and can be applied to other electronic devices capable of storing and providing dictionary data. These electronic devices may include, for example, digital still cameras, digital movie cameras, smartphones, or tablet computers. Furthermore, electronic devices having an imaging function such as a network camera, an in-vehicle camera, a drone camera, and a camera mounted on a robot may be included. In this embodiment, the dictionary data includes a program describing the processing of a machine learning model and learned coefficient parameters. That is, the dictionary data is information that specifies the operation of a learned machine learning model (i.e., the operation of the inference stage).

[0012] The camera 100 and the server 110 are connected, for example, by a wireless communication network. In the information processing system, for example, the camera 100 performs subject detection using a machine learning model. The machine learning model used for subject detection is, for example, composed of a neural network, such as a convolutional neural network (also called CNN). The CNN inputs an image and executes inference processing using dictionary data including processing parameters and the like. In the inference processing, for the input image, the result of detecting the subject in the input image is output. The dictionary data is generated in advance by a learning stage process using teacher data.

[0013] The information processing system according to this embodiment includes a configuration related to the process of generating and managing dictionary data (i.e., including the learning stage process) and a configuration related to the process of using the dictionary data (i.e., including the inference stage process). First, the system configuration will be described below from the viewpoint of the process of generating and managing dictionary data.

[0014] In the camera 100, the image data storage unit 101 stores image data. The image data storage unit 101 writes information at the time of shooting (also simply referred to as shooting information) acquired by the shooting information acquisition unit 103 in a dedicated area for each image data. The shooting information is, for example, information that can be acquired by a sensor in the camera 100 or wireless communication, and can include at least one of information on the location at the time of shooting, information on the time at the time of shooting, and information on the situation at the time of shooting. The information on the location at the time of shooting is, for example, the shooting location, and the information on the time at the time of shooting includes the shooting date, shooting time, time zone, season, etc. Furthermore, the information on the situation at the time of shooting includes the weather at the time of shooting, etc. The image data storage unit 101 transmits the stored image data to the teacher data input unit 102.

[0015] The teacher data input unit 102 transmits image data and subject area information of the image data in which a target subject exists as one set of teacher data to the server 110. The teacher data input unit 102 can input multiple sets of teacher data and transmit them to the server 110.

[0016] In the server 110, the input data acquisition unit 111 acquires image data including shooting information and subject region information transmitted from the teacher data input unit 102. The input data acquisition unit 111 transmits the image data and the subject region information from the acquired data to the dictionary data generation unit 112. The dictionary data generation unit 112 trains a machine learning model using the image data and the subject region information to generate dictionary data. The dictionary data generation unit 112 of this embodiment generates custom dictionary data by additionally learning (also called fine tuning) pre-trained base dictionary data so that highly accurate dictionary data can be created even with a small amount of teacher data. Note that the learning method is not limited to the method of this embodiment, and a method that does not use a base dictionary may be used.

[0017] Dictionary data management unit 113 receives the dictionary data generated by dictionary data generation unit 112, and manages the received dictionary data together with other dictionary data generated in the same manner. Dictionary data management unit 113 assigns a dictionary data ID to the generated dictionary data, and manages the data in a manner that makes it distinguishable from other dictionary data.

[0018] On the other hand, the shooting information added to the image data is extracted from the image data by the shooting information extraction unit 114 and transmitted to the shooting information management unit 115. The shooting information management unit 115 manages the shooting information and the dictionary data ID of the dictionary data generated using the image data to which the corresponding shooting information has been added, by linking them together. That is, the shooting information management unit 115 manages the shooting information of the image used to train the machine learning model related to the dictionary data for each generated dictionary data by associating it with the dictionary data.

[0019] Next, the system configuration will be described from the viewpoint of processing using dictionary data. The dictionary priority acquisition unit 104 of the camera 100 can acquire the priority of the dictionary data by inputting the shooting information obtainable by the shooting information acquisition unit 103 and communicating with the server 110. The priority of the dictionary data indicates the priority of the dictionary data with respect to the current shooting environment of the user of the dictionary data.

[0020] In the server 110, the dictionary priority calculation unit 116 receives the shooting information from the camera 100, and then acquires the shooting information linked to the multiple dictionary data from the shooting information management unit 115, and calculates the priority of the dictionary data. The process of calculating the priority of the dictionary data will be described later.

[0021] The dictionary selection screen 105 of the camera 100 acquires multiple pieces of dictionary data information from the dictionary data management unit 113 of the server 110 and displays them on the dictionary selection screen 105. At this time, the dictionary selection screen 105 reflects the priority of each dictionary data acquired by the dictionary priority acquisition unit 104 on the dictionary selection screen 105. This allows the user to easily select dictionary data suitable for the current shooting environment. The details of the dictionary selection screen 105 will be described later.

[0022] The subject detection unit 106 performs inference processing by CNN based on the dictionary data selected by the user on the dictionary selection screen 105. The detection result obtained by the inference processing is used for imaging control (autofocus, exposure control, etc.).

[0023] 1 is not limited to the configuration including the camera 100 and the server 110. The information processing system may be configured to execute all of the above-mentioned processes within the camera 100, or may be configured to input teacher data using a mobile terminal such as a smartphone instead of the camera 100.

[0024] Such an information processing system enables a user of the camera 100 to provide teacher data to generate dictionary data useful for his / her own photography, or to easily select desired dictionary data from dictionary data stored in the server 110. In a more developed manner, for example, a use case may be realized in which a user creates dictionary data reflecting his / her own preferences, uploads it to a server, and shares it with other users. In such a case, a business model in which a company operates the uploaded dictionary data and provides it to users to secure new customers may be envisioned.

[0025] The configuration shown in camera 100 in Fig. 1 can be realized by CPU 201 shown in Fig. 2 executing a program, or by CPU 201 controlling each unit shown in Figs. 2 and 3. The processes realized by the configuration shown in camera 100 in Fig. 1 correspond to the processes shown in Figs. 6, 7 and 14, for example, and will be described in detail with reference to these figures. The configuration shown in server 110 in Fig. 1 can be realized by CPU 401 shown in Fig. 4 executing a program, or by CPU 401 controlling each unit shown in Fig. 4. The processes realized by the configuration shown in server 110 in Fig. 1 correspond to the processes shown in Figs. 9 and 11, for example, and will be described in detail with reference to these figures.

[0026] One or more of the functional blocks shown in the following figures may be realized by hardware such as an ASIC or a programmable logic array (PLA), or may be realized by a programmable processor such as a CPU or an MPU executing software, or may be realized by a combination of software and hardware.

[0027] (Camera configuration) An example of the configuration of the camera 100 will be described with reference to Fig. 2. As an example, the camera 100 includes a CPU 201, a memory 202, a non-volatile memory 203, an operation unit 204, a neural network processing unit 205, an imaging unit 212, an image processing unit 213, and an encoding processing unit 214. Furthermore, the camera 100 includes a display control unit 215, a display unit 216, a communication control unit 217, a communication unit 218, a GPS receiving module 219, a recording medium control unit 220, an internal bus 230, and a photographing lens 211.

[0028] The camera 100 uses a photographing lens 211 to form an optical image of a subject on a pixel array of an imaging unit 212. The photographing lens 211 may be detachable or non-detachable from the body (housing, main body) of the camera 100. The camera 100 writes and reads image data to and from a recording medium 221 via a recording medium control unit 220. The recording medium 221 may be detachable or non-detachable from the camera 100.

[0029] The CPU 201 includes one or more processors, and controls the operation of each unit (each functional block) of the camera 100 via the internal bus 230 by executing a computer program stored in the non-volatile memory 203 .

[0030] The memory 202 is, for example, a rewritable volatile storage medium. The memory 202 temporarily records a computer program for controlling the operation of each part of the camera 100, information such as parameters related to the operation of each part of the camera 100, information received by the communication control unit 217, etc. The memory 202 temporarily records images acquired by the imaging unit 212, images and information processed by the image processing unit 213, the encoding processing unit 214, etc. The memory 202 has a sufficient storage capacity for temporarily recording these.

[0031] The non-volatile memory 203 is a non-volatile storage medium that can be electrically erased and recorded, and includes, for example, an EEPROM, a hard disk, etc. The non-volatile memory 203 stores information such as computer programs that control the operation of each part of the camera 100 and parameters related to the operation of each part of the camera 100. Various operations performed by the camera 100 are realized by such computer programs. Furthermore, the non-volatile memory 203 stores a computer program describing the processing contents of the neural network used by the neural network processing unit 205, and learned coefficient parameters such as weighting coefficients and bias values. The computer program describing the processing of the neural network and the learned coefficient parameters correspond to dictionary data for subject detection.

[0032] The weighting coefficient is a value indicating the strength of the connection between nodes in the neural network, and the bias is a value for providing an offset to the weighting coefficient and the integrated value of the input data. The non-volatile memory 203 can hold multiple learned coefficient parameters and computer programs describing the processing of the neural network. Multiple computer programs describing the processing of the neural network and multiple learned coefficient parameters may be temporarily stored in the memory 202.

[0033] The operation unit 204 provides a user interface for operating the camera 100. The operation unit 204 includes various buttons such as a power button, a menu button, a release button for shooting, a video recording button, and a cancel button, and the various buttons are configured by switches, a touch panel, or the like. The CPU 201 controls the camera 100 according to a user's instruction input via the operation unit 204. Note that, although the case where the CPU 201 controls the camera 100 based on an operation input via the operation unit 204 has been described here as an example, the present invention is not limited to this. For example, the CPU 201 may control the camera 100 based on a request input via the communication unit 218 from a remote controller or a mobile terminal (not shown).

[0034] The neural network processing unit 205 performs inference processing of the subject detection unit 106 based on the dictionary data. The configuration of the neural network processing unit 205 will be described later with reference to FIG.

[0035] The photographing lens 211 includes a lens unit, and is composed of, for example, a lens group including a zoom lens, a focus lens, etc., a lens control unit (not shown), an aperture (not shown), etc. The photographing lens 211 can function as a zoom means for changing the angle of view. The lens control unit of the photographing lens 211 adjusts the focus and controls the aperture value (F value) according to a control signal transmitted from the CPU 201.

[0036] The imaging unit 212 can function as an acquisition unit that sequentially acquires a plurality of images including a moving image. For example, a CCD (charge-coupled device) image sensor, a CMOS (complementary metal-oxide semiconductor) image sensor, or the like is used as the imaging unit 212. The imaging unit 212 has a pixel array (not shown) in which photoelectric conversion units (pixels) that convert an optical image of a subject into an electric signal are arranged in a matrix, i.e., two-dimensionally. An optical image of the subject is formed on the pixel array by the photographing lens 211. The imaging unit 212 outputs the captured image to the image processing unit 213 or the memory 202. The imaging unit 212 is also capable of acquiring a still image.

[0037] The image processing unit 213 performs predetermined image processing on the image data output from the imaging unit 212 or the image data read from the memory 202. Examples of the image processing include dynamic range conversion processing, interpolation processing, reduction processing (resizing processing), color conversion processing, etc. Furthermore, the image processing unit 213 performs predetermined arithmetic processing for exposure control, distance measurement control, etc., using the image data acquired by the imaging unit 212.

[0038] When the image processing unit 213 performs arithmetic processing, the CPU 201 performs exposure control, distance measurement control, etc., based on the calculation results obtained by the arithmetic processing. For example, the CPU 201 performs AE (Auto Exposure) processing, AWB (Auto White Balance) processing, AF (Auto Focus) processing, etc. These shooting controls are performed with reference to the subject detection results by the neural network processing unit 205.

[0039] The encoding processing unit 214 compresses the size of the image data by performing intraframe predictive coding (intra-screen predictive coding), interframe predictive coding (inter-screen predictive coding), or the like on the image data from the image processing unit 213.

[0040] The display control unit 215 controls the display unit 216. The display unit 216 has a display screen (not shown). The display control unit 215 generates an image signal that can be displayed on the display screen of the display unit 216, and outputs the image signal to the display unit 216. The display control unit 215 is capable of not only outputting image data to the display unit 216, but also outputting image data to an external device via the communication control unit 217.

[0041] The display unit 216 has an OSD (On Screen Display) function that is a function of displaying a setting screen such as a menu on the display screen. The display control unit 215 can superimpose an OSD image on an image signal and output the image signal to the display unit 216. The display unit 216 can also superimpose a subject frame based on a subject detection result by the neural network processing unit 205 on the image signal and display it in response to an instruction from, for example, the CPU 201 or the display control unit 215. The display unit 216 may be configured with, for example, a liquid crystal display, an organic EL display, or the like, and displays the image signal sent from the display control unit 215. The display unit 216 may include, for example, a touch panel. When the display unit 216 includes a touch panel, the display unit 216 can also function as the operation unit 204.

[0042] The communication control unit 217 operates according to instructions from the CPU 201, generates a modulated signal conforming to a wireless communication standard such as IEEE802.11, and outputs the modulated signal to the communication unit 218, while receiving the modulated signal from an external device via the communication unit 218. The communication control unit 217 is capable of transmitting and receiving a control signal for a video signal. For example, the communication unit 217 may be controlled to transmit a video signal conforming to a communication standard such as HDMI (High Definition Multimedia Interface) (registered trademark) or SDI (Serial Digital Interface).

[0043] The communication unit 218 converts the video signal and the control signal into physical electric signals and transmits and receives them to and from external devices. In addition to transmitting and receiving the video signal and the control signal, the communication unit 218 receives dictionary data for subject detection in the neural network processing unit 205 from the server 110 and transmits teacher data to the server 110. The communication unit 218 also receives information such as the shooting date, shooting time, and weather at the time of shooting, which are used as shooting information.

[0044] The GPS receiving module 219 can acquire location information of the camera 100. The location information acquired by the GPS receiving module 219 is added to the image data as shooting information when the teacher data is captured. Note that the acquisition of location information is not limited to using GPS, and other configurations may be used as long as the shooting location can be specified.

[0045] The recording medium control unit 220 controls writing of data to the recording medium 221 and reading of data from the recording medium 221. The recording medium control unit 220 outputs a control signal for controlling the recording medium 221 to the recording medium 221 based on a request from the CPU 201. For example, a non-volatile memory or a magnetic disk is used as the recording medium 221. As described above, the recording medium 221 may be removable or non-removable. The recording medium 221 stores the encoded image data, etc. as a file in a format compatible with the file system of the recording medium 221.

[0046] The internal bus 230 controls transmission and reception of data between the respective functional blocks 201-205, 212-215, 217, and 219-220 of the camera 100. Note that one or more of the functional blocks shown in Fig. 2 are realized by causing a CPU 201 serving as a computer included in the camera 100 to execute a computer program stored in a non-volatile memory 203 or the like serving as a storage medium.

[0047] Fig. 3 shows an example of a schematic configuration of the neural network processing unit 205. As shown in Fig. 3, the neural network processing unit 205 includes a neural core 300 including a CPU 301, a product-sum calculation circuit 302, a DMAC (Dynamic Memory Access Controller) 303, and an internal memory 304.

[0048] The CPU 301 acquires dictionary data from the memory 202 or non-volatile memory 203 via the internal bus 230, or from the internal memory 304, and executes it. The CPU 301 also controls the product-sum calculation circuit 302 and the DMAC 303.

[0049] The product-sum calculation circuit 302 is a circuit that performs product-sum calculation in a neural network. The product-sum calculation circuit 302 has multiple product-sum calculation units, which can perform product-sum calculations in parallel. The product-sum calculation circuit 302 outputs intermediate data calculated when the multiple product-sum calculation units perform product-sum calculations in parallel to the internal memory 304 via the DMAC 303.

[0050] The DMAC 303 is a circuit that transfers data without going through the CPU 301, and transfers data between the memory 202 or nonvolatile memory 203 and the internal memory 304 via the internal bus 230. The DMAC 303 also transfers data between the product-sum calculation circuit 302 and the internal memory 304. Data transferred by the DMAC 303 includes a computer program describing the processing contents of the neural network, learned coefficient parameters (i.e., dictionary data), intermediate data calculated by the product-sum calculation circuit 302, and the like.

[0051] The internal memory 304 stores the above-mentioned dictionary data, intermediate data calculated by the product-sum calculation circuit 302, etc. The internal memory 304 may have a plurality of banks, and the banks may be dynamically switched.

[0052] (Server configuration) Next, a configuration example of the server 110 will be described with reference to Fig. 4. As an example, the server 110 has a CPU 401, a memory 402, a display unit 403, an internal bus 404, an operation unit 405, a recording unit 406, a communication unit 407, and a neural network processing unit 408. Note that some of the functional blocks shown in Fig. 4 may be realized by causing the CPU 401, which serves as a computer included in the server 110, to execute a computer program stored in the recording unit 406 or the like, which serves as a storage medium.

[0053] The CPU 401 executes various processes in the server 110 by executing computer programs stored in the recording unit 406. The memory 402 includes, for example, a volatile storage medium, and is mainly used as a work area for the CPU 401 and a temporary buffer area for data.

[0054] The display unit 403 is configured, for example, with a liquid crystal panel or an organic EL panel, and displays a screen for setting and the like based on an instruction from the CPU 401. Note that instead of including the display unit 403, the server 110 may output information to be displayed to an external device via the communication unit 407.

[0055] The internal bus 404 is a bus for mutually connecting each processing block in the server 110. The operation unit 405 is composed of a keyboard, a mouse, buttons, a touch panel, a remote control, or the like, and receives operation instructions from a user. Operation information input from the operation unit 405 is transmitted to the CPU 401, and the CPU 401 executes control of each processing block based on the operation information. Note that instead of having the operation unit 405, the server 110 may acquire operation information from an external device via the communication unit 407.

[0056] The recording unit 406 includes, for example, a non-volatile storage medium, and stores and reads various data in the storage medium based on instructions from the CPU 401. The storage medium is, for example, an EEPROM, an internal flash memory, an internal hard disk, or a removable memory card. In addition to computer programs, the recording unit 406 stores teacher data used for learning in the neural network processing unit 408, dictionary data that has already been generated, and the like.

[0057] The communication unit 407 includes hardware for performing communication over a wireless LAN and a wired LAN. In the case of a wireless LAN, for example, processing is performed according to the IEEE802.11n / a / g / b method. The communication unit 407 connects to an external access point over a wireless LAN and performs wireless LAN communication with other wireless communication devices via the access point. In the case of a wired LAN, the communication unit 407 performs communication via an external router or a switching hub using an Ethernet cable or the like. The communication unit 407 communicates with external devices including the camera 100 and exchanges information such as teacher data and dictionary data.

[0058] The neural network processing unit 408 performs a learning stage process of the neural network using the teacher data obtained via the communication unit 407. The neural network processing unit 408 performs a learning process for constructing dictionary data corresponding to each of different classes of subjects using the teacher data. The neural network processing unit 408 may be composed of a GPU (Graphic Processing Unit), a DSP (Digital Signal Processor), or the like. The dictionary data that is the result of the learning process performed by the neural network processing unit 408 is held in the recording unit 406.

[0059] (Processing in Information Processing Systems) Next, the processing of the information processing system in this embodiment will be described. As described above, the information processing system in this embodiment performs processing to generate and manage dictionary data, and processing to use the dictionary data. Fig. 5 shows a data flow assuming a use case for explaining the above-mentioned two processes. In the example shown in Fig. 5, a case where there are two parties, a teacher data provider 501 and a dictionary user 502, will be described as an example, but the teacher data provider 501 and the dictionary user 502 may be the same person.

[0060] A teacher data provider 501 specifies teacher data using the camera 100. The specified teacher data is transmitted from the camera 100 to the server 110. The server 110 performs dictionary generation 503 using the received teacher data. The generated dictionary data is managed in dictionary management 504. Meanwhile, a dictionary user 502 can obtain the dictionary data managed in dictionary management 504 through dictionary selection 505 by the server 110 and use it in the camera 100.

[0061] Next, the processes related to dictionary generation 503 and dictionary management 504 in FIG. 5 will be described in detail. Specifically, the camera 100 associates the image used for the teacher data with the shooting information (processing shown in FIG. 6, corresponding to the shooting information acquisition unit 103 and the image data storage unit 101), and then generates the teacher data and transmits it to the server (processing shown in FIG. 7, corresponding to the teacher data input unit 102). After that, in a part of the process shown in FIG. 9, the server 110 generates dictionary data using the acquired teacher data (corresponding to the input data acquisition unit 111, the shooting information extraction unit 114, and the dictionary data generation unit 112). That is, these processes correspond to the dictionary generation 503 shown in FIG. 5. Furthermore, in other processes shown in FIG. 9, the server 110 assigns an ID and statistical information, which will be described later, to the generated dictionary data and stores it together with the shooting information (corresponding to the shooting information management unit 115 and the dictionary data management unit 113). That is, these operations correspond to the dictionary management 504 shown in FIG. 5 described above.

[0062] (Process of associating shooting information with an image) 6, a process of associating shooting information with an image used for teacher data in camera 100 will be described. Note that this process is realized by developing a computer program stored in non-volatile memory 203 into memory 202 and having CPU 201 read and execute the computer program in memory 202 while camera 100 is powered on.

[0063] In S601, the imaging unit 212 captures an image in response to an instruction from the CPU 201. The image captured in S601 is processed by, for example, the image processing unit 213, the encoding processing unit 214, etc., and then temporarily recorded in, for example, the recording medium 221, etc.

[0064] In S602, in response to an instruction from the CPU 201, the communication unit 218 and the GPS receiving module 219 acquire shooting information (that is, shooting location, shooting date, shooting time, weather, etc.).

[0065] In S603, the CPU 201 puts the image recorded in S601 and the shooting information acquired in S602 together into, for example, one file and outputs the file to the recording medium 221. The process of putting the image and the shooting information together into one file may be performed, for example, by writing the shooting information into a predetermined area of ​​the image file. In this way, in this embodiment, the camera 100 generates data in which the image and the shooting information at the time when the image was shot are associated with each other. This allows the shooting information corresponding to the image to be easily and quickly extracted when the server 110 handles the shooting information associated with the learning data. Note that the captured image and the shooting information do not necessarily need to be included in one file, as long as the captured image and the shooting information are associated with each other. The CPU 201 then ends the process.

[0066] (Processing related to generation and transmission of teacher data) Next, a process for generating and transmitting teacher data in camera 100 will be described with reference to Fig. 7. The operation shown in Fig. 7 relates to a flow in which information indicating the correct answer for the subject area is input in camera 100 and learning start is notified to server 110. This process is also realized by developing a computer program stored in non-volatile memory 203 in memory 202 while camera 100 is powered on, and having CPU 201 read and execute the computer program in memory 202.

[0067] In S701, the operation unit 204 accepts a user operation to select an image to be used as teacher data from among the captured images stored in the recording medium 221. FIG. 8 shows an example of an input screen displayed on the display unit 216 for selecting an image to be used as teacher data. In the example shown in FIG. 8(a), 12 captured images 801 stored in the recording medium 221 are displayed. The user can select (one or more) teacher data from among the displayed captured images by touching the operation unit 204, for example. The CPU 201 displays an indication (e.g., a circular area 802) indicating whether or not an image has been selected, for example, in the upper left corner of the captured image. When it indicates that an image has been selected, the captured image becomes an image of teacher data.

[0068] In S702, the operation unit 204 accepts a user operation to specify a subject area in an image selected as training data. FIG. 8(b) shows an example of an input screen for a subject area displayed on the display unit 216. In FIG. 8(b), a rectangular frame 803 indicates a subject area input by a user. The CPU 201 sets a subject area for each image selected as training data. As a method for setting the subject area, a region may be directly selected from an image displayed via a touch panel that is part of the operation unit 204 and is integrated with the display unit 216. Alternatively, the region may be selected by selecting and fine-tuning a subject frame simply detected by the CPU 201 or the like using a feature amount such as an edge.

[0069] In S703, the operation unit 204 accepts a user operation to start dictionary data generation. FIG. 8(c) shows an example of a confirmation screen to start dictionary data generation, displayed on the display unit 216. The user inputs "YES" or "NO" on the confirmation screen. If "YES" shown in 804 is selected, the CPU 201 transmits teacher data to the server 110 via the communication unit 218. At this time, the CPU 201 associates image data with subject area information of the image data in which a subject exists, and transmits the data to the server 110 as one set of teacher data. The CPU 201 may write the subject area information in a predetermined area of ​​the image data, or may treat the image data and the subject area information as separate information. Note that, as exemplified in FIGS. 8(a) to (c), a plurality of subject images may be selected. Therefore, the CPU 201 may transmit one set of teacher data, in which the selected image data and the subject area information are associated, to the server 110 for the number of selected subject images. When the teacher data and the like are transmitted to the server 110, dictionary data, which will be described later, is generated in the server 110. Note that, if "NO" is selected in the example shown in FIG. 8(c), the CPU 201 ends the process without transmitting the teacher data to the server 110.

[0070] (Process for generating and managing dictionary data) Next, a process of generating and managing dictionary data in server 110 will be described with reference to Fig. 9. This process is realized by developing a computer program stored in recording unit 406 into memory 402, and having CPU 401 read and execute the computer program in memory 402.

[0071] In S901, the CPU 401 of the server 110 acquires teacher data from the camera 100 via the communication unit 407. In S902, the CPU 401 extracts shooting information from image data included in the acquired teacher data. Then, the CPU 401 stores the extracted shooting information (for example, in the shooting information management unit 115 configured in the memory 402).

[0072] In S903, the CPU 401 acquires base dictionary data for learning. For example, dictionary data in which a wide variety of objects have been learned in advance can be set as an initial value as the base dictionary data. Alternatively, dictionary data in which specific subjects such as dogs, cats, and motorcycles have been learned can be used as the base dictionary data.

[0073] In S904, the neural network processing unit 408 generates dictionary data in response to an instruction from the CPU 401. For example, the neural network processing unit 408 uses the teacher data including the teacher data acquired in S901 to train a machine learning model to obtain trained coefficient parameters. At this time, the coefficient parameters of the machine learning model at the time of starting the training can use the coefficient parameters included in the base dictionary data. The neural network processing unit 408 sets a program describing the processing of the used machine learning model and the trained coefficient parameters obtained by the training as newly generated dictionary data. The trained coefficient parameters obtained here provide dictionary data for detecting a subject included in an image acquired from the camera 100 or a subject similar to the subject with higher accuracy. S904 functions as a learning means for generating custom dictionary data. In S905, the CPU 401 issues a dictionary data ID for identifying the dictionary data to the generated dictionary data. The dictionary data ID is, for example, a unique number for each dictionary.

[0074] In S906, the CPU 401 stores the generated dictionary data in the dictionary data management unit 113 configured in the memory 402 (or the recording unit 406). The dictionary data managed by the dictionary data management unit 113 can be acquired by specifying a dictionary data ID. At this time, the CPU 401 makes one of the images in the teacher data acquired in S901 into a thumbnail image for display on a dictionary selection screen described later, and stores the thumbnail image in the dictionary data management unit 113. The thumbnail image to be stored may be one that is specified in advance in the camera 100 by the user who generates the dictionary data, or may be automatically selected by the server 110.

[0075] In S907, the CPU 401 associates the shooting information acquired in S902 with the dictionary data ID issued in S905, and stores them in the memory 402 (or the recording unit 406). A method for managing the shooting information will be described with reference to FIG.

[0076] Fig. 10 shows an example of shooting information associated with the dictionary data ID in S907. The columns of the table manage data on the dictionary data ID, shooting location, shooting date, shooting time, and weather. The rows manage the variance and average of each element of the shooting information for each dictionary data ID. Each element of the shooting information associated with each dictionary data ID has the same number of pieces of information as the image data used as training data, and is managed in an array.

[0077] In S908, the CPU 401 calculates the variance and average value of the shooting information managed for each dictionary data ID. Note that the weather of the shooting information according to this embodiment can be quantified as the variance and average value by quantifying it step by step from sunny to rainy depending on the amount of clouds, etc., and managing it as shooting information. The CPU 401 then ends the process of generating and managing dictionary data.

[0078] (Dictionary data priority determination process) Next, referring to Fig. 11, a process for determining the priority of a plurality of dictionaries managed by server 110 based on the shooting information acquired by camera 100 will be described. The process described here corresponds to a process for acquiring appropriate dictionary data from among the dictionary data managed in dictionary management 504 (shown in Fig. 5) through dictionary selection 505 (corresponding to dictionary priority calculation unit 116). This process makes it possible for the present information processing system to provide the priority of dictionary data when dictionary user 502 selects dictionary data. In other words, it is possible to assist dictionary user 502 in easily selecting appropriate dictionary data.

[0079] In the following description, the photographing information acquired by the camera 100 is called the dictionary user photographing information, and the photographing information already managed by the server 110 is called the managed photographing information. The process of determining the priority of the dictionary data is realized by expanding the computer program stored in the recording unit 406 into the memory 402, and by the CPU 401 reading and executing the computer program in the memory 402. In addition, before the start of this process, the camera 100 used by the dictionary user 502 acquires current photographing information (dictionary user photographing information). For example, by acquiring photographing information related to the place and time where the dictionary data is used, it is possible to provide appropriate dictionary data according to the photographing place and the photographing time. However, the dictionary user photographing information is not limited to automatically acquiring information at the time of photographing, and the dictionary user may specify the photographing information. For example, when the dictionary data is to be set in advance on the day before photographing, the dictionary user may be allowed to specify any photographing information.

[0080] In S1101, the CPU 401 of the server 110 acquires dictionary user photographing information from the camera 100. In S1102, the CPU 401 acquires a group of photographing information (i.e., managed photographing information) managed by the photographing information management unit 115 of the memory 402. At this time, the CPU 401 excludes managed photographing information of a dictionary data ID in which the average value of the managed photographing information is deviated from the dictionary user photographing information by a predetermined amount or more from the group of photographing information to be acquired. By excluding the deviated information, the processing amount of the server 110 can be reduced and the processing time can be shortened. In addition, the CPU 401 excludes managed photographing information of a dictionary data ID in which the variance value associated with the dictionary data ID is outside a predetermined range from the group of photographing information to be acquired. By limiting the variance value of the managed photographing information to a specified range, it is possible to switch between a general-purpose dictionary that can be used in a wide range and a specific dictionary that can be used in a spot manner.

[0081] In S1103, the CPU 401 calculates the similarity between the dictionary user photographed information and each piece of managed photographed information acquired in S1102. A method of calculating the similarity will be described later with reference to FIG.

[0082] In S1104, the CPU 401 determines whether there is one or more dictionary data IDs whose similarity calculated in S1103 is equal to or greater than a predetermined level. If the CPU 401 determines that there is no dictionary data ID whose similarity is equal to or greater than a predetermined level, it proceeds to S1105 because it is difficult to prioritize the dictionary data IDs, and otherwise proceeds to S1106. In S1105, the CPU 401 relaxes the parameters for calculating the similarity used in S1103, and returns to S1103. In this way, by relaxing the parameters for calculating the similarity, it becomes possible to increase the similarity calculated in S1103. Details will be described later with reference to FIG. 12.

[0083] In S1106, the CPU 401 determines the priority of the dictionary data ID based on the calculated similarity. For example, the CPU 401 assigns a high priority to a dictionary data ID with a high similarity and a low priority to a dictionary data ID with a low similarity. The CPU 401 may also set the priority of a dictionary data ID with a similarity lower than a predetermined level as undeterminable and not present the dictionary data to the dictionary user. In this way, by not presenting dictionary data that cannot be determined to the dictionary user, the user can select necessary dictionary data from more carefully selected dictionary data.

[0084] In this way, the CPU 401 can select dictionary data in which the distribution of the photographing information obtained for each dictionary data satisfies a predetermined condition with the user photographing information (new photographing information) (for example, the average value of the management photographing information does not deviate from the new photographing information by a predetermined amount or more). Also, the CPU 401 can select one or more dictionary data according to the degree of similarity of the photographing information obtained based on the photographing information for each dictionary data and the user photographing information (new photographing information).

[0085] In S1107, the CPU 401 transmits the priority for each dictionary data ID determined in S1106 to the camera 100. The CPU 401 can transmit a thumbnail image associated with the dictionary data ID to the camera 100 together with the dictionary data ID. The CPU 401 then ends the process. When the priority for each dictionary data ID is transmitted to the camera 100 by the process of S1107, the camera 100 can reflect the priority for each dictionary data ID on the dictionary selection screen (corresponding to the dictionary priority acquisition unit 104 and the dictionary selection screen 105). The camera 100 can also reflect a thumbnail image corresponding to the dictionary data ID on the dictionary selection screen. This allows the user to select appropriate dictionary data from the dictionary data displayed according to the priority. The dictionary selection screen for selecting dictionary data will be described later with reference to FIG. 13.

[0086] The calculation of the similarity of the photographing information in S1103 will be described with reference to FIG. 12. In the example shown in FIG. 12, the case where photographing location information is used as the photographing information will be described as an example. FIG. 12(a) shows a schematic diagram of the relationship between the managed photographing information and the user photographing information. 1201 represents a map. For example, when the information processing system is used in a zoo, 1201 is a map of the zoo. 1202, 1203, 1204, and 1205 are managed photographing information classified by dictionary data ID, and the photographing locations included in the managed photographing information are shown on the map 1201. In FIG. 12(a), the plot shape is changed for each dictionary data ID. 1206 shows the photographing location acquired as the user photographing information on the map 1201.

[0087] FIG. 12(b) is a diagram for explaining the calculation of the similarity of the photographing information. 1207 indicates concentric circles centered on the photographing location 1206 in the user photographing information. If the concentric circles are used as the threshold of the similarity, the similarity between the managed photographing information and the user photographing information for each dictionary data ID can be calculated by counting the managed photographing information inside 1207 for each dictionary data ID. Also, 1208 indicates the distance from the user photographing information to the threshold 1207, and the relaxation of the similarity calculation parameter in S1106 is realized by increasing the distance 1208. Note that the calculation method of the similarity between the managed photographing information and the user photographing information shown in FIG. 12 is an example, and may be realized by using another classification method such as machine learning.

[0088] Next, a dictionary data selection screen (also called a dictionary selection screen, corresponding to dictionary selection screen 105) on display unit 216 of camera 100 will be described with reference to Fig. 13. The example shown in Fig. 13 also illustrates an example assuming a zoo. Fig. 13(a) illustrates an example of a screen display when the map display is reduced (i.e., a wide range display), and Fig. 13(b) illustrates an example of a screen display when the map display is enlarged for a specific area.

[0089] Reference numeral 1301 indicates a dictionary data selection screen, and a dictionary user can select dictionary data by touching the dictionary data selection screen or by operating an operating member. Reference numeral 1302 indicates the scale of the map, and the map is displayed in a larger scale in FIG. 13(b) than in FIG. 13(a). The magnification can be changed by touching the dictionary data selection screen or by operating an operating member.

[0090] 1311 to 1314 indicate areas in the zoo where animals exist. For example, 1311 corresponds to an animal contact area, 1312 corresponds to a bird area, 1313 corresponds to a savanna area, and 1314 corresponds to an underwater area. 1321 to 1325 are thumbnail images managed in association with each dictionary data. The tips of the balloons 1321 to 1325 indicate the average value of the shooting location in the managed shooting information of each dictionary data ID. That is, the CPU 201 displays a thumbnail based on an image used to generate the dictionary data on the display unit in association with one location on the map (for example, the average value of the shooting location). In this way, the dictionary user can intuitively grasp the contents of the dictionary data. In this case, the server 110 can provide the camera 100 with the priority of the dictionary data, a thumbnail for each dictionary data, and one location (for example, the average value for each dictionary data shown in FIG. 10). 13(a) and (b), 1321 indicates dictionary data for dog 1, 1322 indicates dictionary data for birds, 1323 indicates dictionary data for fish, 1324 indicates dictionary data for dog 2, and 1325 indicates dictionary data for dog 3. Note that the examples of FIGS. 13(a) and (b) correspond to a case where the number of dictionary data items displayed is three, and dictionary data items with a priority level of 4 or lower are deemed undeterminable in S1108 described above. The number of dictionary data items displayed may be fixed in advance within server 110, or may be set arbitrarily by the dictionary user via camera 100.

[0091] In the example of FIG. 13(a), since the dictionary user is currently located at 1206, the dictionary data corresponding to 1321 is determined to be the dictionary data with the highest priority according to the distance from the shooting location (i.e., shooting information) of the dictionary use vehicle. For this reason, the CPU 201 of the camera 100 highlights and displays the thumbnail of 1321. That is, the CPU 201 can highlight the thumbnail to be displayed (for example, one) based on the similarity of the shooting information obtained based on the shooting information for each dictionary data and the user shooting information (new shooting information). In this example, only the thumbnail with the highest priority is highlighted, but another highlighting method such as gradually changing the color or density may be used. Also, the dictionary data of animals in the savanna area of ​​1313 is not displayed on the screen because its priority is 4th or lower.

[0092] FIG. 13(b) shows a screen in which the animal petting area portion of 1311 is enlarged. The larger the scale of the display of the camera 100, the lower the threshold value of the variance value of the shooting information in the similarity calculation in S1103 may be set by the CPU 401. That is, the smaller the scale of the map display, the smaller the dictionary data with which the variance calculated for each dictionary data is. In this case, it is possible to display dictionary data for more detailed shooting locations than in FIG. 13(a). As in 1321, 1324, and 1325, the same dog has different subject characteristics depending on the shooting location, so it is possible to select dictionary data that is more suitable for the dictionary user's conditions.

[0093] (Processing using dictionary data) Next, a process of using dictionary data in camera 100 will be described with reference to Fig. 14. This process corresponds to subject detection unit 106. This process is realized in camera 100 by developing a computer program stored in non-volatile memory 203 into memory 202, and having CPU 201 read and execute the computer program in memory 202.

[0094] In S1401, the CPU 201 determines whether dictionary data has been selected by the dictionary user. If dictionary data has been selected, the CPU 201 advances the process to S1402, and if not, the process to S1404.

[0095] In S1402, the neural network processing unit 205 detects a subject included in a newly captured image using the selected dictionary data in response to an instruction from the CPU 201. In S1403, the CPU 201 performs shooting control (for example, autofocus, exposure control, etc.) based on the subject information detected in S1402.

[0096] In S1404, the CPU 201 displays a warning prompting the user to select dictionary data on the display unit 216. Then, in S1405, the CPU 201 performs shooting control without using subject detection, and then ends the process.

[0097] The above embodiment is not limited to the case where dictionary data is necessarily selected by the dictionary user, and dictionary data may be set automatically. For example, the CPU 201 may automatically use the dictionary data with the highest priority at a fixed interval. Alternatively, the CPU 201 may use multiple dictionary data with high priority and use the dictionary data with the highest priority at a high interval.

[0098] In the above example, the shooting location is used as the shooting information, but other shooting information (shooting date, shooting time, weather) may be used to classify the managed shooting information. In this case, a single type of shooting information (e.g., shooting date) may be used, or multiple types of shooting information (e.g., shooting time and shooting location) may be combined and weighted for use.

[0099] As described above, in the above embodiment, images and shooting information used for learning are acquired, and a machine learning model is trained using the acquired images to generate dictionary data. In addition, for each generated dictionary data, shooting information of the image used for generating the dictionary data (i.e., for learning) is associated with the dictionary data. Then, when new shooting information for selecting dictionary data is acquired, one or more dictionary data are selected from a plurality of dictionary data based on the new shooting information and the shooting information for each dictionary data (for example, according to the similarity of the shooting information). In this way, it becomes possible to easily select desired dictionary data from existing dictionary data.

[0100] <Embodiment 2> An information processing system according to embodiment 2 will be described below. In embodiment 2, the dictionary selection screen on which a dictionary user selects dictionary data is different from embodiment 1, but other configurations, i.e., the system configuration, the camera 100, and the server 110, are similar to embodiment 1. For this reason, in this embodiment, the differences will be mainly described, and the same configurations as in embodiment 1 will be assigned the same numbers and redundant description will be omitted.

[0101] An example of a dictionary selection screen according to the second embodiment will be described with reference to Fig. 15. Reference numeral 1501 denotes a dictionary selection screen displayed on the display unit 216 by the camera 100. The camera 100 displays a list of dictionary data information transmitted from the server 110 as thumbnail images on the dictionary selection screen 1501. Reference numeral 1502 denotes a thumbnail image of the dictionary data. The thumbnail image is a thumbnail image that the server 110 manages in the dictionary data management unit in association with the dictionary data, and is an image that the server 110 has transmitted to the camera 100 together with the dictionary data ID (for example, in S1107). A dictionary user can select desired dictionary data using the touch panel of the display unit 216 or an operation member in the operation unit 204.

[0102] Camera 100 rearranges the dictionary data according to the priority of the dictionary data transmitted from server 110, and displays thumbnail images corresponding to the dictionary data. That is, camera 100 can display thumbnails of dictionary data in descending order of priority based on the priority of the shooting information obtained based on the shooting information for each dictionary data and the user's shooting information (new shooting information). By displaying in this manner, the dictionary user can select a dictionary according to the priority.

[0103] In the present embodiment, the case has been described as an example in which camera 100 rearranges dictionary data and displays corresponding thumbnail images in order to display the dictionary selection screen. However, server 110 may generate a list of dictionary data IDs arranged in order of priority of dictionary data, and server 110 may transmit the list and thumbnail images corresponding to each dictionary data ID in the list to camera 100. In this way, camera 100 can display the dictionary selection screen without rearranging the dictionary data (with less processing load).

[0104] As described above, in this embodiment, thumbnail images corresponding to dictionary data are displayed on the dictionary selection screen, and the displayed dictionary data are arranged in order of priority of the dictionary data, thereby enabling a dictionary user to easily select desired dictionary data from existing dictionary data.

[0105] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0106] (Disclosure of the present specification) The disclosure of this specification includes the following information processing device, control method, program, and storage medium. (Item 1) An information acquisition means for acquiring an image used for learning and shooting information relating to shooting of the image; A generating means for generating dictionary data that is information for identifying the behavior of a trained machine learning model by training the machine learning model using the acquired images; A management means for managing, for each of the generated dictionary data, shooting information of an image used to train a machine learning model related to the dictionary data in association with the dictionary data; and a selection means for selecting one or more dictionary data from among a plurality of dictionary data based on new shooting information and shooting information for each dictionary data when new shooting information for selecting dictionary data is acquired. (Item 2) 2. The information processing device according to item 1, wherein the selection means selects the one or more dictionary data in which a distribution of shooting information obtained for each dictionary data satisfies a predetermined condition with respect to the new shooting information. (Item 3) 3. The information processing device according to item 2, wherein the selection means selects the one or more dictionary data according to a similarity of shooting information obtained based on the shooting information for each dictionary data and the new shooting information. (Item 4) The information processing device according to any one of items 1 to 3, characterized in that the selection means determines a priority of the dictionary data based on the shooting information for each dictionary data and the new shooting information, and selects the one or more dictionary data according to the determined priority. (Item 5) 5. The information processing device according to any one of items 1 to 4, wherein the shooting information includes at least one of information relating to a location at the time of shooting, information relating to a time at the time of shooting, and information relating to a situation at the time of shooting. (Item 6) 6. The information processing device according to any one of items 1 to 5, wherein the information acquisition means acquires the shooting information from a file that contains the image. (Item 7) the information processing device is an imaging device, The information processing device according to any one of items 1 to 6, further comprising a display control means for displaying, on a display unit, a thumbnail based on at least one of the images used to train a machine learning model related to the dictionary data, for each of the dictionary data. (Item 8) 8. The information processing device according to item 7, wherein the display control means highlights the thumbnails to be displayed based on a similarity between the shooting information for each dictionary data and the new shooting information. (Item 9) The information processing device described in item 7, characterized in that the display control means displays thumbnails of the dictionary data in order of priority based on the priority of the shooting information obtained based on the shooting information for each dictionary data and the new shooting information. (Item 10) The shooting information includes information regarding the location at the time of shooting, The information processing device includes: an identifying unit that identifies, for each dictionary data, one location according to a distribution of the locations included in the shooting information for each dictionary data; providing means for providing the one or more dictionary data to the imaging device; The information processing device according to any one of items 1 to 6, wherein the providing means further provides the imaging device with the thumbnail for each of the dictionary data and the one location so as to display a thumbnail based on at least one of the images used to train a machine learning model related to the dictionary data in association with the one location on a map. (Item 11) the information processing device is an imaging device, The shooting information includes information regarding the location at the time of shooting, The information processing device includes: an identifying unit that identifies, for each dictionary data, one location according to a distribution of the locations included in the shooting information for each dictionary data; 7. The information processing device according to any one of items 1 to 6, further comprising a display control means for displaying, on a display unit, a thumbnail based on at least one of the images used to train a machine learning model related to the dictionary data in association with the one location on a map. (Item 12) the information processing device is an imaging device, The shooting information includes information regarding the location at the time of shooting, The information processing device includes a calculation unit that calculates a variance of the locations included in the shooting information for each of the dictionary data; and a display control means for displaying the dictionary data on a display unit in association with a location on a map, 7. The information processing device according to any one of claims 1 to 6, characterized in that the display control means displays the dictionary data with smaller variance calculated for each of the dictionary data as the scale at which the map is displayed becomes smaller. (Item 13) 7. The information processing device according to any one of items 1 to 6, further comprising a detection means for detecting a subject included in a newly captured image by using one of the one or more dictionary data selected based on the new shooting information and the shooting information for each dictionary data. (Item 14) A method for controlling an information processing device, comprising: an information acquiring step in which information acquiring means acquires an image used for learning and shooting information which is information related to shooting the image; A generation step in which a generation means generates dictionary data, which is information for specifying the operation of a trained machine learning model, by training the machine learning model using the acquired image; A management step in which the management means manages, for each of the generated dictionary data, shooting information of an image used to train a machine learning model related to the dictionary data in association with the dictionary data; a selection step of selecting one or more dictionary data from among a plurality of dictionary data based on new shooting information and the shooting information for each dictionary data when the selection step acquires new shooting information for selecting dictionary data. (Item 15) A program for causing a computer to function as each of the means of the information processing device according to any one of items 1 to 13. (Item 16) A storage medium for storing a program for causing a computer to function as each of the means of the information processing device according to any one of items 1 to 13.

[0107] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0108] 100: camera, 110: server, 201: CPU, 202: memory, 205: neural network processing unit, 401: CPU, 402: memory, 407: communication unit, 408: neural network processing unit

Claims

1. An information acquisition means for acquiring an image used for learning and shooting information relating to shooting of the image; A generating means for generating dictionary data that is information for identifying the behavior of a trained machine learning model by training the machine learning model using the acquired images; A management means for managing, for each of the generated dictionary data, shooting information of an image used to train a machine learning model related to the dictionary data in association with the dictionary data; An information processing device characterized by having a selection means for selecting one or more dictionary data from a plurality of dictionary data based on the new shooting information and the shooting information for each dictionary data when new shooting information for selecting dictionary data is acquired.

2. 2. The information processing apparatus according to claim 1, wherein the selection means selects the one or more dictionary data in which a distribution of shooting information obtained for each dictionary data satisfies a predetermined condition with respect to the new shooting information.

3. 3. The information processing apparatus according to claim 2, wherein the selection means selects the one or more dictionary data in accordance with a similarity of shooting information obtained based on shooting information for each of the dictionary data and the new shooting information.

4. 2. The information processing apparatus according to claim 1, wherein the selection means determines a priority of the dictionary data based on the shooting information for each of the dictionary data and the new shooting information, and selects the one or more dictionary data in accordance with the determined priority.

5. 2. The information processing apparatus according to claim 1, wherein the image capturing information includes at least one of information relating to a location at the time of image capturing, information relating to a time at the time of image capturing, and information relating to a situation at the time of image capturing.

6. 2. The information processing apparatus according to claim 1, wherein the information acquisition means acquires the photographing information from a file that contains the image.

7. the information processing device is an imaging device, The information processing device according to claim 1 , further comprising a display control means for causing a thumbnail based on at least one of the images used to train a machine learning model related to the dictionary data to be displayed on a display unit for each of the dictionary data.

8. 8. The information processing apparatus according to claim 7, wherein the display control means highlights the thumbnail to be displayed based on a similarity between the shooting information for each of the dictionary data and the new shooting information.

9. 8. The information processing apparatus according to claim 7, wherein the display control means displays thumbnails of the dictionary data in descending order of priority based on the priority of the shooting information obtained based on the shooting information for each of the dictionary data and the new shooting information.

10. The shooting information includes information regarding the location at the time of shooting, The information processing device includes: an identifying unit that identifies, for each dictionary data, one location according to a distribution of the locations included in the shooting information for each dictionary data; providing means for providing the one or more dictionary data to the imaging device; The information processing device according to claim 1, wherein the providing means further provides the imaging device with the thumbnail for each of the dictionary data and the one location so as to display a thumbnail based on at least one of the images used to train a machine learning model related to the dictionary data in association with the one location on a map.

11. the information processing device is an imaging device, The shooting information includes information regarding the location at the time of shooting, The information processing device includes: an identifying unit that identifies, for each dictionary data, one location according to a distribution of the locations included in the shooting information for each dictionary data; The information processing device according to claim 1, further comprising a display control means for causing a thumbnail based on at least one of the images used to train the machine learning model related to the dictionary data to be displayed on a display unit in association with the one location on a map.

12. the information processing device is an imaging device, The shooting information includes information regarding the location at the time of shooting, The information processing device includes a calculation unit that calculates a variance of the locations included in the shooting information for each of the dictionary data; and a display control means for displaying the dictionary data on a display unit in association with a location on a map, 2. The information processing apparatus according to claim 1, wherein the display control means displays the dictionary data with a smaller variance calculated for each of the dictionary data as the scale at which the map is displayed becomes smaller.

13. 2. The information processing device according to claim 1, further comprising a detection means for detecting a subject included in a newly captured image by using one of the one or more dictionary data selected based on the new shooting information and the shooting information for each of the dictionary data.

14. A method for controlling an information processing device, comprising: an information acquiring step in which information acquiring means acquires an image used for learning and shooting information which is information related to shooting the image; A generation step in which a generation means generates dictionary data, which is information for specifying the operation of the trained machine learning model, by training the machine learning model using the acquired image; A management step in which the management means manages, for each of the generated dictionary data, shooting information of an image used to train a machine learning model related to the dictionary data in association with the dictionary data; a selection step of selecting one or more dictionary data from among a plurality of dictionary data based on new shooting information and the shooting information for each dictionary data when the selection step acquires new shooting information for selecting dictionary data.

15. A program for causing a computer to function as each of the means of the information processing apparatus according to any one of claims 1 to 13.

16. A storage medium for storing a program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 13.

Images