Information processing device, electronic device, terminal device, information processing system, information processing method

Abstract

An information processing device collects image data, performs training processing using the image data to create a trained model for autofocus (AF), and outputs the trained model to a requesting device. The terminal device includes a communication function and acquires the trained model output by the information processing device. The electronic device includes an imaging function, acquires the trained model from the terminal device, and performs autofocus based on the trained model. When the electronic device performs autofocus to capture an image of an object and the expected part is not automatically in focus, the user registers the image data (out-of-focus image data) in the terminal device. The AF point is specified in the out-of-focus image data. The information processing device collects the out-of-focus image data from the terminal device, and performs training processing using the out-of-focus image data to create and output a trained model.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of Japanese Priority Patent Application JP 2019-176855 filed on Sep. 27, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to an information processing device, an electronic device, a terminal device, an information processing system, an information processing method, and a program. Background Art

[0004] For example, electronic devices represented by devices such as digital cameras and digital video cameras include on-board imaging devices such as charge coupled devices (CCDs) or complementary metal oxide semiconductor (CMOS) image sensors. The imaging device has an auto focus (AF) function that automatically adjusts the focus. With auto focus, the lens is driven so that the focus is appropriately adjusted, so that a focused (focused) image can be acquired.

[0005] Reference List

[0006] Patent Literature

[0007] PTL1:JP 2017-005443A Summary of the invention

[0008] Technical issues

[0009] However, with the auto focus of the prior art, the focus is not necessarily automatically adjusted to the user's desired spot depending on the object in question. In this case, the user must perform an operation to adjust the focus while shooting, and the user cannot shoot while concentrating on the composition and the correct moment. Therefore, there is a need for a technology to improve the auto focus function so that the focus is automatically adjusted to the user's desired spot without the user having to perform an operation for focus adjustment while shooting.

[0010] Therefore, the present disclosure proposes an information processing device, an electronic device, a terminal device, an information processing system, an information processing method, and a program capable of improving an auto focus function.

[0011] Solution to the problem

[0012] According to the present disclosure, an information processing device includes: a collection unit, which collects image data; a training unit, which uses a plurality of image data collected by the collection unit to extract an object position included in the image data, performs training processing on the relationship between the image data and the object position, and creates a trained model for automatic focus adjustment, wherein the trained model specifies an object from the image data to specify a focus position; and an output unit, which outputs the trained model created by the training unit to a device requesting the trained model.

[0013] An electronic device according to the present disclosure includes: a storage unit, which stores a trained model for automatic focus adjustment; a control unit, which performs automatic focus adjustment based on the trained model stored in the storage unit; and an optical system, which focuses on a predetermined part of an object according to the control of the control unit.

[0014] According to the present disclosure, a terminal device includes: a communication unit capable of communicating with an information processing device; a control unit, which acquires a trained model for automatic focus adjustment from the information processing device through the communication unit; and a providing unit, which provides the trained model acquired from the information processing device to an electronic device that performs automatic focus adjustment based on the trained model.

[0015] According to the present disclosure, the information processing system includes: an information processing device, which collects image data, uses the collected multiple image data to extract the object position included in the image data, performs training processing on the relationship between the image data and the object position, creates a trained model for automatic focus adjustment, and outputs the trained model to a device requesting the trained model, wherein the trained model specifies an object from the image data to specify a focus position; a terminal device, which obtains the trained model from the information processing device; and an electronic device, which obtains the trained model from the terminal device and performs automatic focus adjustment based on the trained model.

[0016] According to the present disclosure, the information processing system includes: an information processing device, which collects image data, uses the collected multiple image data to extract the object position included in the image data, performs training processing on the relationship between the image data and the object position, creates a trained model for automatic focus adjustment, and outputs the trained model to a device requesting the trained model, wherein the trained model specifies an object from the image data to specify a focus position; and an electronic device, which obtains the trained model from the information processing device and performs automatic focus adjustment based on the trained model.

[0017] The information processing method according to the present disclosure includes: collecting image data; using the collected multiple image data to extract the object position included in the image data, performing training processing on the relationship between the image data and the object position, and creating a trained model for automatic focus adjustment, wherein the trained model specifies the object from the image data to specify the focus position; and outputting the created trained model to a device requesting the trained model.

[0018] According to the program of the present disclosure, a computer is caused to perform processing, the processing including: collecting image data; using the collected multiple image data to extract the object position included in the image data, performing training processing on the relationship between the image data and the object position, and creating a trained model for automatic focus adjustment of an electronic device, the trained model specifying an object from the image data to specify a focus position; and outputting the created trained model to a device requesting the trained model. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a block diagram showing one example of the configuration of an information processing system according to the first embodiment of the present disclosure.

[0020] Figure 2 is a block diagram showing one example of the configuration of an electronic device and a terminal apparatus according to the first embodiment of the present disclosure.

[0021] Figure 3 is a block diagram showing one example of the configuration of an information processing apparatus according to the first embodiment of the present disclosure.

[0022] Figure 4 is a flowchart illustrating one example of a process for installing a trained model according to the first embodiment of the present disclosure.

[0023] Figure 5 is a flowchart showing one example of a process for confirming a captured image according to the first embodiment of the present disclosure.

[0024] Figure 6 It is a diagram for explaining the relationship between the degree of coloration and markings of animals and the AF detection rate.

[0025] Figure 7 This is a diagram for explaining a web screen for publishing image data and its aggregation status.

[0026] Figure 8 This is a diagram for explaining an example of specifying an AF point (distance measuring point) on an animal image.

[0027] Fig. 9 It is a diagram for explaining an example of specifying an AF point on an image of a vehicle.

[0028] Fig.10 It is a diagram for explaining confirmation of the degree of autofocus detection based on a trained model.

[0029] Fig.11 is a flowchart showing one example of a process for specifying an AF point before uploading image data.

[0030] Fig.12 This is a diagram for explaining an example in which a portion of image data used in network training for a model is reused in general network training for labeling.

[0031] Fig.13 is a block diagram showing one example of the configuration of an information processing system according to the second embodiment of the present disclosure.

[0032] Fig.14 is a block diagram showing one example of a configuration of an electronic device according to a second embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] Hereinafter, exemplary embodiments of the present disclosure will be described in detail based on the accompanying drawings. Note that in the following embodiments, the same symbols are used to represent the same parts, and repeated descriptions are omitted.

[0034] Hereinafter, the description will be made in the following order.

[0035] <1. First Embodiment>

[0036] <1-1. Configuration of System According to First Embodiment>

[0037] <1-1-1. Configuration of Electronic Device According to First Embodiment>

[0038] <1-1-2. Configuration of Terminal Device According to First Embodiment>

[0039] <1-1-3. Configuration of Information Processing Apparatus According to First Embodiment>

[0040] <1-2. Process for Installing Trained Model According to First Embodiment>

[0041] <1-3. Process for Confirming Photographed Image According to First Embodiment>

[0042] <1-4. Modification (1) of the First Embodiment>

[0043] <1-4-1. AF point specification before uploading image data>

[0044] <1-5. Modification (2) of the first embodiment>

[0045] <1-5-1. Semi-automatic training>

[0046] <2. Second Embodiment>

[0047] <2-1. Configuration of System According to Second Embodiment>

[0048] <2-1-1. Configuration of Electronic Device According to Second Embodiment>

[0049] <2-2. Process for installing a trained model according to the second embodiment>

[0050] <3. Conclusion>

[0051] <1. First Embodiment>

[0052] <1-1. Configuration of System According to First Embodiment>

[0053] Figure 1 1 is a block diagram showing an example of the configuration of an information processing system according to the first embodiment of the present disclosure. Figure 1 As shown in FIG. 1 , the information processing system 100 includes an electronic device 1, a terminal device 20, and an information processing device 30. The electronic device 1 and the terminal device 20 exist on the user side, and the information processing device 30 exists on the network side.

[0054] The electronic device 1 is a device having a function for capturing images including still images and / or moving images. The electronic device 1 is a device such as a digital camera, a digital video camera, a web camera, or a smart phone, a tablet computer, or a portable game console with an imaging function. The electronic device 1 may also be a device such as a surveillance camera, a vehicle-mounted camera, or a medical camera. The electronic device 1 is equipped with an autofocus (AF: automatic focus adjustment) function, and automatically focuses on (focuses on) an object when shooting. The electronic device 1 exchanges various data with the terminal device 20 via a universal serial bus (USB) or a secure digital (SD) memory card, or through wired or wireless communication. The electronic device 1 according to the present embodiment receives a trained model MD for autofocus from the terminal device 20. The electronic device 1 provides image data to the terminal device 20.

[0055] For example, the terminal device 20 is a computer such as a desktop personal computer (PC) or a laptop computer, a smart phone, a tablet computer, an AV device such as a digital TV or a video recorder and player, or a video game console (home console). The terminal device 20 has functions such as a computing function, a storage function, an output function, and a communication function. The terminal device 20 sends and receives various data with the information processing device 30 through the network NW. In the present embodiment, the terminal device 20 downloads the trained model MD from the information processing device 30. In addition, the terminal device 20 outputs the downloaded trained model MD to the electronic device 1. In addition, the terminal device 20 outputs the image data to the information processing device 30.

[0056] For example, the information processing device 30 is a server, such as a PC server or a graphics processing unit (GPU) server, or alternatively a mainframe or a workstation. In addition, the information processing device 30 may be one of a plurality of servers that implement distributed computing or cloud computing. The information processing device 30 is more powerful than the electronic device 1 and the terminal device 20. The information processing device 30 provides various services to the electronic device 1 and the terminal device 20 via the network NW. In the present embodiment, the information processing device 30 publishes the trained model MD on the network (web) in an accessible state via the network NW, and outputs it to the device that has requested the trained model MD. In addition, the information processing device 30 is also capable of collecting various image data via the network NW.

[0057] <1-1-1. Configuration of Electronic Device According to First Embodiment>

[0058] Figure 2 1 is a block diagram showing an example of the configuration of an electronic device and a terminal device according to the first embodiment of the present disclosure. Figure 2As shown in , the electronic device 1 is equipped with an optical system 2, a control unit 3, an image sensor 4, an image processing unit 5, a memory 6, a storage unit 7, a display unit 8, an I / F unit 9, and an input device 10.

[0059] The optical system 2 includes a main lens and a mechanism for driving the main lens, the main lens including one or more lenses. The optical system 2 forms image light (incident light) from the object OBJ onto the photosensitive surface of the image sensor 4 through the main lens. In addition, the optical system 2 is equipped with an autofocus mechanism that adjusts the focus according to a control signal and a zoom mechanism that changes the zoom factor according to a control signal. In addition, the optical system 2 can also be removed from the electronic device 1 and can be replaced with another optical system 2.

[0060] The image sensor 4 includes a plurality of photoelectric transducers arranged, for example, in a matrix of rows and columns. The photoelectric transducer converts received light into electric charge by photoelectric conversion. The image sensor 4 includes a drive circuit for driving the plurality of photoelectric transducers and a signal processing circuit, which reads out electric charge from each of the plurality of photoelectric transducers and generates raw data (unprocessed image data) based on the read out electric charge.

[0061] The image processing unit 5 is, for example, an image processing engine or the like, and performs predetermined image processing on the raw data output from the image sensor 4. For example, the image processing unit 5 is connected to a memory 6 such as a random access memory (RAM), and writes the raw data output from the image sensor 4 to the memory 6. In addition, the image processing unit 5 performs predetermined image processing on the raw data written to the memory 6, and writes the image data generated by the image processing to the memory 6. In addition, in the case where the terminal device 20 instead of the electronic device 1 performs predetermined image processing on the raw data, the image processing unit 5 can also provide the raw data to the terminal device 20 through the I / F unit 9.

[0062] For example, the storage unit 7 is a non-volatile memory such as a flash memory, and permanently stores the image data or raw data output from the image processing unit 5. For example, the display unit 8 includes a display device using a display such as a liquid crystal display (LCD) or an organic electroluminescent display (OEL) and a drive circuit that drives the display device. The display unit 8 is capable of displaying an image based on the image data output from the image processing unit 5. The I / F unit 9 is an interface for exchanging various data with the terminal device 20. For example, an interface such as a USB port or an SD memory card slot can be used as the I / F unit 9. In addition, the I / F unit 9 may also be an interface capable of communicating via wired or wireless communication. For example, the I / F unit 9 may also be an interface that supports a short-range wireless communication standard.

[0063] The input device 10 includes operable elements for receiving user input, etc. If the electronic device 1 is a device such as a digital camera, a digital video camera, or a smart phone, a tablet computer, or a portable game console with an imaging function, the input device 10 may include a shutter button for instructing the image sensor 4 to perform imaging, or an operable element for realizing the function of the shutter button.

[0064] For example, the control unit 3 is a processor such as a central processing unit (CPU). The control unit 3 controls the overall operation of the electronic device 1 by following a program pre-stored in a read-only memory (ROM) and using a cache memory or a RAM as a working memory. Note that all or part of the control unit 3 may include hardware such as a system on a chip (SOC), an application-specific integrated circuit (ASIC), or a field programmable gate array (FPGA). For example, the control unit 3 is capable of controlling the operation of the electronic device 1 according to a user input received by the input device 10. In addition, the control unit 3 is capable of providing the original data or image data stored in the storage unit 7 to the terminal device 20 through the I / F unit 9. In addition, the control unit 3 is capable of storing the trained model MD provided from the terminal device 20 through the I / F unit 9 in the storage unit 7, and controlling the autofocus mechanism of the optical system 2 based on the trained model MD stored in the storage unit 7.

[0065] <1-1-2. Configuration of Terminal Device According to First Embodiment>

[0066] like Figure 2 As shown in , the terminal device 20 is equipped with an I / F unit 21 , an AS control unit 22 , an AS storage unit 23 , a communication unit 24 , an input unit 25 , and a display unit 26 .

[0067] The I / F unit 21 is an interface for exchanging various data with the I / F unit 9 of the electronic device 1. In other words, the I / F unit 21 is an interface corresponding to the I / F unit 9. An example of the I / F unit 21 is similar to the I / F unit 9.

[0068] For example, the AS control unit 22 is a processor such as a CPU, which controls the overall operation of the terminal device 20 by following a program pre-stored in the AS storage unit 23 and using a cache memory or a RAM as a working memory. In addition, the AS control unit 22 executes dedicated application software AS for exchanging image data with the information processing device 30, and registers image data to be uploaded to the information processing device 30. In addition, the AS control unit 22 downloads the trained model MD from the information processing device 30.

[0069] For example, the AS storage unit 23 is a storage device such as a solid state drive (SSD) or a hard disk drive (HDD), or alternatively a non-volatile memory such as a flash memory. The AS storage unit 23 stores information such as the dedicated application software AS executed by the AS control unit 22 and the image data to be uploaded to the information processing device 30. The AS storage unit 23 can be built into the terminal device, or can be an externally attached storage device. In addition, the AS storage unit 23 can also be a combination of a storage medium such as an optical disk and a device (driver) for driving the storage medium. For example, the dedicated application software AS can be stored in a storage medium such as an optical disk.

[0070] The communication unit 24 is an interface that can be connected to the network NW by wired or wireless communication. For example, a network adapter such as a network interface card (NIC), an antenna, etc. can be applied as the communication unit 24.

[0071] For example, the input unit 25 is an input device such as a keyboard and a mouse, a keypad or other buttons. The input unit 25 outputs a signal corresponding to the user's input operation to the AS control unit 22. In addition, the input unit 25 may also be a software key displayed on the display unit 26 and a touch panel linked to the display unit 26. For example, the display unit 26 includes an image processing processor such as a GPU and a display device such as an LCD or an OELD, and displays information such as an image according to an output signal from the AS control unit 22.

[0072] <1-1-3. Configuration of Information Processing Apparatus According to First Embodiment>

[0073] Figure 3 1 is a block diagram showing an example of the configuration of an information processing device according to the first embodiment of the present disclosure. Figure 3 As shown in , the information processing device 30 is equipped with a communication unit 31 , a control unit 32 , and a storage unit 33 .

[0074] The communication unit 31 is an interface that can be connected to the network NW by wired or wireless communication. The communication unit 31 transmits and receives various data with the communication unit 24 of the terminal device 20 through the network NW. An example of the communication unit 31 is similar to the communication unit 24.

[0075] The control unit 32 includes a processor such as a CPU or a GPU, and controls the overall operation of the information processing device 30 by following a program pre-stored in the storage unit 33 and using a cache memory or a RAM as a working memory. In addition, the control unit 32 performs a training process for improving autofocus accuracy.

[0076] The control unit 32 includes a collection unit 321, a labeling unit 322, a training unit 323, an evaluation unit 324, and an output unit 325. The collection unit 321 collects image data from the terminal device 20 and other information sources via the network NW through the communication unit 31. The labeling unit 322 checks the validity of the image data collected by the collection unit 321 and labels the image data. The training unit 323 performs a training process for improving the autofocus accuracy of the electronic device 1 based on the image data collected by the collection unit 321, and generates a trained model MD as a training result. The training unit 323 can also be implemented using a GPU. Because the GPU can perform simple calculations in parallel, the GPU is particularly more suitable for deep learning than the CPU. The evaluation unit 324 evaluates the trained model MD generated by the training unit 323. The output unit 325 outputs the trained model MD in a state that can be used for downloading through the network NW. For example, the output unit 325 also outputs the results calculated by each unit, such as aggregated data about the image data collected by the collection unit 321 and the image data.

[0077] For example, the storage unit 33 is a storage device such as an SSD or an HDD, and stores information such as image data collected through the network NW, various programs executed by the control unit 32, and various data generated due to the processing of the control unit 32. The storage unit 33 can also be independent of the information processing device 30 as a dedicated file server. The storage unit 33 includes an image storage unit 331, an AF point storage unit 332, and a model storage unit 333. The image storage unit 331 stores the image data collected by the collection unit 321. The model storage unit 333 stores the trained model MD generated by the training unit 323.

[0078] <1-2. Process for installing a trained model according to the first embodiment>

[0079] Next, we will use Figure 4 An example of a process for installing a trained model according to the first embodiment of the present disclosure is described. Figure 4 is a flowchart showing an example of a process for installing a trained model according to the first embodiment of the present disclosure. Figure 4 The processing performed by the terminal device 20 and the processing performed by the information processing device 30 are shown as a single processing, but the processing performed by the information processing device 30 may be performed separately from the processing performed by the terminal device 20. Specifically, the processing may be performed independently. Figure 4 The processing from steps S103 to S111.

[0080] The user starts the dedicated application software AS on the terminal device 20 (step S101 ). At this time, the AS control unit 22 reads out and executes the dedicated application software AS stored in the AS storage unit 23 according to the signal from the input unit 25 , and displays a window of the dedicated application screen on the display unit 26 .

[0081] The user uses the input unit 25 to perform an operation on the dedicated application screen displayed on the display unit 26, and registers the image data of the object OBJ that the user wants to recognize through the autofocus of the electronic device 1 as the training image data GD (step S102). The image data of the object OBJ can be image data pre-stored in the AS storage unit 23 or image data acquired through the I / F unit 21 or the communication unit 24. At this time, the AS control unit 22 uploads the training image data GD to the information processing device 30 through the communication unit 24 according to the signal from the input unit 25. Note that the AS control unit 22 can also store the training image data GD in a folder for uploading that has been prepared in the AS storage unit 23. For example, the AS control unit 22 can periodically upload the training image data GD stored in the folder for uploading to the information processing device 30 through the function of the dedicated application software AS.

[0082] Next, the processing performed by the information processing device 30 to which the image data has been uploaded will be described. The information processing device 30 collects the training image data GD uploaded from the terminal device 20 in which the dedicated application software AS is installed (step S103). Note that the information processing device 30 may also loop through a plurality of terminal devices 20 in which the dedicated application software AS is installed according to a technique such as crawling, and collect the training image data GD stored in a folder prepared for uploading in the AS storage unit 23. At this time, the collection unit 321 receives the training image data GD from the terminal device 20 via the network NW through the communication unit 31, and stores the received training image data GD in the image storage unit 331. In addition, the collection unit 321 may also collect image data publicly available on the network NW as the training image data GD through the communication unit 31, and store the collected training image data GD in the image storage unit 331. In addition, the collection unit 321 may also collect image data stored in a storage medium such as an optical disk as the training image data GD.

[0083] The collection unit 321 automatically divides the training image data GD uploaded from the terminal device 20 into categories and types. The output unit 325 creates web-accessible data about the training image data GD and its aggregation status (step S104). The creation of data in step S104 can be performed each time an image is collected, but can also be performed in the case of accessing public information. Figure 7As shown in , the user is able to check the training image data GD collected by the information processing device 30 and its aggregation status on the public screen 261 on the dedicated application screen displayed on the display unit 26. At this time, the user is able to check not only the training image data GD uploaded by the user himself, but also the training image data GD uploaded by other users. In addition, the user is also able to set the training image data GD uploaded by himself as private. The training image data GD that has been set to private is not disclosed to other users, but is used for training processing together with the public training image data GD.

[0084] Next, the collection unit 321 specifies an AF point in the collected image data (step S105). The collection unit 321 reads out information about the AF point specified by the user in the image data. The information processing device 30 can also perform image analysis on the image data to specify the AF point. In addition, the information processing device 30 can also register information about the AF point for an image that has not been registered by the user, or in other words, register information about the AF point for image data that has been stored in a database. Hereinafter, an example of a process of specifying an AF point in image data will be described. The user uses the input unit 25 to perform an operation on a dedicated application screen displayed on the display unit 26, and specifies the desired position to be focused on in the training image data GD collected by the information processing device 30 as an AF point P (ranging point). At this time, the AS control unit 22 displays the registered training image data GD on the dedicated application screen displayed on the display unit 26, and specifies the AF point P by moving the AF point P displayed above the training image data GD according to a signal from the input unit 25. For example, as Figure 7 As shown in , when desired training image data GD is selected from among the training image data GD publicly available on the public screen 261 , the AS control unit 22 displays the training image data GD and the AF point P on the AF point designation screen 262 .

[0085] In this embodiment, if Figure 8 and Fig. 9 As shown in , the user can specify and decide the priority order of multiple AF points P with respect to a single training image data GD. Any number of AF points P can be specified. In the present embodiment, the user specifies up to three AF ​​points P in the order of a first priority AF point P1, a second priority AF point P2, and a third priority AF point P3. Figure 8 An example of training image data GD_D1 of an animal (dog) is shown, and Fig. 9 An example of training image data GD_C1 of a vehicle (racing car) is shown. Figure 8As shown in (Ex. 1), for the uploaded training image data GD_D1 of an animal (dog), a plurality of AF points P are displayed on the web in advance, and the user can specify the AF point P by moving the displayed AF point P. The user may also be able to specify how many AF points P are to be displayed. Alternatively, as Figure 8 As shown in (Ex.2) of FIG. 1 , for the illustration image data GD_D2 of the animal (dog) corresponding to the training image data GD_D1 of the animal (dog), a plurality of AF points P may be displayed in advance on the web. Fig. 9 As shown in (Ex. 1) and (Ex. 2) of FIG. 1 , the positions and priority order of these AF points P can be freely changed by the user.

[0086] The control unit 32 associates the AF point data PD indicating the AF point P specified for the training image data GD with the training image data GD (step S106). The control unit 32 stores the associated data in the AF point storage unit 332. For example, the AF point data PD includes an identifier, a priority, and parameter-related information such as a focus position and an offset of each AF point P. The AF point data PD may be stored together with identification information about the training image data GD in a record that forms a table within a database.

[0087] The labeling unit 322 checks the validity of the training image data GD, the AF point data PD, the classification, etc., and labels the image data (step S107). The labeling unit 322 can also receive labeling related instructions from an external source through the communication unit 31. The labeling includes information such as information indicating the image classification and information associated with the AF point data PD.

[0088] The collection unit 321 confirms whether a predetermined number of training image data GD has been collected for each category and type (step S108). The predetermined number is not particularly limited, but as an example, it can be 10,000 images. In addition, in the case where a trained model can be created using a small amount of image data, the predetermined number can also be a number of images. In the case where the predetermined number has not yet been collected (step S108, no), the collection unit 321 continues to collect training image data GD (return to step S103). In other words, the information processing device 30 does not create a trained model until a predetermined number of image data is collected for the classification in question.

[0089] When a predetermined number of images have been collected for each category and type of training image data GD (step S108, yes), the collection unit 321 regards multiple training image data GD and their AF point data PD as data sets for training, and instructs the training unit 323 to start training processing (proceed to step S109).

[0090] The training unit 323 performs a training process based on a plurality of training image data GD and AF point data PD associated with each training image data GD, and creates a trained model MD as a result of the training process (step S109). For example, the training unit 323 checks an algorithm for detecting an AF point P, and generates a detection code and a parameter as a trained model MD. In this way, the training unit 323 performs a training process by using the training image data GD having the specified AF point P. In addition, the training unit 323 creates a trained model MD for each category and type. At this time, the training unit 323 may create a trained model MD for each user, or a trained model MD shared by all users, for each category and type. The trained model MD for each user is a trained model MD that preferentially detects the AF point P specified by the user. The trained model MD shared by all users is a trained model MD that detects an AF point P similar to the greatest common factor among the AF points P specified by each user. For example, the trained model MD shared by all users may be a trained model MD that detects the AF point P specified by most users. The created trained model MD is stored in the model storage unit 333 .

[0091] The evaluation unit 324 performs a process of evaluating the trained model MD stored in the model storage unit 333 (step S110). For example, the evaluation unit 324 inputs the training image data GD into the trained model MD, and receives the result from the trained model MD as an output. In the case where the result of the evaluation process is determined to be unsatisfactory (step S110, no), the evaluation unit 324 instructs the training unit 323 to start the training process again and recreate the trained model MD (return to step S109). On the other hand, in the case where the result of the evaluation process is determined to be satisfactory (step S110, yes), the evaluation unit 324 instructs the output unit 325 to publish the trained model MD (proceed to step S111).

[0092] The output unit 325 publishes the trained model MD on the network NW in a state that it can be accessed only from the terminal device 20 installed with the dedicated application software AS, and outputs it to the device requesting the trained model MD (step S111). At this time, the output unit 325 assigns an access right to the trained model MD, the access right making the trained model MD accessible only from the terminal device 20 installed with the dedicated application software AS or through the dedicated application software AS; publishes the trained model MD on the network NW through the communication unit 31; and outputs it to the device requesting the trained model MD. For example, the output unit 325 can display the trained model MD in a selectable state only on the application screen displayed by executing the dedicated application software AS.

[0093] The user uses the input unit 25 to perform an operation on the dedicated application screen displayed on the display unit 26, selects and downloads the trained model MD, and checks the degree (accuracy) of autofocus detection of the autofocus based on the downloaded trained model MD (step S112). At this time, the AS control unit 22 acquires the trained model MD through the communication unit 24 according to the signal from the input unit 25, and stores the acquired trained model MD in the AS storage unit 23. Subsequently, the AS control unit 22 inputs the training image data GD into the trained model MD stored in the AS storage unit 23 on the dedicated application screen displayed on the display unit 26 according to the signal from the input unit 25, and as shown in FIG. Fig.10 The result from the trained model MD is displayed as shown in . Note that the AS control unit 22 can also perform simulation of autofocus of the electronic device 1 based on the trained model MD stored in the AS storage unit 23 on the dedicated application screen displayed on the display unit 26 .

[0094] Note that, in practice, before downloading the trained model MD, the degree of autofocus detection based on the trained model MD may be confirmed on the information processing device 30 side rather than the terminal device 20 side. In this case, the control unit 32 inputs the training image data GD into the trained model MD selected by the user via the network NW through the communication unit 31, and outputs the result from the trained model MD to the terminal device 20. The AS control unit 22 displays the result of the trained model MD output from the information processing device 30 on the dedicated application screen displayed on the display unit 26.

[0095] In the case where the result of confirming the degree of autofocus detection based on the trained model MD is determined to be unsatisfactory (step S112, No), the user does not adopt the trained model MD. At this time, the AS control unit 22 may also delete the trained model MD stored in the AS storage unit 23. On the other hand, in the case where the result of confirming the degree of autofocus detection based on the trained model MD is determined to be satisfactory (step S112, Yes), the user adopts the trained model MD (proceeding to step S113).

[0096] The user installs the trained model MD in the electronic device 1 (step S113). At this time, the AS control unit 22 writes the trained model MD stored in the AS storage unit 23 to the electronic device 1 through the I / F unit 21 according to the signal from the input unit 25. The control unit 3 acquires the trained model MD through the I / F unit 9 and stores the trained model MD in the storage unit 7, and sets the trained model MD for autofocus. In the case where a trained model MD of the same category or type has been set, the trained model MD is updated. This series of processing ends here.

[0097] The electronic device 1 does not need to update all (all categories / all kinds) of trained models MD stored in the information processing device 30 at once, and it is sufficient for the electronic device 1 to acquire the trained model MD corresponding to the desired mode based on the user selection. With this arrangement, the user can selectively install only the trained model MD of the object that the user himself desires to photograph in the electronic device 1.

[0098] <1-3. Process for Confirming Photographed Image According to First Embodiment>

[0099] Next, we will use Figure 5 An example of a process for confirming a captured image according to the first embodiment of the present disclosure will be described. Figure 5 is a flowchart showing one example of a process for confirming a captured image according to the first embodiment of the present disclosure.

[0100] The user uses the electronic device 1 installed with the trained model MD to shoot the object OBJ (step S201). The electronic device 1 performs autofocus based on the newly written trained model MD and focuses on a predetermined position on the object OBJ. At this time, the control unit 3 outputs a control signal for controlling the autofocus mechanism of the optical system 2 based on the trained model MD stored in the storage unit 7 according to the signal from the input device 10. Thereafter, the electronic device 1 images the object OBJ and generates image data (which may also be raw data).

[0101] The user checks whether there is image data (out-of-focus image data) in which the expected part (portion) is not in focus among the image data of the object OBJ captured by the electronic device 1 (step S202). If there is no out-of-focus image data (step S202, No), the process ends.

[0102] In the case where out-of-focus image data exists (step S202, yes), the user registers the out-of-focus image data as new training image data GD (proceeding to Figure 4In other words, the user starts the dedicated application software AS (corresponding to the dedicated application software AS) on the terminal device 20 using the input unit 25. Figure 4 In step S101), an operation is performed on the dedicated application screen displayed on the display unit 26, and new training image data GD (corresponding to Figure 4 At this time, the AS control unit 22 reads out and executes the dedicated application software AS stored in the AS storage unit 23 according to the signal from the input unit 25, displays the window of the dedicated application screen on the display unit 26, obtains the training image data GD through the I / F unit 21 or the communication unit 24, and stores the obtained training image data GD in the AS storage unit 23. Thereafter, the process proceeds to Figure 4 The processing in step S103.

[0103] By repetition Figure 4 The process for installing a trained model and Figure 5 By the process for confirming the captured image shown in , the accuracy of the trained model MD is cumulatively improved, so the accuracy of autofocus can be improved.

[0104] <1-4. Modification (1) of the First Embodiment>

[0105] <1-4-1. AF point specification before uploading image data>

[0106] In the above description, Figure 4 In step S105, the AF point P is specified for the training image data GD on the information processing device 30, but in addition, Figure 4 In step S102 , the AF point P may be designated for the training image data GD on the terminal device 20 before uploading the training image data GD.

[0107] Will use Fig.11 The following describes the modification (1) according to the first embodiment. Figure 4 This is an example of the processing in step S102. Fig.11 It is shown Figure 4 Flowchart of one example of the processing in step S102.

[0108] The user performs an operation on the dedicated application screen displayed on the display unit 26 using the input unit 25, and registers the image data of the object OBJ that the user wishes to recognize by the autofocus of the electronic device 1 as the training image data GD (step S102A). The image data of the object OBJ may be image data pre-stored in the AS storage unit 23 or image data acquired through the I / F unit 21 or the communication unit 24.

[0109] The user performs an operation on the dedicated application screen displayed on the display unit 26 using the input unit 25, and specifies a desired position to be focused on in the registered training image data GD as the AF point P (step S102B). At this time, the AS control unit 22 displays the registered training image data GD on the dedicated application screen displayed on the display unit 26, and specifies the AF point P by moving the AF point P displayed on the training image data GD according to the signal from the input unit 25. The display and specification of the AF point P are as follows: Figure 8 and Fig. 9 as shown in .

[0110] The AS control unit 22 associates the AF point data PD indicating the AF point P specified for the training image data GD with the training image data GD, and stores the associated data in the AS storage unit 23 (step S102C). The AF point data PD is as described above. Note that the AF point data PD may also incorporate the training image data GD as metadata.

[0111] The AS control unit 22 uploads the training image data GD together with the associated AF point data PD to the information processing device 30 through the communication unit 24 according to the signal from the input unit 25 (step S102D).

[0112] In this case, Figure 4 In step S103, the information processing device 30 collects the training image data GD and the AF point data PD uploaded from the terminal device 20 on which the dedicated application software AS is installed. At this time, the collecting unit 321 stores the training image data GD received from the terminal device 20 via the network NW through the communication unit 31 in the image storage unit 331, and stores the associated AF point data PD in the AF point storage unit 332.

[0113] With this arrangement, AF points can be specified in two stages, before and after uploading the training image data GD. Before uploading, the user can specify AF points P for the training image data GD that the user owns, and after uploading, the user can correct the AF points P of the training image data GD that the user uploaded, or specify AF points P for the training image data GD uploaded by other users.

[0114] Furthermore, it is expected that visually checking all the training image data GD published together with the aggregated state and specifying the AF point P on the information processing device 30 will be extremely tiring for a single user. Therefore, the burden on the user side can be reduced by having the user specify the AF point P for the training image data GD that the user has before uploading. With this arrangement, each published training image data GD has at least the AF point P specified by the uploading user. If the user is not satisfied with the AF point P specified by the user who uploaded the training image data GD, by having each user give approval, the AF point P specified by the user himself can be set.

[0115] <1-5. Modification (2) of the first embodiment>

[0116] <1-5-1. Semi-automatic training>

[0117] The information processing device 30 can also perform the labeling process performed by the labeling unit 322 and the training process performed by the training unit 323 in parallel. Fig.12 As shown in , in order to create a trained model MD, training image data GD collected from the user is used in network training to create a trained model for the user. However, by reusing part (or all) of the training image data GD in general network training for automatic annotation, automation of annotation can be achieved. Network training refers to machine learning using neural networks, and also includes deep learning using deep neural networks. General network training can be performed by using multiple methods in parallel. Note that manual annotation is performed in cases where it cannot be detected by automatic annotation. With this arrangement, semi-automatic training becomes possible.

[0118] <2. Second Embodiment>

[0119] <2-1. Configuration of System According to Second Embodiment>

[0120] Fig.13 1 is a block diagram showing an example of the configuration of an information processing system according to a second embodiment of the present disclosure. Fig.13 As shown in FIG. 1 , the information processing system 100 includes an electronic device 1 and an information processing apparatus 30. The electronic device 1 exists on the user side, and the information processing apparatus 30 exists on the network side.

[0121] In the second embodiment of the present disclosure, for example, a configuration corresponding to the terminal device 20 in the first embodiment of the present disclosure is set in the electronic device 1, such as in the form of an application processor. In other words, the electronic device 1 also includes the function of the terminal device 20. In the present embodiment, the electronic device 1 is expected to be a high-end device model, such as a smart phone or a tablet computer with an imaging function, or alternatively a digital camera or a digital video camera with a communication function. The information processing device 30 is similar to the first embodiment. In other words, the electronic device 1 according to the second embodiment of the present disclosure and the terminal device 20 according to the first embodiment of the present disclosure can be connected to the same information processing device 30 via the network NW. Therefore, the first embodiment and the second embodiment of the present disclosure can be executed simultaneously.

[0122] <2-1-1. Configuration of Electronic Device According to Second Embodiment>

[0123] Fig.14 1 is a block diagram showing an example of a configuration of an electronic device according to a second embodiment of the present disclosure. Fig.14 As shown in , the electronic device 1 is equipped with an optical system 2, a control unit 3, an image sensor 4, an image processing unit 5, a memory 6, a storage unit 7, a display unit 8, a communication unit 11, and an input device 10.

[0124] The operations of the optical system 2, the control unit 3, the image sensor 4, the image processing unit 5, the memory 6, the storage unit 7, the display unit 8, and the input device 10 are basically similar to those in the first embodiment. In addition, in the present embodiment, the control unit 3 corresponds to the AS control unit 22 in the first embodiment, and the storage unit 7 corresponds to the AS storage unit 23 in the first embodiment. In other words, the control unit 3 executes the dedicated application software AS, and the storage unit 7 stores the dedicated application software AS. In addition, the display unit 8 corresponds to the display unit 26 in the first embodiment, and the input device 10 corresponds to the input unit 25 in the first embodiment. The communication unit 11 corresponds to the communication unit 24 in the first embodiment. In other words, the communication unit 11 is an interface capable of connecting to the network NW by wired or wireless communication.

[0125] <2-2. Process for installing a trained model according to the second embodiment>

[0126] Next, in a manner similar to the first embodiment, Figure 4 An example of the process for installing a trained model according to the second embodiment is described. In this embodiment, since the functions of the terminal device 20 are built into the electronic device 1, the electronic device 1 itself also performs the role of the terminal device 20, and thus the agent that performs the operation changes, but other than that, the basic processing is the same.

[0127] exist Figure 4 In step S101, the user starts the dedicated application software AS on the electronic device 1. At this time, the control unit 3 reads out and executes the dedicated application software AS stored in the storage unit 7 according to the signal from the input device 10, and displays the window of the dedicated application screen on the display unit 8.

[0128] exist Figure 4 In step S102, the user performs an operation on the dedicated application screen displayed on the display unit 8 using the input device 10, and registers the image data of the object OBJ that the user wants to recognize by the autofocus of the electronic device 1 as the training image data GD. At this time, the control unit 3 uploads the training image data GD to the information processing device 30 through the communication unit 11 according to the signal from the input device 10.

[0129] exist Figure 4 In step S103, the information processing device 30 collects the training image data GD uploaded from the electronic device 1 installed with the dedicated application software AS. At this time, the collecting unit 321 receives the training image data GD from the electronic device 1 via the network NW through the communication unit 31, and stores the received training image data GD in the image storage unit 331.

[0130] exist Figure 4 In step S104, the collecting unit 321 automatically divides the training image data GD uploaded from the electronic device 1 into categories and types, and publishes the training image data GD and its aggregation state on the network (web) (step S104).

[0131] exist Figure 4 In step S105, the user performs an operation on the dedicated application screen displayed on the display unit 8 using the input device 10, and specifies a desired position to be focused on in the training image data GD collected by the information processing apparatus 30 as the AF point P. At this time, the control unit 3 displays the registered training image data GD on the dedicated application screen displayed on the display unit 8, and specifies the AF point P by moving the AF point P displayed on the training image data GD according to a signal from the input device 10.

[0132] exist Figure 4 In step S106 , the control unit 32 associates AF point data PD indicating the AF point P specified for the training image data GD with the training image data GD, and stores the associated data in the AF point storage unit 332 .

[0133] exist Figure 4 In step S107, the labeling unit 322 checks the validity of the training image data GD and the AF point data PD, and labels the image data.

[0134] exist Figure 4 In step S108, the collecting unit 321 confirms whether a predetermined number of training image data GD (e.g., 10,000 images) has been collected for each category and type. In the case where the predetermined number has not been collected (step S110, No), the collecting unit 321 continues to collect the training image data GD (returning to step S103). On the other hand, in the case where a predetermined number of images have been collected for each category and type of the training image data GD (step S110, Yes), the collecting unit 321 regards the plurality of training image data GD and their AF point data PD as a data set for training, and instructs the training unit 323 to start the training process (proceeding to step S109).

[0135] exist Figure 4 In step S109 , the training unit 323 performs a training process based on the plurality of training image data GD and the AF point data PD associated with each training image data GD, and creates a trained model MD as a result of the training process.

[0136] exist Figure 4 In step S110, the evaluation unit 324 performs a process of evaluating the trained model MD stored in the model storage unit 333. In the case where the result of the evaluation process is determined to be unsatisfactory (step S110, no), the evaluation unit 324 instructs the training unit 323 to start the training process again and recreate the trained model MD (return to step S109). On the other hand, in the case where the result of the evaluation process is determined to be satisfactory (step S110, yes), the evaluation unit 324 instructs the output unit 325 to publish the trained model MD (proceed to step S111).

[0137] exist Figure 4 In step S111 , the output unit 325 publishes the trained model MD on the network NW in a state accessible only from the electronic device 1 on which the dedicated application software AS is installed, and outputs to the device requesting the trained model MD.

[0138] exist Figure 4In step S112, the user uses the input device 10 to perform an operation on the dedicated application screen displayed on the display unit 8, selects and downloads the trained model MD, and checks the degree of autofocus detection of the autofocus based on the downloaded trained model MD. At this time, the control unit 3 obtains the trained model MD through the communication unit 11 according to the signal from the input device 10 to store the trained model MD in the storage unit 7, inputs the training image data GD into the trained model MD stored in the storage unit 7 on the dedicated application screen displayed on the display unit 8, and displays the result from the trained model MD.

[0139] In the case where the result of confirming the degree of autofocus detection based on the trained model MD is determined to be unsatisfactory (step S112, No), the user does not adopt the trained model MD. At this time, the control unit 3 may also delete the trained model MD stored in the storage unit 7. On the other hand, in the case where the result of confirming the degree of autofocus detection based on the trained model MD is determined to be satisfactory (step S112, Yes), the user adopts the trained model MD (proceeds to step S113).

[0140] exist Figure 4 In step S113, the user installs the trained model MD in the electronic device 1. At this time, the control unit 3 sets the trained model MD stored in the storage unit 7 to be used for autofocus according to the signal from the input device 10. This series of processing ends here.

[0141] Also, in the present embodiment, considering the storage capacity of the electronic device 1, it is not realistic to install all (all categories / all kinds) of trained models MD stored in the information processing device 30 to the electronic device 1 at once. However, in the present embodiment, since the electronic device 1 can directly download the trained model MD from the information processing device 30, even when the user encounters an object that was not expected in advance, the user can download the trained model MD for the object from the information processing device 30 and install the downloaded trained model MD in the electronic device 1 on the spot (in real time). In addition, even in the case where the category and kind of the object are unclear, by photographing the object and uploading the image data to the information processing device 30 as the training image data GD, the category and kind of the object can be determined on the screen where the aggregation status is posted.

[0142] <3. Conclusion>

[0143] As described above, according to each embodiment of the present disclosure, the autofocus function can be improved and a specific object can be detected according to the needs of the user. In addition, the user's desired part (part) on the object can be automatically focused. Therefore, the user becomes able to shoot while focusing on the composition and the correct moment.

[0144] In the case where the object is a living body such as a person or an animal, the electronic device 1 uses an autofocus function to identify the target object and focuses on the eyes of the living body. However, in the case where the object is an animal (such as a bird, a fish, or particularly an insect), there are many kinds of objects, in which the body shape, pattern, marking, etc. vary greatly. Therefore, it is difficult for a single electronic device 1 to learn all these varieties and maintain the performance for focusing on the eyes for all varieties.

[0145] In addition, if Figure 6 As shown in , in the case of a pet (such as a dog or a cat), the detection rate decreases depending on the degree of body coloring (such as dark black) and markings. For example, for Group A of ordinary, clear objects, the detection rate of the object's eyes is about 80%. However, for Group B of dark black objects, it is difficult to distinguish between the eyes and the pattern, and the detection rate of the object's eyes drops to about 50%. In addition, in the case of a cat with a complex pattern, the undetectable rate may increase further. Therefore, in the case where a pet cannot be detected in autofocus, a single user cannot perform appropriate training processing.

[0146] In addition, in the case where the object is a plant or an artificial object (such as a vehicle or a building), the target object is identified and focused on a preset part (portion). For example, in the case where the object is a car, the focus is on the headlights, etc. However, in this case, similar to the above case of animals, etc., because there are a large number of changes in shape, color, etc., it is difficult for a single electronic device 1 to learn all these changes, set the part to be focused on, and maintain the performance for focusing on the set part for all changes.

[0147] Furthermore, although the performance of the electronic device 1 is improving year by year, the commonly used electronic device 1 has not yet reached the same performance level as the information processing device 30 in terms of processing power and the like. Furthermore, since the storage capacity of the electronic device 1 is small compared to the information processing device 30, there is a limit on how much image data can be saved at one time by a single electronic device 1. For these reasons, even if a training process for improving the accuracy of autofocus is performed based on the image data saved in a single electronic device 1, it is not possible to obtain a training result sufficient to satisfy the user.

[0148] This embodiment performs training processing for improving the autofocus accuracy on the information processing device 30, which has higher performance than the electronic device 1 and can store and process a larger amount of image data than the electronic device 1. In addition, by providing the training result, that is, the trained model MD, as feedback to the electronic device 1, a high-precision trained model MD that cannot be obtained by using the electronic device 1 alone can be installed in the electronic device 1, thereby improving the autofocus accuracy.

[0149] For example, the information processing device 30 collects a large amount of image data, and at a point where the amount of collected image data reaches a predetermined amount, the information processing device 30 performs a training process for improving the autofocus accuracy of the electronic device 1 based on the image data, and provides the trained model MD generated as a result of the training to the terminal device 20 through the network NW. The user of the electronic device 1 uses the terminal device 20 to check the operation of the autofocus based on the trained model MD on the terminal device 20 or the information processing device 30, and in the case where it is determined as a result of the operation check that the autofocus accuracy has been improved, the user installs the trained model MD in the electronic device 1 through the terminal device 20. The electronic device 1 performs autofocus according to the installed MD model.

[0150] The information processing device 30 is capable of collecting image data from a large number of unspecified terminal devices 20 via the network NW by means of dedicated application software AS provided to the terminal device 20. Note that the image data used by the information processing device 30 in the training process is not limited to the image data collected from the terminal device 20. For example, the information processing device 30 can also perform training processing for improving the accuracy of autofocus based on data such as freely available image data or commercial image data publicly available on the network NW.

[0151] For the training process performed by the information processing device 30, machine learning such as deep learning using a deep neural network (DNN) can be applied. In addition, other training methods, algorithms, etc. can also be applied.

[0152] According to each of the above-described embodiments, for an object that has not been fully detected by a detector that has been provided, the detection accuracy is improved, so that such an object can be fully detected (improved detection). In addition, according to each of the above-described embodiments, a detection function can be provided for an undetectable object for which a detector is not provided, so that such an object can be detected (new object detection). For example, when photographing wildlife, the eyes of creatures such as wild animals, wild birds, and other types of wildlife can be focused on. The animal newly regarded as an object is selected and used from a menu, etc.

[0153] In addition, according to each of the above-mentioned embodiments, a specific object is distinguished from the successfully detected objects (object distinction). For example, when shooting activities such as school sports events, one can give priority to one's own children and focus on the eyes of the children. In the case where a familiar person, pet, etc. is the object, the object is registered when shooting every day, and the object is regularly set to the highest priority.

[0154] Furthermore, according to each of the above-described embodiments, an AF point can be specified for a detected object (especially an object other than a living body) (AF point specification). For example, when shooting a motorsports, or when shooting a train or an airplane, it is possible to focus on a desired portion of an object targeted by the user. A vehicle and an AF point to be regarded as an object are selected from a menu or the like and used.

[0155] In addition, according to each of the aforementioned embodiments, since continuous improvement can be achieved by increasing the accumulation of image data for training, the autofocus function can be cumulatively improved (cumulative improvement). In addition, according to each of the aforementioned embodiments, by having the user regularly capture and update image data of familiar people, pets, etc., it is also possible to adapt to changes (growth) of the subject over time (adaptation to changes over time).

[0156] The preferred embodiments of the present disclosure have been described above with reference to the accompanying drawings, and the present disclosure is not limited to the above examples. Those skilled in the art may find various changes and modifications within the scope of the appended claims, and it should be understood that they will certainly fall within the technical scope of the present disclosure.

[0157] In addition, the effects described in this specification are merely illustrative or exemplary effects, not restrictive. That is, together with or in place of the above effects, other effects that are clear to those skilled in the art from the description of this specification can be achieved according to the technology disclosed in this specification.

[0158] Furthermore, the present technology may also be configured as follows. (1)

[0160] An information processing device, comprising:

[0161] A collecting unit, wherein the collecting unit collects image data;

[0162] a training unit that extracts a position of an object included in the image data using the plurality of image data collected by the collecting unit, performs a training process on a relationship between the image data and the position of the object, and creates a trained model for automatic focus adjustment that specifies an object from the image data to specify a focus position; and

[0163] An output unit that outputs the trained model created by the training unit to a device that requests the trained model. (2)

[0165] The information processing device according to (1), wherein:

[0166] The collecting unit accumulates the collected image data by automatically classifying the image data by category of the object, and

[0167] When the image data collected by the collecting unit for each category of the object reaches a predetermined number, the training unit starts a training process for the object. (3)

[0169] The information processing device according to (2), wherein the output unit outputs the collected image data and an aggregation state of the collected image data according to the category of the object. (4)

[0171] The information processing device according to (1), wherein distance measurement points are specified in the image data used in the training process. (5)

[0173] The information processing device according to (4), wherein a plurality of distance measuring points can be designated and an order of priority is determined. (6)

[0175] The information processing device according to (5), wherein the priority order of the designated plurality of distance-measuring points is changeable. (7)

[0177] The information processing device according to (4), wherein the collection unit collects image data in which the distance measurement point is specified. (8)

[0179] The information processing apparatus according to (1), further including: a labeling unit that automatically labels the image data by using the image data used by the training unit in the training process. (9)

[0181] An electronic device, comprising:

[0182] a storage unit storing a trained model for automatic focus adjustment;

[0183] a control unit that performs automatic focus adjustment based on the trained model stored in the storage unit; and

[0184] An optical system focuses on a predetermined portion of a subject according to control of the control unit. (10)

[0186] A terminal device, comprising:

[0187] a communication unit capable of communicating with an information processing device;

[0188] a control unit that acquires a trained model for automatic focus adjustment from the information processing device through the communication unit; and

[0189] A providing unit that provides the trained model acquired from the information processing device to an electronic device that performs automatic focus adjustment based on the trained model. (11)

[0191] The terminal device according to (10), wherein the control unit registers image data in the information processing device through the communication unit, and acquires the trained model from the information processing device based on the image data. (12)

[0193] The terminal device according to (11), further comprising:

[0194] a display unit that displays the image data; and

[0195] An input unit, the input unit outputs a signal corresponding to the user's input operation to the control unit, wherein:

[0196] The control unit specifies a distance measurement point for the image data displayed on the display unit according to a signal from the input unit. (13)

[0198] The terminal device according to (12), wherein:

[0199] The display unit displays image data issued by the information processing device, and

[0200] The control unit specifies the distance measurement point for the image data issued by the information processing device according to the signal from the input unit. (14)

[0202] The terminal device according to (12), wherein:

[0203] The display unit displays the image data before the image data is registered in the information processing apparatus, and

[0204] The control unit specifies the distance measuring point for the image data before the image data is registered in the information processing device according to a signal from the input unit, and registers the image data in which the distance measuring point is specified in the information processing device through the communication unit. (15)

[0206] An information processing system, comprising:

[0207] an information processing device that collects image data, extracts a position of an object included in the image data using the collected plurality of image data, performs a training process on a relationship between the image data and the position of the object, creates a trained model for automatic focus adjustment, and outputs the trained model to a device that requests the trained model, wherein the trained model specifies an object from the image data to specify a focus position;

[0208] a terminal device, the terminal device acquiring the trained model from the information processing device; and

[0209] An electronic device that acquires the trained model from the terminal device and performs automatic focus adjustment based on the trained model. (16)

[0211] An information processing system, comprising:

[0212] an information processing apparatus that collects image data, extracts a position of an object included in the image data using the collected plurality of image data, performs a training process on a relationship between the image data and the position of the object, creates a trained model for automatic focus adjustment, and outputs the trained model to a device that requests the trained model, wherein the trained model specifies an object from the image data to specify a focus position; and

[0213] An electronic device that acquires the trained model from the information processing device and performs automatic focus adjustment based on the trained model. (17)

[0215] An information processing method, comprising:

[0216] Collect image data;

[0217] extracting a position of an object included in the image data using the collected plurality of image data, performing a training process on a relationship between the image data and the position of the object, and creating a trained model for automatic focus adjustment, the trained model specifying an object from the image data to specify a focus position; and

[0218] The created trained model is output to a device requesting the trained model. (18)

[0220] A program causing a computer to execute a process, the process comprising:

[0221] Collect image data;

[0222] extracting a position of an object included in the image data using the collected plurality of image data, performing a training process on a relationship between the image data and the position of the object, and creating a trained model for automatic focus adjustment of an electronic device, the trained model specifying an object from the image data to specify a focus position; and

[0223] The created trained model is output to a device requesting the trained model.

[0224] Reference numerals list

[0225] 1 Electronic equipment

[0226] 2 Optical system

[0227] 3 Control unit

[0228] 7 Storage Unit

[0229] 9 I / F unit

[0230] 20 Terminal Devices

[0231] 21 I / F unit

[0232] 22 AS control unit

[0233] 23 AS storage unit

[0234] 24 Communication Unit

[0235] 25 Input Unit

[0236] 26 Display Unit

[0237] 30 Information processing device

[0238] 31 Communication unit

[0239] 32 Control Unit

[0240] 321 Collection Unit

[0241] 322 Annotation Unit

[0242] 323 Training Units

[0243] 324 evaluation units

[0244] 325 Output Unit

[0245] 33 Storage Units

[0246] 331 Image storage unit

[0247] 332 AF point storage unit

[0248] 333 Model storage unit

[0249] 100 Information Processing Systems

Claims

1. An information processing device, include: A collecting unit, wherein the collecting unit collects image data; a training unit that extracts a position of an object included in the image data using the plurality of image data collected by the collecting unit, performs a training process on a relationship between the image data and the position of the object, and creates a trained model for automatic focus adjustment that specifies an object from the image data to specify a focus position; as well as an output unit, the output unit outputting the trained model created by the training unit to a device requesting the trained model, wherein The collecting unit accumulates the collected image data by automatically classifying the image data by the category of the object, When the image data collected by the collecting unit for each category of the object reaches a predetermined number, the training unit starts a training process for the object, and Before the image data used in the training process is input to the training unit, distance measurement points are specified in the image data used in the training process.

2. The information processing device according to claim 1, in, The output unit outputs the collected image data and an aggregation state of the collected image data according to the category of the object.

3. The information processing device according to claim 1, in, Multiple ranging points can be specified and their priority order can be determined.

4. The information processing device according to claim 3, in, The priority order of the designated plurality of ranging points is changeable.

5. The information processing device according to claim 1, in, The collecting unit collects image data in which the distance measuring point is specified.

6. The information processing device according to claim 1, further comprising: include: A labeling unit that automatically labels the image data by using the image data used by the training unit in the training process.

7. An electronic device, include: a storage unit storing a trained model for automatic focus adjustment output from the information processing device according to any one of claims 1 to 6; a control unit that performs automatic focus adjustment based on the trained model stored in the storage unit; as well as An optical system focuses on a predetermined portion of a subject according to control of the control unit.

8. A terminal device, include: a communication unit capable of communicating with the information processing device according to any one of claims 1 to 6; a control unit, the control unit acquiring a trained model for automatic focus adjustment from the information processing device through the communication unit; as well as A providing unit that provides the trained model acquired from the information processing device to an electronic device that performs automatic focus adjustment based on the trained model.

9. The terminal device according to claim 8, in, The control unit registers image data in the information processing device through the communication unit, and acquires the trained model from the information processing device based on the image data.

10. The terminal device according to claim 9, further comprising: include: a display unit, wherein the display unit displays the image data; as well as An input unit, the input unit outputs a signal corresponding to the user's input operation to the control unit, wherein: The control unit specifies a distance measurement point for the image data displayed on the display unit according to a signal from the input unit.

11. The terminal device according to claim 10, in, The display unit displays image data issued by the information processing device, and The control unit specifies the distance measurement point for the image data issued by the information processing device according to the signal from the input unit.

12. The terminal device according to claim 10, in, The display unit displays the image data before the image data is registered in the information processing apparatus, and The control unit specifies the distance measuring point for the image data before the image data is registered in the information processing device according to a signal from the input unit, and registers the image data in which the distance measuring point is specified in the information processing device through the communication unit.

13. An information processing system, include: The information processing device according to any one of claims 1 to 6; a terminal device, the terminal device acquiring the trained model from the information processing device; as well as An electronic device that acquires the trained model from the terminal device and performs automatic focus adjustment based on the trained model.

14. An information processing system, include: The information processing device according to any one of claims 1 to 6; as well as An electronic device that acquires the trained model from the information processing device and performs automatic focus adjustment based on the trained model.

15. An information processing method, include: collecting image data by a collecting unit; extracting, by a training unit, a position of an object included in the image data using the collected plurality of image data, performing a training process on a relationship between the image data and the position of the object, and creating a trained model for automatic focus adjustment, the trained model specifying an object from the image data to specify a focus position; as well as The created trained model is output by an output unit to a device requesting the trained model, wherein: The collecting unit accumulates the collected image data by automatically classifying the image data by the category of the object, When the image data collected by the collecting unit for each category of the object reaches a predetermined number, the training unit starts a training process for the object, and Before the image data used in the training process is input to the training unit, distance measurement points are specified in the image data used in the training process. 16 . A storage medium storing a program for causing a computer to execute the information processing method according to claim 15 .

Images