Information processing device, information processing method, program, information processing system, inspection device, and inspection method

Abstract

To provide an information processing apparatus that can estimate the performance of a lens and easily evaluate the quality of the lens by using a learning model.SOLUTION: An information processing apparatus has: a first acquisition unit that acquires first information recording a usage state after shipping of a lens unit having at least one optical element; a second acquisition unit that acquires second information recording in advance the lens performance or the standard of the lens unit or a lens unit different from the lens unit; and a first processing unit that, with the first information and the second information as input data of a predetermined learning model, generates a learned model obtained by adjusting a parameter in the learning model.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, a program, an information processing system, an inspection device, and an inspection method. [Background technology]

[0002] In recent years, AI technologies such as machine learning have been used in various fields, enabling the analysis of complex phenomena consisting of multiple elements that were previously difficult to achieve. Interchangeable lenses (lens devices, lens units) are precision instruments that combine a variety of elements in a precise and complex manner, and attempts are being made to solve various issues in the field of interchangeable lenses by applying the above-mentioned AI technologies.

[0003] On the other hand, interchangeable lenses require periodic maintenance, and the decision as to whether or not maintenance is necessary is generally left to the user. However, as described above, interchangeable lenses are complex precision instruments, making such a decision difficult. In order to assist in this decision, a method has been disclosed for estimating and evaluating the performance of interchangeable lenses and informing users of their lens quality. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-156643 A Summary of the Invention [Problem to be solved by the invention]

[0005] According to Patent Document 1, performance is estimated by comparing durability information of each drive unit of an interchangeable lens with the operation record of the interchangeable lens. Here, durability information is set for each model of the interchangeable lens, and is uniformly applied to all interchangeable lenses of the same model. However, the degree of change in performance of an interchangeable lens varies depending on the usage environment such as temperature and humidity, and there is a risk of a discrepancy between the quality evaluation result of the interchangeable lens according to the method of Patent Document 1 and the actual quality.

[0006] Therefore, an object of the present invention is to provide an information processing device that can, for example, estimate the performance of a lens using a learning model and easily evaluate the quality of a lens. [Means for solving the problem]

[0007] In order to achieve the above object, an information processing device according to one aspect of the present invention comprises: When information regarding the usage state of the lens unit after shipment is defined as the first information, information regarding the lens performance or specifications obtained during the development of the lens unit is defined as the second information, and information regarding the estimated lens performance including the optical performance of the lens unit is defined as the third information, at least one The first lens unit has an optical element The above a first acquisition unit that acquires first information, and a second lens unit that is different from the first lens unit; The above a second acquisition unit that acquires second information; The first acquisition unit The first information; The second acquisition unit acquires The second piece of information and is the input data for a given learning model. By doing A first processing unit for generating a trained model and a third lens unit different from the first and second lens units. The first and second Information of The trained model By inputting, the third lens unit The third information I live A second processing unit and, The present invention is characterized by having the following. Effect of the Invention

[0008] According to the present invention, it is possible to provide an information processing device that can, for example, estimate the performance of a lens using a learning model and easily evaluate the quality of a lens. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration diagram of an information processing system according to a first embodiment. [Diagram 2] FIG. 2 illustrates an example of a hardware configuration in a learning phase according to the first embodiment. [Diagram 3] FIG. 2 illustrates an example of a hardware configuration of an estimation phase in the first embodiment. [Figure 4] 1 is a flowchart illustrating a lens quality evaluation process that performs a learning phase and an estimation phase in the first embodiment. [Diagram 5] 10 is a flowchart illustrating a measurement process of a lens state when brought into a maintenance base in the first embodiment. [Figure 6] 4 is a flowchart illustrating a learning process of a lens performance estimation model in the first embodiment. [Figure 7] 4 is a flowchart illustrating a process of lens performance estimation and quality evaluation in the first embodiment. [Figure 8] FIG. 11 is a diagram illustrating an example of a configuration diagram of an information processing system according to a second embodiment. [Figure 9] 11 is a flowchart illustrating a process of lens quality evaluation including lens performance measurement at the time of shipment in the second embodiment. [Figure 10] 10 is a flowchart illustrating a process of measuring lens performance at the time of shipment in the second embodiment. [Figure 11] 10 is a flowchart illustrating a measurement flow of a lens state when brought into a maintenance base in the second embodiment. [Figure 12] 10 is a flowchart illustrating a learning process of a lens performance estimation model in the second embodiment. [Figure 13] 11 is a flowchart illustrating a lens performance estimation and quality evaluation process in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In each drawing, the same reference numerals are used to designate the same members or elements, and duplicate descriptions will be omitted or simplified. EXAMPLES

[0011] FIG. 1 is an example of a configuration diagram of an information processing system (quality evaluation system) in this embodiment. The information processing system in this embodiment may include a learning interchangeable lens (hereinafter, interchangeable lens) 100, which is a lens unit, and a lens performance measurement device (hereinafter, measurement device) 200. It may further include a maintenance record server (hereinafter, server device) 300, a lens development data server (hereinafter, server device) 400, and a learning data collection server (hereinafter, server device) 500. It may further include a learning server 600, an estimated interchangeable lens (hereinafter, interchangeable lens) 700, a user terminal 800, an estimation data collection server (hereinafter, server device) 900, and an estimation server 1000. The above-mentioned configurations in the information processing system in this embodiment will be described below with reference to FIG. 2 and FIG. 3.

[0012] Next, the system configuration of the information processing system in the learning phase of this embodiment will be described below with reference to Fig. 2. Fig. 2 is a diagram showing an example of a hardware configuration in the learning phase of this embodiment. The hardware configuration of the information processing system in the learning phase of this embodiment may include an interchangeable lens 100, a measurement device 200, a server device 300, a server device 400, a server device 500, and a learning server 600.

[0013] The interchangeable lens (learning interchangeable lens) 100 is an interchangeable lens whose performance is measured at various facilities where various measuring equipment is installed, such as maintenance bases and production factories set up in various places, in response to requests such as maintenance from users who use the interchangeable lens. In this embodiment, a lens for an interchangeable camera will be described. The interchangeable lens 100 can be composed of an optical element 110, an actuator (ACT) 120, various drivers 130, an internal memory 140, a communication unit 150, and a control unit 160.

[0014] The optical element 110 constitutes an imaging optical system including each lens group, an aperture, a focus mechanism, an anti-vibration mechanism, etc. The optical element 110 has a function of forming an image of various measurement charts displayed on a measurement display (hereinafter, a display) 210 on an image sensor 231 during lens performance measurement, which will be described later.

[0015] Actuator 120 is an actuator for driving optical element 110, and a plurality of actuators may be provided within interchangeable lens 100. Examples of actuator 120 include a group driver for moving a focus lens group or a zoom lens group in the optical axis direction, an anti-vibration driver for driving an anti-vibration mechanism, and a driver for driving an aperture or an ND filter.

[0016] The various drivers 130 function as drivers that transmit electric signals to the actuator 120 to drive the actuator 120. The internal memory 140 appropriately stores first information indicating the usage state of the interchangeable lens 100, which will be described later, in addition to information related to various controls of the interchangeable lens 100. The communication unit 150 transmits and receives various information to and from the imaging device 230, which will be described later, based on commands from the control unit 160.

[0017] The control unit 160 includes a CPU (Central Processing Unit) and is connected to each part of the interchangeable lens 100 via a line. The control unit 160 can comprehensively control the operation of the interchangeable lens 100 according to a program stored in the internal memory 140 of the interchangeable lens 100. The control unit 160 has a function of transmitting various control signals (control commands) to the measurement device 200 and the like.

[0018] In this embodiment, the interchangeable lens 100 is described as a lens for an interchangeable camera, but it may be an image sensor integrated lens device, or an optical element component that does not function alone, such as an extender or various filters. It may also be an optical device such as a telescope, binoculars, rangefinder, or surveying instrument. The same applies to the interchangeable lens 700 described below. Furthermore, if the lens performance of the interchangeable lens 700 has been measured in the past due to a maintenance request or the like, the interchangeable lens 700 at that time may be treated as the above-mentioned interchangeable lens 100.

[0019] The measurement device (lens performance measurement device) 200 is a measurement device that can acquire first information indicating a lens usage state of an interchangeable lens 100, etc., which will be described later, and fourth information related to lens performance. The measurement device 200 can be composed of a display 210, a driving stage 220, an imaging device 230, and a measurement terminal 240.

[0020] The display (measurement display) 210 displays various measurement charts on the display (screen) based on signals from various drivers 241 in the measurement terminal 240 described later.

[0021] The driving stage 220 drives the display 210 and the imaging device 230 in translation along the optical axis and in a direction perpendicular to the optical axis as well as in rotation about each axis as necessary, based on signals from various drivers 241 in the measurement terminal 240. This makes it possible to measure items related to vibration isolation performance, for example, by driving a stage to which an imaging device is attached, and to measure items related to subject tracking performance by driving a stage to which a measurement display is attached.

[0022] The imaging device 230 is a camera body with interchangeable lenses, and may include an imaging element 231, an interchangeable lens communication unit (hereinafter, communication unit) 232, a terminal communication unit (hereinafter, communication unit) 233, an internal memory 234, and a control unit 235.

[0023] The imaging element 231 acquires the image displayed on the measurement display via the interchangeable lens 100. The communication unit (interchangeable lens communication unit) 232 transmits and receives various information to and from the interchangeable lens 100 based on commands from the control unit 235. The communication unit (terminal communication unit) 233 transmits and receives various information to and from the measurement terminal 240 based on commands from the control unit 235. The internal memory 234 stores the various information transmitted from the interchangeable lens 100.

[0024] The control unit 235 includes a CPU and is connected to each unit of the imaging device 230 via a line. The control unit 235 has a function of transmitting various control signals to the imaging device 230 and the interchangeable lens 100 in accordance with a program stored in the internal memory 234, based on a command from the measurement terminal 240.

[0025] In this embodiment, the imaging device 230 is a camera body with interchangeable lenses as described above, but it may be a dedicated measurement device. Information on images and videos captured by the imaging device 230 is stored in the internal memory 234 and transmitted to the measurement terminal 240 by the control unit 235 via the communication unit 233.

[0026] The measurement terminal 240 may include an imaging device communication section (hereinafter, communication section) 242, a data server communication section (hereinafter, communication section) 243, a storage section 244, an input section 245, a control section 246, and various drivers 241.

[0027] The communication unit 242 (communication unit for imaging device) transmits and receives various information to and from the imaging device 230 based on a command from the control unit 246. The communication unit (communication unit for data server) 243 transmits and receives various information to and from the server device 300 based on a command from the control unit 246. The storage unit (memory) 244 saves (holds) various data. The input unit 245 can input various measurement items and measurement conditions, as well as the degree of wear of components of the interchangeable lens 100 and the like. The control unit 246 includes a CPU and is connected to each unit of the measurement terminal 240 via a line. Based on the input from the input unit 245, it transmits various control commands to each unit of the measurement terminal 240 according to the program stored in the storage unit 244. The various drivers 241 transmit electric signals to drive each drive unit of the measurement device 200.

[0028] The server device (maintenance record data server) 300 is a database that holds (saves) and manages the first information and the fourth information acquired by the measuring device 200 for each individual interchangeable lens 100. The server device 300 may include a communication unit 310 that can communicate with the measuring device 200 and the server device 500, a storage unit 320 that holds various data, and a control unit 330 that controls the server device 300 as a whole.

[0029] The server device (lens development data server) 400 is a database that holds and manages various data acquired during lens development, such as second information related to lens durability performance and standards, for each model in advance. The various data acquired during lens development, such as the second information, is information on the interchangeable lens 100 described below or a reference interchangeable lens different from the interchangeable lens 100. The server device 400 may include a communication unit 410 capable of communicating with the server device 500, an input unit 420 to which various information can be input, a storage unit 430 that stores various data, and a control unit 440 that controls the entire server device 400. The second information related to lens durability performance and the like is stored in the storage unit 430 of the server device 400 by the control unit 440 by inputting various information from the input unit. Also, even if no input is made from the input unit 420, the second information stored in, for example, an external server device (storage unit, etc.) may be acquired by the control unit 440 and stored in the storage unit 430. Also, in this case, the server device 400 and the server device 300 may be the same server device (server equipment).

[0030] The server device (learning data collection server) 500 is an information processing device that collects various learning data used in machine learning described later, and may be composed of, for example, at least one computer with a program installed. The server device 500 may include a communication unit 510 that can communicate with the server device 300, the server device 400, and a learning server 600 described later, a storage unit 520 that holds various data, and a control unit 530 that controls the entire server device 500. In this case, the server device 500, the server device 300, and the server device 400 may be the same server device.

[0031] The learning server 600 is an information processing device that performs machine learning to acquire a lens performance estimation function described later, and may be composed of, for example, at least one computer with a program installed. The learning server 600 may include a communication unit 610 that can communicate with the server device 500 and the estimation server 1000, a storage unit 620 that stores various data, and a processing unit (control unit) 630 that processes learning data and performs learning processing. The processing unit 630 includes a CPU and is connected to each unit of the learning server 600 via a line. In addition, the processing unit 630 can comprehensively control the operation of the learning server 600 according to a program stored in the storage unit 620 of the learning server 600. The processing unit 630 also has a function of transmitting various control signals (control commands) to the server device 500, the estimation server 1000, and the like.

[0032] The processing unit 630 executes machine learning, which updates parameters in the model multiple times using a learning model. When learning is performed multiple times using a learning model such as machine learning, it is effective to use a GPU (Graphics Processing Unit) that can efficiently perform parallel data processing. Therefore, in this embodiment, a GPU is used in addition to a CPU for processing by the processing unit 630. Specifically, when a learning program including a learning model is executed, the CPU and the GPU work together to perform calculations to perform learning. Note that the processing of the processing unit 630 may be performed only by the CPU or the GPU. In this case, the learning server 600 and the server device 500 may be the same server device.

[0033] Next, the system configuration in the estimation phase of this embodiment will be described below with reference to Fig. 3. Fig. 3 is a diagram showing an example of a hardware configuration in the estimation phase of this embodiment. The hardware configuration in the estimation phase of this embodiment may include an interchangeable lens 700, a user terminal 800, a server device 900, and an estimation server 1000.

[0034] The interchangeable lens (estimated interchangeable lens) 700 is an interchangeable lens as an evaluation target in this embodiment. In this embodiment, it will be described as a lens for an interchangeable camera. The interchangeable lens 700 can include an optical element 710, an actuator (ACT) 720, various drivers 730, an internal memory 740, a communication unit 750, and a control unit 760.

[0035] The optical element 710 constitutes an imaging optical system including each lens group, an aperture, a focus mechanism, an anti-vibration mechanism, etc. The actuator 720 is an actuator for driving the optical element 710, and a plurality of actuators may be provided in the interchangeable lens 700. Examples of the actuator 720 include a group driver for moving the focus lens group and the zoom lens group in the optical axis direction, an anti-vibration driver for driving the anti-vibration mechanism, and a driver for driving the aperture and the ND filter.

[0036] The various drivers 730 function as drivers that transmit electric signals to the actuator 720 to drive the actuator 720. The internal memory 740 appropriately stores first information indicating the usage state of the interchangeable lens 700 (described later) in addition to information related to various controls of the interchangeable lens 100. The communication unit 750 transmits and receives various information to and from the user terminal 800 (described later) based on commands from the control unit 760.

[0037] The communication unit 750 may be a communication unit such as a USB terminal provided exclusively for the interchangeable lens 700, or may be a type in which the interchangeable lens 700 is attached to a camera body (image capture device) and various communication units provided in the camera body are used. The communication method may be a wired method via various cables, or a wireless method using Wi-Fi (Wireless Fidelity) (registered trademark), Bluetooth (registered trademark), or the like.

[0038] The control unit 760 includes a CPU and is connected to each unit of the interchangeable lens 700 via a line. The control unit 760 also performs overall control of the operation and adjustment of the interchangeable lens 700 in accordance with a program stored in the internal memory 740 or the like of the interchangeable lens 700. The control unit 760 has a function of transmitting various control signals (control commands) to the user terminal 800 or the like.

[0039] In this embodiment, the interchangeable lens 700 is described as a lens for an interchangeable camera, similar to the interchangeable lens 100, but it may be an optical element component that does not function by itself, such as a lens device with an integrated image sensor, an extender, or various filters, etc. Furthermore, it may be an optical device such as a telescope, binoculars, rangefinder, or surveying instrument.

[0040] The user terminal 800 is a device that acquires the first information and the like stored in the internal memory 740 of the interchangeable lens 700. The user terminal 800 may include an interchangeable lens communication unit (hereinafter, communication unit) 810 capable of communicating with the interchangeable lens 700, an input unit 820 through which the user can input various information, and a display unit 830 on which the various input information, lens quality evaluation results, and the like are displayed. In addition, the user terminal 800 may include a storage unit 840 that stores various data, a server communication unit (hereinafter, communication unit) 850 capable of communicating with the server device 900 and the estimation server 1000, and a control unit 860 that controls the entire user terminal 800. Examples of the user terminal 800 include an information processing device (information processing terminal) such as a personal computer or a smartphone, and a lens-interchangeable camera body.

[0041] The server device (estimation data collection server) 900 is an information processing device that collects various estimation data used for estimating lens performance of the interchangeable lens 700 or the like, and may be composed of, for example, at least one computer with a program installed. The server device 900 may include a communication unit 910 that can communicate with the user terminal 800 and the server device 400, a storage unit 920 that stores various data, and a control unit 930 that controls the entire server device 900. The control unit 930 includes a CPU and is connected to each unit of the server device 900 via a line. The control unit 930 can comprehensively control the operation of the server device 900 according to a program stored in the storage unit 920 or the like of the server device 900. The control unit 930 also has a function of transmitting various control signals (control commands) to the server device 400, the user terminal 800, the estimation server 1000, and the like. The server device 900 and the server device 400 may be the same server device.

[0042] The estimation server 1000 is an information processing device that executes lens performance estimation and lens quality evaluation, and may be composed of, for example, at least one computer with a program installed. The estimation server 1000 may include a communication unit 1010 that can communicate with the server device 900 and the learning server 600, a storage unit 1020 that stores various data, and a processing unit (control unit) 1030 that processes estimation data, performs estimation processing, and performs quality evaluation processing. The processing unit 1030 includes a CPU and is connected to each unit of the learning server 600 via a line. The processing unit 1030 can comprehensively control the operation of the estimation server 1000 according to a program stored in the storage unit 1020 of the estimation server 1000. The processing unit 1030 also has a function of transmitting various control signals (control commands) to the learning server 600, the server device 900, and the like.

[0043] The processing unit 1030 may use a GPU in addition to a CPU, as in the processing unit 630 of the above-described learning server 600, or calculations may be performed by only the CPU or the GPU. The estimation server 1000, the learning server 600, and the server device 900 may be the same server device.

[0044] Next, the first information indicating the usage record after shipment of an interchangeable lens such as the interchangeable lens 100 or the interchangeable lens 700 will be described below. First, the first information includes the drive record of the interchangeable lens 100 or the interchangeable lens 700. The drive record includes, for example, the number of times that the drive unit of the interchangeable lens 100 or the interchangeable lens 700 has been driven. The drive unit includes, for example, an aperture, a focus mechanism, a zoom mechanism, an anti-vibration mechanism, and the like. Note that instead of simply adding up the number of operations, a unique number calculation standard may be set, such as setting three times as one step.

[0045] Furthermore, the degree of occurrence of wear of members caused by driving may differ depending on the driving time, driving speed, etc. Thus, the first information may be information (driving information) based on the driving time, driving distance, or driving speed. Furthermore, as the control record, the driving characteristics (control information) of each actuator that drives each driving unit of the interchangeable lens 100 or the interchangeable lens 700, for example, the actuator 120 or the actuator 720, may be the first information. The driving characteristics include, for example, the starting voltage, starting frequency, maximum speed, step-out speed, driving voltage, etc. depending on each actuator, and by acquiring this information, it is possible to evaluate the controllability of the interchangeable lens 100 or the interchangeable lens 700.

[0046] In addition, interchangeable lenses generally undergo various changes in performance depending on the environment in which they are used. For example, parts deterioration and lens mold depending on temperature and humidity, looseness in the lens barrel assembly and parts damage depending on acceleration and impact degree, and foreign matter contamination due to dust and sea breeze may be included. Therefore, information on the environment in which the interchangeable lens 100 or the interchangeable lens 700 is used may be recorded as the first information. For example, various recorded information may be considered, such as temperature records (temperature information) and humidity records (humidity information) inside and outside the interchangeable lens, external force records (external force information) based on acceleration, angular acceleration, and impact degree, and climate records based on location information. In this case, the interchangeable lens 100 or the interchangeable lens 700 may be provided with sensors such as a temperature sensor and a GPS. Furthermore, a dedicated battery or the like may be provided so that various data can be recorded even when the interchangeable lens 100 or the interchangeable lens 700 is not connected to the camera body.

[0047] The first information has been described above, but by enriching the items of the first information, it is possible to specify the usage history of the interchangeable lens 100 or the interchangeable lens 700 in detail, and the accuracy of the estimated lens performance can be improved in the machine learning described below. Therefore, it is preferable that all of the above-mentioned various information be items of the first information. Also, items may be created by combining the above-mentioned various information depending on the situation. Note that, although the above describes the first information in the interchangeable lens 100 or the interchangeable lens 700, the first information is similar in other lens devices other than the interchangeable lens 100 or the interchangeable lens 700.

[0048] Next, the second information related to the quality test that is acquired in the development process before the shipment of the interchangeable lens will be described below. Note that the second information is acquired by an interchangeable lens (a reference lens for testing) or the like that is separate from the interchangeable lens 100 and the interchangeable lens 700. Note that the reference lens for testing is used to acquire measurement data in a factory or the like, and is not generally shipped.

[0049] In the development process, quality tests are conducted assuming various methods of using the interchangeable lens, usage environments, etc. In this embodiment, the test conditions at that time and the lens performance and specifications after the test are set as the second information of the interchangeable lens 100 and the interchangeable lens 700. This allows efficient learning of the degree of influence of each factor that causes a performance change, such as the number of times of driving and temperature, on each performance change in machine learning described later. Examples of test contents in the second information include environmental tests (environmental test information) assuming various climates, durability tests (durability test information) of various driving parts, static pressure tests (load test information), vibration tests (vibration test information), and impact tests (impact test information). Here, the test conditions include temperature, humidity, time, number of times of driving, load load, vibration frequency, etc. according to each test content.

[0050] Items to be evaluated as lens performance after testing include optical performance (optical performance information) based on MTF %, which will be described later, lens barrel operation performance (operation performance information) such as the drive characteristics of each actuator, and dustproof or drip-proof performance (dustproof and drip-proof performance information).Furthermore, items to be evaluated include a wear evaluation of components (wear information) and a sensory evaluation (evaluation information) of appearance and operating state (operational feel).Here, data indicating the lens performance of the interchangeable lens after testing may be a quantitative numerical value indicating each performance, or may be a classification indicating OK / NG according to the standard.

[0051] Furthermore, examples of the fifth information related to product specifications include product catalog data such as size, mass, lens configuration, focal length, and aperture value, product design information such as drive system, constituent materials, and various dimensions, and manufacturing information such as manufacturing plant information at the time of design and manufacturing. This fifth information may be added (appended) to the second information. This allows machine learning, which will be described later, to learn the similarities in specifications between product models of interchangeable lenses, find common trends according to product specifications, and improve the accuracy of estimated lens performance.

[0052] Examples of the common tendency include the way external forces are applied, the way optical performance changes, the way driving characteristics change, and the way various parts wear and tear. This also makes it possible to estimate the performance of a new product with few maintenance records from information on similar past products. Note that the second information has been described as being acquired by an individual interchangeable lens different from the interchangeable lens 100 or the interchangeable lens 700, but this is not limiting, and the second information may be acquired from the interchangeable lens 100 or the interchangeable lens 700.

[0053] Next, the fourth information, which indicates the lens performance based on the results of a user actually using the interchangeable lens 100 or the like, will be described below.

[0054] [Table 1]

[0055] The fourth information is collected at facilities where various measuring equipment is installed, such as various maintenance bases and production factories. For example, Table 1 is an example of the fourth information, which is table data (hereinafter referred to as MTF % data) that stores the percentage of MTF (Modulation Transfer Function), which is one of the indexes indicating the optical performance of a lens, at each image plane position. The table data is expressed as the following equation 1. (Number 1) Table

[0001] "y,z"(x,Co,f,Z,Fo,Iris,g,Di,Ad)

[0056] Here, Table in the above formula 1 indicates that this is table data, and the numbers in [ ] indicate the serial number of the table data. Additionally, the MTF% data is set to 001. The characters in " " are the main variables of the table data, and indicate that the size of the two-dimensional table is an array of y rows by z columns. For convenience, the number of data pieces is expressed as 1 here. The characters in ( ) indicate sub-variables, and indicate that there are as many pieces of table data as there are combinations of sub-variables.

[0057] Next, a detailed description will be given of each variable in the above equation 1. With the position at which the interchangeable lens 100 is attached to the measurement device 200 as a reference, y represents the vertical position and z represents the horizontal position, and each is divided equally into 64 parts.

[0058] x indicates the position (mm) in the optical axis direction, and when the position of the imaging plane is 0, the image plane side is + (plus) and the opposite side is - (minus). Movement can be achieved by moving the imaging plane, moving the object distance, or moving the focus lens group, etc. Co is the color (wavelength) of the light source used during measurement, and in this embodiment, a specific spectral waveform is used as white light.

[0059] f is the spatial frequency (lines / mm), Z is the focal length (mm), Fo is the subject distance (m), Iris is the aperture value, g is the direction of gravity, and Di is the direction of the black and white chart lines when measuring MTF. g is generally distinguished as the sagittal direction or meridional direction, or as a vertical line or horizontal line. Ad indicates the presence or absence of an adapter such as an extender. Note that this data format is not limited to this, and for example, the R, θ coordinate system can be used instead of y, z, and Fo can be substituted by the amount of movement of the focus lens instead of the subject distance.

[0060] Next, an example of data used when the measured MTF % data table is used as the fourth information representing lens performance is shown below. As with Table

[0001] , the MTF % data of the design value for the same model is expressed as Table [001'] using the following formula 2. (Number 2) Table[001'] "y,z"(x,Co,f,Z,Fo,Iris,g,Di,Ad)

[0061] Here, the performance change rate during MTF% measurement is calculated using the following formula (1). Note that the calculations in formula (1) and the formulas described below calculate the numbers in each array data of the same variables. Table

[0001] "y,z"(x,Co,f,Z,Fo,Iris,g,Di,Ad) / Table[001']"y,z"(x,Co,f,Z,Fo,Iris,g,Di,Ad)×100=AAA (1)

[0062] The above formula (1) expresses the degree of deterioration from the design value at the time of measurement under each variable condition as a percentage, and the result of the above formula (1) is expressed as AAA for ease of notation. Also, Table [001'] is stored on each data server as a representative value for each product model.

[0063] Next, as the weighting table data for the MTF % data, Table

[0002] is expressed as in the following Equation 3. (Number 3) Table

[0002] "y,z"(x,Co,f,Z,Fo,Iris,g,Di,Ad)

[0064] In Table

[0002] in the above formula 1, for example, when variables x=-1, 0, +1 are set, the weight of x=0, which is the focal position, is set to 1 under certain conditions for other variables. The weight of x=-1, which reflects the near side of the focal position, is set to 0.5, and the weight of x=+1, which reflects the far side of the focal position, is set to 0.3. For x=-1, in combination with the variable Iris, it also becomes a parameter for evaluating the so-called blur, and the weight of x=-1 is increased to 0.6 under conditions of a large focal length Z where the depth becomes shallow. In addition, when the subject distance Fo is an ∞ distance, since there is usually no subject farther than infinity, the weight of x=-1 is set to 0, and the weight of x=+1 is set to 0.7. However, there are various weighting methods depending on the variables and are not limited to this, so a weighting method different from the above may be used depending on the situation.

[0065] In this embodiment, the weighting data Table

[0002] is stored on each data server as a representative value for each product model. However, it is also possible to use other data or change the amount of weighting based on information from the input unit of each terminal. Here, the following formula (2) is calculated to weight each variable. Table

[0002] "y,z"(x,Co,f,Z,Fo,Iris,g,Di,Ad) ×AAA=BBB (2)

[0066] The result of the above formula (2) is expressed as BBB for ease of notation. The average value of the data stored in this BBB table data can be rephrased as a weighted average value obtained by weighting each variable, and can be regarded as the fourth information in this embodiment as lens performance based on MTF % data.

[0067] The fourth information based on the MTF% data has been described above, but the fourth information may be the MTF% table data itself for each measured variable condition, or image data obtained by photographing a measurement chart. Furthermore, items related to lens performance are not limited to MTF% data, and various other data are possible. For example, items include the amount of deviation of peripheral illumination from the optical axis, the amount of deviation of distortion from the optical axis, the amount of chromatic aberration of magnification, the amount of curvature of field calculated from the MTF% data, the amount of astigmatism of the lens, and the surface precision of spherical and aspherical lenses.

[0068] Furthermore, in addition to the optical performance based on MTF%, information on the lens barrel operation performance such as the driving characteristics of each actuator, dustproof or drip-proof performance, and sensory evaluation information such as appearance and operation state may be added to the lens evaluation items in the fourth information. Furthermore, for example, the degree of wear (wear evaluation) of each lens barrel member constituting the interchangeable lens 100 may be added to the lens evaluation items in the fourth information. For example, items that can be checked by lens inspection or overhaul, such as the appearance and interior, the degree of wear of the cam grooves and cam pins, the amount of grease in each sliding part, the looseness of the lens barrel assembly, and peeling of members, may be included. This makes it possible to evaluate not only the optical performance but also the visual quality, the lens barrel operation accuracy, the dustproof and drip-proof performance, the operation feeling, etc.

[0069] By increasing the fourth item of information through the various measurements described above, it is possible to increase the third item of information, which is the estimated lens performance output by the machine learning model described below.

[0070] Next, the machine learning executed by the learning server 600, and the lens performance estimation process and lens quality evaluation by the estimation server 1000 will be described below.

[0071] In the learning server 600, machine learning is performed using the first information and the second information as input data and the fourth information as teacher data. In this embodiment, the algorithm used in the machine learning model is described in detail as deep learning that uses a neural network to generate features and connection weighting coefficients for learning. However, the present embodiment may be applied to machine learning algorithms such as nearest neighbor method, naive Bayes method, decision tree, and support vector machine.

[0072] Here, the learning model may include an error detection unit and an update unit. The error detection unit obtains an error between the teacher data and output data output from the output layer of the neural network according to input data input to the input layer. The error detection unit may also use a loss function to calculate the error between the output data from the neural network and the teacher data.

[0073] The update unit updates the connection weighting coefficients between the nodes of the neural network based on the error obtained by the error detection unit so as to reduce the error. The update unit updates the connection weighting coefficients using, for example, an error backpropagation method. The error backpropagation method is a method of adjusting the connection weighting coefficients between the nodes of each neural network so as to reduce the error.

[0074] Through the above-mentioned machine learning, the neural network is able to estimate what kind of lens performance a given lens product with given quality test information will have when used in a certain way.

[0075] The reason why lens performance changes with use is because the repeated driving of each part and the deterioration of the components cause the lens barrel of an interchangeable lens to loosen and change in size. The tendency of such performance changes can be confirmed by the various quality tests mentioned above. However, the various quality tests mentioned above are set under extremely harsh conditions and are carried out independently for each test, so they do not necessarily match the conditions under which the user actually uses the lens.

[0076] Therefore, in order to predict performance changes according to the user's usage conditions, it is necessary to consider a complex combination of various quality test information. However, it is extremely difficult to derive the law of performance changes from multiple various test conditions and test results and create a multidimensional map showing the relationship between usage conditions and performance changes. For this reason, machine learning such as a neural network that can analyze multiple elements in a complex manner is used. By inputting the usage conditions and quality test information of the lens used by the user and learning using the fourth information, which is the lens performance based on the actual lens usage results, as training data, it becomes possible to estimate lens performance with reduced discrepancy between lens performance changes due to actual use and quality test information.

[0077] A trained model is generated by the above learning, and the estimation server 1000 inputs the first information and the second information of an arbitrary interchangeable lens, for example, the interchangeable lens 700, to the trained model. This makes it possible to output the third information, which is the estimated lens performance (generation of the third information). Furthermore, when image data obtained by photographing a measurement chart or the like is used as the fourth information, which is the teacher data, the output third information can also be image data. At this time, the above-mentioned MTF % table data is created based on the output image data, and lens performance information can be obtained by performing weighting, weighted averaging, or the like.

[0078] In addition, the lens performance items output as the third information may be increased or decreased depending on the product model, and various driving characteristics in the first information such as start-up voltage, start-up frequency, maximum speed, step-out speed, and driving voltage may be added to the third information as lens performance.

[0079] Next, we will explain the lens quality evaluation based on the performance estimation result, which is the output third information. For example, if the weighted average value based on the above-mentioned MTF % data (BBB calculated by formula (2)) is output as the third information, it can be classified as shown in Table 2 below.

[0080] [Table 2]

[0081] Here, as shown in Table 2, the lens quality is classified into A rank, B rank, and C rank as a quality evaluation result. For example, if the weighted average value is 82 or more, it is classified into A rank, if the weighted average value is 70 or more and less than 82, it is classified into B rank, and if the weighted average value is less than 70, it is classified into C rank. In addition, a notification such as a message according to the rank is notified to the user. For example, as shown in Table 2, the user is notified that no maintenance is required in the case of A rank, that adjustment / cleaning is recommended in the case of B rank, and that overhaul is recommended in the case of C rank.

[0082] Furthermore, the evaluation results may not be classified into the above ranks, and the detailed deterioration degree of each driving unit may be notified to the user, or the output third information itself may be notified to the user. Furthermore, based on the first information acquired at the time of estimation, new first information assuming future lens usage trends may be generated, future lens performance may be estimated, and its progress may be notified. Furthermore, without being limited to this, depending on the situation, the lens quality information notified to the user may be a notification in the form of information other than the above.

[0083] Fig. 4 is a flowchart illustrating a processing flow of lens quality evaluation that performs a learning phase and an estimation phase in this embodiment. Below, with reference to Fig. 4, a flow will be described from the learning phase using the interchangeable lens 100 to evaluating the quality of an interchangeable lens 700, which is an arbitrary interchangeable lens, in the estimation phase. Note that each operation (process) shown in the flowchart in Fig. 4 is controlled by each control unit or each processing unit executing a computer program.

[0084] First, in step S401, it is determined whether or not a lens performance estimation function has already been acquired by machine learning by the processing unit 630, that is, whether or not there is learned data, and whether or not the learning of the learned data is sufficient. If the result of the determination is that there is no learned data, or that there is learned data but the learning of the learned data is insufficient, the process proceeds to step S402, which is the learning phase. If the result of the determination is that there is learned data and the learning of the learned data is sufficient, the process proceeds to step S405, which is the estimation phase. Whether or not the learning is sufficient may be determined by setting a predetermined threshold value and determining whether or not the learning is sufficient based on the threshold value, or may be determined based on the number of times a parameter is updated, or the determination is not limited to this as long as it is determined by a method that can confirm whether or not the learning is sufficient.

[0085] Next, in step S402, the interchangeable lens 100 is measured, and first information representing the interchangeable lens usage state and fourth information representing the lens performance after the interchangeable lens is used are acquired by the measurement device 200 (first acquisition step). Note that in this embodiment, the measurement device 200 also functions as a first acquisition unit that acquires the first information and the fourth information.

[0086] Next, in step S403, second information related to a quality test carried out in the development process of an interchangeable lens is acquired by the server device 400 (second acquisition step). In this embodiment, the server device 400 also functions as a second acquisition unit that acquires the second information. Also, the processing order of steps S402 and S403 may be reversed.

[0087] Next, in step S404, in order to acquire a lens performance estimation function for the interchangeable lens, machine learning using the learning model is performed by the processing unit 630, and a learned model is generated (first processing step). In this embodiment, the processing unit 630 also functions as a first processing unit that generates a learned model. The learned model is a model in which parameters in the learning model are adjusted by machine learning using various first information, second information, and fourth information. The learned model can generate a more up-to-date learned model by updating and adjusting the parameters in the learning model from time to time and optimizing them. Next, in step S405, the processing unit 1030 generates third information from the learned model acquired in step S404, and the processing unit 1030 performs lens quality evaluation of the interchangeable lens 700, which is an arbitrary interchangeable lens, based on the generated third information.

[0088] Next, the process of acquiring the lens state when the interchangeable lens 100 is brought in for maintenance in S402 shown in the flowchart of Fig. 4 will be described in detail below with reference to Fig. 5, which is a subflow of S402. Fig. 5 is a flowchart illustrating the measurement process of the lens state when brought in to a maintenance base in this embodiment.

[0089] First, in step S501, it is determined whether or not an unmeasured interchangeable lens has been brought to a specific maintenance base in response to a request for maintenance or the like from an interchangeable lens user. If an unmeasured interchangeable lens has been brought to the specific maintenance base, the process proceeds to step S502. If an unmeasured interchangeable lens has not been brought to the specific maintenance base, the process proceeds to step S508. The determination is made by a control unit provided in an information processing device or the like of the specific maintenance base based on information recorded in a memory or the like. Furthermore, each maintenance base may be connected by a network so that the control unit can determine whether or not an interchangeable lens is an unmeasured interchangeable lens regardless of which maintenance base it is brought to. Furthermore, bringing in includes a user sending or actually bringing an interchangeable lens to a specific maintenance base, and also includes sending the lens to the maintenance base via a lens manufacturer or a mass retailer where the lens was purchased.

[0090] In addition, even if an interchangeable lens has already been subjected to the processing in steps S502 to S507 below, if a predetermined number of days or years have passed, it may be treated as an unmeasured interchangeable lens. Furthermore, for example, an interchangeable lens that has been subjected to an impact due to being dropped or submerged in water, or an interchangeable lens that has been used in an extreme environment such as hot sand or extremely cold, may also be treated as an unmeasured interchangeable lens. The interchangeable lens is treated as an interchangeable lens 100.

[0091] Next, in step S502, an interchangeable lens, for example the interchangeable lens 100, is mechanically and electrically connected to the measurement device 200. Then, the control unit 160 acquires the first information, such as lens usage state information and lens model information, stored in the internal memory 140 of the interchangeable lens 100. After acquisition, the control unit 160 transmits the first information to the measurement device 200 via the communication unit 150. The transmitted first information is stored in the memory unit 244 of the measurement terminal 240 by the control unit 246 via the communication units 232 and 242 of the measurement device 200.

[0092] Next, in step S503, the items and conditions for performance measurement and the measurement start timing are input from the input unit 245 of the measurement terminal 240, and the control unit 246 transmits a measurement drive command to the imaging device 230 as a performance measurement instruction for the interchangeable lens 100. Next, in S504, based on the measurement drive command transmitted to the imaging device 230 via the communication unit 242, the control unit 235 transmits a measurement drive command to the interchangeable lens 100 via the communication unit 232.

[0093] Next, in step S505, the control unit 160 acquires a video signal of the imaging unit corresponding to the operation of the interchangeable lens 100. After acquisition, the video signal is transmitted to the measurement terminal 240 via the communication units 232 and 242 of the measurement device 200. After transmission, the control unit 246 inputs data into each table of MTF % data, and the data is stored in the storage unit 244 of the measurement terminal 240. At this time, although omitted in the flow in Fig. 5, each performance data of the interchangeable lens 100 is also acquired sequentially by performing the same process, and the control unit 246 stores each acquired performance data in the storage unit 244 of the measurement terminal 240 as fourth information.

[0094] Next, in step S506, an operator performing maintenance on the interchangeable lens 100 visually inspects and performs various operations, disassembly, etc., to check the degree of wear of the various components that make up the interchangeable lens 100. After checking, the degree of wear of the various components that make up the interchangeable lens 100 is input from the input unit 245 to the measurement terminal 240, and the control unit 246 stores this as fourth information in the memory unit 244 of the measurement terminal 240. Furthermore, a sensor may be provided in the measurement device 200 to measure the appearance and degree of wear of the various components that make up the interchangeable lens 100, or measurements may be performed in combination with the checking work performed by the operator.

[0095] Next, in step S507, the control unit 246 transmits the first information and the fourth information acquired in steps S502, S505, and S506 and stored in the memory unit 244 of the measurement terminal 240 to the server device 300 via the communication unit 243. The transmitted first information and the fourth information are stored in the memory unit 320 by the control unit 330 via the communication unit 310 of the server device 300. Then, the process returns to step S501.

[0096] Next, in step S508, the control unit 330 judges whether or not there is any information that has not yet been stored in the server device 300 with respect to each of the above-mentioned pieces of information (information acquired in steps S502 to S506). Note that this judgment is made for an interchangeable lens that has been brought to any maintenance base in the past and has already undergone the work (processing) corresponding to steps S502 to S506. If there is any information that has not yet been stored as a result of the judgment, the process returns to step S507, and the unstored information such as the first information or the fourth information is stored in the storage unit 320 of the server device 300. That is, records of maintenance performed in the past are also stored in the server device 300. If the judgment results in the above-mentioned pieces of information being stored, that is, if all records that should be stored have been stored in the server device 300, the process ends.

[0097] The above-described flowchart shown in FIG. 5 describes the method for collecting and acquiring the first information and the fourth information on the interchangeable lens 100 that is actually used by the user.

[0098] Next, learning of the performance estimation model in step S404 shown in the flowchart of Fig. 4 will be described in detail below with reference to Fig. 6, which is a sub-flow of S404. Fig. 6 is a flowchart illustrating the learning process of the lens performance estimation model in this embodiment.

[0099] First, in step S601, the control unit 330 transmits the first information and the fourth information on the interchangeable lens 100 actually used by the user from the server device 300 to the server device 500 via the communication unit 310. After transmission, the control unit 530 stores the information in the storage unit 520 of the server device 500. Furthermore, the control unit 440 of the server device 400 transmits the second information on each model of the interchangeable lens to the server device 500 via the communication unit 410. After transmission, the control unit 530 stores the information in the storage unit 520 of the server device 500.

[0100] Next, in step S602, the control unit 530 associates the first and fourth information and the second information stored in the storage unit 520 of the server device 500 between interchangeable lenses of the same model. By making the association, learning data that holds the first information, the second information, and the fourth information is created for each interchangeable lens.

[0101] Next, in step S603, the control unit 330 transmits the created learning data to the learning server 600 via the communication unit 510. After transmission, the data is stored in the storage unit 620 of the learning server 600 by the processing unit 630. Next, in step S604, the processing unit 630 selects the learning data of an arbitrary interchangeable lens as sample data.

[0102] Next, in step S605, the processing unit 630 performs conversion processing such as missing value processing, normalization, and categorical variable conversion on the sample data as necessary, and converts the sample data into a format that can be input to the machine learning model. The converted sample data becomes converted sample data. The processing order of steps S604 and S605 may be reversed.

[0103] Next, in step S606, the processing unit 630 inputs the first and second information in the converted sample data as input data and the fourth information as teacher data into a learning model (machine learning model). Next, in step S607, the processing unit 630 adjusts and updates each parameter in the learning model based on the various input data (first information, second information, and fourth information). This makes it possible to obtain a learned model in which each parameter in the learning model is optimized.

[0104] Next, in step S608, the processing unit 630 judges whether or not another sample data (next sample data) exists in the learning data. If it is judged that another sample data exists in the learning data, the process returns to step S603, and the processing of steps S603 to S607 is performed using the sample data. If it is judged that another sample data does not exist in the learning data, the process judges that the learning rate is sufficient, and proceeds to step S609. The processing unit 630 may judge whether or not the learning rate of the learning data is sufficient. In this case, if another sample data does not exist and it is judged that the learning rate is sufficient, the process proceeds to step S609. If it is judged that another sample data does not exist and the learning rate is not sufficient, learning is performed until it is judged that the learning rate is sufficient. For example, repeated learning using the same sample data can be mentioned. If the next sample data exists in making this judgment, the process returns to step S603, regardless of whether the learning rate is sufficient, and the processing of steps S603 to S607 is performed using the sample data.

[0105] Next, in step S609, the processing unit 630 stores the trained model in the memory unit 620 of the learning server 600.

[0106] As described above using the flowcharts in Fig. 5 and Fig. 6, steps S401 to S404 in Fig. 4 correspond to the learning phase in this embodiment. Then, the trained model that has undergone machine learning in the learning phase has a lens performance estimation function.

[0107] Next, the performance estimation and quality evaluation processing of the interchangeable lens 700 using the trained model in step S405 shown in the flowchart of Fig. 4 will be described in detail below with reference to Fig. 7, which is a subflow of S405. Fig. 7 is a flowchart illustrating an example of the processing flow of lens performance estimation and quality evaluation in this embodiment.

[0108] First, in step S701, the interchangeable lens 700 that is the subject of quality evaluation is connected to the user terminal 800. Then, the control unit 760 of the interchangeable lens 700 acquires first information, which is the usage state of the interchangeable lens 700, stored in the internal memory 740, and transmits it to the user terminal 800 via the communication unit 750. After transmission, the control unit 860 stores it in the storage unit 840 of the user terminal 800.

[0109] Next, in step S702, the control unit 860 transmits the first information acquired in step S701 to the server device 900 via the communication unit 850. After transmission, the control unit 930 stores the first information in the memory unit 920 of the server device 900. Next, in step S703, the control unit 440 transmits second information of the same model as the interchangeable lens 700 stored in the memory unit 430 of the server device 400 to the server device 900 via the communication unit 410. After transmission, the control unit 930 stores the second information in the memory unit 920 of the server device 900.

[0110] Next, in step S704, the control unit 930 transmits the first information and the second information stored in the storage unit 920 to the estimation server 1000 via the communication unit 910. After transmission, the first information and the second information are stored in the storage unit 1020 of the estimation server 1000 by the processing unit 1030.

[0111] Next, in step S705, the processing unit 1030 checks and determines whether the lens performance estimation function stored in the storage unit 1020 is the latest trained model by connecting to the learning server 600 via the communication unit 1010. If the result of the determination indicates that it is the latest trained model, the process proceeds to step S707, and if it is not the latest trained model, the process proceeds to step S706.

[0112] Next, in step S706, the processing unit 1030 acquires the trained model from the storage unit 620 of the learning server 600 via the communication unit 1010. After acquiring the trained model, the processing unit 1030 stores it in the storage unit 1020 of the estimation server 1000.

[0113] Next, in step S707, the processing unit 1030 performs missing value processing, normalization, categorical variable conversion, etc., on the first information and second information acquired in step S705 as the case may be, and converts the first information and second information into data that can be input to the trained model. Note that the converted data becomes converted data.

[0114] Next, in step S708, the processing unit 1030 inputs the converted data into the trained model, and generates and acquires third information that is estimated lens performance (second processing step). Note that in this embodiment, the processing unit 1030 also functions as a second processing unit that generates the third information. Then, the processing unit 1030 estimates various lens performances of the interchangeable lens 700 based on the third information.

[0115] Next, in step S709, the processing unit 1030 judges and evaluates the lens quality of the interchangeable lens 700 based on the third information (the estimated result of the lens performance) generated in step S708. Note that in this embodiment, the processing unit 1030 also functions as a judgment unit that makes a predetermined judgment on the interchangeable lens 700 based on the third information and evaluates the lens quality.

[0116] Next, in step S710, the processing unit 1030 transmits the lens quality evaluation result of the interchangeable lens 700 to the user terminal 800 via the communication unit 1010. After transmission, the lens quality evaluation result is displayed on the display unit 830 of the user terminal 800 by the control unit 860.

[0117] In this way, by carrying out the process exemplified in the flowchart of FIG. 7, it is possible to evaluate the lens quality of the interchangeable lens 700 owned (possessed) or temporarily managed by any user.

[0118] As described above, by using the quality evaluation method (information processing method) using the information processing system of this embodiment, it is possible to estimate lens performance while suppressing the discrepancy between durability information and changes in lens performance due to actual operation. This makes it possible to provide an information processing device that can easily evaluate the lens quality of each individual interchangeable lens according to its usage state with high accuracy.

[0119] In this embodiment, a large amount of learning data is required to realize highly accurate lens performance estimation by machine learning. In this embodiment, by utilizing a maintenance base to acquire the learning data as described above, it is possible to easily acquire a large amount of learning data. Conventionally, various interchangeable lenses brought to a maintenance base are subjected to a process of checking the lens performance before maintenance work. In other words, since a large amount of learning data has already been accumulated at maintenance bases around the world, the learning data can be utilized in this embodiment.

[0120] Even after the information processing system (quality evaluation system) in this embodiment starts operating, it is possible to increase the amount of learning data for machine learning without adding a new work process at the maintenance base. Note that, in the process of checking the lens performance before maintenance work, if there are items that have not been checked in the past or items that should be newly checked, these items may be newly added to the process of checking the lens performance.

[0121] Furthermore, to perform a highly accurate quality evaluation of each individual interchangeable lens, it was necessary to directly measure each evaluation item. For example, measurement of the drive characteristics of each drive unit can be performed using a control unit that issues drive commands in the lens and a detection system for the drive unit, and can be performed using, for example, only a commercially available imaging device. On the other hand, measurement of optical characteristics requires dedicated equipment such as a measurement chart and a drive stage, and checking the degree of wear of the lens barrel components requires disassembly of the lens as necessary. Therefore, lens quality evaluation needs to be performed in a place with a suitable measurement environment, such as a maintenance base.

[0122] In the quality evaluation method of this embodiment, the user can evaluate the lens quality as long as he or she can transmit the records in the internal memory 740 of the interchangeable lens 700 to the estimation server 1000 via the user terminal 800. In other words, by applying the information processing system of this embodiment, the burden on the user is reduced, and highly accurate lens quality evaluation becomes possible with ease.

[0123] Furthermore, application of the quality evaluation system of this embodiment makes it possible to easily assist users in determining whether maintenance is necessary, estimate maintenance costs and process time, assess the price of used lenses, and identify faulty parts. In addition, application of the quality evaluation system of this embodiment makes it easy for users themselves to evaluate lens quality. This can have many effects, such as improved reliability of products, construction of a stable used market by providing a basis for pricing used lenses, stabilization of the asset value of interchangeable lenses, and the associated lowering of the barrier to entry for new users. EXAMPLES

[0124] If there is variation in the initial performance of individual lenses, even if they are used in the same way, there will be some difference in the performance of each lens after use. In this embodiment, a quality evaluation method will be described for a case where there is a difference in performance between interchangeable lenses from the time of shipment due to manufacturing errors or assembly errors of the parts in the interchangeable lenses.

[0125] Fig. 8 is an example of a configuration diagram of an information processing system (quality evaluation system) in this embodiment. The information processing system of this embodiment is configured by adding a lens performance measurement device at the time of shipment (hereinafter, measurement device) 1100 and a lens performance record data server at the time of shipment (hereinafter, server device) 1200 to Fig. 1 shown as an example of the system configuration diagram of embodiment 1. Note that, other than the measurement device 1100 and the server device 1200, the configuration is the same as that of the information processing system of embodiment 1, so that explanations of overlapping parts will be omitted.

[0126] The measuring device 1100 is a measuring device that measures the performance of a lens before shipping. The configuration of the measuring device 1100 in this embodiment is similar to that of the measuring device 200 in the first embodiment, and therefore a description thereof will be omitted.

[0127] The server device 1200 is a database that holds and manages sixth information, which is the lens performance of the interchangeable lens 100 at the time of shipment, acquired by the measuring device 1100, for each individual interchangeable lens 100. The server device 1200 may include a communication unit 1210 that can communicate with the measuring device 1100, the server device 500, and the server device 900. The server device 1200 may further include a storage unit 1220 that holds (stores) various data, and a control unit 1230 that controls the server device 1200 as a whole.

[0128] The server apparatus 1200 may be the same as the server apparatus 500 or may be the same as the server apparatus 900 .

[0129] The sixth information indicating the lens performance at the time of shipment will be described below. The sixth information may be the same as the fourth information in the first embodiment.

[0130] Furthermore, the sixth information may be the driving characteristics of various actuators that drive the driving units of the interchangeable lens. The driving characteristics may include, for example, a starting voltage, a starting frequency, a maximum speed, a step-out speed, a driving voltage, etc., depending on the various actuators.

[0131] By acquiring the above-mentioned various items using the measurement device 1100, it is possible to grasp various initial characteristics, performance variations, and the like of each interchangeable lens at the time of shipment. In this embodiment, sixth information is added to the first information, which is the lens quality state, as initial lens performance.

[0132] Fig. 9 is a flowchart showing the process flow of lens quality evaluation including lens performance measurement at the time of shipment. Below, with reference to Fig. 9, a flow will be described from the learning phase to the estimation phase for evaluating the quality of an interchangeable lens whose performance at the time of shipment has been measured. Note that each operation (process) shown in the flowchart in Fig. 9 is controlled by each control unit or each processing unit executing a computer program. Note that the processes in steps S902 to S906 in Fig. 9 are similar to the processes in S401 to S405 in Fig. 4, and therefore description thereof will be omitted.

[0133] First, in step S901, a predetermined interchangeable lens is measured, and individual identification information of the interchangeable lens is generated and then acquired by the measurement device 1100. In addition, sixth information indicating the lens performance of the interchangeable lens 100 (interchangeable lens 700) at the time of shipment is acquired by the measurement device 1200. Thereafter, the processes of steps S902 to S906 are performed.

[0134] Next, the process of acquiring the performance at the time of shipment of an interchangeable lens in S901 shown in the flowchart of Fig. 9 will be described in detail below with reference to Fig. 10, which is a sub-flow of S901. Fig. 10 is a flowchart illustrating the process of measuring the lens performance at the time of shipment in this embodiment.

[0135] First, in step S1001, a predetermined interchangeable lens is mechanically and electrically connected to the measurement device 1100. As a result, individual identification information of the interchangeable lens is generated. Then, the individual identification information is stored in the internal memory 140 (internal memory 740) by the control unit 160 (control unit 760), and in the storage unit 244 by the control unit 246.

[0136] Next, in step S1002, performance measurement items and conditions and measurement start timing are input from the input unit of the measurement terminal 240, and the control unit 246 of the measurement terminal 240 transmits a measurement drive command to the imaging device 230 as a performance measurement instruction for the interchangeable lens. Next, in step S1003, based on the measurement drive command transmitted from the control unit 246 to the imaging device 230 via the communication unit 242, the control unit 235 transmits a measurement drive command to the interchangeable lens 100 (interchangeable lens 700) via the communication unit 232.

[0137] Next, in step S1004, the control unit 235 acquires a video signal of the imaging unit according to the operation of the interchangeable lens 100 (interchangeable lens 700). After acquisition, the control unit 235 transmits the video signal of the imaging unit to the measurement device 1100. The video signal of the imaging unit is transmitted to the measurement terminal 240 via the communication unit 232 and the communication unit 242 of the measurement device 1100. After transmission, the control unit 246 inputs data into each table of MTF% data, and the data is stored in the storage unit 244 of the measurement terminal 240. At this time, although omitted in the flow in FIG. 10, each performance data, driving characteristic data, etc. of the interchangeable lens 100 (interchangeable lens 700) are also sequentially acquired by performing the same process. The acquired performance data, driving characteristic data, etc. are stored in the storage unit 244 of the measurement terminal 240 by the control unit 246 as sixth information.

[0138] Next, in step S1005, the control unit 246 transmits the individual identification information and the sixth information stored in the memory unit 244 in steps S1001 and S1004 to the server device 1200 via the communication unit 243. The transmitted individual identification information and the sixth information are associated with each other by the control unit 1230, and then stored in the memory unit 1220 of the server device 1200. It is also assumed that the above processing is performed in the interchangeable lens 100 and the interchangeable lens 700.

[0139] The flowchart shown in FIG. 10 describes a method for acquiring the sixth piece of information indicating the lens performance at the time of shipment in addition to the individual identification information of an interchangeable lens.

[0140] Next, the process of acquiring the lens state when the interchangeable lens 100 is brought in for maintenance in step S903 shown in the flowchart of Fig. 9 will be described in detail below with reference to Fig. 11, which is a subflow of S903. Fig. 11 is a flowchart illustrating the measurement process of the lens state when brought in to a maintenance base in this embodiment. Note that the processes of steps S1101 and S1103 to S1106 in Fig. 11 are similar to the processes of S501 and S503 to S506 in Fig. 5, respectively, and therefore their explanations will be omitted. Also, the process of step S1108 in Fig. 11 is similar to the process of step S508 in Fig. 5, and therefore its explanation will be omitted.

[0141] First, processing in step S1101 is performed, and then in step S1102, the control unit 160 transmits the first information, such as lens usage state information and lens model information, stored in the internal memory 140 of the interchangeable lens 100, to the measurement device 1100. The transmitted first information is stored in the storage unit 244 of the measurement terminal 240 by the control unit 246 via the communication unit 150. Furthermore, the control unit 160 transmits the individual identification information of the interchangeable lens stored in the internal memory 140 of the interchangeable lens 100 in step S901 to the measurement device 200. The transmitted first information is stored in the storage unit 244 of the measurement terminal 240 by the control unit 246 via the communication unit 150.

[0142] Next, the processing of steps S1103 to S1106 is performed. Next, in step S1107, the control unit 246 associates the individual identification information of the interchangeable lens stored in the storage unit 244 with the first information and the fourth information, and then transmits them to the server device 300 via the communication unit 243. After transmission, the control unit 330 stores them in the storage unit 320 of the server device 300. Then, the processing returns to step S1101. Then, the processing of step S1108 is performed.

[0143] Next, the processing of learning the performance estimation model in step S905 shown in the flowchart of Fig. 9 will be described in detail below with reference to Fig. 12, which is a sub-flow of S905. Fig. 12 is a flowchart illustrating the learning processing of the lens performance estimation model in this embodiment. Note that the processing of steps S1203 to S1210 in Fig. 12 is similar to the processing of steps S602 to S609 in Fig. 6, so the description will be omitted.

[0144] First, in step S1201, the control unit 330 of the server device 300 transmits the individual identification information, the first information, and the fourth information of the interchangeable lens actually used by the user to the server device 500 via the communication unit 310. After transmission, the information is stored in the storage unit 520 of the server device 500 by the control unit 530. Furthermore, the control unit 1230 of the server device 1200 transmits the individual identification information and the sixth information of each interchangeable lens 100 to the server device 500 via the communication unit 1210. After transmission, the information is stored in the storage unit 520 of the server device 500 by the control unit 530. Furthermore, the control unit 440 of the server device 400 transmits the second information of each model of the interchangeable lens to the server device 500 via the communication unit 410. After transmission, the information is stored in the storage unit 520 of the server device 500 by the control unit 530.

[0145] Next, in step S1202, the control unit 530 adds sixth information, which is the lens performance at the time of shipment, to the first information, which is the lens quality state stored in the server device 500 based on the individual identification information of the interchangeable lens. Then, the processing of steps S1203 to S1210 is performed.

[0146] Next, the processing of learning the performance estimation model in step S906 shown in the flowchart of Fig. 9 will be described with reference to Fig. 13, which is a sub-flow of S906. Fig. 13 is a flowchart illustrating the processing of lens performance estimation and quality evaluation in this embodiment. Note that steps S1305 to S1312 in Fig. 13 are similar to the processing in steps S703 to S710 in Fig. 7, and therefore description thereof will be omitted.

[0147] First, in step S1301, the interchangeable lens 700 that is the subject of quality evaluation is connected to the user terminal 800. Then, the control unit 760 of the interchangeable lens 700 acquires the individual identification information of the interchangeable lens stored in the internal memory 740 and first information that is the usage state of the interchangeable lens, and transmits it to the user terminal 800 via the communication unit 750. After transmission, the control unit 860 stores it in the storage unit 840 of the user terminal 800.

[0148] Next, in step S1302, the control unit 860 transmits the individual identification information of the interchangeable lens acquired in step S1301 and the first information to the server device 900 via the communication unit 850. After transmission, the control unit 930 stores the information in the memory unit 920 of the server device 900.

[0149] Next, in step S1303, based on the individual identification information of the interchangeable lens acquired in step S1301, the control unit 930 acquires sixth information of the same individual as the interchangeable lens 700 stored in the storage unit 1220 of the server device 1200 via the communication unit 910. After acquisition, the sixth information is stored in the storage unit 920 of the server device 900 by the control unit 930.

[0150] Next, in S1304, the control unit 930 adds sixth information of the same individual as the interchangeable lens 700 to the first information acquired by the user terminal 800 and stored in the storage unit 920. Thereafter, the processing of steps S1305 to S1312 is performed.

[0151] As described above, by using the quality evaluation method using the information processing system of this embodiment, it is possible to provide an information processing device that can evaluate lens quality taking into account individual variations at the time of shipment by acquiring the relationship between the performance at the time of shipment of each lens and the lens performance and the usage state through machine learning. Furthermore, it is possible to estimate lens performance that reflects the influence of individual variations at the time of shipment.

[0152] In addition, in this embodiment, when evaluating lens quality based on estimated lens performance, it is possible to evaluate not only absolute performance values ​​(absolute values) but also relative values ​​compared to the performance at the time of shipment. This allows the user to understand the lens quality in the form of the degree of performance change compared to the previous lens usage experience. Furthermore, the above relative evaluation method is also effective for individuals whose lens usage conditions are close to those at the time of shipment, where individual variations are likely to appear in the lens quality evaluation.

[0153] In addition, an inspection device that generates third information, which is estimated lens performance, based on the trained model generated by the information processing device described in each of the above embodiments, and further evaluates the quality of a predetermined interchangeable lens based on the third information may be added to the information processing system of each of the above embodiments. In this way, even an inspection device that cannot generate a trained model can perform the process of evaluating the lens quality of the interchangeable lens described in each of the above embodiments. The inspection device may include an acquisition unit (third acquisition unit) that acquires a trained model for a predetermined lens and acquires the first information and the second information. Furthermore, the inspection device may include a processing unit (second processing unit) that generates third information, which is estimated lens performance of the interchangeable lens, based on the first information, second information, and trained model acquired by the acquisition unit, and evaluates the predetermined interchangeable lens quality based on the third information. Note that the inspection device may be connected to the information processing device described in each of the above embodiments by a line, or may be an inspection device incorporating any of the information processing devices described in each of the above embodiments.

[0154] Although the preferred embodiments of the present invention have been described above, the above-mentioned Examples 1 and 2 may be appropriately combined to form an embodiment, and the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0155] In addition, a part or all of the control in the above embodiment may be realized by a computer that realizes the functions of the above embodiment. The computer program may be supplied to each of the information processing devices, server devices, inspection devices, etc., via a network or various storage media. Then, a computer (or a CPU, MPU, etc.) in the information processing device, server device, inspection device, etc. may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0156] 100 Replacement Lenses (Study Lenses) 200 Measurement equipment (lens performance measurement equipment) 300 Server device (maintenance record data server) 400 Server device (lens development data server) 500 Server device (learning data collection server) 600 Learning Server 700 Replacement Lenses (Estimated Replacement Lenses) 800 User Terminals 900 Server device (estimation data collection server) 1000 Estimated Servers

Claims

1. When information regarding the usage state of a lens unit after shipment is defined as first information, information regarding lens performance or specifications obtained during the development of the lens unit is defined as second information, and information regarding estimated lens performance including the optical performance of the lens unit is defined as third information, a first acquisition unit that acquires the first information of a first lens unit having at least one optical element; a second acquisition unit that acquires the second information of the first lens unit or a second lens unit different from the first lens unit; A first processing unit that generates a trained model by using the first information acquired by the first acquisition unit and the second information acquired by the second acquisition unit as input data of a predetermined learning model; A second processing unit that generates the third information of a third lens unit other than the first and second lens units by inputting the first and second information of the third lens unit into the trained model; and 13. An information processing device comprising:

2. 2 . The information processing apparatus according to claim 1 , wherein the first information includes at least one of drive or control information for a lens unit, temperature information, humidity information, position information, and information relating to an external force applied to the lens unit.

3. 3. The information processing apparatus according to claim 1, wherein the second information includes at least one of environmental test information, durability test information, load test information, vibration test information, and impact test information of the lens unit.

4. 4. The information processing device according to claim 3, wherein the second information further includes at least one of optical performance information, operational performance information, dustproof or drip-proof performance information, evaluation information on appearance or operating state, and wear degree information of components of the lens unit after performing a specified test on the lens unit.

5. The information processing device described in Claim 1, characterized in that the learned model is generated by inputting fourth information, which is the performance of the lens in the first lens unit, as teacher data for the learning model.

6. 6. The information processing device according to claim 5, wherein the fourth information includes at least one of optical performance information, operational performance information, dustproof or drip-proof performance information, and information regarding appearance or operating state of the first lens unit.

7. 7. The information processing apparatus according to claim 6, wherein the first information or the fourth information is acquired when a predetermined inspection is performed after shipment.

8. 5. The information processing apparatus according to claim 3, wherein the second information further includes fifth information including at least one of design information, manufacturing information, and catalog data of the lens unit.

9. 5. The information processing device according to claim 3, wherein the second information is pre-recorded information about a lens obtained after various quality tests during development of the lens unit.

10. 3. The information processing device according to claim 1, wherein the first acquisition unit further acquires sixth information including at least one of optical performance information, operational performance information, dustproof or drip-proof performance information, and information regarding appearance or operating status at the time of shipment of the first lens unit.

11. 2 . The information processing apparatus according to claim 1 , wherein the third information includes at least one of operational performance information of a lens unit, dust-proof or drip-proof performance information, and information on an appearance or an operating state.

12. The information processing apparatus according to claim 1 , further comprising a determination unit that performs a predetermined lens quality determination for the third lens unit based on the third information.

13. When information regarding the usage state of a lens unit after shipment is defined as first information, information regarding lens performance or specifications obtained during development of the lens unit is defined as second information, and information regarding estimated lens performance including optical performance of the lens unit is defined as third information, a first acquisition step of acquiring the first information of a first lens unit having at least one optical element; a second acquisition step of acquiring the second information of the first lens unit or a second lens unit different from the first lens unit; A first processing step of generating a trained model by using the first information acquired by the first acquisition step and the second information acquired by the second acquisition step as input data for a predetermined learning model; A second processing step of generating the third information of a third lens unit other than the first and second lens units by inputting the first and second information of the third lens unit into the trained model; 13. An information processing method comprising:

14. A program causing a computer to execute the information processing method according to claim 13.

15. An information processing device according to any one of claims 1 to 12, a determination unit that determines a lens quality of the third lens unit based on the third information; An information processing system comprising:

16. An acquisition unit that acquires the trained model generated by the information processing device according to any one of claims 1 to 12; A processing unit that generates the third information of the lens unit to be inspected by inputting the first and second information of the lens unit to be inspected into the trained model; An inspection device comprising:

17. An acquisition step of acquiring the trained model generated by the information processing method according to claim 13; A processing step of generating the third information of the lens unit to be inspected by inputting the first and second information of the lens unit to be inspected into the trained model; An inspection method comprising the steps of:

Images