Model generation method, model generation device, EVS camera, and camera system

Abstract

To reduce the cost required for deriving an appropriate sensor parameter, for an EVS camera, according to an environment of use.SOLUTION: A model generation method according to the present technology is configured so that an information processing device executes a process for generating a parameter derivation model for deriving a sensor parameter for realizing a target sensor characteristic, in a use environment indicated by designation information with respect to an input of information about the target sensor characteristic and use environment designation information, the process being executed on the basis of: sensor characteristic evaluation information for each of a plurality of trial conditions which each consist of different combinations of a use environment for an EVS camera having EVS and an EVS sensor parameter, the sensor characteristic evaluation information being obtained by evaluating EVS sensor characteristics when an EVS-based event detection operation is executed for each of the trial conditions; use environment information indicating the use environment and the sensor parameter for each trial condition; and parameter information.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present technology relates to a model generation method, a model generation device, an EVS camera, and a camera system, and in particular to a technology for deriving appropriate sensor parameters for an EVS. [Background technology]

[0002] For example, an event-based vision sensor (EVS) is known that detects a change in the amount of received light in a predetermined manner as an event, as disclosed in Patent Document 1 and the like. Compared with an image sensor, that is, a sensor that obtains a grayscale image that indicates the amount of received light for each pixel using a predetermined number of grayscale levels, an EVS can increase the target detection speed, and therefore is being considered for use in surveillance cameras, measurement of fast-moving objects (such as measuring speed), measurement of fast-rotating objects (such as measuring vibration frequency), and the like.

[0003] As a related prior art, the following Patent Document 2 can also be cited. Patent Document 2 discloses a technology for adjusting the event detection sensitivity of an EVS in accordance with the number of events detected. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2022-108329 [Patent Document 2] Patent Publication No. 2021-197710 Summary of the Invention [Problem to be solved by the invention]

[0005] It is known that with EVS, the appropriate sensor parameters, in other words, the sensor parameters that can achieve the target sensor characteristics, vary depending on the usage environment, such as the type of subject in the scene, and it is desirable to be able to adjust the sensor parameters to be appropriate depending on the usage environment.

[0006] Currently, the method of manually deriving appropriate sensor parameters according to the usage environment is to use the sensor characteristics as evaluation indices. However, in EVS, there is a complex correlation between sensor parameters and sensor characteristics, and deriving appropriate sensor parameters manually requires a great deal of effort and time.

[0007] This technology was developed in consideration of the above circumstances, and aims to reduce the cost required to derive appropriate sensor parameters for EVS cameras according to the usage environment. [Means for solving the problem]

[0008] The model generation method according to the present technology is a model generation method in which an information processing device executes a process of generating a parameter derivation model that derives the sensor parameters for realizing the target sensor characteristics under the usage environment indicated by the specification information for the usage environment in response to input of information on target sensor characteristics, which are the target sensor characteristics, and specification information for the usage environment, based on sensor characteristic evaluation information for each trial condition obtained by evaluating the sensor characteristics of the EVS when an event detection operation is performed by the EVS for each of a plurality of trial conditions that differ in combination of the usage environment of an EVS camera having an EVS and the sensor parameters of the EVS, and usage environment information and parameter information that respectively indicate the usage environment and the sensor parameters for each trial condition. By using the parameter derivation model described above, it becomes possible to eliminate the need to manually derive sensor parameters to achieve target sensor characteristics in the assumed EVS camera usage environment. Even if some of the work is done manually, it is possible to at least provide support for adjustment. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram for explaining an overview of a parameter derivation method according to the present technology; [Figure 2] FIG. 1 is a diagram illustrating an example of the configuration of a learning data generation system according to an embodiment. [Figure 3] FIG. 1 is a diagram illustrating an example of the configuration of an EVS camera used to generate learning data. [Figure 4] FIG. 2 is a diagram illustrating an example of the configuration of a pixel included in the EVS. [Figure 5] FIG. 2 is a diagram for explaining an example of the internal configuration of an output control / reset circuit included in the EVS according to the embodiment. [Figure 6] 4A to 4C are diagrams illustrating examples of usage environment information and sensor characteristic information according to an embodiment. [Figure 7] FIG. 4 is a diagram illustrating an example of sensor parameters according to the embodiment. [Figure 8] FIG. 1 illustrates an example of the configuration of a model generating device according to an embodiment. [Figure 9] FIG. 10 is an explanatory diagram of output of compromise proposal information. [Figure 10] FIG. 10 is a diagram illustrating an example of compromise information according to an embodiment. [Figure 11] FIG. 1 is a diagram for explaining a configuration example of an EVS camera according to an embodiment. [Figure 12] 10 is a flowchart illustrating a process for parameter setting according to an embodiment. [Figure 13] 13 is a flowchart of the parameter derivation control process shown in FIG. 12. [Figure 14] FIG. 10 is an explanatory diagram of a first use case. [Figure 15]FIG. 10 is a diagram illustrating examples of sensor characteristic information, usage environment information, sensor parameters, and compromise information used in a first use case. [Figure 16] FIG. 10 is an explanatory diagram of a second use case. [Figure 17] FIG. 10 is a diagram illustrating examples of sensor characteristic information, usage environment information, sensor parameters, and compromise information used in a second use case. [Figure 18] FIG. 10 is an explanatory diagram of a third use case. [Figure 19] FIG. 10 is a diagram illustrating examples of sensor characteristic information, usage environment information, sensor parameters, and compromise information used in a third use case. [Figure 20] FIG. 10 is an explanatory diagram of a fourth use case. [Figure 21] FIG. 10 is a diagram illustrating examples of sensor characteristic information, usage environment information, sensor parameters, and compromise information used in a fourth use case. [Figure 22] FIG. 10 is a diagram illustrating an example of the configuration of an EVS camera to be used when re-learning a parameter derivation model. [Figure 23] FIG. 10 is a diagram illustrating an example of the configuration of a server device to be used when re-learning a parameter derivation model. [Figure 24] FIG. 10 is a diagram illustrating a configuration example of an EVS camera as a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present technology will be described in the following order with reference to the accompanying drawings. <1. Parameter derivation method as an embodiment> [1-1. Overview of derivation method] [1-2. Learning data generation system] [1-3.EVS camera used to generate learning data] [1-4. Examples of usage environment information and sensor characteristic information] [1-5. Methods for generating training datasets] [1-6. Generation of parameter derivation model] [1-7. Parameter derivation model as an embodiment] [1-8. EVS camera as an embodiment] [1-9. Processing for parameter setting] [1-10. Application Examples] <2. Modifications> <3. Summary of the embodiment> <4. This technology>

[0011] <1. Parameter derivation method as an embodiment> [1-1. Overview of derivation method] FIG. 1 is a diagram for explaining an outline of a parameter derivation method according to an embodiment. As a premise, the parameter derivation method of this embodiment derives appropriate parameters as sensor parameters of an EVS (Event-based Vision Sensor) according to the usage environment of an EVS camera having an EVS.

[0012] Here, EVS refers to a sensor that detects a change in the amount of received light (brightness) in a predetermined manner as an event. Specifically, in this embodiment, the EVS refers to a sensor that includes a plurality of pixels each having a light receiving element, each configured to be able to detect a change in the amount of received light in a predetermined manner as an event, and capable of outputting information indicating the position of the pixel where the event was detected and information indicating the time the event was detected.

[0013] In the parameter derivation method of this embodiment, a parameter derivation model MD for deriving sensor parameters is generated. The parameter derivation model MD is a model that derives sensor parameters for realizing the target sensor characteristics in the usage environment indicated by the input of target sensor characteristic information (EVS sensor characteristics) and specification information of the usage environment of the EVS camera. By using such a parameter derivation model, it becomes possible to eliminate the need for manual derivation of sensor parameters to achieve target sensor characteristics in the assumed EVS camera usage environment. Even if some of the derivation is done manually, it is possible to at least provide support for adjustment.

[0014] [1-2. Learning data generation system] FIG. 2 is a diagram showing an example of the configuration of a learning data generation system 10 according to an embodiment. The learning data generation system 10 is a system for generating a learning data set DS used to generate a parameter derivation model MD.

[0015] As shown in the figure, the learning data generation system 10 includes an EVS camera 2′, an information processing device 11, and a subject reproduction device 12. The EVS camera 2' is a camera device equipped with an EVS (EVS 2a, which will be described later) as a light receiving sensor. As shown in the figure, the EVS camera 2' is provided with an environmental information detection unit 15.

[0016] The environmental information detection unit 15 performs processing to detect usage environment information that indicates the usage environment of the EVS camera 2'. There is a limit to the usage environment information that can be detected by the EVS camera 2', and the environmental information detection unit 15 does not necessarily detect all items of usage environment information. In this example, some items of usage environment information can be detected by the environmental information detection unit 12a provided in the subject reproduction device 12. The specific items of information to be detected as the usage environment information will be explained later.

[0017] The EVS camera 2' outputs to the information processing device 11 characteristic evaluation information that is information used to evaluate the sensor characteristics of the EVS 2a.

[0018] The information processing device 11 is configured as a computer device equipped with a microcomputer having, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and the CPU executes processing in accordance with, for example, a program stored in the ROM or a program loaded into the RAM, thereby realizing various functions as the information processing device 11. The information processing device 11 includes a characteristic evaluation unit 16. The characteristic evaluation unit 16 evaluates the sensor characteristics of the EVS 2a based on characteristic evaluation information input from the EVS camera 2′, and obtains sensor characteristic evaluation information indicating the results of the evaluation. Regarding the evaluation of the sensor characteristics, the specific items to be evaluated will be explained later. In this example, the characteristic evaluation information comprehensively indicates information required to evaluate the sensor characteristics for each item described later.

[0019] The subject reproduction device 12 is configured as a device for reproducing a subject of the EVS camera 2'. Specifically, the subject reproduction device 12 in this embodiment is configured to have, for example, multiple light sources and to be able to irradiate the EVS camera 2' with pattern light that is emitted in an arbitrary two-dimensional pattern. The subject reproduction device 12 is configured to be able to adjust the light emission mode of the above-mentioned pattern light, specifically the light emission intensity distribution, light emission cycle, etc., and by adjusting the light emission mode of the pattern light in this way, it is possible to simulate and reproduce various subjects in various scenes. In other words, it is possible to reproduce various usage environments, including different types of subjects. The subject reproduction device 12 of this example is provided with an environmental information detection unit 12a. Examples of environmental information that the environmental information detection unit 12a detects include information on the brightness of the subject and the brightness of the background.

[0020] In the learning data generation method of the embodiment, the information processing device 11 controls the adjustment of the pattern light of the subject reproduction device 12 and controls the setting of the sensor parameters of the EVS camera 2', thereby causing the EVS camera 2' to perform an event detection operation under various sensor parameter setting states and various usage environment reproduction states. In other words, the event detection operation by the EVS 2a is performed for each of a plurality of trial conditions that vary the combination of the usage environment and sensor parameters of the EVS camera 2'.

[0021] In this system, each time an event detection operation is performed for each of the above trial conditions, the environmental information detection unit 15 and the environmental information detection unit 12a detect the above-mentioned usage environment information, and the characteristic evaluation unit 16 evaluates the above-mentioned sensor characteristics to obtain sensor characteristic evaluation information. The usage environment information obtained for each trial condition by the environment information detection unit 15 or the environment information detection unit 12a is input to the information processing device 11 as shown in the figure.

[0022] The information processing device 11 generates a learning dataset DS by associating the usage environment information and sensor characteristic evaluation information for each trial condition with the sensor parameters instructed to be set for the EVS camera 2' for each trial condition. Specifically, the learning dataset DS is generated by associating the usage environment information, sensor characteristic evaluation information, and sensor parameters for each trial condition.

[0023] [1-3.EVS camera used to generate learning data] FIG. 3 is a diagram showing an example of the configuration of the EVS camera 2′ shown in FIG. As shown in the figure, the EVS camera 2' includes an EVS 2a, a camera control unit 13', and a communication unit 14, as well as the environmental information detection unit 15 described above.

[0024] As shown in the figure, the EVS 2a includes a pixel array section 21 in which a plurality of pixels 20 are two-dimensionally arranged, an X arbiter 22, a Y arbiter 23, an event processing circuit 24, and an output I / F (interface).

[0025] In the pixel array section 21, a plurality of pixels 20 are arranged in the row direction (horizontal direction: X direction in the drawing) and the column direction (vertical direction: Y direction in the drawing). Each pixel 20 has a light receiving element and is configured to be able to detect an event. Specifically, each pixel 20 is configured to detect, as an event, that the ratio (contrast) of the post-change luminance to the reference luminance exceeds a threshold. In this example, each pixel 20 is configured to be able to detect two types of events: a "positive polarity event (also called an on event)" in which the brightness changes to an increasing direction, and a "negative polarity event (also called an off event)" in which the brightness changes to a decreasing direction.

[0026] When each pixel 20 detects an event, regardless of whether it is a positive polarity event or a negative polarity event, it outputs a request signal to the X arbiter 22 and the Y arbiter 23 to request that an event signal be read (output) from itself. As shown in the figure, the request signal to the X arbiter 22 is referred to as a "request signal Xrq," and the request signal to the Y arbiter 23 is referred to as a "request signal Yrq." Each pixel 20 outputs an event signal in accordance with arbitration by the X arbiter 22 and the Y arbiter 23 in response to the request signals Xrq and Yrq. Specifically, each pixel 20 outputs a positive event signal Ip indicating the detection result of a positive event and a negative event signal Im indicating the detection result of a negative event to the event processing circuit 24 in response to receiving a response signal Xac output by the X arbiter 22 as a response (ACK) to the request signal Xrq and a response signal Yac output by the Y arbiter 23 as a response to the request signal Yrq.

[0027] It is not essential to provide both the X arbiter 22 and the Y arbiter 23, and it is also possible to provide only one of them.

[0028] The X arbiter 22 and the Y arbiter 23 arbitrate the request signals (the above-mentioned request signals Xrq and Yrq) from each pixel 20, and transmit a response based on the arbitration result (permission / non-permission of the output of the event signal) as the above-mentioned response signal (Xac and Yac) to the pixel 20 that output the request signal.

[0029] The event processing circuit 24 generates event data for the pixel 20 based on the event signals (in this example, the positive event signal Ip and the negative event signal Im) input from the pixel 20. The generated event data includes at least the position information of the pixel where the event was detected (address information in the pixel coordinate system: hereinafter referred to as "position information of the event-occurring pixel") and "detection time information" indicating the time when the event was detected. In this example, since positive and negative polarity events can be detected as events, the generated event data includes the position information and detection time information of the event-occurring pixel, as well as "event type information" indicating the type of the detected event (positive polarity event / negative polarity event).

[0030] Here, the event processing circuit 24 in this example is configured to be able to perform processing related to event filter settings. The event filter here refers to a filter related to the output of detected events, and refers to a filter that limits the output of events on a pixel-by-pixel or block-by-block basis in accordance with set conditions. Specific examples of set conditions for the event filter include, for example, outputting an event when an event is detected at a certain interval, outputting only the first detected event, outputting an event when the event polarity changes, or outputting an event when the same polarity continues a predetermined number of times after the event polarity changes.

[0031] The event processing circuit 24 in this example is also capable of performing processing related to ROI (Region Of Interest) setting and digital mask setting. The ROI setting is the setting of the range of pixels within which events are detected, and for example, the event processing circuit 24 outputs only events detected within the pixel range set as the ROI. The digital mask setting is a setting that masks the event detection results of pixels 20 that should not be used, such as defective pixels, and the event processing circuit 24 masks (does not output) the event detection results of the pixels 20 specified by the digital mask setting.

[0032] Furthermore, the EVS 2a in this example is capable of variably setting the read latency. The read latency here refers to the latency added to the read of each row when an event is detected. This read latency can be set, for example, by the Y-arbiter 23.

[0033] The output I / F 25 sequentially outputs the event data output by the event processing circuit 24 row by row to the outside of the EVS 2a, specifically to the camera control unit 13' in this example.

[0034] FIG. 5 is a diagram showing an example of the circuit configuration of the pixel 20. As shown in the figure, the pixel 20 includes a logarithmic conversion unit 31 including a photodiode PD as a light receiving element, a buffer 32, an event detection circuit 33, and an output control / reset circuit .

[0035] In the pixel 20, the logarithmic converter 31 converts the photocurrent (current corresponding to the amount of received light) obtained by the photodiode PD into a logarithmic voltage signal. The buffer 32 corrects the voltage signal input from the logarithmic converter 31 and outputs it to the event detection circuit 33 .

[0036] As shown in the figure, the logarithmic conversion unit 31 includes a transistor Q1, a transistor Q2, and a transistor Q3. In this example, the various transistors Q included in the pixel 20 are MOSFETs (metal-oxide-semiconductor field-effect transistors).

[0037] In the logarithmic converter 31, the transistors Q1 and Q3 are N-type transistors, and the transistor Q2 is a P-type transistor. The source of transistor Q1 is connected to the cathode of photodiode PD, and the drain is connected to a power supply terminal (reference potential VDD). Transistors Q2 and Q3 are connected in series between the power supply terminal and a ground terminal. The connection point between transistors Q2 and Q3 is connected to the gate of transistor Q1 and the input terminal of buffer 32 (the gate of transistor Q5, described later). A predetermined bias voltage bias_fo is applied to the gate of transistor Q2.

[0038] The drains of transistors Q1 and Q3 are connected to the power supply (reference potential VDD), forming a source follower circuit. These two source followers connected in a loop convert the photocurrent from the photodiode PD into a logarithmic voltage signal. Transistor Q2 also supplies a constant current to transistor Q3.

[0039] The buffer 32 includes a transistor Q4 and a transistor Q5, each of which is a P-type transistor, and these transistors Q4 and Q5 are connected in series between a power supply terminal and a ground terminal. The connection point between the transistor Q4 and the transistor Q5 serves as the output terminal of the buffer 32, and the corrected voltage signal is output from this output terminal to the event detection circuit 33 as a light reception signal.

[0040] The event detection circuit 33 detects a change in the amount of received light as an event by determining the difference between the past level of the light-receiving signal, which is a reference level Lref, and the current level of the light-receiving signal. Specifically, the event detection circuit 33 detects the presence or absence of an event based on whether the level (absolute value) of a differential signal representing the difference between the reference level Lref and the current level of the light-receiving signal is equal to or greater than a predetermined threshold. The event detection circuit 33 in this example is configured to be able to detect and distinguish between positive polarity events, i.e., events in which the difference from the reference level Lref is positive, and negative polarity events, i.e., events in which the difference from the reference level Lref is negative. The event detection circuit 33 generates a positive output voltage Vop as an output voltage indicating the detection result of a positive event, and a negative output voltage Vom as an output voltage indicating the detection result of a negative event.

[0041] Here, the event detection circuit 33 resets the reference level Lref to the current level of the light reception signal based on a reset signal RST, which will be described later. By resetting the reference level Lref in this way, it becomes possible to perform new event detection based on changes in the light-receiving signal level from the time of the reset. In other words, resetting the reference level Lref is equivalent to returning the event detection circuit 33 to a state where it can detect a new event.

[0042] The event detection circuit 33 includes a subtractor 34 and a quantizer 35 . The subtractor 34 reduces the level of the light reception signal (voltage signal) from the buffer 32 in accordance with the reset signal RST, and outputs the reduced light reception signal to the quantizer 35. The quantizer 35 quantizes the received light signal from the subtractor 34, and obtains an output voltage indicating the quantization result. Specifically, in this example, a positive output voltage Vop and a negative output voltage Vom are obtained.

[0043] The subtractor 34 includes capacitors C1 and C2, transistors Q7 and Q8, and a reset switch SWr. The transistor Q7 is a P-type transistor, and the transistor Q8 is an N-type transistor. Transistors Q7 and Q8 are connected in series between a power supply terminal and a ground terminal to form an inverter. Specifically, the source of transistor Q7 is connected to the power supply terminal, the drain is connected to the drain of transistor Q8, and the source of transistor Q8 is connected to the ground terminal. Voltage Vbdif is applied to the gate of transistor Q8. One end of capacitor C1 is connected to the output terminal of buffer 32, and the other end is connected to the gate of transistor Q7 (input terminal of the inverter). One end of capacitor C2 is connected to the other end of capacitor C1, and the other end is connected to the connection point of transistors Q7 and Q8. The reset switch SWr has one end connected to the connection point between the capacitors C1 and C2 and the other end connected to the connection point between the connection point between the transistors Q7 and Q8 and the capacitor C2, and is connected in parallel to the capacitor C2. The reset switch SWr is a switch that is turned ON / OFF in accordance with the reset signal RST. The inverter formed by the transistor Q7 and the transistor Q8 inverts the received light signal input via the capacitor C1 and outputs it to the quantizer 35.

[0044] Here, let us assume that the potential appearing on the buffer 32 side of capacitor C1 at a certain point in time in the subtractor 34 is potential Vinit. At this time, let us assume that the reset switch SWr is turned ON. When the reset switch SWr is ON, the side of capacitor C1 opposite to the buffer 32 becomes a virtual ground terminal. For convenience, let us assume that the potential of this virtual ground terminal is zero. At this time, the charge CHinit stored in capacitor C1 is expressed by the following [Equation 1], where Cp1 is the capacitance of capacitor C1. CHinit=Cp1×Vinit...[Formula 1] When the reset switch SWr is ON, both ends of the capacitor C2 are short-circuited, so that the stored charge therein becomes zero.

[0045] Next, assume that the reset switch SWr is turned OFF. If a change in the amount of received light occurs, the potential of the capacitor C1 on the buffer 32 side changes from the Vinit described above. If the potential after the change is Vafter, the charge CHafter accumulated in the capacitor C1 is expressed by the following [Equation 2]. CHafter=Cp1×Vafter ···[Formula 2]

[0046] On the other hand, the charge CH2 stored in the capacitor C2 is expressed by the following [Equation 3], where Cp2 is the capacitance of the capacitor C2 and Vout is the output voltage of the subtractor 34. CH2=-Cp2×Vout...[Formula 3]

[0047] At this time, the total charge amount of the capacitors C1 and C2 does not change, so the following [Equation 4] holds. CHinit=CHafter+CH2 ···[Formula 4]

[0048] Substituting [Equation 1] through [Equation 3] into [Equation 4] and rearranging it, we obtain the following [Equation 5]. Vout=-(Cp1 / Cp2)×(Vafter-Vinit)...[Formula 5] [Equation 5] represents the subtraction operation of the voltage signal, and the gain of the subtraction result is Cp1 / Cp2.

[0049] From this [Equation 5], it can be seen that the subtractor 34 outputs a signal that represents the difference between the level (Vinit) of the past light-receiving signal and the level (Vafter) of the current light-receiving signal. Here, the potential Vinit corresponds to the reference level Lref described above. As explained above, when the reset switch SWr is turned on, this potential Vinit, i.e., the reference level Lref, is reset to the current level of the light-receiving signal, in other words, the level of the light-receiving signal at the time the reset switch SWr is turned on.

[0050] The quantizer 35 includes a transistor Q9, a transistor Q10, a transistor Q11, and a transistor Q12, and is configured as a 1.5-bit quantizer. The transistors Q9 and Q11 are P-type transistors, and the transistors Q10 and Q12 are N-type transistors. As shown in the figure, transistors Q9 and Q10, and transistors Q11 and Q12 are connected in series between a power supply terminal and a ground terminal, respectively, and the output voltage (Vout) of the subtractor 34 is input to the gates of transistors Q9 and Q11. In addition, a voltage bias_diff_on is applied to the gate of transistor Q10, and a voltage bias_diff_off is applied to the gate of transistor Q12.

[0051] The junction of transistors Q9 and Q10 produces a positive output voltage Vop that indicates the detection result of a positive event, and the junction of transistors Q11 and Q12 produces a negative output voltage Vom that indicates the detection result of a negative event. Specifically, on the transistors Q9 and Q10 side, when the level of the output voltage (Vafter-Vinit) of the subtractor 34 is equal to or higher than a positive threshold corresponding to voltage Vhigh, a high-level positive output voltage Vop is obtained at the connection point between transistors Q9 and Q10, and when the level of the output voltage of the subtractor 34 is lower than the positive threshold, a low-level positive output voltage Vop is obtained. That is, a signal indicating whether the amount of received light has increased by more than a predetermined threshold, i.e., a positive output voltage Vop indicating the detection result of a positive event, is obtained at the connection point between transistors Q9 and Q10. Furthermore, on the transistor Q11, Q12 side, when the level of the output voltage of the subtractor 34 is equal to or lower than a negative threshold corresponding to the voltage Vlow, a high-level negative output voltage Vom is obtained at the connection point between the transistors Q11 and Q12, and when the level of the output voltage of the subtractor 34 is higher than the negative threshold, a low-level negative output voltage Vom is obtained. In this way, a signal indicating whether the amount of received light has decreased by more than a predetermined threshold, i.e., a negative output voltage Vom indicating the detection result of a negative event, is obtained at the connection point between the transistors Q11 and Q12.

[0052] The output control / reset circuit 36 ​​outputs request signals Xrq and Yrq to the X arbiter 22 and Y arbiter 23, respectively, and outputs a positive event signal Ip and a negative event signal Im in response to the response signals Xac and Yac, and also outputs a reset signal RST to the reset switch SWr.

[0053] FIG. 6 is a diagram for explaining an example of the internal configuration of the output control / reset circuit 36, and shows the event detection circuit 33 together with the example of the internal configuration of the output control / reset circuit 36. The output control / reset circuit 36 ​​includes a positive memory 37 p, a negative memory 37 m, an output circuit 38 , an OR circuit 39 , and a delay device 40 .

[0054] The positive output voltage Vop from the event detection circuit 33 is held in the positive output voltage Vop and is input to an OR circuit 39. The negative output voltage Vom from the event detection circuit 33 is held in a negative memory 37m and is input to the OR circuit 39. Here, the positive side memory 37p and the negative side memory 37m hold values ​​(digital values) corresponding to the voltage levels of the input positive side output voltage Vop and negative side output voltage Vom. Specifically, in this example, if the voltage levels of the positive side output voltage Vop and negative side output voltage Vom are H level, they hold "1", and if they are L level, they hold "0".

[0055] The OR circuit 39 outputs a signal at H level when at least one of the input positive output voltage Vop and negative output voltage Vom is at H level, and outputs a signal at L level when both are at L level. In other words, the OR circuit 39 sets its output signal to H level in response to the detection of either a positive event or a negative event, and sets its output signal to L level when neither event is detected. The output signals from the OR circuit 39 are sent to the X arbiter 22 and the Y arbiter 23 as the request signals Xrq and Yrq described above, respectively.

[0056] In response to receiving both the response signal Xac from the X arbiter 22 and the response signal Yac from the Y arbiter 23, the output circuit 38 outputs the value held in the positive side memory 37p and the value held in the negative side memory 37m to the event processing circuit 24 as a positive side event signal Ip and a negative side event signal Im, respectively. When a positive event is detected, a "1" is output as the positive event signal Ip and a "0" is output as the negative event signal Im, allowing the event processing circuit 24 to identify that a positive event has been detected in the corresponding pixel 20. On the other hand, when a negative event is detected, a "0" is output as the positive event signal Ip and a "1" is output as the negative event signal Im, allowing the event processing circuit 24 to identify that a negative event has been detected in the corresponding pixel 20.

[0057] The delay device 40 delays the output signal from the OR circuit 39 and outputs the delayed signal as a reset signal RST to the event detection circuit 33 (reset switch SWr). As a result, in response to the detection of either a positive polarity event or a negative polarity event, the reference level Lref described above is reset, and the event detection circuit 33 is reset to a state where it can detect a new event. The delay time of the delay device 40 should be set to at least the time required for the positive output voltage Vop and the negative output voltage Vom to be stored in the positive memory 37p and the negative memory 37m after they change to H level or L level, thereby preventing events from being overlooked.

[0058] As described above, the EVS2a is configured to output event data for a pixel 20 that has detected an event as a positive polarity event or a negative polarity event, i.e., data including position information of the pixel where the event occurred, detection time information indicating the time when the event was detected, and event type information indicating whether the event is a positive polarity event or a negative polarity event.

[0059] Here, in the EVS2a of this embodiment, it is possible to variably set the bias voltage bias_fo applied to the gate of transistor Q2 in the logarithmic conversion unit 31 shown in Figure 4, the bias voltage bias_hpf applied to the gate of transistor Q4 in the buffer 32, the voltage bias_diff_on applied to the gate of transistor Q10 in the quantizer 35, the voltage bias_diff_off applied to the gate of transistor Q12, and further the voltage bias_refr applied to the delay device 40 in the output control / reset circuit 36. The bias voltage bias_fo is a bias voltage for changing the LPF characteristics (cutoff frequency) of the logarithmic conversion unit 31 that functions as an LPF (Low Pass Filter), and the bias voltage bias_hpf is a bias voltage for changing the HPF characteristics (cutoff frequency) of the buffer 32 that functions as an HPF (High Pass Filter). As can be understood from the above explanation, the voltages bias_diff_on and bias_diff_off are voltages for setting the detection threshold for a positive polarity event and the detection threshold for a negative polarity event, respectively. The voltage bias_refr is a voltage for setting the delay time from the timing of the change in level of the positive output voltage Vop and the negative output voltage Vom until the reset signal RST is output.

[0060] The bias voltage bias_fo, the bias voltage bias_hpf, the voltage bias_diff_on, the voltage bias_diff_off, and the voltage bias_refr correspond to specific examples of "sensor parameters" according to the present technology.

[0061] The sensor parameters may also include the voltage bias_diff applied to the gate of the transistor Q8 of the subtractor .

[0062] In this example, the "sensor parameters" also include the event filter settings, ROI settings, digital mask settings, and readout latency described above.

[0063] In FIG. 3, the event data output by the EVS 2a is input to the camera control unit 13'. The camera control unit 13' is configured with a microcomputer equipped with, for example, a CPU, ROM, RAM, etc., and realizes various functions of the EVS camera 2' by the CPU executing processing in accordance with, for example, a program stored in the ROM or a program loaded into the RAM.

[0064] The camera control unit 13' is connected to a communication unit 14. The communication unit 14 is a communication device for performing wired or wireless data communication with an external device. The camera control unit 13' is capable of performing data communication with an external device such as the information processing device 11 shown in FIG.

[0065] When generating learning data, the camera control unit 13' performs control to set the sensor parameters instructed by the information processing device 11 in the EVS 2a. In addition, the environmental information detection unit 15 performs a process of transmitting the usage environment information detected for each trial condition and the sensor characteristic evaluation information obtained by the characteristic evaluation unit 16 by evaluating the sensor characteristics for each trial condition to the information processing device 11 via the communication unit 14.

[0066] [1-4. Examples of usage environment information, sensor characteristic information, and sensor parameters] Examples of the use environment information, sensor characteristic information, and sensor parameters in this embodiment will be described with reference to FIGS. FIG. 6 is an explanatory diagram of an example of sensor characteristic information and usage environment information. As shown in FIG. 6A, the sensor characteristic information in this example includes the amount of noise, the number of over-responsive pixels, the number of unresponsive pixels, the output event rate, the output data rate, the latency, the contrast threshold, and the dead time.

[0067] The amount of noise mentioned here refers to the BGR (Background Rate), which is the average value of the total number of events that occur per second under constant illumination conditions and no change in the light source, per pixel. The number of over-reacting pixels means pixels whose event detection rate is above a certain value, and the number of unreacting pixels means the number of defective pixels that are always OFF.

[0068] The output event rate means the output rate of events from the output I / F 25 per predetermined unit of time, such as 1 second (e.g., events / second), and the output data rate means the output data rate from the output I / F 25 per predetermined unit of time, such as 1 second (e.g., bytes / second).

[0069] The contrast threshold means a threshold for detecting an event, and in this example, there are both a threshold for detecting a positive polarity event and a threshold for detecting a negative polarity event. Latency refers to the time required from the detection of an event until the aforementioned detection time information (timestamp information indicating the time the event was detected) is assigned. The dead time refers to an insensitive period in which the pixel 20 cannot respond to a change in the amount of received light after an event is detected. This insensitive period as dead time occurs because a delay time is provided by the delay device 40 after the event is detected and before the reference level Lref is reset by the reset signal RST.

[0070] In this example, the characteristic evaluation unit 16 evaluates the sensor characteristics, including the amount of noise (BGR), the number of over-reacting pixels, the number of un-reacting pixels, the output event rate, the output data rate, the latency, the contrast threshold, and the dead time. For example, the amount of noise (BGR) can be calculated based on the output data from the output I / F 25. The number of over-responsive pixels can also be calculated based on the output data from the output I / F 25. The number of unresponsive pixels can be calculated based on the results of the detection process for the pixels 20 that are always OFF. The output event rate and the output data rate can be calculated as the rate of output events and the data rate of output data from the output I / F 25, respectively. The latency can be calculated as the time from the rising edge of the output signal (request signal Xrq or Yrq) of the OR circuit 39 in the pixel 20 until the detection time information is provided, for example. The contrast threshold can be obtained, for example, by detecting the voltage level difference between the time when the event detection starts (the time when resetting by the reset signal RST is completed) and the time of detection for each of the positive output voltage Vop and the negative output voltage Vom. The dead time can be calculated as the time difference between the rising edge of the output signal of the OR circuit 39 in the pixel 20 and the rising edge of the reset signal RST, for example.

[0071] It should be noted that the characteristic evaluation unit 16 may evaluate some sensor characteristics, such as the amount of noise (BGR) and the output data rate, based on the output data from the output I / F 25 by the camera control unit 13 ′.

[0072] FIG. 6B illustrates the usage environment information in this example. As shown in the figure, the usage environment information includes the illuminance on the sensor surface, the sensor temperature, whether or not the subject has periodicity, the duration of contrast change, the number of simultaneously firing rows, the number of simultaneously firing pixels within a row, and the interval between consecutive firings of the same pixel.

[0073] The illuminance on the sensor surface and the sensor temperature refer to the illuminance on the sensor surface (the surface on which the pixels 20 are formed) of the EVS 2a and the temperature of the EVS 2a, respectively. The presence / absence of subject periodicity is information indicating whether or not the movement of the subject has periodicity. The duration of contrast change is a value calculated by "L / V" where V is the object speed (pixels / μsec) and L is the object movement direction length (pixels). The number of simultaneously firing rows means the number of rows in which events occurred simultaneously, and the number of simultaneously firing pixels in a row means the number of pixels in which events occurred simultaneously in the same row. The same pixel consecutive firing interval means the interval between events occurring in pixels 20 where events occur consecutively.

[0074] In this example, the environmental information detection unit 15 detects the usage environment information by detecting the illuminance on the sensor surface, the sensor temperature, the presence or absence of subject periodicity, the duration of contrast change, the number of simultaneously firing rows, the number of simultaneously firing pixels within a row, and the interval between consecutive firings of the same pixel. The illuminance on the sensor surface can be detected, for example, based on the output signal of an illuminance sensor (not shown) that is configured to detect the illuminance on the sensor surface, and the sensor temperature can be detected, for example, based on the output signal of a temperature sensor (not shown) that is configured to detect the temperature of the sensor. The presence or absence of subject periodicity, duration of contrast change, number of simultaneously firing rows, number of simultaneously firing pixels within a row, and interval between consecutive firings of the same pixel can each be calculated based on the event data output from the output I / F 25.

[0075] It is also possible that the camera control unit 13' evaluates some of the environmental information, such as whether or not the subject has periodicity, the duration of contrast change, the number of simultaneously firing rows, the number of simultaneously firing pixels in a row, and the interval between consecutive firings of the same pixel, based on the output data from the output I / F 25. Furthermore, the environment information detection unit 12a can be used to detect the use environment information as needed.

[0076] FIG. 7 shows an example of the sensor parameters in this example. As shown, the sensor parameters in this example include the bias voltage bias_fo, bias voltage bias_hpf, voltage bias_diff_on, voltage bias_diff_off, and voltage bias_refr described above, as well as the event filter setting, ROI setting, digital mask setting, and readout latency described above.

[0077] [1-5. Methods for generating training datasets] 2, the information processing device 11 controls the adjustment of the pattern light of the subject reproduction device 12 and the setting of the sensor parameters of the EVS camera 2', thereby causing the EVS camera 2' to perform an event detection operation under various sensor parameter setting states and various usage environment reproduction states. In other words, the event detection operation by the EVS 2a is performed under multiple trial conditions that vary the combination of the usage environment and sensor parameters of the EVS camera 2'.

[0078] In this example, each time an event detection operation is performed for each of the above trial conditions, the environmental information detection unit 15 detects the usage environment information illustrated in FIG. 6B, and the characteristic evaluation unit 16 evaluates the sensor characteristics illustrated in FIG. 6A to obtain sensor characteristic evaluation information.

[0079] The information processing device 11 generates a learning dataset DS by associating the usage environment information and sensor characteristic evaluation information obtained for each trial condition as described above with the sensor parameters instructed to be set for the EVS camera 2' for each trial condition. Specifically, the learning dataset DS is generated by associating the usage environment information, sensor characteristic evaluation information, and sensor parameters for each trial condition.

[0080] [1-6. Generation of parameter derivation model] Referring to FIG. 8, the learning for generating the parameter derivation model MD, which is performed using the learning dataset DS, will be described. Here, a case where a derivation model is generated using an AI (Artificial Intelligence) model as the parameter derivation model MD will be exemplified.

[0081] FIG. 8 shows an example of the configuration of a model generation device 1 that generates a parameter derivation model MD. As shown in the figure, the model generation device 1 includes a learning device 5, a model generation control unit 6, and a storage unit .

[0082] In this case, the learning device 5 is provided with a calculation unit having a neural network such as a convolutional neural network (CNN) or a recurrent neural network (RNN) as a derivation calculation unit 5a that performs calculations to derive sensor parameters. As shown in the figure, the learning device 5 is provided with a backpropagation unit 5b for updating the parameters of the derivation calculation unit 5a based on the output of the derivation calculation unit 5a and teacher data.

[0083] The model generation control unit 6 is configured with a microcomputer equipped with a CPU, ROM, RAM, etc., and the CPU executes processing in accordance with, for example, a program stored in the ROM or a program loaded into the RAM, thereby realizing various controls related to the generation of the parameter derivation model MD. As shown in the figure, a storage unit 7 consisting of a memory device is connected to the model generation control unit 6, and the model generation control unit 6 is capable of reading out data stored in the storage unit 7. Specifically, in this example, a training dataset DS is stored in the storage unit 7, and the model generation control unit 6 is capable of reading out the data stored as the training dataset DS.

[0084] The model generation control unit 6 reads out the usage environment information and sensor characteristic information for each trial condition from the learning dataset DS stored in the storage unit 7, and sequentially provides them as input data to the derivation calculation unit 5a. Furthermore, when the model generation control unit 6 provides the usage environment information and sensor characteristic information for a certain trial condition as input data to the derivation calculation unit 5a, it provides the sensor parameters for that trial condition (i.e., the sensor parameters corresponding to the input) to the backpropagation unit 5b as teacher data.

[0085] As a result, machine learning is performed on the learning device 5 using the sensor characteristics and information on the usage environment as learning input data and the sensor parameters that were set when the sensor characteristics and information on the usage environment were evaluated and detected as training data, and an algorithm is formed in the derivation calculation unit 5a of the learning device 5 for deriving sensor parameters for realizing the target sensor characteristics in the usage environment indicated by the usage environment information in response to input of information on the target sensor characteristics and information on the usage environment (information specifying the usage environment). In other words, a parameter derivation model MD is generated.

[0086] It should be noted that, with regard to the learning of the parameter derivation model MD, the learning dataset DS is not limited to being stored inside the model generation device 1, and may also be stored outside the model generation device 1.

[0087] [1-7. Parameter derivation model as an embodiment] Here, as shown in FIG. 9, the parameter derivation model MD in this embodiment is configured to be able to output not only sensor parameters but also compromise information as output data in response to input of target sensor characteristic information and usage environment information.

[0088] FIG. 10 is a diagram showing an example of compromise proposal information. As shown in the figure, the compromise information in this example includes information on the illuminance on the sensor surface, the sensor temperature, the background luminance, and the contrast to the background. The background luminance is the luminance of the background of the subject, and the contrast to the background is the luminance contrast between the subject and the background.

[0089] Here, the compromise information means information indicating a compromise for satisfying the target sensor characteristics, such as, for example, that the target sensor characteristics can be satisfied if the sensor temperature is set to XX degrees.

[0090] The parameter derivation model MD in this embodiment is configured to output such compromise information when it is not possible to derive sensor parameters that can realize the input target sensor characteristics. Specifically, in this example, if the parameter derivation model MD cannot derive sensor parameters that can realize the input target sensor characteristics, it determines whether the target sensor characteristics are satisfied with the derived sensor parameters when each item shown in FIG. 10, such as the illuminance on the sensor surface, is changed, and if it is determined that the target sensor characteristics are satisfied, it outputs the values ​​of each item shown in FIG. 10, such as the illuminance on the sensor surface at that time, as compromise information.

[0091] [1-8. EVS camera as an embodiment] FIG. 11 is a diagram for explaining an example of the configuration of the EVS camera 2 according to this embodiment. In the following description, parts that are the same as parts that have already been described will be given the same reference numerals and description thereof will be omitted.

[0092] The EVS camera 2 according to the embodiment is a device including at least an EVS 2a and a camera control unit 13 that performs control to set the sensor parameters derived by the parameter derivation model MD in the EVS 2a.

[0093] As shown in the figure, the EVS camera 2 includes an EVS 2a, the above-mentioned environmental information detection unit 15, a camera control unit 13, and a communication unit 14.

[0094] The EVS camera 2 of this example sets the sensor parameters derived by the parameter derivation model MD of the external device to the EVS 2a. Specifically, in this example, the parameter derivation model MD is provided in the server device 50.

[0095] The server device 50 is configured to be able to communicate data with the EVS camera 2 via a network NT, which is a predetermined communication network such as the Internet, and has a parameter derivation model MD as shown in the figure, as well as a setting control unit 51. The setting control unit 51 is configured with a microcomputer equipped with a CPU, ROM, RAM, etc., and realizes various functions of the server device 50 by the CPU executing processing in accordance with, for example, a program stored in the ROM or a program loaded into the RAM.

[0096] [1-9. Processing for parameter setting] Fig. 12 is a flowchart showing a process for setting sensor parameters. Specifically, it shows an example of a specific processing procedure to be executed by the camera control unit 13 and the setting control unit 51 shown in Fig. 11 for setting sensor parameters. In FIG. 12, the processing of the camera control unit 13 is executed by the CPU of the camera control unit 13 based on a program stored in a predetermined storage device such as its own ROM, and the processing of the setting control unit 51 is executed by the CPU of the setting control unit 51 based on a program stored in a predetermined storage device such as its own ROM.

[0097] First, in step S101, the camera control unit 13 acquires the use environment information, that is, the use environment information detected by the environment information detection unit 15.

[0098] In step S102 following step S101, the camera control unit 13 performs a process of transmitting the usage environment information acquired in step S101 to the server device (setting control unit 51) via the communication unit 14 as a process of transmitting the usage environment information.

[0099] In step S201, the setting control unit 51 waits for reception of the usage environment information transmitted in step S102. If the use environment information has been received, the setting control unit 51 executes a parameter derivation control process in step S202.

[0100] FIG. 13 is a flowchart showing the parameter derivation control process in step S202. In the parameter derivation control process, in step S211, the setting control unit 51 performs a process of inputting the received usage environment information and information on the target sensor characteristics into the parameter derivation model MD. Here, it is considered that the information on the target sensor characteristics is set in advance in the server device 50 by, for example, a user operation.

[0101] In step S212 following step S211, the setting control unit 51 acquires model output data, i.e., data of the sensor parameters output by the parameter derivation model MD in response to the use environment information and the target sensor characteristic information being given as input data in step S211. It is assumed here that the compromise information described above has not been output.

[0102] In step S213 following step S212, the setting control unit 51 determines whether or not the constraint conditions are satisfied. The constraint condition determination process may be performed, for example, by determining whether the sensor parameter values ​​output from the parameter derivation model MD are within a predetermined settable range. By performing such a determination process, it becomes possible to deal with cases where the settable ranges of the sensor parameters differ depending on the product type of the EVS 2a.

[0103] In step S213, for example, if it is determined that the values ​​of the sensor parameters of each item output from the parameter derivation model MD are within a predetermined settable range and satisfy the constraint conditions, the setting control unit 51 ends the parameter derivation control process of step S202.

[0104] On the other hand, if it is determined in step S213 that at least one of the sensor parameter values ​​for each item output from the parameter derivation model MD is not within the predetermined settable range and does not satisfy the constraint condition, the setting control unit 51 proceeds to step S214 and executes target relaxation processing. Specifically, the target sensor characteristic is relaxed so as to tend to satisfy the constraint condition of step S213. Such target relaxation processing can be performed, for example, using correspondence information (e.g., table information, etc.) that indicates the correspondence of how each item of the sensor parameters changes when the sensor characteristics of each item included in the target sensor characteristics are changed.

[0105] In step S215 following step S214, the setting control unit 51 executes a process of causing the parameter derivation model MD to execute a derivation process using the relaxed target sensor characteristics as input data. That is, the setting control unit 51 causes the parameter derivation model MD to execute a process of deriving parameters using the relaxed target sensor characteristics and the usage environment information received from the EVS camera 2 as input data.

[0106] As shown in the figure, the setting control unit 51 advances the process to step S212 in response to having executed the process of step S214. As a result, the target relaxation process in step S214 is repeated until the constraint condition in step S213 is satisfied. As described above, if it is determined in step S213 that the constraint conditions are satisfied, the parameter derivation control process in step S202 ends.

[0107] In FIG. 12, in response to the execution of the parameter derivation control process in step S202, the setting control unit 51 advances the process to step S203 and performs a process of transmitting the derived sensor parameters to the camera control unit 13. After executing the process of step S203, the setting control unit 51 ends the series of processes shown in FIG.

[0108] In step S103 following the previous step S102, the camera control unit 13 waits for reception of the sensor parameters transmitted in step S203. If the camera control unit 13 receives the sensor parameters in step S103, the process proceeds to step S104, where it performs a setting process for the received information. Specifically, it performs a process for setting the sensor parameters received in step S103 in the EVS 2a.

[0109] After executing the process of step S104, the camera control unit 13 ends the series of processes shown in FIG.

[0110] Although the above description is based on the assumption that compromise proposal information is not output, it is conceivable that a corresponding process (not shown) may be performed when compromise proposal information is output. In this case, for example, a process of presenting question information to ask the user whether or not to accept the compromise proposal indicated in the compromise proposal information may be performed between steps S212 and S213 in Fig. 13, and the process may proceed to step S213 on the condition that a response to the question information indicating that the compromise proposal is accepted is obtained.

[0111] The determination process in step S213 may also involve determining whether or not a constraint condition related to the usage environment is satisfied.

[0112] 12 and 13, the sensor parameters suitable for the usage environment are set in the EVS 2a in the EVS camera 2.

[0113] [1-10. Application Examples] 14 to 21, examples of application of the model generation method and the derived parameter setting method as embodiments to each use case will be described. FIG. 14 is an explanatory diagram of a first use case in which the target subject is an object (e.g., a cardboard box or a barcode attached to the cardboard box) moved indoors by a conveyor belt, and FIG. 15 shows examples of the sensor characteristic information, usage environment information, sensor parameters, and compromise information used in the first use case.

[0114] As shown in FIG. 15, in the first use case, it is conceivable that the sensor characteristic information does not include information on the number of over-responsive pixels, the number of unresponsive pixels, and the dead time, but includes only information on the amount of noise, the output event rate, the output data rate, the latency, and the contrast threshold. Furthermore, because there is no particular periodicity in the movement of the target subject, it is conceivable that the information on the usage environment, such as whether or not there is periodicity and the interval between successive firings of the same pixel, as shown in the figure, will not be used, and only the information on the illuminance on the sensor surface, the sensor temperature, the duration of contrast change, the number of simultaneously firing rows, and the number of simultaneously firing pixels within a row will be used. Furthermore, with regard to the sensor parameters, it is conceivable to exclude information on the digital mask settings and use only the bias voltage bias_fo, bias voltage bias_hpf, voltage bias_diff_on, voltage bias_diff_off, and voltage bias_refr, the event filter setting, the ROI setting, and the readout latency. It is possible to use all of the illuminance on the sensor surface, the sensor temperature, the background luminance, and the contrast to the background as compromise information. The reason why the illuminance on the sensor surface is used as compromise information in the first use case is that it is an indoor case and therefore ambient light can be adjusted.

[0115] In this first use case, only the noise amount, output event rate, output data rate, latency, and contrast threshold are used as information on sensor characteristics to be input data to the learning device 5, and information on target sensor characteristics to be input data to the parameter derivation model MD. In addition, in the first use case, only the illuminance on the sensor surface, the sensor temperature, the duration of contrast change, the number of simultaneously firing rows, and the number of simultaneously firing pixels within a row are used as the usage environment information to be input to the learning device 5 and the parameter derivation model MD. In addition, in the first use case, the sensor parameters used as training data for the learning device 5 are only the bias voltage bias_fo, bias voltage bias_hpf, voltage bias_diff_on, voltage bias_diff_off, voltage bias_refr, event filter setting, ROI setting, and read latency, excluding information on the digital mask setting.

[0116] FIG. 16 is an explanatory diagram of a second use case, which is an outdoor surveillance camera application (e.g., people monitoring), and FIG. 17 illustrates examples of sensor characteristic information, usage environment information, sensor parameters, and compromise information used in the second use case.

[0117] As shown in FIG. 17, in the second use case, as in the first use case, it is conceivable that the sensor characteristic information does not use information on the number of over-responsive pixels, the number of unresponsive pixels, and the dead time, but only information on the amount of noise, the output event rate, the output data rate, the latency, and the contrast threshold. Furthermore, as for the usage environment information, since there is no particular periodicity in the movement of the target subject in this case either, information on whether or not there is periodicity is not used, as shown in the figure, and since there is no regularity in the movement of the subject compared to the first use case, it is considered that the duration of contrast change is not used. Furthermore, as with the first use case, it is conceivable to exclude information on digital mask settings from the sensor parameters. In this case, unlike the first use case, it is not possible to adjust the ambient light in the outdoor use case, so the compromise information, including the illuminance on the sensor surface, the sensor temperature, the background luminance, and the contrast to the background, is not used, as shown in the figure.

[0118] In the second use case, only the noise amount, output event rate, output data rate, latency, and contrast threshold are used as information on sensor characteristics to be used as input data to the learning device 5, and as information on target sensor characteristics to be used as input data to the parameter derivation model MD. In addition, in the second use case, the only information used as input data to the learning device 5 and the input data to the parameter derivation model MD is the illuminance on the sensor surface, the sensor temperature, the number of simultaneously firing rows, the number of simultaneously firing pixels within a row, and the interval between consecutive firings of the same pixel. Also, in the second use case, the sensor parameters used as training data for the learning device 5 are only the bias voltage bias_fo, bias voltage bias_hpf, voltage bias_diff_on, voltage bias_diff_off, voltage bias_refr, event filter setting, ROI setting, and readout latency, excluding information on the digital mask setting.

[0119] FIG. 18 is an explanatory diagram of a third use case, which is an outdoor monitoring application of a rotating object (for example, monitoring the wind turbine part of a wind power generator), and FIG. 19 shows examples of the sensor characteristic information, usage environment information, sensor parameters, and compromise information used in the third use case.

[0120] As shown in FIG. 19, in the third use case, it is conceivable to use only information on the amount of noise, output event rate, output data rate, latency, contrast threshold, and dead time as sensor characteristic information, without using information on the number of over-responsive pixels and the number of unresponsive pixels. Furthermore, in this case, since the movement of the target subject is periodic, it is conceivable to use information on all items, including information on whether or not there is periodicity, as shown in the figure, as the usage environment information. Furthermore, as with the first use case, it is conceivable to exclude information on digital mask settings from the sensor parameters. In this case, too, the compromise information is such that, since it is not possible to adjust the ambient light in an outdoor use case, the illuminance on the sensor surface, the sensor temperature, the background luminance, and the contrast to the background are all unused, as shown in the figure.

[0121] In the third use case, only the information on the noise amount, output event rate, output data rate, latency, contrast threshold, and dead time, excluding the items of the number of over-responsive pixels and the number of unresponsive pixels, is used as the information on the sensor characteristics to be input data to the learning device 5, and as the information on the target sensor characteristics to be input data to the parameter derivation model MD. In addition, in the third use case, the usage environment information used as input data to the learning device 5 and the usage environment information used as input data to the parameter derivation model MD includes all of the following information: illuminance on the sensor surface, sensor temperature, presence / absence of periodicity, number of simultaneously firing rows, number of simultaneously firing pixels within a row, and interval between consecutive firings of the same pixel. Also, in the third use case, the sensor parameters used as training data for the learning device 5 are only the bias voltage bias_fo, bias voltage bias_hpf, voltage bias_diff_on, voltage bias_diff_off, voltage bias_refr, event filter setting, ROI setting, and readout latency, excluding information on the digital mask setting.

[0122] FIG. 20 is an explanatory diagram of a fourth use case, which is used to monitor vibrating objects indoors, and FIG. 21 shows examples of the sensor characteristic information, usage environment information, sensor parameters, and compromise information used in the fourth use case.

[0123] As shown in FIG. 21, in the fourth use case, as in the third use case, information on the number of over-responsive pixels and the number of unresponsive pixels is not used as sensor characteristic information, and only information on the amount of noise, output event rate, output data rate, latency, contrast threshold, and dead time is considered to be used. Furthermore, in this case too, since the movement of the target subject is periodic, it is conceivable to use information on all items, including information on whether or not there is periodicity, as in the third use case, as usage environment information. As for the sensor parameters, in this case too, it is conceivable to exclude information on the digital mask settings, as in the first use case. As for the compromise information, since this is an indoor use case, it is conceivable to use all of the illuminance on the sensor surface, the sensor temperature, the background luminance, and the contrast to the background, as in the first use case.

[0124] In the fourth use case, only the information on the noise amount, output event rate, output data rate, latency, contrast threshold, and dead time, excluding the items of the number of over-responsive pixels and the number of unresponsive pixels, is used as the information on the sensor characteristics to be input data to the learning device 5, and as the information on the target sensor characteristics to be input data to the parameter derivation model MD. In addition, in the fourth use case, the usage environment information used as input data to the learning device 5 and the usage environment information used as input data to the parameter derivation model MD includes all of the following information: illuminance on the sensor surface, sensor temperature, presence / absence of periodicity, number of simultaneously firing rows, number of simultaneously firing pixels within a row, and interval between consecutive firings of the same pixel. Also, in the fourth use case, the sensor parameters used as training data for the learning device 5 are only the bias voltage bias_fo, bias voltage bias_hpf, voltage bias_diff_on, voltage bias_diff_off, voltage bias_refr, event filter setting, ROI setting, and readout latency, excluding information on the digital mask setting.

[0125] <2. Modifications> The specific examples described above are merely examples, and the present technology can adopt configurations as various modifications. For example, with respect to the parameter derivation model MD, it is possible to re-learn the model using the usage environment information, sensor characteristic evaluation information, and sensor parameters set in the EVS camera obtained by the EVS camera in which the sensor parameters derived by the parameter derivation model MD are set as re-learning data.

[0126] The re-learning of such a parameter derivation model MD will be described with reference to FIGS. FIG. 22 shows an example of the configuration of an EVS camera 2A to be used when relearning the parameter derivation model MD, and FIG. 23 shows an example of the configuration of a server device 50A to be used in the same case.

[0127] 22, the EVS camera 2A differs from the EVS camera 2 (see FIG. 11) in that a characteristic evaluation unit 16 is added and that a camera control unit 13A is provided instead of the camera control unit 13.

[0128] 23, server device 50A differs from server device 50 (see FIG. 11) in that a model re-learning processing unit 52 is added.

[0129] In the EVS camera 2A, the camera control unit 13A performs the same processing related to setting the sensor parameters of the EVS 2a as the camera control unit 13. Specifically, the camera control unit 13A transmits the usage environment information detected by the environment information detection unit 15 to an external device (in this case, the server device 50A) having the parameter derivation model MD, and performs processing to set the sensor parameters derived by the external device in accordance with the transmitted usage environment information to the EVS 2a.

[0130] Furthermore, during operation of the EVS 2 after the processing for setting the sensor parameters as described above, the camera control unit 13A performs processing to cause the environment information detection unit 15 to detect environmental information and the characteristic evaluation unit 16 to evaluate sensor characteristics at multiple predetermined timings, such as at regular intervals, and to acquire the detected usage environment information and sensor characteristic evaluation information obtained by the evaluation. Furthermore, the camera control unit 13A also performs processing to acquire information on the sensor parameters that were set in the EVS 2a when the usage environment information and sensor characteristic information were obtained. Then, the camera control unit 13A transmits the use environment information, the sensor characteristic evaluation information, and the sensor parameter information thus acquired multiple times to the server device 50A as re-learning data.

[0131] In the server device 50A, the model re-learning processing unit 52 performs a re-learning process on the parameter derivation model MD using the re-learning data transmitted by the camera control unit 13A as described above. Specifically, this is a re-learning process in which the usage environment information and sensor characteristic information acquired as the re-learning data are used as learning input data, and the sensor parameter information is used as training data.

[0132] By performing such a re-learning process, the performance of the parameter derivation model MD can be improved.

[0133] The entity that performs the re-learning process is not necessarily limited to a device that has the parameter derivation model MD and the setting control unit 51, and a configuration in which a device that does not have these parameter derivation model MD and the setting control unit 51 is the entity that performs the re-learning process is also possible.

[0134] 11, in a configuration in which the parameter derivation model MD is provided in a device external to the EVS camera 2, it is also possible to consider detecting some of the usage environment information (for example, information on items that can be detected based on event data, such as the duration of contrast change and the number of simultaneously fired rows) in the external device (server device 50) rather than in the EVS camera 2. In this case, the EVS camera 2 transmits the event data to the server device 50 as RAW data, and the server device 50 detects information on some of the items of the usage environment information based on the RAW data. If there is a device that acts as a repeater between the EVS camera 2 and the server device 50, it is possible to adopt a configuration in which the repeater device detects part of the usage environment information.

[0135] Here, in the above, an example was given of a case where, in order to set the sensor parameters derived by the parameter derivation model MD in the EVS 2a, the EVS camera 2 needs to transmit usage environment information to the server device 50 and receive the sensor parameters from the server device 50. However, it is also possible to adopt the following method, which does not require the transmission and reception of usage environment information and sensor parameters between such external devices.

[0136] FIG. 24 is a diagram showing an example of the configuration of an EVS camera 2B as a modified example in which a method is adopted that does not require transmission and reception of usage environment information and sensor parameters with an external device. As shown in the figure, the EVS camera 2B differs from the EVS camera 2 in that a storage unit 17 is added and that a camera control unit 13B is provided instead of the camera control unit 13.

[0137] As shown in the figure, a candidate parameter set CP is stored in the storage unit 17. The candidate parameter set CP means a plurality of sensor parameters derived by the parameter derivation model MD in response to input of different usage environment information. In obtaining the candidate parameter set CP, the target sensor characteristics given as input data to the parameter derivation model MD may remain unchanged.

[0138] The camera control unit 13B acquires the usage environment information detected by the environment information detection unit 15, selects a set of sensor parameters from the sensor parameters stored as the candidate parameter set CP in the storage unit 17 based on the usage environment information, and sets the selected sensor parameters to the EVS 2a. At this time, the selection process for the set of sensor parameters may be performed, for example, as a process of selecting sensor parameters associated with usage environment information indicating a usage environment that is closest to the usage environment indicated by the usage environment information detected by the environment information detection unit 15. At this time, the determination of whether the usage environments are closest may be performed by calculating a difference value for each item between the usage environment information to be compared, and determining whether a representative value such as the sum or average value of the difference values ​​for each item is smallest.

[0139] By adopting the configuration of the EVS camera 2B as a modified example as described above, it is possible to set appropriate sensor parameters according to the usage environment entirely within the EVS camera without communicating with an external device.

[0140] Furthermore, in the above, an example has been described in which a training dataset is prepared for each use case, such as the first use case to the fourth use case, in other words, an example in which a parameter derivation model MD is generated individually for each of these multiple use cases. However, it is also possible to prepare a training dataset common to multiple use cases and generate a parameter derivation model MD that can be used in common for these multiple use cases.

[0141] In addition, the example above shows how to generate an AI model as a parameter derivation model MD. The parameter derivation model MD can also be generated as a non-AI computational model that derives sensor parameters through rule-based processing.

[0142] <3. Summary of the embodiment> As described above, the model generation method as an embodiment is a model generation method in which an information processing device (model generation device 1) executes a process of generating a parameter derivation model (MD) that derives sensor parameters for realizing target sensor characteristics under a usage environment indicated by the specification information in response to input of information on target sensor characteristics, which are the target sensor characteristics, and specification information on the usage environment, based on sensor characteristic evaluation information for each trial condition, which is obtained by evaluating the sensor characteristics of the EVS when an event detection operation is performed by the EVS for each of a plurality of trial conditions that differ in combination of the usage environment of the EVS camera having the EVS and the sensor parameters of the EVS, and usage environment information and parameter information that indicate the usage environment and sensor parameters for each trial condition. By using the parameter derivation model described above, it becomes possible to eliminate the need to manually derive sensor parameters to achieve target sensor characteristics in the assumed EVS camera usage environment. Even if some of the work is done manually, it is possible to at least provide support for adjustment. Therefore, it is possible to reduce the cost required to derive appropriate sensor parameters for the EVS camera according to the usage environment.

[0143] In addition, in the model generation method of the embodiment, the parameter derivation model is generated by machine learning using the usage environment information and sensor characteristic evaluation information for each trial condition as learning input data and the parameter information for each trial condition as training data. This makes it possible to realize an AI-based parameter derivation model. By using an AI parameter derivation model, it is possible to improve the performance of deriving appropriate sensor parameters.

[0144] Furthermore, in the model generation method according to the embodiment, the usage environment information includes information related to the subject of the EVS camera. This allows appropriate sensor parameters to be derived taking into consideration the type and characteristics of the subject.

[0145] Furthermore, in the model generation method as an embodiment, the sensor parameters include parameters relating to at least one of the event detection threshold, the LPF characteristics of the logarithmic conversion unit, and the HPF characteristics of the buffer unit. These sensor parameters affect the main sensor characteristics of the EVS. Therefore, by deriving these sensor parameters, the target sensor characteristics can be efficiently achieved.

[0146] In the model generation method according to the embodiment, the sensor characteristics include characteristics related to at least one of a contrast threshold, an event output latency, a pixel dead period, and background noise. These characteristics are the main sensor characteristics of the EVS, and by deriving sensor parameters so that target characteristics for such sensor characteristics are realized, appropriate sensor parameters can be derived efficiently.

[0147] Furthermore, in the model generation method of the embodiment, an EVS camera is used to evaluate the sensor characteristics for each trial condition to obtain sensor characteristic evaluation information for each trial condition, and a learning dataset for generating a parameter derivation model is generated based on the usage environment information, parameter information, and sensor characteristic evaluation information for each trial condition (see Figures 22 and 23). This makes it possible to generate an appropriate learning dataset for generating a parameter derivation model.

[0148] Furthermore, in the model generation method of the embodiment, the parameter derivation model is re-learned using, as re-learning data, model characteristic evaluation information obtained by evaluating the sensor characteristics of an EVS camera in which sensor parameters derived by the parameter derivation model are set, information on the usage environment of the EVS camera, and information on the sensor parameters that were set in the EVS camera when the model characteristic evaluation information was evaluated. By performing the above-described re-learning, it is possible to improve the performance of the parameter derivation model.

[0149] The model generation device (1) as an embodiment is equipped with a model generation control unit (6) that performs control to generate a parameter derivation model that derives sensor parameters for realizing target sensor characteristics under a usage environment indicated by the specified information in response to input of information on target sensor characteristics, which are the target sensor characteristics, and specified information on the usage environment, based on sensor characteristic evaluation information for each trial condition obtained by evaluating the sensor characteristics of the EVS when an event detection operation by the EVS is performed for each of a plurality of trial conditions that vary the combination of the usage environment of the EVS camera having the EVS and the sensor parameters of the EVS, and usage environment information and parameter information that indicate the usage environment and sensor parameters for each trial condition, respectively. Such a model generating device can also provide the same functions and effects as the model generating method of the above embodiment.

[0150] An EVS camera (2, 2A, 2B) according to an embodiment includes an EVS (2a), and a camera control unit (13, 13A, 13B) that performs control to set, in the EVS, sensor parameters derived using a parameter derivation model that is generated based on sensor characteristic evaluation information for each trial condition obtained by evaluating the sensor characteristics of the EVS when an event detection operation is performed by the EVS for each of a plurality of trial conditions that vary the combination of the usage environment of the EVS camera having the EVS and the sensor parameters of the EVS, and usage environment information and parameter information that respectively indicate the usage environment and sensor parameters for each trial condition, in response to input of information on target sensor characteristics, which are the target sensor characteristics, and specification information on the usage environment, to derive sensor parameters for realizing the target sensor characteristics under the usage environment indicated by the specification information. By using the parameter derivation model described above to derive appropriate sensor parameters according to the usage environment, it is possible to reduce the cost required to derive appropriate sensor parameters. Furthermore, according to the above EVS camera, it is possible to realize an EVS camera that operates by setting sensor parameters derived at low cost.

[0151] In addition, the EVS camera according to the embodiment is provided with an environmental information detection unit (15) that detects usage environment information indicating the usage environment, and the camera control unit controls the setting of sensor parameters derived by the parameter derivation model in the EVS based on the usage environment information detected by the environmental information detection unit. This allows appropriate sensor parameters to be derived according to the actual usage environment of the EVS camera, and these sensor parameters can be set in the EVS.

[0152] Furthermore, in an EVS camera (ibid. 2) as an embodiment, the camera control unit (ibid. 13) transmits the usage environment information detected by the environment information detection unit to an external device having a parameter derivation model, and controls the external device to set sensor parameters derived using the parameter derivation model based on the transmitted usage environment information in the EVS. Since the external device derives appropriate sensor parameters according to the usage environment, the processing load on the EVS camera can be reduced.

[0153] Furthermore, in an EVS camera (2B) according to an embodiment, a memory unit (17) is provided in which multiple sensor parameters derived by a parameter derivation model in response to input of different usage environment information are stored as candidate parameter sets, and a camera control unit (13B) controls the setting of sensor parameters selected from the candidate parameter set in the EVS based on the usage environment information detected by the environment information detection unit. This allows appropriate sensor parameters to be set according to the usage environment within the EVS camera without communication with an external device.

[0154] The camera system according to an embodiment includes a model generation device (1) having a model generation control unit (6) that controls the generation of a parameter derivation model that derives sensor parameters for realizing target sensor characteristics under a usage environment indicated by input of target sensor characteristics, which are the target sensor characteristics, and specification information for the usage environment, based on sensor characteristic evaluation information for each trial condition obtained by evaluating the sensor characteristics of the EVS when an event detection operation is performed by the EVS under multiple trial conditions that vary the combination of the usage environment of the EVS camera having the EVS and the sensor parameters of the EVS, and usage environment information and parameter information that indicate the usage environment and sensor parameters for each trial condition, and an EVS camera (2, 2A, 2B) that has an EVS and camera control units (13, 13A, 13B) that control the setting of the sensor parameters derived by the parameter derivation model in the EVS. Such a camera system can provide the functions and effects of both the model generation device and the EVS camera of the above-described embodiment.

[0155] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0156] <4. This technology> The present technology can also be configured as follows. (1) An information processing device executes a process of generating a parameter derivation model that derives the sensor parameters for realizing the target sensor characteristics under the usage environment indicated by the specification information for the usage environment, based on sensor characteristic evaluation information for each trial condition obtained by evaluating the sensor characteristics of the EVS when an event detection operation by the EVS is performed for each trial condition, which is obtained by varying the combination of the usage environment of an EVS camera having an EVS and the sensor parameters of the EVS, and usage environment information and parameter information indicating the usage environment and the sensor parameters for each trial condition, in response to input of information on target sensor characteristics that are the sensor characteristics to be targeted and specification information for the usage environment. Model generation method. (2) generating the parameter derivation model, The use environment information and the sensor characteristic evaluation information for each trial condition are used as learning input data, and the parameter information for each trial condition is used as training data, and machine learning is performed. The model generation method according to (1) above. (3) The usage environment information includes information related to a subject of the EVS camera. The model generation method according to (1) or (2) above. (4) The sensor parameters include at least one parameter related to an event detection threshold, an LPF characteristic of a logarithmic converter, and an HPF characteristic of a buffer. The model generation method according to any one of (1) to (3). (5) The sensor characteristics include characteristics related to at least one of a contrast threshold, an event output latency, a pixel dead period, and background noise. The model generation method according to any one of (1) to (4). (6) Using an EVS camera, the sensor characteristics for each of the trial conditions are evaluated to obtain sensor characteristic evaluation information for each of the trial conditions, and a learning dataset for generating the parameter derivation model is generated based on the usage environment information, the parameter information, and the sensor characteristic evaluation information for each of the trial conditions. The model generation method according to any one of (1) to (5) above. (7) The parameter derivation model is re-learned using, as re-learning data, model characteristic evaluation information obtained by evaluating the sensor characteristics of an EVS camera in which the sensor parameters derived by the parameter derivation model are set, usage environment information of the EVS camera, and information on the sensor parameters set in the EVS camera when the model characteristic evaluation information was evaluated. The model generation method according to any one of (1) to (6). (8) The system includes a model generation control unit that performs control to generate a parameter derivation model that derives the sensor parameters for realizing the target sensor characteristics under the usage environment indicated by the specification information for the usage environment, based on sensor characteristic evaluation information for each trial condition obtained by evaluating the sensor characteristics of the EVS when an event detection operation is performed by the EVS for each of a plurality of trial conditions that vary the combination of the usage environment of an EVS camera having an EVS and the sensor parameters of the EVS, and usage environment information and parameter information that respectively indicate the usage environment and the sensor parameters for each trial condition, in response to input of information on target sensor characteristics that are the sensor characteristics to be targeted and specification information for the usage environment. Model generation device. (9) EVS and and a camera control unit that performs control to set, in the EVS, the sensor parameters derived by a parameter derivation model that is generated based on sensor characteristic evaluation information for each trial condition obtained by evaluating sensor characteristics of the EVS when an event detection operation is performed by the EVS for each of a plurality of trial conditions that vary combinations of a usage environment of an EVS camera having an EVS and sensor parameters of the EVS, and usage environment information and parameter information that respectively indicate the usage environment and the sensor parameters for each trial condition, in response to input of information on target sensor characteristics that are the sensor characteristics to be targeted and designation information for the usage environment, the sensor parameters being derived by a parameter derivation model that is generated to derive the sensor parameters for realizing the target sensor characteristics under the usage environment indicated by the designation information. EVS camera. (10) an environment information detection unit that detects use environment information indicating the use environment; The camera control unit The sensor parameters derived by the parameter derivation model based on the usage environment information detected by the environment information detection unit are set in the EVS. The EVS camera described in (9) above. (11) The camera control unit The use environment information detected by the environment information detection unit is transmitted to an external device having the parameter derivation model, and the external device performs control to set the sensor parameters derived by using the parameter derivation model based on the transmitted use environment information to the EVS. The EVS camera according to (10) above. (12) a storage unit in which a plurality of the sensor parameters derived by the parameter derivation model in response to input of different usage environment information are stored as candidate parameter sets; The camera control unit The sensor parameter selected from the candidate parameter set is set to the EVS based on the usage environment information detected by the environment information detection unit. The EVS camera according to (10) above. (13) a model generation device having a model generation control unit that performs control to generate a parameter derivation model that derives the sensor parameters for realizing the target sensor characteristics under the usage environment indicated by the specification information for the usage environment, based on sensor characteristic evaluation information for each trial condition obtained by evaluating the sensor characteristics of the EVS when an event detection operation is performed by the EVS for each of a plurality of trial conditions that vary the combination of the usage environment of an EVS camera having an EVS and the sensor parameters of the EVS, and usage environment information and parameter information that respectively indicate the usage environment and the sensor parameters for each trial condition; an EVS camera having an EVS and a camera control unit that controls setting the sensor parameters derived by the parameter derivation model in the EVS; Camera system. [Explanation of symbols]

[0157] 1. Model generation device 2', 2, 2A, 2B EVS camera 5 Learning Unit 5a Derivation calculation section 5b Backpropagation section 6 Model generation control section 7 Memory section 10 Training data generation system 11 Information processing equipment 12 Subject reproduction device 2a EVS 20 pixels 21 Pixel array section 22 X Arbiter 23 Y Arbiter 24 Event Processing Circuit 25 Output I / F 31 Logarithmic conversion section 32 buffers 33 Event detection circuit 34 Subtractor 35 Quantizer 36 Output control / reset circuit SWr Reset switch 38 Output circuit 39 OR Circuit 40 Delay 13', 13, 13A, 13B Camera control unit 14 Communications Department 15 Environmental information detection unit 16 Characterization Section MD parameter derivation model DS training dataset 50,50A Server Equipment 51 Setting control section 52 Model re-learning processing unit 17 Memory section CP candidate parameter set

Claims

1. The information processing device executes a process of generating a parameter derivation model for deriving the sensor parameters for realizing the target sensor parameters under a usage environment indicated by input of information on a target sensor parameter, which is the sensor parameter to be targeted, and designation information on the usage environment, based on sensor parameter evaluation information for each trial condition obtained by evaluating the sensor parameters of the EVS when an event detection operation is performed by the EVS for each trial condition in which a combination of a usage environment of an EVS camera having an EVS and a sensor parameter of the EVS is different, and on the basis of the usage environment information and parameter information indicating the usage environment and the sensor parameters for each trial condition. Model generation method.

2. Generating the parameter derivation model The use environment information and the sensor characteristic evaluation information for each trial condition are used as learning input data, and the parameter information for each trial condition is used as training data by machine learning. The method of claim 1 .

3. The usage environment information includes information related to a subject of the EVS camera. The method of claim 1 .

4. The sensor parameters include at least one parameter related to an event detection threshold, an LPF characteristic of a logarithmic conversion unit, and an HPF characteristic of a buffer unit. The method of claim 1 .

5. The sensor characteristics include characteristics related to at least one of a contrast threshold, an event output latency, a pixel dead period, and background noise. The method of claim 1 .

6. Using an EVS camera, the sensor characteristics are evaluated for each of the trial conditions to obtain sensor characteristic evaluation information for each of the trial conditions, and a learning data set for generating the parameter derivation model is generated based on the usage environment information, the parameter information, and the sensor characteristic evaluation information for each of the trial conditions. The method of claim 1 .

7. The parameter derivation model is re-learned using model characteristic evaluation information obtained by evaluating the sensor characteristics of an EVS camera in which the sensor parameters derived by the parameter derivation model are set, usage environment information of the EVS camera, and information on the sensor parameters set in the EVS camera when the model characteristic evaluation information was evaluated, as re-learning data. The method of claim 1 .

8. The device includes a model generation control unit that performs control to generate a parameter derivation model that derives the sensor parameters for realizing the target sensor characteristics under a usage environment indicated by the specification information for the usage environment, based on sensor characteristic evaluation information for each trial condition obtained by evaluating the sensor characteristics of the EVS when an event detection operation is performed by the EVS for each of a plurality of trial conditions in which a combination of a usage environment of an EVS camera having an EVS and a sensor parameter of the EVS is different, and on the basis of usage environment information and parameter information indicating the usage environment and the sensor parameters for each trial condition, respectively, in response to input of information on a target sensor characteristic that is the sensor characteristic to be targeted and specification information of the usage environment. Model generation device.

9. EVS and a camera control unit that performs control to set, in the EVS, the sensor parameters derived by a parameter derivation model that is generated so as to derive the sensor parameters for realizing the target sensor characteristics under the usage environment indicated by the specification information for the usage environment in response to input of information on a target sensor characteristic that is the sensor characteristic to be targeted and specification information for the usage environment, based on sensor characteristic evaluation information for each trial condition obtained by evaluating the sensor characteristics of the EVS when an event detection operation is performed by the EVS for each trial condition that differs in combination of a usage environment of an EVS camera having an EVS and a sensor parameter of the EVS, and usage environment information and parameter information indicating the usage environment and the sensor parameters for each trial condition, respectively. EVS camera.

10. an environment information detection unit that detects use environment information indicating the use environment, The camera control unit is The sensor parameters derived by the parameter derivation model based on the usage environment information detected by the environment information detection unit are set in the EVS. The EVS camera of claim 9.

11. The camera control unit is The use environment information detected by the environment information detection unit is transmitted to an external device having the parameter derivation model, and the external device controls to set the sensor parameters derived by using the parameter derivation model based on the transmitted use environment information to the EVS. The EVS camera of claim 10.

12. a storage unit in which a plurality of the sensor parameters derived by the parameter derivation model in response to input of different usage environment information are stored as candidate parameter sets; The camera control unit is The sensor parameters selected from the candidate parameter set are set to the EVS based on the usage environment information detected by the environment information detection unit. The EVS camera of claim 10.

13. a model generation device having a model generation control unit that performs control to generate a parameter derivation model that derives the sensor parameters for realizing a target sensor characteristic under a usage environment indicated by input of information on a target sensor characteristic, which is the sensor characteristic to be targeted, and designation information on the usage environment, based on sensor characteristic evaluation information for each trial condition obtained by evaluating the sensor characteristics of the EVS when an event detection operation is performed by the EVS for each of a plurality of trial conditions that differ in combination of the usage environment of an EVS camera having an EVS and the sensor parameters of the EVS, and on the usage environment information and parameter information that respectively indicate the usage environment and the sensor parameters for each trial condition; and an EVS camera having an EVS and a camera control unit that controls setting the sensor parameters derived by the parameter derivation model in the EVS. Camera system.

Images