Simulation device, simulation system and control method

Abstract

This technology relates to a simulation device, simulation system, and control method that enables the processing of different types of image data. An image input interface is provided, configured to generate instances in which the address of an image memory and the image type are managed, and the image type stored in the instance is determined; and a processing unit is configured to perform processing corresponding to the image type. This technology can be applied, for example, to simulation devices that perform simulations using image data from a rendering device.

Description

Technical Field

[0001] This technology relates to analog devices, analog systems, and control methods, and for example to analog devices, analog systems, and control methods capable of standardizing and processing a variety of different types of image data. Background Technology

[0002] In the development of autonomous driving systems, image recognition algorithms are validated through simulations in virtual space. This validation process involves generating input images that closely resemble real-world images using computer graphics (CG) compositing techniques that simulate sensor models of image sensors.

[0003] PTL 1 proposes that the recognition performance of image recognition algorithms can be correctly evaluated by reproducing noise, etc., when using HDR image sensors.

[0004] Reference List

[0005] Patent documents

[0006] PTL 1: JP 2022-99651 A Summary of the Invention

[0007] Technical issues

[0008] In the simulation system proposed in PTL 1, there is a one-to-one correspondence between the rendering system and the sensor model. Because the sensor model is configured with specifications suitable for the rendering system, it struggles to handle signals from rendering systems with different specifications. Therefore, when the specifications of the rendering system change, it is necessary to redesign the sensor model to match the rendering system.

[0009] This technology was developed in light of this situation, and provides a highly versatile simulation device.

[0010] Solutions to the problem

[0011] An analog device according to one aspect of the present technology includes: an image input interface configured to generate instances, manage the address of an image memory and the image type in the instances, and determine the image type stored in the instances; and a processing unit configured to perform processing corresponding to the type.

[0012] According to one aspect of the present technology, the control method includes the following: generating an instance from an image input interface that controls the input and output of image data; managing the address of the image memory and the image type in the instance; and supplying the image data to a processing unit that performs processing corresponding to the type.

[0013] A simulation system according to one aspect of the present technology includes: a rendering device; and a simulation device configured to perform a simulation using image data from the rendering device, wherein the simulation device includes: an image input interface configured to generate an instance in response to an instruction from the rendering device, manage the address of an image memory in the instance, and store image data and an image type from the rendering device in the image memory; and a processing unit configured to perform processing corresponding to the type.

[0014] In a simulation device according to one aspect of the present technology, an instance is generated, in which the address of the image memory and the image type are managed, and processing corresponding to the type is performed based on the image data and type stored in the image memory.

[0015] In a control method according to one aspect of the present technology, an image input interface is configured to control the input and output of image data, an image input interface generation instance is configured to manage the address of the image memory and the image type in the instance, the image type stored in the instance is determined, and the image data is supplied to a processing unit configured to perform processing corresponding to the type.

[0016] A simulation system according to one aspect of the present technology includes a rendering device and a simulation device configured to perform simulation using image data from the rendering device, wherein the simulation device generates an instance in response to an instruction from the rendering device, manages the address of an image memory in the instance, stores image data and image type from the rendering device in the image memory, and performs processing corresponding to the image data and image type stored in the image memory.

[0017] Note that the analog device can be a standalone device or an internal module that constitutes a device. Attached Figure Description

[0018] Figure 1 This is a diagram illustrating the configuration of an embodiment of a simulation system that applies the present technology.

[0019] Figure 2 It is a diagram used to explain the operation of the simulation.

[0020] Figure 3 It is a diagram used to explain the relationship between instances and image storage.

[0021] Figure 4 This is a diagram used to explain sensor model processing.

[0022] Figure 5 This is a diagram used to explain the sensor model simulation processing.

[0023] Figure 6 It is a diagram used to explain the storage of data in image memory.

[0024] Figure 7 It is a diagram used to explain the reproduction of distortion.

[0025] Figure 8 This is a diagram showing another configuration of the simulation system.

[0026] Figure 9 It is a diagram used to explain the reproduction of distortion.

[0027] Figure 10 This is a diagram showing an example of PC configuration. Detailed Implementation

[0028] The following describes the methods used to implement this technology (hereinafter referred to as embodiments).

[0029] Configuration example of a simulation system

[0030] Figure 1 This is a diagram illustrating a configuration example of an embodiment of a simulation system to which this technology is applied. Figure 1 The simulation system 10 shown can be applied to situations such as simulating the operation of a vehicle's autonomous driving system or a camera system in an advanced driver assistance system (ADAS). The simulation system 10 is configured with hardware, software, or a combination thereof.

[0031] Figure 1 The simulation system 10 shown includes rendering systems (rendering devices) 21-1, 21-2, 21-3 and a sensor model 31 (simulation device). Rendering systems 21-1 to 21-3 generate image data for each pixel of the pixel model used in the sensor model 31 and supply the image data to the sensor model 31.

[0032] Rendering system 21-1 generates image data of spectral irradiance, rendering system 21-2 generates image data of red, green, and blue (RGB) irradiance, and rendering system 21-3 generates image data of photon counts. Any one of rendering systems 21-1 to 21-3 is connected to sensor model 31 and configured to supply data. Figure 1 In the diagram, rendering system 21-1 is connected to sensor model 31, as indicated by a solid arrow. Rendering systems 21-2 and 21-3 are not connected to sensor model 31, as indicated by dashed arrows.

[0033] Note that the state in which a physical connection is established but no data is output is also included in the state in which no connection is established. Rendering systems 21-1 to 21-3 are connected to sensor model 31 in a state in which data can be output, but this embodiment also includes the case in which only one of the rendering systems 21-1 to 21-3 outputs data.

[0034] In this embodiment, the sensor model 31 is configured to process data from multiple rendering systems 21 that output different types of data.

[0035] The sensor model 31 includes an input image normalization unit 41, a spectral irradiance processing unit 42, an RGB irradiance processing unit 43, a photon counting processing unit 44, and a pixel model processing unit 45. The spectral irradiance processing unit 42 includes a distortion reproduction unit 51 and a pixel model generation unit 52, the RGB irradiance processing unit 43 includes a distortion reproduction unit 53 and a pixel model generation unit 54, and the photon counting processing unit 44 includes a distortion reproduction unit 55 and a pixel model generation unit 56.

[0036] The input image normalization unit 41 determines whether the image data supplied from the rendering system 21 is spectral irradiance data, RGB irradiance data, or photon count data, and causes the processing unit in the subsequent stage to perform processing based on the determination result.

[0037] The input image normalization unit 41 performs control related to the input / output of image data from the rendering system 21, determines the type of input image data, and appropriately performs control to enable processing corresponding to the determination results to be performed in subsequent stages. The input image normalization unit 41 serves as the image input interface in the sensor model 31. Note that in the example described herein, the input image normalization unit 41 is located within the sensor model 31; however, the input image normalization unit 41 may be located separately from the sensor model 31 or may be located between the rendering system 21 and the sensor model 31.

[0038] When the data is determined to be spectral irradiance data, processing by the spectral irradiance processing unit 42 is performed. The distortion reproduction unit 51 of the spectral irradiance processing unit 42 performs processing to generate data from the spectral irradiance data in which the distortion of pixels is reproduced. The pixel model generation unit 52 performs processing to transform the spectral irradiance data into pixel model data. For example, the spectral irradiance data supplied from the rendering system 21-1 is a value for each wavelength, and the pixel model generation unit 52 of the spectral irradiance processing unit 42 performs integration processing on the value for each wavelength according to the quantum conversion efficiency of the sensor, and then converts the value into a value within the sensor model 31, such as an electron count, taking into account the pixel size and exposure time.

[0039] When the data is determined to be RGB irradiance data, processing by the RGB irradiance processing unit 43 is performed. The distortion reproduction unit 53 of the RGB irradiance processing unit 43 performs processing to generate data from the RGB irradiance data in which the distorted pixels are reproduced. The pixel model generation unit 54 performs processing to transform the RGB irradiance data into pixel model data. For example, the RGB irradiance data supplied from the rendering system 21-2 is spectral energy values ​​separated into RGB values, and the pixel model generation unit 54 of the RGB irradiance processing unit 43 performs matrix calculation processing based on the spectral energy values ​​separated into RGB values, and converts these values ​​into values ​​within the sensor model 31, such as electron counts, taking into account pixel size and exposure time.

[0040] When the data is determined to be photon counting data, processing by the photon counting processing unit 44 is performed. The distortion reproduction unit 55 of the photon counting processing unit 44 performs processing to generate data from the photon counting data in which the distorted pixels are reproduced. The pixel model generation unit 56 performs processing to transform the photon counting data into pixel model data. For example, the photon counting data is supplied from the rendering system 21-3, and the pixel model generation unit 56 of the photon counting processing unit 44 converts the input image represented by the photon counting data into values ​​within the sensor model 31, such as electron counting.

[0041] Pixel model data from the spectral irradiance processing unit 42, the RGB irradiance processing unit 43, or the photon counting processing unit 44 is supplied to the pixel model processing unit 45, converted into an imaging signal through predetermined processing, and output to the application in a subsequent stage (not shown).

[0042] Operation of the simulation system

[0043] Reference Figure 2 Flowchart description Figure 1 An overview of the operation of the simulation system 10 shown is provided, and references will be made to... Figure 4 and Figure 5 The flowchart describes the detailed operation of sensor model 31. In the example described below, image data of spectral irradiance is supplied to sensor model 31 from rendering system 21 and processed by spectral irradiance processing unit 42.

[0044] In step S21, the rendering system 21 instructs the sensor model 31 to generate instances. When the sensor model 31 receives the instance generation instruction from the rendering system 21 in step S31, the sensor model 31 begins to generate instances. The number of instances generated is equal to the number of inputs required to generate one frame.

[0045] In step S32, the sensor model 31 returns the address of the instance to the rendering system 21.

[0046] Reference Figure 3 Describe the relationship between instances and image storage. For example... Figure 3 As shown in Figure A, sensor model 31 generates instance 101 managed by sensor model 31. Image memory 102 (the address of image memory 102) managed by sensor model 31 is registered in the generated instance 101. In this case, instance 101 and image memory 102 are located in sensor model 31, and the address of instance 101 is returned to rendering system 21.

[0047] like Figure 3 As shown in B, instance 101, managed by sensor model 31, is generated, and image memory 103 (the address of image memory 103), managed externally to sensor model 31, is registered in instance 101. In this case, the address of instance 101 is returned to rendering system 21.

[0048] In the examples described below, such as Figure 3 As shown in A, instance 101 and image memory 102 are managed in sensor model 31, and the address of image memory 102 is registered in instance 101.

[0049] When rendering system 21 is in step S22 ( Figure 2 When the rendering system 21 receives the address of instance 101 from the sensor model 31 in step S23, it issues a control start request to the sensor model 31. When the sensor model 31 receives the control start request from the rendering system 21 in step S33, the sensor model 31 begins control. For example, the control start request is issued by executing the Tick() function.

[0050] In step S34, sensor model 31 requests rendering system 21 to output image data. In step S24, rendering system 21, having received the request to output image data from sensor model 31, registers the image data in image memory 102 in instance 101, and outputs the address of instance 101 in step S25.

[0051] When sensor model 31 receives image data and an address from rendering system 21 in step S35, the received image data is retrieved in step S36 and managed in sensor model 31. For example, the GetImage() function is used for image data transmission and reception. Such image data transmission and reception is repeated an equal number of times as the number of inputs required to generate one frame.

[0052] Sensor model 31 performs the process of generating a pixel model using the managed image data.

[0053] Sensor model processing

[0054] Reference Figure 4 The flowchart shown further describes the processing in sensor model 31.

[0055] In step S101, the input image normalization unit 41 of the sensor model 31 is requested by the rendering system 21 to obtain the number of inputs required to generate one frame. The sensor model 31 then passes the number of inputs required to generate one frame to the rendering system 21.

[0056] In step S102, the rendering system 21 requests the input image normalization unit 41 of the sensor model 31 to generate an image storage class instance for the sensor model input an equal number of times as the number of inputs required to generate one frame. The sensor model 31 generates instance 101 and returns its address to the rendering system 21.

[0057] In step S103, the input image normalization unit 41 receives a control start request from the rendering system 21. In step S104, the input image normalization unit 41 requests the rendering system 21 to output an instance 101 storing image data, and in response to the request, acquires image data supplied from the rendering system 21. The image data acquired at this time is one of the number of images acquired when acquiring the number of inputs required to generate one frame (corresponding to the image data of the processing unit).

[0058] In step S104, the input image normalization unit 41 stores the input image data in the image storage class, and in this case, retrieves the image data whose address is managed by the image memory 102 in instance 101.

[0059] The process up to this point will be described again. In step S102, when an instance is generated, the size of the image memory 102 is set from the rendering system 21 side.

[0060] For reference Figure 3 As described in A, the address of the image memory 102 generated by sensor model 31 is registered in instance 101 and managed by sensor model 31. Alternatively, as referenced... Figure 3 As described in B, the address of the image memory 102 outside the sensor model 31 is registered in instance 101 from the rendering system 21 and managed by the sensor model 31.

[0061] exist Figure 3 In case B, the image width, height, and image type are set from the rendering system 21 during registration. The image size is managed by instance 101 based on the image width, height, and image type. The image storage 102 can be specified as either a graphics processing unit (GPU) or a central processing unit (CPU).

[0062] For reference Figure 6 The content of the data stored in image memory 102, in other words, the storage method (data processed as a single pixel), is different for each type of image. When rendering system 21 registers an image in instance 101, it sets the image type. Therefore, sensor model 31 can acquire the image type stored in the instance.

[0063] When photon counting data is stored in image memory 102, such as Figure 6 As shown in Figure A, the data for one photon count per pixel is stored serially in pixel order. When the RGB irradiance data is stored in image memory 102, as... Figure 6 As shown in Figure B, the data representing the red (R), green (G), blue (B), and alpha channel (A) of a pixel are stored serially in pixel order. Note that the data for the alpha channel (A) does not necessarily need to be stored in the image memory 102.

[0064] When spectral irradiance data is stored in image memory 102, such as Figure 6 As shown in C, all wavelength data is stored serially for each image. All wavelength data consists of data for each wavelength range from the minimum to the maximum wavelength. Figure 6 As shown in Figure C on the right, data from the minimum wavelength [0] to the maximum wavelength [N] is stored in a single pixel.

[0065] In step S106, the input image normalization unit 41 determines whether the acquisition of image data has been repeated a number of times equal to the number of inputs required to generate one frame. If it is determined in step S106 that the acquisition of image data a number of times equal to the number of inputs required to generate one frame has not been performed, the process returns to step S104, and the subsequent processing is repeated. If it is determined in step S106 that the acquisition of image data a number of times equal to the number of inputs required to generate one frame has been performed, the process proceeds to step S107.

[0066] In step S107, simulation processing of the sensor model is performed. (Referencing...) Figure 5 The flowchart describes the simulation processing of the sensor model performed in step S107.

[0067] In step S121, the input image normalization unit 41 determines the type and format of the image supplied from the rendering system 21. The image type may be, for example, spectral irradiance, RGB irradiance, photon count, etc. The format may be, for example, 16-bit floating-point format, 32-bit floating-point format, 64-bit floating-point format, etc.

[0068] For reference Figure 6As described, the image memory 102 stores the data of one pixel in the area corresponding to one pixel, and the image normalization unit 41 obtains the type of the image in instance 101 to determine the type of the image, or refers to the image data stored in the image memory 102 to determine the format.

[0069] In step S122, it is determined whether to perform distortion reproduction. If it is determined in step S122 that distortion reproduction should be performed, the process proceeds to step S123. In step S123, the distortion reproduction unit 51 ( Figure 1 Perform distortion reproduction. Refer to... Figure 7 Describes the processing related to distortion reproduction.

[0070] Rendering system 21 generates render images for very short time intervals, and for each very short time interval, the render image is supplied to sensor model 31. Since the render images for very short time intervals are processed as input image data only once, step S103 is repeated. Figure 4 ) to step S108 ( Figure 4 Furthermore, distortion correction calculations are performed after processing has been performed a number of times corresponding to the time amount constituting a frame, or during repeated execution.

[0071] The distortion reproduction unit 51 of the sensor model 31 reproduces sensor distortion using a rendered image supplied with pixel integration for the exposure object. In step S124, it is determined whether image data with sensor distortion reproduced by the distortion reproduction unit 51 has been generated, and if it has been generated, the image data is supplied to the pixel model generation unit 52. For example, when spectral irradiance data is supplied, image data with spectral irradiance reproducing sensor distortion is generated in the distortion reproduction unit 51 and supplied to the pixel model generation unit 52.

[0072] On the other hand, in step S124, if it is determined that no image data with sensor distortion reproduced by the distortion reproduction unit 51 is generated, the image data is not supplied to the pixel model generation unit 52, and the process waits for the input of the next very short amount of time for a rendered image. That is, in this case, the process returns to step S103. Figure 4 ), and repeat subsequent processing to obtain the necessary rendered image.

[0073] On the other hand, in step S122, if it is determined that distortion reproduction will not be performed, the process proceeds to step S125. In step S125, image conversion processing is performed. Image data is supplied to a processing unit that processes images corresponding to the type determined by the input image normalization unit 41, and the processing is performed. If the input image normalization unit 41 determines that the image type is spectral irradiance, the image data is supplied to the pixel model generation unit 52 of the spectral irradiance processing unit 42, and, for example, processing to convert the spectral irradiance data into electron counting data is performed.

[0074] When the input image normalization unit 41 determines that the image type is RGB irradiance, the image data is supplied to the pixel model generation unit 54 of the RGB irradiance processing unit 43, and, for example, the processing of converting the RGB irradiance data into electron count data is performed.

[0075] If the input image normalization unit 41 determines that the image type is photon counting, the image data is supplied to the pixel model generation unit 56 of the photon counting processing unit 44, and, for example, the processing of converting the photon counting data into electron counting data is performed.

[0076] Data processed by the spectral irradiance processing unit 42, the RGB irradiance processing unit 43, or the photon counting processing unit 44 is supplied to the pixel model processing unit 45. In step S126, the pixel model processing unit 45 performs predetermined processing on the supplied pixel model to generate an imaging signal. When in step S107... Figure 4 When such processing is performed in the process, the sensor model 31 completes the processing of one frame of image, executes a loop via step S108, and the processing returns to step S103, which receives the control start request for the next input. Figure 4 ).

[0077] As described above, the input image normalization unit 41 has the function of generating instances in response to requests from the rendering system 21. The input image normalization unit 41 also has the function of registering and managing the addresses of memory used for images from the rendering system 21. During registration, the width, height, and type of the image are set from the rendering system 21. The input image normalization unit 41 also has the function of passing image information such as width and height to the rendering system. Furthermore, the input image normalization unit 41 also has the function of passing the address of the image memory to the rendering system 21.

[0078] The input image normalization unit 41 has the function of receiving wavelength information (minimum wavelength, maximum wavelength, wavelength resolution) of spectral irradiance from the rendering system 21 and transmitting the wavelength information of spectral irradiance to the rendering system 21. The input image normalization unit 41 also has the function of receiving information from the rendering system 21 regarding each of the R, G, B, and A values ​​of RGB irradiance and transmitting the RGB irradiance information to the rendering system 21. Finally, the input image normalization unit 41 has the function of receiving photon count information from the rendering system 21 and transmitting the photon count information to the rendering system 21.

[0079] The input image normalization unit 41 or instance can receive a pixel value gain setting from the rendering system 21 and request the sensor model 31 to uniformly multiply all pixel values ​​of the image data by that gain. It can also provide the function of passing the pixel value gain to the rendering system 21.

[0080] The input image normalization unit 41 has these functions, so it can uniformly process image data from the rendering system 21 that processes different types of image data.

[0081] Configuration of the sensor model in the second embodiment

[0082] Figure 8 This is a diagram illustrating a configuration example of a sensor model according to the second embodiment. Figure 8 The sensor model 231 shown includes an input image normalization unit 241, a spectral irradiance processing unit 242, an RGB irradiance processing unit 243, a photon counting processing unit 244, and a pixel model processing unit 245. The spectral irradiance processing unit 242 includes a pixel model generation unit 252, the RGB irradiance processing unit 243 includes a pixel model generation unit 254, and the photon counting processing unit 244 includes a pixel model generation unit 256.

[0083] Sensor Model 231 and Figure 1 The difference in the sensor model 31 shown is that the distortion reproduction units 51, 53, and 55 have been removed. The sensor model 231 also includes a linear shutter information output unit 246, which outputs linear shutter information from the pixel model processing unit 245 to the rendering system 21.

[0084] According to the second embodiment, the sensor model 231 is not configured to perform distortion reproduction in the sensor model 231, but is configured to obtain image data from the rendering system 21 that has already undergone distortion reproduction.

[0085] Reference Figure 9Describes the processing related to distortion reproduction. In order for the rendering system 21 to generate image data of an image that has undergone distortion reproduction, the line shutter information output unit 246 of the sensor model 231 obtains shutter control information that associates the exposure start time (open) and exposure end time (closed) with each line from the pixel model processing unit 245, and supplies the shutter control information to the rendering system 21.

[0086] The rendering system 21 uses the shutter control information supplied from the sensor model 231 to generate a frame that reproduces the distortion and supplies the image data to the sensor model 231.

[0087] The sensor model 231 according to the second embodiment operates in essentially the same manner as the sensor model 31 according to the first embodiment, therefore its description is omitted.

[0088] According to this technology, by setting an input image normalization unit 41 (241) in the sensor model 31 (231), the standard of the image input to the sensor model 31 can be unified, and signals from multiple rendering systems 21 can be processed. In other words, different types of image data can be processed. Typically, the rendering system 21 and the sensor model 31 (231) have a one-to-one correspondence, but they can have a many-to-one correspondence. For example, even when the rendering system 21 changes, the sensor model 31 (231) can still process image data from the replaced rendering system 21.

[0089] Even when the rendering system 21 changes, there is no need to redesign the sensor model 31 (231), and a highly versatile sensor model 31 (231) can be provided.

[0090] By setting the distortion reproduction unit 51 or the linear shutter information output unit 246, image sensor-specific distortions generated in the actual image sensor can be reproduced, and more accurate simulations can be performed.

[0091] Recording media

[0092] The aforementioned series of processing steps can be performed by hardware or software. When the series of processing steps are performed by software, the program constituting that software is installed on the computer. Here, the computer includes a computer built into dedicated hardware, or, for example, a general-purpose personal computer that can perform various functions by installing various programs.

[0093] Figure 10This is a block diagram illustrating an example hardware configuration of a computer that performs the aforementioned series of processes via a program. In the computer, a central processing unit (CPU) 2001, a read-only memory (ROM) 2002, and a random access memory (RAM) 2003 are connected to each other via a bus 2004. Furthermore, an input / output interface 2005 is connected to the bus 2004. An input unit 2006, an output unit 2007, a storage unit 2008, a communication unit 2009, and a driver 2010 are connected to the input / output interface 2005.

[0094] Input unit 2006 consists of a keyboard, mouse, microphone, etc. Output unit 2007 consists of a display, speakers, etc. Storage unit 2008 consists of a hard disk, non-volatile memory, etc. Communication unit 2009 consists of a network interface, etc. Driver 2010 drives removable media 2011 such as disks, optical disks, magneto-optical disks, or semiconductor memories.

[0095] In a computer configured as described above, for example, CPU 2001 loads a program stored in storage unit 2008 into RAM 2003 via input / output interface 2005 and bus 2004 and executes the program, thereby performing the series of processes described above.

[0096] The program executed by the computer (CPU 2001) can be provided, for example, by recording it in a removable medium 2011 as an encapsulation medium. The program can also be provided via wired or wireless transmission media (such as a local area network, the Internet, or digital satellite broadcasting).

[0097] In the computer, a program can be installed in the storage unit 2008 via the input / output interface 2005 by installing the removable medium 2011 on the drive 2010. Alternatively, the program can be received by the communication unit 2009 and installed in the storage unit 2008 via a wired or wireless transmission medium. Additionally, the program can be pre-installed in the ROM 2002 or the storage unit 2008.

[0098] Note that a program executed by a computer may be a program for sequentially executing the processes described in this specification, or it may be a program for executing processes in parallel or at necessary time intervals (such as when a call is executed).

[0099] In this specification, "system" refers to an overall device consisting of multiple devices.

[0100] Note that the effects described in this manual are merely examples and are not limited to them; other effects may be provided.

[0101] Note that the embodiments of this technology are not limited to the above embodiments, and modifications can be made without departing from the spirit of this technology.

[0102] This technology can also be configured as follows.

[0103] (1) A simulation device, comprising:

[0104] Image input interface, the image input interface being configured as

[0105] Generate instances,

[0106] In the example described, the address of the image memory and the image type are managed, as well as

[0107] Determine the image type based on the instance; and

[0108] A processing unit configured to perform processing corresponding to an image type.

[0109] (2) According to the simulation device described in (1),

[0110] The simulation device acquires image data from the rendering device for a predetermined amount of time, and

[0111] The image data from the image, taken over the predetermined time period, reproduces the sensor's distortion.

[0112] (3) The simulation device described in (1),

[0113] The simulation device supplies the rendering device with shutter information, including the exposure start time and exposure end time, and...

[0114] Obtain image data from the rendering device that reproduces the distortion of the image.

[0115] (4) The simulation device according to any one of (1) to (3),

[0116] The image type is one of the following: spectral irradiance; red, green, and blue (RGB) irradiance; or photon count.

[0117] (5) The simulation device according to any one of (1) to (4),

[0118] The instance manages the address of the image memory allocated inside the analog device or the address of the image memory allocated outside the analog device.

[0119] (6) The simulation device according to (4),

[0120] Image input interfaces are stored pixel-by-pixel in image memory.

[0121] Image data representing the spectral irradiance of a pixel.

[0122] Image data representing the RGB irradiance of a pixel, or

[0123] Image data representing the photon count of a pixel.

[0124] (7) A control method, comprising:

[0125] An image input interface configured to control the input and output of image data.

[0126] Generate instances;

[0127] In this example, the address of the image memory and the image type are managed;

[0128] Determine the image type based on the example; and

[0129] Image data is supplied to processing units configured to perform processing corresponding to the image type.

[0130] (8) A simulation system, comprising:

[0131] Rendering devices; and

[0132] The simulation device is configured to perform a simulation using image data from a rendering device, wherein

[0133] The simulation equipment includes

[0134] Image input interface, the image input interface being configured as

[0135] In response to an instruction from the rendering device, an instance is generated.

[0136] In this example, the address of the image memory is managed.

[0137] Image data from the rendering device and image type are stored in the image memory, and

[0138] Determine the type of image stored from the image storage device, and

[0139] A processing unit configured to perform processing corresponding to an image type.

[0140] List of reference numerals

[0141] 10 Simulation System 21 Rendering System 31 Sensor Model 41 Input Image Normalization Unit 42 Spectral Irradiance Processing Unit 43 RGB Irradiance Processing Unit 44 Photon Counting Processing Unit 45 Pixel Model Processing Unit 51 Distortion Reproduction Unit 52 Pixel Model Generation Unit 53 Distortion Reproduction Unit 54 Pixel Model Generation Unit 55 Distortion Reproduction Unit 56 Pixel Model Generation Unit 101 Instance 102 Image Memory 103 Image Memory 231 Sensor Model 241 Input Image Normalization Unit 242 Spectral Irradiance Processing Unit 243 RGB Irradiance Processing Unit 244 Photon Counting Processing Unit 245 Pixel Model Processing Unit 246 Linear Shutter Information Output Unit 252 Pixel Model Generation Unit 254 Pixel Model Generation Unit 256 Pixel Model Generation Unit

Claims

1. A simulation device, comprising: Image input interface, the image input interface being configured as Generate instances, In the example described, the address of the image memory and the image type are managed, as well as Determine the image type based on the instance; as well as A processing unit configured to perform processing corresponding to an image type.

2. The simulation device according to claim 1, wherein... The simulation device acquires image data from the rendering device for a predetermined amount of time, and The image data from the image, taken over the predetermined time period, reproduces the sensor's distortion.

3. The simulation device according to claim 1, wherein... The simulation device supplies the rendering device with shutter information, including the exposure start time and exposure end time, and... Obtain image data from the rendering device that reproduces the distortion of the image.

4. The simulation device according to claim 1, wherein... The image type is one of the following: spectral irradiance; red, green, and blue (RGB) irradiance; or photon count.

5. The simulation device according to claim 1, wherein... The instance manages the address of the image memory allocated inside the analog device or the address of the image memory allocated outside the analog device.

6. The simulation device according to claim 4, wherein Image input interfaces are stored pixel-by-pixel in image memory. Image data representing the spectral irradiance of a pixel. Image data representing the RGB irradiance of a pixel, or Image data representing the photon count of a pixel.

7. A control method, comprising: An image input interface configured to control the input and output of image data. Generate instances; In this example, the address of the image memory and the image type are managed; Determine the image type based on the example; as well as The image data is supplied to the processing unit, which is configured to perform the processing corresponding to the type.

8. A simulation system, comprising: Rendering devices; as well as The simulation device is configured to perform a simulation using image data from a rendering device, wherein The simulation equipment includes: Image input interface, the image input interface being configured as In response to an instruction from the rendering device, an instance is generated. In this example, the address of the image memory is managed. Image data from the rendering device and image type are stored in the image memory, and Determine the type of image stored from the image storage device, and A processing unit configured to perform processing corresponding to an image type.

Images