Computer program, information processing method, and information processing apparatus

Abstract

The present invention provides a computer program, an information processing method, and an information processing device that predict temperature changes in an image output device in advance and perform appropriate thermal control. [Solution] The information processing device includes an input layer that inputs a time-series first thermal information based on an image input to an image output device; a thermal reservoir layer having a plurality of virtual nodes that hold a time-series second thermal information observable at each of a plurality of locations as the first thermal information propagates to a plurality of locations of the image output device based on the characteristics including the thermal characteristics of the image output device; a coupling layer that linearly combines the time-series second thermal information held by the plurality of virtual nodes of the thermal reservoir layer using coupling coefficients between the plurality of virtual nodes based on the time-series first thermal information; and an output layer that outputs the linearly combined second thermal information as an output value; and a determination unit that determines the coupling coefficients based on training data including the first thermal information based on the image and the correct values ​​of the output value.

Description

Technical Field

[0001] The present invention relates to a computer program, an information processing method, and an information processing apparatus.

Background Art

[0002] Conventionally, in outdoor events, digital signage, etc., image output devices such as projectors, liquid crystal displays, and organic EL displays have been used. Regarding the temperature control of such image output devices, for example, Patent Document 1 describes a projector that detects the environmental temperature, calculates the temperature of a light modulation device to be cooled, and drives and controls a cooling fan according to the calculated target temperature of the light modulation device.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional temperature control method, since heat control and adjustment are performed after a temperature change occurs, there is a problem that heat generation cannot be predicted and suppressed in advance. For this reason, a heat distribution is formed inside the device, and as a result, image distortion, defocus, non-uniformity of luminance, etc. occur. Particularly in large projectors and business displays with high brightness and large area, such image quality degradation due to thermal non-uniformity becomes significant. Therefore, there is a need to provide a temperature control technology that can predict temperature changes based on image information and projection states and prevent the occurrence of heat distribution.

[0005] This invention has been made in view of the above circumstances, and aims to provide an information processing device, an information processing method, and a computer program that can predict temperature changes in an image output device in advance and perform appropriate thermal control. [Means for solving the problem]

[0006] The computer program according to this embodiment comprises: an input layer that inputs a time-series first thermal information based on an image input to an image output device; a thermal reservoir layer to which the time-series first thermal information is input from the input layer, and which has a plurality of virtual nodes that hold a time-series second thermal information observable at each of a plurality of locations as the first thermal information propagates to a plurality of locations of the image output device based on characteristics including the thermal characteristics of the image output device; and a plurality of virtual nodes of the thermal reservoir layer that, based on the time-series first thermal information input to the thermal reservoir layer, The system functions as follows: a coupling layer that linearly combines the time-series second thermal information held by the virtual nodes using coupling coefficients between multiple virtual nodes; an output layer that outputs the linearly combined second thermal information from the coupling layer as an output value; and a determination unit that, based on training data including the time-series first thermal information and the correct output value based on the first image, determines the coupling coefficients so that when the time-series first thermal information is input to the thermal reservoir layer, the output value, which is obtained by linearly combining the time-series second thermal information held by multiple virtual nodes using coupling coefficients between multiple virtual nodes, approaches the correct value. [Effects of the Invention]

[0007] According to the present invention, it is possible to predict temperature changes in an image output device in advance and perform appropriate thermal control. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the configuration of the information processing device according to this embodiment. [Figure 2] This figure shows an example of the configuration of the thermal reservoir model section. [Figure 3]This figure shows an example of training data used to train the thermal reservoir model. [Figure 4] This figure shows an example of the learning process for the thermal reservoir model unit 60. [Figure 5] This figure shows an example of evaluating an evaluation target based on the output of the output layer. [Figure 6] This figure shows an example of learning processing by an information processing device. [Figure 7] This figure shows an example of inference processing by an information processing device. [Modes for carrying out the invention]

[0009] The present invention will be described below based on the drawings illustrating its embodiments. Figure 1 is a diagram showing an example of the configuration of the information processing device 50 in this embodiment. The information processing device 50 includes a control unit 51 that controls the entire device, an input / output unit 52, a memory 53, a learning processing unit 54, an evaluation unit 55, a storage unit 56, a thermal control unit 58, and a thermal reservoir model unit 60. The information processing device 50 may be composed of multiple devices with distributed processing functions.

[0010] The storage unit 56 can be configured, for example, as a hard disk or semiconductor memory, and stores a computer program 57 (program product) and necessary information. The computer program 57 may be downloaded from an external device and stored in the storage unit 56. Alternatively, the computer program 57 recorded on a recording medium (for example, an optically readable disk storage medium such as a CD-ROM) may be read by a recording medium reading unit and stored in the storage unit 56.

[0011] The control unit 51 is configured with the required number of CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), etc. The control unit 51 can execute the processes defined in the computer program 57. In other words, the processing performed by the control unit 51 is also the processing performed by the computer program 57. By executing the computer program 57, the control unit 51 can perform the functions of the learning processing unit 54 and the evaluation unit 55.

[0012] The input / output unit 52 acquires data necessary for processing by the information processing device 50 and outputs the data obtained as a result of processing by the information processing device 50. The input / output unit 52 may also be equipped with a user interface such as a keyboard, mouse, display panel, or touch panel. The input / output unit 52 can acquire time-series first thermal-related information input to the image output device 200 as the target object.

[0013] The memory 53 can be composed of semiconductor memory such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory. The computer program 57 can be loaded into the memory 53, and the control unit 51 can execute the computer program 57.

[0014] The learning processing unit 54 performs the learning process for the thermal reservoir model unit 60. Details of the learning process will be described later.

[0015] The thermal reservoir model unit 60 is a reservoir computing model and comprises an input layer 61, a thermal reservoir layer 62, a coupling layer 63, and an output layer 64. The thermal reservoir model unit 60 can be implemented using hardware such as an FPGA (Field Programmable Gate Array). The input layer 61 receives first thermal-related information (collectively referred to as "input data") in a time series, which is acquired via the input / output unit 52 under the control of the control unit 51 and input to the image output device 200. The location in the image output device 200 to which the first thermal-related information is input (also referred to as the "input point") is not particularly limited and may be any part of the image output device 200 that emits heat. The first thermal-related information is nonlinearly transformed by the thermal reservoir layer 62 and mapped to a high-dimensional feature space. The mapping to the feature space is represented by the state of multiple virtual nodes in the thermal reservoir layer 62. The coupling layer 63 linearly combines the state values ​​of multiple virtual nodes in the thermal reservoir layer 62 using coupling coefficients (weighting coefficients). The output layer 64 outputs the linearly combined output values. The output values ​​output by the thermal reservoir model unit 60 can be used to evaluate the temperature-related state of the image output device 200. Details of the thermal reservoir model unit 60 will be described later.

[0016] The first heat-related information may include, for example, information regarding an image output by the image output device 200 (also referred to as "image information"). "Image information" is information regarding an image and may include, for example, numerical data set corresponding to pixels constituting the image, or attribute information associated with the numerical data. As the numerical data, RGB values (Red, Green, Blue values), luminance values (Y values), or values indicating transparency, etc. for each pixel may be included. The RGB value is a numerical value indicating the intensity of each of the red component, green component, and blue component in each pixel, and may be represented as an integer value in the range of 0 to 255 in 8-bit units, for example. The color of one pixel is specified by a set of these three component values (R value, G value, B value), and the entire image is configured as a set of RGB values for each of these pixels. Such a set may be held as digital data (e.g., bitmap data) representing the image, and may also be used as input information in various processes such as image processing, object recognition, feature amount extraction, etc. The attribute information may include the resolution of the image, pixel array, type of color space, or compression format, etc. Further, the image information may include feature amounts, detection results, or analysis results, etc. calculated based on the image.

[0017] The evaluation unit 55 evaluates the state related to the temperature of the image output device 200 based on the output value output by the heat reservoir model unit 60. Details of the evaluation unit 55 will be described later.

[0018] The information processing device 50 includes reservoir computing that realizes reservoir computing by using the image output device 200 as a heat reservoir. In this specification, a solid, liquid, gas, etc. through which heat can propagate is referred to as a heat reservoir. Hereinafter, the heat reservoir will be described by taking a solid as an example. Note that the information processing device 50 and the image output device 200 may be integrally configured.

[0019] The image output device 200 according to this embodiment has a video display mechanism such as a display or a projector, and the entire device is configured as a heat reservoir in thermal reservoir computing. In this configuration, the image output device 200 functions as a heat reservoir, and a non-linear mapping between image output (display or projection, etc.) and temperature response is realized by reservoir computing. Here, a heat reservoir refers to the entire object capable of heat propagation, and has a function of accumulating and propagating time-series heat-related information input based on physical characteristics such as the thermal conductivity, thermal diffusivity, and shape of the object. That is, when the image output device 200 outputs light or video based on an input image signal, it holds the time-series change of the heat distribution generated inside the device as its internal state. At this time, physical quantities that change depending on temperature information or temperature changes observed at multiple positions inside the device function as a virtual node group and form the internal state space of reservoir computing.

[0020] In the case of a projector, the entire device including a light source, an optical system, a housing structure, a heat sink, and a fan unit, etc. is treated as a heat reservoir. These have different material characteristics, and the temperature distribution generated in response to video input responds non-linearly. Particularly in a high-brightness projector, since high-output light exceeding 1000 W / cm 2 is irradiated in the light source unit, the temperature field of the entire optical path and housing changes every moment. This temperature response functions as the internal state of the reservoir in thermal reservoir computing. On the other hand, in the case of a thin video projection device, for example, an organic EL display or a liquid crystal display, the entire TV including a back panel, a heat conduction plate, a housing frame, and an internal high thermal conductivity layer is configured as a heat reservoir. Organic EL panels, etc. have high temperature dependence, and the luminous efficiency and gradation response change due to temperature fluctuations, so the thermal changes corresponding to the input image appear in a non-linear form. By utilizing this non-linear response, it becomes possible to estimate the future temperature distribution from the input video signal.

[0021] The image output device 200 includes at least one temperature sensor 201. The temperature sensor 201 detects the temperature at a predetermined location in the image output device 200, and the detected value is associated as a virtual node value. That is, each temperature sensor 201 corresponds to a virtual node and functions as an interface for externally observing the internal temperature state of the image output device 200.

[0022] The image output device 200 includes at least one thermal control device 202. The location of the thermal control device 202 is not particularly limited, but for example, it may be located in a position corresponding to the temperature sensor 201 (such as near the temperature sensor 201). The configuration of the thermal control device 202 is not particularly limited as long as it is a device for thermally controlling the image output device 200, but for example, it may include a cooling device (cooling means) such as a cooling fan and a Peltier element, or a heating device (heating means) such as a heater and a heat generating element. The thermal control device 202 may be located in a position corresponding to the temperature sensor 201 and performs thermal control to dynamically adjust the temperature distribution inside the device.

[0023] In this embodiment, the evaluation unit 55 evaluates the temperature-related state inside the image output device 200 based on the output value obtained by thermal reservoir computing. The evaluation unit 55 analyzes the temperature distribution and time-series changes of the thermal reservoir (i.e., the entire image output device 200) and can detect, for example, a steepening of the temperature gradient, a localized temperature rise, or thermal non-uniformity that affects image quality. When the evaluation target exceeds a predetermined threshold, it is determined to be abnormal, and the stability of the image quality and the degree of image distortion are quantitatively evaluated based on the error between the output value and the correct value. Based on these evaluation results, thermal control is performed in the image output device 200. That is, the evaluation unit 55 can start thermal control before temperature disturbances occur by pre-driving the thermal control device 202 in response to the predicted change in temperature distribution. This prevents bias in the heat distribution that could not be prevented by conventional post-hoc control, which performs thermal control after detecting a temperature rise, and stabilizes the temperature around the heat-generating part of the image output device 200.

[0024] If multiple thermal control devices 202 are provided within the image output device 200, each thermal control device 202 can be controlled in coordination based on the output of the evaluation unit 55. That is, the output of each cooling device, heating device, etc., is individually optimized according to the predicted temperature distribution, and spatially optimized thermal control operation is achieved, for example, by strengthening thermal control in one area and suppressing the output in another area. This coordination control makes it possible to maintain high thermal control efficiency and image quality stability overall while reducing unnecessary power consumption and noise.

[0025] During the training of the thermal reservoir model unit 60, the image output device 200 is made to output an image, and the thermal control device 202 is controlled. The control signals for controlling the thermal control device 202 may also be included in the training data used to train the thermal reservoir model unit 60. By using the thermal control signals as training data (input parameters), the model can understand how the image output device 200 (display or projector) actually dissipates or controls heat by the thermal control device 202, which helps in more accurate temperature change prediction and optimization of proactive thermal control. In this way, by combining proactive thermal control based on the evaluation results of the evaluation unit 55 with cooperative control, it is possible to stabilize the temperature distribution in the image output device 200, improve image quality, and achieve other improvements.

[0026] The image output device 200 can perform thermal control optimized for each environment, even when the environmental conditions (such as ambient temperature and humidity) in which it is installed differ. In this invention, multiple models may be generated for each type of environmental information in order to respond to fluctuations in thermal response due to differences in environmental conditions. That is, multiple thermal reservoir computing models may be switched according to environmental information such as region and climate. As a result, the device becomes less susceptible to the effects of environmental changes and can predict the temperature distribution in the near future and proactively control the thermal control device.

[0027] Figure 2 shows an example of the configuration of the thermal reservoir model section 60. When the input layer 61 receives the first thermal information in a time series, it inputs the first thermal information in a time series to the thermal reservoir layer 62.

[0028] The thermal reservoir layer 62 comprises multiple virtual nodes. The thermal reservoir layer 62 is realized as a physical phenomenon of heat propagation in a thermal reservoir. That is, the thermal reservoir layer 62 comprises multiple virtual nodes that hold time-series second thermal information (for example, including temperature data or physical quantities that change depending on temperature changes) that can be observed at each of multiple locations (observation points) of the thermal reservoir, based on the thermal conductivity or thermal diffusivity (thermal-related properties) of the thermal reservoir that can propagate heat, or the shape (properties) of the thermal reservoir. The construction and design of the thermal reservoir layer 62 can be performed, for example, by determining physical parameters (such as the dimensions, shape, thermal conductivity (or thermal diffusivity), and material of the thermal reservoir) according to the learning target and learning task, based on simulation experiments.

[0029] The coupling layer 63 linearly combines the time-series second thermal-related information held by multiple virtual nodes of the thermal reservoir layer 62 using coupling coefficients between the multiple virtual nodes. The coupling layer 63 and the output layer 64 are collectively referred to as the readout layer.

[0030] The output layer 64 outputs the output values ​​that are linearly combined in the coupling layer 63. The number of dimensions of the output layer 64 can be set arbitrarily.

[0031] Next, we will explain the thermal reservoir computing method, that is, the learning method for the thermal reservoir model unit 60.

[0032] Figure 3 shows an example of training data used to train the thermal reservoir model unit 60. Figure 3 Various states corresponding to the image information output by the image output device 200 Figure 3 The first thermal-related information in time series corresponding to the image is input to the image output device 200. In Figure 3, for convenience, we will call them states 1, 2, ..., m. For example, state 1 corresponds to image information 1 (image information at time t1=t0), state 2 corresponds to image information 2 (image information at time t2=t0+dt), ... state m corresponds to image information m (time t m This corresponds to the image information at t0 + m·dt. The first thermal information may include, for example, image information. The first thermal information may also include temperature data or thermal data. The training input data collected corresponding to states 1, 2, ..., m of the image output device 200 are represented by vectors G1, G2, and Gm. Also, when the state of the image output device 200 is state 1, 2, ..., m, the correct values ​​(true values) of the output values ​​representing the state of the image output device 200 are represented by vectors D1, D2, ..., Dm. Vectors D1, D2, ..., Dm serve as training data.

[0033] Furthermore, when the training input data G1 is input to the thermal reservoir layer 62, the values ​​indicating the state of multiple virtual nodes in the thermal reservoir layer 62 are represented by vector S1. Vector S1 corresponds to the time-series second thermal-related information held by the multiple virtual nodes, and includes, for example, image information, temperature data, or physical quantities that change depending on temperature changes. When vector S1 is input to the coupling layer 63, the vector of output values ​​output by the coupling layer 63 is represented by Y1. Also, when the training input data G2 is input to the thermal reservoir layer 62, the values ​​indicating the state of multiple virtual nodes in the thermal reservoir layer 62 are represented by vector S2, and when vector S2 is input to the coupling layer 63, the vector of output values ​​output by the coupling layer 63 is represented by Y2. Similarly, when the training input data Gm is input to the thermal reservoir layer 62, the values ​​representing the states of multiple virtual nodes in the thermal reservoir layer 62 are represented by vector Sm, and when vector Sm is input to the coupled layer 63, the vector of output values ​​output by the coupled layer 63 is represented by Ym.

[0034] In this embodiment, the correct value is defined based on the temperature at each location when the image information is projected without distortion. That is, when the image output device 200 is displaying or projecting predetermined image information, if the temperature information observed at multiple locations corresponding to the display surface or internal structure does not change before and after projection based on the next image information, then the temperature distribution at that time corresponds to the correct value.

[0035] More specifically, consider a scenario where image information was displayed correctly at time t1, and new image information is input at the next time t2. In this case, if the temperature T1 (before projection) and T2 (after projection) at each observation point are equal, that is, if the temperature change dT = T2 - T1 = 0, then it is determined that no temperature disturbance has occurred. In such a state, if it is a projector, there will be no fluctuation in focus, and if it is a display, additional processing such as brightness correction will not be necessary, so no image disturbance occurs. This temperature distribution represents the ideal correct state.

[0036] The actual temperature information consists of multiple temperature values ​​observed by multiple temperature sensors 201 located inside or on the surface of the image output device 200, and these are treated as vector quantities. Therefore, the correct value can be expressed as any linear combination of temperature vectors at each observation point. That is, not only when the temperature is perfectly constant at all observation points, but also when the image information is projected without distortion by a linear combination of multiple temperature distributions, the temperature distribution is also treated as the correct value. In this way, the correct value in this embodiment represents the stability of the image information, i.e., the thermal equilibrium state corresponding to the presence or absence of image distortion, and is used as a criterion for minimizing the error with the output value vector during the learning stage.

[0037] Figure 4 shows an example of the learning process of the thermal reservoir model unit 60. Conventional recurrent neural networks are complex, nonlinear, and high-dimensional models, requiring large amounts of data and processing time for learning. In thermal reservoir computing (TRC), the thermal reservoir layer 62 is implemented using the entire image output device (display or projector), i.e., a thermal reservoir capable of heat propagation (see Figure 2), making it possible to realize a high-dimensional, nonlinear model of the nonlinear temperature response caused by the input image information, independently of the neural network. In the thermal reservoir layer 62, the state of multiple virtual nodes is randomly set according to the thermal-related characteristics of the thermal reservoir (thermal conductivity, thermal diffusivity, etc.), so there is no need to adjust the parameters within the thermal reservoir layer 62 through learning. That is, the output of the output layer 64 is given by the coupling layer 63 as a linear combination of the states of the thermal reservoir layer 62 (time-series temperature changes at observation points inside the device). The coupling coefficients (weight coefficients) of the coupling layer 63 can be trained using a simple algorithm such as linear regression so that the output of the output layer 64 is the same as or close to the correct value. (Figure 4, Figure 2)

[0038] As shown in Figure 4A, let the virtual nodes of the thermal reservoir layer 62 be s1, s2, s3, ..., sj. Here, q may be the number of observation points and N the number of discrete values, and j = q·N. The virtual node s corresponds to the time-series second thermal information (temperature data, etc.) observed at multiple positions (observation points) of the image output device 200. Let the output values ​​(output nodes) of the output layer 64 be y1, y2, ..., yr. Let the coupling coefficients from virtual node s1 to output values ​​y1, y2, ..., yr be w11, w12, ..., w1r. Similarly, let the coupling coefficients from virtual node sj to output values ​​y1, y2, ..., yr be wj1, wj2, ..., wjr.

[0039] As shown in Figure 4B, if we represent the output value vector as Y, the virtual node value vector corresponding to the temperature sensed at each point of the display or projector as S, and the coupling coefficient matrix as W, then in the equation Y = S·W, we should calculate the coupling coefficients of the coupling coefficient matrix W so that the output value vector Y approaches the correct value vector D.

[0040] As described above, the learning processing unit 54 functions as a decision unit and acquires training data including the first thermal information (thermal data or temperature data) in time series (provided by the input image information) and the correct output value. Based on the acquired training data, when the first thermal information in time series is input to the thermal reservoir layer 62, the thermal reservoir model unit 60 is trained by determining the coupling coefficients so that the output value, which is output by linearly combining the second thermal information in time series held by multiple virtual nodes of the thermal reservoir layer 62 using coupling coefficients between multiple virtual nodes, approaches the correct value. In this case, the correct value refers to the temperature on which the image information is projected without distortion, or a thermal equilibrium state that is represented by a linear combination of temperature vectors at multiple observation points and on which no image distortion occurs.

[0041] As described above, since learning can be performed using a simple configuration such as the coupled layer 63 and a simple algorithm such as linear regression, a large amount of data is not required, the processing time is relatively short, and it is possible to operate with low power consumption, as well as real-time learning. As a result, the information processing device 50 of this embodiment can be applied to edge computing such as IoT devices that predict temperature changes and perform proactive thermal control.

[0042] The output value y and the output value vector Y are not physical quantities with units, but rather numerical values ​​resulting from calculations. It is necessary to determine how these calculation results relate to the state of the thermal reservoir, particularly the temperature-related state, i.e., the thermal non-uniformity related to the image quality stability and resolution degradation of the image output device 200.

[0043] Figure 5 shows an example of evaluating an object to be evaluated based on the output of the output layer 64. The evaluation unit 55 evaluates the state related to the temperature of the image output device (display or projector) to be evaluated, based on the output value (including the prediction of the temperature distribution in the near future) output by the output layer 64. This state of the object to be evaluated includes steepening of the temperature gradient, localized temperature rise, or the degree of thermal non-uniformity that affects image quality. Figure 5A shows the case where the output layer 64 outputs multiple output values. Multiple evaluations 1, 2, ..., r are set corresponding to each output value. If the value of the output value is above a predetermined threshold, the evaluation corresponding to that output value can be determined as the evaluation result of the image output device 200 as the object. The state that evaluations 1, 2, ..., r represent can be determined as appropriate. Evaluations 1, 2, ..., r may each be evaluations of different states (for example, the stability of image quality, the degree of image distortion, the need for thermal control, etc.), or they may be temporal changes in a specific state (for example, the prediction result of the degree of equipment degradation, etc.).

[0044] Figure 5B shows the case where the output layer 64 outputs a single output value y. For example, if the output value y is above a threshold, the image output device 200 determines that it is normal (a thermal equilibrium state in which no image distortion occurs), and if the output value y is below the threshold, the evaluation target can be determined to be abnormal (there is a possibility of temperature disturbance). Based on this evaluation result, the evaluation unit 55 performs proactive thermal control by driving the thermal control device in advance according to the predicted change in temperature distribution, thereby starting thermal control before temperature disturbance occurs. Note that the evaluation method is not limited to the example in Figure 5.

[0045] Figure 6 shows an example of the learning process performed by the information processing device 50. For convenience, the main unit of the process will be described below as the control unit 51. The image output device 200 outputs a predetermined image (first image). This first image may be a test image. The control unit 51 acquires training data (S11) which includes input data (time-series first thermal-related information) and correct values ​​of the output values ​​for the thermal reservoir resulting from the image output device 200 outputting the said image. The time-series first thermal-related information may include, for example, image information of the first image, thermal data, and temperature data. Then, based on the acquired training data, the control unit 51 inputs input data (thermal data for learning the near-future temperature response) to the thermal reservoir layer 62 via the input layer 61 (S12).

[0046] The control unit 51 adjusts the coupling coefficient (weight coefficient) of the coupling layer 63 so that the output value output by the output layer 64 approaches the correct value (S13). Here, the correct value is the temperature at which image information is projected without distortion, that is, the value corresponding to a thermal equilibrium state in which no image distortion occurs. In thermal reservoir computing, learning is performed using simple algorithms such as linear regression, resulting in low learning costs and the ability to learn in real time. The control unit 51 determines whether the difference between the output value and the correct value (the error in the predicted temperature distortion) is within an acceptable range (S14), and if it is not within an acceptable range (NO in S14), it continues the processing from step S12 onward.

[0047] If the difference between the output value and the correct value is within an acceptable range (YES in S14), the control unit 51 stores the adjusted coupling coefficients (weights of the model for predicting the near-future temperature) in the storage unit 56 (S15) and terminates the process. These coupling coefficients are used in the subsequent inference process (Figure 7) to predict the near-future temperature distribution from the input image information and to perform proactive thermal control.

[0048] Figure 7 shows an example of inference processing by the information processing device 50. The control unit 51 acquires a time-series first thermal-related information based on a second image different from the first image (S21). The second image may be, for example, an image that the user wishes to view, rather than a test image. The time-series first thermal-related information may include, for example, image information, thermal data, and temperature data of the second image. The control unit 51 then inputs the time-series first thermal-related information related to the second image acquired via the input layer 61 to the thermal reservoir layer 62 (S22).

[0049] The control unit 51 acquires the output value (including a prediction of the near-future temperature distribution) output by the output layer 64 when the image output device 200 outputs a second image (S23), and evaluates the temperature-related state of the image output device 200 that is the subject of evaluation (such as steepening of the temperature gradient and thermal non-uniformity) based on the acquired output value (S24).

[0050] Based on the evaluation result of step S24 (predicted temperature disturbance or need for thermal control), the control unit 51 performs proactive thermal control by driving the thermal control device 202 in advance before temperature disturbance occurs (S25). If multiple thermal control devices 202 are provided, coordinated control is performed to optimize the output of each thermal control device according to the predicted temperature distribution, thereby reducing unnecessary power consumption and noise while maintaining overall image quality stability. The control unit 51 then terminates the process. The evaluation result of the evaluated object can be output to the required device.

[0051] As described above, the control unit 51 inputs the time-series first thermal information (heat originating from input image information) input to the thermal reservoir layer, which has multiple virtual nodes that hold time-series second thermal information observable at each of the multiple locations, based on the thermal characteristics of the thermal reservoir (the entire image output device) capable of heat propagation. The control unit 51 can read out an output value (predicted thermal state) which is output by linearly combining the time-series second thermal information held by the multiple virtual nodes using coupling coefficients between the multiple virtual nodes. This prevents bias in the heat distribution that could not be prevented by conventional reactive thermal control, and improves resolution (reduces image distortion).

[0052] (Note 1) Computers, An input layer that inputs first thermal information based on a time series of images input to an image output device, A thermal reservoir layer to which time-series first thermal information is input from the input layer, comprising a plurality of virtual nodes that hold time-series second thermal information observable at each of the plurality of locations as the first thermal information propagates to a plurality of locations of the image output device based on characteristics including the thermal characteristics of the image output device, A coupling layer that linearly combines the time-series second heat-related information held by a plurality of virtual nodes of the heat reservoir layer, based on the first heat-related information in time series input to the heat reservoir layer, using coupling coefficients between the plurality of virtual nodes, An output layer that outputs the second thermal information, which is linearly coupled in the coupling layer, as an output value, Based on training data including a time-series first thermal information and the correct value of the output based on the first image, when the time-series first thermal information is input to the thermal reservoir layer, a determination unit that determines the coupling coefficients so that the output value, which is linearly combined using coupling coefficients between the multiple virtual nodes to output the time-series second thermal information held by the multiple virtual nodes, approaches the correct value, functions as follows: Computer program. (Note 2) The first thermal information includes image information, The computer program described in Appendix 1. (Note 3) An acquisition unit that acquires a time-series first thermal information based on a second image different from the first image, The system includes an evaluation unit that evaluates the temperature-related state of the image output device based on the output value output by the output layer when the image output device outputs a second image, using the coupling coefficient determined by the determination unit. The computer program described in Appendix 1. (Note 4) The evaluation unit evaluates at least one of the temperature unevenness and the degree of abnormality of the image output device. The computer program described in Appendix 3. (Note 5) The system further comprises a thermal control unit that controls at least one thermal control means for thermally controlling the image output device based on the evaluation results from the evaluation unit, The computer program described in Appendix 4. (Note 6) The at least one thermal control means includes a plurality of thermal control means corresponding to the plurality of positions, The output layer outputs a plurality of output values ​​corresponding to the plurality of positions, The evaluation unit generates a plurality of evaluation results corresponding to the plurality of positions based on the plurality of output values, The thermal control unit controls the plurality of thermal control means based on the plurality of evaluation results. The computer program described in Appendix 5. (Note 7) The training data further includes time-series control information of the at least one thermal control means. The computer program described in Appendix 5. (Note 8) Based on the characteristics of the image output device, including its thermal characteristics, the first time-series thermal information based on the image input to the image output device is propagated to multiple locations of the image output device, thereby propagating to multiple virtual nodes that hold a second time-series thermal information observable at each of the multiple locations. The first time-series thermal information is input via an input layer to a thermal reservoir layer, which has multiple virtual nodes. Based on the input time-series first thermal information, the time-series second thermal information held by the multiple virtual nodes of the thermal reservoir layer is linearly combined using coupling coefficients between the multiple virtual nodes to output an output value via the output layer. Based on training data including a time-series first thermal information and the correct output value based on the first image, when the time-series first thermal information is input to the thermal reservoir layer, the coupling coefficients are determined such that the output value, which is obtained by linearly combining the time-series second thermal information held by the plurality of virtual nodes using coupling coefficients between the plurality of virtual nodes, approaches the correct value. Information processing methods. (Note 9) An input layer that inputs first thermal information based on a time series of images input to an image output device, A thermal reservoir layer to which time-series first thermal information is input from the input layer, comprising a plurality of virtual nodes that hold time-series second thermal information observable at each of the plurality of locations as the first thermal information propagates to a plurality of locations of the image output device based on characteristics including the thermal characteristics of the image output device, A coupling layer that linearly combines the time-series second heat-related information held by a plurality of virtual nodes of the heat reservoir layer, based on the first heat-related information in time series input to the heat reservoir layer, using coupling coefficients between the plurality of virtual nodes, An output layer that outputs the second thermal information, which is linearly coupled in the coupling layer, as an output value, Based on training data including a time-series first thermal information and the correct value of the output, the system includes a determination unit that determines the coupling coefficients such that when the time-series first thermal information is input to the thermal reservoir layer, the output value, which is obtained by linearly combining the time-series second thermal information held by the plurality of virtual nodes using coupling coefficients between the plurality of virtual nodes, approaches the correct value. Information processing device.

[0053] The matters described in each embodiment can be combined with each other. Furthermore, the independent and dependent claims described in the claims can be combined with each other in any combination, regardless of the form of reference. In addition, the claims use a form in which claims referencing two or more other claims (multi-claim form), but are not limited to this. A form in which multi-claims referencing at least one multi-claim (multi-multi-claim) may also be used. [Explanation of Symbols]

[0054] 10...Control Unit, 20...Interface Unit, 30...Communication Unit, 40...Storage Unit, 50...Information Processing Unit, 51...Control Unit, 52...Input / Output Unit, 53...Memory, 54...Learning Processing Unit, 55...Evaluation Unit, 56...Storage Unit, 57...Computer Program, 58...Thermal Control Unit, 60...Thermal Reservoir Model Unit, 61...Input Layer, 62...Thermal Reservoir Layer, 63...Coupling Layer, 64...Output Layer, 200...Image Output Device

Claims

1. Computers, An input layer that receives first thermal information in a time series, which includes numerical data set to correspond to the pixels that make up the image input to the image output device, A thermal reservoir layer to which time-series first thermal information is input from the input layer, and the image A thermal reservoir layer comprising a plurality of virtual nodes that hold a time-series of second thermal information observable at each of the plurality of locations, by propagating the first thermal information to a plurality of locations of the image output device based on characteristics including the thermal characteristics of the output device, A coupling layer that linearly combines the time-series second heat-related information held by a plurality of virtual nodes of the heat reservoir layer, based on the first heat-related information in time series input to the heat reservoir layer, using coupling coefficients between the plurality of virtual nodes, An output layer that outputs the second thermal information, which is linearly coupled in the coupling layer, as an output value, Based on training data including a time-series first thermal information and the correct value of the output, when the time-series first thermal information is input to the thermal reservoir layer, a determination unit that determines the coupling coefficients so that the output value, which is linearly combined using coupling coefficients between the multiple virtual nodes to output the time-series second thermal information held by the multiple virtual nodes, approaches the correct value, functions as follows: Computer program.

2. The first thermal information includes image information, The computer program according to claim 1.

3. An acquisition unit that acquires a time-series first thermal information based on a second image different from the first image, The system includes an evaluation unit that evaluates the temperature-related state of the image output device based on the output value output by the output layer when the image output device outputs a second image, using the coupling coefficient determined by the determination unit. The computer program according to claim 1.

4. The evaluation unit evaluates at least one of the temperature unevenness and abnormality of the image output device. The computer program according to claim 3.

5. The system further comprises a thermal control unit that controls at least one thermal control means for thermally controlling the image output device based on the evaluation results from the evaluation unit, The computer program according to claim 4.

6. The at least one thermal control means includes a plurality of thermal control means corresponding to the plurality of positions, and the output layer outputs a plurality of output values ​​corresponding to the plurality of positions. The evaluation unit generates a plurality of evaluation results corresponding to the plurality of positions based on the plurality of output values, The thermal control unit controls the plurality of thermal control means based on the plurality of evaluation results. The computer program according to claim 5.

7. The training data further includes time-series control information of the at least one thermal control means. The computer program according to claim 5.

8. Based on the characteristics of the image output device, including its thermal characteristics, a time-series first thermal information, which includes numerical data set corresponding to the pixels constituting the image input to the image output device, is propagated to multiple locations of the image output device, thereby inputting the time-series first thermal information to a thermal reservoir layer equipped with multiple virtual nodes that hold time-series second thermal information observable at each of the multiple locations, via an input layer. Based on the input time-series first thermal information, the time-series second thermal information held by the multiple virtual nodes of the thermal reservoir layer is linearly combined using coupling coefficients between the multiple virtual nodes to output an output value via the output layer. Based on training data including a time-series first thermal information and the correct output value based on the first image, when the time-series first thermal information is input to the thermal reservoir layer, the coupling coefficients are determined such that the output value, which is obtained by linearly combining the time-series second thermal information held by the plurality of virtual nodes using coupling coefficients between the plurality of virtual nodes, approaches the correct value. Information processing methods.

9. An input layer that receives first thermal information in a time series, which includes numerical data set to correspond to the pixels that make up the image input to the image output device, A thermal reservoir layer to which time-series first thermal information is input from the input layer, comprising a plurality of virtual nodes that hold time-series second thermal information observable at each of the plurality of locations, as the first thermal information propagates to a plurality of locations of the image output device based on characteristics including the thermal characteristics of the image output device, A coupling layer that linearly combines the time-series second heat-related information held by a plurality of virtual nodes of the heat reservoir layer, based on the first heat-related information in time series input to the heat reservoir layer, using coupling coefficients between the plurality of virtual nodes, The system comprises: an output layer that outputs the second thermal information linearly combined in the coupling layer as an output value; and a determination unit that, based on training data including the first thermal information in time series based on the first image and the correct value of the output value, determines the coupling coefficients such that when the first thermal information in time series is input to the thermal reservoir layer, the output value, which is obtained by linearly combining the second thermal information in time series held by the plurality of virtual nodes using coupling coefficients between the plurality of virtual nodes, approaches the correct value. Information processing device.

Images