Calibration system, monitoring method, and computer program

Abstract

In this monitoring method, automatic adjustment processing of a display device that displays a video is monitored. In the monitoring method, in a display system including a display device that displays a video, an imaging device that captures an image of the display state of the display device, and an adjustment device that adjusts the display state of the display device, the adjustment executed by the adjustment device is monitored in accordance with a predetermined monitoring schedule. The adjustment device includes an arithmetic circuit capable of accessing a storage device that stores adjustment history data including a history of one or more parameter values indicating the display states of the display device obtained from images captured by the imaging device in past adjustments. In the monitoring method, the success or failure of the adjustment is determined by the arithmetic circuit according to the display state of the display device, the image of which has been captured by the imaging device, if the adjustment is determined to be unauthorized, a history of past adjustments is extracted from the adjustment history data in the storage device, and the display device is adjusted using the extracted history of adjustments.

Description

Calibration system, monitoring method and computer program

[0001] The present disclosure relates to a calibration system, a monitoring method, and a computer program for calibrating and monitoring a display device that displays an image.

[0002] When a display device displays an image, the display device may undergo a calibration process. For example, the display device may be initially calibrated when it is first used. Furthermore, the display device may be recalibrated after it has been used to accommodate changes over time. However, there are cases where the process scheduled for the display device does not execute normally (see, for example, Patent Documents 1 and 2).

[0003] JP 2010-152131 A JP 2002-108657 A

[0004] However, for example, there is no way for the user to know that the calibration process has not been performed properly. Furthermore, if the calibration process is not performed properly, the display device may display an image that requires adjustment.

[0005] The present disclosure provides a calibration system, a monitoring method, and a computer program for monitoring the adjustment status of a display device that displays an image.

[0006] The monitoring method disclosed herein monitors adjustments performed by the adjustment device according to a predetermined monitoring schedule in a display system including a display device that displays video, an imaging device that captures the display state of the display device, and an adjustment device that adjusts the display state of the display device. The adjustment device includes an arithmetic circuit that can access a storage device that stores adjustment history data including a history of one or more parameter values ​​that indicate the display state of the display device obtained from images captured by the imaging device in past adjustments. The monitoring method uses the arithmetic circuit to determine whether the adjustment was successful based on the display state of the display device captured by the imaging device, and if the adjustment is determined to be incorrect, extracts a history of past adjustments from the adjustment history data in the storage device and adjusts the display device using the extracted adjustment history.

[0007] Such general and specific aspects may be realized by a system, a method, and a computer program, as well as a combination thereof.

[0008] The calibration system, calibration method, and computer program of the present disclosure can monitor the automatic adjustment process of a display device that displays an image.

[0009] 1 is a block diagram showing the configuration of a projection system and an adjustment device according to the present disclosure. FIG. 2 is a block diagram showing an example of the projection device of FIG. 1. FIG. 3 is a block diagram showing an example of the imaging device of FIG. 1. FIG. 4 is a block diagram showing an example of a user terminal of FIG. 1. FIG. 5 is a flowchart showing an example of an initial calibration process. FIG. 6 is a flowchart showing an example of a recalibration process and a monitoring process. FIG. 7 is a flowchart showing an example of a health monitoring process. FIG. 8 is a schematic diagram showing an example of data recording in an adjustment management module and a health monitoring module. FIG. 9 is a flowchart showing another example of a health monitoring process. FIG. 10 is a flowchart showing an example of a recovery process.

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings as appropriate. However, in the detailed description, unnecessary parts of the description of the prior art and substantially identical configurations may be omitted. This is for the sake of simplicity. Furthermore, the following description and the accompanying drawings are disclosed to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter of the claims.

[0011] The calibration system, monitoring method, and computer program according to the present disclosure monitor whether adjustments are being properly performed on a display device in response to environmental changes, changes over time, etc. In the following, an example will be described in which the display device is a projection device that projects an image onto a screen.

[0012] <Projection System> As shown in Fig. 1, a projection system 1 according to the present disclosure may include an adjustment device 10, a projection device 20, an image capture device 30, and a user terminal 40. The projection system 1 can project an image onto a screen 50 using the projection device 20. The adjustment device 10 executes an "initial adjustment process," a "recalibration process," or a "monitoring process" at a predetermined timing. Furthermore, when the adjustment device 10 is started, the adjustment device 10 starts executing a "life and death monitoring process." Furthermore, the adjustment device 10 executes a "recovery process" depending on the result of the life and death assessment.

[0013] <<Initial Adjustment Process>> In this specification, the initial adjustment process refers to the initial adjustment of each setting value of the projection device 20 after installation. Adjustment items in the initial adjustment include geometric distortion, color brightness, black level, focus, etc. The initial adjustment process will be described in detail later using the flowchart in FIG. 3.

[0014] <<Recalibration Process>> In this specification, the recalibration process refers to correcting or readjusting the projection device 20 to the state it was in immediately after the initial adjustment at a predetermined timing (referred to as the "first predetermined timing" as necessary) after the initial adjustment of the projection device 20. The predetermined timing for performing the recalibration process may be, for example, when the position of the projection device 20 or the color of the screen 50 is changed. Alternatively, the recalibration process may be when a position shift or color change occurs in the projected image due to aging of the internal circuitry of the projection device 20. Adjustment items for the recalibration process include geometric distortion, color brightness, black level, focus, etc. The recalibration process will be described in detail later using the flowchart of FIG. 4.

[0015] <<Monitoring Process>> In this specification, the monitoring process refers to determining whether a positional shift, color change, or the like of a projected image has occurred at a predetermined timing (referred to as a "second predetermined timing" as necessary) after the initial adjustment of the projection device 20. This monitoring process can determine whether a recalibration process is necessary. The predetermined timing for performing the monitoring process may be, for example, a periodic timing after the initial adjustment. Alternatively, the predetermined timing may be timing according to a predetermined schedule. Monitoring items of the monitoring process include geometric distortion, color brightness, black level, focus, and the like. The monitoring process will be described in detail later using the flowchart of FIG. 4. Note that in the example flowchart shown in FIG. 4, the recalibration process and the monitoring process are described in a single flowchart.

[0016] In any of the initial adjustment process, recalibration process, and monitoring process, the adjustment device 10 controls the projection device 20 to project a predetermined test pattern image onto the screen 50. The adjustment device 10 also acquires a photographed image of the screen 50 onto which the test pattern image is projected, photographed by the photographing device 30. The adjustment device 10 then uses the test pattern image captured in the photographed image to perform the initial adjustment, recalibration, or monitoring.

[0017] <<Alive / Dead Monitoring Processing>> In this specification, the term "alive / dead monitoring processing" refers to monitoring whether the recalibration processing or monitoring processing is being performed normally according to a predetermined schedule. The term "alive / dead monitoring processing" may also include monitoring whether the recalibration processing or monitoring processing is being performed normally when the recalibration processing or monitoring processing is performed from the GUI of an application installed on the adjustment device 10 connected to the projection device 20 or when the recalibration processing or monitoring processing is performed from the UI of the user terminal 40. The adjustment device 10 performs alive / dead monitoring using progress information of the recalibration processing or monitoring processing. The alive / dead monitoring processing will be described in detail later using the flowcharts of FIGS. 5 and 6. In this disclosure, the term "monitoring processing" simply refers to the processing of determining whether the above-described recalibration processing is necessary. The term "alive / dead monitoring processing" refers to monitoring whether the recalibration processing or monitoring processing is being performed normally according to a predetermined schedule.

[0018] <<Recovery Processing>> Here, when it is determined in the alive monitoring processing that the recalibration processing and / or monitoring processing is not being performed normally, the recovery processing refers to adjusting the projection by the projection device 20 to a normal state. Note that when the recalibration processing and / or monitoring processing is not being performed normally, the adjustment device 10 itself may restore the projection by the projection device 20 to normal. Alternatively, the adjustment device 10 may notify the user that recovery processing is required and have the user perform the necessary recovery processing. The recovery processing will be described in detail later using the flowchart of FIG. 7.

[0019] In the example shown in FIG. 1 , the projection system 1 is described as including one projection device 20 and one image capture device 30, but the number of projection devices 20 and image capture devices 30 included in the projection system 1 is not limited. For example, if the projection system 1 includes multiple projection devices 20, each projection device 20 may project a different image to generate a single video content. In addition, the following description will be given as an example in which the projection device 20 is used as a display device and projects an image onto a screen 50 by the projection device 20. However, the type of display device is not limited to the projection device 20. It may also be a display that displays an image.

[0020] 1, the adjustment device 10 is an information processing device including an arithmetic circuit 11, an input device 12, an output device 13, a communication circuit 14, and a storage device 15. For example, the adjustment device 10 may be a personal computer.

[0021] The arithmetic circuit 11 is a controller that controls the entire adjustment device 10. For example, the arithmetic circuit 11 can function as an image adjustment module by reading and executing an image adjustment program P1 stored in the storage device 15. Furthermore, for example, the arithmetic circuit 11 can function as an alive monitoring module by reading and executing an alive monitoring program P2 stored in the storage device 15. In this way, the arithmetic circuit 11 realizes various processes such as initial adjustment processing, recalibration processing, and monitoring processing. The arithmetic circuit 11 may be various processors such as a CPU, MPU, GPU, FPGA, DSP, or ASIC, or a dedicated hardware circuit.

[0022] The input device 12 is used for user operations and data input. The input device 12 can be, for example, an operation button, a keyboard, a mouse, a touch panel, a microphone, etc. The output device 13 is used for outputting processing results and data. The output device 13 can be, for example, output means such as a display, a speaker, etc.

[0023] The communication circuit 14 performs data communication with the projection device 20, the image capture device 30, and the user terminal 40. The data communication is performed wired and / or wirelessly, and may be in accordance with, for example, a known communication standard. For example, wired data communication may be performed by using a communication controller of a semiconductor integrated circuit that operates in accordance with the Ethernet (registered trademark) standard and / or the USB (registered trademark) standard as the communication circuit 14. Wireless data communication may be performed by using a communication controller of a semiconductor integrated circuit that operates in accordance with the IEEE 802.11 standard for wireless local area networks (LANs) and / or the fourth-, fifth-, and sixth-generation mobile communication systems, known as 4G, 5G, and 6G, for mobile communications.

[0024] The storage device 15 is a recording medium for recording various information. The storage device 15 is realized by, for example, a RAM, a ROM, a flash memory, an SSD (Solid State Drive), a hard disk drive, or other storage devices, or an appropriate combination thereof. The storage device 15 can store, for example, a video adjustment program P1, a live monitoring program P2, adjustment history data 151, adjustment backup data 152, setting backup data 153, monitoring schedule data 154, and the like.

[0025] The image adjustment program P1 executes an initial adjustment process, a recalibration process, and a monitoring process. The image adjustment program P1 executes the initial adjustment process, the recalibration process, and the monitoring process, and the arithmetic circuit 11 functions as an image adjustment module.

[0026] The alive-or-death check program P2 executes the alive-or-death check process and the recovery process. The alive-or-death check process and the recovery process are executed by the alive-or-death check program P2, so that the arithmetic circuit 11 functions as an alive-or-death check module.

[0027] The adjustment history data 151 may include data used for adjusting the projection device 20 and for various adjustments of the image capture device 30. For example, the adjustment history data 151 may include data detected from an image captured by the image capture device 30 when a test pattern image is projected onto the screen 50 by the projection device 20. Specifically, the data detected from the captured image may be coordinate data of a predetermined feature detected from the captured image or pixel value data for which color adjustment is required. The adjustment history data 151 may also include parameters required for various adjustments of the image capture device 30, such as exposure setting data of the image capture device 30 used for adjustment. The adjustment history data 151 may also include image data captured by the image capture device 30. The adjustment history data 151 is generated by the arithmetic circuit 11 in the initial adjustment process. The adjustment history data 151 is also used and updated by the arithmetic circuit 11 in the recalibration process and monitoring process. The adjustment history data 151 may also include information on whether the initial adjustment process or recalibration process was successful or failed when the data was used to perform the initial adjustment process or recalibration process. This allows the user to check the history of successful and unsuccessful cases.

[0028] The adjustment backup data 152 is the same data as the adjustment history data 151. Of the data included in the adjustment history data 151, only data that is added when the initial adjustment process or the recalibration process is successful is added to the adjustment backup data 152. The adjustment backup data 152 is generated at the start of the recalibration process and at a predetermined timing after the start of the monitoring process. The adjustment backup data 152 is also deleted at the end of the recalibration process or the monitoring process. Alternatively, the adjustment backup data 152 is deleted at the end of the recovery process after the recalibration process or the monitoring process. By referring to the adjustment backup data 152, it is possible to determine whether the automatic recovery process has been completed successfully or whether manual recovery process is required.

[0029] The setting backup data 153 can include various parameters of the projection device 20 at a predetermined timing. Specifically, the setting backup data 153 can include each parameter value used in the projection device 20 for geometric distortion correction, edge blending, black level correction, white balance correction, color correction, light source output adjustment, etc. The setting backup data 153 is generated by the arithmetic circuit 11 at the completion of the initial adjustment process based on the parameter values ​​set in the projection device 20 during the initial adjustment. The setting backup data 153 is also generated based on the parameter values ​​set in the projection device 20 at each timing of the recalibration process or the monitoring process. The number of pieces of data stored as setting backup data 153 in the storage device 15 can be determined depending on the storage capacity of the storage device 15, etc.

[0030] The monitoring schedule data 154 is data indicating the timing for executing the monitoring process. The arithmetic circuit 11 executes the monitoring process in accordance with the monitoring schedule data 154. The monitoring schedule data 154 may also include the timing for executing the recalibration process. Therefore, the arithmetic circuit 11 can execute the recalibration process in accordance with the monitoring schedule data 154.

[0031] In the example shown in FIG. 1A , the adjustment device 10 is shown as being realized by a single device, but is not limited to this. For example, the adjustment device 10 may include multiple information processing devices. Specifically, the video adjustment module and the alive monitoring module may be executed by different information processing devices. Also, for example, the adjustment device 10 may store information in an external storage device (not shown) and read and use it as needed. Specifically, the monitoring schedule data 154 may be stored in an external storage device, and the adjustment device 10 may read and use it from the external storage device.

[0032] <Projection Device> Fig. 2A is a block diagram showing the configuration of the projection device 20. As shown in Fig. 2A, the projection device 20 can include an arithmetic circuit 21, an input device 22, an output device 23, a communication circuit 24, an HDMI (registered trademark) receiving circuit 25, an HDMI (registered trademark) transmitting circuit 26, a storage device 27, an optical mechanism 28, and the like.

[0033] The arithmetic circuit 21, the input device 22, the output device 23, the communication circuit 24, and the storage device 27 can be realized by the same specific means as the arithmetic circuit 11, the input device 12, the output device 13, the communication circuit 14, and the storage device 15 described above with reference to FIG. 1.

[0034] The HDMI (registered trademark) receiving circuit 25 enables reception of data from an external device (not shown). The HDMI (registered trademark) transmitting circuit 26 enables transmission of data to an external device (not shown). The projection device 20 is capable of communicating video and audio signals with the external device using the HDMI (registered trademark) receiving circuit 25 and the HDMI (registered trademark) transmitting circuit 26. The projection device 20 also includes an optical mechanism 28 that projects an image. The optical mechanism 28 includes a light source such as a laser diode, LED, or lamp. The optical mechanism 28 also includes an optical unit that includes a liquid crystal element that modulates light emitted from the light source, a light modulation element such as a DMD (Digital Mirror Device), and a projection lens system that guides the image light modulated by the light modulation element to the projection surface.

[0035] <Photographing Device> Fig. 2B is a block diagram showing the configuration of the photographing device 30. As shown in Fig. 2B, the photographing device 30 can include an arithmetic circuit 31, an input device 32, an output device 33, a communication circuit 34, a storage device 35, an optical mechanism 36, and the like.

[0036] The arithmetic circuit 31, the input device 32, the output device 33, the communication circuit 34, and the storage device 35 can be realized by the same specific means as the arithmetic circuit 11, the input device 12, the output device 13, the communication circuit 14, and the storage device 15 described above with reference to FIG. 1.

[0037] The optical mechanism 36 includes a series of mechanisms related to capturing an image. The optical mechanism 36 acquires an image in accordance with control from the arithmetic circuit 31. Specifically, the optical mechanism 36 may include an image sensor, an optical element, an A / D converter 113, and the like.

[0038] <Configuration of User Terminal> The user terminal 40 is, for example, a terminal used by an operator who handles projection by the projection device 20 and adjustments of the projection. FIG. 2C is a block diagram showing the configuration of the user terminal 40. As shown in FIG. 2C , the user terminal 40 is an information processing device including an arithmetic circuit 41, an input device 42, an output device 43, a communication circuit 44, a storage device 45, and the like. For example, the user terminal 40 may be a personal computer, a smartphone, a tablet, or the like. The arithmetic circuit 41, the input device 42, the output device 43, the communication circuit 44, and the storage device 45 may be realized by the same specific means as the arithmetic circuit 11, the input device 12, the output device 13, the communication circuit 14, and the storage device 15 described above with reference to FIG. 1 .

[0039] The user terminal 40 can also hold a user who is located in a location remote from the installation location of the projection device 20. Therefore, the user can operate the projection device 20 even when the user is located in a location remote from the installation locations of the adjustment device 10 and the projection device 20. Furthermore, even when the user is located in a location remote from the installation location of the projection device 20, the user can grasp changes in the environment in which the projection device 20 is installed. Furthermore, when the user terminal 40 accesses the adjustment device 10 via a network, processing may be performed via a server (not shown) such as a cloud server.

[0040] 3 is a flowchart showing the initial adjustment process executed by the adjustment device 10. The initial adjustment process is executed after the projection device 20 is installed and before the projection device 20 starts to be used. For example, the initial adjustment process is started based on an operation by a user.

[0041] The arithmetic circuit 11 controls the projection device 20 to project a predetermined pattern image onto the screen 50 (S101).

[0042] The arithmetic circuit 11 acquires the photographed image photographed by the photographing device 30 and analyzes the pattern image in the photographed image (S102).

[0043] The arithmetic circuit 11 changes each parameter value of the projection device 20 so that the analysis result becomes a predetermined state (S103). In changing the parameter values, the arithmetic circuit 11 repeatedly projects the pattern image and analyzes the captured image while adjusting each parameter value of the projection device 20. Note that the case where the analysis result becomes a predetermined state can include a case where the contrast of the captured image is maximized. Note that the case where the analysis result becomes a predetermined state can also include a case where the distance between multiple feature points detected from the pattern image becomes an ideal interval. Note that the case where the analysis result becomes a predetermined state can also include a case where multiple predetermined conditions are satisfied.

[0044] When the projection device 20 is initially adjusted, the arithmetic circuit 11 generates adjustment history data 151 and stores it in the storage device 15 (S104). Specifically, the arithmetic circuit 11 generates data including image data captured by the image capturing device 30 when the analysis result reaches a predetermined state as the adjustment history data 151. The adjustment history data 151 can also include each parameter value of the image capturing device 30.

[0045] Furthermore, when the projection device 20 is initially adjusted, the arithmetic circuit 11 generates setting backup data 153 and stores it in the storage device 15 (S104). Specifically, the arithmetic circuit 11 generates, as the setting backup data 153, data including the parameter values ​​of the projection device 20 when the analysis result reaches a predetermined state.

[0046] The projection device 20 is ready for use after the initial adjustment has been performed as described above.

[0047] <Recalibration Processing and Monitoring Processing> Fig. 4 is a flowchart showing the recalibration processing and monitoring processing executed by the adjustment device 10. The processing shown in Fig. 4 is processing executed by the arithmetic circuit 11 as an adjustment module. The recalibration processing and / or monitoring processing is started at a predetermined timing after the initial adjustment. Note that the recalibration processing and the monitoring processing are partially the same processing. Therefore, as shown in Fig. 4, whether the adjustment device 10 executes the processing as a recalibration processing or a monitoring processing may be determined in an intermediate determination processing.

[0048] At a predetermined timing in the recalibration process or monitoring process, the arithmetic circuit 11 generates progress information for the recalibration process or monitoring process (S201). Here, the progress information is described as a progress rate R indicating the degree of progress of the recalibration process or monitoring process. Immediately after the start of the process, the recalibration process or monitoring process is not progressing, so the progress rate R is 0%. The generated progress information is also output from the adjustment module to the alive monitoring module.

[0049] For example, the progress rate R increases at a predetermined rate each time the processing of an item determined by the recalibration processing or the monitoring processing is completed. In the example of Fig. 4, when the adjustment backup data 152 and the setting backup data 153 are generated in step S202, the progress rate R1% and progress information are generated.

[0050] Thereafter, progress information is generated at a predetermined rate, for example, when the image capture device 30 is recognized, when the projection device 20 is recognized, when deviation detection of the image capture device 30 is completed, when exposure setting of the image capture device 30 is completed, etc. Note that in the example shown in Fig. 1, one projection device 20 and one image capture device 30 are connected to the adjustment device 10. However, the rate at which the progress rate increases varies depending on the number of connected devices 20, 30 and / or the number of items to be adjusted.

[0051] Furthermore, the arithmetic circuit 11 generates adjustment backup data 152 and setting backup data 153 (S202).

[0052] After generating each backup data 152, 153, the arithmetic circuit 11 generates new progress information (S203). The generated progress information is output from the adjustment module to the alive monitoring module. As the processing has progressed, the progress rate R is, for example, 1%. Note that, although the present disclosure will be described with reference to an example in which the progress rate R changes at 1% intervals, the present disclosure is not limited to this.

[0053] The arithmetic circuit 11 determines whether the processing being executed is a recalibration processing (S204). For example, whether the processing is a recalibration processing may be determined according to the monitoring schedule data 154 included in the storage device 15. Alternatively, whether the processing is a recalibration processing is determined by the user based on the results of periodic monitoring executed according to the monitoring schedule data 154, and specified from the user terminal 40.

[0054] When the recalibration process is being executed (YES in S204), the arithmetic circuit 11 controls the projection device 20 to project a predetermined pattern image onto the screen 50 (S206). The projection of the pattern image is performed in the same manner as in step S101 described above.

[0055] The arithmetic circuit 11 acquires the photographed image captured by the photographing device 30 and analyzes the pattern image in the photographed image (S207). The acquisition and analysis of the photographed image are performed in the same manner as in step S102 described above.

[0056] The arithmetic circuit 11 changes each parameter value of the projection device 20 so that the analysis result becomes a predetermined state (S208). The change of each parameter value is performed in the same manner as in step S103 described above.

[0057] When each parameter value is determined, the arithmetic circuit 11 generates the setting backup data 153 (S209).

[0058] The arithmetic circuit 11 also generates new progress information (S210). The generated progress information is output from the adjustment module to the alive monitoring module. Note that the processes of steps S206 to S210 are executed while the progress information is in the range of 1%<R<99%, and progress information is generated multiple times (S205).

[0059] On the other hand, when the recalibration process is not being performed (NO in S204), the arithmetic circuit 11 performs a monitoring process. Specifically, the arithmetic circuit 11 controls the projection device 20 to project a predetermined pattern image onto the screen 50 (S212). The pattern image is projected in the same manner as in step S101 described above.

[0060] The arithmetic circuit 11 acquires the photographed image captured by the photographing device 30 and analyzes the pattern image in the photographed image (S213). The acquisition and analysis of the photographed image are performed in the same manner as in step S102 described above.

[0061] The arithmetic circuit 11 determines, from the analysis results, whether or not it is necessary to change the parameter values ​​of the projection device 20 (S214). Whether or not it is necessary to change the parameter values ​​can be determined, for example, based on whether or not the pattern image in the captured image matches a predetermined condition.

[0062] When the parameter value needs to be changed (YES in S214), the arithmetic circuit 11 generates notification data indicating that the parameter value needs to be changed. The arithmetic circuit 11 also transmits the generated notification data to the user terminal 40 (S215). Although not shown in the figure, when the parameter value needs to be changed, the process may proceed to recalibration processing.

[0063] The notification data may include a level of fraud when a parameter value needs to be changed, for example, a level defined in advance for each state as follows:

[0064] Level 1: Indicates a state (incident) in which an image is definitely corrupted and an immediate response is required. Specifically, a case in which a connection error occurs in the projection device 20 while a test pattern image is being projected from the projection device 20 can be classified as Level 1. Also, a case in which the adjustment device 10 abnormally terminates the calibration process or monitoring process and the user manually performs recovery can be classified as Level 1.

[0065] Level 2: Indicates a state in which no emergency is required, but there is slight distortion in the image. Specifically, when the adjustment device 10 detects a change over time in the projection device 20 during monitoring processing (when it detects that recalibration processing is necessary), it can be set to Level 2.

[0066] Level 3: This indicates a state in which an error has occurred in the projection system 1, but the image displayed by the projection device 20 is normal. Specifically, this occurs when the adjustment device 10 detects an error during the recalibration or monitoring process and terminates the process by restoring each parameter value of the projection device 20 to the state before the process (the value of the setting backup data 153). Because the adjustment device 10 restores each parameter value of the projection device 20 to the value of the setting backup data 153, the projected image is normal. The errors referred to here include correction convergence errors, feature point detection errors, file read errors, connection errors, etc. A correction convergence error refers to a failure to converge to optimal setting values ​​during color correction or focus adjustment processes. A feature point detection error refers to a failure to detect feature points in a pattern image. A file read error refers, for example, to a failure to properly access a file containing setting information for equipment required during adjustment or adjustment data from initial adjustment. A connection error refers, for example, to a failure to properly access the image capture device 30 and the adjustment device 10.

[0067] Level 4: Indicates that the adjustment is successful. The adjustment device 10 may also notify the adjustment success when the adjustment is successful.

[0068] Although not shown in FIG. 4, if an error occurs in any of the steps of the recalibration process and the monitoring process, notification data as shown in step S215 is transmitted.

[0069] Furthermore, the arithmetic circuit 11 generates new progress information (S216). The generated progress information is output from the adjustment module to the alive monitoring module. Note that the processes of steps S212 to S216 are executed while the progress information is in the range of 1%<R<99%, and progress information is generated multiple times (S211).

[0070] When the progress information reaches a predetermined value (S210 or S216), the arithmetic circuit 11 generates the predetermined progress information (R=99% in the example of FIG. 4) (S217). Furthermore, before terminating the series of processes, the arithmetic circuit 11 deletes the adjustment backup data 152 stored in the storage device 15 (S218). Furthermore, the arithmetic circuit 11 updates the progress information (R=100% in the example of FIG. 4) and terminates the process (S219).

[0071] <Alive / Dead Monitoring Process 1> Fig. 5A is an example of alive / dead monitoring process executed by the alive / dead monitoring module of the adjustment device 10. The alive / dead monitoring process starts simultaneously with the startup of the adjustment device 10. The alive / dead monitoring process shown in Fig. 5A performs alive / dead monitoring in response to changes in progress information acquired from the adjustment module executing the recalibration process or the monitoring process. Fig. 5B also shows an example of a log of progress information recorded by the adjustment device 10 in the alive / dead monitoring process.

[0072] The arithmetic circuit 11 acquires progress information generated by the adjustment module (S401). Here, the progress information will be described as a progress rate R indicating the degree of progress of the recalibration process or the monitoring process. The arithmetic circuit 11 stores the acquired progress information as a log in the storage device 15. The arithmetic circuit 11 also starts timing at the timing when the progress information is first acquired. In this way, the arithmetic circuit 11 measures the time from the start of the recalibration process or the start of the monitoring process.

[0073] The arithmetic circuit 11 determines whether the recalibration process or the monitoring process has been completed based on the progress information acquired in step S401 (S402). Specifically, when the recalibration process or the monitoring process has been completed, the progress rate R becomes 100%.

[0074] If the progress rate R is not 100% (NO in S402), the arithmetic circuit 11 determines whether or not there is a recalibration process or a monitoring process (S403).

[0075] If there is no process (YES in S403), the arithmetic circuit 11 determines that the recalibration process or the monitoring process has stopped suddenly (S404). The abrupt stopping of the recalibration process or the monitoring process means that the recalibration process or the monitoring process has stopped due to an error.

[0076] If there is a process (NO in S403), the arithmetic circuit 11 determines whether or not a predetermined time has elapsed since the start of the recalibration process or the monitoring process (S405).

[0077] If the predetermined time has not elapsed (NO in S405), the arithmetic circuit 11 returns to the process of step S401.

[0078] When the predetermined time or more has elapsed (YES in S405), the arithmetic circuit 11 determines that the recalibration process or the monitoring process has fallen into an infinite loop (S406). Note that the predetermined time is set taking into consideration the time that would normally be sufficient to complete the recalibration process or the monitoring process. In other words, if the process continues even after the time required for the recalibration process or the monitoring process has elapsed, it is considered that the recalibration process or the monitoring process has fallen into a situation where it cannot be completed.

[0079] If it is determined in step S406 that an infinite loop has occurred, the arithmetic circuit 11 forcibly terminates the recalibration process or the monitoring process (S407).

[0080] If it is determined in step S404 that the process has suddenly died, or if it is determined in step S406 that the process is an infinite loop, the arithmetic circuit 11 proceeds to recovery processing (S5). The recovery processing will be described later with reference to FIG.

[0081] <Alive / Dead Monitoring Process 2> Fig. 6 shows another example of alive / dead monitoring process executed by the alive / dead monitoring module of the adjustment device 10. The alive / dead monitoring process shown in Fig. 6 executes alive / dead monitoring depending on whether a recalibration process or a monitoring process is being executed according to a preset monitoring process schedule.

[0082] The arithmetic circuit 11 reads out the monitoring schedule data 154 from the storage device 15 (S411).

[0083] The arithmetic circuit 11 determines whether or not progress information has been acquired in accordance with the monitoring schedule data 154 read in step S411 (S412).

[0084] When the progress information is not acquired in accordance with the monitoring schedule data 154 (NO in S412), the arithmetic circuit 11 determines that a schedule error has occurred in the monitoring process (S413).

[0085] If a schedule error is determined in step S413, the arithmetic circuit 11 generates notification data indicating the occurrence of a schedule error in the monitoring process, and transmits the generated notification data to the user terminal 40 (S414).

[0086] Note that the alive-or-dead monitoring process 1 shown in FIG. 5A and the alive-or-dead monitoring process 2 shown in FIG. 6 can be executed in parallel at the same time.

[0087] <Recovery Processing> Fig. 7 shows an example of recovery processing executed when it is determined that the recalibration processing or the monitoring processing has suddenly failed by the alive monitoring module of the adjustment device 10. The alive monitoring processing shown in Fig. 7 is a detailed example of the processing of step S5 in the flowchart shown in Fig. 5A.

[0088] The arithmetic circuit 11 determines whether the progress rate R is 0%≦R<1% (S501).

[0089] When 0%≦R<1% (YES in S501), the arithmetic circuit 11 determines that recovery is unnecessary (S502). In this case, the degree of progress is not large, and essentially, recalibration processing or monitoring processing has not been performed. Therefore, it can be determined that each parameter of the projection device 20 and the image capture device 30 has been adjusted in the initial calibration processing or the previous recalibration processing, etc.

[0090] If 0%≦R<1% is not satisfied (NO in S501), the arithmetic circuit 11 determines whether the progress rate R is 1%≦R<99% (S503).

[0091] If 1%≦R<99% (YES in S503), the arithmetic circuit 11 determines that recovery is necessary (S504). In this case, a recalibration process or a monitoring process is being executed. Therefore, the parameters of the projection device 20 and the image capture device 30 may have changed from the state adjusted by the initial calibration process or the previous recalibration process. Therefore, it is considered that the parameters of the projection device 20 and the image capture device 30 may not have been adjusted.

[0092] The arithmetic circuit 11 also executes recovery processing (S505). Specifically, the arithmetic circuit 11 recovers the parameter values ​​of the projection device 20 by using the setting backup data 153 stored in the storage device 15 when the recalibration processing was successfully executed. The arithmetic circuit 11 also recovers the adjustment history data 151 by using the adjustment backup data 152 stored in the storage device 15 when the recalibration processing was successfully executed.

[0093] If 1%≦R<99% is not true (NO in S503), the arithmetic circuit 11 determines that recovery is unnecessary (S506). In this case, 99%≦R<100%, and it can be determined that the parameters of the projection device 20 and the image capture device 30 have been adjusted. Therefore, it is determined that the recalibration process or monitoring process is almost complete, and the recovery process is terminated.

[0094] When the processing of steps S501 to S506 is completed, in either case, the recalibration processing or the monitoring processing has been improperly terminated, so the arithmetic circuit 11 generates notification data indicating the improper termination, and transmits the generated notification data (S507).

[0095] The recovery process shown in FIG. 7 is also executed when the adjustment device 10 is started up. For example, suppose that the power to the adjustment device 10 is turned off while the recalibration process or the monitoring process is being executed. In such a case, since the power to the adjustment device 10 is turned off, notification data indicating the occurrence of an error is not sent from the adjustment device 10 to the user terminal 40. Furthermore, the recovery process is not executed. Therefore, in consideration of the fact that the power to the adjustment device 10 was turned off while the recalibration process or the monitoring process was being executed, the adjustment device 10 executes the recovery process shown in FIG. 7 when it is started up. Furthermore, if the record of the progress information generated by the recovery process is not 100% when the adjustment device 10 is started up, the arithmetic circuit 11 can execute the recovery process.

[0096] Overview of the Embodiments (1) A monitoring method of the present disclosure is a monitoring method for monitoring adjustments performed by an adjustment device in accordance with a predetermined monitoring schedule in a display system including a display device that displays an image, an imaging device that captures an image of the display state of the display device, and an adjustment device that adjusts the display state of the display device, wherein the adjustment device includes an arithmetic circuit that can access a storage device that stores adjustment history data including a history of one or more parameter values ​​that indicate a display state obtained from an image captured by the imaging device in a past adjustment, and the arithmetic circuit determines whether the adjustment according to the monitoring schedule was successful or not depending on the display state of the display device, and if the adjustment is determined to be fraudulent, extracts a history of past adjustments from the adjustment history data in the storage device, and adjusts the display device using the extracted adjustment history.

[0097] This makes it possible to monitor whether the adjustment of the display device is being performed in accordance with the monitoring schedule.

[0098] (2) In the monitoring method of (1), the adjustment history data may be parameter values ​​that indicate the state of the image displayed on the display device.

[0099] This allows the video to be adjusted according to the parameter value indicating the video state.

[0100] (3) In the monitoring method of (1) or (2), the arithmetic circuit may be accessible to a terminal of a user using the display device via a network and / or a server, and when the arithmetic circuit determines that the adjustment is fraudulent, it may send a notification that the adjustment is fraudulent to the terminal.

[0101] This allows the user to grasp the adjustment status of the display device even when the user is in a remote location from the display device.

[0102] (4) In the monitoring method described in any one of (1) to (3), the arithmetic circuit may be accessible to a terminal of a user who uses the display device, and when the arithmetic circuit determines that the adjustment is invalid, the captured image may be sent to the terminal together with the notification.

[0103] This allows the user to grasp the adjustment status of the display device by using the captured image of the display video, even when the user is in a location remote from the display device.

[0104] (5) In the monitoring method described in any one of (1) to (4), the arithmetic circuit may, upon determining whether the adjustment was successful, determine the level of the fraud, and transmit a notification including the determined level of the fraud to the terminal.

[0105] This allows the user to grasp the adjustment status of the display device from the level even when the user is in a remote location from the display device, making it easier to determine the degree of urgency of the required response.

[0106] (6) In the monitoring method described in any one of (1) to (5), the adjustment managed by the monitoring schedule may include at least one of a recalibration process performed at a first predetermined timing after the initial adjustment and a monitoring process for determining whether a recalibration process performed at a second predetermined timing after the initial adjustment is necessary.

[0107] In this way, it can be determined whether the recalibration process and the monitoring process are both being performed according to schedule.

[0108] (7) In the monitoring method described in any one of (1) to (6), the arithmetic circuit generates one or more parameter values ​​indicating the display state of the display device from the image captured by the imaging device at multiple predetermined times set from the start to the end of adjustment execution, and stores each of the one or more parameter values ​​in the storage device as adjustment backup data; deletes the adjustment backup data from the storage device at the time when the adjustment is completed or the process of recovering the display device and / or adjustment history data is completed; and may notify the terminal of notification data that the process of recovering the display device has not completed normally when the adjustment backup data is stored in the storage device at the time when a new adjustment is started.

[0109] This allows the user to grasp the recovery status of the display device in addition to the adjustment status of the display device.

[0110] (8) In the monitoring method described in any one of (1) to (7), the arithmetic circuit may acquire one or more parameter values ​​indicating the state of the display device at multiple predetermined timings set from the start to the end of adjustment execution, store each of the one or more parameter values ​​as setting backup data in the storage device, and provide the setting backup data in response to a request.

[0111] This allows the user to use the setting backup data when restoring the display device, thereby facilitating the restoration process.

[0112] (9) An adjustment device of the present disclosure is connected to a display device that displays an image and a photographing device that photographs the display state of the display device, and adjusts the display state of the display device, the adjustment device including an arithmetic circuit that can access a storage device, the storage device storing data indicating a predetermined monitoring schedule for adjusting the display state of the display device, and adjustment history data including a history of one or more parameter values ​​that indicate the display state obtained from an image photographed by the photographing device in a past adjustment, the arithmetic circuit determining whether the adjustment was successful or not depending on the display state of the display device, and if the adjustment is determined to be invalid, extracting a history of past adjustments from the adjustment history data in the storage device, and adjusting the display device using the extracted adjustment history.

[0113] This makes it possible to monitor whether the adjustment of the display device is being performed in accordance with the monitoring schedule.

[0114] (10) A computer program according to the present disclosure causes a computer to execute the monitoring method according to any one of (1) to (8).

[0115] This makes it possible to monitor whether the adjustment of the display device is being performed in accordance with the monitoring schedule.

[0116] The calibration system, calibration method and computer program according to all claims of the present disclosure are realized by hardware resources, such as a processor, a memory, and cooperation with a computer program.

[0117] The calibration system, monitoring method, and computer program of the present disclosure are useful for monitoring the adjustment status of a display device that displays an image.

Claims

1. In a display system including a display device that displays an image, a photographing device that photographs the display state of the display device, and an adjustment device that adjusts the display state by the display device, a monitoring method for monitoring the adjustment executed by the adjustment device according to a predetermined monitoring schedule, wherein the adjustment device includes an arithmetic circuit accessible to a storage device that stores adjustment history data including a history of one or more parameter values indicating the display state of the display device obtained from a photographed image of the photographing device in past adjustments, and by the arithmetic circuit, determining whether the adjustment is successful or not according to the display state of the display device photographed by the photographing device, and when it is determined that the adjustment is unauthorized, extracting the history of past adjustments from the adjustment history data of the storage device and using the extracted history of the adjustment to adjust the display device. Monitoring method.

2. The monitoring method according to claim 1, wherein the adjustment history data includes, as the history, parameter values indicating the state of the video displayed on the display device.

3. The arithmetic circuit is accessible to a terminal of a user who uses the display device via a network and / or a server, and when it is determined by the arithmetic circuit that the adjustment is unauthorized, the arithmetic circuit transmits a notification that the adjustment is unauthorized to the terminal. Monitoring method according to claim 1.

4. The arithmetic circuit is accessible to a terminal of a user who uses the display device, and when it is determined by the arithmetic circuit that the adjustment is unauthorized, the arithmetic circuit transmits the photographed image to the terminal together with the notification. Monitoring method according to claim 3.

5. When determining whether the adjustment is successful or not, the arithmetic circuit determines the level of the unauthorizedness and transmits a notification including the determined level of the unauthorizedness to the terminal. Monitoring method according to claim 4.

6. The adjustment managed by the monitoring schedule includes at least one of a recalibration process executed at a first predetermined timing after the initial adjustment and a monitoring process for determining whether a recalibration process is required at a second predetermined timing after the initial adjustment. Monitoring method according to claim 1.

7. The arithmetic circuit generates, at a plurality of predetermined timings set from the start to the end of the execution of the adjustment, one or more parameter values indicating the display state of the display device from the captured image of the imaging device, stores each one or more parameter values in the storage device as adjustment backup data, deletes the adjustment backup data from the storage device at the timing when the adjustment ends or at the timing when the process of restoring the display device and / or the adjustment history data ends, and when the adjustment backup data is stored in the storage device at the timing of starting a new adjustment, notifies the terminal of notification data indicating that the process of restoring the display device has not ended normally. The monitoring method according to claim 3.

8. The arithmetic circuit acquires, at a plurality of predetermined timings set from the start to the end of the execution of the adjustment, one or more parameter values indicating the state of the display device, stores each one or more parameter values in the storage device as setting backup data, and provides the setting backup data in response to a request. The monitoring method according to claim 1.

9. An adjustment device that is connected to a display device that displays video and an imaging device that captures the display state of the display device, and adjusts the display state of the display device. The adjustment device includes an arithmetic circuit that can access a storage device. The storage device stores data indicating a predetermined monitoring schedule for adjusting the display state of the display device, and adjustment history data including a history of one or more parameter values indicating the display state obtained from the captured image of the imaging device in past adjustments. The arithmetic circuit determines whether the adjustment is successful according to the display state of the display device, extracts the history of past adjustments from the adjustment history data of the storage device when the adjustment is determined to be incorrect, and adjusts the display device using the extracted history of the adjustment. An adjustment device.

10. A computer program for causing a computer to execute the monitoring method according to any one of claims 1 to 8.

Images