Notification method and notification system

Abstract

This notification method includes: a first acquisition step for acquiring a first image captured at a first timing, the first image including a display area for video displayed by a projection display device; a second acquisition step for acquiring a second image captured at a second timing, the second image including the display area; a comparison step for comparing first analysis data based on the first image and second analysis data based on the second image; a first assessment step for assessing, on the basis of the result of comparison, whether or not first misalignment that represents misalignment of display parameters relating to display has occurred and whether or not second misalignment that represents misalignment of image-capture parameters relating to image-capture has occurred; and a notification step for notifying a user of misalignment information indicating the first misalignment or the second misalignment that is assessed to have occurred.

Description

Notification method and notification system

[0001] The present disclosure relates to a notification method for a projection display system.

[0002] Japanese Patent Application Laid-Open No. 2009-102663 discloses a method for identifying misalignment of an image displayed by a display device by using an image obtained by capturing the image displayed by the display device with an installed camera through a recalibration process of the image projection device. Japanese Patent Application Laid-Open No. 2009-102663 also discloses notifying the user that misalignment has occurred. This can prompt the user to perform maintenance to correct the misalignment.

[0003] Japanese Patent Application Laid-Open No. 2018-207373

[0004] However, if a camera that captures images for detecting misalignment of the video becomes misaligned, it will be impossible to correctly detect the misalignment of the video, so it is necessary to detect the misalignment of the camera. In this case, too, it is conceivable to notify the user to perform camera maintenance.

[0005] The present disclosure provides a notification system that can notify whether a camera or a display device needs maintenance.

[0006] A notification method according to one aspect of the present disclosure includes a first acquisition step of acquiring a first image captured by a camera at a first timing, the first image including a display area of ​​an image to be displayed by a display device; a second acquisition step of acquiring a second image captured by the camera at a second timing a predetermined period after the first timing, the second image including the display area; a comparison step of comparing first analysis data based on the first image with second analysis data based on the second image; a first determination step of determining, based on the comparison result in the comparison step, whether a first deviation has occurred, which is a deviation in a display parameter related to display by the display device, and whether a second deviation has occurred, which is a deviation in a shooting parameter related to shooting by the camera; and a notification step of notifying a user of deviation information indicating which of the first deviation and the second deviation has been determined to have occurred in the first determination step.

[0007] According to another aspect of the present disclosure, there is provided a notification system including a display device, a camera, and an information processing device, wherein the information processing device executes a notification method including: a first acquisition step of acquiring a first image captured by the camera at a first timing, the first image including a display area of ​​an image to be displayed by the display device; a second acquisition step of acquiring a second image captured by the camera at a second timing a predetermined time after the first timing, the second image including the display area; a comparison step of comparing first analysis data based on the first image with second analysis data based on the second image; a first determination step of determining, based on a comparison result in the comparison step, whether a first deviation has occurred, which is a deviation in a display parameter related to display by the display device, and whether a second deviation has occurred, which is a deviation in a shooting parameter related to shooting by the camera; and a notification step of notifying a user of deviation information indicating which of the first deviation and the second deviation has occurred, which is determined to have occurred in the first determination step.

[0008] These general or specific aspects may be realized by an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized by any combination of an apparatus, a system, a method, an integrated circuit, a computer program, and a recording medium.

[0009] A notification method according to an aspect of the present disclosure can notify which of the camera and the display device needs maintenance.

[0010] FIG. 1 is a diagram showing the configuration of a projection display system according to an embodiment. FIG. 2 is a diagram showing an example of a flowchart of the operation of the projection display system. FIG. 3 is a flowchart showing an example of an initial calibration process. FIG. 4 is a flowchart showing an example of a feature point detection process. FIG. 5 is a flowchart showing an example of a calibration process. FIG. 6 is a flowchart showing an example of a comparison process. FIG. 7 is a flowchart showing an example of a determination process. FIG. 8 is a diagram showing an example of a first amount of deviation generated in a projection display device. FIG. 9 is a diagram showing an example of a second amount of deviation generated in a camera. FIG. 10 is a flowchart showing an example of a correction limit notification process according to Modification 1. FIG. 11 is a diagram for explaining the correction limit. FIG. 12 is a diagram showing the configuration of a projection display system according to Modification 2. FIG. 13 is a diagram showing the configuration of a projection display system according to Modification 3.

[0011] A notification method according to one aspect of the present disclosure includes a first acquisition step of acquiring a first image including a display area of ​​an image to be displayed by a display device, the first image being captured by a camera at a first timing; a second acquisition step of acquiring a second image including the display area, the second image being captured by the camera at a second timing a predetermined period after the first timing; a comparison step of comparing first analysis data based on the first image with second analysis data based on the second image; a first determination step of determining, based on the comparison result in the comparison step, whether a first deviation has occurred, which is a deviation in a display parameter related to display by the display device, and whether a second deviation has occurred, which is a deviation in a shooting parameter related to shooting by the camera; and a notification step of notifying a user of deviation information indicating which of the first deviation and the second deviation has been determined to have occurred in the first determination step.

[0012] In this way, the deviation information notified to the user includes the deviation that occurred between the first deviation and the second deviation, and therefore the user can be notified of the device having the parameter that is the source of the deviation and the deviation itself, and can be notified of which of the camera and the display device needs maintenance.

[0013] Furthermore, the method may further include a changing step of accepting an adjustment of the display parameter in the display device and changing the display parameter in accordance with the accepted adjustment.

[0014] In addition, the change in the display parameter is made by an adjustment value within a predetermined adjustment range, and the notification method further includes a second determination step of determining whether the adjustment value exceeds a predetermined adjustment value, and in the notification step, if the adjustment value exceeds the predetermined adjustment value in the second determination step, the user may be further notified that the limit of the adjustment is approaching.

[0015] This makes it possible to notify the user that the limit of adjustment is approaching, and therefore that the time when maintenance will be required is approaching.

[0016] The display device may be a projector that projects an image onto the display area, and the display parameters may include a position of the image projected by the display device relative to the display area.

[0017] This makes it possible to notify the user that the limit of adjustment of the image projection position is approaching, and therefore the user can be notified that the time for maintenance is approaching.

[0018] The display device may be a projector that projects an image onto the display area, and the display parameters may include a focus state of the image projected by the display device.

[0019] This makes it possible to notify the user that the limit of image focus adjustment is approaching, and therefore that the time when maintenance will be required is approaching.

[0020] The display parameters may include the color of an image displayed by the display device.

[0021] This makes it possible to notify the user that the color adjustment limit for the image is approaching, and therefore to notify the user that the time for maintenance is approaching.

[0022] The display parameters may include brightness of an image displayed by the display device.

[0023] This makes it possible to notify the user that the image brightness adjustment limit is approaching, and therefore to notify the user that the time for maintenance is approaching.

[0024] In addition, the notification method may be executed by a control device connected to the camera and the display device via a local network, and the notification method may further include a storage step of storing the deviation information notified in the notification step and at least one of the first image, the second image, the first analysis data, the second analysis data, and the comparison result in different storage.

[0025] Therefore, the deviation information and at least one of the first image, the second image, and the comparison result can be managed separately. For example, the security levels of the two can be managed separately. Furthermore, for example, they can be managed separately based on whether they are to be used as data to be communicated.

[0026] In addition, in the storage step, (i) the deviation information may be stored in a first storage device connected to the camera and the display device via an external network, and (ii) at least one of the first image, the second image, and the comparison result may be stored in a second storage device connected to the camera and the display device via a local network.

[0027] This reduces the communication load on the network that is required to transmit the deviation information.

[0028] Furthermore, a notification system according to one aspect of the present disclosure includes a display device, a camera, and an information processing device, and the information processing device executes a notification method including: a first acquisition step of acquiring a first image captured by the camera at a first timing, the first image including a display area of ​​an image to be displayed by the display device; a second acquisition step of acquiring a second image captured by the camera at a second timing a predetermined period after the first timing, the second image including the display area; a comparison step of comparing first analysis data based on the first image with second analysis data based on the second image; a first determination step of determining, based on a comparison result in the comparison step, whether a first deviation has occurred, which is a deviation in a display parameter related to display by the display device, and whether a second deviation has occurred, which is a deviation in a shooting parameter related to shooting by the camera; and a notification step of notifying a user of deviation information indicating which of the first deviation and the second deviation has occurred, which is determined to have occurred in the first determination step.

[0029] In this way, the deviation information notified to the user includes the deviation that occurred between the first deviation and the second deviation, and therefore the user can be notified of the device having the parameter that is the source of the deviation and the deviation itself, and can be notified of which of the camera and the display device needs maintenance.

[0030] These general or specific aspects may be realized by an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized by any combination of an apparatus, a system, a method, an integrated circuit, a computer program, and a recording medium.

[0031] Hereinafter, embodiments will be described with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection forms, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not described in independent claims will be described as optional components.

[0032] It should be noted that the drawings are schematic diagrams and are not necessarily strict illustrations. In addition, in the drawings, substantially the same components are denoted by the same reference numerals, and overlapping descriptions may be omitted or simplified.

[0033] (Embodiment) [Configuration] First, the configuration of a projection display system according to an embodiment will be described. Fig. 1 is a diagram showing the configuration of a projection display system according to an embodiment.

[0034] The projection display system 10 is a system that supports multi-projection, in which a single image is constructed using images projected by two projection display devices 20. The projection display system 10 is an example of a notification system. Furthermore, the projection display system 10 performs a calibration process using images captured by a camera 30 to project an image aligned with a screen 70 (along the area inside the frame 70a of the screen 70). The projection display system 10 includes two projection display devices 20, a camera 30, and an information processing device 40. The projection display system 10 may further include a server 51 and information terminals 52 and 53. The projection display system 10 is only required to include at least one projection display device 20; multi-projection support is not essential. The projection display device 20 may also include multiple cameras 30.

[0035] The projection display device 20 projects an image onto the screen 70 under the control of the information processing device 40. The projection display device 20 is realized by an optical system including, for example, a laser light source, a phosphor wheel, an image display element, and a projection lens. Specific examples of the image display element include a digital micromirror device (DMD) or a reflective liquid crystal panel (LCOS: Liquid Crystal On Silicon). The projection display device 20 is an example of a display device.

[0036] When multi-projection is achieved using two projection display devices 20, a technique called edge blending is used, for example. The two images projected by the two projection display devices 20 have an overlapping portion 70b where they overlap each other, and edge blending projects the images so that the brightness of the two images cross-fade in the overlapping portion 70b. This prevents the seam between the two images from being noticeable.

[0037] The camera 30 captures an image showing the entire screen 70 under the control of the information processing device 40. The screen 70 is an example of an area (i.e., a display area) onto which an image is projected by the projection display device 20. The camera 30 is realized by an image sensor, a lens, and the like.

[0038] The information processing device 40 performs information processing to realize the above-described multi-projection. Such information processing includes a calibration process using images captured by the camera 30 to project images aligned with the screen 70. The information processing device 40 is, for example, a general-purpose device such as a personal computer on which an application program for executing the above-described information processing is installed, but it may also be a device dedicated to the projection display system 10. Specifically, the information processing device 40 includes an input device 41, a monitor 42, a communication circuit 43, a control circuit 44, and storage devices 45 and 46. As shown in FIG. 1 , the information processing device 40 (an example of a control device) is connected to the camera 30 and two projection display devices 20 via a local network.

[0039] The input device 41 receives operations from a user. The input device 41 is, for example, a keyboard and a mouse, but may also be a touch panel or the like.

[0040] The monitor 42 displays an image. The monitor 42 is realized by a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel. The monitor 42 may be a device separate from the information processing device 40.

[0041] The communication circuit 43 is a communication circuit that enables the information processing device 40 to communicate with the two projection display devices 20 and the camera 30. The communication circuit 43 communicates with the two projection display devices 20 and the camera 30, for example, via a local communication network. The communication performed by the communication circuit 43 is, for example, wired communication, but may also be wireless communication. There are no particular limitations on the communication standard used for communication.

[0042] The control circuit 44 performs the above-described information processing. Specifically, the control circuit 44 is realized by a processor or a microcomputer. The functions of the control circuit 44 are realized when the processor or the microcomputer constituting the control circuit 44 executes a computer program stored in the storage device 45.

[0043] The storage devices 45 and 46 are storage devices that store information necessary for the above-mentioned information processing, such as the computer program executed by the control circuit 44. Specifically, the storage devices 45 and 46 are realized by semiconductor memories or HDDs (Hard Disk Drives). The storage devices 45 and 46 do not have to be realized by different storage devices, but may be realized by different storage areas of the same storage device. The storage device 45 may be accessible only to specific users (e.g., a servicer, as described below), while the storage device 46 may be accessible only to users including the specific user and other users. Note that the storage device 46 does not necessarily have to be accessible only to a limited number of users.

[0044] The server 51 is connected to the information processing device 40 via a network 60. The server 51 may also be connected to an information processing device (not shown) of another projection display system via the network 60. The server 51 may receive and store various data generated in the projection display system 10 via the network 60, or may receive and store processing results in the information processing device 40 via the network 60. The network 60 may be a general-purpose communication line such as the Internet, or may be a dedicated communication line.

[0045] The server 51 may store the various accumulated data in a state that prevents it from being viewed from the information terminal 52, and store the processing results of the various data in a state that allows it to be viewed from the information terminal 52. Here, the various data includes the first image and second image generated by the projection display system 10. The first image and second image are images captured by the camera 30. The processing results include first analysis data, second analysis data, deviation information, and the like. The first analysis data, second analysis data, and deviation information are data generated by processing in the information processing device 40. Details of the various data will be described later.

[0046] The information terminal 52 is a terminal used by a user of the projection display system 10. The information terminal 52 is connected to the information processing device 40 via the network 60. The information terminal 52 may accept operations by the user to remotely control the information processing device 40. The information terminal 52 may transmit a control signal for remote control to the information processing device 40 and receive a processing result of a process executed in the information processing device 40 in response to the control signal. The information terminal 52 is, for example, a computer such as a PC (Personal Computer), a smartphone, or a tablet.

[0047] The information terminal 53 is a terminal used by a servicer that provides the projection display system 10 to users. The information terminal 53 is connected to the information processing device 40 via the network 60. The information terminal 53 may accept operations by the servicer and remotely operate the information processing device 40. The information terminal 53 may transmit a control signal for remote operation to the information processing device 40 and receive a processing result of a process executed in the information processing device 40 in response to the control signal, or may transmit the processing result to the server 51 and store the processing result in the server 51.

[0048] The information terminal 53 may also transmit a control signal for remote control to the server 51, and process various data generated in the projection display system 10 and stored in the server 51 in response to the control signal. The information terminal 53 may also set the results of processing the various data in the server 51 so that they can be viewed from the information terminal 52. The information terminal 53 is, for example, a computer such as a PC (Personal Computer), a smartphone, or a tablet.

[0049] [Operation] Next, a description will be given of the operation of the projection display system 10. Fig. 2 is a diagram showing an example of a flowchart of the operation of the projection display system 10.

[0050] First, the control circuit 44 executes an initial calibration process (S11). In the initial calibration process, the control circuit 44 acquires a first image from the camera 30 and generates first analysis data based on the first image. The first image is an image captured by the camera 30 at a first timing, and is an image that includes the display area (screen 70 in this embodiment) of the image displayed by the projection display device 20. Details of the initial calibration will be described later using FIG. 3.

[0051] Next, control circuit 44 executes a calibration process (S12). In the calibration process, control circuit 44 acquires a second image from camera 30 and generates second analysis data based on the second image. The second image is an image captured by camera 30 at a second timing that is a predetermined time after the first timing, and includes the display area of ​​projection display device 20. Details of the calibration process will be described later using FIG. 5.

[0052] Next, the control circuit 44 compares the initial calibration result with the calibration result (S13).The control circuit 44 compares the first analysis data with the second analysis data.

[0053] Next, the control circuit 44 determines whether a deviation has occurred (S14). Based on the comparison result of step S13, the control circuit 44 determines whether a first deviation, which is a deviation in the display parameters related to the display by the projection display device 20, has occurred, and whether a second deviation, which is a deviation in the shooting parameters related to the shooting by the camera 30, has occurred.

[0054] The display parameters include parameters relating to the position of the image projected by the projection display device 20, the focus state of the projected image, the color of the projected (displayed) image, and the brightness of the projected (displayed) image.

[0055] The shooting parameters include the position, orientation, and focal length of the camera 30. The shooting parameters are also called camera parameters.

[0056] Next, the control circuit 44 notifies the user of the deviation information (S15). The control circuit 44 notifies the user by, for example, transmitting the deviation information to the information terminal 52 using the communication circuit 43. The information terminal 52, which has received the deviation information, displays the deviation information (e.g., the graphs shown in FIGS. 8 and 9 , described below). The deviation information indicates the deviation determined to have occurred in step S14. For example, if it is determined that a first deviation has occurred in step S14, the deviation information includes the first deviation of the display parameters determined by the projection display device 20. If it is not determined that the first deviation has occurred in step S14, the deviation information does not include the first deviation. Furthermore, for example, if it is determined that a second deviation has occurred in step S14, the deviation information includes the second deviation of the shooting parameters determined by the camera 30. If it is not determined that the second deviation has occurred in step S14, the deviation information does not include the second deviation. In other words, depending on the judgment result of step S14, the deviation information may include only the first deviation, only the second deviation, or both the first deviation and the second deviation.

[0057] The first deviation occurs when a deviation (difference) of a first threshold or more occurs in the display parameters in the projection display device 20. The second deviation occurs when a deviation (difference) of a second threshold or more occurs in the shooting parameters in the camera 30.

[0058] [Initial Calibration Processing] Next, details of the initial calibration processing (S11) of the projection display system 10 will be described. The initial calibration processing is processing that is performed when the projection display system 10 starts operating. Fig. 3 is a flowchart showing an example of the initial calibration processing.

[0059] First, the control circuit 44 sets the projection positions of the images of the two projection display devices 20 so that they are aligned with the screen 70 (S21). Specifically, the control circuit 44 transmits control signals to each of the two projection display devices 20 via the communication circuit 43, thereby causing each of the two projection display devices 20 to project an image.

[0060] At this time, the control circuit 44 adjusts the projection direction, lens shift amount, zoom magnification, focal length, etc. of each of the two projection display devices 20 so that the edges of the projection ranges of the two images projected by the two projection display devices 20 are aligned with the frame 70a of the screen 70.

[0061] The following describes the process of storing the projection position of the test image for one projection display device 20, but this process is performed for each of the two projection display devices 20.

[0062] The control circuit 44 transmits a control signal to the projection display device 20 via the communication circuit 43, causing the projection display device 20 to project a test image (S22). The test image may be any image suitable for calculating the analysis value in the next step S23, such as an image having a predetermined color pattern. Here, the analysis value refers to the evaluation value of the display parameters of the projection display device 20. The analysis value includes the coordinates of feature points in the captured image, color information of specific areas in the captured image, and focus information that indicates the focus state of the test image in the captured image.

[0063] Next, the control circuit 44 calculates an analysis value for the captured image of the test image projected on the screen 70, and stores first analysis data indicating the calculated analysis value in the storage device 45 (S23). The calculation of the analysis value in step S23 is a process for calculating evaluation values ​​of the display parameters of the projection display device 20. Specifically, the coordinates of the feature points as analysis values ​​are used to identify the projection position of the image. In other words, they are used to identify the relative position between the projection display device 20 and the screen 70. Furthermore, the color information as analysis values ​​is used to identify the color of the color pattern projected by the projection display device 20. Furthermore, the focus information as analysis values ​​is used to identify the focus state of the test image projected by the projection display device 20. The first analysis data is calculated based on the image captured at the first timing, and therefore includes the first display parameters, which are display parameters at the first timing.

[0064] Specifically, the control circuit 44 transmits a control signal to the camera 30 via the communication circuit 43, causing the camera 30 to capture an image (still image) including the screen 70 with the test image projected thereon. The control circuit 44 acquires the captured image (more specifically, image information of the captured image) from the camera 30 via the communication circuit 43 and processes the acquired image to detect multiple feature points, color information, and a focus state. At this time, the control circuit 44 uses, for example, an algorithm optimized for detecting feature points appearing in the test image. Similarly, the control circuit 44 uses, for example, an algorithm optimized for detecting color information of a color pattern in the test image. Similarly, the control circuit 44 uses, for example, an algorithm optimized for detecting a focus state in the test image.

[0065] For example, in detecting feature points, intersections of areas of different colors are detected as feature points in a test image. Hereinafter, the coordinates of the detected N feature points are collectively expressed as FP(N) = {FP1, FP2, FP3, ..., FPn, ..., FPN}. 1, 2, 3, ..., N are identification information (ID) of the feature points, and FP1, FP2, FP3, ... are the coordinates of each feature point. FP(N) is stored in the storage device 45.

[0066] In detecting color information, color information of a specific color area in the test image is detected. The detected color information is stored in the storage device 45.

[0067] In the detection of focus information, the focus state in the test image is detected. The detected focus information is stored in the storage device 45.

[0068] Furthermore, the control circuit 44 uses the first analysis data stored in step S23 to perform geometric correction on each of the two images projected by the two projection display devices 20, as necessary. The control circuit 44 also uses the first analysis data stored in step S23 to perform edge blending processing on the overlapping portion 70b of the two images projected by the two projection display devices 20. The control circuit 44 automatically executes this series of processes using the images captured by the camera 30 (more specifically, image information obtained from the camera 30 via the communication circuit 43) (S24).

[0069] Immediately after step S24, the control circuit 44 performs a feature point detection process (S25). The feature point detection process here is a process for identifying the relative positions of the camera 30 and the screen 70. The feature point detection process will be described in detail later.

[0070] [Example 1 of Feature Point Detection Processing] A specific example of the feature point detection processing performed in step S25 will now be described. Fig. 4 is a flowchart showing an example of the feature point detection processing.

[0071] For example, upon receiving an operation from a user via the input device 41, the control circuit 44 transmits a control signal to the camera 30 via the communication circuit 43, thereby causing the camera 30 to capture an image (still image) (S31). At this time, the image capture conditions (white balance, shutter speed, ISO sensitivity, F-number, etc.) instructed by the information processing device 40 (control circuit 44) to the camera 30 by the control signal may be set to preset values. Hereinafter, the image captured in step S31 is an example of a first image.

[0072] When the first image is captured by the camera 30, the control circuit 44 acquires the first image (more specifically, image information of the first image) from the camera 30 via the communication circuit 43 and detects a plurality of feature points from the acquired first image as a processing target (S32). At this time, the control circuit 44 uses, for example, an algorithm for detecting feature points that appear at the four corners of the rectangular screen 70. Note that any existing algorithm may be used to detect the feature points, and open source or the like may be used as appropriate.

[0073] The control circuit 44 associates the first image with the coordinates of each of the detected feature points in the first image and stores the associated data in the storage device 45 (S33). The coordinates of the feature points may be assigned identification information (ID) of the feature points. The information stored in step S33 may be included in the first analysis data. The information stored in step S33 is calculated based on the first image captured at the first timing, and therefore includes first shooting parameters, which are shooting parameters at the first timing.

[0074] The first image in step S31 may include a test image projected by the projection display device 20. In other words, the process of step S32 may be performed on the first image including the test image. In other words, the image that is the subject of step S23 and the first image may be different images or may be the same image.

[0075] [Calibration Process] Next, the calibration process (S12) performed after the first timing at which the initial calibration process (S11) is performed will be described in detail. Fig. 5 is a flowchart showing an example of the calibration process.

[0076] When a predetermined second timing arrives, which is later than the first timing, the control circuit 44 uses this as an opportunity to send a control signal to the camera 30 via the communication circuit 43, thereby causing the camera 30 to capture an image (still image) (S41). For example, the control circuit 44 stores the shooting conditions during the initial calibration process in the storage device 45, and by referring to the storage device 45, causes the camera to capture an image under the same shooting conditions as when the initial calibration process was performed. Hereinafter, the image captured in step S41 is an example of a second image.

[0077] Note that the second image is captured automatically without user operation. For example, the control circuit 44 manages schedule information for the calibration process (e.g., once per predetermined period, such as once a day, once a week, or once a month) and periodically captures the second image based on the schedule information. The schedule information is stored (registered) in advance in the storage device 45 by, for example, a user operating the input device 41. Note that the capture schedule may be managed by a device higher than the information processing device 40, such as a cloud server (not shown), and the information processing device 40 may capture the second image based on a command from the higher-level device.

[0078] Furthermore, the second image may be captured based on a user operation. For example, when the user views the image projected on the screen 70 and determines that calibration is necessary, the user may perform a predetermined operation on the input device 41 to input a command to capture the second image, which then causes the second image to be captured.

[0079] When the second image is captured by the camera 30, the control circuit 44 acquires the second image (more specifically, image information of the second image) from the camera 30 via the communication circuit 43 and detects a plurality of feature points from the acquired second image as a processing target (S42). The method of detecting the plurality of feature points is the same as the method described in step S32, except that the processing target is the second image. The second image and the plurality of feature points detected in step S42 may be included in second analysis data. The second analysis data may include coordinates in the second image of each of the detected plurality of feature points. The second image, the plurality of feature points, and the coordinates of each of the plurality of feature points are calculated based on the second image captured at the second timing, and therefore include second shooting parameters that are shooting parameters at the second timing.

[0080] Next, the control circuit 44 reads out the coordinates of the first image and the plurality of feature points stored in the storage device 45 in step S33 (S43). The control circuit 44 then calculates the coordinate deviation amount at the current time (second timing) for each of the read feature points (S44). Specifically, the control circuit 44 identifies the feature amount of each of the read feature points by comparing the coordinates of each of the read feature points with the first image. For each of the plurality of feature points, the control circuit 44 determines one feature point from the plurality of feature points detected in step S42 that has a feature amount most similar to the feature amount of the feature point, and calculates the coordinate deviation amount between the coordinates of the determined feature point and the coordinates of the feature point.

[0081] The coordinate shift amount of each of the multiple feature points indicates a shift (change) in the relative position between the camera 30 and the screen 70. In other words, it can be said that step S44 detects a shift in the relative position between the camera 30 and the screen 70. In other words, the coordinate shift amount calculated in step S44 is an example of the second shift amount of the second shift.

[0082] Next, the control circuit 44 calculates a correction coefficient for the coordinate system of the camera 30 based on the coordinate shift amount of each of the plurality of feature points (S45). The control circuit 44 calculates, as a correction coefficient, a homography matrix H for converting the coordinate system of the camera 30 at the current time (second timing) into the coordinate system of the camera 30 at the time when the initial calibration process was executed (first timing) based on the coordinate shift amount of each of the plurality of feature points. The homography matrix H is expressed by the following equation.

[0083]

[0084] Here, if the coordinates of the second image (the image captured at the current time) are (x, y), the corresponding coordinates (x', y') of the first image (the image captured at the time the initial calibration process was performed) are expressed by the following equation:

[0085]

[0086] Note that a method for calculating the homography matrix H from the displacement amounts of four or more feature points in a two-axis coordinate system is well known, and therefore a detailed description thereof will be omitted here.

[0087] Next, the control circuit 44 performs a process to correct the projection position of the image for each of the two projection display devices 20. The following describes the process to correct the projection position of the image for one projection display device 20, but this process is performed for each of the two projection display devices 20.

[0088] First, the control circuit 44 causes the projection display device 20 to project a test image (S46) by transmitting a control signal to the projection display device 20 via the communication circuit 43. The test image is the same as the test image projected in step S22 of the initial calibration process.

[0089] Next, control circuit 44 calculates analysis values ​​for the image obtained by capturing the test image projected on screen 70, and stores second analysis data indicating the calculated analysis values ​​in storage device 45 (S47). This process is similar to step S23 of the initial calibration process. The second analysis data is calculated based on the image captured at the second timing, and therefore includes second display parameters that are display parameters at the second timing.

[0090] Next, the control circuit 44 corrects the coordinates FP'(N) of the feature points detected in step S47 based on the correction coefficients (homography matrix H) calculated in step S45 (S48). That is, the control circuit 44 corrects the coordinates of the feature points detected in the current positional relationship between the camera 30 and the screen 70 to coordinates that would be obtained if the feature points were detected in the positional relationship between the camera 30 and the screen 70 when the initial calibration process was performed. Specifically, the control circuit 44 calculates the corrected coordinates FP''(N) using the following equation:

[0091]

[0092] The second image in step S41 may include a test image projected by the projection display device 20. In other words, the process of step S42 may be performed on the second image including the test image. In other words, the image that is the target of step S47 and the second image may be different images or the same image.

[0093] [Comparison Process] Next, the comparison process (S13) performed after the calibration process (S12) will be described in detail. Fig. 6 is a flowchart showing an example of the comparison process.

[0094] The control circuit 44 calculates the amount of deviation between the analysis value indicated by the first analysis data stored in step S23 of the initial calibration process and the analysis value indicated by the second analysis data obtained in step S48 of the calibration process (S51). For example, the amount of deviation is the amount of deviation between the coordinate FP(N) of the feature point and the coordinate FP''(N) of the feature point. Note that the corrected coordinate FP''(N) has the deviation (change) in the relative position between the camera 30 and the screen 70, which is included in the uncorrected coordinate FP'(N), removed. Therefore, the amount of deviation between the coordinate FP(N) and the coordinate FP''(N) indicates the deviation (change) in the relative position between the projection display device 20 and the screen 70. In other words, in step S51, the deviation in the relative position between the projection display device 20 and the screen 70 is detected. In other words, the amount of deviation calculated in step S51 is an example of a first deviation.

[0095] The first deviation amount may include a deviation amount between color information stored in the initial calibration and color information stored in the calibration.The first deviation amount may include a deviation amount between focus information stored in the initial calibration and focus information stored in the calibration.

[0096] Next, the control circuit 44 calculates new correction parameters based on the second deviation amount calculated in step S51 (S52). Specifically, for each feature point n, the control circuit 44 calculates a homography matrix Hn such that the coordinate FP''n moves to the coordinate FPn, and calculates a new geometric correction parameter (homography matrix) Hnewn by multiplying the current geometric correction parameter (homography matrix) Holdn by the homography matrix Hn. In other words, the new geometric correction parameter Hnewn is calculated based on the formula Hnewn = Holdn × Hn.

[0097] Based on the deviation of the color information, the control circuit 44 may also calculate color correction parameters that adjust the luminance values ​​of each of the RGB colors of the projection display device 20 so that the color information matches a predetermined reference. Based on the deviation of the focus state, the control circuit 44 may also calculate focus correction parameters that adjust the focus state of the projection display device 20 so that the optimal focus state is achieved.

[0098] Then, the control circuit 44 accepts the new correction parameters calculated in step S52 and sets them in the projection display device 20 (S53). Specifically, the control circuit 44 transmits a setting command including the new correction parameters to the projection display device 20 via the communication circuit 43. As a result, the correction parameters in the projection display device 20 are changed. In other words, the image projected by the projection display device 20 is corrected.

[0099] If the control circuit 44 determines that the first deviation amount calculated in step S51 is smaller than a predetermined value, it may omit the processes of steps S52 and S53. For example, if the coordinate FP"(N) and the coordinate FP(N) are substantially the same and there is no deviation in the relative positions of the projection display device 20 and the screen 70, there is no need to correct the image projected by the projection display device 20.

[0100] [Determination Process] Next, the determination process (S14) performed after the comparison process (S13) will be described in detail. Fig. 7 is a flowchart showing an example of the determination process.

[0101] The control circuit 44 determines whether the first deviation amount calculated in step S51 is greater than a first threshold value (S61).

[0102] If the control circuit 44 determines that the first deviation amount is greater than the first threshold value (Yes in S61), it determines whether the second deviation amount calculated in step S44 is greater than the second threshold value (S62).

[0103] When the control circuit 44 determines that the second deviation amount is greater than the second threshold value (Yes in S62), it determines that both the first deviation and the second deviation exist (S63).

[0104] When the control circuit 44 determines that the second deviation amount is equal to or less than the second threshold value (No in S62), it determines that there is a first deviation and that there is no second deviation (S64).

[0105] If the control circuit 44 determines that the first deviation amount is less than or equal to the first threshold value (No in S61), it determines whether the second deviation amount calculated in step S44 is greater than the second threshold value (S65).

[0106] When the control circuit 44 determines that the second deviation amount is greater than the second threshold value (Yes in S65), it determines that there is no first deviation and that there is a second deviation (S66).

[0107] When the control circuit 44 determines that the second deviation amount is equal to or less than the second threshold value (No in S65), it determines that there is no first deviation and no second deviation (S67).

[0108] In step S14, the control circuit 44 determines that a deviation has occurred if it is determined that at least one of the first deviation and the second deviation exists, whereas the control circuit 44 determines that no deviation has occurred if it is determined that neither the first deviation nor the second deviation exists.

[0109] In the judgment of step S61, the deviation amount of the coordinates of the feature point, the deviation amount of the color information, and the deviation amount of the focus information included in the first deviation amount are each compared with a first threshold value set for each deviation amount.

[0110] [Notification Process] Next, the deviation information notified in the notification process will be described with reference to Fig. 8 and Fig. 9. Fig. 8 is a diagram showing an example of a first deviation amount occurring in the projection display device 20. Fig. 9 is a diagram showing an example of a second deviation amount occurring in the camera 30.

[0111] 8A is a graph showing the change over time in the positional deviation of the projection position of the image projected by the projection display device 20. In this graph, the vertical axis represents the amount of positional deviation in the X-axis direction (left-right or horizontal direction) or the amount of positional deviation in the Y-axis direction. The horizontal axis represents the elapsed time (date and time). The X-axis direction is the left-right direction (horizontal direction) of the image projected by the projection display device 20. The Y-axis direction is the up-down direction (vertical direction) of the image projected by the projection display device 20.

[0112] 8B is a graph showing the change over time in the positional deviation of the angle of view of an image projected by the projection display device 20. In this graph, the vertical axis represents the size of the angle of view, which indicates the amount of change in the size of the projected image in the width direction (horizontal direction) or the amount of change in the size of the projected image in the height direction (vertical direction), and the horizontal axis represents the elapsed time (date and time).

[0113] 8C is a graph showing the transition of the projection position of the image projected by the projection display device 20. In this graph, the vertical axis represents the amount of deviation in the Y-axis direction, and the horizontal axis represents the amount of deviation in the X-axis direction.

[0114] 9A is a graph showing the change over time in the positional deviation of the screen 70 in the image captured by the camera 30. In this graph, the vertical axis indicates the amount of deviation in the X-axis direction (left-right or horizontal direction) or the amount of deviation in the Y-axis direction. The horizontal axis indicates the elapsed time (date and time). The X-axis direction is the left-right direction (horizontal direction) of the image captured by the camera 30. The Y-axis direction is the up-down direction (vertical direction) of the image captured by the camera 30.

[0115] 9B is a graph showing the transition of the position of the screen 70 in the image captured by the camera 30. In this graph, the vertical axis represents the amount of deviation in the Y-axis direction, and the horizontal axis represents the amount of deviation in the X-axis direction.

[0116] In this way, when a first deviation occurs, the control circuit 44 may notify the user of deviation information including the graphs shown in FIGS. 8A to 8C (graphs showing the first deviation) as the first deviation amount via the communication circuit 43. Furthermore, when a second deviation occurs, the control circuit 44 may notify the user of deviation information including the graphs shown in FIGS. 9A and 9B (graphs showing the second deviation) as the second deviation amount via the communication circuit 43. Although not shown, the deviation information may include, as the first deviation amount, a graph showing the time-dependent change in color information in the projection display device 20, or a graph showing the time-dependent change in focus information in the projection display device 20. Furthermore, when both the first deviation and the second deviation occur, the control circuit 44 may display both the graph showing the first deviation and the graph showing the second deviation on the monitor 42. Furthermore, when the first deviation has occurred but the second deviation has not, the control circuit 44 may display a graph showing the first deviation on the monitor 42, but may not display a graph showing the second deviation on the monitor 42. Furthermore, when the first deviation has not occurred but the second deviation has occurred, the control circuit 44 may not display a graph showing the first deviation on the monitor 42, but may display a graph showing the second deviation on the monitor 42. This makes it possible to properly notify which of the camera 30 and the projection display device 20 requires maintenance.

[0117] The control circuit 44 may store the deviation information to be notified and other information in different storages (e.g., storage devices 45 and 46). For example, the control circuit 44 may store the deviation information in storage device 46, which is not restricted to specific users, and store the other information in storage device 45, which is restricted to specific users. The other information may include, for example, the first image, the second image, the first analysis data, the second analysis data, and the comparison results. In this way, the deviation information to be notified and various other data used to correct the projection display device 20 and the camera 30 can be stored separately. This makes it easier to manage the two data with different security levels, for example. In other words, the security level of storage device 45 (an example of a second storage device) may be higher than the security level of storage device 46 (an example of a first storage device) that stores the deviation information.

[0118] Furthermore, the control circuit 44 may store the deviation information in a server 51 (another example of a first storage device) connected via an external network 60, and store other information in the storage device 45. The deviation information has a smaller data volume than other information, and since the deviation information is transmitted to the server 51 via the network 60, the communication load on the network 60 can be reduced.

[0119] [Effects, etc.] As described above, the notification method executed by a computer such as the projection display system 10 (information processing device 40) includes: a first acquisition step (S11) of acquiring a first image captured by the camera 30 at a first timing, the first image including the display area of ​​an image to be displayed by the projection display device 20; a second acquisition step (S12) of acquiring a second image captured by the camera 30 at a second timing a predetermined period after the first timing, the second image including the display area; a comparison step (S13) of comparing first analysis data based on the first image with second analysis data based on the second image; a first determination step (S14) of determining, based on the comparison result in the comparison step, whether a first deviation has occurred, which is a deviation in a display parameter related to the display by the projection display device 20, and whether a second deviation has occurred, which is a deviation in an imaging parameter related to the imaging by the camera 30; and a notification step (S15) of notifying a user of deviation information indicating which of the first deviation and the second deviation has occurred, which deviation has been determined to have occurred in the determination step.

[0120] In this way, the deviation information notified to the user includes the deviation that occurred between the first deviation and the second deviation, and therefore the user can be notified of the device (i.e., the projection display device 20 or the camera 30) that has the parameter that is the source of the deviation and the deviation itself, thereby being able to notify the user which of the camera 30 and the projection display device 20 needs maintenance.

[0121] [Modification 1] Next, a description will be given of the correction limit notification process in the projection display system according to Modification 1. Fig. 10 is a flowchart showing an example of the correction limit notification process according to Modification 1.

[0122] The control circuit 44 accepts adjustment of the display parameters of the projection display device 20 based on input received from the user via the input device 41 (S71). For example, the control circuit 44 may accept input from the user while the projection display device 20 is projecting a test image, adjust the display parameters, and project the test image according to the adjusted display parameters. This allows the user to adjust the display parameters to their desired values ​​while visually viewing the projected test image. In other words, the user can adjust the position, color, brightness, and focus state of the image projected by the projection display device 20.

[0123] Next, the control circuit 44 determines whether or not the correction limit is being approached (S72).

[0124] FIG. 11 is a diagram for explaining the correction limit.

[0125] The correction limit is the limit value for adjusting a display parameter. Display parameters are changed using adjustment values ​​within a predetermined adjustment range. In other words, the projection display device 20 cannot change a display parameter to an adjustment value outside the predetermined adjustment range. The control circuit 44 determines whether the correction limit is being approached by determining whether the adjustment value exceeds the predetermined adjustment value. The predetermined adjustment value is set between the initial adjustment value of each display parameter and the minimum or maximum value of the adjustment range. The first adjustment value is set between the initial adjustment value and the maximum value of the adjustment range, and the second adjustment value is set between the initial adjustment value and the minimum value of the adjustment range. The predetermined adjustment value is set to a value closer to the minimum or maximum value of the adjustment range than the initial value. The control circuit 44 determines whether the adjustment value has moved from a value closer to the initial value than the predetermined adjustment value to closer to the minimum or maximum value than the predetermined adjustment value, and if it determines that the adjustment value has approached the minimum or maximum value, it determines that the correction limit is being approached.

[0126] This determination may be made for each of the adjustment values ​​for the positional deviation of the display position of the projection display device 20 (positional deviation of the coordinates of the feature points), the adjustment value for the color of the image projected by the projection display device 20, and the adjustment value for the focus state of the image projected by the projection display device 20, or may be made for at least one of these adjustment values. The predetermined adjustment value is a value set for each of these adjustment values.

[0127] When the control circuit 44 determines that the correction limit is approaching (Yes in S72), it notifies the user that the correction limit is approaching (S73). Note that, when deviation information is notified, the notification of the correction limit may be included in the deviation information to be notified.

[0128] If the control circuit 44 determines that the correction limit is not approaching (No in S72), the process ends.

[0129] In the comparison process described using FIG. 6, new correction parameters are automatically calculated and set as the new correction parameters, but even in this case, the processes of steps S72 and S73 may be performed.

[0130] As described above, the display parameters are changed by adjustment values ​​within a predetermined adjustment range in the information processing device 40. Then, the information processing device 40 determines whether the adjustment value exceeds a predetermined adjustment value, and if the adjustment value exceeds the predetermined adjustment value, notifies the user that the correction limit is approaching.

[0131] Therefore, it is possible to notify the user that the correction limit (adjustment limit) is approaching, and therefore it is possible to notify the user that the time when maintenance is required is approaching.

[0132] Furthermore, for example, the display parameters include the position of the image projected by the projection display device 20 relative to the display area. This makes it possible to notify the user that the adjustment limit for the projection position of the image on the projection display device 20 is approaching. This makes it possible to notify the user that maintenance is soon required.

[0133] Furthermore, for example, the display parameters include the focus state of the image projected by the projection display device 20. Therefore, it is possible to notify the user that the limit of image focus adjustment in the projection display device 20 is approaching. Therefore, it is possible to notify the user that the time for maintenance is approaching.

[0134] Furthermore, for example, the display parameters include the color of the image displayed by the projection display device 20. Therefore, it is possible to notify the user that the color adjustment limit of the image on the projection display device 20 is approaching, and therefore it is possible to notify the user that the time for maintenance is approaching.

[0135] Furthermore, for example, the display parameters include the brightness of the image displayed by the projection display device 20. Therefore, it is possible to notify the user that the brightness adjustment limit of the image on the projection display device 20 is approaching. Therefore, it is possible to notify the user that the time for maintenance is approaching.

[0136] [Modification 2] Next, a description will be given of the configuration of a projection display system according to Modification 2. Fig. 12 is a diagram showing the configuration of a projection display system according to Modification 2.

[0137] The projection display system 10a differs from the projection display system 10 in that it projects an image onto a wall surface 80 instead of a screen 70, but other configurations are the same as those of the projection display system 10, and therefore a description thereof will be omitted.

[0138] Wall surface 80 is used as an area onto which an image is projected by projection display device 20. Four non-luminous markers 81 are provided on wall surface 80. The number and arrangement of non-luminous markers 81 are not particularly limited.

[0139] In the feature point detection process executed by the projection display system 10a, feature points appearing in (or around) the plurality of non-luminous markers 81 displayed in the first image are detected, and the control circuit 44 uses an algorithm suitable for detecting such feature points. The process for detecting feature points in the second image is similar to the feature point detection process (the process for detecting feature points in the first image).

[0140] [Modification 3] Next, we will explain the configuration of a projection display system according to Modification 3. Fig. 13 is a diagram showing the configuration of a projection display system according to Modification 3.

[0141] The projection display system 10b projects an image onto a screen 90. Unlike the screen 70, the screen 90 does not have a frame 70a, and instead has luminous markers 91 at positions corresponding to the four vertices (corners) of the screen 90.

[0142] Unlike the projection display system 10, the projection display system 10b includes a marker control device 50. The other configurations of the projection display system 10b are the same as those of the projection display system 10, and therefore a description thereof will be omitted.

[0143] The marker control device 50 turns on and off the light-emitting markers 91 under the control of the information processing device 40. The light-emitting markers 91 are realized by, for example, LED (Light Emitting Diode) elements. The number and arrangement of the light-emitting markers 91 are not particularly limited.

[0144] In the feature point detection process executed by the projection display system 10b, the first image is captured with the plurality of light-emitting markers 91 turned on by the marker control device 50, and feature points appearing in (or around) the plurality of light-emitting markers 91 shown in the first image are detected. The control circuit 44 uses an algorithm suitable for detecting such feature points.

[0145] Other Embodiments Although notification methods and projection display systems according to the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

[0146] For example, in the above embodiment, the frame of the screen or a marker is used to detect the feature points, but other objects may be used to detect the feature points. As long as the relative position between the camera and the surface onto which the image is projected can be identified, the object is not particularly limited.

[0147] For example, in the above embodiment, the projection display system is realized by multiple devices, but it may also be realized as a single device. For example, the projection display system may be realized as a single device equivalent to an information processing device. When the projection display system is realized by multiple devices, the components of the projection display system may be distributed among the multiple devices in any manner.

[0148] For example, in the above embodiment, the display device is the projection display device 20, but this is not limiting. For example, a non-projection display device such as a liquid crystal display or an organic EL display may also be used. Even in this case, notification regarding display parameters other than the projection position of the image can be provided.

[0149] For example, in the above embodiment, the deviation to be notified is described as a deviation from the initial state. However, with regard to deviations in focus, color, and brightness, the optimal state for each changes depending on changes in the surrounding environment, etc. Therefore, information about deviations from the optimal display state at each time may be notified to the user. Here, methods for adjusting color and brightness to the optimal display state at each time may include adjusting the display state of another display device to the darkest display device at that time, or adjusting to an optimal display state previously determined by the system or the user.

[0150] In the above-described embodiment, the processing performed by a specific processing unit may be performed by another processing unit. The order of multiple processing operations may be changed, or multiple processing operations may be performed in parallel.

[0151] In the above-described embodiments, each component may be realized by executing a software program suitable for that component, or by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0152] Furthermore, each component may be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0153] Furthermore, the general or specific aspects of the present disclosure may be realized as a system, device, method, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM. Also, the present disclosure may be realized as any combination of a system, device, method, integrated circuit, computer program, and recording medium. For example, the present disclosure may be realized as the projection display system or information processing device 40 of the above-described embodiment. The present disclosure may be realized as a program (computer program product) for causing a computer to execute the calibration method of the above-described embodiment, or as a computer-readable non-transitory recording medium on which such a program is stored.

[0154] In addition, this disclosure also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art would think of, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope that does not deviate from the intent of this disclosure.

[0155] The notification method of the present disclosure is useful as a notification system that can notify which of a camera and a display device needs maintenance.

[0156] REFERENCE SIGNS LIST 10, 10a, 10b Projection display system 20 Projection display device 30 Camera 40 Information processing device 41 Input device 42 Monitor 43 Communication circuit 44 Control circuit 45, 46 Storage device 50 Marker control device 51 Server 52, 53 Information terminal 60 Network 70, 90 Screen 70a Frame 70b Superimposed portion 80 Wall surface 81 Non-luminous marker 91 Luminous marker

Claims

1. a first acquisition step of acquiring a first image captured by a camera at a first timing, the first image including a display area of ​​an image to be displayed on a display device; a second acquisition step of acquiring a second image captured by the camera at a second timing a predetermined period after the first timing, the second image including the display area; a comparing step of comparing first analysis data based on the first image with second analysis data based on the second image; a first determination step of determining whether or not a first deviation, which is a deviation in a display parameter related to display by the display device, has occurred, and whether or not a second deviation, which is a deviation in a shooting parameter related to shooting by the camera, has occurred, based on a comparison result in the comparison step; a notification step of notifying a user of deviation information indicating a deviation determined to have occurred in the first determination step, out of the first deviation and the second deviation. Notification method.

2. moreover, and a changing step of accepting an adjustment of the display parameter and changing the display parameter in accordance with the accepted adjustment. The notification method according to claim 1 .

3. The display parameters are changed by adjustment values ​​within a predetermined adjustment range, The notification method further includes: a second determination step of determining whether the adjustment value exceeds a predetermined adjustment value; In the notifying step, if the adjustment value exceeds the predetermined adjustment value in the second determining step, the user is further notified that the adjustment limit is approaching. The notification method according to claim 2 .

4. moreover, an evaluation step of evaluating a deviation of the display parameters based on the first image and the second image; In the first determination step, it is determined whether a first deviation, which is a deviation of the evaluated display parameter, has occurred. The notification method according to claim 1 .

5. the display device is a projector that projects an image onto the display area, The display parameters include a position of an image projected by the projector relative to the display area. The notification method according to any one of claims 1 to 4.

6. the display device is a projector that projects an image onto the display area, The display parameters include a focus state of an image projected by the projector. The notification method according to any one of claims 1 to 4.

7. The display parameters include a color of an image displayed by the display device. The notification method according to any one of claims 1 to 4.

8. The display parameters include brightness of an image displayed by the display device. The notification method according to any one of claims 1 to 4.

9. the notification method is executed by a control device connected to the camera and the display device via a local network; The notification method further includes: a first storage step of storing the deviation information notified in the notification step in a first storage device; a second storage step of storing at least one of the first image, the second image, the first analysis data, the second analysis data, and the comparison result in a second storage device different from the first storage device. The notification method according to any one of claims 1 to 4.

10. the control device includes the first storage device and the second storage device; The second storage device has a security level higher than the security level of the first storage device. The notification method according to claim 9.

11. the first storage device is connected to the camera and the display device via an external network; The second storage device is connected to the camera and the display device via the local network. The notification method according to claim 9.

12. moreover, The method further includes a display step of displaying a graph indicating the first deviation and a graph indicating the second deviation when it is determined that the first deviation has occurred and when it is determined that the second deviation has occurred. The notification method according to claim 1 .

13. moreover, The method further includes a display step of displaying a graph showing the first deviation when it is determined that the first deviation has occurred and the second deviation has not occurred. The notification method according to claim 1 .

14. moreover, The method further includes a display step of displaying a graph showing the second deviation when it is determined that the first deviation has not occurred and the second deviation has occurred. The notification method according to claim 1 .

15. A program for causing a computer to execute the notification method according to any one of claims 1 to 4.

16. a display device; A camera and an information processing device; The information processing device includes: a first acquisition step of acquiring a first image captured by a camera at a first timing, the first image including a display area of ​​an image to be displayed on a display device; a second acquisition step of acquiring a second image captured by the camera at a second timing a predetermined period after the first timing, the second image including the display area; a comparing step of comparing first analysis data based on the first image with second analysis data based on the second image; a first determination step of determining whether or not a first deviation, which is a deviation in a display parameter related to display by the display device, has occurred, and whether or not a second deviation, which is a deviation in a shooting parameter related to shooting by the camera, has occurred, based on a comparison result in the comparison step; a notification step of notifying a user of deviation information indicating a deviation determined to have occurred in the first determination step, out of the first deviation and the second deviation. Notification system.