Information processing device, information processing method, and program

Abstract

To detect a positional deviation between two cameras in a camera system including a plurality of cameras.SOLUTION: A calculation unit (12) calculates movement of a first camera with respect to a predetermined position, based on an image captured by the first camera. A detection unit (14) detects a positional deviation between the first camera and a second camera, based on the movement of the first camera.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program, and more particularly to an information processing device, an information processing method, and a program that detect positional deviation between two cameras in a camera system equipped with multiple cameras. [Background technology]

[0002] A camera system that uses a visible light camera to perform face recognition has been developed. To add infrared thermography functionality to this camera system, a thermal camera is attached near the visible light camera. In this way, the relative positions of the visible light camera and the thermal camera are fixed.

[0003] First, both cameras are calibrated. In one example, a common marker is photographed using a visible light camera and a thermal camera. Then, based on the position of the marker on the images acquired by each camera, the coordinate system of the visible light camera and the coordinate system of the thermal camera are calculated (Patent Document 1). This makes it possible to obtain a transformation formula between the coordinate system of the visible light camera and the coordinate system of the thermal camera. This completes the preparation for photographing.

[0004] A visible light camera is used to capture a person's face. By detecting the face from the visible light image, the position of the face in the coordinate system of the visible light camera is identified. Then, based on the image obtained from the visible light camera (called the visible light image), a machine-learned classifier is used to recognize the person's face.

[0005] The same person is photographed using a thermal camera. The image obtained from the thermal camera (called a thermal distribution image) is used to measure body temperature. At this time, the conversion formula obtained in the preparation stage is used to obtain the position coordinates of the person's face in the thermal camera's coordinate system from the position coordinates of the face in the visible light camera's coordinate system. By using the position coordinates of the person's face in the thermal camera's coordinate system, it is possible to perform stable body temperature measurements in the area of ​​the person's face on the thermal distribution image, especially in areas where the skin is widely exposed, such as between the eyebrows and the inner corners of the eyes. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. 2017 / 056473 [Patent Document 2] Japanese Patent Application Laid-Open No. 2002-232875 [Patent Document 3] Japanese Patent Application Laid-Open No. 2012-34147 [Patent Document 4] Japanese Patent Application Publication No. 2019-129403 [Patent Document 5] Japanese Patent Application Publication No. 2019-75776 Summary of the Invention [Problem to be solved by the invention]

[0007] This disclosure aims to improve upon the techniques disclosed in the prior art documents mentioned above. [Means for solving the problem]

[0008] An information processing device according to one aspect of this disclosure includes a calculation means for calculating the movement of a first camera relative to a predetermined position based on an image captured by the first camera, and a detection means for detecting a positional deviation between the first camera and a second camera based on the movement of the first camera.

[0009] In an information processing method according to one aspect of this disclosure, the movement of a first camera relative to a predetermined position is calculated based on an image captured by the first camera, and the positional deviation between the first camera and a second camera is detected based on the movement of the first camera.

[0010] A program according to one aspect of the present disclosure causes a computer to perform a process of calculating the movement of a first camera relative to a predetermined position based on an image captured by the first camera, and a process of detecting a positional deviation between the first camera and a second camera based on the movement of the first camera. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram schematically illustrating a configuration of a system including an information processing device according to any one of first to fourth embodiments. [Figure 2] 2 is a diagram showing an example of the arrangement of a visible light camera and a thermal camera provided in the system shown in FIG. 1. FIG. [Figure 3] 2 is a diagram showing an example of the configuration of an alignment device included in the system shown in FIG. 1. FIG. [Figure 4] 2 is a flowchart showing the operation of an alignment device provided in the system shown in FIG. 1. [Figure 5] 2 is a diagram showing an example of the configuration of a cognitive body temperature measuring device included in the system shown in FIG. 1. FIG. [Figure 6] 2 is a flowchart showing the operation of the cognitive body temperature measuring device provided in the system shown in FIG. [Figure 7] 1 is a block diagram showing the configuration of an information processing apparatus according to a first embodiment. [Figure 8] 4 is a flowchart showing the operation of the information processing device according to the first embodiment. [Figure 9] FIG. 10 is a block diagram showing the configuration of an information processing apparatus according to a second embodiment. [Figure 10] 10 is a flowchart showing the operation of the information processing device according to the second embodiment. [Figure 11]FIG. 10 is a block diagram showing the configuration of an information processing device according to a third embodiment. [Figure 12] 10 is a flowchart showing the operation of the information processing device according to the third embodiment. [Figure 13] FIG. 10 is a block diagram showing the configuration of an information processing device according to a fourth embodiment. [Figure 14] 10 is a flowchart showing the operation of the information processing device according to the fourth embodiment. [Figure 15] FIG. 1 is a diagram showing a hardware configuration of an information processing device according to any one of the first to fourth embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0012] First, an example of the configuration of a camera system to which information processing devices according to first to fourth embodiments described below are applied will be described.

[0013] (Camera System 1) Fig. 1 is a diagram illustrating an example of the configuration of a camera system 1. As shown in Fig. 1, the camera system 1 includes an alignment device 100, a recognition body temperature measurement device 200, an information processing device 10 (20, 30, 40), a storage 300, a visible light camera 400, and a thermal camera 500. Here, the "information processing device 10 (20, 30, 40)" refers to any of the information processing devices 10 to 40 according to embodiments 1 to 4 described below.

[0014] The storage 300 stores various data, which will be described later. The storage 300 is communicably connected to the alignment device 100, the recognition body temperature measurement device 200, and the information processing device 10 (20, 30, 40) via wired or wireless communication. The storage 300 may be formed in a local server or a cloud server.

[0015] The visible light camera 400 is a camera capable of capturing images in the visible light band. The thermal camera 500 is a camera capable of detecting infrared rays and measuring the temperature of a heat-generating body. The thermal camera 500 is sometimes called a thermography camera. In this case, the subject to be captured is a person's face, and the subject to be measured is the body temperature. The visible light camera 400 and the thermal camera 500 are fixed in a pre-positioned state.

[0016] 2 is a diagram showing an example of the positional relationship between the visible light camera 400 and the thermal camera 500. In the example shown in FIG. 2, the visible light camera 400 and the thermal camera 500 are fixed side by side in the vertical direction on the same support. The field of view of the visible light camera 400 and the field of view of the thermal camera 500 at least partially overlap. This makes it possible for both the visible light camera 400 and the thermal camera 500 to simultaneously capture images of the face of the same person.

[0017] The visible light camera 400 and the thermal camera 500 do not necessarily have to be fixed to the same pole as in the example shown in Fig. 2. In another example, the visible light camera 400 and the thermal camera 500 may be fixed to different poles, stands, rails, walls, or ceilings.

[0018] The alignment device 100 and the recognition body temperature measurement device 200, which are components of the camera system 1, will be described below.

[0019] (Alignment device 100) The alignment device 100 associates the coordinate system of the visible light camera 400 with the coordinate system of the thermal camera 500 when the visible light camera 400 and the thermal camera 500 are respectively in predetermined positions.

[0020] The configuration of the alignment device 100 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the configuration of the alignment device 100. As shown in Fig. 3, the alignment device 100 includes a marker detection unit 101, a conversion formula creation unit 102, and a default value setting unit 103.

[0021] First, for calibration, the operator places one or more markers within the common field of view of the visible light camera 400 and the thermal camera 500. In one example, the markers may be spherical or other three-dimensional objects, or boards with predetermined two-dimensional patterns drawn on them. Alternatively, the markers may be parts of the human body, such as a face or hand.

[0022] The marker detection unit 101 captures an image of one or more installed markers using the visible light camera 400 and the thermal camera 500. The marker detection unit 101 detects a plurality of feature points corresponding to the captured one or more markers from the visible light image obtained from the visible light camera 400. The marker detection unit 101 also detects a plurality of feature points corresponding to the captured one or more markers from the thermal distribution image obtained from the thermal camera 500.

[0023] The marker detection unit 101 outputs to the conversion formula creation unit 102 information indicating the position coordinates where one or more markers are detected on the visible light image and the thermal distribution image, respectively.

[0024] The conversion formula creation unit 102 obtains position coordinates where one or more markers are detected on the visible light image and the thermal distribution image. The conversion formula creation unit 102 solves simultaneous equations that represent the relationship between the position coordinates of one or more markers on the visible light image and the position coordinates of one or more markers on the thermal distribution image. This results in a conversion formula for converting from the coordinate system of the visible light camera 400 to the coordinate system of the thermal camera 500. The conversion formula creation unit 102 notifies the operator that the creation of the conversion formula is complete. Note that if the position of the marker changes during calibration, the information processing device 10 (20, 30, 40), described below, may erroneously detect a misalignment between the visible light camera 400 and the thermal camera 500. Therefore, the alignment device 100 may output a warning message, such as "Do not move the markers," after starting calibration and until calibration is completed.

[0025] After the conversion formula is created, the operator removes one or more markers from the fields of view of the visible light camera 400 and the thermal camera 500. Next, the default value setting unit 103 causes the visible light camera 400 and the thermal camera 500 to capture an image of the background. Here, the background is preferably an object that is semi-permanent, such as a building or tree, or that moves little at least while the visible light camera 400 and the thermal camera 500 are installed. Alternatively / in addition, even if the background moves, it is preferable that the change in position is small.

[0026] Specifically, the default value setting unit 103 continuously captures images multiple times using the visible light camera 400 and the thermal camera 500. The default value setting unit 103 extracts, as the background in the visible light image, an area that does not change or an area that changes little between the multiple visible light images obtained by capturing the images multiple times. The default value setting unit 103 also extracts, as the background in the thermal distribution image, an area that does not change or an area that changes little between the multiple thermal distribution images obtained by capturing the images multiple times. In another example, the default value setting unit 103 may extract the background from one visible light image obtained by capturing the images once using the visible light camera 400 using a general image recognition technique. Alternatively, the background may be extracted manually instead of by the default value setting unit 103.

[0027] The default value setting unit 103 extracts features from the background of the visible light image and the thermal distribution image.

[0028] For example, the default setting unit 103 extracts features of any of the following objects in the background: lighting, pillars, windows, doors, displays, or black bodies, from the visible light image and the thermal distribution image, respectively. In particular, the default setting unit 103 preferably extracts features from heat-generating objects such as lighting and displays. This is because heat-generating objects have clear positions and shapes in thermal distribution images, making them easily distinguishable from the surrounding background. However, due to the temperature difference between the outside air and the inside air, the temperatures of windows and doors are also easily distinguishable from the temperature of the interior walls of buildings. Therefore, the default setting unit 103 does not necessarily need to extract features only from heat-generating objects, as long as the objects are easily distinguishable from the surrounding background in terms of temperature. Furthermore, because the temperature difference between the outside air and the inside air changes depending on the season, day, or weather, the default setting unit 103 may determine from which object features to extract based on the season, day, or weather. In this case, an administrator inputs information about the season, day, or weather into the alignment device 100.

[0029] Note that the features extracted from the background of the visible light image and the features extracted from the background of the thermal distribution image may be features of the same object. However, as will be described in a first embodiment below, the features extracted from the background of the visible light image are matched with predetermined values ​​of the features of the background of the visible light image. Furthermore, as will be described in a modified example of the first embodiment, the features extracted from the background of the thermal distribution image are matched with predetermined values ​​of the features extracted from the background of the thermal distribution image. In other words, the features extracted from the background of the visible light image and the features extracted from the background of the thermal distribution image are not compared. Therefore, the features extracted from the background of the visible light image and the features extracted from the background of the thermal distribution image may be features of different objects.

[0030] The default value setting unit 103 stores data of features extracted from the background of the visible light image and data of features extracted from the background of the thermal distribution image in the storage 300 as default values ​​of background features. Examples of background features include window frames and door frames. Alternatively, in another example, background features include lighting and the edges of a black body. The default values ​​of background features are used by the information processing device 10 (20, 30, 40) in the first to fourth embodiments described below as references for background features when there is no positional deviation between the visible light camera 400 and the thermal camera 500.

[0031] (Operation of Alignment Device 100) The operation of the alignment device 100 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of processing executed by each unit of the alignment device 100.

[0032] As shown in FIG. 4, first, the marker detection unit 101 captures an image of one or more markers using the visible light camera 400 and the thermal camera 500, which are respectively placed at predetermined positions (S101).

[0033] Next, the marker detection unit 101 detects a plurality of feature points corresponding to the one or more captured markers from the visible light image obtained from the visible light camera 400. The marker detection unit 101 also detects a plurality of feature points corresponding to the one or more captured markers from the thermal distribution image obtained from the thermal camera 500 (S102).

[0034] The conversion formula creation unit 102 creates a conversion formula for converting from the coordinate system of the visible light camera 400 to the coordinate system of the thermal camera 500 based on the relationship between the position coordinates of one or more markers on the visible light image and the position coordinates of one or more markers on the thermal distribution image (S103). The conversion formula creation unit 102 stores the created conversion formula in the storage 300.

[0035] Next, the default value setting unit 103 uses the visible light camera 400 and the thermal camera 500 to capture an image of the background (S104).

[0036] The default value setting unit 103 extracts any feature from the background of the visible light image and also extracts any feature from the background of the thermal distribution image (S105).

[0037] Then, the default value setting unit 103 records the features extracted from the background of the visible light image and the features extracted from the background of the thermal distribution image in the storage 300 as default values ​​of the background features (S106).

[0038] This completes the operation of the alignment device 100.

[0039] (Recognized body temperature measuring device 200) The configuration of the recognition body temperature measuring device 200 will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the configuration of the recognition body temperature measuring device 200. As shown in Fig. 5, the recognition body temperature measuring device 200 includes a face recognition unit 201, a coordinate conversion unit 202, and a body temperature measuring unit 203.

[0040] The face recognition unit 201 uses the visible light camera 400 to capture an image of a person's face. The face recognition unit 201 acquires a visible light image obtained by capturing an image of the person's face from the visible light camera 400. The face recognition unit 201 detects the person's face from the visible light image. For example, the face recognition unit 201 detects the area of ​​the person's face from the visible light image. Next, the face recognition unit 201 recognizes (identifies) the face detected from the visible light image. Furthermore, the face recognition unit 201 generates position coordinate data that identifies the area of ​​the person's face in the visible light image.

[0041] In one example of face recognition processing, the face recognition unit 201 extracts facial features detected from a visible light image. Then, the face recognition unit 201 calculates the similarity between the facial features detected from the visible light image and facial features of people registered in advance in a database (not shown). The database of facial features of registered people may be stored in the storage 300.

[0042] The face recognition unit 201 recognizes the person whose facial features registered in the database have a similarity to the facial features detected from the visible light image that is equal to or greater than a threshold value and has the highest similarity as the person detected from the visible light image.

[0043] The face recognition unit 201 recognizes the identity of the person detected from the visible light image based on the calculation result of the similarity. The face recognition unit 201 outputs the recognition result of the person detected from the visible light image to the body temperature measurement unit 203. The face recognition unit 201 outputs position coordinate data of the person's face in the coordinate system of the visible light camera 400 to the coordinate conversion unit 202.

[0044] If the similarity between the facial features detected from the visible light image and the facial features of a person registered in the database is equal to or greater than a threshold value and there is no person registered in the database who meets the condition of the highest similarity, the face recognition unit 201 outputs to the body temperature measurement unit 203 a recognition result that the person whose face was detected from the visible light image is not registered in the database, or a recognition result that recognition failed.

[0045] When the coordinate conversion unit 202 receives the position coordinate data from the face recognition unit 201, it uses the conversion formula stored in the storage 300 by the conversion formula creation unit 102 to convert the position coordinates of the person's face in the coordinate system of the visible light camera 400 into position coordinates in the coordinate system of the thermal camera 500. The coordinate conversion unit 202 outputs information indicating the position coordinates of the person's face in the coordinate system of the thermal camera 500 to the body temperature measurement unit 203.

[0046] The body temperature measurement unit 203 photographs the face of a person using the thermal camera 500. The body temperature measurement unit 203 acquires a heat distribution image obtained by photographing the face of the person from the thermal camera 500. The body temperature measurement unit 203 receives the recognition result of the person's face detected from the visible light image from the face recognition unit 201. The body temperature measurement unit 203 also receives information indicating the position coordinates of the person's face in the coordinate system of the thermal camera 500 from the coordinate conversion unit 202.

[0047] The body temperature measurement unit 203 uses information indicating the position coordinates of the person's face in the coordinate system of the thermal camera 500 to identify the position of the person's face on the thermal distribution image.

[0048] Then, the body temperature measurement unit 203 identifies areas of the model person's face that correspond to the forehead, cheeks, etc., using information indicating the general positions of the forehead and cheeks within the facial contours of the model person. The information indicating the general positions of the forehead and cheeks may be stored in the storage 300 or in a recording medium (not shown).

[0049] The body temperature measurement unit 203 then extracts information indicating the person's body temperature from areas on the thermal distribution image that correspond to parts of the person's face, such as the forehead and cheeks. The body temperature measurement unit 203 associates the recognition result of the person's face with the information indicating the person's body temperature, and stores them in the storage 300. However, if the body temperature measurement unit 203 receives a recognition result from the face recognition unit 201 that the person has not been recognized, the body temperature measurement unit 203 stores only the information indicating the person's body temperature as a new person in the storage 300. Alternatively, the body temperature measurement unit 203 may store the information indicating the person's body temperature in the storage 300 together with data of the visible light image of the new person's face.

[0050] Unless a predetermined condition is satisfied, the above-described operation of the recognition body temperature measuring device 200 is repeated. When the predetermined condition is satisfied, the operation of the recognition body temperature measuring device 200 is switched to the operation of the information processing devices 10 to 40.

[0051] The information processing devices 10 to 40 detect the relative positional deviation between the visible light camera 400 and the thermal camera 500 based on the images acquired from the visible light camera 400 and the thermal camera 500. After the information processing devices 10 to 40 detect the positional deviation, the operation of the recognition body temperature measuring device 200 is resumed.

[0052] The information processing devices 10 to 40 are configured as in the first to fourth embodiments described later, respectively.

[0053] (Operation of the cognitive body temperature measuring device 200) The operation of the recognition body temperature measuring device 200 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of processing executed by each unit of the recognition body temperature measuring device 200.

[0054] As shown in FIG. 6, the face recognition unit 201 acquires an image obtained by capturing a person's face using the visible light camera 400 (S201).

[0055] The face recognition unit 201 detects a face from the acquired image (S202).

[0056] The face recognition unit 201 refers to a database of people stored in the storage 300 and recognizes the detected face (S203).

[0057] The face recognition unit 201 identifies the position coordinates of the person's face in the visible light image (S204).

[0058] The coordinate conversion unit 202 converts the position coordinates of the person's face in the visible light image into the position coordinates of the person's face in the thermal distribution image using the conversion formula stored in the storage 300 (S205).

[0059] The body temperature measurement unit 203 acquires a heat distribution image from the thermal camera 500 (S206).

[0060] The body temperature measurement unit 203 measures the temperature of the face (that is, body temperature) at the position of a specific part of the face in the thermal distribution image (S207).

[0061] The body temperature measurement unit 203 associates the person's face recognition result received from the face recognition unit 201 with the body temperature measurement result, and stores them in the storage 300 (S208). Thereafter, the body temperature measurement unit 203 notifies the face recognition unit 201 that the processing of step S208 has been completed.

[0062] Upon receiving the notification from the body temperature measurement unit 203, the face recognition unit 201 determines whether or not a predetermined condition is met (S209).

[0063] If the predetermined condition is not met (No in S209), the flow returns to step S201. On the other hand, if the predetermined condition is met (Yes in S209), the operation of the recognition body temperature measuring device 200 transitions to the operation of the information processing devices 10 to 40 described in the first to fourth embodiments.

[0064] In one example, the predetermined condition is that a certain amount of time has elapsed since the information processing devices 10 to 40 completed their previous operations. In this case, the recognition body temperature measuring device 200 uses a timer to measure the time elapsed since the information processing devices 10 to 40 completed their previous operations. However, if the information processing devices 10 to 40 have never operated, the recognition body temperature measuring device 200 determines whether a certain amount of time has elapsed since the timer was started. In another example, the predetermined condition is that an impact to the visible light camera 400 or the thermal camera 500 has been detected. In this case, an impact detection unit such as an acceleration sensor is provided for each of the visible light camera 400 and the thermal camera 500. The recognition body temperature measuring device 200 receives sensor data from the impact detection unit. When the impact detection unit detects an impact (large acceleration), the operation of the recognition body temperature measuring device 200 transitions to the operation of the information processing devices 10 to 40 described in embodiments 1 to 4.

[0065] This completes the operation of the cognitive body temperature measuring device 200.

[0066] [Embodiment 1] The first embodiment will be described with reference to FIGS.

[0067] (Information processing device 10) FIG. 7 is a block diagram showing the configuration of an information processing device 10 according to the first embodiment. As shown in FIG. 7, the information processing device 10 includes a calculation unit 12 and a detection unit 14. The calculation unit 12 calculates the movement of the visible light camera 400 relative to a predetermined position based on an image captured by the visible light camera 400. The detection unit 14 detects a positional deviation between the visible light camera 400 and the thermal camera 500 based on the amount of change in the positional relationship. The information processing device 10 will be described in detail below. The visible light camera 400 is an example of a first camera, and the thermal camera 500 is an example of a second camera.

[0068] The calculation unit 12 calculates the movement of the visible light camera 400 relative to a predetermined position based on the image captured by the visible light camera 400. The calculation unit 12 is an example of a calculation means.

[0069] In one example, the calculation unit 12 acquires a plurality of visible light images using a method similar to the method used by the default value setting unit 103 of the above-described alignment device 100. First, the calculation unit 12 acquires a plurality of visible light images by taking images consecutively multiple times using the visible light camera 400.

[0070] The calculation unit 12 extracts, as the background in the visible light image, an area that does not change or changes little between multiple visible light images obtained by multiple captures. In another example, the calculation unit 12 may extract the background from a single visible light image obtained by a single capture by the visible light camera 400 using a general image recognition technique. Alternatively, the background may be extracted manually instead of by the calculation unit 12.

[0071] The calculation unit 12 acquires default values ​​of the features of the background of the visible light image by referring to the storage 300. The calculation unit 12 matches the features extracted from the background of the visible light image with the default values ​​of the features of the background of the visible light image.

[0072] Through matching, calculation unit 12 quantitatively evaluates how much the background in the visible light image has changed compared to when the background was photographed by alignment device 100. Specifically, calculation unit 12 evaluates the difference between the features of the background in the visible light image and the default values ​​of the features of the background in the visible light image in the left-right and up-down directions within the visible light image.

[0073] For example, the background features of the visible light image are represented by feature points a i (i=0,1,2,...) is a feature vector A. On the other hand, the default value of the background feature of the visible light image is set to the feature point b i (i=0, 1, 2, ...) is used as the feature vector B. In this case, the calculation unit 12 calculates the feature point a i The position of (i=0,1,2,...) corresponds to the feature point b i For the position (i=0, 1, 2, ...), the number of pixels that differ in the left / right and up / down directions in the visible light image is evaluated.

[0074] The calculation unit 12 then calculates the movement of the visible light camera 400 relative to the predetermined position based on the change in the background of the visible light image evaluated as described above. Here, movement refers to rotation (either pan, tilt, or roll) and translation. The predetermined positions are the respective positions of the visible light camera 400 and thermal camera 500 when the alignment device 100 captures the background of the visible light image.

[0075] Specifically, the calculation unit 12 uses a camera alignment technique that is generally used for camera alignment. First, the calculation unit 12 calculates the feature point b i (i=0,1,2,...) and the corresponding feature point a i(i=0, 1, 2, ...) and the angle of rotation (either pan, tilt, or roll) and distance of translation of the visible light camera 400 are generated as simultaneous equations. The angle of rotation (either pan, tilt, or roll) and distance of translation of the visible light camera 400 are parameters of the simultaneous equations. The calculation unit 12 then solves the generated simultaneous equations using a matrix operation to obtain the angle of rotation and distance of translation of the visible light camera 400.

[0076] The calculation unit 12 outputs data on the movement of the visible light camera 400 relative to the predetermined position to the detection unit 14.

[0077] The detection unit 14 detects a positional deviation between the visible light camera 400 and the thermal camera 500 based on the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500. The detection unit 14 is an example of a detection means.

[0078] In one example, the detection unit 14 receives data on the movement of the visible light camera 400 relative to a predetermined position from the calculation unit 12. The detection unit 14 determines whether or not a positional shift has occurred based on the movement of the visible light camera 400. Specifically, if the amount of movement of the visible light camera 400 is equal to or greater than a predetermined value, the detection unit 14 determines that a positional shift has occurred in the visible light camera 400.

[0079] The predetermined value may be zero, or may be a value greater than zero. For example, if the change in the positional relationship between the visible light camera 400 and the thermal camera 500 is small, such as when the visible light camera 400 moves by 0.01 degrees in the Pan direction, the effect on measuring body temperature using the thermal distribution image is small. Therefore, in such a case, the detection unit 14 may determine that there is no relative movement of the visible light camera 400 with respect to the thermal camera 500.

[0080] The detection unit 14 detects the relative movement of the visible light camera 400 with respect to the thermal camera 500 as a positional deviation between the visible light camera 400 and the thermal camera 500. The positional deviation detection result is information indicating whether or not a positional deviation has been detected.

[0081] The relative movement of the visible light camera 400 with respect to the thermal camera 500 refers to the rotation (pan, tilt, or roll) of the visible light camera 400 when the thermal camera 500 is stationary. Alternatively, the relative movement of the visible light camera 400 with respect to the thermal camera 500 refers to the translation of the visible light camera 400 in a direction along the support (FIG. 2).

[0082] When the visible light camera 400 and the thermal camera 500 are moved parallel to each other without changing their shooting direction, or when the support (FIG. 2) that secures the visible light camera 400 and the thermal camera 500 rotates, the visible light camera 400 does not move relative to the thermal camera 500. Therefore, the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 is zero. In such cases, the detection unit 14 does not detect a positional deviation between the visible light camera 400 and the thermal camera 500. However, even in the above cases, it is expected that there will be a need to return the visible light camera 400 and the thermal camera 500 to their default positions (i.e., the positions before they were moved or rotated). Therefore, when the detection unit 14 detects that the visible light camera 400 has moved, it may notify the administrator to return the visible light camera 400 to its default position. Note that, if the information processing device 10 includes a notification unit 35 as in the third or fourth embodiment described below, the notification unit 35 may notify the administrator instead of the detection unit 14.

[0083] Here, for simplicity of explanation, it is assumed that only the visible light camera 400 moves. However, there is a possibility that both the visible light camera 400 and the thermal camera 500 move independently from their respective predetermined positions. Taking such a possibility into consideration, the detection unit 14 may detect a positional deviation between the visible light camera 400 and the thermal camera 500 based on the respective movements of the visible light camera 400 and the thermal camera 500 (a modified example described later). In this case, the method of calculating the movement of the thermal camera 500 based on the background features of the thermal distribution image is the same as the method of calculating the movement of the visible light camera 400 based on the background features of the visible light image.

[0084] (Operation of information processing device 10) The operation of the information processing device 10 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the flow of processing executed by each unit of the information processing device 10. The operation of the information processing device 10 starts when a predetermined condition is satisfied (Yes in step S209) in the operation of the cognitive body temperature measuring device 200 described above (Fig. 6).

[0085] 8, the calculation unit 12 obtains visible light images by capturing images using the visible light camera 400 (S1). The calculation unit 12 extracts, as the background in the visible light images, an area that does not change between multiple visible light images obtained by capturing images multiple times.

[0086] Next, the calculation unit 12 extracts features from the background of the visible light image (S2).

[0087] The calculation unit 12 calculates the movement of the visible light camera 400 by comparing the background features of the visible light image with predetermined values ​​recorded in the storage 300 (S3).

[0088] The detection unit 14 detects the relative movement of the visible light camera 400 with respect to the thermal camera 500 as a positional deviation between the visible light camera 400 and the thermal camera 500 (S4).

[0089] This completes the operation of the information processing device 10 according to the present embodiment 1. According to the above-described operation, the information processing device 10 can detect a positional deviation between two cameras in the camera system 1 that includes multiple cameras (here, the visible light camera 400 and the thermal camera 500).

[0090] (Variation) In one modified example, the information processing device 10 may focus on the movement of the thermal camera 500, rather than the movement of the visible light camera 400. In this modified example, the thermal camera 500 is an example of a first camera, and the visible light camera 400 is an example of a second camera.

[0091] In this modification, the calculation unit 12 calculates the movement of the thermal camera 500. The detection unit 14 detects the relative movement of the visible light camera 400 with respect to the thermal camera 500 as a positional shift between the visible light camera 400 and the thermal camera 500. In this way, in this modification as well, it is possible to detect a positional shift between the visible light camera 400 and the thermal camera 500.

[0092] In another modification, the information processing device 10 may detect a positional deviation between the visible light camera 400 and the thermal camera 500 based on the movements of both the visible light camera 400 and the thermal camera 500 .

[0093] In this modification, the calculation unit 12 calculates the movement of the thermal camera 500 relative to the predetermined position in addition to the movement of the visible light camera 400 relative to the predetermined position.

[0094] The detection unit 14 then detects the relative movement of the visible light camera 400 with respect to the thermal camera 500 as a positional deviation between the visible light camera 400 and the thermal camera 500.

[0095] According to the configuration of this modified example, it is possible to detect a positional shift between the visible light camera 400 and the thermal camera 500 even when only one of the visible light camera 400 and the thermal camera 500 moves, or even when both move.

[0096] (Effects of this embodiment) In the related technology described in Patent Document 1, if the relative positions of the visible light camera and the thermal camera are shifted, the above conversion formula becomes invalid. Therefore, it is not possible to accurately calculate the position coordinates of a person's face in the thermal camera's coordinate system from the position coordinates of the person's face in the visible light camera's coordinate system. As a result, there is a possibility that body temperature measurement using a thermal distribution image will fail.

[0097] On the other hand, according to the configuration of this embodiment, the calculation unit 12 calculates the movement of the visible light camera 400 (an example of a first camera) relative to a predetermined position based on an image captured by the visible light camera 400, and the detection unit 14 detects the positional deviation between the visible light camera 400 and the thermal camera 500 based on the amount of change in the positional relationship.

[0098] Alternatively, according to the configuration described in one variation of this embodiment, the thermal camera 500 corresponds to an example of a first camera, and the visible light camera 400 corresponds to an example of a second camera, which is the opposite of the configuration of this embodiment. The calculation unit 12 calculates the movement of the thermal camera 500 relative to a predetermined position based on an image captured by the thermal camera 500 (an example of a first camera). The detection unit 14 detects a positional deviation between the visible light camera 400 and the thermal camera 500 based on the amount of change in their positional relationship.

[0099] Alternatively, according to the configuration described in another modified example of this embodiment, both the visible light camera 400 and the thermal camera 500 correspond to an example of the first camera and also correspond to an example of the second camera. The calculation unit 12 calculates the movement of each of the visible light camera 400 and the thermal camera 500 relative to a predetermined position based on images captured by each of the visible light camera 400 and the thermal camera 500 (corresponding to both the first camera and the second camera). The detection unit 14 detects a positional deviation between the visible light camera 400 and the thermal camera 500 based on the amount of change in their positional relationship.

[0100] In this way, in a camera system 1 equipped with multiple cameras, by measuring the amount of change in the positional relationship between a first camera and a second camera, it is possible to detect the positional deviation between the two cameras.

[0101] [Embodiment 2] A second embodiment will be described with reference to Figures 9 and 10. In this second embodiment, a configuration for automatically correcting positional deviation between two cameras will be described.

[0102] (information processing device 20) Fig. 9 is a block diagram showing the configuration of an information processing device 20 according to the second embodiment. As shown in Fig. 9, the information processing device 20 includes a calculation unit 12, a measurement unit 13, and a detection unit 14. The information processing device 20 also includes a correction unit 25. The information processing device 20 differs in configuration from the information processing device 10 according to the first embodiment in that it also includes the measurement unit 13 and the correction unit 25.

[0103] In the second embodiment, the detection unit 14 outputs the detection result of the positional deviation between the visible light camera 400 and the thermal camera 500 to the correction unit 25.

[0104] The measurement unit 13 measures the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 based on the movement of the visible light camera 400. The measurement unit 13 is an example of a measurement means.

[0105] In one example, the measurement unit 13 receives data on the movement of the visible light camera 400 relative to a predetermined position from the calculation unit 12. Here, it is assumed that the thermal camera 500 has not moved from the predetermined position. In this case, the amount of movement of the visible light camera 400 relative to the predetermined position directly corresponds to the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500. The measurement unit 13 quantitatively evaluates how the positional relationship between the visible light camera 400 and the thermal camera 500 has changed based on the movement of the visible light camera 400. In this way, the measurement unit 13 measures the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500.

[0106] The measurement unit 13 outputs data on the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 to the correction unit 25. When a positional deviation between the visible light camera 400 (an example of a first camera) and the thermal camera 500 (an example of a second camera) is detected, the correction unit 25 corrects the transformation formula between the coordinate system of the visible light camera 400 and the coordinate system of the thermal camera 500. The correction unit 25 is an example of a correction means.

[0107] In one example, the correction unit 25 refers to the storage 300 (FIG. 1) to obtain a conversion formula between the coordinate system of the visible light camera 400 and the coordinate system of the thermal camera 500. The correction unit 25 receives data on the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 from the measurement unit 13. The correction unit 25 also receives a positional deviation detection result from the detection unit 14. The positional deviation detection result is information indicating whether or not a positional deviation has been detected.

[0108] When the detection unit 14 detects a positional deviation between the visible light camera 400 and the thermal camera 500, the correction unit 25 corrects the conversion formula based on data on the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500.

[0109] More specifically, the correction unit 25 corrects the parameters of the conversion formula so as to cancel the change in the positional relationship between the visible light camera 400 and the thermal camera 500. In this way, it is possible to obtain a conversion formula that indicates the correspondence between the position coordinates of the visible light image and the position coordinates of the thermal distribution image after the change in the positional relationship between the visible light camera 400 and the thermal camera 500 occurs.

[0110] (Operation of information processing device 20) The operation of the information processing device 20 will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the flow of processing executed by each unit of the information processing device 20. The operation of the information processing device 20 starts when a predetermined condition is satisfied (Yes in step S209) in the operation of the cognitive body temperature measuring device 200 described above (Fig. 6).

[0111] Steps S1 to S5 shown in FIG. 10 are common to steps S1 to S5 described with reference to FIG. 7 in the first embodiment.

[0112] 10, the calculation unit 12 obtains visible light images by capturing images using the visible light camera 400 (S1). The calculation unit 12 extracts, as the background in the visible light images, an area that does not change between multiple visible light images obtained by capturing images multiple times.

[0113] Next, the calculation unit 12 extracts features from the background of the visible light image (S2).

[0114] The calculation unit 12 calculates the movement of the visible light camera 400 by comparing the background features of the visible light image with predetermined values ​​recorded in the storage 300 (S3).

[0115] The detection unit 14 detects the relative movement of the visible light camera 400 with respect to the thermal camera 500 as a positional deviation between the visible light camera 400 and the thermal camera 500 (S4). The detection unit 14 outputs the detection result of the positional deviation between the visible light camera 400 and the thermal camera 500 (whether or not it has been detected) to the correction unit 25.

[0116] The measurement unit 13 measures the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 based on the movement of the visible light camera 400 (S5). The measurement unit 13 outputs data on the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 to the correction unit 25.

[0117] The correction unit 25 determines whether or not a positional deviation between the visible light camera 400 and the thermal camera 500 has been detected (S206).

[0118] If a positional deviation between the visible light camera 400 and the thermal camera 500 is detected (Yes in S206), the correction unit 25 corrects the conversion formula (S207). On the other hand, if a positional deviation between the visible light camera 400 and the thermal camera 500 is not detected (No in S206), the correction unit 25 does not correct the conversion formula.

[0119] This completes the operation of the information processing device 20 according to the second embodiment.

[0120] (Effects of this embodiment) According to the configuration of this embodiment, the calculation unit 12 calculates the movement of the visible light camera 400 (an example of a first camera) relative to a predetermined position based on an image captured by the visible light camera 400, and the detection unit 14 detects the positional deviation between the visible light camera 400 and the thermal camera 500 based on the amount of change in the positional relationship.

[0121] In this way, in a camera system 1 equipped with multiple cameras, by measuring the amount of change in the positional relationship between a first camera and a second camera, it is possible to detect the positional deviation between the two cameras.

[0122] Furthermore, according to the configuration of this embodiment, the measurement unit 13 measures the amount of change in the positional relationship between the visible light camera 400 (an example of a first camera) and the thermal camera 500 (an example of a second camera) based on the movement of the visible light camera 400. When a positional deviation between the visible light camera 400 (an example of a first camera) and the thermal camera 500 (an example of a second camera) is detected, the correction unit 25 corrects the conversion formula between the coordinate system of the visible light camera 400 and the coordinate system of the thermal camera 500. This makes it possible to obtain a conversion formula that indicates the correspondence between the position coordinates of the visible light image and the position coordinates of the thermal distribution image after a change in the positional relationship between the visible light camera 400 and the thermal camera 500 occurs.

[0123] [Embodiment 3] A third embodiment will be described with reference to Figures 11 and 12. In this third embodiment, a configuration will be described in which a positional deviation between two cameras is automatically reported (alert).

[0124] (Information processing device 30) Fig. 11 is a block diagram showing the configuration of an information processing device 30 according to the third embodiment. As shown in Fig. 11, the information processing device 30 includes a calculation unit 12 and a detection unit 14. The information processing device 20 further includes a notification unit 35. The information processing device 30 differs in configuration from the information processing device 10 according to the first embodiment in that it further includes the notification unit 35.

[0125] In the third embodiment, the detection unit 14 outputs the detection result of the positional deviation between the visible light camera 400 and the thermal camera 500 to the notification unit 35.

[0126] When a positional deviation between the visible light camera 400 (an example of a first camera) and the thermal camera 500 (an example of a second camera) is detected, the notification unit 35 notifies that a positional deviation has occurred between the visible light camera 400 and the thermal camera 500, and outputs information indicating the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500. The notification unit 35 is an example of a notification means.

[0127] In one example, the notification unit 35 receives a result of the detection of misalignment from the detection unit 14. The result of the detection of misalignment is information indicating whether or not a misalignment has been detected.

[0128] When the detection unit 14 detects a positional deviation between the visible light camera 400 and the thermal camera 500, the notification unit 35 notifies that a positional deviation has occurred between the visible light camera 400 and the thermal camera 500. Any notification method may be used. For example, the notification unit 35 outputs visible light images captured by the visible light camera 400 during the time that the visible light camera 400 was deviated from the predetermined position, and a list of objects captured by the visible light camera 400 during the time that the visible light camera 400 was deviated from the predetermined position.

[0129] For example, the notification unit 35 notifies the administrator of the alert by displaying a warning on the administrator's terminal or by flashing a lamp.

[0130] Additionally, the notification unit 35 may instruct the administrator by displaying information on which camera should be turned, in which direction, and by how many degrees, or may instruct the administrator by voice using a speaker, etc. In this case, the notification unit 35 receives data on the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 from the measurement unit 13, and generates instructions for the administrator based on the received data.

[0131] The notification unit 35, in response to an instruction from the administrator, causes the laser pointers provided on the visible light camera 400 and the thermal camera 500 to emit light. The laser pointers provided on the visible light camera 400 and the thermal camera 500 pass through the optical axes, i.e., the center of the angle of view, of the visible light camera 400 and the thermal camera 500. The administrator adjusts the optical axes of the visible light camera 400 and the thermal camera 500 while observing the direction of irradiation of the laser pointers of the visible light camera 400 and the thermal camera 500 so that they overlap at a position where a person's face is detected. The notification unit 35 may assist in correcting the position or orientation of the visible light camera 400 and the thermal camera 500. After the administrator manually corrects the position or orientation of the visible light camera 400 and the thermal camera 500, the notification unit 35 may stop issuing the alert.

[0132] Furthermore, the notification unit 35 may identify the time when the visible light camera 400 or the thermal camera 500 deviated from the predetermined position, and output a list of visible light images taken by the visible light camera 400 within the identified time, or a list of targets whose faces were recognized by the face recognition unit 201 of the recognition body temperature measuring device 200 within the identified time.

[0133] When the visible light camera 400 and the thermal camera 500 are moved in parallel together, or when the support (FIG. 2) that supports the visible light camera 400 and the thermal camera 500 is rotated, the visible light camera 400 does not move relative to the thermal camera 500. In such cases, the detection unit 14 does not detect a positional deviation between the visible light camera 400 and the thermal camera 500. Therefore, the notification unit 35 does not send an alert to the administrator's terminal.

[0134] (Operation of information processing device 30) The operation of the information processing device 30 will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the flow of processing executed by each unit of the information processing device 30. The operation of the information processing device 30 starts when a predetermined condition is satisfied (Yes in step S209) in the operation of the cognitive body temperature measuring device 200 described above (Fig. 6).

[0135] Steps S1 to S5 shown in FIG. 12 are common to steps S1 to S5 described with reference to FIG. 7 in the first embodiment.

[0136] 12, the calculation unit 12 obtains visible light images by capturing images using the visible light camera 400 (S1). The calculation unit 12 extracts, as the background in the visible light images, an area that does not change between multiple visible light images obtained by capturing images multiple times.

[0137] Next, the calculation unit 12 extracts features from the background of the visible light image (S2).

[0138] The calculation unit 12 calculates the movement of the visible light camera 400 by comparing the background features of the visible light image with predetermined values ​​recorded in the storage 300 (S3).

[0139] The measurement unit 13 measures the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 based on the movement of the visible light camera 400 (S4).

[0140] The detection unit 14 detects the relative movement of the visible light camera 400 with respect to the thermal camera 500 as a positional shift between the visible light camera 400 and the thermal camera 500 (S5). The detection unit 14 outputs the detection result of the positional shift between the visible light camera 400 and the thermal camera 500 (whether or not it has been detected) to the notification unit 35.

[0141] The notification unit 35 determines whether or not a positional deviation between the visible light camera 400 and the thermal camera 500 has been detected (S306).

[0142] If a positional deviation between the visible light camera 400 and the thermal camera 500 is detected (Yes in S306), the notification unit 35 notifies that a positional deviation between the visible light camera 400 and the thermal camera 500 has occurred (S307). On the other hand, if a positional deviation between the visible light camera 400 and the thermal camera 500 is not detected (No in S306), the notification unit 35 does not issue a notification.

[0143] This completes the operation of the information processing device 30 according to the third embodiment.

[0144] (Effects of this embodiment) According to the configuration of this embodiment, the calculation unit 12 calculates the movement of the visible light camera 400 (an example of a first camera) relative to a predetermined position based on an image captured by the visible light camera 400, and the detection unit 14 detects the positional deviation between the visible light camera 400 and the thermal camera 500 based on the amount of change in the positional relationship.

[0145] In this way, in a camera system 1 equipped with multiple cameras, by measuring the amount of change in the positional relationship between a first camera and a second camera, it is possible to detect the positional deviation between the two cameras.

[0146] Furthermore, according to the configuration of this embodiment, when a positional deviation between the visible light camera 400 (an example of a first camera) and the thermal camera 500 (an example of a second camera) is detected, the notification unit 35 notifies that a positional deviation has occurred between the visible light camera 400 and the thermal camera 500. This allows the administrator to be prompted to manually correct the position or orientation of the visible light camera 400 / thermal camera 500.

[0147] [Embodiment 4] A fourth embodiment will be described with reference to Figures 13 and 14. In the fourth embodiment, a configuration will be described in which, based on the magnitude of the change in the positional relationship between the first camera and the second camera, either automatic correction is performed as in the third embodiment, or automatic notification (alert) is issued as in the fourth embodiment.

[0148] (information processing device 40) Fig. 13 is a block diagram showing the configuration of an information processing device 40 according to the fourth embodiment. As shown in Fig. 13, the information processing device 40 includes a calculation unit 12, a measurement unit 13, and a detection unit 14'. The information processing device 40 also includes a correction unit 25 and a notification unit 35. The information processing device 40 differs in configuration from the information processing device 10 according to the first embodiment in that it also includes the measurement unit 13, the correction unit 25, and the notification unit 35.

[0149] When a positional deviation between the visible light camera 400 (an example of a first camera) and the thermal camera 500 (an example of a second camera) is detected, the correction unit 25 corrects the transformation formula between the coordinate system of the visible light camera 400 and the coordinate system of the thermal camera 500. The correction unit 25 is an example of a correction means.

[0150] When a positional deviation between the visible light camera 400 (an example of a first camera) and the thermal camera 500 (an example of a second camera) is detected, the notification unit 35 notifies that a positional deviation has occurred between the visible light camera 400 and the thermal camera 500, and outputs information indicating the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500. The notification unit 35 is an example of a notification means.

[0151] As described above, in the present embodiment 4, the process executed by the correction unit 25 is the same as that executed by the correction unit 25 in the above-described embodiment 2. Furthermore, in the present embodiment 4, the process executed by the notification unit 35 is the same as that executed by the notification unit 35 in the above-described embodiment 3.

[0152] In the fourth embodiment, the measurement unit 13 measures the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 based on the movement of the visible light camera 400. The measurement unit 13 outputs data on the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 to the detection unit 14' and the correction unit 25. The detection unit 14' outputs a positional deviation detection result to either the correction unit 25 or the notification unit 35 based on the magnitude of the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500.

[0153] More specifically, when the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 is equal to or less than the threshold, the detection unit 14' outputs the positional deviation detection result to the correction unit 25. On the other hand, when the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 exceeds the threshold, the detection unit 14' outputs the positional deviation detection result to the notification unit 35.

[0154] (Operation of information processing device 40) The operation of the information processing device 40 will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the flow of processing executed by each unit of the information processing device 40. The operation of the information processing device 40 is started when a predetermined condition is satisfied (Yes in step S209) in the operation of the cognitive body temperature measuring device 200 described above (Fig. 6).

[0155] Steps S1 to S5 shown in FIG. 14 are common to steps S1 to S5 described with reference to FIG. 7 in the first embodiment.

[0156] 14, the calculation unit 12 obtains visible light images by capturing images using the visible light camera 400 (S1). The calculation unit 12 extracts, as the background in the visible light images, an area that does not change between multiple visible light images obtained by capturing images multiple times.

[0157] Next, the calculation unit 12 extracts features from the background of the visible light image (S2).

[0158] The calculation unit 12 calculates the movement of the visible light camera 400 by comparing the background features of the visible light image with predetermined values ​​recorded in the storage 300 (S3).

[0159] The detection unit 14' detects the relative movement of the visible light camera 400 with respect to the thermal camera 500 as a positional deviation between the visible light camera 400 and the thermal camera 500 (S4).

[0160] The measurement unit 13 measures the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 based on the movement of the visible light camera 400 (S5).

[0161] The detection unit 14' determines whether the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 is equal to or less than a threshold value (S406).

[0162] If the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 is less than or equal to the threshold value (Yes in S406), the detection unit 14' outputs the detection result of the positional deviation between the visible light camera 400 and the thermal camera 500 to the correction unit 25.

[0163] The correction unit 25 corrects the transformation formula between the coordinate system of the visible light camera 400 and the coordinate system of the thermal camera 500 based on the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 (S407A). Note that a case where no positional deviation between the visible light camera 400 and the thermal camera 500 is detected is also included when the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 is equal to or less than the threshold. In this case, the correction unit 25 does not need to correct the transformation formula.

[0164] If the change in the positional relationship between the visible light camera 400 and the thermal camera 500 exceeds the threshold value (No in S406), the detection unit 14' outputs the detection result of the positional deviation between the visible light camera 400 and the thermal camera 500 to the notification unit 35.

[0165] The notification unit 35 notifies that a positional deviation has occurred between the visible light camera 400 and the thermal camera 500 (S407B).

[0166] This completes the operation of the information processing device 40 according to the fourth embodiment.

[0167] (Effects of this embodiment) According to the configuration of this embodiment, the calculation unit 12 calculates the movement of the visible light camera 400 (an example of a first camera) relative to a predetermined position based on an image captured by the visible light camera 400, the measurement unit 13 measures the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 (an example of a second camera) based on the movement of the visible light camera 400, and the detection unit 14' detects the positional deviation between the visible light camera 400 and the thermal camera 500 based on the amount of change in the positional relationship.

[0168] In this way, in a camera system 1 equipped with multiple cameras, by measuring the amount of change in the positional relationship between a first camera and a second camera, it is possible to detect the positional deviation between the two cameras.

[0169] Furthermore, according to the configuration of this embodiment, when the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 is equal to or less than a threshold, the detection unit 14' outputs a positional deviation detection result to the correction unit 25. In this case, the correction unit 25 corrects the conversion formula between the coordinate system of the visible light camera 400 and the coordinate system of the thermal camera 500. This makes it possible to obtain a conversion formula that indicates the correspondence between the position coordinates of the visible light image and the position coordinates of the thermal distribution image after a change in the positional relationship between the visible light camera 400 and the thermal camera 500 occurs.

[0170] On the other hand, if the amount of change in the positional relationship between the visible light camera 400 and the thermal camera 500 exceeds the threshold, the detection unit 14' outputs the positional deviation detection result to the notification unit 35. In this case, the notification unit 35 notifies that a positional deviation has occurred between the visible light camera 400 and the thermal camera 500. This makes it possible to prompt the administrator to manually correct the position or orientation of the visible light camera 400 / thermal camera 500.

[0171] (About hardware configuration) Each of the components of the information processing devices 10 to 40 described in the first to fourth embodiments represents a functional block. Some or all of these components are realized by an information processing device 900 as shown in Fig. 15. Fig. 15 is a block diagram showing an example of the hardware configuration of the information processing device 900.

[0172] As shown in FIG. 15, the information processing device 900 includes the following configuration, for example.

[0173] ·CPU(Central Processing Unit)901 ROM (Read Only Memory) 902 ·RAM(Random Access Memory)903 Program 904 loaded into RAM 903 A storage device 905 for storing a program 904 A drive device 907 for reading and writing data from and to the recording medium 906 A communication interface 908 for connecting to a communication network 909 Input / output interface 910 for inputting and outputting data Bus 911 connecting each component

[0174] Each of the components of the information processing devices 10 to 40 described in the first to fourth embodiments is realized by the CPU 901 reading and executing a program 904 that realizes the function of each component. The program 904 that realizes the function of each component is stored, for example, in advance in the storage device 905 or the ROM 902, and is loaded into the RAM 903 and executed by the CPU 901 as needed. The program 904 may be supplied to the CPU 901 via the communication network 909, or may be stored in advance in the recording medium 906, and the drive device 907 may read out the program and supply it to the CPU 901.

[0175] According to the above configuration, the information processing devices 10 to 40 described in the first to fourth embodiments are realized as hardware, and therefore the same effects as those described in the first to fourth embodiments can be achieved.

[0176] [Note] One embodiment of the present invention is also described as in the following supplementary notes, but is not limited to the configuration described below.

[0177] (Appendix 1) a calculation means for calculating a motion of the first camera relative to a predetermined position based on an image captured by the first camera; a measuring means for measuring a change in the positional relationship between the first camera and the second camera based on the movement of the first camera; a detection means for detecting a positional deviation between the first camera and the second camera based on the amount of change in the positional relationship; An information processing device comprising:

[0178] (Appendix 2) The calculation means extracts features from the background of the image captured by the first camera, and calculates the motion of the first camera by comparing the features extracted from the background of the image with pre-recorded features of the background. 2. The information processing device according to claim 1,

[0179] (Appendix 3) a measuring means for measuring a change in the positional relationship between the first camera and the second camera based on the movement of the first camera; The image capturing apparatus further includes a correction unit that corrects a transformation formula between the coordinate system of the first camera and the coordinate system of the second camera when a positional deviation between the first camera and the second camera is detected. 3. The information processing device according to claim 1 or 2.

[0180] (Appendix 4) When an impact occurs to the first camera, each unit of the information processing device starts processing. 4. The information processing device according to claim 1, wherein:

[0181] (Appendix 5) The camera further includes a notification unit that notifies a user that a positional deviation has occurred between the first camera and the second camera when the positional deviation between the first camera and the second camera is detected, and outputs information indicating the amount of change in the positional relationship between the first camera and the second camera. 5. The information processing device according to claim 1, wherein:

[0182] (Appendix 6) The notification means The image captured by the first camera during a time period in which the first camera was displaced from the predetermined position; and a list of objects photographed by the first camera during the time the first camera was displaced from the predetermined position; Output 6. The information processing device according to claim 5,

[0183] (Appendix 7) The predetermined position is identified by imaging the human body with the first camera. 7. The information processing device according to any one of Supplementary Notes 1 to 6,

[0184] (Appendix 8) The first camera and the second camera are a visible light camera capable of capturing images in the visible light band and a thermal camera capable of measuring body temperature, respectively. 8. The information processing device according to any one of Supplementary Notes 1 to 7,

[0185] (Appendix 9) The transformation equation between the coordinate system of the first camera and the coordinate system of the second camera is obtained by calibrating the first camera and the second camera using a common marker. 4. The information processing device according to claim 3,

[0186] (Appendix 10) Calculating a motion of the first camera relative to a predetermined position based on images captured by the first camera; Detecting a positional deviation between the first camera and the second camera based on the movement of the first camera. Information processing methods.

[0187] (Appendix 11) calculating a motion of the first camera relative to a predetermined position based on an image captured by the first camera; detecting a positional deviation between the first camera and the second camera based on the movement of the first camera; A program that causes a computer to execute the following.

[0188] (Appendix 12) An information processing device according to any one of Supplementary Notes 1 to 9; a setting alignment device that defines the default position when the first camera and the second camera are fixed in a pre-positioned state; a recognition body temperature measurement device that uses the first camera and the second camera to recognize a person's face and measure their body temperature; A camera system with [Explanation of symbols]

[0189] 1 camera system 10. Information processing equipment 11 Acquisition Department 12 Calculation section 13 Measuring part 14(14´) Detector 20 Information processing equipment 25 Correction unit 30 Information processing equipment 35 Notification Department 40 Information processing equipment 100 Alignment device 200 Recognition body temperature measuring device

Claims

1. a calculation means for calculating a motion of the first camera relative to a predetermined position based on an image captured by the first camera; a detection means for detecting a positional deviation between the first camera and the second camera based on the movement of the first camera; Equipped with a notification unit configured to notify the occurrence of a positional deviation between the first camera and the second camera when a positional deviation between the first camera and the second camera is detected, and to output information indicating an amount of change in the positional relationship between the first camera and the second camera; The notification means The image captured by the first camera during a time period in which the first camera was displaced from the predetermined position; and a list of objects photographed by the first camera during the time the first camera was displaced from the predetermined position; Output Information processing device.

2. The calculation means extracts features from the background of the image captured by the first camera, and calculates the motion of the first camera by comparing the features extracted from the background of the image with pre-recorded features of the background.

2. The information processing apparatus according to claim 1, wherein:

3. a measuring means for measuring a change in the positional relationship between the first camera and the second camera based on the movement of the first camera; and a correction unit that corrects a transformation formula between the coordinate system of the first camera and the coordinate system of the second camera when a positional deviation between the first camera and the second camera is detected.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

4. When an impact occurs to one or more of the first camera and the second camera, each unit of the information processing device starts processing.

4. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

5. Calculating a motion of the first camera relative to a predetermined position based on images captured by the first camera; An information processing method for detecting a positional deviation between the first camera and a second camera based on a movement of the first camera, comprising: when a positional deviation between the first camera and the second camera is detected, notifying that a positional deviation has occurred between the first camera and the second camera, and outputting information indicating an amount of change in the positional relationship between the first camera and the second camera; When notifying that a positional deviation has occurred between the first camera and the second camera, The image captured by the first camera during a time period in which the first camera was displaced from the predetermined position; and a list of objects photographed by the first camera during the time the first camera was displaced from the predetermined position; Output Information processing methods.

6. calculating a motion of the first camera relative to a predetermined position based on an image captured by the first camera; detecting a positional deviation between the first camera and the second camera based on the movement of the first camera; A program for causing a computer to execute the above, The computer, when a positional deviation between the first camera and the second camera is detected, a process of notifying that a positional deviation has occurred between the first camera and the second camera and outputting information indicating an amount of change in the positional relationship between the first camera and the second camera is further executed; When notifying that a positional deviation has occurred between the first camera and the second camera, The image captured by the first camera during a time period in which the first camera was displaced from the predetermined position; and a list of objects photographed by the first camera during the time the first camera was displaced from the predetermined position; Output program.

Images