Information processing device, projection device, system, control method, and program

Abstract

To provide an information processing device capable of solving difficulty of viewing of a presentation material.SOLUTION: An information processing device comprises: first detection means of detecting a first region indicating a region of at least one part region of a human body from an image imaged by imaging means; second detection means of detecting a second region indicating a region of at least one part region of the image projected by the projection means from the image imaged; and outputting means of generating and outputting control information for controlling the projection means so that the region where the first region and the second region are overlapped is reduced.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing device, a projection device, a system, a control method, and a program. [Background technology]

[0002] In recent years, there are increasing opportunities for students to attend lectures and the like given by presenters using presentation materials via a network. An example of such a lecture is an online class where a lecture is attended from a remote location via a network. Viewers can attend a lecture with the same sense of realism as in a lecture room by watching the distributed video using a smartphone, tablet, or the like. For example, the distributed video is a video captured by a camera installed in a lecture room of a presenter such as a lecturer giving a lecture using presentation materials. In addition, the presentation materials used by the presenter are increasingly being transmitted as images stored in an information processing device or the like to a projector and projected from the projector onto a blackboard, whiteboard, or the like.

[0003] However, there was an issue that depending on the position and angle of view of the camera recording the lesson, the presentation materials could be difficult to see in the streamed video.

[0004] Patent Document 1 discloses a technique for automatically adjusting the angle of view by detecting non-descriptive matter and controlling the pan mechanism and zoom mechanism of a camera. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2010-278916 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, with the conventional technology disclosed in the above-mentioned patent document, it is difficult to eliminate the difficulty in viewing the presentation materials that occurs when the presenter and the presentation materials overlap when both are imaged together.

[0007] SUMMARY OF THE PRESENT EMBODIMENTS An object of the present invention is to provide an information processing device, a projection device, a system, a control method, and a program that can eliminate the difficulty in viewing presentation materials. [Means for solving the problem]

[0008] In order to solve this problem, for example, an information processing device of the present invention includes the following configuration: a first detection unit that detects a first region indicating at least a part of a region of a human body from an image captured by an imaging unit; a second detection means for detecting a second region indicating at least a part of an image projected by the projection means from the captured image; The projection device further includes an output unit that generates and outputs control information for controlling the projection unit so as to reduce an overlapping area between the first area and the second area. Effect of the Invention

[0009] According to the present invention, it is possible to provide an information processing device, a projection device, a system, a control method, and a program that can eliminate the difficulty in viewing presentation materials regardless of the position of the presenter. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic configuration diagram of a presentation system according to a first embodiment. [Diagram 2] FIG. 1 is a block diagram showing the configuration of an information processing device according to a first embodiment. [Diagram 3] 5A to 5C are views showing an example of detection of a presenter region in the first embodiment. [Figure 4] 5A to 5C are diagrams showing an example of detection of a presentation material area in the first embodiment. [Diagram 5]5A to 5C are views showing an example of detection of a projectable area in the first embodiment. [Figure 6] 6 is a flowchart illustrating a generation process of projection control information in the first embodiment. [Figure 7] 5A to 5C are views showing an example of overlap determination in the first embodiment. [Figure 8] FIG. 4 is a diagram showing an example of a distributed video before application of processing in the first embodiment. [Figure 9] FIG. 11 is a block diagram showing the configuration of a presentation system according to a second embodiment. [Figure 10] 10 is a flowchart illustrating a projection process of a projection device according to a second embodiment. [Figure 11] FIG. 11 is a block diagram showing the configuration of a presentation system according to a third embodiment. [Figure 12] 13 is a flowchart illustrating a generation process of projection control information according to the third embodiment. [Figure 13] 13A to 13C are diagrams illustrating an example of processing of an image capture angle of view in the third embodiment. [Figure 14] FIG. 2 is a configuration diagram illustrating hardware of an information processing device. [Figure 15] 13A and 13B are diagrams illustrating an example of overlap determination between a second frame and a third frame in the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0012] <First embodiment> The presentation system according to the first embodiment will be described below with reference to FIGS.

[0013] Fig. 1 is a schematic configuration diagram of a presentation system including an information processing device according to a first embodiment. The presentation system in Fig. 1 includes an imaging device 10 as an imaging means, a projection device 20 as a projector, an information processing device 30, and a terminal 40. The imaging device 10, the projection device 20, the information processing device 30, and the terminal 40 are connected via a network 80 so as to be able to transmit and receive images, control information, and the like to each other.

[0014] The imaging device 10 is, for example, a PTZ (pan-tilt-zoom) camera capable of changing the imaging angle of view, and is installed at a predetermined position in a lecture room to capture images of a presenter 50, which is an example of a human body, and presentation materials 60. The imaging device 10 generates and stores the captured images. The imaging device 10 transmits the captured images to the information processing device 30 via the network 80.

[0015] The projection device 20 is, for example, a projector that projects an image acquired from the information processing device 30 onto the projection board 70. The projection device 20 may be capable of changing the projection angle of view. In order to change the projection angle of view, the projection device 20 includes a shift mechanism for moving the projection position of the image up, down, left, and right, and a zoom mechanism for enlarging or reducing the projection size of the image. The projection device 20 acquires, via the network 80, presentation materials 60 used by the presenter 50 in the lecture, and control commands for controlling the projection, from the information processing device 30. The projection device 20 controls the projection angle of view, etc., according to the control commands, and projects images of the presentation materials 60, etc. onto the projection board 70 in the lecture room.

[0016] The information processing device 30 is, for example, a computer such as a personal computer, and is connected to the imaging device 10 and the projection device 20 via a network 80. The information processing device 30 can display and distribute the captured image received from the imaging device 10. The information processing device 30 also transmits an image of a presentation material 60 prepared and saved in advance by the presenter 50 to the projection device 20 for display. The information processing device 30 can switch the presentation material 60 in response to an operation input by the presenter 50 or another operator. In this embodiment, the imaging device 10 and the information processing device 30 are described as separate devices, but the imaging device 10 may be included in the information processing device 30 by the information processing device 30 assuming the role of the imaging device 10.

[0017] The terminal 40 is, for example, a smartphone, and receives and displays the video distributed by the information processing device 30 via the network 80. Viewers of the terminal 40 can attend lectures from remote locations by live distribution that allows viewing in real time, or on-demand distribution that allows viewing at any time.

[0018] Fig. 2 is a block diagram of an information processing device 30 in the first embodiment. The information processing device 30 in Fig. 2 has an I / F unit 301 as an image input unit, a presenter detection unit 302 as a first detection unit, a document detection unit 303 as a second detection unit, a projection detection unit 304 as a third area detection unit, and a control information output unit 305 as an output unit.

[0019] The I / F (Interface) unit 301 is an interface for transmitting and receiving video signals and control signals via the network 80. The I / F unit 301 receives captured images from the imaging device 10, and transmits the captured images to a presenter detection unit 302, a material detection unit 303, and a projection detection unit 304.

[0020] The presenter detection unit 302 detects a first region, which is at least a part of the region of the presenter 50, by analyzing the captured image received from the I / F unit 301. The detection target of the presenter detection unit 302 is, for example, the face, upper body, and whole body of the presenter 50. Fig. 3(A) to Fig. 3(C) are diagrams showing an example of detection of the presenter region. Fig. 3(A) is a detection example when the detection target is a face. Fig. 3(B) is a detection example when the detection target is the upper body. Fig. 3(C) is a detection example when the detection target is the whole body. For the sake of explanation, Fig. 3(A) to Fig. 3(C) display a dashed rectangle superimposed as a frame of the detection target (hereinafter, first frame 400), but the first frame 400 does not need to be superimposed as long as the center coordinate of the detection target, the width of the first frame 400, and the height of the first frame 400 can be detected as the region information of the first region. The presenter detection unit 302 transmits the region information of the detected first region to the control information output unit 305 as the first region information.

[0021] The document detection unit 303 detects a second region, which is at least a part of the presentation document 60, from the image received from the I / F unit 301. The detection target of the document detection unit 303 is, for example, the region of the projected image and the region of text and figures. For example, a method of determining the region of the projected image based on the luminance difference between the projection board 70 and the projected image may be used to detect the region of text and figures. For example, pattern matching may be used to detect the region of text and figures. Here, an example of region detection is shown, but the method is not limited to this as long as a similar effect is obtained. Figures 4(A) and 4(B) are diagrams showing an example of detection of the presentation document region. Figure 4(A) is a detection example when the detection target detects the entire region of the projected image as the second region. Figure 4(B) is a detection example when the detection target detects the region of text and figures included in the projected image as the second region. 4(A) and 4(B) show a dashed rectangle superimposed as a frame of the detection target (hereinafter, second frame 402) for the sake of explanation, but as long as the center coordinates of the detection target, the width of the second frame 402, and the height of the second frame 402 can be detected as area information of the second area, the second frame 402 does not have to be superimposed. The document detection unit 303 transmits the area information of the detected second area to the control information output unit 305 as second area information.

[0022] The projection detection unit 304 detects a third area, which is an area where the projection board 70 can project, by analyzing the image received from the I / F unit 301. The projection board 70, which is the detection target of the projection detection unit 304, is, for example, a blackboard, a whiteboard, or a screen. The detection target is not limited to this as long as it is an area where the projection device 20 can project. Figs. 5(A) to 5(C) are diagrams showing an example of detection of a projectable area. Fig. 5(A) is a detection example when the detection target is a blackboard. Fig. 5(B) is a detection example when the detection target is a whiteboard. Fig. 5(C) is a detection example when the detection target is a screen. For the purpose of explanation, Figs. 5(A) to 5(C) display a dashed rectangle superimposed as a frame of the detection target (hereinafter, third frame 404), but the third frame 404 does not need to be superimposed if the center coordinates of the detection target, the width of the third frame 404, and the height of the third frame 404 can be detected as area information of the third area. The projection detection unit 304 transmits area information of the detected third area to the control information output unit 305 as third area information.

[0023] The control information output unit 305 generates and outputs projection control information for controlling the projection device 20 so as to reduce an overlapping area between the first and second regions based on the first region indicated by the first region information received from the presenter detection unit 302 and the second region indicated by the second region information received from the document detection unit 303. The control information output unit 305 generates, for example, information for controlling the projection angle of view of the projection device 20 as the projection control information. For example, the control information output unit 305 calculates a first frame that surrounds at least a part of the presenter 50 from the first region information, and a second frame that surrounds at least a part of the presentation document 60 from the second region information. The control information output unit 305 may generate projection control information for controlling the projection angle of view so as to reduce an overlapping area between the first frame and the second frame.

[0024] Furthermore, the control information output unit 305 generates projection control information for controlling the projection angle of view based on the second region indicated by the second region information and the third region indicated by the third region information received from the projection detection unit 304. For example, the control information output unit 305 calculates a third frame surrounding the projection plate 70 from the third region information. The control information output unit 305 may correct and update the projection control information generated based on the first region and the second region so that the second frame is superimposed on the third frame, in other words, so that the second frame is included in the third frame.

[0025] Fig. 14 is a configuration diagram for explaining hardware of the information processing device 30. As shown in Fig. 14, the information processing device 30 includes a CPU 350, a memory 352, a storage 354, an external I / F 356, a network I / F 358, a display unit 360, and a bus 362. The CPU 350, the memory 352, the storage 354, the external I / F 356, the network I / F 358, and the display unit 360 are connected via the bus 362 so as to be able to transmit and receive data to and from each other.

[0026] The CPU 350 is an abbreviation of Central Processing Unit, and is a programmable processor that processes various calculations in the information processing device 30. For example, the CPU 350 expands a program stored in the storage 354 into the memory 352, executes the program, and realizes various functions. In this embodiment, the CPU 350 executes a program that generates projection control information, and realizes the functions of the I / F unit 301, the presenter detection unit 302, the document detection unit 303, the projection detection unit 304, and the control information output unit 305 described above. Note that the information processing device 30 may have other programmable processors such as an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), and a QPU (Quantum Processing Unit) instead of the CPU 350 or in addition to the CPU 350. Also, the information processing device 30 may have an ASIC (Application Specific Integrated Circuit) instead of the programmable processor or in addition to the programmable processor. In this case, the information processing device 30 may have another programmable processor or ASIC realize some of the functions of the above-mentioned I / F unit 301, presenter detection unit 302, material detection unit 303, projection detection unit 304 and control information output unit 305.

[0027] The memory 352 is a volatile storage device such as a random access memory (RAM). The memory 352 is capable of reading and writing faster than the storage 354. The memory 352 temporarily stores the programs executed by the CPU 350, data necessary for the execution of the programs, and the like.

[0028] The storage 354 is, for example, one or more non-volatile storage devices selected from a read only memory (ROM), a hard disk drive (HDD), a solid state drive (SSD), etc. The storage 354 has a larger capacity than the memory 352. The storage 354 stores programs and the like, and deploys them in the memory 352 in accordance with commands from the CPU 350.

[0029] The external I / F 356 is, for example, an interface such as a USB (Universal Serial Bus). The external I / F 356 can be connected to an input device such as a keyboard or a mouse, and an external storage device such as an external memory. The external I / F 356 receives input from a user such as the presenter 50 via the input device, and outputs the input to the CPU 350.

[0030] The network I / F 358 is, for example, an RJ-45 terminal into which a network cable connected to the network 80 is inserted. The network I / F 358 outputs data such as images received via the network 80 to the CPU 350, and transmits images, control commands, and the like to the imaging device 10 or the projection device 20 via the network 80 in response to commands from the CPU 350.

[0031] The display unit 360 is, for example, a display device for displaying images such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 360 displays, for example, a captured image acquired from the imaging device 10 and an input screen for receiving input from a user. The display unit 360 may be an external display device connected via the external I / F 356.

[0032] The hardware configurations of the imaging device 10, the projection device 20, and the terminal 40 are almost the same as that of the information processing device 30, and therefore a description thereof will be omitted.

[0033] Hereinafter, the generation process of the projection control information by the control information output unit 305 in this embodiment will be described with reference to the flowchart in Fig. 6. This process is started when at least one of the first region information, the second region information, and the third region information is updated. Note that when there is no update to the region information, the processes of S11, S12, and S16 corresponding to the region of the first region information, the second region information, and the third region information that has not been updated may be omitted.

[0034] In S11, the control information output unit 305 calculates information about the first frame 400 from first region information, which is information about the region of the presenter 50. The information about the first frame 400 may be, for example, coordinate information in the captured image that can identify the position of the first frame 400. First, when the resolution of the captured image is 1920 x 1080, the coordinates of the top left corner of the captured image are set to (0, 0) and the coordinates of the bottom right corner are set to (1919, 1079). Next, the first region information is composed of center coordinates (x, y), width (w), and height (h). In this case, the control information output unit 305 calculates the following as the information about the first frame 400: The upper left coordinate of the first frame 400 is calculated as (xw / 2, yh / 2), and the lower right coordinate of the first frame 400 is calculated as (x+w / 2, y+h / 2). The control information output unit 305 can calculate the first frame 400 in the captured image from the above values. When the control information output unit 305 completes the calculation process of the first frame 400, it proceeds to S12.

[0035] In S12, the control information output unit 305 calculates information about the second frame 402 from second area information, which is information about the area of ​​the presentation material 60. The information about the second frame 402 may be, for example, coordinate information in the captured image that can identify the position of the second frame 402. The calculation of the information about the second frame 402 is similar to the calculation method for the information about the first frame 400, so a description thereof will be omitted. When the control information output unit 305 completes the calculation process for the second frame 402, the process proceeds to S13.

[0036] In S13, the control information output unit 305 judges whether the first frame 400 and the second frame 402 overlap based on the information of the first frame 400 calculated in S11 and the information of the second frame 402 calculated in S12. FIG. 7(A) and FIG. 7(B) are diagrams showing an example of the overlap judgment of the first frame 400 and the second frame 402. The control information output unit 305 first detects an overlap area 410, which is an area where the first frame 400 and the second frame 402 overlap, from the coordinates of the first frame 400 surrounding the presenter 50 and the coordinates of the second frame 402 surrounding the presentation material 60. In FIG. 7(A) and FIG. 7(B), the first frame 400 and the second frame 402 are shown by dashed lines, and the overlap area 410 is shown by a filled area. Next, the control information output unit 305 calculates the ratio of the overlap area 410 in the first frame 400 or the second frame 402. For example, the control information output unit 305 may calculate the ratio of the overlapping area 410 within the frame of the second frame 402 (=area of ​​the overlapping area 410 ÷ area of ​​the second frame 402) as the ratio of the overlapping area 410. The control information output unit 305 compares the calculated ratio of the overlapping area 410 with a predetermined threshold and determines whether the ratio exceeds the threshold, thereby determining whether the first frame 400 and the second frame 402 overlap. In this case, the threshold may be 0.

[0037] Here, the threshold value may be changed to a value larger than the above threshold value, for example, a value larger than 0, so that the control information output unit 305 determines that the first frame and the second frame do not overlap even if they overlap to a certain extent. In other words, by increasing the threshold value, a dead zone is provided, and it is possible to prevent the control information output unit 305 from determining that the projection angle of view is changed even if the presentation material 60 is visible to the audience. For example, as shown in FIG. 7(A), if the ratio of the overlapping area 410 does not exceed the threshold value, the control information output unit 305 does not determine that the first frame 400 and the second frame 402 overlap. On the other hand, as shown in FIG. 7(B), if the ratio of the overlapping area 410 exceeds the threshold value, the control information output unit 305 may determine that the first frame 400 and the second frame 402 overlap. As a result, for example, if the first frame 400 overlaps part of the blank area on the periphery of the second frame 402 that does not contain the text of the presentation material 60, i.e., if the viewer can view the presentation material 60 without any problems, the projection angle of view will not be changed.

[0038] As another method of providing the dead zone, the document detection unit 303 may use a method of resetting the second frame 402 so that it is a predetermined amount inside the second frame 402 calculated in S12. In this case, the document detection unit 303 may reset the second frame 402 inside the initially set second frame 402 in the presentation document 60 and outside the area including the characters. The document detection unit 303 may reset the second frame 402 so that it has a certain margin from the area including the characters. The information processing device 30 may also include an operation unit (not shown) that accepts operations from the user, and the user may be able to set the degree of the dead zone as appropriate. If the control information output unit 305 determines that the first frame 400 and the second frame 402 overlap (Yes), it proceeds to S14, and if it determines that they do not overlap (No), it ends this flow.

[0039] In S14, the control information output unit 305 calculates projection control information. The projection control information is information for controlling the projection angle of view so that the first frame 400 and the second frame 402 do not overlap with each other, and for changing at least one of the position or size of the second frame 402. For example, the projection control information is information for moving the second frame 402 so that the difference between the central coordinates of the first region information, which is the region information of the presenter 50, and the central coordinates of the second region information, which is the region information of the presentation material 60, becomes large. The movement direction of the second frame 402 may be either left and right or up and down, or a combination of these. For example, the control information output unit 305 may set the direction in which the center of the second frame 402 moves away from the overlapping region as the movement direction. If the purpose is to control so that the first frame 400 and the second frame 402 do not overlap with each other, the information for moving the second frame 402 may be a combination of up, down, left and right information, or may be information for changing the size of the second frame 402. To improve the quality of the movement, the control information output unit 305 may calculate the amount of movement according to the proportion of the overlapping area 410 calculated in S13. When the control information output unit 305 completes the generation of the projection control information, the process proceeds to S15.

[0040] In S15, the control information output unit 305 calculates information on the third frame 404 from the third region information. The information on the third frame 404 may be, for example, information on coordinates in the captured image that can identify the position of the third frame 404. The calculation of the information on the third frame 404 is similar to the calculation method for the information on the first frame 400, and therefore a description thereof will be omitted. When the calculation of the third frame 404 is completed, the control information output unit 305 proceeds to S16.

[0041] In S16, the control information output unit 305 determines whether the second frame 402 and the third frame 404 overlap based on the information of the second frame 402 calculated in S12 and the information of the third frame 404 calculated in S15. FIG. 15(A) and FIG. 15(B) are diagrams showing an example of overlap determination between the second frame 402 and the third frame 404. The control information output unit 305 first detects an overlap area 412 between the second frame 402 surrounding the projected presentation material 60 and the third frame 404 surrounding the projection board 70. Next, the control information output unit 305 calculates the ratio of the overlap area 412 within the frame of the second frame 402 (=area of ​​the overlap area 412÷area of ​​the second frame 402). As shown in Fig. 15(A), when the ratio of the overlapping region 412 exceeds a predetermined threshold, the control information output unit 305 determines that the second frame 402 and the third frame 404 overlap, and in other cases, as shown in Fig. 15(B), determines that the second frame 402 and the third frame 404 do not overlap. Here, "the second frame 402 and the third frame 404 do not overlap" means that at least a part of the second frame 402 is not included in the third frame 404. Therefore, the threshold may be, for example, 1.

[0042] The purpose of this process is to prevent the presentation material 60 from being projected outside the presentable area. Therefore, the threshold value may be appropriately set so that the second frame 402 can be kept contained within the third frame 404. As another method for achieving the above, the control information output unit 305 may use a method of resetting the second frame 402 so that the second frame 402 is a predetermined amount inside the third frame 404. In this case, the fact that the second frame 402 and the third frame 404 do not overlap means that at least a part of the second frame 402 is not contained within the third frame 404, or that there is a high possibility that at least a part of the second frame 402 will not be contained within the third frame 404. Therefore, the projection detection unit 304 may reset the third frame 404 to a frame formed by moving each side of the third frame 404 set in S15 inward by a certain distance. The control information output unit 305 may determine the overlap with the second frame 402 based on the reset third frame 404. By using this method, it is possible to apply the above-mentioned determination process before a part of the second frame 402 exceeds the third frame 404. If the control information output unit 305 determines that the second frame 402 and the third frame 404 overlap (No), it ends this flow, and if it determines that the second frame 402 and the third frame 404 do not overlap (Yes), it proceeds to S17.

[0043] In S17, the control information output unit 305 corrects and updates the projection control information calculated in S14 so that the second frame 402 is included in the third frame 404. The purpose of this process is to correct the projection control information calculated in S14 to prevent a part of the presentation material 60 from moving outside the area of ​​the projection board 70. Therefore, as a correction process, the projection control information calculated in S14 is changed so that the presentation material 60 does not move outside the area of ​​the projection board 70. For example, the control information output unit 305 may add to the projection control information calculated in S14 a movement of the second frame 402 in a direction away from the side of the third frame 404 to which the second frame 402 is closest, or a change in the size of the second frame 402, so that a part of the presentation material 60 does not move outside the area of ​​the projection board 70. When the control information output unit 305 completes the correction process, it ends this flow.

[0044] The control information output unit 305 outputs the projection control information to the projection device 20 when the generation process of the projection control information in FIG. 6 is completed. When outputting the projection control information, the control information output unit 305 may convert the projection control information as appropriate in accordance with a protocol supported by the projection device 20. For example, the control information output unit 305 may convert the projection control information indicating the movement up, down, left, and right into a control command for lens shift. In addition, the control information output unit 305 may convert the projection control information indicating the change in size of the second frame 402 into a control command for zooming. FIG. 8(A) and FIG. 8(B) are examples of the distributed video of the terminal 40 before and after the application of the process by the control information output unit 305. FIG. 8(A) shows a state where this process is not applied. FIG. 8(B) shows a state where this process is applied. In FIG. 8(A), the presentation material 60 and the presenter 50 overlap, making the presentation material 60 difficult to see. On the other hand, in FIG. 8(B), the overlap between the presentation material 60 and the presenter 50 is eliminated, and the difficulty in seeing the presentation material 60 is eliminated.

[0045] In this embodiment, the imaging device 10 and the information processing device 30 have been described as separate devices, but it is also possible to configure the imaging device 10 and the information processing device 30 as the same imaging device. Similarly, it is also possible to configure the projection device 20 and the information processing device 30 as the same projection device. This allows the presentation system of this embodiment to have a simplified configuration.

[0046] The information processing device 30 of the first embodiment generates projection control information for controlling the projection device 20 so as to reduce overlap between the first region and the second region based on a first region indicating at least a partial region of the presenter 50 and a second region indicating at least a partial region of the image of the presentation material 60. This allows the information processing device 30 to generate projection control information so as to prevent overlap between the presenter 50 and the presentation material 60. As a result, the information processing device 30 can eliminate the difficulty in viewing the presentation material 60 for the viewers of the terminal 40, regardless of the position of the presenter 50.

[0047] The information processing device 30 calculates an overlapping area 410 where a first frame 400 surrounding a first area and a second frame 402 surrounding a second area overlap, and generates projection control information by comparing the ratio of the overlapping area 410 to the second frame 402 with a threshold value. In this way, the information processing device 30 can adjust the projection angle of view indicated by the projection control information based on the threshold value to better meet the needs of the presenter 50 or the audience of the terminal 40. For example, by setting the threshold value to 0, the information processing device 30 can prevent the presenter 50 and the presentation material 60 from overlapping. On the other hand, by setting the threshold value to a value greater than 0, the information processing device 30 will not change the projection angle of view even if the presenter 50 and the presentation material 60 overlap slightly, so that frequent changes in the position or size of the presentation material 60 that make it difficult to view can be reduced.

[0048] The information processing device 30 can detect the second region inside the image of the presentation material 60. As a result, the information processing device 30 can detect the region including the characters and image of the presentation material 60 as the second region, and delete the blank region from the second region. As a result, even if the presenter 50 overlaps with the blank region of the presentation material 60, the information processing device 30 does not change the projection angle of view of the projection control information, and therefore unnecessary changes to the presentation material 60 can be reduced.

[0049] The information processing device 30 detects a third area indicating the projection board 70, which is an area onto which the presentation material 60 can be projected, and corrects and updates the projection control information based on the second area and the third area indicating the area of ​​the presentation material 60. This allows the information processing device 30 to correct the projection control information including the projection angle of view so that the presentation material 60 is included in the projection board 70, making it easier to view the presentation material 60.

[0050] <Second embodiment> The projection device according to the second embodiment will be described below with reference to Figures 9 and 10. Note that the same parts as those in the first embodiment are given the same reference numerals, and duplicated descriptions will be omitted.

[0051] The presentation system of this embodiment has a configuration in which the projection device 20 in the presentation system of the first embodiment shown in Fig. 1 is replaced with a projection device 21. The configuration other than that described above is the same, so a description thereof will be omitted.

[0052] Fig. 9 is a block diagram showing an example of the configuration of a presentation system in the second embodiment. The presentation system of this embodiment has an imaging device 10, a projection device 21 as a projector, and an information processing device 30. The projection device 21 in Fig. 9 has an I / F unit 211 as a control information acquisition means, a projection control unit 212 as a projection control means, and a projection unit 213. Each function of the I / F unit 211 and the projection control unit 212 is realized by a processor such as a CPU expanding a program stored in a storage such as an SSD into a memory such as a RAM.

[0053] The I / F unit 211 transmits and receives control signals via an input / output terminal such as an RJ-45 terminal and a network. The I / F unit 211 receives projection control information from the information processing device 30 and outputs it to the projection control unit 212. The I / F unit 211 may also transmit information of the projection device 21 to the information processing device 30. The transmitted information includes, for example, inversion information, projection information, and angle of view control information. The inversion information is information that is appropriately changed depending on the installation form of the projection device, such as up-down inversion in the case of ceiling installation and left-right inversion in the case of rear projection. The projection information is information regarding the projection position, projection size, and focal position of the projected image. The angle of view control information is information regarding controllable items such as lens shift and zoom and their control ranges.

[0054] The projection control unit 212 controls the projection unit 213 based on the projection control information received from the I / F unit 211 .

[0055] The projection unit 213 is composed of, for example, a light source, a display element, and a projection optical system. The light source has, for example, a lamp light source and a microprocessor for controlling the lamp light source. The light source emits light to the display element via an integrator optical system or a color separation optical system (neither of which are shown). The light source may be an ultra-high pressure mercury lamp, a metal halide lamp, a xenon lamp, a halogen lamp, or a solid light source such as an LED (Light Emitting Diode) or a laser. The display element has, for example, a transmissive liquid crystal panel and a microprocessor for controlling the display element. The display element emits modulated light to the projection optical system. The display element may be a reflective liquid crystal panel or a DMD (Digital Micromirror Device). The projection optical system has a plurality of lenses, an actuator for driving the lenses, an encoder for detecting the position of the lenses, and a microprocessor for controlling the lenses. The projection optical system emits light to a screen. The projection optical system also includes a zoom mechanism, a focus mechanism, a peripheral focus mechanism, a lens shift mechanism, and the like, and is capable of controlling the projection angle of view and the focus.

[0056] The projection process of the projection device 21 in the second embodiment will be described with reference to the flowchart of Fig. 10. This process starts when projection control information is received from the information processing device 30 via the I / F unit 211.

[0057] In S21, the projection control unit 212 controls the projection angle of view based on the projection control information, and proceeds to S22. A control command or the like is used as the projection control information. For example, when the projection control unit 212 receives a lens shift control command as the projection control information, the projection control unit 212 controls the lens shift mechanism of the projection unit 213 based on a specified control value. In addition, the control value may be specified by a relative specification method or an absolute specification value. When an absolute specification method is used, the projection device 21 may transmit information about the projection device 21 to the information processing device 30 in advance via the I / F unit 211.

[0058] In S22, the projection control unit 212 determines whether or not the control of the projection unit 213 has been completed according to the control value. If the control has been completed (Yes), the projection control unit 212 ends this flow, otherwise (No), the projection control unit 212 proceeds to S23.

[0059] In S23, the projection control unit 212 compares the current control value of the projection unit 213 with the control range and determines whether the control limit has been reached. If the control value has reached the control limit (Yes), the projection control unit 212 proceeds to S24, and otherwise (No), the projection control unit 212 proceeds to S21.

[0060] In S24, if the control limit has been reached, the projection control unit 212 notifies the information processing device 30, the presenter 50, or the like that the control limit has been reached. The notification method may be to notify the information processing device 30 via the I / F unit 211, or to notify the presenter 50 by superimposing a warning image on the projected image via a display element. Note that the information processing device 30 may include an information presentation means for presenting information about the control limit to the presenter 50 or an administrator when the information processing device 30 receives the above notification. When the notification is complete, the projection control unit 212 ends this flow.

[0061] As described above, according to the second embodiment, the projection device 21 can control the projection angle of view based on the projection control information for controlling the projection angle of view generated by the information processing device 30 based on the captured image of the imaging device 10. In this way, in addition to the effect of the first embodiment, the projection device 21 can project the presentation material 60 so that it is easy for the audience of the terminal 40 to view.

[0062] The projection device 21 notifies the information processing device 30 or the presenter 50 that the angle of view control has reached the limit of the control range. This allows the information processing device 30 to notify the presenter 50 or the like that the limit of the control range has been reached. As a result, the notified presenter 50 can reduce the difficulty in viewing the presentation material 60 by moving his / her position, etc.

[0063] <Third embodiment> The presentation system in the third embodiment will be described below with reference to Figures 11 and 12. Note that the same parts as those in the first and second embodiments are given the same reference numerals, and duplicated descriptions will be omitted.

[0064] 1 of the second embodiment, the presentation system is configured by replacing the imaging device 10 with an imaging device 12 and the information processing device 30 with an information processing device 32. The configuration other than the above is the same, so a description thereof will be omitted.

[0065] Fig. 11 is a block diagram showing an example of the configuration of a presentation system in the third embodiment. The presentation system of this embodiment has an imaging device 12, a projection device 21, and an information processing device 30. The projection device 21 in Fig. 11 has an I / F unit 121, an imaging control unit 122, and an imaging unit 123. The information processing device 32 in Fig. 2 has an I / F unit 301, a presenter detection unit 302, a material detection unit 303, a projection detection unit 304, and a control information output unit 325. Each function of the I / F unit 121 and the imaging control unit 122 is realized by a processor such as a CPU expanding a program stored in a storage such as an SSD into a memory such as a RAM.

[0066] The I / F unit 121 transmits and receives a video signal and imaging control information for controlling the imaging unit 123 via an input / output terminal such as an RJ-45 terminal and a network. The I / F unit 121 transmits a video signal to the information processing device 32 and receives the imaging control information.

[0067] The imaging control unit 122 controls the imaging unit 123 based on the imaging control information acquired from the I / F unit 121 .

[0068] The imaging unit 123 is composed of, for example, an imaging optical system, an imaging element, and a pan / tilt mechanism. The imaging optical system is composed of a zoom lens, a focus lens, a blur correction lens, an aperture, a shutter, etc. The imaging optical system collects light from a subject onto the imaging element. The imaging element is a charge-integrated solid-state imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device) that converts the collected light into a current value. The pan / tilt mechanism is a pan mechanism that changes the imaging angle of view in the pan direction, and a tilt mechanism that changes it in the tilt direction. In addition, a mechanism such as a camera platform that is configured separately from the imaging device may be used.

[0069] The control information output unit 325 generates imaging control information for controlling the imaging angle of view and projection control information for controlling the projection angle of view.

[0070] Hereinafter, the control information generation process of the control information output unit 325 in the first embodiment will be described with reference to the flowchart in Fig. 12. This process is started when at least one of the first region information and the second region information is updated.

[0071] In S31, the control information output unit 325 calculates information of the first frame 400 from first area information, which is information on the area of ​​the presenter 50. This step is similar to the processing content of S11, so a description thereof will be omitted. When the calculation processing in this step is completed, the process proceeds to S32.

[0072] In S32, the control information output unit 325 calculates information of the second frame 402 from the second area information, which is information on the area of ​​the presentation material 60. This step is similar to the processing content of S12, so the explanation is omitted. When the calculation processing in this step is completed, the process proceeds to S33.

[0073] In S33, the control information output unit 325 calculates imaging control information for controlling the imaging angle of view and projection control information for controlling the projection angle of view. First, the control of the imaging angle of view will be described. The imaging control information for controlling the imaging angle of view is, for example, information for panning and tilting the imaging device 12 so that the first frame 400, which is a frame showing the presenter 50, fits within the captured image even when the first frame 400 moves.

[0074] 13(A) to 13(C) are diagrams showing examples of the imaging angle of view when the presenter 50 moves. FIG. 13(A) shows the state before the application of the imaging control information in this step when the presenter 50 moves. FIG. 13(B) shows the state after the application of the imaging control information. As shown in FIG. 13(A), the presenter 50 does not fall within the imaging angle of view in the state before the application. On the other hand, in the state shown in FIG. 13(B) after the application, the presenter 50 falls within the imaging angle of view. However, as shown in FIG. 13(B), since the imaging angle of view has been changed to track the presenter 50, the presentation material 60 does not fall within the imaging angle of view.

[0075] Next, the control of the projection angle of view will be described. The projection control information of the projection angle of view is information for controlling the lens shift of the projection device 21 so that the second frame 402 surrounding the presentation material 60 is included in the captured image even if the second frame 402 does not fit within the capture angle of view due to a change in the capture angle of view, as shown in FIG. 13(B). FIG. 13(C) shows a state after the control of the capture angle of view and the projection angle of view is applied in this step. As shown in FIG. 13(C), the presenter 50 and the presentation material 60 are included within the capture angle of view by applying the control of the capture angle of view and the projection angle of view. Note that, depending on the relationship between the presenter 50 and the presentation material 60, both may not be included within the capture angle of view. In this case, the information processing device 32 may be configured to receive a priority setting of which of the presenter 50 and the presentation material 60 is to be given priority. The information processing device 32 may determine which of the presenter 50 and the presentation material 60 is to be included in the angle of view with priority based on the above-mentioned priority.

[0076] In addition, when the imaging device 12 and the projection device 21 control both the imaging angle of view and the projection angle of view, at least one of the control direction, the control amount, and the control speed may be controlled in synchronization in order to improve the quality of the control. In this case, the control information output unit 325 may generate the imaging control information and the projection control information so as to synchronize them, and may control the output timing of the imaging control information and the projection control information. Furthermore, the control information output unit 325 may receive the angle of view control information from the imaging device 12 and the projection device 21, and correct the imaging control information and the projection control information based on the angle of view control information so that the amount of change in the imaging angle of view and the amount of change in the projection angle of view within the imaging angle of view are the same. If necessary, the control information output unit 325 may receive the inversion information from the imaging device 12 and the projection device 21, and correct the imaging control information and the projection control information so that the control direction is consistent even when the installation form is different based on the inversion information. When the control information output unit 325 completes the calculation process, it proceeds to S34.

[0077] In S34, the control information output unit 325 determines whether the first frame 400 calculated in S31 and the second frame 402 calculated in S32 overlap. This step is similar to the process content of S13, so a description thereof will be omitted. If the control information output unit 325 determines in the determination process that there is overlap (Yes), it proceeds to S35, and if it determines that there is no overlap (No), it ends this flow.

[0078] In S35, the control information output unit 325 corrects the projection control information calculated in S33. The control information output unit 325 corrects the projection control information for controlling the projection of the second frame 402 so that the first frame 400 calculated in S31 and the second frame 402 calculated in S32 do not overlap. This step is similar to the processing content of S14, so a description thereof will be omitted. When the control information output unit 325 completes the correction processing, it ends this flow.

[0079] 12 is completed, the control information output unit 325 outputs imaging control information for controlling the imaging angle of view to the imaging device 12, and outputs projection control information for controlling the projection angle of view to the projection device 21. When outputting the imaging control information and projection control information, the control information output unit 325 may convert the imaging control information and projection control information as appropriate in accordance with the protocols supported by the imaging device 12 and the projection device 21. Furthermore, the control information output unit 325 may appropriately adjust the output timing of the imaging control information and the projection control information in consideration of the delay time until the imaging device 12 and the projection device 21 are controlled, or may output control information first to a device with a slow control speed.

[0080] In the present embodiment, the angle of view is controlled by combining pan / tilt control of the imaging device 12 and lens shift control of the projection device 21, but the angle of view may be controlled by controlling the zoom of both.

[0081] As described above, according to this embodiment, the information processing device 32 in the presentation system can output information for controlling the angle of view of the imaging device 12 and the projection device 21 based on the image captured by the imaging device 12. As described above, the information processing device 32 can prevent the audience of the terminal 40 from having difficulty viewing the presentation material 60 even if the position of the presenter 50 changes during the presentation.

[0082] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0083] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0084] The disclosure of this specification includes the following information processing device, projection device, system, control method, and program. (Item 1) a first detection means for detecting a first region indicating at least a part of a region of a human body from an image captured by the imaging means; a second detection means for detecting a second region indicating at least a part of an image projected by the projection means from the captured image; an output unit that generates and outputs control information for controlling the projection unit so as to reduce an overlapping area between the first area and the second area. (Item 2) The output means includes: Calculating an overlapping area where a first frame surrounding the first area and a second frame surrounding the second area overlap; 2. The information processing device according to item 1, characterized in that the control information is output by comparing a ratio of the overlapping area to the second frame with a predetermined threshold value. (Item 3) The second detection means is 3. The information processing device according to item 2, wherein the second region is detected on the inside of the image. (Item 4) a third area detection unit that detects a third area indicating an area in which the image can be projected from the captured image; 2. The information processing device according to item 1, wherein the output means generates and outputs the control information based on the second area and the third area. (Item 5) 2. The information processing device according to item 1, wherein the output means generates and outputs imaging control information for controlling the imaging means based on the first area and the second area. (Item 6) The output means is 6. The information processing device according to item 5, wherein the control information and the imaging control information are generated and output so that the projection means and the imaging means are synchronized in at least a portion of their operations. (Item 7) 2. The information processing device according to item 1, comprising the imaging means. (Item 8) an acquisition means for acquiring control information generated so as to reduce an overlapping area between a first area indicating at least a partial area of ​​a human body included in an image captured by the imaging means and a second area indicating at least a partial area of ​​an image projected from the captured image by the projection means; a projection control unit that controls a projection angle at which the image is projected based on the control information. (Item 9) A system including a projection device, an imaging device, and an information processing device, The projection device is a projection control means for controlling a projection angle for projecting an image; The imaging device includes: an imaging means for imaging a human body and the projected image to generate an image; The information processing device includes: a first detection means for detecting a first region indicating at least a part of a region of a human body from the image captured by the imaging means; a second detection means for detecting a second region indicating at least a part of an image projected from the projection means included in the captured image; and an output unit that generates and outputs control information for controlling the projection unit so as to reduce an overlapping area between the first area and the second area. (Item 10) a first detection step of detecting a first region indicating at least a part of a human body from an image captured by the imaging means; a second detection step of detecting a second region indicating at least a part of an image projected by a projection means from the captured image; and an output step of generating and outputting control information for controlling the projection means so as to reduce overlap between the first area and the second area. (Item 11) A program for causing a computer to execute each step of the control method according to item 10 by reading and executing the program by the computer.

[0085] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0086] 10...imaging device, 20...projection device, 30...information processing device, 50...presenter, 60...presentation materials, 70...projection board, 302...presenter detection unit, 303...material detection unit, 304...projection detection unit, 305...control information output unit, 400...first frame, 402...second frame, 404...third frame, 410...overlapping area, 412...overlapping area, 21...projection device, 212...projection control unit, 12...imaging device, 32...information processing device, 123...imaging unit, 325...control information output unit.

Claims

1. a first detection means for detecting a first region indicating at least a part of a region of a human body from an image captured by the imaging means; a second detection means for detecting a second region indicating at least a part of an image projected by the projection means from the captured image; an output unit that generates and outputs control information for controlling the projection unit so as to reduce an overlapping area between the first area and the second area.

2. The output means includes: Calculating an overlapping area where a first frame surrounding the first area and a second frame surrounding the second area overlap; The information processing apparatus according to claim 1 , wherein the control information is output by comparing a ratio of the overlapping area to the second frame with a predetermined threshold value.

3. The second detection means is The information processing apparatus according to claim 2 , wherein the second area is detected on the inside of the image.

4. a third area detection unit that detects a third area indicating an area in which the image can be projected from the captured image; 2 . The information processing apparatus according to claim 1 , wherein the output means generates and outputs the control information based on the second area and the third area.

5. 2 . The information processing apparatus according to claim 1 , wherein the output means generates imaging control information for controlling the imaging means based on the first area and the second area, and outputs the imaging control information.

6. The output means is 6. The information processing apparatus according to claim 5, wherein the control information and the imaging control information are generated and output so that the projection means and the imaging means are synchronized in at least a part of their operations.

7. 2. The information processing apparatus according to claim 1, further comprising the imaging means.

8. an acquisition means for acquiring control information generated so as to reduce an overlapping area between a first area indicating at least a partial area of ​​a human body included in an image captured by the imaging means and a second area indicating at least a partial area of ​​an image projected from the captured image by the projection means; a projection control unit that controls a projection angle at which the image is projected based on the control information.

9. A system including a projection device, an imaging device, and an information processing device, The projection device is a projection control means for controlling a projection angle for projecting an image; The imaging device includes: an imaging means for imaging a human body and the projected image to generate an image; The information processing device includes: a first detection means for detecting a first region indicating at least a part of a region of a human body from the image captured by the imaging means; a second detection means for detecting a second region indicating at least a part of an image projected from the projection means included in the captured image; and an output unit that generates and outputs control information for controlling the projection unit so as to reduce an overlapping area between the first area and the second area.

10. a first detection step of detecting a first region indicating at least a part of a human body from an image captured by the imaging means; a second detection step of detecting a second region indicating at least a part of an image projected by a projection means from the captured image; and an output step of generating and outputting control information for controlling the projection means so as to reduce overlap between the first area and the second area.

11. A program for causing a computer to execute each step of the control method according to claim 10 by reading and executing the program by the computer.

Images