Image capture control device, image capture control method

Abstract

To provide a technique for determining an angle of view including interrelated descriptions on an information sharing board.SOLUTION: The continuity between a description added to an imaging subject from a frame in which the imaging subject was imaged and a description that has been added to the imaging subject before the description is determined. On the basis of the determination result, descriptions of the imaging subject are divided into groups. On the basis of the time on each group, the angle of view of imaging apparatus is controlled to image the description of the imaging subject of a group to be distributed in the frame.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] This disclosure relates to imaging control technology.

Background Art

[0002] In places for information sharing such as lectures, generally, information sharing boards such as blackboards, whiteboards, and screens are used to assist the understanding of the audience. Therefore, when distributing the video of a lecture to the audience outside the lecture room, it is desirable to include the information presented on the information sharing board in the distributed video. At that time, if the shooting angle is too wide, the characters included in the distributed video will be too small to read, while if the shooting angle is too narrow, there is a possibility that the necessary information will not be included in the distributed video, making it difficult to understand the lecture. Therefore, it is important to image the information sharing board at an appropriate shooting angle.

[0003] On the other hand, for the purpose of labor saving, several technologies have been proposed to determine an appropriate shooting angle of the distributed video without the need for an operator such as a cameraman. In Patent Document 1, the change in the feature amount of a preset imaging area is detected, and the cutout range is selectively determined. In addition, in Patent Document 2, in accordance with the description, the shooting angle for pan control is determined while maintaining the zoom magnification at which characters can be read.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, a method has not been established to recognize the relationships between information on an information sharing board and appropriately control the field of view each time, similar to a cameraman. Even when using the method disclosed in Patent Document 1, if there was a highly related piece of writing on the board located far from the instructor, it was not possible to simultaneously distribute the video to remote students. Similarly, even when using the method disclosed in Patent Document 2, only the area last written by the instructor was captured, and continuous writing at distant locations was not included in the capture. Therefore, if there was no spatial continuity between the latest and past writings, remote students could not simultaneously refer to highly related pieces of writing on the board. This disclosure provides a technique for determining the field of view that includes mutually related writings on an information sharing board. [Means for solving the problem]

[0006] One aspect of this disclosure is the process of adding to the image target from a frame in which the image target has been captured. 1 Description and the 1 The following have been added to the imaging targets prior to the description: 2nd A grouping means that determines the continuity of the description and, based on the result of the determination, groups the descriptions in the imaged object. Based on the time for each group obtained by the grouping means, a control means controls the field of view of the imaging device to capture the description of the imaging target of the group to be distributed in the frame. Equipped with 、 The grouping means determines, from the frame in which the object to be imaged has been captured, whether there is any overlap between the projected area of ​​the page projected onto the object to be imaged and the area of ​​an existing group, and based on the result of this determination, groups the projected area of ​​the object to be imaged. It is characterized by the following. [Effects of the Invention]

[0007] According to this disclosure, a technique can be provided for determining the field of view, which includes descriptions on an information sharing board that are closely related to each other. [Brief explanation of the drawing]

[0008] [Figure 1] A block diagram showing an example of the system configuration. [Figure 2] A block diagram showing an example of the functional configuration of the imaging control device 101. [Figure 3] A flowchart for grouping information on a shared information board. [Figure 4] A schematic diagram illustrating how font size is detected. [Figure 5] A flowchart for the grouping process related to projection onto the information sharing board. [Figure 6] Flowchart for the time update process. [Figure 7] Flowchart for recording target location. [Figure 8] Flowchart for updating the target location. [Figure 9] A diagram showing an example of the structure of descriptive information. [Figure 10] This figure illustrates the content represented by the descriptive information in Figure 9. [Figure 11] Flowchart for determining the field of view. [Figure 12] Flowchart for angle-of-view compositing. [Figure 13] A diagram illustrating the field-of-view compositing process. [Modes for carrying out the invention]

[0009] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the scope of this disclosure as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to this disclosure, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0010] [First Embodiment] First, a configuration example of the system according to this embodiment will be described using the block diagram of FIG. 1. The system according to this embodiment is a system that captures a moving image of a state in which a lecture is being conducted using a flat member such as a blackboard, a whiteboard, a screen (hereinafter referred to as an information sharing board), and distributes the captured moving image. As shown in FIG. 1, the system according to this embodiment includes an imaging device 102, an imaging control device 101, and a viewing terminal 104. The imaging control device 101 and the viewing terminal 104 are configured to be able to perform data communication with each other via a wired and / or wireless network 103. Note that the network configuration between the imaging control device 101 and the viewing terminal 104 is not limited to a specific network configuration.

[0011] First, the imaging device 102 will be described. The imaging device 102 is installed and set in a lecture room where an information sharing board is installed so that all information sharing boards in the lecture room can be imaged. The imaging device 102 has a single high-resolution camera, and the high-resolution camera captures a moving image of "the state in which a lecture is being conducted using the information sharing board", and sequentially outputs each frame in the captured moving image to the imaging control device 101. The imaging device 102 and the imaging control device 101 may be connected by wire or wirelessly.

[0012] Next, the imaging control device 101 will be described. The imaging control device 101 determines the range (angle of view) of an area (low-image-quality area) to be cut out from the frames (high-resolution frames) received from the imaging device 102, cuts out the area of the determined angle of view, and generates a distribution frame including the cut-out area. In the following description, the zoom ratio of the angle of view refers to the magnification when the cut-out area is enlarged or reduced to match the distribution frame. Then, the imaging control device 101 distributes the generated distribution frame to the viewing terminal 104 via the network 103. Although the audio is not mentioned below, it is assumed that the frames distributed by the imaging control device 101 to the viewing terminal 104 also include audio. However, it is not essential for the frame to include audio, and it may not include audio.

[0013] The CPU 201 executes various processes using the computer programs and data stored in the storage unit 202. As a result, the CPU 201 controls the operation of the entire imaging control device 101 and executes or controls various processes described as being performed by the imaging control device 101.

[0014] The storage unit 202 is a non-volatile memory, and the storage unit 202 stores computer programs and data for causing the CPU 201 to execute or control various processes described as being performed by the imaging control device 101.

[0015] The interface 203 has an interface for connecting the imaging control device 101 to the network 103 and an interface for connecting the imaging control device 101 to the imaging device 102.

[0016] Next, the viewing terminal 104 will be described. The viewing terminal 104 displays / plays back the distribution frame transmitted from the imaging control device 101 via the network 103. As a result, a student who remotely attends a lecture being imaged by the imaging device 102 can view the lecture (video stream) using the viewing terminal 104.

[0017] An example of the functional configuration of the imaging control device 101 is shown in the block diagram of FIG. 2. In the present embodiment, it is assumed that the functional units shown in FIG. 2 are implemented by a computer program. Hereinafter, when the functional units shown in FIG. 2 are described as the main body of the process, in reality, the functions of the corresponding functional units are realized by the CPU 201 executing the computer program corresponding to the functional units. Note that one or more of the functional units shown in FIG. 2 may be implemented by hardware.

[0018] Next, the angle-of-view determination process performed by the imaging control device 101 will be described. The angle-of-view determination process performed by the imaging control device 101 is roughly divided into a front stage, middle row , and a rear stage. First, the processes in the front stage and the middle stage will be described.

[0019] In the first stage of processing, the frames received from the imaging device 102 are analyzed to generate information used in determining the field of view in the second stage of processing. In the second stage of processing, the information generated in the first stage of processing is stored in the storage unit 202. In addition, as the first and second stages of processing are executed, the stored information is read out as needed and provided to each respective process.

[0020] The "generation of information" in the preceding processing may include grouping processing related to descriptions on the information sharing board, grouping processing related to projection onto the information sharing board, time update processing based on instructions for descriptions and projections on the information sharing board, and target position recording and target position update processing related to the information sharing board. In other words, "generation of information" may include all or some of the above processes that "generation of information" may include. Next, we will explain each of the above processes that "generation of information" may include.

[0021] <Grouping process for descriptions on information sharing boards> The grouping process for entries on the information sharing board is performed in response to the instructor adding entries to the board. The grouping process for entries on the information sharing board will be explained according to the flowchart in Figure 3.

[0022] In step S301, the description detection unit 10 detects newly added descriptions (such as characters or line drawings) to the information sharing board from the frame captured by the imaging device 102. Description detection can be performed using known techniques, such as background subtraction methods using images and character recognition methods for images.

[0023] In step S302, the description analysis unit 11 determines whether the description detected in step S301 has continuity with a description that was added to the information sharing board before (in the past) that description. There are various methods for determining whether or not there is continuity between descriptions.

[0024] For example, if the difference in detection timing (time interval) between a newly detected description (Description A) and a description detected more recently than Description A (Description B) is within a certain time frame, then it is determined that there is continuity (temporal continuity) between Description A and Description B.

[0025] For example, if the distance (spatial distance) between a newly detected description (description A) and a description (description B) that was detected more recently but earlier than description A is within a certain distance, then it is determined that there is continuity (spatial continuity) between description A and description B.

[0026] In other cases, that is, if a certain amount of time has passed between the detection timing of description A and the detection timing of description B, or if no previous description is within a certain distance of description A, it may be determined that there is no continuity.

[0027] Alternatively, the continuity between descriptions can be determined by considering both the temporal and spatial relationships between them. For example, both the temporal interval and spatial distance can be scored, and if the weighted average exceeds a threshold, continuity can be determined.

[0028] Furthermore, there may be one or more indicators used to determine whether or not there is continuity between descriptions. Thus, there are various methods for determining whether or not there is continuity between descriptions, and it is not limited to a specific method.

[0029] If it is determined that there is continuity between the description detected in step S301 and the description already present on the information sharing board, the process proceeds to step S305. On the other hand, if it is determined that there is no continuity between the description detected in step S301 and the description already present on the information sharing board, the process proceeds to step S303.

[0030] In step S303, the description analysis unit 11 creates a new group for the descriptions detected in step S301. If the process proceeds from step S303 to step S304, in step S304, the description generation unit 18 generates description information for the group created in step S303 based on the descriptions detected in step S301.

[0031] On the other hand, in step S305, the description analysis unit 11 identifies a group of descriptions that are determined to be continuous with the descriptions detected in step S301. If the process proceeds from step S305 to step S304, in step S304, the description generation unit 18 updates the description information of the group identified in step S305 based on the descriptions detected in step S301, so that the group identified in step S305 includes the descriptions detected in step S301.

[0032] The description information generated by the description generation unit 18 for each group includes the position of the group, the allowable zoom range for the group, and the time for the group. The position of the group is position information that defines a rectangle (group area) that surrounds the descriptions included in the group. The position information that defines the rectangle may be the positions of the four corners of the rectangle, the positions of the upper left corner and the lower right corner of the rectangle, or the position of the upper left corner of the rectangle and the height and width of the rectangle.

[0033] The allowable zoom range for a group is a constraint on the zoom range when including the group in the field of view, and is used later when determining the field of view. The time for a group is a logical time, and is information that allows for the identification of the generation / update order among groups. The time for a group may be the actual time (real time) when the group was created / updated, or it may be something that defines the ordering relationship between groups, such as a counter value that is incremented when a group is created / updated.

[0034] The description analysis unit 11 calculates the acceptable zoom range for a group based on the size of the descriptions included in the group. The method for calculating the size of the descriptions differs depending on whether the descriptions contain characters. Known character recognition techniques can be used to determine whether the descriptions contain characters. If the descriptions contain characters, the acceptable zoom range is calculated based on the size of the characters and information about the viewing terminal 104 (such as the screen resolution of the viewing terminal 104). The lower limit of the zoom range is the zoom rate at which the characters are most zoomed out while still being readable on the screen of the viewing terminal 104. The upper limit of the zoom range is the zoom rate at which the entire group is most zoomed in while still being visible. On the other hand, if the descriptions do not contain characters, a default value is used as the acceptable zoom range. The default value may be expressed as a range or as a single number. In the case of a single number, it indicates that the upper limit of the zoom range and the lower limit of the zoom range are equal. Alternatively, the acceptable zoom range may be calculated for each group using a predetermined function that takes the properties of each group, such as the size of the rectangle surrounding the group, as input.

[0035] A lightweight method is used to detect character size by estimating it from character height. Typically, the writing direction of a description is fixed. For example, in horizontal writing, descriptions are always added horizontally within a single line. The description analysis unit 11 utilizes this characteristic to detect the writing direction. Then, using a direction perpendicular to the writing direction as the axis, it calculates the maximum height of a continuous group of strokes as the size of the character belonging to that group. Figure 4 schematically shows the method of character size detection by the description analysis unit 11. In Figure 4, since the writing direction is horizontal, the axis perpendicular to it is vertical. The tallest character in a continuous group of strokes in the vertical direction is the character "あ". Therefore, the size of the characters included in this description is calculated using the height of "あ".

[0036] <Grouping process related to projection to the information sharing board> The grouping process related to projection onto the information sharing board is performed in response to the projection of images onto the board from information equipment such as a projector. For example, if related descriptions are grouped into individual pages beforehand, the projector or other information equipment switches the projected image of each page by advancing through the pages, thereby changing the image of the pages projected onto the information sharing board. Therefore, the projection area corresponding to an entire page is treated as a single group. Furthermore, for simplicity, the switching of projection content is detected as a new projection and grouped accordingly. Note that the projection area is a simple rectangle and is brighter than its surroundings, so it can be easily detected using known image processing techniques. Similarly, changes in projection content due to page advancing can also be detected using known image processing techniques.

[0037] The grouping process related to projection onto the information sharing board will be explained according to the flowchart in Figure 5. In step S501, the projection detection unit 12 detects the projection of page images onto the information sharing board and changes in the projected content due to page turning from frames captured by the imaging device 102.

[0038] In step S502, the projection analysis unit 13 detects the projection area on the information sharing board where the page image is projected from the frame captured by the imaging device 102, and determines whether or not the detected projection area overlaps with the group area of ​​an existing group.

[0039] If the result of this determination indicates that there is overlap, the process proceeds to step S505; if it indicates that there is no overlap, the process proceeds to step S503. In determining overlap, even if the entire projection area does not overlap, if a percentage of the projection area exceeds a threshold, it may be determined that there is overlap. As mentioned above, overlap occurs when the projection content is paginated.

[0040] In step S505, the projection analysis unit 13 identifies the group that has been determined to overlap with the projection region from among the existing groups. If the process proceeds from step S505 to step S504, in step S504, the description generation unit 18 updates the "group location" included in the description information of the group identified in step S505 so that the projection region is included in the group region represented by the "group location".

[0041] Meanwhile, in step S503, the projection analysis unit 13 creates a new group for the projection region detected in step S502. If the process proceeds from step S503 to step S504, in step S504, the description generation unit 18 generates description information for the group created in step S503 based on the projection region detected in step S502.

[0042] The description information generated by the description generation unit 18 in step S504 for each group includes the position of the group, the allowable zoom range for the group, and the time for the group, similar to the description information generated in step S304 above.

[0043] The group's position is positional information that defines a rectangle (group area) that encloses the projection region included in the group. The information of the rectangle represented by the positional information is the same as that of the group's position in the description.

[0044] The allowable zoom range for a group is a constraint on the zoom range when including the group in the field of view, and is calculated in the same way as the method for groups related to descriptions. That is, if the projection area contains text, it is calculated based on the size of the text, and otherwise a default value is used. The time for a group is a logical time, and is information that allows the generation / update order between groups to be identified.

[0045] <Time update process> The time update process is performed in response to the instructor pointing to descriptions or projection areas previously written for explanation purposes. The time update process will be explained according to the flowchart in Figure 6.

[0046] In step S601, the instruction detection unit 14 uses frames captured by the imaging device 102 to detect actions performed by the instructor. There are various methods for detecting actions performed by the instructor.

[0047] For example, when the instruction detection unit 14 detects a group region in a frame where the distance between the instructor's hand and the group region falls below a threshold, it determines that the action "the instructor pointed to the group corresponding to the group region" has occurred.

[0048] For example, when the instruction detection unit 14 detects a group region where the distance between the laser pointer operated by the instructor and the position within the frame where the laser pointer is irradiated falls below a threshold, it determines that the instructor has pointed to the group corresponding to the group region with the laser pointer. The same applies when the instructor points to a projection region using the laser pointer. When the target pointed to by the instructor using the laser pointer is a projection region, it may also be indicated by the operation of an information processing device using a mouse pointer or the like. In this case, the operation of the information processing device by the instructor is detected and treated as an instruction to the projection region. To improve the accuracy of detection, the instructor's statements may be recognized and their content added to the determination criteria.

[0049] In step S602, the instruction analysis unit 15 identifies the group that the instructor indicated in the processing in step S601. In step S603, the description generation unit 18 updates the time included in the description information of the group identified in step S602. When updating the time included in the description information, for example, if real time is used as the time, the time is updated to the current time (instruction time).

[0050] <Target location recording process, target location update process> The target position recording and target position updating processes for the information sharing board are designed to efficiently update the position of writing and projection when the blackboard used as the information sharing board is vertically slidable or when the whiteboard used as the information sharing board is movable. First, the target position recording process records the position of the information sharing board at the start of imaging by the imaging device 102. Then, the target position updating process updates the recorded position of the information sharing board in accordance with changes in the position of the information sharing board during imaging by the imaging device 102.

[0051] First, the target position recording process, which records the initial position of the information sharing board (the position of the information sharing board at the start of imaging by the imaging device 102) in the storage unit 202, will be explained according to the flowchart in Figure 7. In step S701, the target detection unit 16 detects the information sharing board from the frame captured by the imaging device 102.

[0052] In step S702, the target analysis unit 17 detects the position of the information sharing board detected in step S701. For example, the target analysis unit 17 converts the position of the information sharing board within the frame detected in step S701 (the positions of the four corners of the information sharing board, the positions of two corresponding corners on the information sharing board, etc.) into a position in the world coordinate system of the lecture room. The world coordinate system of the lecture room is a coordinate system in which one point in the lecture room is the origin, and the three mutually orthogonal axes at the origin are the x-axis, y-axis, and z-axis, respectively. In this embodiment, the position and orientation of the imaging device 102 in the world coordinate system of the lecture room are assumed to have been acquired in advance as position and orientation information and stored in the storage unit 202. Therefore, the target analysis unit 17 acquires the position obtained by converting the position of the information sharing board within the frame captured by the imaging device 102 using a known coordinate transformation with the position and orientation information as the "position in the world coordinate system of the lecture room".

[0053] In step S703, the description generation unit 18 generates position information that defines a rectangle adjacent to the area of ​​the information sharing board defined by the position determined in step S702. The position information defining the rectangle may be the positions of the four corners of the rectangle, the positions of the upper left corner and the lower right corner of the rectangle, or the position of the upper left corner of the rectangle and the height and width of the rectangle. The description generation unit 18 then stores the generated position information in the storage unit 202 as the "initial position of the information sharing board".

[0054] Next, the target position update process for updating the position of the information sharing board will be explained according to the flowchart in Figure 8. In step S801, the target detection unit 16 detects the movement of the information sharing board from the frame captured by the imaging device 102. In step S802, the target analysis unit 17 identifies which information sharing board has moved.

[0055] Step S803 describes Generate Unit 18 determines the current position of the information sharing board (the information sharing board identified in step S802) in the world coordinate system as described above, based on the frame captured by the imaging device 102. In step S804, the description Generate Unit 18 stores the position of the "moved information sharing board" in the world coordinate system, which was determined in step S803, in the storage unit 202.

[0056] Here, an example of the structure of the descriptive information is shown in Figure 9. Figure 9 shows the structure of the descriptive information for each group, with each row corresponding to the descriptive information for the corresponding group. In Figure 9, the descriptive information includes the group's position, the allowable zoom range for the group, the time for the group, the subject to be described, and the position of the subject to be described.

[0057] The group positions in Figure 9 represent the positional information of the upper-left and lower-right corners of the rectangle that surrounds the description included in the group, in a local coordinate system based on the information sharing board. The local coordinate system based on the information sharing board is, for example, a coordinate system in which one point on the information sharing board is the origin, the axis along one side of the information sharing board is the x-axis, and the axis along the orthogonal side perpendicular to that side is the y-axis.

[0058] The permissible zoom range for a group is a constraint on the zoom range when including the group in the field of view. For groups containing text, the permissible zoom range is the upper and lower limits of the zoom range in which the viewer on the viewing terminal 104 can read the text, while for groups without text, the permissible zoom range is a specified zoom ratio. The time for a group is a logical time as described above. The subject of description is a unique identifier for each information sharing board.

[0059] The locations to be described are the upper left and lower right corners of the information sharing board in the lecture hall's world coordinate system. While it would be more efficient to manage the locations to be described separately, they are included here for the sake of explanation. The structure of the described information is not limited to a specific configuration; for example, the information group shown in Figure 9 could be divided into two groups for management. Furthermore, the device that holds each of the above pieces of information is not limited to a specific device; it is sufficient if the imaging control device 101 can acquire the information as needed.

[0060] Figure 10(a) shows the content represented by the descriptive information of the group corresponding to the first row in Figure 9 (referred to as Group A). ​​The upper left and lower right corners of the information sharing board assigned identifier b1 (information sharing board b1) in the world coordinate system of the lecture room are (gx1s, gy1s) and (gx1e, gy1e), respectively, and Group A belongs to information sharing board b1. Group A contains the string "abc def ghi" as its description, and the upper left and lower right corners of the rectangle surrounding this string are (x1s, y1s) and (x1e, y1e), respectively. The latest generation / update / instruction time for this Group A is t1. The upper limit of the zoom range for strings belonging to this Group A is 1.5, and the lower limit is 0.5. Figure 10(b) shows the content represented by the descriptive information of the group corresponding to the second row in Figure 9 (referred to as Group B). The positions of the upper left corner and lower right corner of the information sharing board assigned identifier b2 (information sharing board b2) in the world coordinate system of the lecture room are (gx2s, gy2s) and (gx2e, gy2e), respectively, and Group B belongs to information sharing board b2. Group B includes a line drawing as its description, and the positions of the upper left corner and lower right corner of the rectangle surrounding the line drawing are (x2s, y2s) and (x2e, y2e), respectively. The latest generation time / update time / instruction time for this Group B is t2. The zoom range for the line drawing belonging to this Group B is 1.

[0061] <Subsequent processing> Next, we will explain the subsequent processing. The subsequent processing may include the determination of the field of view and the compositing process.

[0062] <Field of View Determination Process> The field of view determination process according to this embodiment will be explained with reference to the flowchart in Figure 11. In step S1101, the field of view determination unit 19 refers to the descriptive information of each group and obtains the descriptive information including the latest time from the descriptive information of groups that are not included in the currently set field of view (set field of view) in the frame captured by the imaging device 102. The initial value of the set field of view is an appropriate initial value, and depending on the initial value, no groups may be included.

[0063] In step S1102, the field of view determination unit 19 updates the set field of view so that the group corresponding to the descriptive information acquired in step S1101 is included in the set field of view. In other words, the field of view determination unit 19 updates the set field of view so that the position of the group included in the descriptive information acquired in step S1101 is included in the set field of view. In this embodiment, the group with the most recent time is given priority in being included in the set field of view.

[0064] In step S1103, the angle of view determination unit 19 determines whether the zoom ratio corresponding to the set angle of view is included in the allowable zoom range contained in the descriptive information of each group included in the set angle of view.

[0065] As a result of this determination, if the zoom rate corresponding to the set field of view is included in the allowable zoom range included in the description information of each group included in the set field of view, the process proceeds to step S1101. On the other hand, if the zoom rate corresponding to the set field of view is not included in the allowable zoom range included in the description information of one or more groups included in the set field of view, the process proceeds to step S1104. In step S1104, the field of view determination unit 19 returns the set field of view to the set field of view before the update in step S1102.

[0066] The imaging control device 101 may also extract an image within the set field of view determined by the process in accordance with the flowchart in Figure 11 from the frame captured by the imaging device 102, and distribute the extracted image as a distribution frame to the viewing terminal 104. The image extracted from the frame may also be resized to an appropriate size (for example, the size of the distribution frame) as needed before being distributed to the viewing terminal 104 as a distribution frame.

[0067] <Field of view compositing process> The field-of-view compositing process is a process that, when closely related descriptions are far apart and cannot both fit into a single field of view, sets a new field of view for the description, and then combines multiple fields of view, including the set field of view, to create a frame for distribution. At that time, the same field-of-view determination process as described above is performed on each of the fields of view that have already been combined. The field-of-view compositing process according to this embodiment will be explained with reference to the flowchart in Figure 12.

[0068] In step S1201, the field of view determination unit 19 refers to the descriptive information of each group and obtains the descriptive information including the latest time from the descriptive information of groups that are not yet included in the combined field of view in the frames captured by the imaging device 102.

[0069] In step S1202, the field of view determination unit 19 modifies the combined field of view (all or part of the combined field of view) so that the combined field of view and the field of view of the group of descriptive information acquired in step S1201 fit within the field of view for transmission (for example, a frame for distribution).

[0070] In step S1203, the field of view determination unit 19 determines whether one or more of the modified composite field of view have a zoom rate that is within the acceptable zoom range. If, as a result of this determination, one or more of the modified composite field of view have a zoom rate that is within the acceptable zoom range, the process proceeds to step S1208. On the other hand, if none of the modified composite field of view have a zoom rate that is within the acceptable zoom range, the process proceeds to step S1204.

[0071] In step S1208, the field of view determination unit 19 selects the field of view from the modified composite field of view in which the zoom rate corresponding to the modified field of view falls within the acceptable zoom range as the selected field of view. The field of view determination unit 19 then returns the fields of view other than the selected field of view from the modified composite field of view to their original state. As a result, the field of view of the group of descriptive information acquired in step S1201 is included in the composite field of view. On the other hand, in step S1204, the field of view determination unit 19 returns the field of view modified in the most recent step S1202 to its original field of view.

[0072] In step S1205, the field of view compositing unit 20 generates a frame for distribution by arranging the partial images within each field of view included in the composite field of view such that the positional relationship of the partial images within each field of view is maintained in at least one direction, either horizontal or vertical.

[0073] For example, as shown on the left side of Figure 13(A), suppose that in a frame captured by the imaging device 102, partial image X is located to the upper left of partial image Y. In this case, as shown on the right side of Figure 13(A), a distribution frame is generated in which partial image X and partial image Y are arranged so as to maintain their vertical positional relationship.

[0074] For example, as shown on the left side of Figure 13(B), suppose that in a frame captured by the imaging device 102, partial image U is located to the lower left of partial image V and to the upper left of partial image W, and partial image V is located to the upper right of partial image W. In this case, as shown on the right side of Figure 13(B), a distribution frame is generated in which partial image U, partial image V, and partial image W are arranged so as to maintain the horizontal positional relationship between partial image U and partial image V, and the vertical positional relationship between partial image U / partial image V and partial image W.

[0075] In step S1206, the field of view compositing unit 20 determines whether the distribution frame generated in step S1205 includes each image portion within the composite field of view without any loss.

[0076] If, as a result of this determination, it is determined that the distribution frame generated in step S1205 contains all the image portions within each of the composited fields of view without any loss, the process proceeds to step S1201.

[0077] On the other hand, if it is determined that the distribution frame generated in step S1205 does not contain all the image portions within each of the composited fields of view without any loss, the process proceeds to step S1207.

[0078] In step S1207, the field of view synthesis unit 20 generates a distribution frame that was generated in the previous step S1205, that is, a distribution frame before the placement of the "image portion of the field of view of the group of descriptive information acquired in step S1201" which was placed last.

[0079] Thus, according to this embodiment, information described on the information sharing board can be grouped, and the field of view for simultaneously distributing related information can be determined according to the priority based on the group's creation time / update time / instruction time, etc.

[0080] [Second Embodiment] In the following embodiments, including this embodiment, the differences from the first embodiment will be described, and unless otherwise specified below, they will be the same as the first embodiment. In this embodiment, the case in which the imaging device 102 has a camera (PTZ camera) capable of pan, tilt, and zoom control will be described. When a PTZ camera is used, the field of view to be controlled is the imaging field of view of the PTZ camera itself, and is changed by controlling the pan, tilt, and zoom. Also, the zoom ratio of the field of view corresponds to the zoom ratio of the PTZ camera.

[0081] In this embodiment, the field of view determined by the processing according to the first embodiment is notified to the imaging device 102. In the first embodiment, the field of view change was achieved solely by the imaging control device 101, whereas in this embodiment, control of the PTZ camera of the imaging device 102 is required. Therefore, when the field of view determination unit 19 determines a new field of view, it outputs the determined field of view as a setting related to pan, tilt, and zoom to the imaging device 102 via the interface 203.

[0082] Furthermore, it is not necessary to communicate the candidate angles of view generated during the angle of view determination process to the imaging device 102 each time. The imaging control device 101 incorporates a logical model of the PTZ camera, making it possible to calculate the zoom ratio after changing the angle of view through simulation. Therefore, even without actually changing the angle of view of the camera, it is possible to determine in step S1103 whether the zoom ratio falls within the acceptable zoom range.

[0083] Thus, this embodiment is also applicable to imaging devices 102 having one or more PTZ cameras. This makes it possible to obtain the same effects as in the first embodiment.

[0084] [Third Embodiment] In the configuration shown in Figure 1, the imaging device 102 and the imaging control device 101 are described as separate devices. However, it is also possible to have a configuration in which the imaging device 102 and the imaging control device 101 are combined, for example, a configuration in which the imaging device 102 has the imaging control device 101.

[0085] Furthermore, in the configuration shown in Figure 1, the imaging control device 101 and the viewing terminal 104 are described as being connected via the network 103. However, for purposes such as ensuring scalability, a distribution server may be provided on the network 103, and this server may distribute the distribution frames generated by the imaging control device 101 to the viewing terminal 104.

[0086] Furthermore, in the second embodiment, video streams from multiple PTZ cameras are used without processing; however, if the frames captured by the PTZ cameras are sufficiently high-resolution, the method of the first embodiment may be used in combination. That is, video extracted from the video streams from the PTZ cameras may be combined with the distributed video.

[0087] Furthermore, some of the processes described above, which are performed by the imaging control device 101, may be performed by the viewing terminal 104. For example, if there is variation in the resolution or size of the screen of the viewing terminal 104, the above-mentioned field of view determination process may be performed on the viewing terminal 104, and the allowable zoom range may be adjusted to suit the viewing terminal 104. Also, if field of view selection by the learner is only possible on viewing terminals 104 with a resolution above a certain level, the field of view determination process may be repeated on the viewing terminal 104 to generate multiple field of view options to be selected.

[0088] Furthermore, the numerical values, processing timing, processing order, processing entity, data (information) destination / source / storage location, etc., used in each of the above embodiments are given as examples for the purpose of providing a concrete explanation, and are not intended to limit the scope to such examples.

[0089] Furthermore, some or all of the embodiments described above may be used in appropriate combinations. Alternatively, some or all of the embodiments described above may be used selectively.

[0090] (Other embodiments) This disclosure can also be implemented by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0091] The disclosures herein include the following imaging control devices, imaging control methods, and computer programs.

[0092] (Item 1) A grouping means that determines the continuity between a description added to the target and a description previously added to the target, based on the result of the determination, from a frame in which the target has been captured, Based on the time for each group obtained by the grouping means, a control means controls the field of view of the imaging device to capture the description of the imaging target of the group to be distributed in the frame. An imaging control device characterized by comprising:

[0093] (Item 2) The imaging control device according to item 1, characterized in that the grouping means determines the spatial and / or temporal continuity between a description added to the imaging target and a description previously added to the imaging target.

[0094] (Item 3) The imaging control device according to item 1 or 2, characterized in that, if the grouping means determines that there is continuity between the description added to the imaging target and the description previously added to the imaging target, the description added to the imaging target is included in the group of descriptions previously added to the imaging target.

[0095] (Item 4) The imaging control device according to any one of items 1 to 3, characterized in that the grouping means generates a group of descriptions added to the imaging target when it is determined that there is no continuity between the description added to the imaging target and the description previously added to the imaging target.

[0096] (Item 5) The imaging control device according to any one of items 1 to 4, characterized in that the grouping means determines whether there is any overlap between the projected area of ​​the page projected onto the imaging target and the area of ​​an existing group, and groups the projected area of ​​the imaging target based on the result of the determination.

[0097] (Item 6) The imaging control device according to item 5, characterized in that, when the grouping means determines that there is an overlap between the projected area of ​​the page projected onto the imaging target and the area of ​​the existing group, the projected area of ​​the page projected onto the imaging target is included in the existing group.

[0098] (Item 7) The imaging control device according to item 5 or 6, characterized in that the grouping means generates a group of projected areas of the page projected onto the imaging target when it determines that there is no overlap between the projected area of ​​the page projected onto the imaging target and the area of ​​the existing group.

[0099] (Item 8) moreover, An imaging control device according to any one of items 1 to 7, characterized by comprising means for detecting the movement of the object to be imaged and obtaining the position of the object to be imaged.

[0100] (Item 9) The image capture control device according to item 1, characterized in that the control means updates the set field of view to include the groups in order from the most recent time, and the set field of view is set to the field of view after updating until the zoom rate corresponding to the set field of view is no longer included in the allowable zoom range of each group included in the set field of view.

[0101] (Item 10) The image capture control device according to any one of items 1 to 9, characterized in that the control means determines the allowable zoom range of a group including characters according to the size of the characters.

[0102] (Item 11) moreover, The imaging control device according to item 1, characterized by comprising means for generating a frame for distribution in which partial images within each of the fields of view included in the aforementioned field of view are arranged such that the positional relationship of the partial images within each of the fields of view is maintained in at least one direction, either horizontal or vertical.

[0103] (Item 12) moreover, An imaging control device according to any one of items 1 to 10, characterized by comprising means for outputting the determined angle of view to an imaging device having a camera capable of pan, tilt, and zoom control.

[0104] (Item 13) The imaging control device according to any one of items 1 to 12, characterized in that the time relating to the group includes the time of creation of the group, the update time for the group, and the instruction time for the group.

[0105] (Item 14) The imaging control device according to any one of items 1 to 13, characterized in that the imaging target includes a blackboard and a screen.

[0106] (Item 15) An imaging control method performed by an imaging control device, The grouping means of the imaging control device determines the continuity between a description added to the imaging target and a description previously added to the imaging target from a frame in which the imaging target has been captured, and groups the descriptions in the imaging target based on the result of this determination. The control means of the imaging control device controls the field of view of the imaging device to capture the description of the imaging target of the group to be distributed in the frame, based on the time for each group obtained in the grouping step. An imaging control method characterized by comprising:

[0107] (Item 16) A computer program for causing a computer to function as one of the means of an imaging control device described in any one of items 1 to 14.

[0108] This disclosure is not limited to the embodiments described above, and various modifications and alterations are possible without departing from the spirit and scope of this disclosure. Accordingly, the claims are attached to make the scope of this disclosure public. [Explanation of Symbols]

[0109] 10: Description detection unit 11: Description analysis unit 12: Projection detection unit 13: Projection analysis unit 14: Instruction detection unit 15: Instruction analysis unit 16: Target detection unit 17: Target analysis unit 18: Description generation unit 19: Field of view determination unit 20: Field of view synthesis unit

Claims

1. A grouping means that determines the continuity between a first description added to the target and a second description that was added to the target before the first description, based on the result of the determination, and groups the descriptions in the target. Based on the time for each group obtained by the grouping means, a control means controls the field of view of the imaging device to capture the description of the imaging target of the group to be distributed in the frame. Equipped with, The imaging control device is characterized in that the grouping means determines whether there is any overlap between the projected area of ​​the page projected onto the imaging target and the area of ​​an existing group, based on the result of the determination, and groups the projected area of ​​the imaging target.

2. The imaging control device according to claim 1, characterized in that the grouping means determines the spatial and / or temporal continuity between a first description added to the imaging target and a second description that was added to the imaging target prior to the first description.

3. The imaging control device according to claim 1, characterized in that, if the grouping means determines that there is continuity between a first description added to the imaging target and a second description that was added to the imaging target in the past prior to the first description, the first description added to the imaging target is included in the group of second descriptions that were added to the imaging target in the past prior to the first description.

4. The imaging control device according to claim 1, characterized in that if the grouping means determines that there is no continuity between the first description added to the imaging target and the second description that was added to the imaging target in the past prior to the first description, it generates a group of the first descriptions added to the imaging target.

5. The imaging control device according to claim 1, characterized in that, if the grouping means determines that there is an overlap between the projected area of ​​the page projected onto the imaging target and the area of ​​the existing group, the projected area of ​​the page projected onto the imaging target is included in the existing group.

6. The imaging control device according to claim 1, characterized in that the grouping means generates a group of projected areas of the page projected onto the imaging target when it determines that there is no overlap between the projected area of ​​the page projected onto the imaging target and the area of ​​the existing group.

7. moreover, The imaging control device according to claim 1, further comprising means for detecting the movement of the object to be imaged and obtaining the position of the object to be imaged.

8. A grouping means for determining the continuity between a first description added to the imaging target and a second description that was added to the imaging target prior to the first description, and for grouping the descriptions in the imaging target based on the result of the determination, Based on the time for each group obtained by the grouping means, a control means controls the field of view of the imaging device to capture the description of the imaging target of the group to be distributed in the frame. Equipped with, The image capture control device is characterized in that the control means updates the set field of view to include the groups in order from the most recent group, and updates the set field of view until the zoom rate corresponding to the set field of view is no longer included in the allowable zoom range of each group included in the set field of view, and the updated set field of view is set as the field of view.

9. A grouping means for determining the continuity between a first description added to the imaging target and a second description that was added to the imaging target prior to the first description, and for grouping the descriptions in the imaging target based on the result of the determination, Based on the time for each group obtained by the grouping means, a control means controls the field of view of the imaging device to capture the description of the imaging target of the group to be distributed in the frame. Equipped with, The control means is characterized by determining the permissible zoom range of a group including characters according to the size of the characters.

10. A grouping means for determining the continuity between a first description added to the imaging target and a second description that was added to the imaging target prior to the first description, and for grouping the descriptions in the imaging target based on the result of the determination, A control means for controlling the field of view of the imaging device to capture the description of the imaging target of the group to be distributed in the frame, based on the time for each group obtained by the grouping means, Means for generating a frame for distribution, in which partial images within each of the fields of view included in the aforementioned field of view are arranged such that the positional relationship of the partial images within each field of view is maintained in at least one of the horizontal and vertical directions. An imaging control device characterized by comprising:

11. moreover, The imaging control device according to claim 1, characterized in that it includes means for outputting the determined angle of view to an imaging device having a camera capable of pan, tilt, and zoom control.

12. The imaging control device according to claim 1, characterized in that the time relating to the group includes the time of creation of the group, the update time for the group, and the instruction time for the group.

13. The imaging control device according to claim 1, characterized in that the imaging target includes a blackboard and a screen.

14. An imaging control method performed by an imaging control device, The grouping means of the imaging control device determines the continuity between a first description added to the imaging target and a second description that was added to the imaging target prior to the first description, and groups the descriptions in the imaging target based on the result of this determination. The control means of the imaging control device controls the field of view of the imaging device to capture the description of the imaging target of the group to be distributed in the frame, based on the time for each group obtained in the grouping step. Equipped with, The imaging control method is characterized in that, in the grouping step, it is determined whether there is any overlap between the projected area of ​​the page projected onto the imaging target and the area of ​​an existing group, and based on the result of the determination, the projected area of ​​the imaging target is grouped.

15. A computer program for causing a computer to function as one of the means of an imaging control device according to any one of claims 1 to 13.

Images