Communication management system, communication system, communication management device, image processing method and program

Abstract

To prevent an operation different from the intention of a user performed to a stroke image by an operation from a plurality of terminal devices, in a session which shares the stroke image among the plurality of terminal devices.SOLUTION: A communication management system, which manages a session which shares a stroke image among a plurality of terminal devices, includes: a stroke information management unit which manages stroke information including a plurality of stroke data which represent the stroke image; a receiver unit which receives, from the first terminal device, group operation information designating one or more stroke data to be operated among the plurality of stroke data; and an operation restriction unit which, based on the group operation information, restricts an operation, related to the one or more stroke data, from a second terminal device which is different from the first terminal device.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a communication management system, a communication system, a communication management device, an image processing method, and a program. [Background technology]

[0002] A conference system in which a conference is held among a plurality of terminal devices via a communication network such as the Internet has become widespread. In such a conference system, a stroke image is shared by transmitting and receiving stroke data for reproducing a stroke image drawn by a user with an electronic pen or the user's hand, using a terminal device such as an electronic whiteboard, between the terminal devices.

[0003] In addition, in such conference systems, a technique is known for displaying information regarding operations by users at other locations on a display or the like so that users at each location can understand operations by users at other locations in advance (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] In a conference where stroke images are shared among multiple terminal devices, when operations are performed on the shared stroke images from multiple terminal devices, there is a problem that operations different from the user's intention may be performed on the stroke images depending on the timing of the operations.

[0005] Note that such a problem is not limited to conference systems that hold conferences, but is common to various communication systems in which sessions for sharing stroke images are held, such as remote classes and remote medical treatments.

[0006] The embodiment of the present invention has been made in consideration of the above-mentioned problems, and aims to prevent operations from being performed on a stroke image from multiple terminal devices in a session in which stroke images are shared among multiple terminal devices, which operations are different from the user's intention. [Means for solving the problem]

[0007] In order to solve the above problem, a communication management system according to an embodiment of the present invention is a communication management system that manages a session for sharing stroke images among a plurality of terminal devices, comprising: The device includes a stroke information management unit that manages stroke information including a plurality of stroke data representing the stroke image, a receiving unit that receives group operation information from a first terminal device that designates one or more stroke data to be operated among the plurality of stroke data, and an operation restriction unit that restricts an operation on the one or more stroke data by a second terminal device different from the first terminal device based on the group operation information. Effect of the Invention

[0008] According to one embodiment of the present invention, in a session in which stroke images are shared among multiple terminal devices, it is possible to prevent operations from being performed on a stroke image from multiple terminal devices that are different from the user's intention. [Brief description of the drawings]

[0009] [Figure 1] FIG. 2 is a schematic diagram illustrating an example of a communication route in a communication system according to an embodiment. [Diagram 2] FIG. 1 is a diagram for explaining an image of how an electronic whiteboard is used according to an embodiment; [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of an electronic whiteboard according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to an embodiment. [Diagram 5]FIG. 1 is a diagram illustrating an example of a system configuration of a communication system according to an embodiment. [Figure 6] FIG. 1 is a diagram illustrating an example of a functional configuration of a communication system according to an embodiment. [Figure 7] FIG. 13 is a diagram illustrating an example of an authentication management table according to an embodiment. [Figure 8] FIG. 13 is a diagram illustrating an example of a terminal management table according to an embodiment. [Figure 9] FIG. 13 is a diagram illustrating an example of a destination list management table according to an embodiment. [Figure 10] FIG. 13 is a diagram illustrating an example of a session management table according to an embodiment. [Figure 11] FIG. 13 is a diagram illustrating an example of a relay device management table according to an embodiment. [Figure 12] FIG. 11 is a diagram illustrating stroke information according to an embodiment. [Figure 13] FIG. 11 is a sequence diagram illustrating an example of a preparatory process according to an embodiment. [Figure 14] FIG. 11 is a sequence diagram illustrating an example of a session start process according to an embodiment. [Figure 15] FIG. 11 is a sequence diagram illustrating an example of a stroke image sharing process according to an embodiment. [Figure 16] FIG. 11 is a sequence diagram illustrating an example of a stroke image moving process according to an embodiment. [Figure 17] FIG. 13 is a diagram illustrating an example of a display screen according to an embodiment. [Figure 18] FIG. 1 is a sequence diagram (1) showing an example of an operation restriction process according to the first embodiment. [Figure 19] FIG. 11 is a diagram illustrating an example of group operation information according to an embodiment. [Figure 20] FIG. 11 is a sequence diagram (2) illustrating an example of the operation restriction process according to the first embodiment. [Figure 21] FIG. 11 is a sequence diagram illustrating an example of an operation restriction process according to the second embodiment. [Figure 22] FIG. 11 is a diagram showing an example of a display screen according to the second embodiment. [Diagram 23]FIG. 13 is a sequence diagram illustrating an example of an operation restriction process according to the third embodiment. [Figure 24] FIG. 13 is a diagram showing an example of a display screen according to the third embodiment. [Diagram 25] FIG. 11 is a diagram illustrating another example of a usage scene according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0011] [Outline of the communication system] <Communication route> First, a communication system 100 for conducting a video conference while drawing on multiple electronic whiteboards 1a and 1b will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing communication routes in this embodiment. Also, a "video conference" may be called a "television conference" or a "remote conference", etc.

[0012] Note that a "video conference", a "television conference", and a "remote conference" are examples of a session in which a stroke image is shared among a plurality of terminal devices. For example, the session in which a stroke image is shared may be a session other than a conference, such as a remote lesson, a remote medical treatment, a meeting, or a simple conversation.

[0013] The communication system 100 is constructed by, for example, a plurality of electronic whiteboards 1a and 1b, a relay device 3, a communication management system 5, an image storage device 7, and the like.

[0014] The electronic whiteboards (terminal devices) 1a and 1b are terminal devices that perform mutual communication of content data such as video data and audio data for calls, image data for sharing, and stroke data, and are also called IWBs (Interactive White Boards).

[0015] The communication system 100 may have three or more electronic whiteboards, not limited to two electronic whiteboards 1a and 1b. In the following description, the term "electronic whiteboard 1" will be used to refer to any one of the electronic whiteboards included in the communication system 100.

[0016] The electronic whiteboard 1 is an example of a terminal device according to the present embodiment. The terminal device may be another terminal device having a communication function, a drawing function, a display function, etc. For example, the terminal device may be an information terminal such as a PC (Personal Computer), a tablet terminal, or a smartphone in which an application program corresponding to the communication system 100 is installed.

[0017] The stroke data is data necessary to reproduce (reproduce) a stroke image, and includes coordinate data, line width data, line color data, vector data, etc. In addition, the electronic whiteboards 1a and 1b can transmit and receive video data and audio data for a call, reproducing the video and audio of the other party, thereby enabling a video call.

[0018] The electronic whiteboards 1a and 1b transmit and receive image data of a shared document image, allowing users of the communication system 100 to share the same document image. The document image is an image displayed on the display of the electronic whiteboard 1, and may include, for example, meeting documents, background images displayed on the display, and images of a captured screen when the display screen is captured.

[0019] Moreover, the electronic whiteboards 1a and 1b transmit and receive stroke data of a shared stroke image, so that participants using the communication system can share the same stroke image. A stroke image is an image showing lines drawn by a user using handwritten strokes with an electronic pen or the like. A stroke image is displayed by stroke data showing points that specify coordinates on a display.

[0020] 1, an electronic whiteboard equipped with a video conference function is shown as an example of the electronic whiteboards 1a and 1b. Note that the image of the video data may be a moving image or a still image.

[0021] In the following description, the electronic whiteboard 1 that requests the start of a video conference is called the "start terminal," and the electronic whiteboard 1 that is the destination (relay destination) of the request is called the "destination terminal." For example, in FIG. 1, when electronic whiteboard 1a requests electronic whiteboard 1b to start a video conference, electronic whiteboard 1a is the start terminal, and electronic whiteboard 1b is the destination terminal. Note that each electronic whiteboard 1b, 1b may be used not only for communication between multiple offices or between different rooms in the same office, but also for communication within the same room, or communication between outdoors and indoors, or outdoors and outdoors.

[0022] The relay device 3 is configured by a computer, and performs a process of relaying content data for communication between the plurality of electronic whiteboards 1a and 1b.

[0023] The communication management system 5 is configured by a computer, and centrally manages the authentication of logins from the electronic whiteboards 1a and 1b, the management of the communication status of the electronic whiteboards 1a and 1b, the management of destination lists, and the communication status of the relay device 3. The communication management system 5 also relays shared stroke data between the multiple electronic whiteboards 1a and 1b.

[0024] The image storage device 7 is configured by a computer, and stores image data of the shared document image uploaded from the electronic whiteboard 1a and downloads it to the electronic whiteboard 1b. The reverse is also performed. That is, the image storage device 7 stores image data uploaded from the electronic whiteboard 1b and downloads it to the electronic whiteboard 1a.

[0025] In addition, the relay device 3, the communication management system 5, and the image storage device 7 may be constructed by a single computer, or may be constructed by multiple computers in which each part (function or means) is divided and arbitrarily assigned.

[0026] In addition, in the communication system 100, a management information session sei (hereinafter referred to as "session sei") for transmitting and receiving various management information is established between the electronic whiteboards 1a and 1b via the communication management system 5. Furthermore, four sessions for transmitting and receiving four types of data, high-resolution image data, medium-resolution image data, low-resolution image data, and sound data, are established between the electronic whiteboards 1a and 1b via the relay device 3. In FIG. 1, these four sessions are collectively shown as an image and sound data session sed (hereinafter simply referred to as "session"). Note that the number of sessions does not necessarily have to be four, and the number of sessions may be less or more than four. Furthermore, a session may be established directly between the start terminal and the destination terminal without the relay device 3.

[0027] Furthermore, in the communication system 100, stroke data can be transmitted and received between the electronic whiteboards 1a and 1b using the session sei.

[0028] Here, the image resolution of the image data handled in this embodiment will be described. Low-resolution image data is, for example, 160 pixels wide and 120 pixels high, and serves as a base image. Medium-resolution image data is, for example, 320 pixels wide and 240 pixels high. High-resolution image data is, for example, 640 pixels wide and 480 pixels high. Among these, when passing through a narrowband path, low-quality image data consisting only of low-resolution image data serving as a base image is relayed. When the band is relatively wide, low-resolution image data serving as a base image and medium-quality image data consisting of medium-resolution image data are relayed. Also, when the band is very wide, high-quality image data consisting of low-resolution image data serving as a base image, medium-resolution image data, and high-resolution image data are relayed. Since the amount of sound data is smaller than that of video data, it is relayed even through a narrowband path.

[0029] <Image of using the electronic whiteboard> Fig. 2 is a diagram for explaining an image of use of an electronic whiteboard according to an embodiment. As shown in Fig. 2, the electronic whiteboard 1 has a leg 151 with a plurality of casters provided on the lower side, a support 152 provided on the upper side of the leg 151, a main body 153 of the electronic whiteboard 1 provided on the upper side of the support 152, and a display 160 provided on the front side of the main body 153. The main body 153 has a built-in CPU 101, which will be described later, and the like. A user can input (draw) a stroke image such as a character on the display 160 using an electronic pen 140 or the like.

[0030] [Hardware configuration] Next, the hardware configuration of each device according to this embodiment will be described.

[0031] <Hardware configuration of the electronic whiteboard> Fig. 3 is a diagram showing an example of a hardware configuration of an electronic whiteboard according to an embodiment. As shown in Fig. 3, the electronic whiteboard 1 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an SSD (Solid State Drive) 104, a network I / F (Interface) 105, and an external device connection I / F 106.

[0032] Of these, the CPU 101 is a calculation device that controls the overall operation of the electronic whiteboard 1. The ROM 102 is a non-volatile memory that stores the CPU 101 and programs used to start up the CPU 101, such as an IPL (Initial Program Loader). The RAM 103 is a volatile memory that is used as a work area for the CPU 101. The SSD 104 is a large-capacity storage device that stores various data such as programs for the electronic whiteboard 1.

[0033] The network I / F 105 is a communication interface for connecting the electronic whiteboard 1 to a communication network and performing communication. The external device connection I / F 106 is an interface for connecting various external devices. In this case, the external devices include, for example, a USB (Universal Serial Bus) memory 131, external devices (a microphone 132, a speaker 133, a camera 134), and the like.

[0034] The electronic whiteboard 1 also includes a capture device 111, a GPU (Graphics Processing Unit) 112, a display controller 113, a contact sensor 114, a sensor controller 115, an electronic pen controller 116, a short-range communication circuit 119, an antenna 119a of the short-range communication circuit 119, a power switch 117, and selection switches 118.

[0035] Of these, the capture device 111 captures (acquires) a display screen or the like displayed on a display or the like of an external PC (Personal Computer) 150 as a still image or a moving image. The GPU 112 is a semiconductor chip (processor) that specializes in graphics. The display controller 113 controls and manages the screen display in order to output an output image from the GPU 112 to a display 160 or the like. The contact sensor 114 detects that the electronic pen 140, the user's hand H, or the like has come into contact with the display 160. The sensor controller 115 controls the processing of the contact sensor 114.

[0036] The contact sensor 114 performs input of coordinates and detection of coordinates by, for example, an infrared blocking method. The method of inputting coordinates and detecting coordinates is such that two light receiving / emitting devices installed at both ends of the upper side of the display 160 emit a plurality of infrared rays parallel to the display 160, and receive the light reflected by a reflecting member provided around the display 160 and returning on the same optical path as the optical path of the light emitted by the light receiving element. The contact sensor 114 outputs the ID of the infrared rays emitted by the two light receiving / emitting devices that are blocked by an object to the sensor controller 115, and the sensor controller 115 specifies the coordinate position that is the contact position of the object. The electronic pen controller 116 communicates with the electronic pen 140 to determine whether the pen tip or the pen tail touches the display 160. The near field communication circuit 119 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark).

[0037] The power switch 117 is a switch for switching on / off the power of the electronic whiteboard 1. The selection switches 118 are a group of switches for adjusting the brightness and color of the display 160, for example.

[0038] Furthermore, the electronic whiteboard 1 includes a bus line 120. The bus line 120 includes an address bus, a data bus, various control signals, and the like for electrically connecting the components such as the CPU 101 shown in FIG.

[0039] The contact sensor 114 is not limited to an infrared blocking type, and may be a capacitive touch panel that identifies the contact position by detecting a change in capacitance, or a resistive film type touch panel that identifies the contact position by a voltage change between two opposing resistive films. The contact sensor 114 may also be an electromagnetic induction type touch panel that identifies the contact position by detecting electromagnetic induction caused by a contact object touching the display unit. Furthermore, the electronic pen controller 116 may determine the presence or absence of a touch not only on the tip and end of the electronic pen 140, but also on the part of the electronic pen 140 that the user holds and other parts of the electronic pen.

[0040] <Hardware configuration of management device, relay device, image storage device, PC, etc.> The communication management system 5, the relay device 3, the image storage device 7, and the PC 150, etc., each have a hardware configuration of a computer 400 as shown in Fig. 4. Alternatively, the communication management system 5, the relay device 3, and the image storage device 7 may be realized by a plurality of computers 400.

[0041] 4 is a diagram showing an example of a hardware configuration of a computer according to an embodiment. The computer 400 includes, for example, a CPU 401, a ROM 402, a RAM 403, a hard disk (HD) 404, a hard disk drive (HDD) controller 405, a display 406, an external device connection I / F 407, a network I / F 408, a keyboard 409, a pointing device 410, a digital versatile disk rewritable (DVD-RW) drive 412, a media I / F 414, and a bus line 415.

[0042] Of these, the CPU 401 is an arithmetic unit that controls the overall operation of the computer 400. The ROM 402 is a non-volatile memory that stores programs such as an IPL used to drive the CPU 401. The RAM 403 is a volatile memory used as a work area for the CPU 401. The HD 404 is a large-capacity storage device that stores programs such as an OS (Operating System) and applications, various data, etc. The HDD controller 405 controls the reading and writing of various data from and to the HD 404 under the control of the CPU 401.

[0043] The display 406 displays various information such as a cursor, a menu, a window, characters, or an image. The external device connection I / F 407 is an interface for connecting various external devices. The network I / F 408 is a communication interface for performing data communication using a communication network. The keyboard 409 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 410 is a type of input means for selecting and executing various instructions, selecting a processing target, moving the cursor, etc.

[0044] The DVD-RW drive 412 controls reading and writing (storing) of data from and to the DVD-RW 411. The DVD-RW drive 412 may control reading and writing (storing) of data from and to a disc such as a BD-RE (Blu-ray (registered trademark) Disc Rewritable) instead of the DVD-RW 411. The media I / F 414 controls reading and writing (storing) of data from and to a recording medium 413 such as a flash memory. The bus line 415 includes an address bus, a data bus, various control signals, and the like for electrically connecting each component such as the CPU 401 shown in FIG. 4.

[0045] [Overall configuration of communication system] Next, a description will be given of the overall configuration of the communication system 100. Fig. 5 is a diagram showing an example of the system configuration of a communication system according to an embodiment.

[0046] In FIG. 5, electronic whiteboard 1a is installed at location A, and electronic whiteboard 1b is installed at location B. For example, location A is an office in Tokyo, Japan, and location B is an office in Osaka, Japan. At location A, user A uses electronic whiteboard 1a, and at location B, user B and user C use electronic whiteboard 1b. Electronic whiteboard 1a is an example of a first terminal device, and electronic whiteboard 1b is an example of a second terminal device.

[0047] Moreover, the electronic whiteboards 1a and 1b, the relay device 3, the communication management system 5, and the image storage device 7 can transmit and receive data to and from each other via a communication network 500 such as the Internet or a LAN (Local Area Network). The communication network 500 may include a wireless communication portion. In Fig. 5, the electronic whiteboards 1a and 1b are assumed to be electronic whiteboards capable of transmitting and receiving conference videos.

[0048] [Functional configuration of communication system] Next, a functional configuration of the communication system 100 according to the present embodiment will be described. Fig. 6 is a diagram showing an example of the functional configuration of the communication system according to an embodiment. In Fig. 6, the electronic whiteboard 1b has the same functional configuration as the electronic whiteboard 1a.

[0049] <Functional configuration of the electronic whiteboard> Each electronic whiteboard 1 realizes a transmitting / receiving unit 11, a receiving unit 12, a video / audio processing unit 13, a display control unit 14, an image processing unit 15, an information transmitting unit 16, a short-range communication unit 17, and a storage / reading processing unit 18 by executing a predetermined program with a CPU 101 in Fig. 3. At least a part of the above-mentioned functional configurations may be realized by hardware. In addition, the electronic whiteboard 1 has a memory unit 1001 realized by the RAM 103, SSD 104, etc. in Fig. 3.

[0050] (Functional configuration of electronic whiteboard 1) The transmission / reception unit 11 transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 500. The transmission / reception unit 11 also serves as an initiation unit, and performs a login process to the communication management system 5 or the like, and a process of initiating communication with other terminal devices.

[0051] The receiving unit 12 receives various inputs from a user using the electronic pen 140 or the like. For example, the receiving unit 12 receives a designation operation for designating one or more stroke data to be the target of an operation, a change operation related to the stroke data, a drawing operation of a stroke image, and the like.

[0052] The video / audio processor 13 performs main processing of the video conference function. For example, the video / audio processor 13 performs digital processing such as encoding video data and audio data based on the output signal of the microphone 132 and the output signal of the camera 134. The video / audio processor 13 also generates video signals and audio signals based on the video data and audio data received by the transmission / reception unit 11. The video / audio processor 13 also performs processing such as combining video data with different resolutions.

[0053] The display control unit 14 outputs a video signal (image signal) and the like to the display 160, and performs control for displaying a display screen.

[0054] The image processing unit 15 performs main processing of the electronic whiteboard function. For example, the image processing unit 15 performs processing to create a stroke image and stroke data based on strokes of the electronic pen 140 or the like received by the reception unit 12, processing to create a stroke image based on stroke data received by the transmission / reception unit 11, etc. The image processing unit 15 also performs processing to generate an image signal based on image data of a document image received by the transmission / reception unit 11.

[0055] The information transmitting unit 16 transmits the stroke data created by the image processing unit 15 to the communication management system 5. In addition, the information transmitting unit 16 transmits group operation information designating one or more stroke data in response to a designation operation that is accepted by the accepting unit 12 and that designates one or more stroke data to be the target of an operation to the communication management system 5.

[0056] The short-range communication unit 17 obtains and provides data between each terminal having a short-range communication unit through short-range wireless communication.

[0057] The storage / read processing unit 18 performs processing such as storing various data in the storage unit 1001 or a recording medium 1002 such as a USB memory 131, reading out various data stored in the storage unit 1001 or the recording medium 1002, and the like.

[0058] Furthermore, video data and audio data received during communication with other terminals are overwritten and stored every time they are received in the storage unit 1001. Of these, images are displayed on the display 160 using the video data before being overwritten, and audio is output from the speaker 133 using the audio data before being overwritten.

[0059] <Functional configuration of the communication management system> The communication management system (communication management device) 5, for example, realizes a transmission / reception unit 51, an authentication unit 52, a terminal management unit 53, a session management unit 54, a relay device management unit 55, a stroke information management unit 56, an operation restriction unit 57, and a storage / read processing unit 58 by executing a predetermined program in the CPU 401 of FIG. 4. Each of the above-mentioned functional configurations may be realized by a program executed by a plurality of computers 400. Also, at least a part of each of the above-mentioned functional configurations may be realized by hardware. Also, the communication management system 5 has a storage unit 5000 realized by the HD 504 of FIG. 4 or the like. The storage unit 5000 stores, for example, an authentication management DB (Database) 5001, a terminal management DB 5002, a destination list management DB 5003, a session management DB 5004, a relay device management DB 5005, and stroke information 5006.

[0060] (Authentication Management Table) Fig. 7 is a diagram showing an example of an authentication management table according to an embodiment. For example, an authentication management DB 5001 configured by an authentication management table 700 as shown in Fig. 7 is stored in the storage unit 5000. In this authentication management table 700, each password is associated with each terminal ID of all the electronic whiteboards 1 managed by the communication management system 5 and managed. For example, the authentication management table 700 shown in Fig. 7 indicates that the password corresponding to the terminal ID "01aa" of the electronic whiteboard 1a (terminal device) is "aaaa".

[0061] The terminal ID is identification information for identifying the electronic whiteboard (terminal device) 1, and is an example of identification information for the communication management system 5 to manage terminal devices such as the electronic whiteboard 1. For example, the communication management system 5 may manage the electronic whiteboard 1 using a user ID for identifying a user, a contract ID for identifying a contractor, a communication ID for identifying a communication source, or the like, instead of the terminal ID.

[0062] Moreover, the password is an example of authentication information for authenticating a terminal device such as the interactive whiteboard 1. The authentication information may be authentication information other than a password, such as an access token, for example.

[0063] (Terminal Management Table) Fig. 8 is a diagram showing an example of a terminal management table according to an embodiment. The storage unit 5000 stores a terminal management DB 5002 configured by a terminal management table 800 as shown in Fig. 8. In this terminal management table 800, for each terminal ID for identifying each electronic whiteboard 1 (terminal device), the destination name when each electronic whiteboard 1 is the destination, the operation status of each electronic whiteboard 1, the reception date and time when login request information described later is received by the communication management system 5, and the IP address of each electronic whiteboard 1 (terminal device) are associated and managed.

[0064] 8, the terminal management table 800 indicates that the electronic whiteboard 1a with the terminal ID "01aa" has the terminal name "Japan Tokyo Office AA Terminal" and the operating status "Online (communication available)". The electronic whiteboard 1a with the terminal ID "01aa" also indicates that the date and time when the login request information was received by the communication management system 5 was "April 10, 2015, 13:40" and the IP address was "1.2.1.3". The terminal ID, destination name, and terminal IP address, etc. are stored when each electronic whiteboard 1 is pre-registered to receive services provided by the communication management system 5.

[0065] (Destination list management table) FIG. 9 is a diagram showing an example of a destination list management table according to an embodiment. The storage unit 5000 stores a destination list management DB 5003 configured by a destination list management table 900 as shown in FIG. 9. In this destination list management table 900, all terminal IDs of destination terminals registered as candidates of the electronic whiteboard 1 (destination terminal) are associated with the terminal ID of the electronic whiteboard 1 (start terminal) requesting the start of communication and are managed. For example, the destination list management table 900 shown in FIG. 9 shows that the destination terminal candidates that can request the start of communication from the start terminal (electronic whiteboard 1a) with the terminal ID "01aa" include the electronic whiteboard 1b with the terminal ID "01ba". The destination terminal candidates are updated by being added or deleted in response to an addition or deletion request from any start terminal to the communication management system 5.

[0066] The destination list is an example of destination information, and the destination information may not be in a list format, but may instead include a list of information related to the destinations, such as terminal IDs.

[0067] (Session management table) FIG. 10 is a diagram showing an example of a session management table according to an embodiment. The storage unit 5000 stores a session management DB 5004 configured by a session management table 1000 as shown in FIG. 10, for example. In this session management table 1000, for each session ID for identifying a session, the device ID of the relay device 3 to be used, the terminal ID of the starting terminal, the terminal ID of the destination terminal, and the terminal IDs of one or more participating terminals are associated and managed. In addition, in the session management table 1000, for example, information such as a reception delay time (ms) when the video data is received at the destination terminal and a reception date and time when the delay information indicating this delay time is received from the destination terminal are further associated and managed. For example, in the session management table 1000 shown in FIG. 10, in a session with a session ID "se01", a relay device 3 with a relay device ID "111a" relays communication between a start terminal with a terminal ID "01aa" and a destination terminal with a terminal ID "01ba". It also shows that in the session with session ID "se01", the delay time of the video data at "13:41 on April 10, 2015" is 200 (ms).

[0068] (Relay device management table) FIG. 11 is a diagram showing an example of a relay device management table according to an embodiment. The storage unit 5000 stores a relay device management DB 5005 configured by a relay device management table 1100 as shown in FIG. 11, for example. In this relay device management table 1100, the operation state of each relay device 3, the reception date and time of the state information indicating the operation state, the IP address of the relay device 3, and the maximum data transmission rate (Mbps) are associated and managed for each relay device ID of a plurality of relay devices 3. For example, in the relay device management table 1100 shown in FIG. 11, it is shown that the operation state of the relay device 3 with the relay device ID "111a" is "online" and the date and time when the state information was received by the communication management system 5 is "12:00 on April 10, 2015". In addition, it is shown that the IP address of the relay device 3 with the relay device ID "111a" is "1.2.1.2" and the maximum data transmission rate of the relay device 3 is 100 Mbps.

[0069] (Stroke information) 12 is a diagram for explaining stroke information according to an embodiment. This diagram shows an example of stroke information 5006 stored in the storage unit 5000.

[0070] In the example of FIG. 12, the stroke information 5006 has a hierarchical structure of "meeting," "pages (pages 1 to 3)," and "strokes (strokes 1 to 5)."

[0071] A "stroke" corresponds to stroke data according to this embodiment, and is managed for each "page" by the stroke information management unit 56. For example, stroke 1 includes a stroke ID (id=aaaaaa) that identifies stroke 1, information that specifies parent page 1 (parent=hijkl), a Gid "g001" that identifies drawing data 1, and drawing data 1. For example, when an operation such as moving is performed on a stroke image represented by stroke 1, the stroke ID (id=aaaaaa) is carried over as is, but drawing data 1 and the Gid "g001" that identifies drawing data 1 are updated.

[0072] The drawing data 1 is drawing data for reproducing the stroke 1, and includes, for example, coordinate data, line width data, line color data, vector data, and the like.

[0073] The electronic whiteboard 1 allows pages to be switched, and a "page" corresponds to each page of the electronic whiteboard 1. For example, page 1 includes information such as a page ID (id=hijkl) that identifies page 1, and information for specifying the parent "meeting" (parent=abcdefg).

[0074] A "conference" corresponds to a session managed by the communication system 100, and is managed by, for example, a session ID (id=abcdefg) that identifies the session.

[0075] For example, the image processor 15 of the electronic whiteboard 1 can create a stroke image drawn on a page 1 by using drawing data 1 to 5 included in strokes 1 to 5.

[0076] The session ID (id=abcdefg) is an example of a first identification information for identifying a session (conference), and the stroke ID (id=aaaaaa) is an example of a second identification information for identifying stroke data. Moreover, the Gid "g001" for identifying drawing data 1 is an example of a third identification information for identifying drawing data.

[0077] (Functional configuration of communication management system) Returning now to FIG. 6, the functional configuration of the communication management system 5 will be described.

[0078] The transmitting / receiving unit 51 transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 500. For example, the transmitting / receiving unit 51 functions as a receiving unit that receives group operation information that designates one or more stroke data to be the target of an operation from the electronic whiteboard 1.

[0079] The authentication unit 52 authenticates the login request received via the transmission / reception unit 51. For example, when a combination of a terminal ID and a password included in the login request received from the electronic whiteboard 1 is stored in the authentication management table 700 as shown in FIG.

[0080] The terminal management unit 53 manages the terminal management DB 5002, and updates information such as the operating state, reception date and time, and terminal IP address in the terminal management table 800 shown in FIG.

[0081] The terminal manager 53 also manages a destination list management table 900, and provides destination list information including the terminal IDs of destination terminals in the destination list management table 900 as shown in FIG. 9 in response to a request from the electronic whiteboard 1, for example.

[0082] The session management unit 54 manages sessions held in the communication system 100. For example, the session management unit 54 generates a session ID for identifying a session in response to start request information requesting the start of communication from the electronic whiteboard 1. In addition, the session management unit 54 stores and manages various information related to the session in a session management table 1000 as shown in FIG. 10 in association with the session ID.

[0083] The relay device management unit 55 manages the relay device management DB 5005 and selects a relay device 3 to be used for relaying a session, for example, by using a relay device management table 1100 as shown in Fig. 11. For example, the relay device management unit 55 may select a relay device 3 near the starting terminal based on the IP address of each relay device 3 stored in the relay device management table 1100 and the IP address of the starting terminal. Alternatively, the relay device management unit 55 may select a relay device 3 based on the maximum data transmission speed of each relay device 3 stored in the relay device management table 1100.

[0084] In this embodiment, the method of selecting the relay device 3 to be used for relaying a session may be any method. Also, the number of relay devices 3 included in the communication system 100 may be one.

[0085] The stroke information management unit 56 stores and manages stroke data transmitted, for example, in the session sei from the electronic whiteboard 1 participating in the session, in the storage unit 5000 or the like in association with the session ID. For example, the stroke information management unit 56 stores stroke data received from the electronic whiteboard 1 in stroke information 5006 as shown in Fig. 12. In addition, the stroke information management unit 56 transfers stroke data received from the electronic whiteboard 1 to another electronic whiteboard 1 participating in the same session as the electronic whiteboard 1.

[0086] The operation limiting unit 57 limits operations on the designated stroke data by other electronic whiteboards 1 based on group operation information that specifies one or more stroke data to be operated and that is received by the transmitting / receiving unit 51 from the electronic whiteboard 1. The process by the operation limiting unit 57 and the group operation information will be described later.

[0087] The storage / read processing unit 58 performs processing such as storing various data in the storage unit 5000 and reading out various data stored in the storage unit 5000.

[0088] <Functional configuration of the image storage device> The image storage device 7 realizes a transmission / reception unit 71, a storage / readout processing unit 72, etc., by executing a predetermined program in the CPU 401 in Fig. 4. The image storage device 7 also has a storage unit 7000 realized by the RAM 403, HD 404, etc. in Fig. 4.

[0089] (Functional configuration of image storage device) The transmitting / receiving unit 71 transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 500. The storage / reading processing unit 72 stores various data in the storage unit 7000 and reads out various data stored in the storage unit 7000.

[0090] With the above configuration, the image storage device 7 stores, for example, image data uploaded from the electronic whiteboard 1 in a specified URL (Uniform Resource Locator), and provides the image data stored in the specified URL in response to a request from the electronic whiteboard 1.

[0091] <Functional configuration of relay device> The relay device 3 realizes a transmission / reception unit 31, a determination unit 32, a storage / readout processing unit 33, etc., by executing a predetermined program in the CPU 401 in Fig. 4. The relay device 3 also has a storage unit 3000 realized by the RAM 403, HD 404, etc. in Fig. 4.

[0092] (Functional configuration of relay device) The transmitting / receiving unit 31 transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 500. The transmitting / receiving unit 31 also serves as a transfer unit, and transfers, for example, video data and audio data received from the electronic whiteboard 1 to other electronic whiteboards 1 participating in the same session as the electronic whiteboard 1. The determining unit 32 performs various determinations, such as determining the state of data delays, etc.

[0093] The storage / read processing unit 33 performs processing such as storing various data in the storage unit 3000 and reading out various data stored in the storage unit 3000.

[0094] 6 is an example, and the communication system 100 can be modified in various ways. For example, each functional configuration included in the communication management system 5 may be realized by a plurality of computers 400. Furthermore, the function of the image storage device 7 may be realized by a storage server or the like external to the communication system 100. Furthermore, at least a part of each functional configuration of the image storage device 7 and the relay device 3 may be included in the communication management system 5. Furthermore, the electronic whiteboard 1 is an example of a terminal device, and the terminal device may be an information terminal that executes an application corresponding to the communication system 100.

[0095] [Processing flow] Next, the process flow of the communication management method and image processing method according to this embodiment will be described.

[0096] <Preparation for remote communication> 13 is a sequence diagram showing an example of a preparatory process according to an embodiment. This process shows an example of a preparatory process executed before the interactive whiteboard 1 participates in a session (video conference) in the communication system 100.

[0097] First, when the power switch 117 of the interactive whiteboard 1a is turned on, the reception unit 12 receives the power-on command (step S22).

[0098] Next, the transmitting / receiving unit 11 transmits login request information requesting login to the communication management system 5 via the communication network 500 (step S23). This login request information includes, for example, the terminal ID and password of the electronic whiteboard 1a.

[0099] Next, the authentication unit 52 of the communication management system 5 searches the authentication management table 700 as shown in Fig. 7 using the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys. Furthermore, if the combination of the terminal ID and password included in the login request information is managed in the authentication management table 700, the authentication unit 52 permits login to the electronic whiteboard 1a (step S24). Here, if the combination of the terminal ID and password included in the login request information is managed in the authentication management table 700, the processing from step S25 onwards is executed.

[0100] When the login of the electronic whiteboard 1a is permitted, the terminal management unit 53 updates the information corresponding to the terminal ID "01aa" of the electronic whiteboard 1a in the terminal management table 800 as shown in FIG. 8. For example, the terminal management unit 53 changes the information of "operation status" corresponding to the terminal ID "01aa" to "online (communication possible)" and updates the information of "received date and time" to the date and time when the login request information was received (step S25). The IP address information of the terminal is not registered in advance, and the IP address transmitted from the electronic whiteboard 1a in the above step S23 may be used. As a result, in the terminal management table 800, for example, as shown in FIG. 8, the terminal ID "01aa" is managed in association with the operation status "online (communication possible)", the received date and time "2015.4.10.13:40", the IP address "1.2.1.3", and the like.

[0101] Next, the session management unit 54 adds a new record in which the terminal ID "01aa" of the electronic whiteboard 1a received in step S23 is set as the "terminal ID of the starting terminal" to the session management table 1000 as shown in Fig. 10 and manages the new record (step S26). Then, the transmission / reception unit 51 of the communication management system 5 transmits authentication result information indicating the authentication result obtained by the processing of step 24 to the electronic whiteboard 1a that made the login request via the communication network 500 (step S27).

[0102] When the transmitting / receiving unit 11 of the electronic whiteboard 1a receives the authentication result information indicating that the login is permitted, it transmits destination list request information indicating a request for a destination list to the communication management system 5 via the communication network 500 (step S28). As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the destination list request information.

[0103] Next, the terminal management unit 53 searches the destination list management table 900 as shown in Fig. 9 using the terminal ID "01aa" of the electronic whiteboard 1a as a search key, and reads out the terminal IDs of destination candidates that can communicate with the electronic whiteboard 1a. The terminal management unit 53 also reads out the destination names corresponding to the terminal IDs of the destination candidates from the terminal management table 800 as shown in Fig. 8 (step S29). As a result, the terminal IDs of the destination candidates corresponding to the terminal ID "01aa" of the electronic whiteboard 1a and the destination names corresponding to these are extracted.

[0104] Next, the transmitting / receiving unit 51 of the communication management system 5 reads, for example, data of the destination list frame and data of the icon indicating the operating state stored in the memory unit 5000 via the memory / readout processing unit 58 (step S30). The transmitting / receiving unit 51 of the communication management system 5 also transmits "destination list information (destination list frame, icon, terminal ID, destination name)" including the read destination list frame and icon, and the terminal ID and destination name extracted in step S29, to the electronic whiteboard 1a (step S31). As a result, in the electronic whiteboard 1a, the transmitting / receiving unit 11 receives the destination list information, and the memory / readout processing unit 18 stores the received destination list information in the memory unit 1001 (step S32).

[0105] In this manner, in this embodiment, the destination list information of all terminals is centrally managed by the communication management system 5, rather than managing the destination list information on each electronic whiteboard 1. Therefore, even when a new electronic whiteboard 1 is registered in the communication management system 5, when a new model of terminal device is registered in place of an electronic whiteboard 1 already included, when the appearance of the destination list frame is changed, etc., it is possible to eliminate the need to change the destination list information on each electronic whiteboard 1.

[0106] 8 using the terminal IDs of the destination candidates extracted in step S29 as a search key, and reads out the corresponding operating status for each of the destination candidate terminal IDs. As a result, the terminal management unit 53 acquires the operating status of each electronic whiteboard 1 corresponding to the destination candidate terminal ID (step S33).

[0107] Next, the transmitter / receiver unit 51 of the communication management system 5 transmits "terminal status information" including the terminal ID used as the search key in step S33 and the operating status of each corresponding destination terminal to the electronic whiteboard 1a via the communication network 500 (step S34).

[0108] Next, the storage / readout processing unit 18 of the electronic whiteboard 1a sequentially stores the terminal status information received from the communication management system 5 in the storage unit 1001 (step S35). Thus, by receiving the above-mentioned electronic whiteboard 1 status information, the electronic whiteboard 1a can obtain the current operating status of the electronic whiteboards 1b, 1c, etc., which are potential destinations that can communicate with the electronic whiteboard 1a.

[0109] Next, the display control unit 14 of the electronic whiteboard 1a creates a destination list that reflects the status of the terminal as a destination candidate, based on the destination list information and the terminal status information stored in the storage unit 1001. In addition, the display control unit 14 uses the created destination list to display a destination list screen on the display 160 of the electronic whiteboard 1a (step S36).

[0110] Meanwhile, the terminal management unit 53 of the communication management system 5 extracts the terminal IDs of other terminals that have registered the electronic whiteboard 1a as a destination candidate by searching the destination list management table 900 as shown in FIG. 9 based on the terminal ID "01aa" of the electronic whiteboard 1a (step S37).

[0111] Next, the terminal management unit 53 of the communication management system 5 searches the terminal management table 800 as shown in FIG. 8 based on the terminal ID "01aa" of the electronic whiteboard 1a, and acquires the operation status of the electronic whiteboard 1a (step S38).

[0112] Then, the transmitting / receiving unit 51 transmits "terminal status information" including the terminal ID and the operating status of the electronic whiteboard 1a to the electronic whiteboard 1 of the terminal ID whose operating status is "online" in the terminal management table 800 among the terminal IDs extracted in step S37 (step S39). When transmitting the terminal status information to each electronic whiteboard 1, the transmitting / receiving unit 51 refers to the IP address of each electronic whiteboard 1 managed in the terminal management table 800 based on each terminal ID. This makes it possible to transmit the terminal ID "01aa" of the electronic whiteboard 1a and the operating status "online" to each electronic whiteboard 1 that can communicate with the electronic whiteboard 1a as a destination candidate. This makes it possible to display the status of the electronic whiteboard 1a on each electronic whiteboard 1 that is a destination candidate (step S40).

[0113] <Session start processing> 14 is a sequence diagram showing an example of a session start process according to an embodiment. This process shows an example of a session start process executed in the communication system 100 when, for example, the electronic whiteboard 1a requests the electronic whiteboard 1b to start a session.

[0114] In step S1401, when the electronic whiteboard 1a, which is the start terminal, accepts the destination selection by the user A, the processes from step S1402 onwards are executed. Here, it is assumed that the electronic whiteboard 1b is selected as the destination terminal by the user A.

[0115] In step S1402, when the electronic whiteboard 1a accepts the destination selection by user A, the transmitting / receiving unit 11 of the electronic whiteboard 1a transmits start request information requesting the start of a session to the communication management system 5. This start request information includes, for example, the terminal ID "01aa" of the electronic whiteboard 1a which is the start terminal, the terminal ID "01ba" of the electronic whiteboard 1b which is the destination terminal, and the IP address of the electronic whiteboard 1a which is the start terminal. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the start request information and also receives the IP address of the start terminal (electronic whiteboard 1a) which is the sender.

[0116] In step S1403, the terminal management unit 53 of the communication management system 5 updates the terminal management DB 5002 based on the terminal ID of the starting terminal and the terminal ID of the destination terminal included in the start request information. For example, based on the terminal ID "01aa" of the starting terminal and the terminal ID "01ba" of the destination terminal included in the start request information, the terminal management unit 53 updates the "operating status" corresponding to the terminal IDs "01aa" and "01ba" in the terminal management table 800 to "online (communicating)". Note that in this state, the electronic whiteboard 1a, which is the starting terminal, and the electronic whiteboard 1b, which is the destination terminal, are not participating in the session, but are managed as being in a communicating state.

[0117] In step S1404, the session management unit 54 of the communication management system 5 generates a session ID for identifying a session held between the electronic whiteboard 1a and the electronic whiteboard 1b. Here, it is assumed that the session management unit 54 generates the session ID "se01".

[0118] In step S1405, the session management unit 54 stores and manages the generated session ID, the terminal ID "01aa" of the initiating terminal, the terminal ID "01ba" of the destination terminal, etc. in a session management table 1000 as shown in FIG. 10 in association with each other.

[0119] In step S1406, the relay device management unit 55 of the communication management system 5 selects a relay device 3 that will relay the session between the starting terminal (electronic whiteboard 1a) and the destination terminal (electronic whiteboard 1b). For example, the relay device management unit 55 selects a relay device 3 having an IP address close to the IP address of the starting terminal (electronic whiteboard 1a) in the terminal management table 800 from among the device IDs of relay devices 3 whose operating status is "online" in the relay device management table 1100. Here, it is assumed that the relay device 3 with the device ID "111a" is selected.

[0120] In steps S1407a and S1407b, the session management unit 54 of the communication management system 5 transmits relay device selection information to the start terminal (electronic whiteboard 1a) and the destination terminal (electronic whiteboard 1b) via the transmission / reception unit 51. This relay device selection information includes, for example, the IP address of the relay device 3 selected in step S1406, the session ID generated in step S1404, and the IP address of the communication management system 5.

[0121] In steps S1408a and S1408b, when the transmitter / receiver 11 of the initiating terminal (electronic whiteboard 1a) and the destination terminal (electronic whiteboard 1b) receive the relay device selection information, they transmit reception completion information indicating that the relay device selection information has been received to the communication management system 5. This reception completion information includes, for example, the session ID "se01" and the like that was included in the relay device selection information.

[0122] In steps S1409a and S1409b, the transmitting / receiving units 11 of the initiating terminal (electronic whiteboard 1a) and the destination terminal (electronic whiteboard 1b) establish a session with the relay device 3 using the device ID and session ID of the relay device 3 included in the relay device selection information. As a result, the electronic whiteboard 1a and the electronic whiteboard 1b participate in the session with the same session ID "se01" and can transmit and receive video data such as conference video, audio data, and the like to each other via the relay device 3.

[0123] By the above process, a video conference can be held between user A of the electronic whiteboard 1a and user B of the electronic whiteboard 1b, transmitting and receiving conference video and the like.

[0124] <Sharing of stroke images> In the communication system 100, a user A of the electronic whiteboard 1a and users B and C of the electronic whiteboard 1b participating in a video conference can display a stroke image drawn with an electronic pen 140 or the like on each electronic whiteboard 1 as a shared image on the other electronic whiteboards 1. Here, an example of a process for sharing a stroke image between the electronic whiteboard 1a and the electronic whiteboard 1b participating in the same session will be described.

[0125] Fig. 15 is a sequence diagram showing an example of a stroke image sharing process according to an embodiment. At the start of the process shown in Fig. 15, the electronic whiteboard 1a and the electronic whiteboard 1b are assumed to be participating in the same session by, for example, the session start process shown in Fig. 14.

[0126] Furthermore, the electronic whiteboards 1a and 1b share document images via the image storage device 7 as necessary, and display the document images on the display 160.

[0127] In step S1501, user A draws (moves) the electronic pen 140 or hand H on the display 160 of the electronic whiteboard 1a, and the accepting unit 12 accepts input of the drawn stroke (trajectory).

[0128] In step S1502, the image processing unit 15 of the electronic whiteboard 1a creates stroke data for displaying a stroke image corresponding to the stroke accepted by the acceptance unit 12.

[0129] In step S1503, the display control unit 14 of the electronic whiteboard 1a causes the display 160 to display a stroke image based on the stroke data created by the image processing unit 15. As a result, the stroke image drawn by the user A on the display 160 with the electronic pen 140 or the like is displayed on the display 160.

[0130] In step S1504, the transmitting / receiving unit 11 of the electronic whiteboard 1a transmits stroke data for reproducing the stroke image drawn by the user A to the communication management system 5, for example, through the session sei. As a result, the transmitting / receiving unit 51 of the communication management system 5 receives the stroke data transmitted from the electronic whiteboard 1a.

[0131] In step S1505, the stroke information management unit 56 of the communication management system 5 registers the stroke data received from the electronic whiteboard 1a in, for example, stroke information 5006 as shown in FIG.

[0132] In step S1506, the stroke information management unit 56 of the communication management system 5 transfers the stroke data to the electronic whiteboard 1b participating in the same session as the electronic whiteboard 1a. As a result, the transmission / reception unit 11 of the electronic whiteboard 1b receives the stroke data.

[0133] In step S1507, the image processing unit 15 of the interactive whiteboard 1b creates a stroke image based on the stroke data received by the transmitting / receiving unit 11.

[0134] In step S1508, the display control unit 14 of the electronic whiteboard 1b causes the display 160 to display the stroke image created by the image processing unit 15, that is, the stroke image drawn by the user A in step S1501.

[0135] In addition, when the user B or C draws on the electronic whiteboard 1b with the electronic pen 140 or the like, the stroke image can be displayed on the electronic whiteboard 1a in the same manner.

[0136] In this way, stroke images can be shared between the electronic whiteboard 1a and the electronic whiteboard 1b participating in the same session.

[0137] <Stroke image movement processing> In a conference where stroke images are shared between multiple electronic whiteboards (terminal devices) 1a and 1b, the multiple electronic whiteboards 1a and 1b can perform editing processes such as moving processes on the shared stroke images. In this case, for example, when a moving operation or the like is performed on the shared stroke images from the multiple electronic whiteboards 1a and 1b, an operation different from the user's intention may be performed on the stroke images depending on the timing of the operation.

[0138] In order to solve such a problem, the communication management system 5 has an operation restriction function that restricts an editing operation by a user B on a stroke image when a user A is performing an editing operation on the stroke image to be operated, for example.

[0139] Before describing the operation limiting function, an example of a stroke image movement process in which the operation limiting function is not used will be described.

[0140] Fig. 16 is a sequence diagram showing an example of a stroke image movement process according to an embodiment. At the start of the process shown in Fig. 16, for example, a display screen 1710 including stroke images ST1 to 5 is displayed on the displays of the electronic whiteboard 1a and the electronic whiteboard 1b by the stroke image sharing process shown in Fig. 15, as shown in Fig. 17(A). Also, for example, stroke data (strokes 1 to 5) corresponding to the stroke images ST1 to 5 are stored in the stroke information 5006 managed by the stroke information management unit 56 of the communication management system 5, as shown in Fig. 12.

[0141] Also, as an example for the purposes of explanation, it is assumed here that user A of the electronic whiteboard 1a performs an operation to sequentially move the stroke images ST1 to 5 to the left, and user B of the electronic whiteboard 1b performs an operation to sequentially move the stroke images ST2 to 4 downward.

[0142] In step S1601, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST1 (Gid=g001) to the left by a predetermined distance in response to a moving operation on the stroke image ST1 by the user A to the communication management system 5. This operation information includes, for example, coordinate information of the stroke image ST1 after the movement.

[0143] In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 1 of the stroke 1 corresponding to the stroke image ST1 (Gid=g001) and Gid in the stroke information 5006 shown in Fig. 12. As a result, for example, the stroke image ST1 shared between the electronic whiteboard 1a and the electronic whiteboard 1b moves to the left as shown in the display screen 1720 shown in Fig. 17(B).

[0144] In step S1602, the stroke information management unit 56 of the communication management system 5 transmits a success notification indicating that the movement of the stroke image ST1 has been successful to the electronic whiteboard 1a. The success notification includes the updated gid (for example, Gid=g006) of the stroke image ST1.

[0145] In step S1603, the information transmission unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST2 (Gid=g002) to the left by a predetermined distance in response to the moving operation on the stroke image ST2 by the user A to the communication management system 5. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 2 and Gid of the stroke 2 corresponding to the stroke image ST2 (Gid=g002) in the stroke information 5006 shown in FIG. 12, for example. As a result, the stroke image ST2 shared between the electronic whiteboard 1a and the electronic whiteboard 1b moves to the left, for example, as shown in a display screen 1730 shown in FIG. 17(C).

[0146] In step S1604, the stroke information management unit 56 of the communication management system 5 transmits a success notification indicating that the movement of the stroke image ST2 has been successful to the electronic whiteboard 1a. The success notification includes the updated gid (for example, Gid=g007) of the stroke image ST2.

[0147] In step S1605, the information transmitting unit 16 of the electronic whiteboard 1b transmits operation information for moving the stroke image ST2 (Gid=g002) downward by a predetermined distance to the communication management system 5 in response to the moving operation on the stroke image ST2 by the user B. However, since the gid of the stroke image ST2 has been updated to "g007", the stroke information managing unit 56 of the communication management system 5 cannot perform the moving process of the stroke image ST2 (Gid=g002).

[0148] In step S1606, the stroke information management unit 56 of the communication management system 5 transmits a failure notification indicating that the movement of the stroke image ST2 has failed to the electronic whiteboard 1b.

[0149] In step S1607, the information transmission unit 16 of the electronic whiteboard 1b transmits operation information for moving the stroke image ST3 (Gid=g003) downward by a predetermined distance to the communication management system 5 in response to the movement operation on the stroke image ST3 by the user B. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 3 and Gid of the stroke 3 corresponding to the stroke image ST3 (Gid=g003) in the stroke information 5006 shown in FIG. 12, for example. As a result, the stroke image ST3 shared between the electronic whiteboard 1a and the electronic whiteboard 1b moves downward, for example, as shown in a display screen 1740 shown in FIG. 17(D).

[0150] In step S1608, the stroke information management unit 56 of the communication management system 5 transmits a success notification indicating that the movement of the stroke image ST3 has been successful to the electronic whiteboard 1b. The success notification includes the updated gid (for example, Gid=g008) of the stroke image ST3.

[0151] In step S1609, the information transmission unit 16 of the electronic whiteboard 1b transmits operation information for moving the stroke image ST4 (Gid=g004) downward by a predetermined distance to the communication management system 5 in response to the movement operation on the stroke image ST4 by the user B. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 3 and Gid of the stroke 4 corresponding to the stroke image ST4 (Gid=g004) in the stroke information 5006 shown in FIG. 12, for example. As a result, the stroke image ST4 shared between the electronic whiteboard 1a and the electronic whiteboard 1b moves downward, for example, as shown in a display screen 1750 shown in FIG. 17(E).

[0152] In step S1610, the stroke information management unit 56 of the communication management system 5 transmits a success notification indicating that the movement of the stroke image ST4 has been successful to the electronic whiteboard 1b. The success notification includes the updated gid (for example, Gid=g009) of the stroke image ST4.

[0153] In step S1611, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST3 (Gid=g003) a predetermined distance to the left in response to the move operation on the stroke image ST3 by the user A to the communication management system 5. However, since the gid of the stroke image ST3 has been updated to "g008", the stroke information managing unit 56 of the communication management system 5 cannot perform the move process of the stroke image ST3 (Gid=g003).

[0154] In step S1612, the stroke information management unit 56 of the communication management system 5 transmits a failure notification indicating that the movement of the stroke image ST3 has failed to the electronic whiteboard 1a.

[0155] In step S1613, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST4 (Gid=g004) a predetermined distance to the left in response to the move operation on the stroke image ST4 by the user A to the communication management system 5. However, since the gid of the stroke image ST4 has been updated to "g009", the stroke information managing unit 56 of the communication management system 5 cannot perform the move process of the stroke image ST4 (Gid=g004).

[0156] In step S1614, the stroke information management unit 56 of the communication management system 5 transmits a failure notification indicating that the movement of the stroke image ST4 has failed to the electronic whiteboard 1a.

[0157] In step S1615, the information transmission unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST5 (Gid=g005) to the left by a predetermined distance in response to the moving operation on the stroke image ST5 by the user A to the communication management system 5. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 2 and Gid of the stroke 2 corresponding to the stroke image ST5 (Gid=g005) in the stroke information 5006 shown in FIG. 12, for example. As a result, the stroke image ST5 shared between the electronic whiteboard 1a and the electronic whiteboard 1b moves to the left, for example, as shown in a display screen 1760 shown in FIG. 17(F).

[0158] In this way, when the operation limiting function is not used, as shown in the display screen 1760 of Fig. 17(F), the stroke images ST1, ST2, and ST5 move leftward in response to the operation of user A, while the stroke images ST3 and ST4 move downward in response to the operation of user B. As a result, the stroke images ST1 to 5 end up being moved to positions different from the intentions of user A and user B.

[0159] In order to solve such a problem, the communication management system 5 has an operation restriction function that restricts an editing operation by a user B on a stroke image when a user A is performing an editing operation on the stroke image to be operated, for example.

[0160] [First embodiment] 18 is a sequence diagram (1) showing an example of the operation restriction process according to the first embodiment. This process shows an example of the operation restriction process by the operation restriction unit 57 of the communication management system 5.

[0161] 18, for example, the display screen 1710 including the stroke images ST1 to ST5 is displayed on the displays of the electronic whiteboard 1a and the electronic whiteboard 1b as shown in Fig. 17(A) by the stroke image sharing process shown in Fig. 15. Also, for example, stroke data (strokes 1 to 5) corresponding to the stroke images ST1 to ST5 are stored in the stroke information 5006 managed by the stroke information management unit 56 of the communication management system 5 as shown in Fig. 12.

[0162] Also, here, as an example for explanation, it is assumed that a user A of the electronic whiteboard 1a performs an operation to sequentially move the stroke images ST1 to ST5 to the left.

[0163] In step S1801, the reception unit 12 of the interactive whiteboard 1a receives a group selection operation by user A.

[0164] In this embodiment, before performing an editing operation on a shared stroke image, a user can select the stroke image to be operated on and notify the communication management system 5 of a list of the stroke data to be operated on.

[0165] In step S1802, the information transmitting unit 16 of the electronic whiteboard 1a transmits a group operation request 1 specifying one or more stroke data to be operated in response to the group selection operation by the user A to the communication management system 5. Note that the group operation request 1 is an example of group operation information specifying one or more stroke data to be operated.

[0166] Fig. 19(a) shows an example of group operation information 1910 (group operation request 1) that the information transmission unit 16 of the electronic whiteboard 1a transmits to the communication management system 5. In the group operation information 1910 shown in Fig. 19(a), the character string ""lock":"abcdefg"" is information that specifies identification information that identifies the conference (session) to be operated. The example of Fig. 19(a) shows that the stroke information to be operated is, for example, the stroke information 5006 of the conference (id=abcdefg) shown in Fig. 12.

[0167] In addition, the character string "list":["aaaaa", "bbbbb", "ccccc", "ddddd", "eeeee"]" is information that specifies the stroke ID of the stroke data to be operated. In the example of FIG. 19(a), it is shown that the stroke data to be operated are stroke 1, stroke 2, stroke 3, stroke 4, and stroke 5.

[0168] In step S1803 of Figure 18, when the operation restriction unit 57 of the communication management system 5 receives the group operation request 1 transmitted from the electronic whiteboard 1a, it determines whether the operation target (stroke data) specified in the group operation request 1 includes an operation target (stroke data) that is under operation restriction.

[0169] If the operation targets specified in the group operation request 1 do not include any operation targets under operation restriction, the processing from step S1804 onwards is executed. On the other hand, if the operation targets specified in the group operation request 1 include any operation targets under operation restriction, the operation restriction unit 57 of the communication management system 5 transmits a rejection notice to the electronic whiteboard 1 that is the request source, as described below.

[0170] In step S1804, the operation restriction unit 57 of the communication management system 5 starts an operation restriction process for the stroke data specified in the group operation request 1. As a result, the operation restriction unit 57 prohibits operations from the other electronic whiteboards 1b regarding the stroke image specified in the group operation request 1 transmitted from the electronic whiteboard 1a.

[0171] In step S1805, the operation restriction unit 57 of the communication management system 5 transmits a reception notification indicating that the group operation request 1 has been received to the electronic whiteboard 1a that is the request source.

[0172] In step S1806, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST1 (Gid=g001) a predetermined distance to the left in response to the moving operation on the stroke image ST1 by the user A to the communication management system 5. In response to this, the stroke information managing unit 56 of the communication management system 5 updates the drawing data 1 of the stroke 1 corresponding to the stroke image ST1 (Gid=g001) in the stroke information 5006 shown in Fig. 12, for example, and updates Gid to "g006".

[0173] In step S1807, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST2 (Gid=g002) a predetermined distance to the left in response to the moving operation on the stroke image ST2 by the user A to the communication management system 5. In response to this, the stroke information managing unit 56 of the communication management system 5 updates the drawing data 2 of the stroke 2 corresponding to the stroke image ST2 (Gid=g002) in the stroke information 5006 shown in Fig. 12, for example, and updates Gid to "g007".

[0174] Here, for example, it is assumed that in step S1808, the reception unit 12 of the interactive whiteboard 1b receives a group selection operation by user B.

[0175] In step S1809, the information transmitting unit 16 of the electronic whiteboard 1b transmits a group operation request 2 specifying one or more stroke data to be operated on to the communication management system 5 in response to the group selection operation by the user B. Note that the group operation request 2 is another example of group operation information specifying one or more stroke data to be operated on.

[0176] Fig. 19(b) shows an example of group operation information 1920 (group operation request 2) that the information sending unit 16 of the electronic whiteboard 1b sends to the communication management system 5. In the group operation information 1920 shown in Fig. 19(b), the character string ""lock":"abcdefg"" indicates that the stroke information to be operated is, for example, the stroke information 5006 of the meeting (id=abcdefg) shown in Fig. 12.

[0177] In addition, the character string "list":["bbbbb","ccccc","ddddd"]" indicates that the stroke data to be operated on is stroke 2, stroke 3, and stroke 4.

[0178] In step S1810 of Figure 18, when the operation restriction unit 57 of the communication management system 5 receives the group operation request 2 sent from the electronic whiteboard 1b, it determines whether the operation target (stroke data) specified in the group operation request 2 includes an operation target (stroke data) that is under operation restriction.

[0179] Here, in step S1804, operation restriction processing is started for stroke data with stroke IDs "aaaaaa," "bbbbbb," "cccccc," [dddddd], and "eeeeee" based on group operation information 1910 shown in Fig. 19(a). Also, in group operation information 1920 shown in Fig. 19(b), stroke IDs "bbbbbb," "cccccc," and [dddddd] are specified as stroke data to be operated.

[0180] In this way, when a part or all of the specified operation target (stroke data) is included in the operation target (stroke data) under the operation restriction, the operation restriction unit 57 transmits a rejection notice (or a failure notice) to the requesting electronic whiteboard 1b in step 1811. In this case, the users B and C of the electronic whiteboard 1b cannot operate the stroke data ST2, ST3, and ST4 that are the operation targets.

[0181] In step S1812, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST3 (Gid=g003) a predetermined distance to the left in response to the moving operation on the stroke image ST3 by the user A to the communication management system 5. In response to this, the stroke information managing unit 56 of the communication management system 5 updates the drawing data 3 of the stroke 3 corresponding to the stroke image ST3 (Gid=g003) in the stroke information 5006 shown in Fig. 12, for example, and updates Gid to "g008".

[0182] In step S1813, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST4 (Gid=g004) a predetermined distance to the left in response to the move operation on the stroke image ST4 by the user A to the communication management system 5. In response to this, the stroke information managing unit 56 of the communication management system 5 updates the drawing data 4 of the stroke 4 corresponding to the stroke image ST4 (Gid=g004) in the stroke information 5006 shown in Fig. 12, for example, and updates Gid to "g009".

[0183] In step S1814, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST5 (Gid=g005) a predetermined distance to the left in response to the moving operation on the stroke image ST5 by the user A to the communication management system 5. In response to this, the stroke information managing unit 56 of the communication management system 5 updates the drawing data 5 of the stroke 4 corresponding to the stroke image ST5 (Gid=g005) in the stroke information 5006 shown in FIG. 12, for example, and updates Gid to "g010".

[0184] In step S1815, for example, when the operation restriction unit 57 of the communication management system 5 accepts operations on all stroke data specified as operation targets, it determines that the group operation is complete and ends the operation restriction process.

[0185] As another example, the operation limiting unit 57 may determine that the group operation is completed when end information of the group operation is received from the interactive whiteboard 1a.

[0186] Preferably, the operation limiting unit 57 ends the group limiting process when a predetermined time has elapsed since the operation limiting process started. This prevents the operation target from becoming unable to be edited when, for example, the electronic whiteboard 1a transmits group operation information to the communication management system 5 and leaves the session due to a failure or the like before completing operations on all stroke information to be operated.

[0187] In step S1816, the operation restriction unit 57 of the communication management system 5 transmits completion information indicating that the group operation has been completed to the electronic whiteboard 1a. The Gid of each stroke data item updated by the group operation may be notified to the electronic whiteboard 1a using this completion information, or may be notified to the electronic whiteboard 1a using, for example, a success notification as shown in step S1602 of FIG.

[0188] By the above process, for example, as shown on a display screen 1770 in FIG. 17(G), the stroke images ST1 to ST5 to be operated are moved to the left as intended by the user A.

[0189] Fig. 20 is a sequence diagram (2) showing an example of the operation restriction process according to the first embodiment. This process shows another example of the operation restriction process by the operation restriction unit 57 of the communication management system 5. Among the processes shown in Fig. 20, the processes of steps S1801 to S1807 and S1812 to S1816 are similar to the processes described in Fig. 18, so the following description will focus on the differences from the processes shown in Fig. 18.

[0190] In step S2001, the receiving unit 12 of the electronic whiteboard 1b receives a moving operation by user B to move the stroke image ST3 downward by a predetermined distance while user A is performing a group operation on the stroke images ST1 to ST5.

[0191] In step S2002, the information transmitting unit 16 of the electronic whiteboard 1b transmits operation information for moving the stroke image ST3 (Gid=g003) downward by a predetermined distance to the communication management system 5 in response to the moving operation on the stroke image ST3 by the user B. However, since the operation of the stroke image ST3 is restricted by the operation restriction process by the operation restriction unit 57 of the communication management system 5, the stroke information managing unit 56 cannot perform the moving process of the stroke image ST3 (Gid=g003).

[0192] In step S2003, the stroke information management unit 56 of the communication management system 5 transmits a failure notification indicating that the movement of the stroke image ST3 has failed to the electronic whiteboard 1b.

[0193] In this way, when the operation restriction unit 57 according to the present embodiment receives group operation information transmitted from the electronic whiteboard 1, it restricts operations from other electronic whiteboards 1 relating to the stroke image specified in the group operation information.

[0194] Therefore, according to this embodiment, in a session in which stroke images are shared among multiple terminal devices, it is possible to prevent operations from being performed on a stroke image from multiple terminal devices that are different from the user's intention.

[0195] [Second embodiment] 21 is a sequence diagram showing an example of an operation restriction process according to the second embodiment. This process shows an example of an operation restriction process by the communication management system 5 and the electronic whiteboard 1.

[0196] In the following description, it is assumed that the communication system 100 has three electronic whiteboards 1a, 1b, and 1c, and that the three electronic whiteboards 1a, 1b, and 1c are participating in a video conference in which the same stroke image is shared.

[0197] In step S2101, the reception unit 12 of the electronic whiteboard 1a receives a group selection operation by the user A who uses the electronic whiteboard 1a. Here, it is assumed that "Page 1" of the electronic whiteboard 1a is selected by the group selection operation. In this manner, the group selection operation that designates one or more stroke data to be the target of the operation may be performed on a page-by-page basis.

[0198] In step S2102, in response to the group selection operation by user A, the information transmission unit 16 of the electronic whiteboard 1a transmits a group operation request to the communication management system 5, the request including the information to be operated on, "Page 1", and the terminal ID "10aa" of the electronic whiteboard 1a.

[0199] In step S2103, when the operation restriction unit 57 of the communication management system 5 receives the group operation request sent from the electronic whiteboard 1a, it determines whether the "Page 1" specified in the group operation request includes an operation target (stroke data, page, etc.) that is under operation restriction.

[0200] If the operation targets specified in the group operation request do not include operation targets under operation restriction, the processing from step S2104 onwards is executed. On the other hand, if the operation targets specified in the group operation request include operation targets under operation restriction, the operation restriction unit 57 of the communication management system 5 stops the execution of the processing from step S2104 onwards and transmits a rejection notice to the requesting electronic whiteboard 1a. Here, the following explanation will be given on the assumption that the operation targets specified in the group operation request do not include operation targets under operation restriction.

[0201] In step S2104, the operation restriction unit 57 of the communication management system 5 starts the operation restriction process for page 1 specified in the group operation request.

[0202] In step S2105, the operation restriction unit 57 of the communication management system 5 transmits a reception notification indicating that the group operation request has been received to the electronic whiteboard 1a that is the request source.

[0203] In addition, in steps S2106 and S2107, the operation restriction unit 57 of the communication management system 5 according to the second embodiment transmits an operation restriction notification indicating that the operation restriction process has started to all other electronic whiteboards 1b and 1c participating in the video conference. This operation restriction notification includes, for example, information on the operation target that is the subject of the operation restriction (e.g., "page 1") and information on the electronic whiteboard 1a that has imposed the operation restriction on the operation target (e.g., terminal ID "10aa"). Note that the information on the electronic whiteboard 1a may be the terminal name of the electronic whiteboard 1a, the destination name managed in the terminal management table 800, or the like.

[0204] In steps S2108 and S2109, the display control units 14 of the electronic whiteboards 1b and 1c that have received the operation restriction notification from the communication management system 5 display, for example, a display screen 2200 as shown in FIG. 22, indicating that operation on page 1 is restricted.

[0205] FIG. 22 is a diagram showing an example of a display screen according to the second embodiment. This diagram shows an example of a display screen 2200 displayed by the electronic whiteboards 1b and 1c in steps S2108 and S2109 in FIG. 21. In FIG. 22, it is assumed that page 1 is displayed on the display screen 2200 of the electronic whiteboards 1b and 1c. In addition, a message 2201 indicating that operation of page 1 is restricted is displayed on the display screen 2200. This message 2201 can prevent operation on page 1 by the electronic whiteboards 1b and 1c. Preferably, this message 2201 includes information (for example, a terminal name, a destination name, a terminal ID, etc.) about the electronic whiteboard 1a that has locked (restricted operation) page 1.

[0206] The message 2201 is an example of display information indicating that operations are restricted. For example, the display information indicating that operations are restricted may be information other than character strings, such as pictograms or icons.

[0207] By the above process, user A who uses the electronic whiteboard 1a can preferentially operate the stroke images included in the operation target "Page 1."

[0208] For example, in step S2111, when user A moves the stroke image ST1 on page 1 of the electronic whiteboard 1a to the right, the processes from step S2112 onwards are executed.

[0209] In step S2112, the information transmitting unit 16 of the electronic whiteboard 1a transmits to the communication management system 5 operation information for moving the stroke image ST1 (stroke1) on page 1 by a predetermined distance to the right.

[0210] In steps S2113 and S2114, the stroke information management unit 56 of the communication management system 5 transmits operation information for moving the stroke image ST1 (stroke1) a predetermined distance to the right to the electronic whiteboards 1b and 1c.

[0211] As a result, in steps S2115 and S2116, the stroke image ST1 on page 1 of the electronic whiteboards 1b and 1c is moved to the right, and the same stroke image is displayed on page 1 of the electronic whiteboards 1a, 1b, and 1c.

[0212] In this way, the communication system 100 according to the second embodiment can prevent operations from being performed on a stroke image that are different from the user's intention due to operations from the multiple electronic whiteboards 1a, 1b, and 1c.

[0213] [Third embodiment] Fig. 23 is a sequence diagram showing an example of an operation restriction process according to the third embodiment. Note that the processes in steps S2101 to S2104 in Fig. 23 are similar to the processes according to the second embodiment described in Fig. 21, and therefore the following description will focus on the differences from the second embodiment.

[0214] In step S2104 of Figure 23, after the operation restriction unit 57 of the communication management system 5 starts the operation restriction process, there is a risk that operations will be performed on page 1 on the electronic whiteboards 1b and 1c until an operation restriction notification is sent to the electronic whiteboards 1b and 1c.

[0215] For example, assume that user B who uses the electronic whiteboard 1b moves the stroke image ST1 on page 1 to the left in step S2301 in Fig. 23. In this case, the communication management system 5 and the electronic whiteboard 1 execute the processes of steps S2302 to S2304.

[0216] In step S2302, the information transmitting unit 16 of the electronic whiteboard 1b transmits to the communication management system 5 operation information for moving the stroke image ST1 (stroke1) on page 1 by a predetermined distance to the left.

[0217] However, since the operation of page 1 has already been restricted by the operation restriction process performed by the operation restriction unit 57 of the communication management system 5, the stroke information management unit 56 cannot move the stroke image ST1 of page 1. In this case, in step S2303, the stroke information management unit 56 of the communication management system 5 transmits a failure notification indicating that the movement of the stroke image ST1 of page 1 has failed to the electronic whiteboard 1b.

[0218] In step S2304, when the image processor 15 of the electronic whiteboard 1b receives a failure notification from the communication management system 5, the image processor 15 of the electronic whiteboard 1b returns the stroke image ST1 of the page 1 that has failed to be moved to its original position. This allows the stroke images of the page 1 displayed on the electronic whiteboards 1a, 1b, and 1c to be maintained in the same state.

[0219] In the second embodiment, when the electronic whiteboard 1b displays the display screen 2200 as shown in FIG. 22, if a move operation is received on the stroke image ST1 of page 1, the processing of steps S2302 to S2304 in FIG. 23 can also be applied.

[0220] Next, a process after the operation limiting unit 57 of the communication management system 5 transmits a reception notification indicating that the group operation request has been received to the requesting electronic whiteboard 1a in step S2305 will be described.

[0221] In steps S2306 and S2307, the operation restriction unit 57 of the communication management system 5 An operation restriction notice indicating that the operation restriction process has started is transmitted to all other electronic whiteboards 1b and 1c participating in the video conference.

[0222] In steps S2308 and S2309, the electronic whiteboards 1b and 1c that have received the operation restriction notification from the communication management system 5 prohibit operations on the stroke image of page 1 that is subject to the operation restriction. For example, the display control units 14 of the electronic whiteboards 1b and 1c display a display screen 2400 as shown in FIG. 24, and the reception units 12 of the electronic whiteboards 1b and 1c prohibit reception of editing operations on the stroke image of page 1.

[0223] 24 is a diagram showing an example of a display screen according to the third embodiment. This diagram shows an example of a display screen 2400 displayed by the electronic whiteboards 1b and 1c in steps S2308 and S2309 in FIG.

[0224] 24, it is assumed that page 1 is displayed on the display screen 2400 of the electronic whiteboards 1b and 1c. However, page 1 is displayed, for example, grayed out so that it cannot be operated. Also, a message 2401 is displayed on the display screen 2400 indicating that operation of page 1 is restricted and cannot be operated. Preferably, this message 2401 includes information (for example, terminal name, destination name, terminal ID, etc.) related to the electronic whiteboard 1a that has locked page 1 (operation restriction).

[0225] By the above process, editing operations cannot be performed on the stroke images on page 1 from the electronic whiteboards 1b and 1c, so that user A who uses the electronic whiteboard 1a can operate the stroke images included in page 1 with priority.

[0226] In the second and third embodiments, as an example, an example in which the operation restriction is applied to the stroke image of page 1 has been described, but the stroke data to be the object of the operation restriction may be one or more selected stroke images, as in the first embodiment. Also, the stroke data to be the object of the operation restriction may be stroke images of multiple pages.

[0227] As described above, according to each embodiment of the present invention, in a session in which stroke images are shared among a plurality of terminal devices, it is possible to prevent operations from being performed on a stroke image that are different from the user's intention due to operations from the plurality of terminal devices.

[0228] <Supplementary Information> Each function of each embodiment described above can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above.

[0229] In the above-described embodiment, the terminal device is described as being the electronic whiteboard 1, but the present invention is not limited to this. For example, each embodiment of the present invention can also be applied to an interactive projector 2501 as shown in Fig. 25. The interactive projector 2501 is a terminal device that projects a display screen 2503 onto a projection surface 2502, and allows a user to write stroke images and select menus with a pen, a user's finger, or the like, in the same manner as the electronic whiteboard 1.

[0230] Other examples of the terminal device may include information terminals such as a PC, a smartphone, a smart watch, and a car navigation device that executes an application corresponding to the communication system 100. The terminal device may also include medical equipment. In the case of a medical equipment, the document image may include an image of a patient, etc.

[0231] Furthermore, in each of the above-described embodiments, the communication system 100 is used for video conferencing, but the present invention is not limited to this and may be used for meetings, general conversations between family members or friends, remote medical treatment, or one-way presentation of information. [Explanation of symbols]

[0232] 1. Electronic whiteboard (an example of a terminal device) 5. Communications management system (communications management device) 51 Transmitting / receiving unit (receiving unit) 56 Stroke information management section 57 Operation Restriction Section 100 Communication Systems 1910, 1920 Group Operation Information 5006 Stroke Information [Prior art documents] [Patent documents]

[0233] [Patent Document 1] JP 2015-60592 A

Claims

1. A communication management system that manages a session between a plurality of terminal devices including a first terminal device and a second terminal device, in which a user using the first terminal device and a user using the second terminal device share stroke images showing handwritten strokes, comprising: a stroke information management unit that manages stroke information including a plurality of stroke data representing the stroke image; a transmitting / receiving unit that receives, from the first terminal device, information regarding an operation received from a user using the first terminal device, the information regarding an operation related to editing one or more stroke data that are targets of an editing operation among the plurality of stroke data; an operation restriction unit that restricts an operation on one or more stroke data by the second terminal device different from the first terminal device, based on information regarding an operation received from a user using the first terminal device, the information being information regarding an operation related to editing one or more stroke data that are targets of an editing operation among the plurality of stroke data; and A communication management system in which the transmitter / receiver unit transmits information identifying a user using the first terminal device to the second terminal device when the operation is restricted, so that the second terminal device displays information related to the user using the first terminal device when the operation is restricted.

2. A communication management system as described in Claim 1, wherein information regarding an operation related to editing one or more stroke data that are the target of the editing operation is a group operation request that specifies the one or more stroke data that are the target of the editing operation among the plurality of stroke data.

3. A communication management system as described in claim 1 or 2, characterized in that the information identifying the user using the first terminal device includes at least one of a user ID identifying the user, a terminal name of the first terminal device used by the user, and a terminal name of the second terminal device used by the user.

4. 3. The communication management system according to claim 2, wherein the operation restriction unit prohibits the second terminal device from operating on the one or more stroke data if the one or more stroke data specified by the group operation request do not include stroke data for which operation is being restricted.

5. 5. The communication management system according to claim 2, wherein the operation restriction unit, when the one or more stroke data specified by the group operation request includes stroke data for which operation is being restricted, discontinues the process of prohibiting operation of the one or more stroke data by the second terminal device.

6. The operation limiting unit A communication management system as described in claim 2 or 4, wherein after prohibiting operations on the one or more stroke data by the second terminal device, if the first terminal device performs an operation on each of the one or more stroke data that are the target of the operation, the prohibition on operations on the one or more stroke data is terminated.

7. the stroke information management unit manages the plurality of stroke data in association with first identification information that identifies the session; 5. The communication management system according to claim 2, wherein the stroke data includes second identification information for identifying the stroke data, drawing data for reproducing the stroke image, and third identification information for identifying the drawing data.

8. The communication management system according to claim 7 , wherein the group operation request includes the first identification information and the second identification information that identifies the one or more stroke data.

9. 3. A communication system including the communication management system according to claim 2 and a plurality of terminal devices participating in a session managed by the communication management system, The terminal device a receiving unit that receives a designation operation for designating one or more stroke data to be operated on and an editing operation for the one or more stroke data; an information sending unit that sends a group operation request to the communication management system in response to the specifying operation, the group operation request specifying one or more stroke data items to be subjected to the editing operation; A communication system comprising:

10. The communication system described in claim 9, wherein the terminal device has a display control unit that displays display information on a display screen indicating that operations regarding one or more stroke data are restricted in response to an operation restriction notification notified from the communication management system that restricts operations regarding the one or more stroke data.

11. The communication system according to claim 9 , wherein the reception unit prohibits the reception of editing operations on one or more stroke data in response to an operation restriction notification that restricts operations on the one or more stroke data, which is notified from the communication management system.

12. The operation restriction unit of the communication management system 10. The communication system according to claim 9, wherein, based on a group operation request received from the first terminal device and specifying one or more stroke data to be subjected to the editing operation, an operation restriction notification indicating that operations relating to the one or more stroke data are restricted is transmitted to each of the second terminal devices among the plurality of terminal devices, the second terminal device being different from the first terminal device.

13. A communication management device that manages a session between a plurality of terminal devices including a first terminal device and a second terminal device, in which a user using the first terminal device and a user using the second terminal device share stroke images showing handwritten strokes, comprising: a stroke information management unit that manages stroke information including a plurality of stroke data representing the stroke image; a transmitting / receiving unit that receives, from the first terminal device, information regarding an operation received from a user using the first terminal device, the information regarding an operation related to editing one or more stroke data that are targets of an editing operation among the plurality of stroke data; an operation restriction unit that restricts an operation on one or more stroke data by the second terminal device different from the first terminal device, based on information regarding an operation received from a user using the first terminal device, the information being information regarding an operation related to editing one or more stroke data that are targets of an editing operation among the plurality of stroke data; and A communication management device in which the transmitter / receiver unit transmits information identifying a user using the first terminal device to the second terminal device when the operation is restricted, so that the second terminal device displays information related to the user using the first terminal device when the operation is restricted.

14. A communication management system that manages a session in which a user using a first terminal device and a user using a second terminal device share stroke images showing handwritten strokes between a plurality of terminal devices including a first terminal device and a second terminal device, comprising: a process of managing stroke information including a plurality of stroke data representing the stroke image; a transmission / reception process for receiving, from the first terminal device, information regarding an operation received from a user using the first terminal device, the information regarding an operation related to editing one or more stroke data that are targets of an editing operation among the plurality of stroke data; a process of restricting an operation on one or more stroke data by the second terminal device different from the first terminal device, based on information regarding an operation received from a user using the first terminal device, the information being information regarding an operation related to editing one or more stroke data that are targets of an editing operation among the plurality of stroke data; Run The transmission / reception process is an image processing method in which, when the operation is restricted, information identifying a user using the first terminal device is transmitted to the second terminal device so that the second terminal device displays information related to the user using the first terminal device when the operation is restricted.

15. A computer that manages a session in which a user using a first terminal device and a user using a second terminal device share stroke images showing handwritten strokes between a plurality of terminal devices including the first terminal device and a second terminal device, a stroke information management unit that manages stroke information including a plurality of stroke data representing the stroke image; a transmitting / receiving unit that receives, from the first terminal device, information regarding an operation received from a user using the first terminal device, the information regarding an operation related to editing one or more stroke data that are targets of an editing operation among the plurality of stroke data; an operation restriction unit that restricts an operation on one or more stroke data by the second terminal device different from the first terminal device, based on information regarding an operation received from a user using the first terminal device, the information being information regarding an operation related to editing one or more stroke data that are targets of an editing operation among the plurality of stroke data; It functions as The program includes a transmitter / receiver unit that transmits information identifying a user using the first terminal device to the second terminal device when the operation is restricted, so that the second terminal device displays information related to the user using the first terminal device when the operation is restricted.

16. The program described in Claim 15, wherein information regarding an operation related to editing one or more stroke data that are the target of the editing operation is a group operation request that specifies the one or more stroke data that are the target of the editing operation among the plurality of stroke data.

17. The program described in Claim 15 or 16, characterized in that the information identifying the user using the first terminal device includes at least one of a user ID that identifies the user, a terminal name of the first terminal device used by the user, and a terminal name of the second terminal device used by the user.

18. The program described in Claim 16, wherein the operation restriction unit prohibits the second terminal device from operating on the one or more stroke data if the one or more stroke data specified by the group operation request does not include stroke data for which operation is being restricted.

19. A program as described in claim 16 or 18, wherein the operation restriction unit, if the one or more stroke data specified by the group operation request includes stroke data for which operation is being restricted, discontinues the process of prohibiting operation of the one or more stroke data by the second terminal device.

20. The operation restriction unit:

19. The program according to claim 16 or 18, wherein after prohibiting an operation on the one or more stroke data by the second terminal device, if the first terminal device performs an operation on each of the one or more stroke data that are the target of the operation, the prohibition on the operation on the one or more stroke data is terminated.

21. The stroke information management unit manages the plurality of stroke data in association with first identification information that identifies the session; 19. The program according to claim 16, wherein the stroke data includes second identification information for identifying the stroke data, drawing data for reproducing the stroke image, and third identification information for identifying the drawing data.

22. The program described in Claim 21, wherein the group operation request includes the first identification information and the second identification information that identifies the one or more stroke data.

Images