COMMUNICATION MANAGEMENT SYSTEM, COMMUNICATION ...MANAGEMENT DEVICE, IMAGE PROCESSING METHOD, AND PROGRAM

Abstract

To prevent an operation different from the intention of a user performed to a stroke image by an operation from a plurality of terminal devices, in a session which shares the stroke image among the plurality of terminal devices.SOLUTION: A communication management system, which manages a session which shares a stroke image among a plurality of terminal devices, includes: a stroke information management unit which manages stroke information including a plurality of stroke data which represent the stroke image; a receiver unit which receives, from the first terminal device, group operation information designating one or more stroke data to be operated among the plurality of stroke data; and an operation restriction unit which, based on the group operation information, restricts an operation, related to the one or more stroke data, from a second terminal device which is different from the first terminal device.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a communication management system, a communication system, a communication management device, an image processing method, and a program. [Background technology]

[0002] Conference systems in which a conference is held among a plurality of terminal devices via a communication network such as the Internet are becoming widespread. In such conference systems, stroke images drawn by users with an electronic pen, their hands, or the like are shared by transmitting and receiving stroke data for reproducing the stroke images using terminal devices such as electronic whiteboards.

[0003] In addition, in such conference systems, a technique is known in which information regarding operations by users at other locations is displayed on a display or the like so that users at each location can understand operations by users at other locations in advance (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] In a conference where stroke images are shared among multiple terminal devices, when operations are performed on the shared stroke images from multiple terminal devices, there is a problem that, depending on the timing of the operations, operations on the stroke images may be performed differently from the user's intention.

[0005] Note that this problem is not limited to conference systems that hold conferences, but is also common to various communication systems that hold sessions in which stroke images are shared, such as remote lessons and remote medical treatments.

[0006] The embodiment of the present invention has been made in consideration of the above-mentioned problems, and aims to prevent operations on stroke images that are different from the user's intention from being performed on the stroke images through operations from multiple terminal devices in a session in which stroke images are shared among multiple terminal devices. [Means for solving the problem]

[0007] In order to solve the above problem, a communication management system according to an embodiment of the present invention comprises: A communication management system for managing a session for sharing stroke images between a plurality of terminal devices including a first terminal device and a second terminal device, the system comprising: a stroke information management unit for managing stroke information including a plurality of stroke data representing the stroke images; a receiving unit for receiving, from the first terminal device, a group operation request for specifying one or more stroke data to be subjected to an editing operation among the plurality of stroke data; and an operation restriction unit for restricting an editing operation on the one or more stroke data by the second terminal device different from the first terminal device, based on the group operation request for specifying the one or more stroke data to be subjected to the editing operation, The operation restriction unit request The one or more stroke data specified by Editing by operation restriction processing If the stroke data during operation restriction is not included, By starting the operation restriction process, an editing operation on the one or more stroke data from the first terminal device can be accepted; Regarding the one or more stroke data by the second terminal device edit Prohibit operation and when the one or more stroke data specified by the group operation request includes stroke data for which editing operations are being restricted by the operation restriction process, editing operations on the one or more stroke data from the first terminal device are not accepted, thereby prohibiting editing operations on the one or more stroke data by the first terminal device. [Effects of the Invention]

[0008] According to one embodiment of the present invention, in a session in which stroke images are shared among multiple terminal devices, it is possible to prevent operations from being performed on the stroke images from multiple terminal devices that are different from the user's intention. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a schematic diagram illustrating an example of a communication route in a communication system according to an embodiment. [Figure 2] FIG. 1 is a diagram for explaining an image of how an electronic whiteboard is used according to an embodiment. [Figure 3] FIG. 1 is a diagram illustrating an example of a hardware configuration of an interactive whiteboard according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to an embodiment. [Figure 5] FIG. 1 is a diagram illustrating an example of a system configuration of a communication system according to an embodiment. [Figure 6]FIG. 1 is a diagram illustrating an example of a functional configuration of a communication system according to an embodiment. [Figure 7] FIG. 10 is a diagram illustrating an example of an authentication management table according to an embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a terminal management table according to an embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of a destination list management table according to an embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a session management table according to an embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of a relay device management table according to an embodiment. [Figure 12] FIG. 10 is a diagram illustrating stroke information according to an embodiment. [Figure 13] FIG. 10 is a sequence diagram illustrating an example of a preparatory stage process according to an embodiment. [Figure 14] FIG. 10 is a sequence diagram illustrating an example of a session start process according to an embodiment. [Figure 15] FIG. 10 is a sequence diagram illustrating an example of a stroke image sharing process according to an embodiment. [Figure 16] FIG. 10 is a sequence diagram illustrating an example of a stroke image movement process according to an embodiment. [Figure 17] FIG. 10 is a diagram illustrating an example of a display screen according to an embodiment. [Figure 18] FIG. 1 is a sequence diagram (1) showing an example of an operation restriction process according to the first embodiment. [Figure 19] FIG. 10 is a diagram illustrating an example of group operation information according to an embodiment. [Figure 20] FIG. 10 is a sequence diagram (2) illustrating an example of the operation restriction process according to the first embodiment. [Figure 21] FIG. 10 is a sequence diagram illustrating an example of an operation restriction process according to the second embodiment. [Figure 22] FIG. 10 is a diagram showing an example of a display screen according to the second embodiment. [Figure 23] FIG. 11 is a sequence diagram illustrating an example of an operation restriction process according to the third embodiment. [Figure 24]FIG. 11 is a diagram showing an example of a display screen according to the third embodiment. [Figure 25] FIG. 10 is a diagram illustrating another example of a usage scene according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0011] [Communication system overview] <Communication Route> First, a communication system 100 for conducting a video conference while drawing on multiple electronic whiteboards 1a and 1b will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing communication routes in this embodiment. Also, a "video conference" may be called a "teleconference" or "remote conference," etc.

[0012] Note that a "video conference," "teleconference," or "remote conference" is an example of a session in which stroke images are shared among multiple terminal devices. For example, the session in which stroke images are shared may be a session other than a conference, such as a remote lesson, a remote medical consultation, a meeting, or a simple conversation.

[0013] The communication system 100 is constructed by, for example, a plurality of electronic whiteboards 1a and 1b, a relay device 3, a communication management system 5, an image storage device 7, and the like.

[0014] The electronic whiteboards (terminal devices) 1a and 1b are terminal devices that mutually communicate content data such as video data and audio data for calls, image data for sharing, and stroke data, and are also called IWBs (Interactive White Boards).

[0015] The communication system 100 is not limited to two electronic whiteboards 1a and 1b, but may have three or more electronic whiteboards. In the following description, the term "electronic whiteboard 1" will be used to refer to any one of the electronic whiteboards included in the communication system 100.

[0016] The electronic whiteboard 1 is an example of a terminal device according to the present embodiment. The terminal device may be another terminal device having a communication function, a drawing function, a display function, etc. For example, the terminal device may be an information terminal such as a PC (Personal Computer), a tablet terminal, or a smartphone, on which an application program compatible with the communication system 100 is installed.

[0017] The stroke data is data necessary to reproduce (reproduce) a stroke image, and includes coordinate data, line width data, line color data, vector data, etc. Furthermore, the electronic whiteboards 1a and 1b transmit and receive video data and audio data for a call, thereby reproducing the video and audio of the other party, enabling video calls.

[0018] The electronic whiteboards 1a and 1b transmit and receive image data of shared document images, allowing users of the communication system 100 to share the same document images. The document images are images displayed on the display of the electronic whiteboard 1, and may include, for example, meeting materials, background images displayed on the display, and images such as a captured screen when the display screen is captured.

[0019] Furthermore, the electronic whiteboards 1a and 1b transmit and receive stroke data of a shared stroke image, allowing participants using the communication system to share the same stroke image. A stroke image is an image showing lines drawn by a user using handwritten strokes with an electronic pen or the like. A stroke image is displayed using stroke data that indicates points that specify coordinates on the display.

[0020] 1 shows an electronic whiteboard equipped with a video conferencing function as an example of the electronic whiteboards 1a and 1b. Note that the image of the video data may be a moving image or a still image.

[0021] In the following description, the electronic whiteboard 1 that requests the start of a videoconference will be referred to as the "start terminal," and the electronic whiteboard 1 that is the destination (relay destination) of the request will be referred to as the "destination terminal." For example, in FIG. 1, when electronic whiteboard 1a requests electronic whiteboard 1b to start a videoconference, electronic whiteboard 1a is the start terminal and electronic whiteboard 1b is the destination terminal. Note that electronic whiteboards 1b, 1b may be used not only for communication between multiple offices or between different rooms in the same office, but also for communication within the same room, or between indoors and outdoors, or between outdoors and outdoors.

[0022] The relay device 3 is configured by a computer, and performs a process of relaying content data for communication between the plurality of electronic whiteboards 1a and 1b.

[0023] The communication management system 5 is configured by a computer and centrally manages the authentication of logins from the electronic whiteboards 1a and 1b, the management of the communication status of the electronic whiteboards 1a and 1b, the management of the destination list, and the communication status of the relay device 3. The communication management system 5 also relays shared stroke data between the multiple electronic whiteboards 1a and 1b.

[0024] The image storage device 7 is configured by a computer, and stores image data of shared document images uploaded from the electronic whiteboard 1a and downloads the data to the electronic whiteboard 1b. The reverse is also possible. That is, the image storage device 7 stores image data uploaded from the electronic whiteboard 1b and downloads the data to the electronic whiteboard 1a.

[0025] The relay device 3, communication management system 5, and image storage device 7 may be constructed by a single computer, or may be constructed by multiple computers in which each part (function or means) is divided and arbitrarily assigned.

[0026] In addition, in the communication system 100, a management information session sei (hereinafter referred to as "session sei") is established between the electronic whiteboards 1a and 1b via the communication management system 5 for transmitting and receiving various types of management information. Furthermore, four sessions are established between the electronic whiteboards 1a and 1b via the relay device 3 for transmitting and receiving four types of data: high-resolution image data, medium-resolution image data, low-resolution image data, and sound data. In FIG. 1, these four sessions are collectively shown as an image and sound data session sed (hereinafter simply referred to as "session"). The number of sessions does not necessarily have to be four, and the number of sessions may be more or less than four. Furthermore, a session may be established directly between the initiating terminal and the destination terminal without going through the relay device 3.

[0027] Furthermore, in the communication system 100, stroke data can be transmitted and received between the electronic whiteboards 1a and 1b using the session sei.

[0028] Here, the video resolution of the video data handled in this embodiment will be described. Low-resolution video data consists of, for example, 160 pixels horizontally and 120 pixels vertically, and serves as a base image. Medium-resolution video data consists of, for example, 320 pixels horizontally and 240 pixels vertically. High-resolution video data consists of, for example, 640 pixels horizontally and 480 pixels vertically. Of these, when passing through a narrowband path, low-quality video data consisting only of low-resolution video data that serves as a base image is relayed. When the bandwidth is relatively wide, medium-quality video data consisting of low-resolution video data that serves as a base image and medium-resolution video data is relayed. Furthermore, when the bandwidth is very wide, high-quality video data consisting of low-resolution video data that serves as a base image, medium-resolution video data, and high-resolution video data is relayed. Audio data has a smaller data volume than video data, so it can be relayed even through a narrowband path.

[0029] <Image of how the electronic whiteboard is used> 2 is a diagram illustrating an image of how an electronic whiteboard according to one embodiment is used. As shown in FIG. 2, the electronic whiteboard 1 has legs 151 with a plurality of casters on the lower side, a support 152 provided on the upper side of the legs 151, a main body 153 of the electronic whiteboard 1 provided on the upper side of the support 152, and a display 160 provided on the front side of the main body 153. The main body 153 incorporates a CPU 101, which will be described later, and the like. A user can input (draw) stroke images such as characters on the display 160 using an electronic pen 140 or the like.

[0030] [Hardware configuration] Next, the hardware configuration of each device according to this embodiment will be described.

[0031] <Hardware configuration of the electronic whiteboard> Fig. 3 is a diagram illustrating an example of a hardware configuration of an electronic whiteboard according to an embodiment. As illustrated in Fig. 3, the electronic whiteboard 1 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an SSD (Solid State Drive) 104, a network I / F (Interface) 105, and an external device connection I / F 106.

[0032] Of these, the CPU 101 is a computing device that controls the overall operation of the electronic whiteboard 1. The ROM 102 is a non-volatile memory that stores the CPU 101 and programs used to start up the CPU 101, such as an IPL (Initial Program Loader). The RAM 103 is a volatile memory that is used as a work area for the CPU 101. The SSD 104 is a large-capacity storage device that stores various data, such as programs for the electronic whiteboard 1.

[0033] The network I / F 105 is a communication interface for connecting the interactive whiteboard 1 to a communication network and performing communication. The external device connection I / F 106 is an interface for connecting various external devices. In this case, the external devices include, for example, a USB (Universal Serial Bus) memory 131, external devices (a microphone 132, a speaker 133, a camera 134), etc.

[0034] The electronic whiteboard 1 also includes a capture device 111, a GPU (Graphics Processing Unit) 112, a display controller 113, a contact sensor 114, a sensor controller 115, an electronic pen controller 116, a short-range communication circuit 119, an antenna 119a of the short-range communication circuit 119, a power switch 117, and selection switches 118.

[0035] Of these, the capture device 111 captures (acquires) a display screen or the like displayed on a display or the like of an external PC (Personal Computer) 150 as a still image or a moving image. The GPU 112 is a semiconductor chip (processor) that specializes in graphics. The display controller 113 controls and manages the screen display to output an output image from the GPU 112 to a display 160 or the like. The contact sensor 114 detects contact of the electronic pen 140, the user's hand H, or the like on the display 160. The sensor controller 115 controls the processing of the contact sensor 114.

[0036] The contact sensor 114 inputs and detects coordinates using, for example, an infrared blocking method. This coordinate input and coordinate detection method involves two light receiving and emitting devices installed at both ends of the upper side of the display 160 emitting multiple infrared rays parallel to the display 160, which are reflected by a reflecting member provided around the display 160 and return along the same optical path as the light emitted by the light receiving element. The contact sensor 114 outputs the IDs of the infrared rays radiated by the two light receiving and emitting devices that are blocked by an object to the sensor controller 115, and the sensor controller 115 identifies the coordinate position, which is the contact position of the object. The electronic pen controller 116 communicates with the electronic pen 140 to determine whether the pen tip or the pen tail has touched the display 160. The near-field communication circuit 119 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark).

[0037] The power switch 117 is a switch for switching on / off the power of the interactive whiteboard 1. The selection switches 118 are a group of switches for adjusting the brightness and color of the display 160, for example.

[0038] Furthermore, the electronic whiteboard 1 includes a bus line 120. The bus line 120 includes an address bus, a data bus, and various control signals for electrically connecting the components such as the CPU 101 shown in FIG.

[0039] The contact sensor 114 is not limited to an infrared blocking type, and may be a capacitive touch panel that identifies the contact position by detecting a change in capacitance, or a resistive film touch panel that identifies the contact position by a voltage change between two opposing resistive films. The contact sensor 114 may also use various detection means, such as an electromagnetic induction touch panel that identifies the contact position by detecting electromagnetic induction caused by an object touching the display unit. Furthermore, the electronic pen controller 116 may determine whether or not a touch has occurred not only on the tip and bottom of the electronic pen 140, but also on the part of the electronic pen 140 that the user holds or on other parts of the electronic pen.

[0040] <Hardware configuration of management device, relay device, image storage device, PC, etc.> The communication management system 5, the relay device 3, the image storage device 7, and the PC 150, for example, have the hardware configuration of a computer 400 as shown in Fig. 4. Alternatively, the communication management system 5, the relay device 3, and the image storage device 7 may be realized by a plurality of computers 400.

[0041] 4 is a diagram showing an example of the hardware configuration of a computer according to an embodiment. The computer 400 includes, for example, a CPU 401, a ROM 402, a RAM 403, a hard disk (HD) 404, a hard disk drive (HDD) controller 405, a display 406, an external device connection I / F 407, a network I / F 408, a keyboard 409, a pointing device 410, a digital versatile disk rewritable (DVD-RW) drive 412, a media I / F 414, and a bus line 415.

[0042] Of these, the CPU 401 is an arithmetic unit that controls the overall operation of the computer 400. The ROM 402 is a non-volatile memory that stores programs used to drive the CPU 401, such as an IPL. The RAM 403 is a volatile memory that is used as a work area for the CPU 401. The HD 404 is a large-capacity storage device that stores programs such as an OS (Operating System) and applications, as well as various types of data. The HDD controller 405 controls the reading and writing of various types of data from and to the HD 404 under the control of the CPU 401.

[0043] The display 406 displays various types of information such as a cursor, menus, windows, characters, or images. The external device connection I / F 407 is an interface for connecting various types of external devices. The network I / F 408 is a communication interface for performing data communication using a communication network. The keyboard 409 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 410 is a type of input means for selecting and executing various instructions, selecting a processing target, moving the cursor, etc.

[0044] The DVD-RW drive 412 controls reading and writing (storing) of data from and to the DVD-RW 411. The DVD-RW drive 412 may control reading and writing (storing) of data from and to a disc such as a BD-RE (Blu-ray (registered trademark) Disc Rewritable) instead of the DVD-RW 411. The media I / F 414 controls reading and writing (storing) of data from and to a recording medium 413 such as a flash memory. The bus line 415 includes an address bus, a data bus, various control signals, and the like for electrically connecting the components such as the CPU 401 shown in FIG. 4.

[0045] [Overall configuration of communication system] Next, a description will be given of the overall configuration of the communication system 100. Fig. 5 is a diagram showing an example of the system configuration of a communication system according to an embodiment.

[0046] In FIG. 5, electronic whiteboard 1a is installed at location A, and electronic whiteboard 1b is installed at location B. For example, location A is an office in Tokyo, Japan, and location B is an office in Osaka, Japan. At location A, user A uses electronic whiteboard 1a, and at location B, users B and C use electronic whiteboard 1b. Note that electronic whiteboard 1a is an example of a first terminal device, and electronic whiteboard 1b is an example of a second terminal device.

[0047] Furthermore, the electronic whiteboards 1a and 1b, the relay device 3, the communication management system 5, and the image storage device 7 can transmit and receive data to and from each other via a communication network 500 such as the Internet or a LAN (Local Area Network). The communication network 500 may include a wireless communication section. In Fig. 5, the electronic whiteboards 1a and 1b are assumed to be electronic whiteboards capable of transmitting and receiving conference videos.

[0048] [Functional configuration of communication system] Next, a functional configuration of the communication system 100 according to this embodiment will be described. Fig. 6 is a diagram showing an example of the functional configuration of the communication system according to one embodiment. In Fig. 6, the electronic whiteboard 1b has the same functional configuration as the electronic whiteboard 1a.

[0049] <Functional configuration of the electronic whiteboard> Each electronic whiteboard 1 implements a transmitting / receiving unit 11, a receiving unit 12, a video / audio processing unit 13, a display control unit 14, an image processing unit 15, an information transmitting unit 16, a short-range communication unit 17, and a storage / readout processing unit 18 by executing a predetermined program on the CPU 101 shown in Fig. 3. At least a part of the above functional configurations may be implemented by hardware. The electronic whiteboard 1 also has a storage unit 1001 implemented by the RAM 103, SSD 104, etc. shown in Fig. 3.

[0050] (Each function configuration of the electronic whiteboard 1) The transmitting / receiving unit 11 transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 500. The transmitting / receiving unit 11 also serves as an initiating unit, and performs a login process to the communication management system 5 or the like, and a process of initiating communication with other terminal devices.

[0051] The receiving unit 12 receives various inputs from a user using the electronic pen 140 or the like. For example, the receiving unit 12 receives a designation operation for designating one or more stroke data to be operated, a change operation related to the stroke data, a drawing operation of a stroke image, and the like.

[0052] The video / audio processor 13 performs the main processing of the video conferencing function. For example, the video / audio processor 13 performs digital processing such as encoding of video data and audio data based on the output signal of the microphone 132 and the output signal of the camera 134. The video / audio processor 13 also generates video signals and audio signals based on the video data and audio data received by the transmitter / receiver 11. The video / audio processor 13 also performs processing such as combining video data with different resolutions.

[0053] The display control unit 14 outputs a video signal (image signal) and the like to the display 160, and performs control to display a display screen.

[0054] The image processing unit 15 performs the main processing of the electronic whiteboard function. For example, the image processing unit 15 performs processing to create a stroke image and stroke data based on strokes of the electronic pen 140 or the like received by the receiving unit 12, and processing to create a stroke image based on stroke data received by the transmitting / receiving unit 11. The image processing unit 15 also performs processing to generate an image signal based on image data of a document image received by the transmitting / receiving unit 11.

[0055] The information transmitting unit 16 transmits the stroke data created by the image processing unit 15 to the communication management system 5. Furthermore, in response to a designation operation that is accepted by the accepting unit 12 and that designates one or more stroke data to be operated, the information transmitting unit 16 transmits group operation information that designates one or more stroke data to the communication management system 5.

[0056] The short-distance communication unit 17 acquires and provides data between each terminal having a short-distance communication unit through short-distance wireless communication.

[0057] The storage / readout processing unit 18 performs processing such as storing various data in the storage unit 1001 or a recording medium 1002 such as a USB memory 131, reading out various data stored in the storage unit 1001 or the recording medium 1002, and so on.

[0058] Furthermore, video data and audio data received during communication with other terminals are overwritten and stored in storage unit 1001. Of these, images are displayed on display 160 using the video data before being overwritten, and audio is output from speaker 133 using the audio data before being overwritten.

[0059] <Functional configuration of communication management system> The communication management system (communication management device) 5, for example, implements a transmitting / receiving unit 51, an authentication unit 52, a terminal management unit 53, a session management unit 54, a relay device management unit 55, a stroke information management unit 56, an operation restriction unit 57, and a storage / readout processing unit 58 by executing a predetermined program on the CPU 401 of FIG. 4 . Note that each of the above functional components may be implemented by a program executed by multiple computers 400. Furthermore, at least a portion of each of the above functional components may be implemented by hardware. The communication management system 5 also includes a storage unit 5000 implemented by the HD 504 of FIG. 4 or the like. The storage unit 5000 stores, for example, an authentication management database (DB) 5001, a terminal management DB 5002, a destination list management DB 5003, a session management DB 5004, a relay device management DB 5005, and stroke information 5006.

[0060] (Authentication management table) 7 is a diagram showing an example of an authentication management table according to an embodiment. The storage unit 5000 stores an authentication management DB 5001 configured with an authentication management table 700 as shown in FIG. 7. In this authentication management table 700, each password is associated with each terminal ID of all the electronic whiteboards 1 managed by the communication management system 5 and managed. For example, the authentication management table 700 shown in FIG. 7 indicates that the password corresponding to the terminal ID "01aa" of the electronic whiteboard 1a (terminal device) is "aaaa."

[0061] The terminal ID is identification information that identifies the electronic whiteboard (terminal device) 1, and is an example of identification information that the communication management system 5 uses to manage terminal devices such as the electronic whiteboard 1. For example, instead of the terminal ID, the communication management system 5 may manage the electronic whiteboard 1 using a user ID that identifies a user, a contract ID that identifies a contractor, a communication ID that identifies the source of communication, or the like.

[0062] Furthermore, the password is an example of authentication information for authenticating a terminal device such as the interactive whiteboard 1. The authentication information may be authentication information other than a password, such as an access token, for example.

[0063] (Terminal management table) Fig. 8 is a diagram showing an example of a terminal management table according to an embodiment. The storage unit 5000 stores a terminal management DB 5002 configured with a terminal management table 800 as shown in Fig. 8. In this terminal management table 800, for each terminal ID for identifying each electronic whiteboard 1 (terminal device), the destination name when each electronic whiteboard 1 is the destination, the operating status of each electronic whiteboard 1, the reception date and time when login request information (described below) was received by the communication management system 5, and the IP address of each electronic whiteboard 1 (terminal device) are associated and managed.

[0064] 8, the terminal management table 800 indicates that the electronic whiteboard 1a with the terminal ID "01aa" has the terminal name "Japan Tokyo Office Terminal AA" and an operating status of "online (communication available)." The electronic whiteboard 1a with the terminal ID "01aa" also indicates that the date and time when the communication management system 5 received login request information was "April 10, 2015, 1:40 PM" and the IP address was "1.2.1.3." The terminal ID, destination name, and terminal IP address are stored when each electronic whiteboard 1 is pre-registered to receive services provided by the communication management system 5.

[0065] (Address list management table) FIG. 9 is a diagram illustrating an example of a destination list management table according to an embodiment. The storage unit 5000 stores a destination list management DB 5003 configured with a destination list management table 900 as shown in FIG. 9. In this destination list management table 900, the terminal IDs of all destination terminals registered as candidates for the electronic whiteboard 1 (destination terminal) are associated with the terminal ID of the electronic whiteboard 1 (starting terminal) requesting the start of communication. For example, the destination list management table 900 shown in FIG. 9 indicates that the destination terminal candidates to which a communication can be requested from the starting terminal (electronic whiteboard 1a) with the terminal ID "01aa" include the electronic whiteboard 1b with the terminal ID "01ba." These destination terminal candidates are updated by being added or deleted in response to an addition or deletion request from any starting terminal to the communication management system 5.

[0066] The destination list is an example of destination information, and the destination information may not be in a list format, but may instead be a list of information about destinations such as terminal IDs.

[0067] (Session management table) FIG. 10 is a diagram illustrating an example of a session management table according to an embodiment. The storage unit 5000 stores a session management DB 5004 configured with, for example, a session management table 1000 as shown in FIG. 10. In this session management table 1000, for each session ID used to identify a session, the device ID of the relay device 3 used, the terminal ID of the initiating terminal, the terminal ID of the destination terminal, and the terminal IDs of one or more participating terminals are associated and managed. The session management table 1000 also associates and manages information such as the reception delay time (ms) when video data is received at the destination terminal and the reception date and time when delay information indicating this delay time was received from the destination terminal. For example, in the session management table 1000 illustrated in FIG. 10, for a session with session ID "se01," a relay device 3 with relay device ID "111a" relays communication between an initiating terminal with terminal ID "01aa" and a destination terminal with terminal ID "01ba." It also shows that in the session with session ID "se01", the delay time of the video data as of "13:41 on April 10, 2015" is 200 (ms).

[0068] (Relay device management table) FIG. 11 is a diagram illustrating an example of a relay device management table according to an embodiment. The storage unit 5000 stores a relay device management DB 5005 configured with a relay device management table 1100 as shown in FIG. 11. In this relay device management table 1100, the operating status of each relay device 3, the date and time when status information indicating the operating status was received, the IP address of the relay device 3, and the maximum data transmission rate (Mbps) are associated and managed for each relay device ID of multiple relay devices 3. For example, the relay device management table 1100 shown in FIG. 11 indicates that the operating status of the relay device 3 with the relay device ID "111a" is "online" and that the date and time when the status information was received by the communication management system 5 is "12:00 on April 10, 2015." Furthermore, it is indicated that the IP address of the relay device 3 with the relay device ID "111a" is "1.2.1.2" and that the maximum data transmission rate of the relay device 3 is 100 Mbps.

[0069] (Stroke information) 12 is a diagram for explaining stroke information according to an embodiment. This diagram shows an example of stroke information 5006 stored in the storage unit 5000.

[0070] In the example of FIG. 12, the stroke information 5006 has a hierarchical structure of "meeting," "pages (pages 1 to 3)," and "strokes (strokes 1 to 5)."

[0071] A "stroke" corresponds to stroke data according to this embodiment and is managed for each "page" by the stroke information management unit 56. For example, stroke 1 includes a stroke ID (id=aaaaaa) that identifies stroke 1, information that specifies parent page 1 (parent=hijkl), a Gid "g001" that identifies drawing data 1, and drawing data 1. For example, when an operation such as moving is performed on the stroke image represented by stroke 1, the stroke ID (id=aaaaaa) remains the same, but drawing data 1 and the Gid "g001" that identifies drawing data 1 are updated.

[0072] The drawing data 1 is drawing data for reproducing the stroke 1, and includes, for example, coordinate data, line width data, line color data, vector data, and the like.

[0073] The electronic whiteboard 1 allows pages to be switched, and a "page" corresponds to each page of the electronic whiteboard 1. For example, page 1 includes information such as a page ID (id=hijkl) that identifies page 1, and information for specifying the parent "meeting" (parent=abcdefg).

[0074] A "conference" corresponds to a session managed by the communication system 100, and is managed by, for example, a session ID (id=abcdefg) that identifies the session.

[0075] For example, the image processing unit 15 of the electronic whiteboard 1 can create a stroke image drawn on page 1 using drawing data 1 to 5 included in strokes 1 to 5.

[0076] The session ID (id=abcdefg) is an example of first identification information for identifying a session (conference), and the stroke ID (id=aaaaaa) is an example of second identification information for identifying stroke data. Furthermore, the Gid "g001" for identifying drawing data 1 is an example of third identification information for identifying drawing data.

[0077] (Functional configuration of communication management system) Returning to FIG. 6, the functional configuration of the communication management system 5 will now be described.

[0078] The transmitting / receiving unit 51 transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 500. For example, the transmitting / receiving unit 51 functions as a receiving unit that receives, from the electronic whiteboard 1, group operation information that specifies one or more stroke data to be operated.

[0079] The authentication unit 52 authenticates the login request received via the transmission / reception unit 51. For example, if the combination of the terminal ID and password included in the login request received from the electronic whiteboard 1 is stored in the authentication management table 700 as shown in FIG.

[0080] The terminal management unit 53 manages the terminal management DB 5002, and updates information such as the operating state, reception date and time, and terminal IP address in the terminal management table 800 shown in FIG.

[0081] The terminal management unit 53 also manages a destination list management table 900, and provides destination list information including the terminal IDs of destination terminals in the destination list management table 900 as shown in FIG. 9 in response to a request from the electronic whiteboard 1, for example.

[0082] The session management unit 54 manages sessions held in the communication system 100. For example, the session management unit 54 generates a session ID for identifying a session in response to start request information requesting the start of communication from the electronic whiteboard 1. The session management unit 54 also stores and manages various pieces of information related to the session in association with the session ID, for example, in a session management table 1000 as shown in FIG.

[0083] The relay device management unit 55 manages the relay device management DB 5005 and selects a relay device 3 to be used to relay a session, for example, by using a relay device management table 1100 such as that shown in Fig. 11. For example, the relay device management unit 55 may select a relay device 3 near the starting terminal based on the IP address of each relay device 3 stored in the relay device management table 1100 and the IP address of the starting terminal. Alternatively, the relay device management unit 55 may select a relay device 3 based on the maximum data transmission speed of each relay device 3 stored in the relay device management table 1100.

[0084] In this embodiment, the method for selecting the relay device 3 to be used for relaying the session may be any method. Also, the number of relay devices 3 included in the communication system 100 may be one.

[0085] The stroke information management unit 56 associates stroke data transmitted, for example, in session sei from an electronic whiteboard 1 participating in a session with a session ID, and stores and manages the stroke data in the storage unit 5000 or the like. For example, the stroke information management unit 56 stores stroke data received from the electronic whiteboard 1 in stroke information 5006 as shown in FIG. 12. In addition, the stroke information management unit 56 transfers stroke data received from the electronic whiteboard 1 to another electronic whiteboard 1 participating in the same session as the electronic whiteboard 1, for example.

[0086] The operation restriction unit 57 restricts operations related to the specified stroke data by other electronic whiteboards 1, based on group operation information that specifies one or more stroke data to be operated, and that is received by the transmission / reception unit 51 from the electronic whiteboard 1. The processing by the operation restriction unit 57 and the group operation information will be described later.

[0087] The storage / readout processing unit 58 performs processing such as storing various data in the storage unit 5000 and reading out various data stored in the storage unit 5000.

[0088] <Functional configuration of the image storage device> The image storage device 7 realizes a transmission / reception unit 71, a storage / readout processing unit 72, etc. by executing a predetermined program in the CPU 401 in Fig. 4. The image storage device 7 also has a storage unit 7000 realized by the RAM 403, HD 404, etc. in Fig. 4.

[0089] (Functional configuration of image storage device) The transmitting / receiving unit 71 transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 500. The storage / reading processing unit 72 stores various data in the storage unit 7000 and reads out various data stored in the storage unit 7000.

[0090] With the above configuration, the image storage device 7 stores, for example, image data uploaded from the electronic whiteboard 1 in a specified URL (Uniform Resource Locator), and provides the image data stored in the specified URL in response to a request from the electronic whiteboard 1.

[0091] <Functional configuration of relay device> The relay device 3 implements a transmitting / receiving unit 31, a determining unit 32, a storage / reading processing unit 33, etc. by executing a predetermined program in the CPU 401 of Fig. 4. The relay device 3 also has a storage unit 3000 implemented by the RAM 403, HD 404, etc. of Fig. 4.

[0092] (Functional configuration of relay device) The transmitting / receiving unit 31 transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 500. The transmitting / receiving unit 31 also functions as a transferring unit, and, for example, transfers video data and audio data received from the electronic whiteboard 1 to other electronic whiteboards 1 participating in the same session as the electronic whiteboard 1. The determining unit 32 performs various determinations, such as determining the state of data delays.

[0093] The storage / readout processing unit 33 performs processing such as storing various data in the storage unit 3000 and reading out various data stored in the storage unit 3000.

[0094] 6 is an example, and the communication system 100 can be modified in various ways. For example, each functional component included in the communication management system 5 may be realized by a plurality of computers 400. Furthermore, the function of the image storage device 7 may be realized by a storage server or the like external to the communication system 100. Furthermore, at least some of the functional components of the image storage device 7 and the relay device 3 may be included in the communication management system 5. Furthermore, the electronic whiteboard 1 is an example of a terminal device, and the terminal device may be an information terminal that executes an application corresponding to the communication system 100.

[0095] [Processing flow] Next, the processing flow of the communication management method and image processing method according to this embodiment will be described.

[0096] <Preparation for remote communication> 13 is a sequence diagram illustrating an example of a preparatory process according to an embodiment. This process illustrates an example of a preparatory process that is executed before the interactive whiteboard 1 participates in a session (video conference) in the communication system 100.

[0097] First, when the power switch 117 of the electronic whiteboard 1a is turned on, the reception unit 12 receives the power-on command (step S22).

[0098] Next, the transmitter / receiver 11 transmits login request information requesting login to the communication management system 5 via the communication network 500 (step S23). This login request information includes, for example, the terminal ID and password of the electronic whiteboard 1a.

[0099] Next, the authentication unit 52 of the communication management system 5 searches the authentication management table 700 as shown in Fig. 7 using the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys. Furthermore, if the combination of the terminal ID and password included in the login request information is managed in the authentication management table 700, the authentication unit 52 permits login to the electronic whiteboard 1a (step S24). Here, if the combination of the terminal ID and password included in the login request information is managed in the authentication management table 700, the processing from step S25 onwards is executed.

[0100] When login to the electronic whiteboard 1a is permitted, the terminal management unit 53 updates the information corresponding to the terminal ID "01aa" of the electronic whiteboard 1a in the terminal management table 800 as shown in FIG. 8. For example, the terminal management unit 53 changes the "operation status" information corresponding to the terminal ID "01aa" to "online (communication available)" and updates the "received date and time" information to the date and time when the login request information was received (step S25). Note that the IP address information of the terminal does not need to be registered in advance, and the IP address transmitted from the electronic whiteboard 1a in step S23 may be used. As a result, for example, as shown in FIG. 8, the terminal management table 800 manages the terminal ID "01aa" in association with the operation status "online (communication available)," the received date and time "2015.4.10.13:40," the IP address "1.2.1.3," and the like.

[0101] Next, the session management unit 54 adds a new record in which the terminal ID "01aa" of the electronic whiteboard 1a received in step S23 above is set as the "terminal ID of the starting terminal" to the session management table 1000 as shown in Fig. 10 and manages the new record (step S26). Then, the transmitter / receiver 51 of the communication management system 5 transmits authentication result information indicating the authentication result obtained by the processing of step 24 via the communication network 500 to the electronic whiteboard 1a that made the login request (step S27).

[0102] When the transmitter / receiver 11 of the electronic whiteboard 1a receives the authentication result information indicating that the login is permitted, the transmitter / receiver 11 transmits destination list request information indicating a request for a destination list to the communication management system 5 via the communication network 500 (step S28). As a result, the transmitter / receiver 51 of the communication management system 5 receives the destination list request information.

[0103] Next, the terminal management unit 53 searches the destination list management table 900 as shown in Fig. 9 using the terminal ID "01aa" of the electronic whiteboard 1a as a search key, and reads out the terminal IDs of destination candidates that can communicate with the electronic whiteboard 1a. The terminal management unit 53 also reads out the destination names corresponding to the terminal IDs of the destination candidates from the terminal management table 800 as shown in Fig. 8 (step S29). As a result, the terminal IDs of the destination candidates that correspond to the terminal ID "01aa" of the electronic whiteboard 1a and the corresponding destination names are extracted.

[0104] Next, the transmitting / receiving unit 51 of the communication management system 5 reads, for example, data of the destination list frame and data of the icon indicating the operating status stored in the memory unit 5000 via the storage / readout processing unit 58 (step S30). The transmitting / receiving unit 51 of the communication management system 5 also transmits "destination list information (destination list frame, icon, terminal ID, destination name)" including the read destination list frame and icon, and the terminal ID and destination name extracted in step S29, to the electronic whiteboard 1a (step S31). As a result, the transmitting / receiving unit 11 of the electronic whiteboard 1a receives the destination list information, and the storage / readout processing unit 18 stores the received destination list information in the memory unit 1001 (step S32).

[0105] As described above, in this embodiment, the communication management system 5 centrally manages the destination list information of all terminals, rather than managing the destination list information on each electronic whiteboard 1. Therefore, even when a new electronic whiteboard 1 is registered in the communication management system 5, when a new model of terminal device is registered in place of an electronic whiteboard 1 already included, or when the appearance of the destination list frame is changed, the effort of changing the destination list information on each electronic whiteboard 1 can be eliminated.

[0106] 8 using the terminal IDs of the destination candidates extracted in step S29 as search keys, and reads out the corresponding operating status for each terminal ID of the destination candidate. As a result, the terminal management unit 53 acquires the operating status of each electronic whiteboard 1 corresponding to the terminal ID of the destination candidate (step S33).

[0107] Next, the transmitter / receiver 51 of the communication management system 5 transmits "terminal status information" including the terminal ID used as the search key in step S33 and the operating status of each corresponding destination terminal to the electronic whiteboard 1a via the communication network 500 (step S34).

[0108] Next, the storage / readout processing unit 18 of the electronic whiteboard 1a sequentially stores the terminal status information received from the communication management system 5 in the storage unit 1001 (step S35). Therefore, by receiving the above-mentioned electronic whiteboard 1 status information, the electronic whiteboard 1a can obtain the current operating status of the electronic whiteboards 1b, 1c, etc., which are candidate destinations that can communicate with the electronic whiteboard 1a.

[0109] Next, the display control unit 14 of the electronic whiteboard 1a creates a destination list that reflects the status of the terminal as a destination candidate, based on the destination list information and the terminal status information stored in the storage unit 1001. Furthermore, the display control unit 14 uses the created destination list to display a destination list screen on the display 160 of the electronic whiteboard 1a (step S36).

[0110] Meanwhile, the terminal management unit 53 of the communication management system 5 extracts the terminal IDs of other terminals that have registered the electronic whiteboard 1a as a destination candidate by searching the destination list management table 900 as shown in Figure 9 based on the terminal ID "01aa" of the electronic whiteboard 1a (step S37).

[0111] Next, the terminal management unit 53 of the communication management system 5 searches the terminal management table 800 shown in FIG. 8 based on the terminal ID "01aa" of the electronic whiteboard 1a, and acquires the operating status of the electronic whiteboard 1a (step S38).

[0112] The transmitter / receiver 51 then transmits "terminal status information" including the terminal ID and operating status of the electronic whiteboard 1a to the electronic whiteboards 1 having terminal IDs whose operating status is "online" in the terminal management table 800, among the terminal IDs extracted in step S37 (step S39). When transmitting the terminal status information to each electronic whiteboard 1, the transmitter / receiver 51 references the IP address of each electronic whiteboard 1 managed in the terminal management table 800 based on each terminal ID. This allows the terminal ID "01aa" of the electronic whiteboard 1a and the operating status "online" to be communicated to each electronic whiteboard 1 that can communicate with the electronic whiteboard 1a as a candidate destination. This allows the status of the electronic whiteboard 1a to be displayed on each candidate destination electronic whiteboard 1 (step S40).

[0113] <Session start processing> 14 is a sequence diagram illustrating an example of a session start process according to an embodiment. This process illustrates an example of a session start process executed in the communication system 100 when, for example, the electronic whiteboard 1a requests the electronic whiteboard 1b to start a session.

[0114] In step S1401, when the electronic whiteboard 1a, which is the starting terminal, accepts the destination selection by user A, the processes from step S1402 onward are executed. Here, it is assumed that user A has selected the electronic whiteboard 1b as the destination terminal.

[0115] In step S1402, when the electronic whiteboard 1a accepts the destination selection by user A, the transmitter / receiver 11 of the electronic whiteboard 1a transmits start request information requesting the start of a session to the communication management system 5. This start request information includes, for example, the terminal ID "01aa" of the electronic whiteboard 1a, which is the starting terminal, the terminal ID "01ba" of the electronic whiteboard 1b, which is the destination terminal, and the IP address of the electronic whiteboard 1a, which is the starting terminal. As a result, the transmitter / receiver 51 of the communication management system 5 receives the start request information and also receives the IP address of the starting terminal (electronic whiteboard 1a), which is the sender.

[0116] In step S1403, the terminal management unit 53 of the communication management system 5 updates the terminal management DB 5002 based on the terminal ID of the starting terminal and the terminal ID of the destination terminal included in the start request information. For example, based on the terminal ID "01aa" of the starting terminal and the terminal ID "01ba" of the destination terminal included in the start request information, the terminal management unit 53 updates the "operating status" corresponding to the terminal IDs "01aa" and "01ba" in the terminal management table 800 to "online (communicating)." Note that in this state, the electronic whiteboard 1a, which is the starting terminal, and the electronic whiteboard 1b, which is the destination terminal, are not participating in the session, but are managed as being in a communicating state.

[0117] In step S1404, the session management unit 54 of the communication management system 5 generates a session ID for identifying the session held between the electronic whiteboard 1a and the electronic whiteboard 1b. Here, it is assumed that the session management unit 54 generates the session ID "se01."

[0118] In step S1405, the session management unit 54 stores and manages the generated session ID, the terminal ID "01aa" of the initiating terminal, the terminal ID "01ba" of the destination terminal, etc. in association with each other in a session management table 1000 as shown in FIG.

[0119] In step S1406, the relay device management unit 55 of the communication management system 5 selects a relay device 3 that will relay the session between the starting terminal (electronic whiteboard 1a) and the destination terminal (electronic whiteboard 1b). For example, the relay device management unit 55 selects a relay device 3 that has an IP address close to the IP address of the starting terminal (electronic whiteboard 1a) in the terminal management table 800 from among the device IDs of relay devices 3 whose operating status is "online" in the relay device management table 1100. Here, it is assumed that the relay device 3 with device ID "111a" is selected.

[0120] In steps S1407a and S1407b, the session management unit 54 of the communication management system 5 transmits relay device selection information to the start terminal (electronic whiteboard 1a) and the destination terminal (electronic whiteboard 1b) via the transmission / reception unit 51. This relay device selection information includes, for example, the IP address of the relay device 3 selected in step S1406, the session ID generated in step S1404, and the IP address of the communication management system 5.

[0121] In steps S1408a and S1408b, upon receiving the relay device selection information, the transmitter / receiver 11 of the initiating terminal (electronic whiteboard 1a) and the destination terminal (electronic whiteboard 1b) transmit reception completion information indicating that the relay device selection information has been received to the communication management system 5. This reception completion information includes, for example, the session ID "se01" that was included in the relay device selection information.

[0122] In steps S1409a and S1409b, the transmitter / receiver 11 of the initiating terminal (electronic whiteboard 1a) and the destination terminal (electronic whiteboard 1b) establish a session with the relay device 3 using the device ID and session ID of the relay device 3 included in the relay device selection information. As a result, the electronic whiteboard 1a and the electronic whiteboard 1b participate in the session with the same session ID "se01" and can mutually transmit and receive video data such as conference video, audio data, and the like via the relay device 3.

[0123] By the above process, a video conference can be held between user A of the electronic whiteboard 1a and user B of the electronic whiteboard 1b, transmitting and receiving conference video and the like.

[0124] <Sharing of stroke images> In the communication system 100, user A of the electronic whiteboard 1a and users B and C of the electronic whiteboard 1b participating in a video conference can display stroke images drawn with an electronic pen 140 or the like on each electronic whiteboard 1 as shared images on the other electronic whiteboards 1. Here, an example of a process for sharing stroke images between the electronic whiteboard 1a and the electronic whiteboard 1b participating in the same session will be described.

[0125] Fig. 15 is a sequence diagram showing an example of a stroke image sharing process according to an embodiment. At the start of the process shown in Fig. 15, the electronic whiteboard 1a and the electronic whiteboard 1b are assumed to be participating in the same session through the session start process shown in Fig. 14, for example.

[0126] Furthermore, the electronic whiteboards 1a and 1b share document images via the image storage device 7 as necessary, and display the document images on the display 160.

[0127] In step S1501, user A draws (moves) the electronic pen 140 or hand H on the display 160 of the electronic whiteboard 1a, and the receiving unit 12 receives input of the drawn stroke (trajectory).

[0128] In step S1502, the image processing unit 15 of the electronic whiteboard 1a creates stroke data for displaying a stroke image corresponding to the stroke accepted by the accepting unit 12.

[0129] In step S1503, the display control unit 14 of the electronic whiteboard 1a displays a stroke image on the display 160 based on the stroke data created by the image processing unit 15. As a result, the stroke image drawn by the user A on the display 160 with the electronic pen 140 or the like is displayed on the display 160.

[0130] In step S1504, the transmitter / receiver 11 of the electronic whiteboard 1a transmits stroke data for reproducing the stroke image drawn by user A to the communication management system 5, for example, via session sei. As a result, the transmitter / receiver 51 of the communication management system 5 receives the stroke data transmitted from the electronic whiteboard 1a.

[0131] In step S1505, the stroke information management unit 56 of the communication management system 5 registers the stroke data received from the electronic whiteboard 1a in, for example, stroke information 5006 as shown in FIG.

[0132] In step S1506, the stroke information management unit 56 of the communication management system 5 transfers the stroke data to the electronic whiteboard 1b that is participating in the same session as the electronic whiteboard 1a. As a result, the transmission / reception unit 11 of the electronic whiteboard 1b receives the stroke data.

[0133] In step S1507, the image processing unit 15 of the interactive whiteboard 1b creates a stroke image based on the stroke data received by the transmitting and receiving unit 11.

[0134] In step S1508, the display control unit 14 of the electronic whiteboard 1b causes the display 160 to display the stroke image created by the image processing unit 15, that is, the stroke image drawn by user A in step S1501.

[0135] In addition, when user B or user C draws on the electronic whiteboard 1b with the electronic pen 140 or the like, the stroke image can be displayed on the electronic whiteboard 1a in the same manner.

[0136] In this way, stroke images can be shared between the electronic whiteboard 1a and the electronic whiteboard 1b participating in the same session.

[0137] <Moving stroke images> In a conference where stroke images are shared between multiple electronic whiteboards (terminal devices) 1a, 1b, each of the multiple electronic whiteboards 1a, 1b can perform editing operations such as moving the shared stroke images. In this case, for example, when a moving operation or the like is performed on a shared stroke image from the multiple electronic whiteboards 1a, 1b, depending on the timing of the operation, an operation different from the user's intention may be performed on the stroke image.

[0138] To solve such problems, the communication management system 5 has an operation restriction function that restricts editing operations by user B on a stroke image when user A is performing an editing operation on the stroke image to be operated, for example.

[0139] Before describing the operation restriction function, an example of a stroke image movement process when the operation restriction function is not used will be described.

[0140] Fig. 16 is a sequence diagram showing an example of stroke image movement processing according to an embodiment. At the start of the processing shown in Fig. 16, for example, it is assumed that a display screen 1710 including stroke images ST1 to ST5 is displayed on the displays of the electronic whiteboard 1a and the electronic whiteboard 1b as a result of the stroke image sharing processing shown in Fig. 15, as shown in Fig. 17(A). It is also assumed that stroke data (strokes 1 to 5) corresponding to the stroke images ST1 to ST5 are stored in the stroke information 5006 managed by the stroke information management unit 56 of the communication management system 5, as shown in Fig. 12, for example.

[0141] Also, as an example for the purpose of explanation, it is assumed here that user A of the electronic whiteboard 1a performs an operation to sequentially move stroke images ST1 to ST5 to the left, and user B of the electronic whiteboard 1b performs an operation to sequentially move stroke images ST2 to ST4 downward.

[0142] In step S1601, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST1 (Gid=g001) to the left by a predetermined distance in response to the moving operation on the stroke image ST1 by the user A to the communication management system 5. This operation information includes, for example, coordinate information of the stroke image ST1 after it has been moved.

[0143] In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 1 and Gid of the stroke 1 corresponding to the stroke image ST1 (Gid=g001) in the stroke information 5006 shown in Fig. 12, for example. As a result, the stroke image ST1 shared between the electronic whiteboard 1a and the electronic whiteboard 1b moves to the left, for example, as shown in a display screen 1720 in Fig. 17(B).

[0144] In step S1602, the stroke information management unit 56 of the communication management system 5 transmits a success notification indicating that the stroke image ST1 has been successfully moved to the electronic whiteboard 1a. The success notification includes the updated gid (e.g., Gid=g006) of the stroke image ST1.

[0145] In step S1603, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST2 (Gid=g002) to the communication management system 5 in response to the user A's operation to move the stroke image ST2 (Gid=g002) to the left by a predetermined distance. In response to this, the stroke information managing unit 56 of the communication management system 5 updates the drawing data 2 and Gid of the stroke 2 corresponding to the stroke image ST2 (Gid=g002) in the stroke information 5006 shown in FIG. 12, for example. As a result, the stroke image ST2 shared between the electronic whiteboard 1a and the electronic whiteboard 1b moves to the left, for example, as shown in a display screen 1730 in FIG. 17(C).

[0146] In step S1604, the stroke information management unit 56 of the communication management system 5 transmits a success notification indicating that the stroke image ST2 has been successfully moved to the electronic whiteboard 1a. The success notification includes the updated gid (e.g., Gid=g007) of the stroke image ST2.

[0147] In step S1605, the information transmitting unit 16 of the electronic whiteboard 1b transmits operation information for moving the stroke image ST2 (Gid=g002) downward by a predetermined distance in response to the move operation on the stroke image ST2 by user B to the communication management system 5. However, since the gid of the stroke image ST2 has been updated to "g007", the stroke information managing unit 56 of the communication management system 5 cannot perform the move process on the stroke image ST2 (Gid=g002).

[0148] In step S1606, the stroke information management unit 56 of the communication management system 5 transmits a failure notification indicating that the movement of the stroke image ST2 has failed to the interactive whiteboard 1b.

[0149] In step S1607, the information transmitting unit 16 of the electronic whiteboard 1b transmits operation information for moving the stroke image ST3 (Gid=g003) downward by a predetermined distance to the communication management system 5 in response to the user B's operation to move the stroke image ST3. In response to this, the stroke information managing unit 56 of the communication management system 5 updates the drawing data 3 and Gid of the stroke 3 corresponding to the stroke image ST3 (Gid=g003) in the stroke information 5006 shown in FIG. 12, for example. As a result, the stroke image ST3 shared between the electronic whiteboard 1a and the electronic whiteboard 1b moves downward, for example, as shown in a display screen 1740 in FIG. 17(D).

[0150] In step S1608, the stroke information management unit 56 of the communication management system 5 transmits a success notification indicating that the stroke image ST3 has been successfully moved to the electronic whiteboard 1b. The success notification includes the updated gid (e.g., Gid=g008) of the stroke image ST3.

[0151] In step S1609, the information transmitting unit 16 of the electronic whiteboard 1b transmits operation information for moving the stroke image ST4 (Gid=g004) downward by a predetermined distance to the communication management system 5 in response to the user B's operation to move the stroke image ST4. In response to this, the stroke information managing unit 56 of the communication management system 5 updates the drawing data 3 and Gid of the stroke 4 corresponding to the stroke image ST4 (Gid=g004) in the stroke information 5006 shown in FIG. 12, for example. As a result, the stroke image ST4 shared between the electronic whiteboard 1a and the electronic whiteboard 1b moves downward, for example, as shown in a display screen 1750 in FIG. 17(E).

[0152] In step S1610, the stroke information management unit 56 of the communication management system 5 transmits a success notification indicating that the stroke image ST4 has been successfully moved to the electronic whiteboard 1b. The success notification includes the updated gid (e.g., Gid=g009) of the stroke image ST4.

[0153] In step S1611, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST3 (Gid=g003) to the left by a predetermined distance in response to the move operation on the stroke image ST3 by user A to the communication management system 5. However, since the gid of the stroke image ST3 has been updated to "g008", the stroke information managing unit 56 of the communication management system 5 cannot perform the move process on the stroke image ST3 (Gid=g003).

[0154] In step S1612, the stroke information management unit 56 of the communication management system 5 transmits a failure notification indicating that the movement of the stroke image ST3 has failed to the interactive whiteboard 1a.

[0155] In step S1613, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST4 (Gid=g004) a predetermined distance to the left in response to the move operation on the stroke image ST4 by user A to the communication management system 5. However, since the gid of the stroke image ST4 has been updated to "g009," the stroke information managing unit 56 of the communication management system 5 cannot perform the move process on the stroke image ST4 (Gid=g004).

[0156] In step S1614, the stroke information management unit 56 of the communication management system 5 transmits a failure notification indicating that the movement of the stroke image ST4 has failed to the electronic whiteboard 1a.

[0157] In step S1615, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST5 (Gid=g005) to the communication management system 5 in response to the user A's operation to move the stroke image ST5 (Gid=g005) to the left by a predetermined distance. In response to this, the stroke information managing unit 56 of the communication management system 5 updates the drawing data 2 and Gid of the stroke 2 corresponding to the stroke image ST5 (Gid=g005) in the stroke information 5006 shown in FIG. 12, for example. As a result, the stroke image ST5 shared between the electronic whiteboard 1a and the electronic whiteboard 1b moves to the left, for example, as shown in a display screen 1760 in FIG. 17(F).

[0158] 17(F), the stroke images ST1, ST2, and ST5 move leftward in response to the operation of user A, while the stroke images ST3 and ST4 move downward in response to the operation of user B. As a result, the stroke images ST1 to ST5 end up being moved to positions that are different from the intentions of both user A and user B.

[0159] To solve such problems, the communication management system 5 has an operation restriction function that restricts editing operations by user B on a stroke image when user A is performing an editing operation on the stroke image to be operated, for example.

[0160] [First embodiment] 18 is a sequence diagram (1) showing an example of the operation restriction process according to the first embodiment. This process shows an example of the operation restriction process by the operation restriction unit 57 of the communication management system 5.

[0161] 18, for example, it is assumed that the display screen 1710 including the stroke images ST1 to ST5 is displayed on the displays of the electronic whiteboard 1a and the electronic whiteboard 1b as a result of the stroke image sharing process shown in Fig. 15, as shown in Fig. 17(A). It is also assumed that the stroke information 5006 managed by the stroke information management unit 56 of the communication management system 5 stores stroke data (strokes 1 to 5) corresponding to the stroke images ST1 to ST5, as shown in Fig. 12, for example.

[0162] Also, as an example for explanation, it is assumed here that a user A of the electronic whiteboard 1a performs an operation to sequentially move the stroke images ST1 to ST5 to the left.

[0163] In step S1801, the reception unit 12 of the interactive whiteboard 1a receives a group selection operation by user A.

[0164] In this embodiment, before performing an editing operation on a shared stroke image, a user can select the stroke image to be operated on and notify the communication management system 5 of a list of stroke data to be operated on.

[0165] In step S1802, the information transmitting unit 16 of the electronic whiteboard 1a transmits a group operation request 1 specifying one or more stroke data to be operated on to the communication management system 5 in response to the group selection operation by user A. Note that the group operation request 1 is an example of group operation information specifying one or more stroke data to be operated on.

[0166] 19(a) shows an example of group operation information 1910 (group operation request 1) that the information sending unit 16 of the electronic whiteboard 1a sends to the communication management system 5. In the group operation information 1910 shown in FIG. 19(a), the character string ""lock":"abcdefg"" is information that specifies identification information that identifies the conference (session) to be operated. The example of FIG. 19(a) shows that the stroke information to be operated is, for example, the stroke information 5006 of the conference (id=abcdefg) shown in FIG. 12.

[0167] Furthermore, the string "list":["aaaaa","bbbbb","ccccc","ddddd","eeeee"]" is information specifying the stroke IDs of the stroke data to be operated. In the example of FIG. 19(a), the stroke data to be operated are shown to be stroke 1, stroke 2, stroke 3, stroke 4, and stroke 5.

[0168] In step S1803 of Figure 18, when the operation restriction unit 57 of the communication management system 5 receives the group operation request 1 sent from the electronic whiteboard 1a, it determines whether the operation target (stroke data) specified in the group operation request 1 includes an operation target (stroke data) that is under operation restriction.

[0169] If the operation targets specified in the group operation request 1 do not include any operation targets under operation restriction, the processing from step S1804 onwards is executed. On the other hand, if the operation targets specified in the group operation request 1 include any operation targets under operation restriction, the operation restriction unit 57 of the communication management system 5 sends a rejection notice to the requesting electronic whiteboard 1, as will be described later.

[0170] In step S1804, the operation restriction unit 57 of the communication management system 5 starts operation restriction processing for the stroke data specified in the group operation request 1. As a result, the operation restriction unit 57 prohibits operations from the other electronic whiteboard 1b related to the stroke image specified in the group operation request 1 transmitted from the electronic whiteboard 1a.

[0171] In step S1805, the operation restriction unit 57 of the communication management system 5 transmits a reception notification indicating that the group operation request 1 has been received to the requesting interactive whiteboard 1a.

[0172] In step S1806, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST1 (Gid=g001) to the left by a predetermined distance in response to the move operation on the stroke image ST1 by user A to the communication management system 5. In response to this, the stroke information managing unit 56 of the communication management system 5 updates the drawing data 1 of the stroke 1 corresponding to the stroke image ST1 (Gid=g001) in the stroke information 5006 shown in FIG. 12, for example, and updates Gid to "g006."

[0173] In step S1807, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST2 (Gid=g002) to the left by a predetermined distance in response to the move operation on the stroke image ST2 by user A to the communication management system 5. In response to this, the stroke information managing unit 56 of the communication management system 5 updates the drawing data 2 of the stroke 2 corresponding to the stroke image ST2 (Gid=g002) in the stroke information 5006 shown in FIG. 12, for example, and updates Gid to "g007."

[0174] Here, for example, it is assumed that in step S1808, the accepting unit 12 of the interactive whiteboard 1b accepts the group selection operation by user B.

[0175] In step S1809, the information transmitting unit 16 of the electronic whiteboard 1b transmits a group operation request 2 specifying one or more stroke data to be operated on to the communication management system 5 in response to the group selection operation by user B. Note that the group operation request 2 is another example of group operation information specifying one or more stroke data to be operated on.

[0176] 19(b) shows an example of group operation information 1920 (group operation request 2) that the information sending unit 16 of the electronic whiteboard 1b sends to the communication management system 5. In the group operation information 1920 shown in FIG. 19(b), the character string "lock:abcdefg" indicates that the stroke information to be operated is, for example, the stroke information 5006 of the meeting (id=abcdefg) shown in FIG.

[0177] Furthermore, the character string "list":["bbbbb","ccccc","ddddd"]" indicates that the stroke data to be operated on is stroke 2, stroke 3, and stroke 4.

[0178] In step S1810 of Figure 18, when the operation restriction unit 57 of the communication management system 5 receives the group operation request 2 sent from the electronic whiteboard 1b, it determines whether the operation target (stroke data) specified in the group operation request 2 includes an operation target (stroke data) that is under operation restriction.

[0179] Here, in step S1804, operation restriction processing is started for stroke data with stroke IDs "aaaaaa," "bbbbbb," "cccccc," [dddddd], and "eeeeee" based on group operation information 1910 shown in Fig. 19(a). Also, in group operation information 1920 shown in Fig. 19(b), stroke IDs "bbbbbb," "cccccc," and [dddddd] are specified as stroke data to be operated.

[0180] In this way, if a part or all of the specified operation target (stroke data) is included in the operation target (stroke data) under operation restriction, the operation restriction unit 57 transmits a rejection notice (or failure notice) to the requesting electronic whiteboard 1b in step 1811. In this case, users B and C of the electronic whiteboard 1b cannot operate the stroke data ST2, ST3, and ST4 that are the operation targets.

[0181] In step S1812, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST3 (Gid=g003) a predetermined distance to the left in response to the move operation on the stroke image ST3 by user A to the communication management system 5. In response to this, the stroke information managing unit 56 of the communication management system 5 updates the drawing data 3 of the stroke 3 corresponding to the stroke image ST3 (Gid=g003) in the stroke information 5006 shown in FIG. 12, for example, and updates Gid to "g008."

[0182] In step S1813, the information sending unit 16 of the electronic whiteboard 1a sends operation information for moving the stroke image ST4 (Gid=g004) a predetermined distance to the left in response to the move operation on the stroke image ST4 by user A to the communication management system 5. In response to this, the stroke information management unit 56 of the communication management system 5 updates the drawing data 4 of the stroke 4 corresponding to the stroke image ST4 (Gid=g004) in the stroke information 5006 shown in FIG. 12, for example, and updates Gid to "g009."

[0183] In step S1814, the information transmitting unit 16 of the electronic whiteboard 1a transmits operation information for moving the stroke image ST5 (Gid=g005) a predetermined distance to the left in response to the move operation on the stroke image ST5 by user A to the communication management system 5. In response to this, the stroke information managing unit 56 of the communication management system 5 updates the drawing data 5 of the stroke 4 corresponding to the stroke image ST5 (Gid=g005) in the stroke information 5006 shown in FIG. 12, for example, and updates Gid to "g010."

[0184] In step S1815, for example, when the operation restriction unit 57 of the communication management system 5 receives operations for all stroke data designated as operation targets, it determines that the group operation is complete and ends the operation restriction process.

[0185] As another example, the operation restriction unit 57 may determine that the group operation is complete when it receives information indicating the end of the group operation from the interactive whiteboard 1a.

[0186] Preferably, the operation restriction unit 57 terminates the group restriction process when a predetermined time has elapsed since the start of the operation restriction process. This prevents the operation target from becoming unable to be edited, for example, when the electronic whiteboard 1a transmits group operation information to the communication management system 5 and then leaves the session due to a failure or the like before completing operations on all stroke information that are the operation target.

[0187] In step S1816, the operation restriction unit 57 of the communication management system 5 transmits completion information indicating that the group operation has been completed to the electronic whiteboard 1a. Note that the Gid of each stroke data item updated by the group operation may be notified to the electronic whiteboard 1a using this completion information, or may be notified to the electronic whiteboard 1a using, for example, a success notification as shown in step S1602 of FIG.

[0188] By the above process, for example, as shown on the display screen 1770 of FIG. 17(G), the stroke images ST1 to ST5 to be operated are moved to the left as intended by the user A.

[0189] Fig. 20 is a sequence diagram (2) showing an example of operation restriction processing according to the first embodiment. This processing shows another example of operation restriction processing by the operation restriction unit 57 of the communication management system 5. Of the processing shown in Fig. 20, the processing of steps S1801 to S1807 and S1812 to S1816 is the same as the processing described in Fig. 18, so the following description will mainly focus on the differences from the processing shown in Fig. 18.

[0190] In step S2001, the accepting unit 12 of the electronic whiteboard 1b accepts a move operation by user B to move the stroke image ST3 downward by a predetermined distance while user A is performing a group operation on the stroke images ST1 to ST5.

[0191] In step S2002, the information transmitting unit 16 of the electronic whiteboard 1b transmits operation information for moving the stroke image ST3 (Gid=g003) downward by a predetermined distance in response to the move operation on the stroke image ST3 by the user B to the communication management system 5. However, since the operation of the stroke image ST3 is restricted by the operation restriction process performed by the operation restriction unit 57 of the communication management system 5, the stroke information managing unit 56 cannot perform the move process on the stroke image ST3 (Gid=g003).

[0192] In step S2003, the stroke information management unit 56 of the communication management system 5 transmits a failure notification indicating that the movement of the stroke image ST3 has failed to the electronic whiteboard 1b.

[0193] In this way, when the operation restriction unit 57 according to this embodiment receives group operation information transmitted from the electronic whiteboard 1, it restricts operations from other electronic whiteboards 1 relating to the stroke image specified in the group operation information.

[0194] Therefore, according to this embodiment, in a session in which stroke images are shared among multiple terminal devices, it is possible to prevent operations from being performed on the stroke images from multiple terminal devices that are different from the user's intention.

[0195] [Second embodiment] 21 is a sequence diagram showing an example of operation restriction processing according to the second embodiment. This processing shows an example of operation restriction processing by the communication management system 5 and the electronic whiteboard 1.

[0196] In the following description, it is assumed that the communication system 100 has three electronic whiteboards 1a, 1b, and 1c, and that the three electronic whiteboards 1a, 1b, and 1c are participating in a video conference in which the same stroke image is shared.

[0197] In step S2101, the reception unit 12 of the electronic whiteboard 1a receives a group selection operation by user A who uses the electronic whiteboard 1a. Here, it is assumed that "Page 1" of the electronic whiteboard 1a has been selected by the group selection operation. In this way, the group selection operation that specifies one or more stroke data to be operated on may be performed on a page-by-page basis.

[0198] In step S2102, in response to user A's group selection operation, the information sending unit 16 of the electronic whiteboard 1a sends a group operation request to the communication management system 5, which includes the information for the selected operation target, "Page 1," and the terminal ID "10aa" of the electronic whiteboard 1a.

[0199] In step S2103, when the operation restriction unit 57 of the communication management system 5 receives the group operation request sent from the electronic whiteboard 1a, it determines whether the "Page 1" specified in the group operation request includes an operation target (stroke data, page, etc.) that is under operation restriction.

[0200] If the operation targets specified in the group operation request do not include any operation targets under operation restriction, the processing from step S2104 onwards is executed. On the other hand, if the operation targets specified in the group operation request include any operation targets under operation restriction, the operation restriction unit 57 of the communication management system 5 cancels the execution of the processing from step S2104 onwards and sends a rejection notice to the requesting electronic whiteboard 1a. Here, the following explanation will be given assuming that the operation targets specified in the group operation request do not include any operation targets under operation restriction.

[0201] In step S2104, the operation restriction unit 57 of the communication management system 5 starts the operation restriction process for page 1 specified in the group operation request.

[0202] In step S2105, the operation restriction unit 57 of the communication management system 5 transmits a reception notification indicating that the group operation request has been received to the requesting interactive whiteboard 1a.

[0203] Furthermore, in steps S2106 and S2107, the operation restriction unit 57 of the communication management system 5 according to the second embodiment transmits an operation restriction notification to all other electronic whiteboards 1b and 1c participating in the videoconference, indicating that the operation restriction process has begun. This operation restriction notification includes, for example, information about the operation target that is the subject of the operation restriction (e.g., "page 1") and information about the electronic whiteboard 1a that has imposed the operation restriction on the operation target (e.g., terminal ID "10aa"). Note that the information about the electronic whiteboard 1a may be the terminal name of the electronic whiteboard 1a, the destination name managed in the terminal management table 800, or the like.

[0204] In steps S2108 and S2109, the display control units 14 of the electronic whiteboards 1b and 1c, which have received an operation restriction notification from the communication management system 5, display a display screen 2200 indicating that operation of page 1 is restricted, as shown in FIG. 22, for example.

[0205] FIG. 22 is a diagram showing an example of a display screen according to the second embodiment. This diagram shows an example of a display screen 2200 displayed by the electronic whiteboards 1b and 1c in steps S2108 and S2109 of FIG. 21, for example. In FIG. 22, it is assumed that page 1 is displayed on the display screen 2200 of the electronic whiteboards 1b and 1c. Also, a message 2201 indicating that operations on page 1 are restricted is displayed on the display screen 2200. This message 2201 can prevent operations on page 1 from being performed on the electronic whiteboards 1b and 1c. Preferably, this message 2201 includes information (e.g., terminal name, destination name, terminal ID, etc.) about the electronic whiteboard 1a that has locked (restricted operations on) page 1.

[0206] The message 2201 is an example of display information indicating that operations are restricted. For example, the display information indicating that operations are restricted may be information other than character strings, such as pictograms or icons.

[0207] By the above process, user A who uses the electronic whiteboard 1a can preferentially operate the stroke images included in the operation target "Page 1."

[0208] For example, in step S2111, when user A moves the stroke image ST1 on page 1 of the electronic whiteboard 1a to the right, the processes from step S2112 onwards are executed.

[0209] In step S2112, the information transmitting unit 16 of the interactive whiteboard 1a transmits to the communication management system 5 operation information for moving the stroke image ST1 (stroke1) on page 1 by a predetermined distance to the right.

[0210] In steps S2113 and S2114, the stroke information management unit 56 of the communication management system 5 transmits operation information for moving the stroke image ST1 (stroke1) to the right by a predetermined distance to the electronic whiteboards 1b and 1c.

[0211] As a result, in steps S2115 and S2116, the stroke image ST1 on page 1 of the electronic whiteboards 1b and 1c is moved to the right, and the same stroke image is displayed on page 1 of the electronic whiteboards 1a, 1b, and 1c.

[0212] In this way, the communication system 100 according to the second embodiment can prevent operations from being performed on a stroke image that are different from the user's intention due to operations from the multiple electronic whiteboards 1a, 1b, and 1c.

[0213] [Third embodiment] Fig. 23 is a sequence diagram showing an example of operation restriction processing according to the third embodiment. Note that the processing of steps S2101 to S2104 in Fig. 23 is similar to the processing according to the second embodiment described in Fig. 21, and therefore the following description will focus on the differences from the second embodiment.

[0214] In step S2104 of Figure 23, after the operation restriction unit 57 of the communication management system 5 starts the operation restriction process, there is a risk that operations will be performed on page 1 on the electronic whiteboards 1b and 1c until an operation restriction notification is sent to the electronic whiteboards 1b and 1c.

[0215] For example, assume that user B using the electronic whiteboard 1b moves the stroke image ST1 on page 1 to the left in step S2301 in Fig. 23. In this case, the communication management system 5 and the electronic whiteboard 1 execute the processes of steps S2302 to S2304.

[0216] In step S2302, the information transmitting unit 16 of the interactive whiteboard 1b transmits to the communication management system 5 operation information for moving the stroke image ST1 (stroke1) on page 1 to the left by a predetermined distance.

[0217] However, since the operation of page 1 has already been restricted by the operation restriction process performed by the operation restriction unit 57 of the communication management system 5, the stroke information management unit 56 cannot move the stroke image ST1 of page 1. In this case, in step S2303, the stroke information management unit 56 of the communication management system 5 transmits a failure notification to the electronic whiteboard 1b, indicating that the movement of the stroke image ST1 of page 1 has failed.

[0218] In step S2304, upon receiving the failure notification from the communication management system 5, the image processing unit 15 of the electronic whiteboard 1b returns the stroke image ST1 of page 1 that failed to be moved to its original position. This allows the stroke image of page 1 displayed on the electronic whiteboards 1a, 1b, and 1c to be maintained in the same state.

[0219] In the second embodiment, when the electronic whiteboard 1b displays the display screen 2200 as shown in FIG. 22, if a move operation is received on the stroke image ST1 of page 1, the processing of steps S2302 to S2304 in FIG. 23 can also be applied.

[0220] Next, a description will be given of the processing that occurs after the operation limiting unit 57 of the communication management system 5 transmits a reception notification indicating that the group operation request has been received to the requesting interactive whiteboard 1a in step S2305.

[0221] In steps S2306 and S2307, the operation restriction unit 57 of the communication management system 5 An operation restriction notice indicating that the operation restriction process has started is sent to all other electronic whiteboards 1b and 1c participating in the video conference.

[0222] In steps S2308 and S2309, the electronic whiteboards 1b and 1c that have received the operation restriction notification from the communication management system 5 prohibit operations on the stroke image of page 1 that is the target of the operation restriction. For example, the display control units 14 of the electronic whiteboards 1b and 1c display a display screen 2400 as shown in FIG. 24, and the reception units 12 of the electronic whiteboards 1b and 1c prohibit reception of editing operations on the stroke image of page 1.

[0223] 24 is a diagram showing an example of a display screen according to the third embodiment. This diagram shows an example of a display screen 2400 displayed by the interactive whiteboards 1b and 1c in steps S2308 and S2309 of FIG.

[0224] 24, it is assumed that page 1 is displayed on the display screen 2400 of the electronic whiteboards 1b and 1c. However, page 1 is displayed, for example, grayed out, so that it cannot be operated. Also, a message 2401 indicating that operation of page 1 is restricted and that it cannot be operated is displayed on the display screen 2400. Preferably, this message 2401 includes information (for example, the terminal name, the destination name, the terminal ID, etc.) about the electronic whiteboard 1a that has locked page 1 (operation restricted).

[0225] By the above process, editing operations cannot be performed on the stroke images of page 1 from the electronic whiteboards 1b and 1c, so user A who uses the electronic whiteboard 1a can operate the stroke images included in page 1 with priority.

[0226] In the second and third embodiments, an example of restricting the operation of the stroke image of page 1 has been described as an example, but the stroke data to be restricted may be one or more selected stroke images, as in the first embodiment. Also, the stroke data to be restricted may be stroke images of multiple pages.

[0227] As described above, according to each embodiment of the present invention, in a session in which stroke images are shared among multiple terminal devices, it is possible to prevent operations from being performed on the stroke images from multiple terminal devices that are different from the user's intention.

[0228] <Supplementary information> Each function of each embodiment described above can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the functions described above.

[0229] Furthermore, in the above-described embodiment, the terminal device has been described as being the electronic whiteboard 1, but the present invention is not limited to this. For example, each embodiment of the present invention can also be applied to an interactive projector 2501 as shown in Fig. 25. The interactive projector 2501 is a terminal device that projects a display screen 2503 onto a projection surface 2502, and allows users to write stroke images, select menus, and so on with a pen, a user's finger, or the like, just like the electronic whiteboard 1.

[0230] Other examples of the terminal device may include information terminals such as PCs, smartphones, smart watches, and car navigation devices that execute applications compatible with the communication system 100. The terminal device may also include medical equipment. In the case of medical equipment, the document images may include images of patients, etc.

[0231] Furthermore, in each of the above-described embodiments, the communication system 100 is used for video conferencing, but the present invention is not limited to this and may be used for meetings, general conversations between family members or friends, remote medical treatment, or one-way presentation of information. [Explanation of symbols]

[0232] 1. Electronic whiteboard (an example of a terminal device) 5. Communication management system (communication management device) 51 Transmitting and receiving unit (receiving unit) 56 Stroke information management section 57 Operation Restriction Section 100 Communication Systems 1910, 1920 Group Operation Information 5006 Stroke Information [Prior art documents] [Patent documents]

[0233] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-60592

Claims

1. A communication management system for managing a session for sharing stroke images between a plurality of terminal devices including a first terminal device and a second terminal device, comprising: a stroke information management unit that manages stroke information including a plurality of stroke data representing the stroke image; a receiving unit that receives, from the first terminal device, a group operation request that specifies one or more stroke data to be subjected to an editing operation among the plurality of stroke data; an operation restriction unit that restricts an editing operation on the one or more stroke data by the second terminal device different from the first terminal device based on a group operation request that specifies the one or more stroke data to be the target of the editing operation; having and when the one or more stroke data specified by the group operation request include stroke data for which editing operations are restricted by the operation restriction process, the operation restriction unit starts the operation restriction process, thereby allowing editing operations on the one or more stroke data to be accepted from the first terminal device and prohibiting editing operations on the one or more stroke data by the second terminal device. and when the one or more stroke data specified by the group operation request include stroke data for which editing operations are restricted by the operation restriction process, the operation restriction unit does not allow editing operations on the one or more stroke data to be accepted from the first terminal device, thereby prohibiting editing operations on the one or more stroke data by the first terminal device.

2. 2. The communication management system according to claim 1, wherein the operation restriction unit discontinues processing to prohibit the second terminal device from operating the one or more stroke data when the one or more stroke data specified by the group operation request includes stroke data for which operation is being restricted.

3. The operation limiting unit is 3. A communication management system as described in claim 1 or 2, wherein after prohibiting the second terminal device from operating on the one or more stroke data, when the first terminal device performs an operation on each of the one or more stroke data that are the target of the operation, the prohibition on the operation on the one or more stroke data is terminated.

4. the stroke information management unit manages the plurality of stroke data in association with first identification information for identifying the session; 4. A communication management system according to claim 1, wherein the stroke data includes second identification information for identifying the stroke data, drawing data for reproducing the stroke image, and third identification information for identifying the drawing data.

5. The communication management system according to claim 4 , wherein the group operation request includes the first identification information and the second identification information that identifies the one or more stroke data.

6. 6. A communication system including the communication management system according to claim 1 and a plurality of terminal devices participating in a session managed by the communication management system, The terminal device a reception unit that receives a designation operation for designating one or more pieces of stroke data to be operated on and an editing operation for the one or more pieces of stroke data; an information sending unit that sends a group operation request to the communication management system, the group operation request specifying one or more stroke data items to be subjected to the editing operation, in response to the specifying operation; A communication system comprising:

7. The communication system according to claim 6, wherein the terminal device has a display control unit that displays on a display screen display information indicating that operations relating to one or more stroke data are restricted in response to an operation restriction notification notified from the communication management system, the operation restriction notification restricting operations relating to the one or more stroke data.

8. The communication system according to claim 6 or 7, wherein the reception unit prohibits the reception of editing operations regarding one or more stroke data in response to an operation restriction notification notified from the communication management system, the operation restriction notification restricting operations regarding the one or more stroke data.

9. The operation restriction unit of the communication management system includes: A communication system according to any one of claims 6 to 8, further comprising: a group operation request received from a first terminal device, the group operation request specifying one or more stroke data, and an operation restriction notification indicating that operation relating to the one or more stroke data is restricted is sent to each of the second terminal devices among the plurality of terminal devices, the second terminal device being different from the first terminal device.

10. A communication management device that manages a session for sharing a stroke image between a plurality of terminal devices including a first terminal device and a second terminal device, a stroke information management unit that manages stroke information including a plurality of stroke data representing the stroke image; a receiving unit that receives, from the first terminal device, a group operation request that specifies one or more stroke data to be subjected to an editing operation among the plurality of stroke data; an operation restriction unit that restricts an editing operation on the one or more stroke data by the second terminal device different from the first terminal device based on a group operation request that specifies the one or more stroke data to be the target of the editing operation; having the operation restriction unit, when the one or more stroke data specified by the group operation request does not include stroke data for which editing operations are restricted by operation restriction processing, starts the operation restriction processing, thereby allowing editing operations on the one or more stroke data to be accepted from the first terminal device and prohibiting editing operations on the one or more stroke data by the second terminal device; and, when the one or more stroke data specified by the group operation request includes stroke data for which editing operations are restricted by the operation restriction processing, does not allow editing operations on the one or more stroke data to be accepted from the first terminal device, thereby prohibiting editing operations on the one or more stroke data by the first terminal device.

11. A communication management system for managing a session for sharing a stroke image between a plurality of terminal devices including a first terminal device and a second terminal device, comprising: A process of managing stroke information including a plurality of stroke data representing the stroke image; receiving, from the first terminal device, a group operation request for designating one or more stroke data to be subjected to an editing operation among the plurality of stroke data; a process of restricting an editing operation on the one or more stroke data by the second terminal device different from the first terminal device based on a group operation request specifying the one or more stroke data to be subjected to the editing operation; Run The process of restricting the editing operation, when the one or more stroke data specified by the group operation request does not include stroke data for which editing operations are restricted by an operation restriction process, starts the operation restriction process, thereby making it possible to accept editing operations on the one or more stroke data from the first terminal device and prohibiting editing operations on the one or more stroke data by the second terminal device; and when the one or more stroke data specified by the group operation request includes stroke data for which editing operations are restricted by an operation restriction process, does not make it possible to accept editing operations on the one or more stroke data from the first terminal device, thereby prohibiting editing operations on the one or more stroke data by the first terminal device. Image processing methods.

12. A program for causing a communication management system to execute the image processing method according to claim 11.

Images