Remote meeting support system, remote meeting support device, and method for supporting remote meeting

Abstract

To support, in a remote meeting in which a plurality of participants are in a first region and a plurality of participants are in a second region which is different from the first region, a participant in the second region to grasp situations in the first region.SOLUTION: A remote meeting support system (1) for supporting a remote meeting between a plurality of first participants (20a to e) in a first region (10) and a plurality of second participants (110a to c) in a second region (100a to c), acquires detection information regarding detection of the state of the first region (10), analyzes operations of the first participants (20a to e) on the basis of the detection information, and notifies terminals (120a to c) used by the second participants (110a to c) of the results of analysis.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a remote conference supporting system, a remote conference supporting device, and a remote conference supporting method. [Background technology]

[0002] In today's world, where teleworking is promoted and information and communication technology is developing, remote meetings using online conferencing systems are being held frequently.

[0003] Patent Document 1 describes a technique for controlling the display mode of an avatar corresponding to each user in an online conference system according to the situation of each user participating in the online conference. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2022-95256 A Summary of the Invention [Problem to be solved by the invention]

[0005] While remote conferences are being held frequently, there is also a demand to utilize the advantages of face-to-face communication. Therefore, in addition to the conference format in which all participants participate remotely as described in Patent Document 1, a conference format has emerged in which some participants gather in a conference room, while other participants participate remotely, and all participants proceed with the conference together.

[0006] In this type of conference, participants in the conference room can grasp the situation in the conference room, but participants participating remotely may find it difficult to grasp the situation in the conference room.

[0007] One aspect of the present invention has been made in consideration of the above problems. One example of the purpose of the present invention is to provide a technology for supporting a participant in a second area to grasp a situation in a first area in a remote conference in which there are multiple participants in a first area and participants in a second area different from the first area. [Means for solving the problem]

[0008] A remote conference supporting system according to one embodiment of the present invention is a remote conference supporting system that supports a remote conference between a plurality of first participants in a first area and a second participant in a second area different from the first area, and includes an acquisition means for acquiring detection information that detects a state within the first area, an analysis means for analyzing the behavior of the plurality of first participants based on the detection information, and a notification means for notifying a terminal used by the second participant of the analysis result of the analysis means.

[0009] A remote conference supporting device according to one embodiment of the present invention is a remote conference supporting device that supports a remote conference between a plurality of first participants in a first area and a second participant in a second area different from the first area, and includes an acquisition unit that acquires detection information that detects a state within the first area, an analysis unit that analyzes the actions of the plurality of first participants based on the detection information, and a notification unit that notifies a terminal used by the second participant of the analysis result of the analysis unit.

[0010] A remote conference support method according to one embodiment of the present invention is a remote conference support method for supporting a remote conference between a plurality of first participants in a first area and a second participant in a second area different from the first area, and includes acquiring detection information that detects a state within the first area, analyzing the actions of the plurality of first participants based on the detection information, and notifying a terminal used by the second participant of the results of the analysis. Effect of the Invention

[0011] According to one aspect of the present invention, in a remote conference in which there are multiple participants in a first area and also participants in a second area different from the first area, it is possible to assist participants in the second area in understanding the situation in the first area. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an example of a configuration of a remote conference support system according to an exemplary embodiment 1 of the present invention. [Diagram 2] FIG. 1 is a block diagram showing an example of an environment in which a remote conference is held in exemplary embodiment 1 of the present invention. [Diagram 3] 1 is a flowchart showing an example of a remote conference support method according to an exemplary embodiment 1 of the present invention. [Figure 4] 1 is a block diagram showing an example of a configuration of a remote conference support device according to an exemplary embodiment 1 of the present invention. [Diagram 5] FIG. 11 is a block diagram showing an example of a configuration of a remote conference supporting system according to an exemplary embodiment 2 of the present invention and an example of an environment in which a remote conference is implemented. [Figure 6] FIG. 11 is a diagram showing an example of a conference room in the second exemplary embodiment of the present invention. [Figure 7] 11 is a flowchart showing an example of a remote conference support method according to an exemplary embodiment 2 of the present invention. [Figure 8] FIG. 11 is a block diagram showing an example of a connection in a remote conference support system according to an exemplary embodiment 2 of the present invention. [Figure 9] FIG. 11 is a block diagram showing an example of a connection of a remote conference support device according to an exemplary embodiment 2 of the present invention. [Figure 10] FIG. 2 is a conceptual block diagram showing a computer used as an information processing device according to each exemplary embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Example 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described later.

[0014] (Remote meeting support system) Fig. 1 is a block diagram showing an example of a configuration of a remote conference supporting system 1 according to this exemplary embodiment. Fig. 2 is a block diagram showing an example of an environment in which a remote conference is held in this exemplary embodiment.

[0015] In one example, the remote conference support system 1 supports a remote conference between a plurality of first participants 20a-e in a first area 10 and second participants 110a-c in second areas 100a-c different from the first area 10. The remote conference support system 1 includes an acquisition means 3 that acquires detection information that detects a state in the first area 10, an analysis means 6 that analyzes the actions of the plurality of first participants 20a-e based on the detection information, and a notification means 9 that notifies the terminals 120a-c used by the second participants 110a-c of the analysis results of the analysis means 6.

[0016] In this specification, a remote conference refers to a conference in which some conference participants use means other than the five senses to communicate with other conference participants, such as, for example, information transmission using a communication network and information display on an information display device. In one embodiment, the remote conference is conducted using a WEB conference system such as ZOOM (registered trademark).

[0017] In one embodiment, the first area 10 refers to a company's conference room, for example, but is not limited thereto, and may be any place where multiple people can gather and hold a meeting.

[0018] In one embodiment, the first participants 20a to 20e refer to people who participate in a remote conference held in the first domain 10, connecting the first domain 10 and the second domains 100a to 100c.

[0019] In one embodiment, the second area 100a-c refers to a location different from the first area 10, and as an example, refers to the locations (e.g., homes, offices, etc.) of people participating in a conference held in the first area 10 using a WEB conferencing system, but is not limited to this.

[0020] In one embodiment, the second participants 110a to 110c refer to people who participate in a remote conference held by connecting the first domain 10 and the second domains 100a to 100c in the second domains 100a to 100c.

[0021] In one embodiment, detection information detecting the state within the first area 10 refers to data obtained by acquiring physical quantities representing the state within the first area using a sensor or the like, and examples include, but are not limited to, still image or video data captured by a camera, motion data of the first participants 20a-e detected by a motion sensor, and position data of the first participants 20a-e detected by a human presence sensor.

[0022] In one embodiment, the acquisition means 3 is connected to these sensors and acquires the detection information.

[0023] In one embodiment, the analysis results obtained by analyzing the behavior of the first participants 20a-e refer to, for example, the results of identifying the behavior of the first participants 20a-e related to the progress of the meeting held in the first area 10, such as the behavior of each of the first participants 20a-e (e.g., speaking, folding arms, operating an object in the first area 10 (e.g., a whiteboard, a laptop, a prototype to be discussed in the meeting, etc.), dozing, leaving the first area 10, etc.), the relationships between the first participants 20a-e (e.g., other first participants paying attention to a first participant who is speaking, the first participants 20a-e gathering in front of an object, other first participants looking at the words written by a first participant on the whiteboard, the first participants talking to each other, etc.), but are not limited to these.

[0024] In one embodiment, the analysis means 6 obtains the above-mentioned analysis result by analyzing the detection information. For example, the analysis means 6 may analyze the movement of each of the first participants 20a-e based on still images, video data, motion data, position data, etc. The analysis means 6 may also identify the position and orientation of each of the first participants 20a-e based on still images, video data, motion data, position data, etc., and analyze the interrelationships between the first participants 20a-e.

[0025] In one embodiment, the terminals 120a-c used by the second participants 110a-c refer to devices used by the second participants 110a-c who participate in the remote conference in the second areas 100a-c to participate in the remote conference. Examples include, but are not limited to, information processing devices (for example, personal computers, tablet terminals, smartphones, etc.) equipped with a communication function, an audio playback function, and an image display function.

[0026] In one embodiment, the notification means 9 notifies the terminals 120a-c used by the second participants 110a-c of data indicating the analysis result obtained by the analysis means 6 analyzing the state in the first area 10 via a network or the like. As an example, the notification means 9 may display characters indicating the analysis result on the image display units (displays) of the terminals 120a-c used by the second participants 110a-c, but is not limited to this. When there are multiple second participants 110a-c, the notification means 9 may notify each second participant of the same information, or may notify different information appropriately depending on the second participant to be notified. By the second participants 110a-c recognizing the notified analysis result, the second participants 110a-c can know matters related to the conference held in the first area 10.

[0027] (How to support remote meetings) A remote meeting assistance method according to this exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the remote meeting assistance method.

[0028] In step S1, the acquiring means 3 acquires detection information on the state within the first area 10.

[0029] In step S2, the analysis means 6 analyzes the actions of the multiple first participants 20a to 20e based on the detection information acquired in step S1.

[0030] In step S3, the notification means 9 notifies the terminals 120a to 120c used by the second participants 110a to 110c of the analysis result in step S2.

[0031] (Remote meeting support device) In one aspect, the present exemplary embodiment may be realized by a remote conference supporting device 150 instead of the remote conference supporting system 1.

[0032] The configuration of the remote conference supporting apparatus 150 according to this exemplary embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing an example of the configuration of the remote conference supporting apparatus 150.

[0033] The remote conference supporting device 150 is a remote conference supporting device that supports a remote conference between a plurality of first participants 20a-e in a first area 10 and second participants 110a-c in second areas 100a-c different from the first area 10. The remote conference supporting device 150 includes an acquisition unit 160 that acquires detection information that detects a state in the first area 10, an analysis unit 170 that analyzes the actions of the plurality of first participants 20a-e based on the detection information, and a notification unit 180 that notifies the terminals 120a-c used by the second participants 110a-c of the analysis results of the analysis unit 170.

[0034] (Effects of this embodiment) As described above, according to this exemplary embodiment, it is possible to inform conference participants who are not in the first area 10 (the second participants 110a-c in the second areas 100a-c) of actions related to the progress of the conference being performed by the multiple first participants 20a-e in the first area 10. This makes it possible for remote conference participants to recognize the status of the conference, and it is expected that the conference will proceed smoothly.

[0035] Exemplary embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and the description thereof will be omitted as appropriate.

[0036] 5 is a block diagram showing an example of a configuration of a remote conference supporting system 1 according to this exemplary embodiment and an example of an environment in which a remote conference is implemented. The remote conference supporting system 1 is connected to an in-house system 240 and a WEB conference system 270.

[0037] The in-house system 240 is connected to conference rooms 300a to 300z, each of which corresponds to the first area 10 in the exemplary embodiment 1. The conference rooms 300a to 300z may be collectively referred to as conference rooms 300. The number of conference rooms 300 connected to the in-house system 240 is not particularly limited as long as it is one or more.

[0038] The WEB conference system 270 is connected to the terminals 120a to 120z used by the second participants. The terminals 120a to 120z may be collectively referred to as the terminals 120. The number of terminals 120 connected to the WEB conference system 270 is not particularly limited as long as it is one or more.

[0039] 6 is a diagram showing an example of the state of the conference room 300. In the conference room 300, there are a whiteboard 310, a camera 330, a human presence sensor 340, a motion sensor 350, an entrance door 360, a table 370, and first participants 320a-f. Note that the first participants 320a-f may be collectively referred to as the first participants 320.

[0040] The human presence sensor 340 refers to a sensor for detecting the presence of a human being.

[0041] The camera 330 is attached in the conference room 300 at a position where it can capture still images or videos in the conference room 300. The camera 330 also has a performance that can capture images with a sufficient resolution to analyze the movements of any first participant in the entire conference room 300. Although only one camera 330 is shown in FIG. 6, multiple cameras may be installed to satisfy the above conditions of capturing an image of the entire conference room and capturing an image with a sufficient resolution for motion analysis. Also, multiple cameras may be installed so that the first participant 320 in the conference room 300 can be captured from various angles, which contributes to improving the performance of the motion analysis.

[0042] The motion sensor 340 is attached in the conference room 300 at a position where the position of the first participant 320 can be detected. Although only one motion sensor 340 is shown in FIG. 4, multiple motion sensors may be installed to satisfy the above-mentioned condition of presence detection in all areas. Furthermore, multiple motion sensors 340 may be installed so that the first participant 320 in the conference room 300 can be detected from various angles, thereby contributing to improving the performance of presence detection. Examples of the motion sensor 340 include, but are not limited to, those that use infrared rays, those that use ultrasonic waves, those that use visible light, and combinations of two or more of these.

[0043] The motion sensor 350 is attached in the conference room 300 at a position where it can detect the movement of the first participant 320. For example, in the example shown in Fig. 6, the movement of the first participant 320b is the movement of writing characters on the whiteboard 310, and the movements of the first participants 320a and 320c turning their bodies toward the written characters to look at the characters, but is not limited thereto. For example, the movement may be the first participant 320 speaking, the first participant 320 folding his arms, the first participant 320 operating an object in the first area 10, the first participant 320 falling asleep, the first participant 320 leaving the conference room 300, or the like.

[0044] The motion sensor 350 may be appropriately placed according to the position where it is desired to detect the movement of the first participant 320 in the conference room 300. The motion sensor may be placed so as to detect the movement of the first participant 320 throughout the entire area of ​​the conference room 300. The movement of the first participant 320 may also be detected by the camera 330.

[0045] The camera 330, the human presence sensor 340, and the motion sensor 350 may be used not only individually but also in any combination to detect the state in the conference room 300. Also, sensors that detect the state in the conference room 300 other than the camera 330, the human presence sensor 340, and the motion sensor 350 may be used alone or in combination. An example of a sensor that detects the state in the conference room 300 other than the camera, the human presence sensor, and the motion sensor is a carbon dioxide concentration sensor, which may be used to detect the carbon dioxide concentration in the conference room 300 and use it as an index of the intensity of the discussion between the first participants 320.

[0046] The analysis means 6 analyzes the behavior of the first participant 320 based on the detection information detected by the camera 330, the human presence sensor 340, and the motion sensor 350. In light of the purpose of notifying the second participant of the status of the multiple first participants in the conference room 300, it is preferable that the analysis means 6 analyzes information on the reaction of other participants to the behavior of any one of the first participants 320. By analyzing the information detected by such sensors by the analysis means 6, it is possible to obtain information representing the state of interaction between the multiple first participants, which is suitable as information representing the status of the multiple first participants to be notified to the second participant.

[0047] In addition, in light of the purpose of notifying the second participant of the status of multiple first participants in the conference room 300, it is preferable that the analysis means 6 analyzes, for example, the target of gaze of each of the multiple first participants, the entry and exit of the first participants into the conference room, and coordinated actions performed by at least two of the multiple first participants.

[0048] By identifying the target of gaze, it becomes easier to estimate the target on which attention is focused, and when the target of gaze of multiple first participants is the same, it becomes easier to estimate that the focus of the entire conference is fixed on the target of gaze. Also, it may be confirmed that first participants 320d and 320e, who are sitting across from each other at table 370, are indeed talking because their gazes are directed at each other. Analysis information on gaze may be used to detect whether participants are dozing off.

[0049] In addition, since the number of people participating in a conference is important information for the progress of the conference because decisions may be made by majority vote, the analysis means 6 can know the increase or decrease in the number of people participating in the conference by analyzing the entry and exit of the first participant 320 to the conference room 300. The analysis of entry and exit can be performed by analyzing information around the door 360 for entering and exiting the conference room 300. An opening and closing detection sensor that is attached to the door 360 and detects the opening and closing of the door 360 may be used in combination with another sensor.

[0050] In addition, if at least two of the multiple first participants 320 are performing an action in coordination, this can be useful information for estimating changes in the situation of the entire conference, so it is useful to detect this and notify the terminal 120.

[0051] As described above, since the number of people participating in a conference is important information for the progress of the conference, it is desirable to be able to identify the number of first participants by various sensors that detect the situation in the conference room 300. This identification may be performed, for example, by analyzing information detected by the human presence sensor 340, by analyzing information detected by the camera 330 using artificial intelligence, or by analyzing information detected by various other sensors.

[0052] When the number of first participants 320 is small (for example, 0 or 1), there is little point in analyzing the overall situation of the conference room 300. Therefore, the notification means 9 may perform control based on the number of first participants 320 so as not to analyze the overall situation of the conference room 300 and notify the terminal 120 of the second participant of the analysis result.

[0053] The control to not analyze the overall situation of the conference room 300 and notify the terminal of the second participant of the analysis results may be performed in any situation, not limited to when the number of first participants is 0 or 1.

[0054] Furthermore, the analysis of the detection information from the various sensors may be performed every time a predetermined time period that can be determined arbitrarily has elapsed, and the predetermined time period may be adjustable according to the progress speed of the discussion in the conference.

[0055] The analysis may be performed by using a suitable method according to the sensor used, but as an example, a feature of a person may be extracted from a camera image and the motion of the captured person may be analyzed. In this specification, the feature refers to a quantitative expression of a feature for understanding the situation in the conference room 300, and an example is the motion of the first participant 320.

[0056] Furthermore, if the analysis result is the same as the previous analysis result, the notification means 9 may not notify the terminal 120 of the analysis result, allowing the second participant to concentrate more on the discussion.

[0057] In one embodiment, it is assumed that there are multiple conference rooms 300, and each conference room 300 has a different type of sensor for detecting the state inside the conference room 300. In such a case, the acquisition means 3 may acquire conference information related to the conference room 300 where the remote conference is held from the WEB conference system 270, and acquire detection information based on the acquired conference information. This can be realized by including a database in the remote conference support system, storing conference information related to the conference room where the remote conference is held and information on various sensors equipped in each conference room in the database so that the acquisition means 3 can search for it.

[0058] When an action such as a speech is made in a meeting, the subject of the action is also considered to be useful information for the second participant. Therefore, the analysis means 6 may specify who the subject of each action included in the analysis result is (i.e., identify an individual) and include it in the notification to the terminal 120. The subject can be specified, for example, by obtaining face information and skeletal information of employees who may attend the meeting in advance, and then combining camera information, face recognition technology, and skeletal detection technology, but is not limited to this. The face recognition technology or skeletal detection technology may be used alone, or may be used in combination with technology other than the face recognition technology or skeletal detection technology.

[0059] The notification to the terminal 120 may be, for example, by displaying text information on the display screen of the terminal 120 used by the second participant, but is not limited to this. Audio information, vibration of the terminal, etc. may also be used, or these methods may be used in combination. If the display screen of the terminal 120 used by the second participant is small, it may be difficult to recognize the information depending on the amount of text information, so it is particularly preferable to use other methods or use them in combination.

[0060] Even when notifying information by text information, the more the amount of text information increases, the more difficult it becomes to recognize the information. For example, the importance of the information to be notified may be analyzed by AI, and the more important the information, the longer the time from when it is displayed on the screen until it automatically disappears, and the less important the information, the shorter the time it is displayed on the screen and the sooner it automatically disappears. Also, the second participant may wear AR (Augmented Reality) glasses on his eyes, and the information to be notified may be displayed on the AR glasses. When selling a product for realizing the remote conference support system according to this exemplary embodiment, for example, an information notification function to the AR glasses may be added as a premium plan function.

[0061] 7 is a flowchart showing an example of a process flow of the remote conference support system 1 according to this exemplary embodiment. The remote conference support system 1 is assumed to be connected to a database having information on the schedule of conferences, reservation information for each of a plurality of conference rooms 300, and information on the camera 330 provided in each conference room 300. It is also assumed that the first area 10 is the conference room 300, and the acquisition means 3 is the camera 330.

[0062] In step S11, the acquisition means 3 acquires, from the Web conference system 270, conference information indicating a schedule of a conference.

[0063] In step S12, the acquisition means 3 acquires information as to whether or not there is a conference room 300 that has been reserved at the scheduled time of the conference.

[0064] In step S12, if there is no conference room 300 reserved for the scheduled time of the conference, the obtaining means 3 ends the process.

[0065] In step S12, if there is a conference room 300 reserved for the scheduled time of the conference, the obtaining means 3 proceeds to step S13.

[0066] In step S13, the acquisition means 3 acquires information on the cameras 330 provided in the conference rooms 300 that are reserved for the scheduled time of the conference, and identifies the cameras 330 that can be used.

[0067] In step S14, the acquisition means 3 acquires information as to whether or not there is a meeting that is scheduled to start at the current time.

[0068] In step S14, if there is no conference starting at the current time, the obtaining means 3 waits until it is determined that there is a conference starting at the current time.

[0069] In step S14, if there is a conference that is to start at the current time, the obtaining means 3 proceeds to step S15.

[0070] In step S15, the acquisition means 3 captures images of the conference room 300 in which the conference that was determined to start in step S14 will take place using the camera 330 provided in the conference room 300, and starts acquiring video showing the situation inside the conference room 300. In each step from step S15 to the end of processing, the acquisition means 3 may continue to capture images of the conference room 300 using the camera and acquire the video. Furthermore, the acquisition means may stop capturing images of the inside of the conference room 300 and acquiring the video as appropriate. If the stop is performed appropriately, it is expected that the operating costs can be reduced.

[0071] In step S16, the analysis means 6 analyzes the video of the video acquired in step S15, and ascertains the number of first participants 320.

[0072] In step S17, the analysis means 6 judges whether the number of first participants 320 in the conference room 300 is two or more.

[0073] In step S17, if the analysis means 6 determines that the number of first participants 320 in the conference room 300 is not two or more, that is, that there is zero or one, the process ends. When there is one participant, the information that the second participant should obtain about the first participant 320 in order to understand the situation of the conference is considered to be sufficient with the image information of the first participant 320 and the voice information of the first participant 320, and there is basically no difference from the second participant.

[0074] In step S17, if the analysis means 6 determines that the number of first participants 320 in the conference room 300 is two or more, the process proceeds to step S18.

[0075] In step S18, the analysis means 6 analyzes the video of the inside of the conference room 300 that has been continuously acquired since step S15, and analyzes the actions of the first participants 320.

[0076] In step S19, the analysis means 6 calculates an analysis result and determines whether or not the analysis result differs from the analysis result obtained in the previous derivation of the analysis result.

[0077] In step S19, if the analysis means 6 determines that the calculated analysis result is the same as the previous analysis result, the process returns to step S18 and continues analyzing the actions of the multiple first participants 320 in the conference room 300. In this case, the analysis result does not need to be notified to the terminal 120.

[0078] In step S19, if the analysis means 6 determines that the calculated analysis result differs from the previous analysis result, the process proceeds to step S20.

[0079] In step S20, the notification means 9 transmits the analysis result of the analysis means 6 to the terminal 120.

[0080] In step S21, the acquisition means 3 determines whether the conference being imaged has reached its end time. If the acquisition means 3 determines in step S21 that the conference being imaged has not reached its end time, the process returns to step S18, and the analysis means 6 analyzes the video of the conference room 300 that the acquisition means 3 has continued to acquire since step S15, and analyzes the actions of the first participants 320.

[0081] If it is determined in step S21 that the time for the conference being imaged has come to an end, the process ends.

[0082] (Connection example) 8 is a block diagram showing an example of connections of the remote conference support system 1 in this exemplary embodiment. As shown in FIG. 8, the remote conference support system 1, the conference room 300, the in-house system 240, the WEB conference system 270, and the terminal 120 may be connected to each other via a network.

[0083] Also in this exemplary embodiment, a remote conference supporting device 150 may be used instead of the remote conference supporting system 1. Fig. 9 is a block diagram showing an example of a connection of the remote conference supporting device 150 in this exemplary embodiment. As shown in Fig. 9, the remote conference supporting device 150 may be directly connected to a conference room 300, and the remote conference supporting device 150, the in-house system 240, the WEB conference system 270, and the terminal 120 may be connected to each other via a network.

[0084] In this manner, the remote conference support device 150 may be configured as an MEC server provided near the conference room 300. By using the MEC server near the conference room 300 to analyze the detection information, the analysis can be completed in a shorter time, and the analysis results can be promptly transmitted to the second remote participant.

[0085] (Modification) In one embodiment, the remote meeting support system 1 or remote meeting support apparatus 150 may be adapted to operate in a meeting hosted by a first participant 320 in a meeting room 300 .

[0086] The purpose of this device is to prevent a second remote participant from being aware of what is going on in the conference room 300, so it is more effective when used in a conference dominated by a first participant 320 in the conference room 300.

[0087] The above configuration is realized by the acquisition means 3 of the remote conference support system 1 or the acquisition unit 160 of the remote conference support device 150 acquiring the audio of the conference, and the analysis means 6 or the analysis unit 170 performing the analysis. For example, the acquisition means 3 or the acquisition unit 160 accesses the WEB conference system 270 and acquires the audio in the conference. Then, when the audio of multiple people (first participant 320) is input from an account participating from the conference room 300 (for example, the WEB conference account of the conference room 300) and there is no audio input from the second participant remotely for a certain period of time, the analysis means 6 or the analysis unit 170 analyzes that "the conference room is the main conference room." When the above judgment is made, the remote conference support system 1 or the remote conference support device 150 may execute the remote conference support method according to this exemplary embodiment.

[0088] In addition, in one embodiment, the remote conference supporting system 1 or the remote conference supporting device 150 may be configured to perform video analysis and transmit the analysis results based on the request of a second participant participating remotely when the second participant wishes to obtain the analysis results.

[0089] In one embodiment, the remote meeting support system 1 or the remote meeting support device 150 may be incorporated into a Web conference system 270.

[0090] In one embodiment, the remote conference supporting system 1 or the remote conference supporting device 150 may cooperate with a schedule management system such as the in-house system 240 instead of the WEB conference system 270. In that case, the acquisition means 3 or the acquisition unit 160 acquires conference information indicating the reservation of the conference room 300 and the presence or absence of remote participants from the schedule management system. For example, the acquisition means 3 or the acquisition unit 160 may make a determination based on whether or not a link to participate in a WEB conference is included in the schedule of the schedule management system such as the in-house system 240.

[0091] In one embodiment, the remote meeting support system 1 or the remote meeting support apparatus 150 may cooperate with both the schedule management system and the Web conference system 270 .

[0092] In one embodiment, the notification means 9 or the notification unit 180 may notify the terminal 120 not only with text but also with an image such as an avatar.

[0093] (Effects of this exemplary embodiment) As described above, according to this exemplary embodiment, by using various sensors, information useful for the second participant to know the state of the conference among the conditions in the conference room 300 can be notified in a manner suitable for the second participant. Also, based on the number of first participants 320 in the conference room 300 and the analysis result of the behavior of the first participants 320, information useful to be notified to the second participant among the conditions in the conference room 300 can be selectively notified to the second participant. This allows a conference in which there are multiple first participants 320 in the conference room 300 and a second participant who participates there remotely to proceed smoothly.

[0094] [Software implementation example] A part or all of the functions of the remote conference supporting system 1 and the remote conference supporting apparatus 150 may be realized by hardware such as an integrated circuit (IC chip), or may be realized by software.

[0095] In the latter case, the remote meeting supporting system 1 and the remote meeting supporting device 150 are realized, for example, by a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter, referred to as computer C) is shown in FIG. 10. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the remote meeting supporting system 1 and the remote meeting supporting device 150. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the remote meeting supporting system 1 and the remote meeting supporting device 150.

[0096] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0097] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may further include a communication interface for transmitting and receiving data to and from other devices. The computer C may further include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0098] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. Furthermore, the program P can be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.

[0099] [Additional Note 1] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiment are also included in the technical scope of the present invention.

[0100] [Additional Note 2] Some or all of the above-described embodiments can be described as follows. However, the present invention is not limited to the following described aspects.

[0101] (Appendix 1) A remote conference support system that supports a remote conference between a plurality of first participants in a first area and a second participant in a second area different from the first area, the remote conference support system comprising: an acquisition means that acquires detection information that detects a state within the first area; an analysis means that analyzes the actions of the plurality of first participants based on the detection information; and a notification means that notifies a terminal used by the second participant of the analysis result of the analysis means.

[0102] The above configuration can assist participants in the second area in understanding the situation in the first area.

[0103] (Appendix 2) The remote conference support system according to claim 1, wherein the analysis means analyzes the reactions of the other first participants to the actions of any one of the first participants.

[0104] The above configuration can assist participants in the second area in understanding the situation in the first area.

[0105] (Appendix 3) The remote conference support system described in Appendix 1 or 2, wherein the analysis means performs at least one of the following: identifying the target of gaze of each of the first participants; identifying entry and exit of the first participants into a conference room; and identifying coordinated actions performed by at least two of the first participants.

[0106] According to the above configuration, it is possible to assist the participants in the second area in understanding the situation in the first area.

[0107] (Appendix 4) The remote conference support system according to any one of appendices 1 to 3, wherein the detection information is information detected by one or more of a camera, a motion sensor, and a human presence sensor arranged in the first area.

[0108] According to the above configuration, it is possible to obtain detection information for analyzing the situation in the first area.

[0109] (Appendix 5) The remote conference support system according to any one of appendices 1 to 4, wherein the analysis means identifies the number of the first participants, and the notification means decides whether or not to notify the terminal depending on the number of participants identified by the analysis means.

[0110] According to the above configuration, in the case where the second participant can easily grasp the situation in the first area, taking into account the number of first participants, it is expected that the second participant will be able to concentrate more on the conference discussion by not sending a notification to the second participant's terminal.

[0111] (Appendix 6) The remote conference support system of any one of Appendices 1 to 5, wherein the analysis means further comprises a control means for repeatedly performing the analysis and determining whether a certain analysis result of the analysis means is different from a previous analysis result of the certain analysis result, and when the control means determines that the certain analysis result is different from the previous analysis result, the control means causes the notification means to notify the certain analysis result.

[0112] According to the above configuration, only when it is determined that a certain analysis result of the analysis means is different from the previous analysis result, the notification means can notify the certain analysis result, so that information that is less necessary to be notified is not notified to the second participant. This is expected to enable the second participant to concentrate more on the discussion of the meeting.

[0113] (Appendix 7) The remote conference support system of any one of Appendices 1 to 6, wherein the first area is a conference room, and the acquisition means acquires conference information regarding the conference room where the remote conference is held, and acquires the detection information based on the acquired conference information.

[0114] According to the above configuration, when there are multiple conference rooms, necessary detection information can be automatically acquired.

[0115] (Appendix 8) 8. The remote conference support system according to claim 1, wherein the analysis means identifies a subject of each action included in the analysis result.

[0116] According to the above configuration, the second participant can know the subject of the information of each action that is useful for knowing the situation of the conference room, and therefore can know the situation of the conference room better.

[0117] (Appendix 9) A remote conference support device that supports a remote conference between a plurality of first participants in a first area and a second participant in a second area different from the first area, the remote conference support device comprising: an acquisition unit that acquires detection information that detects a state within the first area; an analysis unit that analyzes the behavior of the plurality of first participants based on the detection information; and a notification unit that notifies a terminal used by the second participant of the analysis result of the analysis unit.

[0118] According to the above configuration, the same effect as in Supplementary Note 1 can be obtained.

[0119] (Appendix 10) A remote conference support method for supporting a remote conference between a plurality of first participants in a first area and a second participant in a second area different from the first area, the remote conference support method including: acquiring detection information that detects a state within the first area; analyzing the actions of the plurality of first participants based on the detection information; and notifying a terminal used by the second participant of the results of the analysis.

[0120] According to the above configuration, the same effect as in Supplementary Note 1 can be obtained.

[0121] [Additional Note 3] The present disclosure may further be expressed as the following appended claims. However, the present disclosure is not limited to the following appended claims.

[0122] (Appendix 1) A remote conference support system that supports a remote conference between a plurality of first participants in a first area and a second participant in a second area different from the first area, at least one processor, the at least one processor comprising: An acquisition process for acquiring detection information that detects a state within the first region; an analysis process for analyzing the actions of the first participants based on the detection information; A notification process of notifying a terminal used by the second participant of an analysis result of the analysis process; A remote meeting support system that:

[0123] (Appendix 2) The at least one processor: The remote conference support system according to claim 1, wherein the analysis process analyzes the reactions of other first participants to the actions of any one of the first participants.

[0124] (Appendix 3) The at least one processor: The remote conference support system described in Appendix 1, wherein the analysis process performs at least one of identifying the target of each of the first participants' gaze, identifying the first participants' entry into and exit from the conference room, and identifying actions being performed in coordination by at least two of the first participants.

[0125] (Appendix 4) The remote conference support system described in Appendix 1, wherein the detection information is information detected by one or more of a camera, a motion sensor, and a human presence sensor arranged in the first area.

[0126] (Appendix 5) The at least one processor: In the analysis process, a number of the first participants is identified; The remote conference support system according to claim 1, wherein in the notification process, it is determined whether or not to notify the terminal depending on the number of people identified by the analysis means.

[0127] (Appendix 6) The at least one processor: In the analysis process, the analysis is repeatedly performed; further performing a control process for determining whether a certain analysis result of the analysis process is different from a previous analysis result of the certain analysis result; The remote conference support system according to claim 1, wherein, in the control process, when it is determined that the certain analysis result is different from the previous analysis result, the notification means is caused to notify the certain analysis result.

[0128] (Appendix 7) the first area is a conference room; The at least one processor: The remote conference support system according to claim 1, wherein in the acquisition process, conference information regarding the conference room in which the remote conference is held is acquired, and the detection result is acquired based on the acquired conference information.

[0129] (Appendix 8) The at least one processor: The remote conference support system according to claim 1, wherein the analysis process identifies who is the subject of each action included in the analysis result.

[0130] (Appendix 9) A remote conference support device that supports a remote conference between a plurality of first participants in a first area and a second participant in a second area different from the first area, at least one processor, the at least one processor comprising: An acquisition process for acquiring detection information that detects a state within the first region; an analysis process for analyzing the actions of the first participants based on the detection information; A notification process of notifying a terminal used by the second participant of an analysis result of the analysis process; A remote meeting support device that performs the following:

[0131] (Appendix 10) A remote conference support method for supporting a remote conference between a plurality of first participants in a first area and a second participant in a second area different from the first area, the remote conference support method including: acquiring detection information that detects a state within the first area; analyzing the actions of the plurality of first participants based on the detection information; and notifying a terminal used by the second participant of the results of the analysis. [Explanation of symbols]

[0132] 1. Remote meeting support system 10...1st area 20a~e 1st participant 3...Method of acquisition 6...Means of analysis 9 ...Notification means 100a~c...Second area 110a~c...2nd participant 120a~c ···Terminal 150 ···Remote conference support device 160...Acquisition part 170...Analysis Department 180...Notification Department 300 ···Conference room 310 ···Whiteboard 320a~f...1st participant 330 ···Camera 340 ···Human Sensor 350 ···Motion Sensor 360···Door 370 Table

Claims

1. A remote conference support system that supports a remote conference between a plurality of first participants in a first area and a second participant in a second area different from the first area, an acquisition means for acquiring detection information obtained by detecting a state in the first area; an analysis means for analyzing actions of the plurality of first participants based on the detection information; a notification means for notifying a terminal used by the second participant of an analysis result of the analysis means; A remote meeting support system equipped with the above.

2. The remote conference support system according to claim 1 , wherein the analysis means analyzes reactions of the other first participants to an action of any one of the first participants.

3. 2. The remote conference support system of claim 1, wherein the analysis means performs at least one of identifying the target of gaze of each of the first participants, identifying the entry and exit of the first participants into the conference room, and identifying actions being performed in coordination by at least two of the first participants.

4. The remote conference support system according to claim 1 , wherein the detection information is information detected by one or more of a camera, a motion sensor, and a human presence sensor disposed in the first area.

5. The analysis means identifies a number of the first participants; The remote conference support system according to claim 1 , wherein the notification unit determines whether or not to notify the terminal depending on the number of participants identified by the analysis unit.

6. The analysis means repeatedly performs the analysis, a control means for determining whether a certain analysis result of the analysis means is different from a previous analysis result of the certain analysis result, 2. The remote conference support system according to claim 1, wherein the control means, when determining that the certain analysis result is different from the previous analysis result, causes the notification means to notify the certain analysis result.

7. the first area is a conference room; 2 . The remote conference support system according to claim 1 , wherein the acquisition means acquires conference information related to a conference room in which the remote conference is held, and acquires the detection information based on the acquired conference information.

8. The remote conference support system according to claim 1 , wherein the analysis means identifies a subject of each action included in the analysis result.

9. A remote conference support device that supports a remote conference between a plurality of first participants in a first area and a second participant in a second area different from the first area, an acquisition unit that acquires detection information obtained by detecting a state in the first area; an analysis unit that analyzes actions of the plurality of first participants based on the detection information; a notification unit that notifies a terminal used by the second participant of an analysis result of the analysis unit; A remote meeting support device equipped with the above.

10. A remote conference support method for supporting a remote conference between a plurality of first participants in a first area and a second participant in a second area different from the first area, the method comprising: Obtaining detection information on a state within the first area; Analyzing actions of the plurality of first participants based on the sensed information; notifying a result of the analysis to a terminal used by the second participant; A method for supporting a remote meeting, comprising:

Images