Communication system and communication control method

Abstract

A communication system includes terminals 5, a media server 2, and a signaling server 1. The signaling server 1: holds an inter-UE connection definition including the media type and the media direction transmitted and received among the terminals 5 according to a combination of terminal types; receives connection requests each including connection destination resource information from the terminals 5; and generates the definition of the media type and the media direction transmitted and received among the terminals on the basis of the UE type, the connection destination resource information, and the inter-UE connection definition. The signaling server 1 generates, on the basis of the definition of the media type and the media direction transmitted and received among the terminals, a control signal for setting a media connection path in the media server 2 and SDP for transmitting and receiving media between the media server 2 and each of the terminals 5. The signaling server 1 transmits the control signal to the media server 2, and transmits the SDP to the terminals 5. The media server 2 sets the media connection path on the basis of the control signal. The terminals 5 each set a session with the media server 2 on the basis of the SDP.

Description

Communication system and communication control method

[0008] ,

[0007] , ,

[0006] ,

[0001] The present disclosure relates to a communication system and a communication control method.

[0002] Online meeting services using Web Real-Time Communication (WebRTC) and the like have been diversely developed, and recently, due to the development of metaverse technology, the service form has been advanced. In online communication services including the metaverse, development is proceeding individually for each service, and it takes time and cost to build a media communication infrastructure such as voice communication and video communication.

[0003] In the base technology of session control found in conventional telephone services, in various terminals and service controls, by separating the sending and receiving, the combination of terminals and services was limited by dividing into sending-side control and receiving-side control (Patent Document 1).

[0004] Japanese Patent Application Laid-Open No. 9-93625

[0005] In Patent Document 1, by separating the sending and receiving and sharing the interface between the sending side and the receiving side, an increase in the control pattern due to an increase in the variations of the sending-side control and the receiving-side control was suppressed. However, in the conventional sending-receiving separation control, for example, in the receiving-side control, it is not possible to perform receiving-side control due to differences in the terminal conditions of the sending side.

[0006] In recent metaverse services, it is necessary to change the receiving-side control according to the terminal conditions of the sending side due to security and privacy requirements, and the use of sending-receiving separation is not possible. For example, although users communicate with each other using avatars and do not display each other's videos, when a user permits it, it is not possible to realize media control that permits videos for an analyzer and provides added value by video image analysis using AI.

[0007] The present disclosure has been made in view of the above, and an object thereof is to more easily realize media control according to the terminal type.

[0008] A communication system according to one aspect of the present disclosure is a communication system including a terminal, a media server, and a signaling server, wherein the signaling server maintains an inter-terminal connection definition including media type and media direction to be transmitted and received between terminals, depending on a combination of terminal types, receives a connection request including destination information from the terminal, generates inter-terminal media information including media type and media direction to be transmitted and received between terminals based on the terminal type, destination information, and the inter-terminal connection definition, generates a control signal for setting a media connection path in the media server and session information for sending and receiving media between the terminal and the media server based on the inter-terminal media information, transmits the control signal to the media server, transmits the session information to the terminal, the media server sets a media connection path based on the control signal, and the terminal sets a session with the media server based on the session information.

[0009] According to this disclosure, media control according to the terminal type can be implemented more easily.

[0010] Figure 1 shows an example of the signaling server configuration. Figure 2 shows an example of the UE type definition. Figure 3 shows an example of the inter-UE connection definition. Figure 4 is a sequence diagram showing an example of the processing flow when a general UE starts media communication. Figure 5 is a sequence diagram showing an example of the processing flow when a system UE connects. Figure 6 is a sequence diagram showing an example of the processing flow of the signaling server. Figure 7 is a sequence diagram showing an example of the processing flow when a general UE starts media communication. Figure 8 shows an example of how various terminals communicate via the media server. Figure 9 shows an example of setting the media connection path in the media server. Figure 10 shows an example of the UE type definition. Figure 11 shows an example of the inter-UE connection definition. Figure 12 shows an example of the signaling server hardware configuration.

[0011] [Media Server Configuration] The embodiments of this disclosure will be described below with reference to the drawings.

[0012] Referring to Figure 1, an example of the configuration of the signaling server 1 in this embodiment will be described. The signaling server 1 exchanges information necessary when establishing communication between terminals (also called User Equipment (UE)). The signaling server 1 shown in the figure comprises a signaling control unit 11, a media server control unit 12, a media control generation unit 13, and a storage unit 14.

[0013] The signaling control unit 11 receives connection requests containing destination resource information from general UEs 5A, 5B and system UE 5C (hereinafter referred to as terminal 5). For example, when general UE 5A participates in a remote meeting, general UE 5A sends a connection request to the signaling server 1 that includes the conference room number of the remote meeting as the destination resource information. When general UE 5A and general UE 5B communicate, general UE 5A may send a connection request to the signaling server 1 that includes general UE 5B's information as the destination resource information.

[0014] The signaling control unit 11 transmits destination resource information and requesting UE type to the media control generation unit 13, and receives destination resource information from the media control generation unit 13, which includes information on media type and media direction transmitted and received between terminals 5. Media types include video, audio, and data, and media direction includes transmission, reception, or transmission and reception. For example, the destination resource information includes information such as audio and data being transmitted and received bidirectionally between general UE 5A and general UE 5B, and video, audio, and data being transmitted from general UEs 5A and 5B to system UE 5C.

[0015] The signaling control unit 11 generates a Session Description Protocol (SDP) for setting up a media communication session based on the connection destination resource information updated by the media control generation unit 13, and sends it to the terminal 5.

[0016] The media server control unit 12 generates control signals (also called media server control commands) to control the media server 2 based on the connection destination resource information updated by the media control generation unit 13, and sends them to the media server 2. The media server 2 delivers media such as video, audio, and data transmitted from terminal 5 to the destination terminal 5 according to the control signals.

[0017] The media control generation unit 13 refers to the UE type definition and inter-UE connection definition stored in the storage unit 14 and generates inter-terminal media information including the media type and media direction to be transmitted and received between terminals. Based on the inter-terminal media information, the media control generation unit 13 updates the information on the media type and media direction between UEs in the destination resource information.

[0018] Figures 2 and 3 show examples of UE type definitions and UE connection definitions stored in the memory unit 14.

[0019] The UE type definitions shown in Figure 2 define UE types UE A (General), UE B (Analyzer), and UE C (Pet), and each UE type includes items for attributes, connection timing, and connection limit. Note that the items included in the UE types shown here are examples and are not limited to these. UE A is a terminal used by a user. UE B and UE C are applications and services provided by service providers. For example, UE B receives video from UE A and provides added value through video analysis, or receives audio and data from UE A and generates meeting minutes. UE C is an AI Bot. The attribute indicates whether it is a UE type used by a user or a UE type provided by a service provider. Based on the attribute, the authentication method and authentication logic when connecting to the UE are determined. If the attribute is a system, the authentication method and authentication logic may be simplified. The connection timing indicates when a terminal of the UE type can connect. For example, if the connection timing is later, other terminals must be connected to media server 2. The connection limit indicates the number of terminals that can connect simultaneously. For example, if the connection limit is 500, 500 devices can connect to the same meeting.

[0020] In the UE connection definition shown in Figure 3, information about the type and direction of media transmitted and received between terminals is defined according to the combination of terminal types. For example, in the example in Figure 3, it is defined that voice and data are transmitted and received bidirectionally between UE A and UE A, voice, video, and data are transmitted from UE A to UE B between UE A and UE B, and voice and data are transmitted from UE C to UE A between UE A and UE C.

[0021] UE type definitions and UE connection definitions are registered in advance from the Web server 4 via the API server 3.

[0022] [Overall System Operation] Referring to the sequence diagram in Figure 4, an example of the processing flow when a general UE5A starts media communication will be explained.

[0023] In step S11, the UE type definition and inter-UE connection definition are registered in the signaling server 1 via the API server 3 from the web server 4.

[0024] In step S12, the general UE 5A sends a connection request to the signaling server 1 that includes its UE type and destination resource information. For example, the destination resource information specifies the conference room number that the general UE 5A wishes to join. Communication takes place between terminals 5 that have connected by specifying the same conference room number. In the case of one-to-one communication, the destination resource information may specify either the general UE 5B or the system UE 5C.

[0025] In step S13, the signaling server 1 refers to the inter-UE connection definition and updates the media type and media direction information between UEs in the destination resource information. For example, the signaling server 1 updates the destination resource information so that audio and data can be sent and received bidirectionally between general UE 5A and other general UEs participating in the conference (UE A in Figure 3). Based on the updated destination resource information, the signaling server 1 generates a control signal to set general UE 5A as the media transmission and reception destination in media server 2, and an SDP to set up a session between general UE 5A and media server 2. Details of the processing in step S13 will be described later.

[0026] In step S14, the signaling server 1 sends a control signal to the media server 2. Based on the control signal, the media server 2 sets the destination for sending and receiving media.

[0027] In step S15, the signaling server 1 sends the SDP to the general UE 5A.

[0028] In step S16, the general UE 5A generates a peer connection with the media server 2 based on the SDP.

[0029] In step S17, media communication becomes possible between the general UE 5A and the media server 2. From this point onward, audio and data transmitted from the general UE 5A are distributed to other general UEs via the media server 2, and audio and data transmitted by other general UEs are distributed to the general UE 5A via the media server 2.

[0030] Next, referring to the sequence diagram in Figure 5, we will explain an example of the processing flow when System UE5C connects to a meeting in which General UE5A is participating. Assume that the UE type of System UE5C is UE B (Analyzer) as shown in Figures 2 and 3. Also, assume that, as a result of the processing in Figure 4, a media connection has been established between General UE5A and Media Server 2 in step S17.

[0031] In step S21, system UE5C sends a connection request to signaling server 1 that includes its own UE type (UE B) and destination resource information. For example, the destination resource information specifies the conference room number in which general UE5A is participating.

[0032] In step S22, the signaling server 1 refers to the inter-UE connection definition and updates the media type and media direction information between UEs in the destination resource information. Specifically, the signaling server 1 updates the destination resource information so that system UE 5C receives audio, video, and data from general UE 5A. Based on the updated destination resource information, the signaling server 1 generates a control signal to add system UE 5C as a media transmission / reception destination in media server 2, an SDP to connect system UE 5C and media server 2, and an SDP to add media transmission to system UE 5C to the session between general UE 5A and media server 2.

[0033] In step S23, the signaling server 1 sends a control signal to the media server 2. Based on the control signal, the media server 2 adds system UE5C as a destination for sending and receiving media.

[0034] In step S24, the signaling server 1 sends the SDP to the system UE5C.

[0035] In step S25, system UE5C generates a peer connection with media server 2 based on SDP.

[0036] In step S26, media communication becomes possible between system UE5C and media server 2.

[0037] In step S27, the signaling server 1 sends the SDP to the general UE 5A.

[0038] In step S28, the general UE5A updates the peer connection with the media server 2 based on the SDP.

[0039] In step S29, the general UE5A becomes capable of media communication with the system UE5C via the media server 2.

[0040] [Signaling Server Operation] Referring to the sequence diagram in Figure 6, an example of the processing flow of the signaling server 1 will be explained.

[0041] In step S12, the signaling control unit 11 receives a connection request including the UE type and connection destination resource information from the general UE 5A.

[0042] In step S131, the signaling control unit 11 acquires the UE type and connection destination resource information from the connection request. Note that, as in another embodiment described later, the signaling control unit 11 may acquire the UE type from an external server such as the web server 4.

[0043] In step S132, the signaling control unit 11 transmits the UE type and connection destination resource information to the media control generation unit 13.

[0044] In step S133, the media control generation unit 13 acquires the UE type definition corresponding to the UE type and the UE indirect connection definition corresponding to the connection destination resource information from the storage unit 14.

[0045] In step S134, the media control generation unit 13 updates the media type and media direction information in the connection destination resource information based on the UE type definition and the UE indirect connection definition. For example, when the storage unit 14 holds the UE indirect connection definition shown in FIG. 3 and a connection request from a terminal of UE type UE A (general) to a remote conference is received, the media control generation unit 13 updates the media type in the connection destination resource information to audio and data and updates the media direction to two-way. When UE A (general) is already connected to the media server 2 and a connection request is received from a terminal of UE type UE B (analyzer), the media control generation unit 13 updates the media type in the connection destination resource information to audio, video, and data and updates the media direction from UE A to UE B.

[0046] In step S135, the media control generation unit 13 transmits the updated connection destination resource information to the media server control unit 12.

[0047] In step S136, the media server control unit 12 generates a control signal for setting the general UE 5A as the media transmission / reception destination in the media server 2 based on the connection destination resource information.

[0048] In step S14, the media server control unit 12 transmits a control signal to the media server 2.

[0049] In step S137, the media control generation unit 13 transmits the updated connection destination resource information to the signaling control unit 11.

[0050] In step S138, the signaling control unit 11 generates an SDP for setting a session between the general UE 5A and the media server 2 based on the connection destination resource information.

[0051] In step S15, the signaling control unit 11 transmits the SDP to the general UE 5A.

[0052] [Another Example of Terminal Connection Processing] Referring to the sequence diagram of FIG. 7, an example of the processing flow when the general UE 5A starts media communication will be described. This is different from the processing of FIG. 4 in that the UE type is not included in the connection request.

[0053] In step S11, in advance, the UE type definition and the UE connection definition are registered in the signaling server 1 from the Web server 4 via the API server 3. It is assumed that the UE type of the general UE 5A is associated with the user ID and registered in the Web server 4. [[ID=十六]] [[ID=十七]]

[0054] In step S31, the general UE 5A transmits a connection request including the source user ID and the connection destination resource information to the signaling server 1.

[0055] In step S32, the signaling server 1 acquires the source user ID and the connection destination resource information from the connection request, and inquires the Web server 4 about the source user ID and the connection destination resource information. [[ID=2三十]]

[0056] In step S33, the Web server 4 returns the UE type of the source user ID to the signaling server 1.

[0057] The processing after step S13 is the same as the processing of FIG. 4, so the description here is omitted.

[0058] [Example of Media Connection Path Configuration] Next, we will explain an example of configuring the media connection path in media server 2. The media connection path is a path that indicates the direction of media transfer within media server 2.

[0059] Here, we will describe a configuration where two UEs, UE A (general) and UE D (general), are connected to UE B (analyzer) and UE C (pet), as shown in Figure 8. It is assumed that the signaling server 1 holds the inter-UE connection definition shown in Figure 3.

[0060] Audio and data are transmitted bidirectionally between the two UEs A and D. Audio, video, and data are transmitted from UEs A and D to UE B. Audio and data are transmitted from UE C to UEs A and D. In Figure 8, media communication between UE A and UE D is shown with solid arrows, media communication between UEs A and UE D and UE B is shown with dashed arrows, and media communication between UEs A and UE D and UE C is shown with a dotted line.

[0061] Figure 9 shows an example of media connection path settings in media server 2 to realize the media communication shown in Figure 8. A media connection path, shown by a solid line, is set up between UE A and UE D so that audio and data are transmitted and received. A media connection path, shown by a dashed line, is set up so that audio, video, and data are transmitted from UE A and UE D to UE B. A media connection path, shown by a dotted line, is set up so that audio and data are transmitted from UE C to UE A and UE D.

[0062] In this embodiment, SDP control and media server control based on connection definitions are patterned, and the patterned processing is reused according to the connected terminal, thereby reducing development costs even when connecting various terminals such as AI. For example, when connecting UE C to a state where UE A, UE D, and UE B are connected, the media connection path shown by the dashed line in Figure 9 should be set according to the inter-UE connection definition. More specifically, the control signal to media server 2 includes commands to add voice reception and data reception for UE A and UE D, and commands to add voice transmission and data transmission for UE C, and includes commands to set a media connection path connecting voice reception of UE A and UE D to voice transmission of UE C, and data reception of UE A and UE D to data transmission of UE C. The SDP to UE A and UE D is set to receive voice and data, and the SDP to UE C is set to transmit voice and data. The signaling server 1 can generate control signals and SDP according to the pattern as described above based on the inter-UE connection definition.

[0063] [Variation] The combination of UE type definition and inter-UE connection definition is managed as an Apptype for each application use case. When the signaling server 1 receives a connection request, it retrieves the UE type definition and inter-UE connection definition corresponding to the Apptype and updates the destination resource information.

[0064] For example, signaling server 1 manages the UE type definitions and inter-UE connection definitions shown in Figures 2 and 3 as Apptype: NvsN, and manages the UE type definitions and inter-UE connection definitions shown in Figures 10 and 11 as Apptype: 1vs1.

[0065] When the signaling server 1 receives a connection request from the terminal 5, it obtains a UE type definition and an inter-UE connection definition corresponding to the Apptype. The connection request may include information indicating the Apptype, or the signaling server 1 may query the web server 4 for the Apptype. For example, the signaling server 1 obtains the source user ID and destination resource information from the connection request and queries the web server 4 for the source user ID and destination resource information. The web server 4 returns the UE type corresponding to the source user ID and the Apptype corresponding to the destination resource information to the signaling server 1.

[0066] As described above, the communication system of this embodiment includes a terminal 5, a media server 2, and a signaling server 1. The signaling server 1 maintains an inter-UE connection definition that includes the media type and media direction to be transmitted and received between terminals 5, depending on the combination of terminal types. The signaling server 1 receives a connection request from terminal 5 that includes destination resource information, and generates inter-terminal media information that includes the media type and media direction to be transmitted and received between terminals, based on the UE type, destination resource information, and inter-UE connection definition. Based on the inter-terminal media information, the signaling server 1 generates a control signal to set the media connection path in the media server 2 and an SDP for sending and receiving media between terminal 5 and the media server 2. The signaling server 1 sends the control signal to the media server 2 and the SDP to terminal 5. The media server 2 sets the media connection path based on the control signal. Terminal 5 sets up a session with the media server 2 based on the SDP. As a result, when adding a terminal of a new UE type, the signaling server 1 can easily establish media communication with the new UE type terminal by adding the new UE type to the UE type definition and the inter-UE connection definition. This is done by generating the control signals for the media server 2 and the SDP for the session between terminal 5 and media server 2 based on the defined pattern.

[0067] The signaling server 1 described above can be a general-purpose computer system, such as the one shown in Figure 12, which includes a central processing unit (CPU) 901, memory 902, storage 903, communication device 904, input device 905, and output device 906. In this computer system, the signaling server 1 is realized when the CPU 901 executes a predetermined program loaded onto the memory 902. This program can be recorded on a computer-readable non-temporary recording medium such as a magnetic disk, optical disk, or semiconductor memory, or it can be distributed via a network.

[0068] 1 Signaling Server 11 Signaling Control Unit 12 Media Server Control Unit 13 Media Control Generation Unit 14 Storage Unit 2 Media Server 3 API Server 4 Web Server 5A, 5B General UE 5C System UE

Claims

1. A communication system comprising a terminal, a media server, and a signaling server, wherein the signaling server maintains an inter-terminal connection definition that includes the type of media to be transmitted and received between terminals and the media direction, depending on the combination of terminal types; receives a connection request from the terminal that includes destination information; generates inter-terminal media information that includes the type of media to be transmitted and received between terminals and the media direction, based on the terminal type, the destination information, and the inter-terminal connection definition; generates a control signal for setting a media connection path in the media server and session information for sending and receiving media between the terminal and the media server based on the inter-terminal media information; transmits the control signal to the media server; transmits the session information to the terminal; the media server sets a media connection path based on the control signal; and the terminal sets a session with the media server based on the session information.

2. A communication system according to claim 1, wherein the signaling server maintains a terminal type definition including attributes for each terminal type, and determines the authentication method for the terminal according to the attributes.

3. A communication system according to claim 1, wherein the signaling server maintains the inter-terminal connection definition for each application used, and switches the inter-terminal connection definition referenced according to the application.

4. A communication control method by a signaling server, comprising: maintaining an inter-terminal connection definition including the type of media to be transmitted and received between terminals and the media direction, according to the terminal type; receiving a connection request including destination information from a terminal; generating a definition of the type of media to be transmitted and received between terminals and the media direction based on the terminal type, the destination information, and the inter-terminal connection definition; generating a control signal for setting a media connection path in a media server and session information for transmitting and receiving media between the terminal and the media server based on the definition of the type of media to be transmitted and received between terminals and the media direction; transmitting the control signal to the media server; and transmitting the session information to the terminal.

Images