Information processing device, association method, and program
The information processing device associates media data with communication channels, addressing the challenge of switching between channels based on network quality, thereby enhancing media communication efficiency and compatibility with existing technologies.
Patent Information
- Application Number
- PCT/JP2023/017615
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-10
- Publication Date
- 2024-11-14
Smart Images

Figure JP2023017615_14112024_PF_FP_ABST
Abstract
Description
Information processing device, association method, and program
[0001] The present invention relates to a technique for establishing a communication path and communicating media data (media communication).
[0002] Various conventional techniques for establishing a communication path have been proposed. For example, Non-Patent Document 1 discloses a technique for establishing a communication path between a sending communication device and a receiving communication device by signaling via a signaling server, and performing P2P media communication.
[0003] In the technology disclosed in Non-Patent Document 1, signaling is required to exchange information such as port numbers between terminals before media communication begins, which means that it takes time before media communication can begin.
[0004] WebRTC (RFC8825) https: / / www.rfc-editor.org / rfc / rfc8825.pdf
[0005] In media communications, there is a demand for switching between multiple established communication paths in accordance with the characteristics (quality and connectivity) of the communication network. Note that the communication path in this specification refers to a logical communication path from the application's perspective, such as a TCP connection.
[0006] To achieve this switching, it is necessary to group multiple communication paths and associate them with specific media data. However, conventional technologies do not have a mechanism for grouping multiple communication paths and associating them with specific media data.
[0007] The present invention has been made in view of the above points, and has an object to provide a technique that enables a communication path to be associated with specific media data in media communication.
[0008] According to the disclosed technology, there is provided an information processing device capable of communicating with each of a first communication device and a second communication device, the information processing device comprising: an association resolution unit that, after a communication path for transmitting specific media data is established between the first communication device and the second communication device, associates the specific media data with the communication path based on information received from each of the first communication device and the second communication device; and an association database that stores association information indicating the association between the specific media data and the communication path.
[0009] According to the disclosed technology, it is possible to associate a communication path with specific media data.
[0010] FIG. 1 is a diagram showing an example of a schematic configuration for performing association. FIG. 2 is a diagram showing an example of the overall configuration of a communication system. FIG. 3 is a diagram showing an example of the functional configuration of each device constituting the communication system. FIG. 4 is a sequence diagram for explaining the operation of the communication system. FIG. 5 is a sequence diagram for explaining the operation of the communication system. FIG. 6 is a diagram showing another example of the configuration of a communication system. FIG. 7 is a diagram showing another example of the configuration of a communication system. FIG. 8 is a diagram showing an example of the hardware configuration of a device.
[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.
[0012] As described above, there is a demand for switching between multiple established communication paths in accordance with the characteristics (quality and connectivity) of a communication network. For example, it is conceivable to seamlessly switch the communication path for transmitting media data between "communication path 1 established using a normal TCP / IP protocol stack" and "communication path 2 established using a protocol stack specific to IOWN (registered trademark)."
[0013] By realizing the above-mentioned switching, for example, telecommunications carriers will be able to provide users with high-quality, high-function, high-value-added services that utilize new communications technologies while maintaining compatibility with existing technologies and services.
[0014] To achieve the above switching, it is necessary to group a plurality of logically established communication paths and associate them with a specific medium. In this embodiment, a technique for this association will be described.
[0015] An example of a schematic configuration for performing the association is shown in Fig. 1. In the example shown in Fig. 1, communication path 1 and communication path 2 are established between communication unit 120 of communication device 100 on the calling side and communication unit 220 of communication device 200 on the receiving side, and the two communication paths are associated with the same media.
[0016] Communication path 1 is a communication path established using, for example, a normal TCP / IP protocol stack, and communication path 2 is a communication path established using, for example, a protocol stack specific to IOWN (registered trademark). Note that the communication paths may also be called "connections."
[0017] Using the above association, it is possible to seamlessly switch the communication path over which specific media data is transmitted between communication path 1 and communication path 2. For example, media processing unit 230 of receiving communication device 200 can process media data in the same manner regardless of the communication path over which the media data has been transmitted.
[0018] When two communication devices transmit and receive media data using multiple communication paths, it is necessary to determine which media data is associated with each of these communication paths, and by determining this, it is possible to seamlessly switch between the multiple communication paths. For example, if two communication paths are established between two communication devices, an application that transmits and receives audio media can determine that the two communication paths are associated with the same audio media, and can switch between the two communication paths.
[0019] In this embodiment, as shown in A in Fig. 1, media data is associated with a communication path by exchanging endpoint information between a sender and a receiver using an out-channel connection separate from the connection through which media data is exchanged. The configurations of the communication system and devices for this association, as well as the operation of the devices, will be described in detail below.
[0020] (System Configuration Example) An example of the overall configuration of a communication system according to this embodiment is shown in Fig. 2. As shown in Fig. 2, this communication system includes a calling communication device 100, a receiving communication device 200, and an association server 300.
[0021] Communication is possible between the originating communication device 100 and the recipient communication device 200 via a communication network. Similarly, communication is also possible between the originating communication device 100 and the association server 300, and between the recipient communication device 200 and the association server 300 via a communication network. These communication networks are, for example, IP networks.
[0022] Both the communication device of the originating party 100 and the communication device of the recipient 200 may be a single physical device or may consist of multiple devices. In addition, both the communication device of the originating party 100 and the communication device of the recipient 200 may be virtual machines on the cloud.
[0023] Similarly, the association server 300 may be a single physical device or may consist of multiple devices. The association server 300 may also be a virtual machine on a cloud. The association server 300 may also be called an information processing device.
[0024] For ease of explanation, the sender (originator) of a connection request is assumed to be originator communication device 100 , and the receiver of the connection request is assumed to be receiver communication device 200 .
[0025] (Device Configuration Example) Fig. 3 shows an example of the functional configuration of each device constituting the communication system. As shown in Fig. 3, the originating communication device 100 has an association management unit 110, a communication unit 120, and a media processing unit 130. The receiving communication device 200 has an association management unit 210, a communication unit 220, and a media processing unit 230. Furthermore, the association server 300 has an association resolution unit 310 and an association database 320. The association database may also be called an association storage unit.
[0026] The association management unit 110 in the originating communication device 100 manages the association between media data and communication paths (specifically, ports, etc.) by transmitting and receiving signals (signal data) to and from the association server 300. The association management unit 110 includes a storage unit that stores association information.
[0027] Similarly, the association management unit 210 in the receiver communication device 200 manages the association between media data and communication paths (specifically, ports, etc.) by transmitting and receiving signals (signal data) to and from the association server 300. The association management unit 210 includes a storage unit that stores association information.
[0028] The communication unit 120 in the communication device of the originator 100 manages ports and transmits and receives data. Similarly, the communication unit 220 in the communication device of the recipient 200 manages ports and transmits and receives data.
[0029] The media processing unit 130 in the communication device 100 of the originating side processes media data and exchanges the media data with the communication unit 120. The media processing unit 230 in the communication device 200 of the receiving side processes the media data in the same manner.
[0030] The association resolution unit 310 of the association server 300 resolves the association by executing the procedure described below, and stores the association information in the association database 310. The association database 310 stores the association information. The association information is, for example, the information shown below: - media x: - pair #1 - host xxx.xxx.xxx.xxx=port a - host yyy.yyy.yyy.yyy=port b hash: abcdef - pair #2 - host xxx.xxx.xxx.xxx=port c - host yyy.yyy.yyy.yyy=port d hash: ghijkl The above information indicates that bear #1 and pair #2 are associated with media x. Pair #1 is a connection (communication path) connecting "host xxx.xxx.xxx.xxx=port a" and "host yyy.yyy.yyy.yyy=port b", and pair #2 is a connection (communication path) connecting "host xxx.xxx.xxx.xxx=port c" and "host yyy.yyy.yyy.yyy=port d". Also, as described above, for example, a hash value is included in the association information.
[0031] The above association information may also be written as "media x((pair #1(host xxx.xxx.xxx.xxx=port a, host yyy.yyy.yyy.yyy=port b, hash: abcdef)), (pair #2(host xxx.xxx.xxx.xxx=port c, host yyy.yyy.yyy.yyy=port d, hash: ghijkl)))."
[0032] When the association server 300 manages the association by storing the above association information, the association management unit 110 of the calling communication device 100 and the association management unit 210 of the receiving communication device 200 each store the above association information.
[0033] In a communication system having the above configuration, only media data is transmitted and received over the established connection, and information regarding association is transmitted and received over the out-channel. In this embodiment, the association procedure is executed after the connection is established. This allows communication to start without waiting for the completion of signaling. Therefore, the effects of technologies such as 0-RTT (technology for establishing communication with zero round-trip communication) are not diminished.
[0034] (Sequence) Next, an example of the operation of the communication system according to this embodiment will be described with reference to the sequences of Figures 4 and 5. Note that the step numbers (S101, etc.) given in Figures 4 and 5 do not necessarily match the order of processing.
[0035] <New / Additional Connection> FIG. 4 shows a sequence for associating a connection with media data when a new connection is established and then an additional connection is established.
[0036] In S101, communication unit 120 of communication device 100 of the originating party transmits a connection request for the first connection. In S102, communication unit 220 of communication device 200 of the recipient receives the connection request and returns a connection response. This establishes the first connection, and in S103, communication of media data using the connection is started between communication device 100 of the originating party and communication device 200 of the recipient.
[0037] Next, the association phase is executed. In the following description, it is assumed that each of the communication device of the originating party (100) and the communication device of the receiving party (200) performs address resolution of the association server (300). However, if the address of the association server (300) is known in advance, address resolution does not have to be performed.
[0038] In S104, the association management unit 110 of the originating communication device 100 performs address resolution of the association server 300. In S105, the association management unit 110 transmits information for association regarding the media communication in the first connection to the association server 300. The information for association includes "information such as the IP addresses of the destination and source, the first several hundred bytes of information sent to the receiving side (or a hash value generated based on these), originating side port ID #1, and media ID #1." The first several hundred bytes of information may be, for example, a fragment of a video or part of an HTTP request.
[0039] The above hash value is, for example, a hash value of "destination and source IP address information and the first X bytes of information sent to the recipient." Alternatively, the above hash value may be, for example, a hash value of "the first X bytes of information sent to the recipient."
[0040] Similarly, in S106 and S107, the association management unit 210 of the receiving communication device 200 performs address resolution for the association server 300 and transmits information for association regarding the media communication in the first connection to the association server 300. The information for association includes "information such as the IP addresses of the destination and source, information for the first several hundred bytes received from the sending side (or a hash value generated based on this), and receiving side port ID #1."
[0041] The hash value is, for example, a hash value of "IP address information of the destination and source and the first X bytes of information received from the sender." Alternatively, the hash value may be, for example, a hash value of "the first X bytes of information received from the sender."
[0042] In S108, the association resolution unit 310 of the association server 300 resolves the association and assigns pair ID #1, and stores the association information in the association database 320.
[0043] For example, let us assume that the association resolution unit 310 determines, based on the information received from the calling communication device 100 and the information received from the receiving communication device 200, that the calling side information corresponding to the media data of media ID #1 is "IP address = xxx.xxx.xxx.xxx, port number = calling side port ID #1," and that the receiving side information corresponding to the media data of media ID #1 is "IP address = yyy.yyy.yyy.yyy, port number = receiving side port ID #1." Furthermore, let us assume that the association resolution unit 310 receives a hash value = abcdef from each of the calling communication device 100 and the receiving communication device 200.
[0044] In this case, the association resolution unit 310 stores, for example, "media ID#1 (pair ID#1 (host xxx.xxx.xxx.xxx = calling port ID#1, host yyy.yyy.yyy.yyy = receiving port ID#1, hash: abcdef))" as association information in the association database 320. The above association information indicates that media ID #1 is associated with pair #1, and the information of pair #1 is "host xxx.xxx.xxx.xxx = calling port ID#1, host yyy.yyy.yyy.yyy = receiving port ID#1, hash: abcdef."
[0045] Regarding the above association, the association resolution unit 310 uses the hash value (or the first few hundred bytes of information) to confirm the originating and receiving processes associated with the established connection. For example, suppose two processes (A1, A2) exist in the originating communication device 100, and each establishes a connection with the same receiving communication device 200. Similarly, suppose the receiving communication device 200, which is the receiving side of the established connection, has two processes (B1, B2) that terminate each connection. In this case, it is unclear to which process, B1 or B2, the connection established by A1 is associated. Therefore, the association resolution unit 310 compares the hash value received from the originating communication device 100 (information on the initially sent data) with the hash value received from the receiving communication device 200 (information on the initially received data). If they match, it can be confirmed that the connection established by A1 is associated with the port information (i.e., the process) received from the receiving communication device 200.
[0046] The association resolution unit 310 of the association server 300 transmits the association information to the communication device of the originator 100 in S109, and also transmits the association information to the communication device of the recipient 200 in S110.
[0047] The association information sent from the association resolution unit 310 to the calling communication device 100 and the association information sent to the receiving communication device 200 may be the same as the association information held by the association server 300, or may be information obtained by excluding information known to the calling communication device 100 / receiving communication device 200 (e.g., IP address) from the association information held by the association server 300.
[0048] For example, the association information sent to the calling communication device 100 and the association information sent to the receiving communication device 200 may be the same information, which may be "calling port ID #1, receiving port ID #1, pair ID #1, media ID #1."
[0049] In S111, the association management unit 110 of the originating communication device 100 stores the association information. Similarly, in S112, the association management unit 210 of the receiving communication device 200 stores the association information. For the target connection, the association information stored by the association management unit 110 / association management unit 210 is the same as the association information held by the association server 300.
[0050] With the first connection (pare ID#1) established, in S113 and S114, a connection request / connection response is sent and received to establish a second connection for streaming media data of media ID#1, and the second connection is established.
[0051] In S115, the association phase is executed for the second connection using the same processing as the association phase for the first connection.
[0052] For example, suppose the IP addresses are the same for the first and second connections, the port numbers for the second connection are sender port ID#2 and receiver port ID#2, and the pair ID for the second connection is pare ID#2.
[0053] At this time, the association information stored in each of the calling communication device 100, the receiving communication device 200, and the association server 300 by S115 is, for example, "media ID#1((pair ID#1(host xxx.xxx.xxx.xxx=calling port ID#1, host yyy.yyy.yyy.yyy=receiving port ID#1, hash: abcdef)), (pair ID#2(host xxx.xxx.xxx.xxx=calling port ID#2, host yyy.yyy.yyy.yyy=receiving port ID#2, hash: ghijkl)))."
[0054] The above information indicates that media ID#1 is associated with a connection with pair ID#1 and a connection with pair ID#2.
[0055] 5 is a sequence diagram showing how to change the association when media data having media ID#1 is changed to media data having media ID#2 while being communicated over a connection having pare ID#1. Pare ID#1 and media ID#1 are associated with each other, and for example, the aforementioned association information “media ID#1 (pair ID#1 (host xxx.xxx.xxx.xxx=calling-side port ID#1, host yyy.yyy.yyy.yyy=receiving-side port ID#1, hash: abcdef))” is stored in each of calling communication device 100, receiving communication device 200, and association server 300.
[0056] In S201 of FIG. 5, media data of media ID#1 is being communicated between the communication device of the originating side 100 and the communication device of the receiving side 200 via the connection of pare ID#1.
[0057] In S202, the communication unit 120 of the originating communication device 100 transmits an association change request to the association server 300. The association change request is a request to change the media data associated with pair ID #1 from media ID #1 to media ID #2.
[0058] In S203, the association server 300 transfers the association change request received from the communication device of the originator 100 to the communication device of the recipient 200.
[0059] In the recipient communication device 200 that has received the association change request, the association management unit 210 changes the media data associated with pair ID #1 from media ID #1 to media ID #2 in S204. Specifically, the association management unit 210 changes the current association information stored in the storage unit from "media ID#1 (pair ID#1 (host xxx.xxx.xxx.xxx = calling port ID#1, host yyy.yyy.yyy.yyy = receiving port ID#1, hash: abcdef))" to "media ID#2 (pair ID#1 (host xxx.xxx.xxx.xxx = calling port ID#1, host yyy.yyy.yyy.yyy = receiving port ID#1, hash: abcdef))."
[0060] In S205, the communication device of the recipient 200 returns an association change response to the association server 300. The association change response includes pair ID#1, which is the target of the association change.
[0061] In S206, the association resolution unit 310 of the association server 300 that has received the association change response changes the media data associated with pair ID #1 from media ID #1 to media ID #2. Specifically, the association resolution unit 310 changes the current association information stored in the association database 310, "media ID#1 (pair ID#1 (host xxx.xxx.xxx.xxx = calling port ID#1, host yyy.yyy.yyy.yyy = receiving port ID#1, hash: abcdef))" to "media ID#2 (pair ID#1 (host xxx.xxx.xxx.xxx = calling port ID#1, host yyy.yyy.yyy.yyy = receiving port ID#1, hash: abcdef))."
[0062] In S207, the association server 300 returns an association change response to the transmitter communication device 100. The association change response includes pair ID#1, which is the target of the association change.
[0063] Upon receiving the association change response, association management unit 110 of originating communication device 100 changes the media data associated with pair ID #1 from media ID #1 to media ID #2. Specifically, association management unit 110 changes the current association information stored in the storage unit from "media ID#1 (pair ID#1 (host xxx.xxx.xxx.xxx = originating port ID#1, host yyy.yyy.yyy.yyy = receiving port ID#1, hash: abcdef))" to "media ID#2 (pair ID#1 (host xxx.xxx.xxx.xxx = originating port ID#1, host yyy.yyy.yyy.yyy = receiving port ID#1, hash: abcdef))."
[0064] In S209, with media ID #2 associated with pair ID #1, communication for media ID #2 is performed over the connection for pair ID #1.
[0065] If a connection for pair ID#2 is established in addition to the connection for pair ID#1, the media data associated with pair ID#2 can be changed from media ID#1 to media ID#2 using a process similar to the association change phase described above.
[0066] (Other Configuration Examples) The functional arrangement in the communication system according to this embodiment is not limited to that shown in FIG.
[0067] For example, as shown in Fig. 6, the functions of the association server 300 may be provided in the communication device of the recipient 200. Also, as shown in Fig. 7, the functions of the association server 300 may be provided in the communication device of the sender 100.
[0068] 6 and 7, the processing procedure for association is the same as that described above. However, if the function of association server 300 is located within transmitter communication device 100, communication between association server 300 and transmitter communication device 100 is communication within transmitter communication device 100. The same applies when the function of association server 300 is located within receiver communication device 200.
[0069] (Hardware Configuration Example) Any of the devices described in this embodiment (the originating communication device 100, the receiving communication device 200, and the association server device 100) can be realized, for example, by causing a computer to execute a program. This computer may be a physical computer or a virtual machine on the cloud. Hereinafter, these devices will be collectively referred to as information processing devices.
[0070] That is, the information processing device can be realized by executing a program corresponding to the processing performed by the information processing device using hardware resources such as a CPU and memory built into the computer. The program can be recorded on a computer-readable recording medium (such as a portable memory) and stored or distributed. The program can also be provided via a network such as the Internet or email.
[0071] Fig. 8 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 8 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, and the like, all of which are interconnected by a bus BS. The computer may further include a GPU.
[0072] The program that realizes the processing on the computer is provided by a recording medium 1001, such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.
[0073] The memory device 1003 reads and stores a program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes functions related to the information processing device in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) or the like according to the program. The input device 1007 is composed of a keyboard, mouse, buttons, a touch panel, etc., and is used to input various operation instructions. The output device 1008 outputs the results of calculations.
[0074] (Effects of the embodiment) As described above, the technology described in the embodiment makes it possible to group a plurality of communication paths and associate them with a specific medium.
[0075] The following additional notes are provided regarding the above-described embodiments.
[0076] <Additional Notes> (Additional Item 1) An information processing device capable of communicating with each of a first communication device and a second communication device, comprising: a memory; and at least one processor connected to the memory, wherein after a communication path for transmitting specific media data between the first communication device and the second communication device is established, the processor associates the specific media data with the communication path based on information received from each of the first communication device and the second communication device, and the memory stores association information indicating the association between the specific media data and the communication path. (Additional Item 2) The information processing device according to Additional Item 1, wherein the processor transmits information including a pair ID identifying the communication path and a media data ID identifying the specific media data to each of the first communication device and the second communication device. (Additional Item 3) The information processing device according to Additional Item 1 or 2, wherein the processor receives, from each of the first communication device and the second communication device, address information related to the communication path, port information related to the communication path, and information related to the specific media data, and generates the association information based on this information. (Supplementary Item 4) The information processing device according to Supplementary Item 3, wherein the information related to the specific media data is initial information of the specific media data or a hash value generated based on the initial information. (Supplementary Item 5) The information processing device according to any one of Supplementary Items 1 to 4, wherein the processor receives an association change request from the first communication device or the second communication device to change the media data associated with the communication path from the specific media data to other media data based on the association change request.(Supplementary Item 6) An association method executed by an information processing device capable of communicating with each of a first communication device and a second communication device, the association method comprising the steps of: after a communication path for transmitting specific media data is established between the first communication device and the second communication device, associating the specific media data with the communication path based on information received from the first communication device and the second communication device, and storing association information indicating the association between the specific media data and the communication path in an association database. (Supplementary Item 7) A non-transitory storage medium storing a program for causing a computer to function as each unit in an information processing device described in any one of Supplementary Items 1 to 5.
[0077] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.
[0078] REFERENCE SIGNS LIST 100: Calling communication device 110: Association management unit 120: Communication unit 130: Media processing unit 200: Receiving communication device 210: Association management unit 220: Communication unit 230: Media processing unit 300: Association server 310: Association resolution unit 320: Association database 1000: Drive device 1001: Recording medium 1002: Auxiliary storage device 1003: Memory device 1004: CPU 1005: Interface device 1006: Display device 1007: Input device 1008: Output device
Claims
1. An information processing device capable of communicating with each of a first communication device and a second communication device, comprising: an association resolution unit that, after a communication path for transmitting specific media data is established between the first communication device and the second communication device, associates the specific media data with the communication path based on information received from the first communication device and the second communication device; and an association database that stores association information indicating the association between the specific media data and the communication path.
2. The information processing device according to claim 1, wherein the association resolution unit transmits information including a pair ID for identifying the communication path and a media data ID for identifying the specific media data to each of the first communication device and the second communication device.
3. The information processing device of claim 1, wherein the association resolution unit receives address information regarding the communication path, port information regarding the communication path, and information regarding the specific media data from each of the first communication device and the second communication device, and generates the association information based on this information.
4. The information processing device according to claim 3, wherein the information relating to the specific media data is initial information of the specific media data or a hash value generated based on the initial information.
5. The information processing device of claim 1, wherein the association resolution unit receives an association change request from the first communication device or the second communication device to change the media data associated with the communication path, and changes the media data associated with the communication path from the specific media data to other media data based on the association change request.
6. An association method executed by an information processing device capable of communicating with each of a first communication device and a second communication device, comprising: after a communication path for transmitting specific media data is established between the first communication device and the second communication device, associating the specific media data with the communication path based on information received from each of the first communication device and the second communication device; and storing association information indicating the association between the specific media data and the communication path in an association database.
7. A program for causing a computer to function as each unit in an information processing device according to any one of claims 1 to 5.
Citation Information
Patent Citations
Multimedia data transmitter
JP2003204575A
Apparatus and method for recording 'pathenhanced' multimedia data
WO2004100166A2
