Real-time communication control device, metaverse management device, and program

Abstract

A real-time communication control device (100, 100A, 100B, 100C, 100D) comprising a media processing thread (133), a packet transfer unit (140), and a session control thread (132) for real-time communication comprises an integrated management unit (110) that, when the packet transfer unit (140) and the media processing thread (133) preliminarily secure a processor resource for media processing, performs control of the idle state so that the processor can transition to a deep idle state and stay in the deep idle state without consuming extra power.

Description

Real-time Communication Control Device, Metaverse Management Device, and Program

[0001] The present invention relates to a real-time communication control device, a metaverse management device, and a program.

[0002] Attention has been paid to the metaverse in which users act via avatars in a virtual space. In the metaverse (virtual space), there is a function to create a room among nearby users (such as avatars) in order to perform real-time communication such as audio and video between the nearby users.

[0003] FIG. 15 is a diagram showing a configuration example of a metaverse service system. The metaverse service system 1 shown in FIG. 15 is such that a plurality of users simultaneously participate in a virtual space 2 on the Internet using avatars 3, and freely move within the virtual space 2 using the avatars 3. The avatars 3 participating in the virtual space 2 are marked with identification marks 4, and the identification marks 4 are color-coded according to the attributes of the participating users.

[0004] The avatar 3 is a character object that serves as the user's alter ego. The virtual space composed of 2D or 3D includes, in addition to the room that displays the avatar 3 and the game space where the avatar 3 can be operated, a metaverse space for the purpose of communication with other users, etc.

[0005] In the virtual space 2, the user can perform communication with other users, shopping, video distribution, appreciation meetings of various contents, etc. By surrounding and designating (reference numeral aa in FIG. 15) the avatars 3 within the virtual space 2, the user controls so that an audio-visual call can be made between nearby users (arrow bb in FIG. 15). The metaverse service system 1 is realized by a server capable of data communication via a network not shown. The server includes a virtual space control unit 10, a session control unit 11, and a media processing unit 12.

[0006] The virtual space control unit 10 has functions such as avatar position control, avatar profile management, and game scenarios. The session control unit 11 performs session control between the mobile information terminal 2 and personal computer 6 and the media processing unit 12, for example using SIP (Session Initiation Protocol). The media processing unit 12 realizes real-time communication via voice communication, for example using RTP (Real-time Transport Protocol). The media processing unit 12 realizes voice communication by resampling and encoding media data such as voice data and generating RTP packets, etc.

[0007] The metaverse service system 1 can be connected to a mobile information terminal 5 and a personal computer 6. The mobile information terminal 2 and the personal computer 6 can communicate via the network with the session control unit 11 and the media processing unit 12. The session control unit 11 and the media processing unit 12 perform real-time communication control with the mobile information terminal 5 and the personal computer 6.

[0008] The virtual space control unit 10 instructs the session control unit 11 and the media processing unit 12 to create a room so that real-time communication can be performed between nearby users (code cc in Figure 15).

[0009] On the other hand, the CPU (Central Processing Unit) has a function to control the CPU's idle state through hardware control, which is called LPI (Low Power Idle). LPI is also often referred to as CPUidle or C-state, and below, LPI will be described as C-state. C-state attempts to conserve power by turning off the power to some of the CPU's circuits when the CPU load is low (Patent Documents 1 and 2).

[0010] Patent Document 1 discloses a task scheduler device that takes C-state into consideration. The task scheduler device described in Patent Document 1 colors the CPU cores into "busy" and "idle" groups in task management, setting shallow C-states for the "busy" CPU cores and deep C-states for the "idle" CPU cores.

[0011] Patent Document 2 discloses a packet forwarding system that takes CPU idle into consideration. In the packet forwarding system described in Patent Document 2, the packet receiving function transitions to a deep C-state when the packet arrival frequency is low, and in order to mitigate the recovery time from the deep C-state, it anticipates packet arrival and performs pre-wake-up.

[0012] Figure 16 is a schematic diagram of the metaverse service system. As shown in Figure 16, the metaverse service system 1 (Figure 15) includes a server 1A. Server 1A has hardware (HW) 20, an OS 30, and room threads (thread for room #1, thread for room #2, thread for room #3, ...) 50 for displaying avatars in the metaverse service program on the user space 40.

[0013] The hardware 20 includes a processor 21 such as a CPU and a NIC (Network Interface Card) 22. The processor 21 has multiple CPU cores #1, CPU core #2, CPU core #3, etc. The NIC 22 is NIC hardware that realizes the NW interface.

[0014] The OS 30 includes a packet forwarding unit 31. The packet forwarding unit 31 includes a packet processing API (Application Programming Interface) and implements packet processing functions when a packet arrives at the NIC 22 and when a packet is forwarded to the NIC 22.

[0015] The room threads (thread for Room #1, thread for Room #2, thread for Room #3, ...) 50 are threads of the program that displays avatars, and a number of room threads are created equal to the number of avatars participating in the metaverse. Figure 16 shows the thread for Room #1, thread for Room #2, thread for Room #3, ..., but in reality, a huge number of room threads are launched based on a vast number of room creation instructions. Each room thread 50 has a session control thread (labeled as session control in Figure 16) 11 for connecting to the session control unit 11 in Figure 15, and a media processing thread (labeled as media processing in Figure 16) 12 for connecting to the media processing unit 12 in Figure 15.

[0016] International Publication No. 2022 / 027327, International Publication No. 2022 / 017943

[0017] However, in conventional metaverse service systems, when there is a high concentration of users in the virtual space, a huge number of room creation instructions may be issued. In particular, even when real-time communication is not actually taking place between users, if rooms are created as a precaution, resource reservations are made in the media processing unit 12, which can activate CPU cores and consume a large amount of power.

[0018] Figure 17 illustrates the challenges of the metaverse service system 1 shown in Figure 15. As shown by the code dd in Figure 17, when the user density is excessive, a huge number of real-time communication session establishment requests are generated. The virtual space control unit 10 issues a huge number of room creation instructions (code ee in Figure 17) when there is a high concentration of users in the virtual space.

[0019] Figure 18 shows an example of the internal structure of the media processing unit 12. The same reference numerals are used for components identical to those in Figure 16. The room threads 50 are room threads based on room creation instructions, and represent threads for room #0, room #1, ..., room #7. Tasks for executing these room #0, room #1, ..., room #7 threads are assigned to the CPU cores (CPU core #0, CPU core #1, CPU core #2, ..., CPU core #7) of the processor 21 by a task scheduler (not shown).

[0020] As shown by the symbol dd in Figure 17, when users are densely clustered in the virtual space, a huge number of room creation instructions are issued. When rooms are proactively created simply by users passing each other, resources (CPU cores) of the media processing unit 12 (Figure 17) are reserved even when real-time communication is not actually taking place between users. The threads for Room #1, Room #2, ..., Room #8, ... shown in Figure 18 are "idle processing" threads that are not actually performing real-time communication between users. Despite being "idle processing," the metaverse service system 1 reserves CPU cores of the processor 21 (CPU core #0, CPU core #1, CPU core #2, ..., CPU core #7...) as a resource reservation, resulting in many CPU cores being in operation and consuming a large amount of power. In this way, the CPU resources reserved in advance consume power (symbol ff in Figure 18).

[0021] In light of this background, the present invention was made, and its objective is to achieve power saving of processor resources even when a thread is launched as a precaution.

[0022] To solve the aforementioned problems, a real-time communication control device is provided, comprising a session control unit and a media processing unit for real-time communication, wherein the media processing unit, when preemptively securing processor resources for media processing, includes an integrated management unit that controls the idle state so that the processor does not consume excess power, transitions to a deep idle state, and remains in the deep idle state.

[0023] According to the present invention, power saving of processor resources can be achieved even when threads are launched as a precaution.

[0024] This is a schematic diagram of a real-time communication control system according to an embodiment of the present invention. This is a diagram showing an example of the cooperation of each functional unit of the real-time communication control device of the real-time communication control system according to an embodiment of the present invention. This is a diagram showing an example of the configuration of a real-time communication control device in which the packet forwarding unit of the real-time communication control system according to an embodiment of the present invention is placed in user space. This is a diagram showing an example of the configuration of a real-time communication control device in which the processor idle state management unit of the real-time communication control system according to an embodiment of the present invention is placed in the OS kernel. This is a diagram showing an example of the configuration of a real-time communication control device in which the packet forwarding unit of the real-time communication control device according to an embodiment of the present invention is placed in user space and the processor idle state management unit is placed on the OS kernel. This is a diagram showing an example of the configuration of a real-time communication control device in which a task management unit independent of the room thread of the real-time communication control device according to an embodiment of the present invention is provided. This is a flowchart showing the operation of the processor idle state management unit of the real-time communication control device in the real-time communication control system according to an embodiment of the present invention. This is a flowchart showing the task execution operation of the room thread of the real-time communication control device in the real-time communication control system according to an embodiment of the present invention. This is a diagram showing an example of the task execution timing by the room thread of the real-time communication control device in the real-time communication control system according to an embodiment of the present invention. This is a flowchart showing the task execution operation of the task management unit of the real-time communication control device in the real-time communication control system according to an embodiment of the present invention. This is a flowchart showing the packet forwarding operation of the packet forwarding unit of the real-time communication control device in the real-time communication control system according to an embodiment of the present invention. This is a diagram showing an example of the packet forwarding timing by the packet forwarding unit of the real-time communication control device in the real-time communication control system according to an embodiment of the present invention. This figure shows an example of application where the real-time communication control device shown in Figure 1 is used as a metaverse management device. This figure shows an example of application where the real-time communication control device shown in Figure 1 is used as a metaverse management device. This figure shows an example of application where the real-time communication control device shown in Figure 1 is used as a metaverse management device.This is a hardware configuration diagram showing an example of a computer that implements the functions of a real-time communication control device in a real-time communication control system according to an embodiment of the present invention. This is a diagram showing an example of the configuration of a metaverse service system. This is a schematic configuration diagram of a metaverse service system. This is a diagram illustrating the problems of the metaverse service system in Figure 15. This is a diagram showing an example of the internal structure of the media processing unit of a metaverse service system.

[0025] The following describes a real-time communication control device, etc., in an embodiment for carrying out the present invention (hereinafter referred to as "this embodiment") with reference to the drawings.

[0026] [Overall Configuration] Figure 1 is a schematic diagram of a real-time communication control system according to an embodiment of the present invention. The same reference numerals are used for components identical to those in Figure 16. The real-time communication control system 1000 is an example applied to a metaverse service system. The present invention is not limited to media processing, but also includes session control units for real-time communication and packet forwarding units for real-time communication.

[0027] As shown in Figure 1, the real-time communication control system 1000 includes hardware (HW) 20, an OS 30, and a user space 40. The OS 30 and user space 40 are equipped with a real-time communication control device 100 (metaverse management device). The real-time communication control device 100 (metaverse management device) provides an application for communication between multiple users. The real-time communication control device 100 (metaverse management device) sets power saving settings for threads related to a room according to the actions of avatars in the room within the application (see Application Example 1-3 in Figures 13A-13C below).

[0028] The real-time communication control device 100 has an integrated management unit 110, a processor idle state management unit 120, and room threads (threads for room #1, room #2, room #3, etc.) 130 (threads for media processing, etc.) on the user space 40, and a packet transfer unit 140 (media processing unit) and a processor idle state setting unit 150 on the OS 30 (described later). The above threads may also be "processes".

[0029] The hardware 20 includes a processor 21 such as a CPU and a NIC 22. The processor 21 has multiple CPU cores #1, CPU core #2, CPU core #3, etc. The NIC 22 is NIC hardware that realizes the NW interface. The processor 21 is composed of a CPU. In addition to the CPU, it may also include processors such as a GPU (Graphic Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), and DPU (Data Processing Unit).

[0030] The OS 30 includes a packet forwarding unit 140 and a processor idle state setting unit 150. In addition to the functions of the packet forwarding unit 31 shown in Figure 16, the packet forwarding unit 140 also includes a packet receiving unit 141 (described later).

[0031] [Real-time communication control device] As shown in the dashed box in Figure 1, the real-time communication control device 100 includes an integrated management unit 110, a processor idle state management unit 120, a room thread 130, a packet forwarding unit 140, and a processor idle state setting unit 150.

[0032] <Integrated Management Unit 110> When the packet forwarding unit 140 (media processing unit) reserves processor resources for media processing in advance, the integrated management unit 110 controls the idle state so that the processor does not consume extra power, transitions to a deep idle state, and remains in the deep idle state.

[0033] Specifically, the integrated management unit 110 manages the state of threads or the state of tasks processed by threads so that threads for media processing, etc. (room threads 130) do not consume excessive power from the processor 21. The integrated management unit 110 controls the idle state of the processor idle state management unit 120 so that it can remain in a deep idle state for a long time depending on the arrival state of packets (described later). In addition, in the case of "preliminary resource reservation," if packet data for session updates or statistical information arrives, the integrated management unit 110 instructs the packet forwarding unit 140 not to wake up sleep threads (extending the sleep time).

[0034] In metaverse use cases, "preliminary" refers to situations where, even if real-time communication is not actually taking place between users, a room is created preliminarily and processor resources are reserved so that nearby avatars can immediately begin conversing. "Preliminary resource reservation" refers to a state where a communication session for real-time communication is established preliminarily, for example, when users pass each other, but real-time communication between users has not yet taken place.

[0035] <Processor idle state management unit 120> The processor idle state management unit 120 configures the processor so that it enters a deep idle state when the processing thread is in a reserve resource reservation state. Specifically, when the processing thread is in a reserve resource reservation state, the processor idle state management unit 120 requests the processor idle state setting unit 150 to configure the processor so that the CPU core of the processor 21 can transition to a deep idle state (C-state).

[0036] The "deep idle state" described in the claim refers to a predetermined grade when the CPU idle state is defined by a grade. Similarly, the "shallow idle state" also refers to a predetermined grade when it is defined by a grade. For example, LPI (Low Power Idle) hardware control has a function to control the idle state of the CPU by hardware control. LPI is often referred to as CPUidle or C-state, and below, LPI will be described as C-state. The CPU idle state has grades C0 to C6. The "deep idle state" described in the claim refers to a predetermined grade (for example, grade C6).

[0037] The definition of C-states varies depending on the CPU hardware. For example, some models lack C4 and C5 states, while others have a state where C1 is followed by C1E. Furthermore, the depth to which the CPUidle state transitions is controlled by the CPU hardware and is product-dependent (often not controllable by software such as the kernel).

[0038] When the system is in a "preliminary resource reservation state," the room thread 130 detects this and transmits it to the integrated management unit 110, which then controls the idle state to allow the processor idle state management unit 120 to remain in a deep idle state for a long period of time. In other words, when the system is in a "preliminary resource reservation state," the room thread 130 has established a session for real-time communication, such as for voice calls, but the user is not actually speaking or making any other actions, and there is no communication of user data.

[0039] The processor idle state management unit 120 monitors the status of each room thread 130. The monitoring triggers are (1) periodic monitoring, (2) when a room thread 130 is started, and (3) when there is a change in the status of a room thread 130. Regarding the status management of the room thread 130, when there is a change in the status of a room thread 130, the room thread 130 may notify the processor idle state management unit 120 of the status change according to the status of each thread.

[0040] <Room thread 130> The room thread 130 includes a queuing thread 131, a session control thread (indicated as session control in Figure 1) 13, and a media processing thread (indicated as media processing in Figure 1) 133.

[0041] The room threads (thread for Room #1, thread for Room #2, thread for Room #3, ...) 130 are threads for the program that displays avatars, and a number of room threads are created equal to the number of avatars participating in the metaverse. Figure 1 shows the threads for Room #1, Room #2, Room #3, ..., but in reality, a vast number of room threads are launched based on a huge number of room creation instructions.

[0042] The queuing system 131 holds data to be sent to the integrated management unit 110 in a data area called a queue, and moves on to the next process without waiting for the receiving side to complete its processing (session control, media processing). When sending data, the room thread 130 can deliver the data even if it is not synchronized with the destination by placing the data in the queue. The queuing system 131 makes the coordination between the room thread 130 and the integrated management unit 110 loose, allowing the integrated management unit 110 to integrate flexibly.

[0043] Here, when the real-time communication control device 100 is applied to a metaverse management device that provides an application for communication between multiple users, it corresponds to a session control thread 132 (the "session control unit" as described in the claim) and a media processing thread 133 (the "processing thread" as described in the claim), and the media processing thread 133, together with the packet transfer unit 140, corresponds to the "media processing unit" as described in the claim. The media processing unit (media processing thread 133 and packet transfer unit 140) of the real-time communication control device 100 (metaverse management device) creates a room provisionally when in a "provisional resource reservation state". A "provisional resource reservation state" refers to a state in which a communication session for real-time communication is provisionally established when users pass each other in the vicinity, but real-time communication is not yet performed between users.

[0044] Each thread 130 for a room that performs processing enables the processor 21 to be used to maintain the sleep state as much as possible, so that "unnecessary state monitoring and the like are not performed" (for example, using interrupt processing instead of monitoring by polling, etc.). "Not performing unnecessary state monitoring" means, for example, in packet arrival monitoring, instead of using 100% of the CPU resources to monitor in polling mode, the corresponding thread is put to sleep and wakes up by an interrupt from the NIC 22. There are also cases such as reducing the frequency of monitoring the life and death of threads if it is being monitored.

[0045] Also, each thread 130 for a room that performs processing, when "there is a task that must be processed", executes it by shifting as much as possible in the "time direction" to ensure the sleep time. Here, "when there is a task that must be processed" means that even if there is one task. That is, if there is even one task, the corresponding thread wakes up the processor. The granularity of shifting in the time axis direction is determined within the delay requirements defined by the service and the range of the bit rate of the media.

[0046] <Packet transfer unit 140> The packet transfer unit 140 is a transfer function unit for real-time communication. Also, since the packet transfer unit 140 performs packet processing for media processing, it is described as the "media processing unit" described in the claims. Note that the packet transfer unit 140 only needs to be a transfer function for real-time communication and is not limited to media processing.

[0047] The packet transfer unit 140 has a packet receiving unit 141 that performs reception processing of the packets queued in the packet reception queue 142 when a packet arrives at the NIC 22. When the packet data of session update or statistical information arrives in the situation of preliminary resource reservation, the packet transfer unit 140 causes the packet receiving unit 141 to perform shifting in the time axis direction by queuing so that the thread is not woken up while the processor is sleeping.

[0048] Specifically, the packet transfer unit 140 has a packet reception unit 141 for packet reception processing. When a packet arrives at the NIC 22, the packet reception unit 141 performs reception processing on the packets queued in the packet reception queue 142. The packet transfer unit 140 transfers packets to the room thread 130 until the packet reception queue 142 of the packet reception unit 141 becomes empty. Also, the packet transfer unit 140 realizes a packet processing function when transferring packets to the NIC 22.

[0049] In the situation of "preliminary resource reservation", when there is an update of the session or packet data of statistical information arrives, the packet transfer unit 140 performs skew in the time axis direction, such as queuing, in the packet reception unit 141 so as not to wake up the sleeping thread (lengthen the sleep time). The granularity of the skew in the time axis direction is determined within the delay requirements defined by the service and the range of the media bit rate.

[0050] When there is a packet arrival, the packet transfer unit 140 does not immediately perform packet reception processing, but waits for a certain period (FIG. 12 described later) and then performs packet reception processing. It is desirable to determine the certain period in consideration of service requirements (delay requirements) demanded by the room thread 130 and the like.

[0051] While waiting for packet reception processing for a certain period (FIG. 12 described later), in order to prevent the processor from waking up due to the arrival of a new packet, the packet transfer unit 140 stops the hardware interrupt from the NIC 22 due to packet arrival for a certain period. When the certain period expires, it performs reception processing on the packets queued in the packet reception queue 142 and transfers the packets to the room thread 130 (executes until the packet reception queue becomes empty). Here, by separating the processor core used by the packet transfer unit 140 from the core used by the room thread 130, it is possible to prevent the room thread 130 from waking up due to packet reception processing.

[0052] <Processor idle state setting unit 150> The processor idle state setting unit 150 basically configures the CPU core used by the room thread 130 to be able to transition to a deep idle state. However, it should be noted that the above configuration often requires privileged access to ring0, such as kernel mode. The processor idle state setting unit 150 configures the CPU core used by the room thread 130 to be limited to transitioning to a shallow idle state.

[0053] Here, the idle state suitable for setting a transition to a shallow idle state is determined according to the QoS requirements for the thread 130 for the relevant room. Generally, recovery after transitioning to a deep idle state often involves a wake-up latency of about 1us to 150us, so it is desirable to set an appropriate idle state according to the service requirements (latency requirements) required for each room.

[0054] [Coordination of Each Functional Unit of the Real-Time Communication Control Device] Figure 2 shows an example of coordination of each functional unit of the real-time communication control device 100 shown in Figure 1. The room thread 130 detects when it is in a "preliminary resource reservation state" and transmits this to the integrated management unit 110 (arrow a in Figure 2). The integrated management unit 110 controls the idle state of the processor idle state management unit 120 so that it can stay in a deep idle state for a long time, depending on the arrival state of the packet (arrow b in Figure 2). When the integrated management unit 110 receives packet data for session updates or statistical information in the "preliminary resource reservation" state, it instructs the packet forwarding unit 140 not to wake up the sleep thread (arrow c in Figure 2).

[0055] The processor idle state management unit 120, when a processing thread is in a "preliminary resource reservation" state, requests the processor idle state setting unit 150 to configure the processor so that the CPU core of the processor 21 can transition to a deep idle state (C-state) (arrow d in Figure 2). The processor idle state setting unit 150 basically configures the CPU core used by the room thread 130 to be able to transition to a deep idle state (arrow e in Figure 2), but for the CPU core used by the room thread 130 in question, it configures it to limit the transition to a shallow idle state.

[0056] When a packet arrives at the NIC 22 (arrow f in Figure 2), the packet forwarding unit 140 processes the packet that was queued in the packet reception queue 142. The packet forwarding unit 140 forwards packets to the room thread 130 until the packet reception queue 142 of the packet reception unit 141 is empty (arrow g in Figure 2).

[0057] [Arrangement of Functional Units of the Real-Time Communication Control Device] Figures 3 and 6 illustrate variations in the arrangement of each functional unit of the real-time communication control device. The same reference numerals are used for components identical to those in Figure 1.

[0058] <Package forwarding unit 140 in user space 40> Figure 3 shows an example configuration of a real-time communication control device 100A (metaverse management device) in which the packet forwarding unit 140 is placed in the user space 40.

[0059] The real-time communication control device 100A shown in Figure 3 is an example in which the packet forwarding unit 140 is placed on the user space 40, compared to the real-time communication control device 100 in Figure 1.

[0060] <Placement of Processor Idle State Management Unit 120 in OS 30> Figure 4 shows an example configuration of a real-time communication control device 100B (metaverse management device) in which the processor idle state management unit 120 is placed in the kernel of OS 30.

[0061] The real-time communication control device 100B shown in Figure 4 is an example in which the processor idle state management unit 120 is placed on the kernel of the OS 30, compared to the real-time communication control device 100 in Figure 1.

[0062] <Placement of the packet forwarding unit 140 in user space 40 and placement of the processor idle state management unit 120 on the OS 30> Figure 5 shows an example configuration of a real-time communication control device 100C (metaverse management device) in which the packet forwarding unit 140 is placed in user space 40 and the processor idle state management unit 120 is placed on the kernel of the OS 30.

[0063] The real-time communication control device 100C shown in Figure 5 is an example in which the packet forwarding unit 140 is located in the user space 40 and the processor idle state management unit 120 is located on the kernel of the OS 30, compared to the real-time communication control device 100 in Figure 1.

[0064] Thus, the processor idle state management unit 120 may be located in the user space 40 (Figures 1 and 3), or it may be located within the kernel of the OS 30 (for example, the Linux kernel; "Linux" is a registered trademark) (Figures 4 and 5).

[0065] Furthermore, the packet forwarding unit 140 may be located within the Linux kernel, such as NAPI which is standard in the Linux kernel (Figures 1 and 4), or it may perform packet reception processing in user space 40, such as Intel DPDK (Data Plane Development Kit) ("Intel" is a registered trademark) (Figures 3 and 5).

[0066] This section describes data transfer technologies within servers. These technologies include New API (NAPI), DPDK, and KBP (Kernel Busy Poll). KBP constantly monitors packet arrivals within the kernel using a polling model. This suppresses softIRQs and enables low-latency packet processing. New API (NAPI) processes packets upon arrival via a hardware interrupt request followed by a software interrupt request.

[0067] DPDK implements packet processing functionality in user space where applications run, and immediately harvests packets upon arrival from user space using a polling model. Specifically, DPDK is a framework for controlling NICs in user space, a task traditionally handled by the Linux kernel. The biggest difference from Linux kernel processing is the presence of a polling-based receiving mechanism called PMD (Pull Mode Driver). Normally, in the Linux kernel, an interrupt occurs upon data arrival at the NIC, triggering the execution of the receiving process. In contrast, PMD uses a dedicated thread to continuously check for data arrival and perform receiving processing. By eliminating overhead such as context switching and interrupts, it enables high-speed packet processing. DPDK significantly improves packet processing performance and throughput, allowing more time for data plane application processing. However, DPDK exclusively uses computer resources such as the CPU and NIC.

[0068] The present invention can be applied when there is a polling thread within the kernel, such as in NAPI and KBP (Figures 1 and 4). Furthermore, the present invention can be applied when there is a polling thread in user space, such as in DPDK (Figures 3 and 5).

[0069] <Introduction of Task Management Unit> In the real-time communication control devices 100 to 100C shown in Figures 1 to 5, the room thread 130 performs task management (Figure 8). As an alternative variation, a task management unit 135 independent of the room thread 130 may be provided.

[0070] Figure 6 shows an example configuration of a real-time communication control device 100D (metaverse management device) equipped with a task management unit 135 independent of the room threads 130. The same reference numerals are used for components identical to those in Figure 2. As shown in Figure 6, the real-time communication control device 100D is equipped with a task management unit 135 independent of the room threads 130. The task management unit 135 detects when a "preliminary resource reservation state" is in place and transmits this information to the integrated management unit 110 (arrow a in Figure 2).

[0071] The task management unit 135, when it has a task to be executed, does not execute the task immediately, but instead performs task management by waiting for a certain period of time (Figure 9 below) before executing the task (Figure 10 below). In the case of the real-time communication control devices 100 to 100C shown in Figures 1 to 5, which do not have a task management unit 135, this task management is performed by the room thread 130 (Figure 8 below). By providing a task management unit 135 that is independent of the room thread 130, modifications to the room thread 130 become unnecessary.

[0072] The operation of the real-time communication control devices 100 to 100D, configured as described above, will be explained below.

[0073] <Operation of the Integrated Management Unit 110> The integrated management unit 110 of the real-time communication control devices 100 to 100D shown in Figures 1 to 6 manages the room thread 130, the processor idle state management unit 120, and the packet forwarding unit 140 in an integrated manner, regardless of the variations in the arrangement of each functional unit.

[0074] The integrated management unit 110 monitors the status of the room thread 130. Specifically, the integrated management unit 110 receives notifications of status changes from the room thread 130 periodically, when the room thread 130 is started, or when there is a change in the status of the room thread 130. The integrated management unit 110 requests the room thread 130 to confirm whether or not there has been a status change and receives notifications of status changes from the room thread 130 (bidirectional arrow a in Figure 2). The integrated management unit 110 requests the room thread 130 to confirm whether or not there has been a status change and receives notifications of status changes from the room thread 130 (bidirectional arrow a in Figure 2).

[0075] The integrated management unit 110 instructs the processor idle state management unit 120 (arrow b in Figure 2) to control the idle state so that it can remain in a deep idle state for a long time, depending on the arrival status of the packet (Figures 7 to 10). In addition, in the case of "preliminary resource reservation," if packet data for session updates or statistical information arrives, the integrated management unit 110 instructs the packet forwarding unit 140 (arrow c in Figure 2) to prevent sleep threads from being woken up (Figures 11 and 12).

[0076] <Operation of Processor Idle State Management Unit 120> When a thread performing processing is in a "preliminary resource reservation" state, the Processor Idle State Management Unit 120 requests the Processor Idle State Setting Unit 150 to configure the processor so that the CPU core of the processor 21 can transition to a deep idle state (C-state) (arrow d in Figure 2).

[0077] Figure 7 is a flowchart showing the operation of the processor idle state management unit 120 of the real-time communication control devices 100 to 100D shown in Figures 1 to 6. In step S11, the integrated management unit 110 monitors the state of each room thread 130. Monitoring is triggered periodically, when a room thread 130 is started, or when there is a change in the state of a room thread 130. The integrated management unit 110 can also request the room thread 130 to confirm whether or not there has been a change in state, and can receive notifications of state changes from the room thread 130. In this embodiment, the integrated management unit 110 monitors the state of each room thread 130, but the processor idle state management unit 120 may also monitor the state of each room thread 130. That is, regarding the state management of the room thread 130, when there is a change in the state of a room thread 130, the room thread 130 may notify the processor idle state management unit 120 of the state change according to the state of each thread.

[0078] In step S12, the processor idle state management unit 120 determines whether the room thread 130 is in a "preliminary resource reservation state". As described above, the "preliminary resource reservation state" refers to a state in which a communication session for real-time communication is established provisionally, for example, when users pass each other, but real-time communication between users has not yet been executed.

[0079] If the room thread 130 is in a "reserve resource reservation state" (S12: Yes), in step S13, the processor idle state management unit 120 requests the processor idle state setting unit 150 to set the CPU core used by the room thread so that it can transition to a deep idle state.

[0080] In step S14, the processor idle state management unit 120 configures the CPU core used by the room thread to enable transition to a deep idle state and terminates the processing of this flow. As described above, the "deep idle state" described in the claim refers to a predetermined grade when the CPU idle state is defined by a grade. However, this configuration often requires privileged access to ring0, such as kernel mode.

[0081] If the room thread 130 is not in a "preliminary resource reservation state" in step S12 (S12: No), then in step S15, the processor idle state management unit 120 determines whether the room thread 130 is in a communication state and whether there is a sufficient period of time for it to sleep. Examples of cases where there is a sufficient period of time for it to sleep are as follows: - When there is little incoming user data such as voice, and the room thread 130 has few tasks to process. - When tasks and packet processing are concentrated along the time axis to secure a period of time for it to sleep (Figures 11 and 12 below).

[0082] If the room thread 130 is in a communication state and there is a sufficient period of time for it to sleep (S15: Yes), in step S16 the processor idle state management unit 120 requests the processor idle state setting unit 150 to set the CPU core used by the room thread so that it can transition to a deep idle state.

[0083] In step S17, the processor idle state management unit 120 configures the CPU core used by the room thread to enable a transition to a deep idle state, and then terminates the processing of this flow.

[0084] If, in step S15, the room thread 130 is not in a communication state, or if the room thread 130 is in a communication state and there is not enough time for it to sleep (S15: No), then in step S18, the processor idle state management unit 120 requests the processor idle state setting unit 150 to limit the transition of the CPU core used by the room thread to a shallow idle state. As described above, a "shallow idle state" is a predetermined grade when the CPU idle state is defined by a grade.

[0085] In step S19, the processor idle state setting unit 150 terminates the processing of this flow by setting the CPU core used by the thread for the room in question to be limited to transitions to a shallow idle state. Here, the predetermined idle state set by the processor idle state setting unit 150 (hereinafter referred to as the appropriate idle state) is determined, for example, according to the QoS (Quality of Service) requirements for the room in question.

[0086] By executing the above flow, the integrated management unit 110 monitors the status of each room thread 130. If a room thread 130 is in a "backup resource reservation state," or even if it is not in a "backup resource reservation state," but is in a communication state and has a sufficient period of time to sleep, the processor idle state management unit 120 requests that the CPU core used by the room thread be configured to transition to a deep idle state. This allows the room threads that were started as a backup to sleep as much as possible, thereby transitioning the processor on which the room threads operate to a deep idle state and maintaining that state, thus achieving power saving.

[0087] <Task Execution Operation of Room Thread 130> Figure 8 is a flowchart showing the task execution operation of the room thread 130. Figure 9 is a diagram showing an example of the timing of task execution by the room thread 130. In step S21, if the room thread 130 has a task to be executed, it does not execute the task immediately, but waits for a certain period of time to execute the task, as shown in Figure 9. The "certain period of time" is determined by considering the service requirements (delay requirements, etc.) required of the room thread 130.

[0088] The task execution of the room thread 130 will be explained in more detail. As shown in Figure 1, the room thread 130 has a queuing 131. The queuing 131 holds data to be sent to the integrated management unit 110 in a data area called a queue. In this case, even if the room thread 130 has a task to be executed, it does not execute the task immediately by placing the data in the queue, but waits for a certain period of time for the task to be executed (in other words, it creates a period of time when the task is not executed by shifting the task on the time axis).

[0089] In step S22, the room thread 130 shifts tasks to one side in the time direction to secure time for sleep (dashed arrow h in Figure 9).

[0090] In step S23, the integrated management unit 110 manages the transition of the processor to the sleep state while the room thread 130 waits for a certain period of time for task execution. Specifically, the integrated management unit 110 instructs the processor idle state management unit 120 to transition the processor to the sleep state so that the processor can transition to a deep idle state. The processor idle state management unit 120 requests the processor idle state setting unit 150 to configure the processor so that the CPU core of the processor 21 can transition to a deep idle state (C-state) to the sleep state.

[0091] Alternatively, the room thread 130 may instruct the processor idle state management unit 120 to transition to the sleep state so that the processor can transition to a deep idle state, without going through the integrated management unit 110. The processor idle state setting unit 150 sets the CPU core used by the room thread 130 so that it can transition to a deep idle state. This shifts the tasks to one side in the time direction, creating the sleep time shown by the symbol i in Figure 9. Therefore, while the room thread 130 waits for task execution for a certain period of time, it transitions to the sleep state so that the processor can transition to a deep idle state.

[0092] In step S24, if a certain period of time has elapsed, the room thread 130 executes the queued tasks 131 (executing until the task queue is empty) and completes the processing of this flow. The above task management may be performed by the room thread 130, or as an alternative, a task management unit 135 independent of the room thread 130 may be provided, and the task management unit 135 may assign tasks to the room thread 130 considering the above certain period of time. The following describes an example of task execution by the task management unit 135.

[0093] <Task Execution Operation of Task Management Unit 135> Figure 10 is a flowchart showing the task execution operation of the task management unit 135. An example of the task execution timing by the task management unit 135 is the same as in Figure 9, and is therefore omitted from the illustration. In step S31, if the task management unit 135 has a task to be executed, it does not execute the task immediately, but waits for a certain period of time to execute the task, as shown in Figure 9. The "certain period of time" is determined by the task management unit 135 considering the service requirements (delay requirements, etc.) required of the room thread 130.

[0094] In step S32, the task management unit 135 shifts the tasks to one side in the time direction to secure time for sleep (dashed arrow h in Figure 9).

[0095] In step S33, the integrated management unit 110 manages the transition of the processor to the sleep state while the room thread 130 is waiting for task execution by the task management unit 135. Specifically, the integrated management unit 110 instructs the processor idle state management unit 120 to transition to the sleep state so that the processor can transition to a deep idle state. The processor idle state management unit 120 requests the processor idle state setting unit 150 to configure the processor so that the CPU core of the processor 21 can transition to a deep idle state (C-state) to the sleep state.

[0096] Alternatively, the task management unit 135 may instruct the processor idle state management unit 120 to transition to the sleep state so that the processor can transition to a deep idle state, without going through the integrated management unit 110. The processor idle state setting unit 150 sets the CPU core used by the room thread 130 so that it can transition to a deep idle state. This shifts the tasks to one side in the time direction, creating the sleep time shown by the symbol i in Figure 9. Therefore, while the task management unit 135 waits for a certain period of time for task execution, the processor transitions to the sleep state so that it can transition to a deep idle state.

[0097] In step S34, if a certain period has expired, the task management unit 135 executes the queued tasks 131 (executing them until the task queue is empty) and finishes processing this flow.

[0098] <Packet forwarding operation of the packet forwarding unit 140> Figure 11 is a flowchart showing the packet forwarding operation of the packet forwarding unit 140. Figure 12 is a diagram showing an example of packet forwarding timing by the packet forwarding unit 140 (Figure 1). In step S41, when a packet arrives, the packet forwarding unit 140 does not immediately process the packet reception, but waits for a certain period of time. The "certain period of time" is determined by considering the service requirements (delay requirements) etc. required for the room thread 130.

[0099] In step S42, the packet forwarding unit 140 concentrates packet processing in the time direction to secure time for sleep (dashed arrow j in Figure 12).

[0100] In step S43, the packet forwarding unit 140 stops hardware interrupts from the NIC 22 (Figure 1) caused by packet arrivals for a certain period of time in order to prevent the processor from waking up due to the arrival of a new packet while waiting for a certain period of time to process the received packet. This shifts packet processing to one side in the time direction, creating a sleep time as shown by the code k in Figure 12. Therefore, in the case of a preliminary resource reservation, if packet data for session updates or statistical information arrives, the packet receiving unit 141 performs queuing to shift the processing to one side in the time direction, preventing threads from waking up during sleep (extending the sleep time).

[0101] In step S44, if a certain period of time has elapsed, the packet forwarding unit 140 processes the packets that were queued in the packet reception queue 142 (Figure 1) of the packet receiving unit 141, and forwards the packets to the room thread 130 (this process continues until the packet reception queue 142 is empty), thereby ending the processing of this flow.

[0102] <Points to note regarding the task execution operation of the room thread 130 and the packet forwarding operation of the packet forwarding unit 140> The processor core used by the packet forwarding unit 140 is kept separate from the core used by the room thread 130. By adopting this configuration, it is possible to prevent the room thread 130 from being woken up by packet reception processing.

[0103] [Hardware Configuration] The real-time communication control devices 100, 100A, 100B, 100C, and 100D according to the above embodiment are configured by a server such as the metaverse service system shown in Figure 17. When the real-time communication control devices 100, 100A, 100B, 100C, and 100D are configured by a server, the server is realized by a computer 900 configured as shown in Figure 14, for example. Figure 14 is a hardware configuration diagram showing an example of a computer 900 that realizes the functions of the real-time communication control device 100. The computer 900 has a CPU 901, ROM 902, RAM 903, HDD 904, communication interface (I / F) 906, input / output interface (I / F) 905, and media interface (I / F) 907.

[0104] The CPU 901 operates based on programs stored in the ROM 902 or HDD 904, and controls the real-time communication control devices 100, 100A, 100B, 100C, and 100D shown in Figures 1 to 4. The ROM 902 stores boot programs executed by the CPU 901 when the computer 900 starts up, as well as programs that depend on the computer 900's hardware.

[0105] The CPU 901 controls input devices 910, such as a mouse or keyboard, and output devices 911, such as a display, via the input / output interface 905. The CPU 901 acquires data from the input devices 910 and outputs the generated data to the output devices 911 via the input / output interface 905. In addition to the CPU 901, a GPU (Graphics Processing Unit) or the like may also be used as a processor.

[0106] The HDD 904 stores programs executed by the CPU 901 and data used by those programs. The communication I / F 906 receives data from other devices via a communication network (e.g., NW (Network) 920) and outputs it to the CPU 901, and also transmits data generated by the CPU 901 to other devices via the communication network.

[0107] The media interface 907 reads a program or data stored in the recording medium 912 and outputs it to the CPU 901 via the RAM 903. The CPU 901 loads the program related to the desired processing from the recording medium 912 onto the RAM 903 via the media interface 907 and executes the loaded program. The recording medium 912 is an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto Optical disk), a magnetic recording medium, a conductive memory tape medium, or a semiconductor memory.

[0108] For example, when computer 900 functions as a real-time communication control device 100, 100A, 100B, 100C, 100D configured as one of the devices according to this embodiment, the CPU 901 of computer 900 realizes the functions of the real-time communication control devices 100, 100A, 100B, 100C, 100D by executing a program loaded onto RAM 903. Furthermore, data from RAM 903 is stored in HDD 904. CPU 901 reads and executes a program related to the desired processing from recording medium 912. Alternatively, CPU 901 may read a program related to the desired processing from another device via a communication network (NW 920).

[0109] [Application Example] The above-mentioned real-time communication control devices 100, 100A, 100B, 100C, and 100D can be used as metaverse management devices for a metaverse service system. A metaverse management device limits the session control unit and packet forwarding unit for real-time communication to media processing.

[0110] The phrase "in response to the actions of an avatar in a room within the application" in the claim encompasses all behaviors resulting from the actions of an avatar in a room. For example, "in response to the actions of an avatar in a room within the application" could mean "in response to the distance of the avatar controlled by the user." It could also mean "in response to situations where it is not expected that a large amount of information such as turning the voice call function ON / OFF or other media processing will occur." Below, we will explain an example of avatar actions in a room within the application, using the distance of the avatar controlled by the user as an example.

[0111] <Application Example 1> Figure 13A shows an application example 1 in which the real-time communication control device 100 of Figure 1 is applied as a metaverse management device. The same reference numerals are used for the same components as in Figure 1.

[0112] As shown in Figure 13A, the real-time communication control system 1000 includes hardware 20, an OS 30, and a user space 40, and the OS 30 and user space 40 are equipped with a metaverse management device 100. The metaverse management device 100 shown in Figure 13A includes a power saving function unit 200A that performs power saving settings for threads related to a room according to the actions of avatars in a room within an application. The power saving function unit 200A corresponds to the integrated management unit 110, the processor idle state management unit 120, and the processor idle state setting unit 150.

[0113] The integrated management unit 110 shown in Figure 13A controls the threads for media processing (threads related to the room in response to the actions of avatars in the room) to stay in a deep idle state for a long time, depending on the distance of the avatar operated by the user, so as not to consume extra processor power. The processor idle state management unit 120 shown in Figure 13A configures the processor so that if a thread performing media processing is in a state of reserve resource reservation, the processor can transition to a deep idle state (C-state), according to the instructions of the integrated management unit 110. The processor idle state setting unit 150 shown in Figure 13A configures the CPU core used by the room thread 130 so that it can transition to a deep idle state, according to the instructions of the processor idle state management unit 120.

[0114] As a result, the metaverse management device 100 shown in Figure 13A, equipped with a power saving function unit 200A, can set power saving settings for the room thread according to the distance of the avatar operated by the user, linked to the user's actions on the metaverse.

[0115] <Application Example 2> Figure 13B shows an application example 2 in which the real-time communication control device 100 of Figure 1 is applied as a metaverse management device. The same reference numerals are used for the same components as in Figure 13A.

[0116] The metaverse management device 100 shown in Figure 13B includes a power saving function unit 200B, which is an additional queuing function 131 for room threads 130 added to the power saving function unit 200A shown in Figure 13A. In addition to the power saving function of the power saving function unit 200A, the power saving function unit 200B performs queuing 131, which places data in a queue even when there are tasks that need to be executed for the room threads 130, thereby shifting tasks on the time axis and creating periods when tasks are not executed. Even when there are tasks that need to be processed by media, it shifts them as much as possible in the time direction to secure sleep time. The power saving function unit 200B can further enhance the power saving function of the power saving function unit 200A by putting threads related to the room to sleep as much as possible in response to the movements of avatars in the room within the application.

[0117] <Application Example 3> Figure 13C shows an application example 3 in which the real-time communication control device 100 of Figure 1 is applied as a metaverse management device. The same reference numerals are used for the same components as in Figure 13A.

[0118] The metaverse management device 100 shown in Figure 13C includes a power saving function unit 200C, which is an extension of the power saving function unit 200A shown in Figure 13A. Furthermore, the packet receiving unit 141 of the packet forwarding unit 140 includes a packet receiving queue 142.

[0119] In the "preliminary resource reservation" situation, when packet data for session updates or statistical information arrives, the packet forwarding unit 140 uses the packet reception queue 142 in the packet reception unit 141 to queue the data, thereby shifting it along the time axis and preventing sleep threads from being woken. The power saving function unit 200C can further enhance the power saving function of the power saving function unit 200A by preventing sleep threads from being woken if threads related to the room are sleepy, depending on the actions of the avatar in the room within the application.

[0120] [Effects] As described above, the real-time communication control devices 100, 100A, 100B, 100C, and 100D (Figures 1 to 6) are equipped with a session control unit (session control thread 132) and a media processing unit (packet transfer unit 140, media processing thread 133) for real-time communication, and include an integrated management unit 110 (Figures 1 to 6) that controls the idle state so that when the media processing unit (packet transfer unit 140, media processing thread 133) reserves processor resources for media processing in advance, the system transitions to a deep idle state and remains in the deep idle state so that the processor does not consume extra power.

[0121] By doing so, the real-time communication control devices 100, 100A, 100B, 100C, and 100D can sleep the threads that were started in advance as much as possible, thereby transitioning the processor on which the threads are running into a deep idle state and maintaining that state, thereby achieving power saving of processor resources.

[0122] In the real-time communication control devices 100, 100A, 100B, 100C, and 100D, a processor idle state management unit 120 (Figures 1 to 6) is further provided to configure the processor so that it enters a deep idle state when a processing thread (room thread 130) (Figures 1 to 6) is in a preliminary resource reservation state.

[0123] By doing so, the real-time communication control devices 100, 100A, 100B, 100C, and 100D can cause the processor to transition to a deep idle state (C-state) when a processing thread is in a state of reserve resource allocation, thereby achieving power saving of processor resources.

[0124] In the real-time communication control devices 100, 100A, 100B, 100C, and 100D, the processing threads (room threads 130) (Figures 1 to 6) do not perform polling monitoring so that the processor they use can maintain a sleep state.

[0125] By doing so, the real-time communication control devices 100, 100A, 100B, 100C, and 100D can extend the sleep time by ensuring that each thread performing processing does not perform unnecessary state monitoring (for example, by using interrupt processing instead of monitoring by polling) so that the processor being used can maintain the sleep state for as long as possible.

[0126] In the real-time communication control devices 100, 100A, 100B, 100C, and 100D, the processing threads (room threads 130) (Figures 1 to 6) execute tasks by shifting them to one side in the time direction if there are tasks that need to be processed, thereby securing sleep time for the processor.

[0127] By doing so, the real-time communication control devices 100, 100A, 100B, 100C, and 100D can extend the sleep time by consolidating the tasks that need to be processed in the time direction and executing the tasks accordingly.

[0128] In the real-time communication control devices 100, 100A, 100B, 100C, and 100D, the media processing unit (packet transfer unit 140) (Figures 1 to 6) has a packet receiving unit 141 (Figure 1) that processes packets that have been queued in the packet receiving queue 142 (Figure 1) when a packet arrives at the NIC 22 (Figures 1 to 6). When the media processing unit (packet transfer unit 140) is in a preliminary resource reservation state, if packet data for session updates or statistical information arrives, the packet receiving unit 141 queuing the packets to shift them along the time axis, preventing threads from being woken up while the processor is sleeping.

[0129] By doing so, the real-time communication control devices 100, 100A, 100B, 100C, and 100D can extend the sleep time by shifting packets along the time axis and preventing the processor from waking up threads during sleep.

[0130] The processing threads, as determined by the real-time communication control devices 100, 100A, 100B, 100C, and 100D (Figures 1 to 6), determine the granularity of the time axis alignment within the delay requirements and media processing bitrate range defined by the service.

[0131] By doing so, the real-time communication control devices 100, 100A, 100B, 100C, and 100D can extend the sleep time while meeting the delay requirements and media bitrate requirements specified in the service.

[0132] A metaverse management device (real-time communication control devices 100, 100A, 100B, 100C, 100D) (Figures 1 to 6) provides an application for communication between multiple users, and includes power saving function units 200A, 200B, 200C (Figures 13A, 13B, 13C) that perform power saving settings for threads related to a room according to the actions of avatars in the room within the application.

[0133] Conventional metaverse service systems did not have a function to configure power saving settings for room threads in response to avatar actions. In other words, there was no technology that focused on power saving functions in session control such as audio and video linked to avatars. In this embodiment, the metaverse management device (real-time communication control devices 100, 100A, 100B, 100C, 100D) can configure power saving settings for the corresponding room threads in conjunction with the user's actions on the metaverse.

[0134] For example, as an example of the behavior of avatars in a room within an application, if the avatar controlled by the user is far away, power-saving settings are applied to prevent the processor from consuming unnecessary power, while if the avatar controlled by the user is close, power-saving settings are not applied. The criteria for determining whether the avatar is far or close can be predetermined by setting a specific distance in advance.

[0135] Furthermore, the power-saving function units 200A, 200B, and 200C are not limited to, but for example, the media processing unit (packet transfer unit 140, media processing thread 133) and the integrated management unit 110 (Figures 1 to 6) can be used. As a result, the metaverse management device can achieve power saving of processor resources in the metaverse service system without degrading the operation of the metaverse service (such as causing operational delays).

[0136] A metaverse management device (real-time communication control devices 100, 100A, 100B, 100C, 100D) (Figures 1 to 6) that provides an application for communication between multiple users, comprises a media processing unit (packet forwarding unit 140, media processing thread 133) that provisionally creates a room when a provisional resource reservation state occurs in which a communication session for real-time communication is provisionally established when users pass each other in the vicinity, but real-time communication between users has not yet been executed, and an integrated management unit 110 (Figures 1 to 6) that, when the media processing unit provisionally secures processor resources for media processing, transitions to a deep idle state (C-state) to prevent the processor from consuming extra power, and controls the idle state so that it can remain in the deep idle state.

[0137] In metaverse use cases, even when real-time communication is not actually taking place between users, rooms are created in advance and processor resources are reserved so that nearby avatars can immediately begin conversing. When rooms are proactively created simply because users pass each other, resources (CPU cores) of the media processing unit are reserved even when real-time communication is not actually taking place between users. When there is a high density of users on the metaverse, a huge number of room creation commands will be issued.

[0138] The metaverse management device (real-time communication control devices 100, 100A, 100B, 100C, 100D) can shift the processor on which the threads operate into a deep idle state by keeping the threads that were started in advance to sleep as much as possible, and maintain that state, thereby achieving power saving of processor resources. Therefore, it is suitable for application to sleep control of resource reservation functions such as media processing in session control of audio and video in conjunction with avatars in a virtual space.

[0139] Furthermore, among the processes described in the above embodiments and application examples, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. Moreover, each component of each illustrated device is a functional concept and does not necessarily have to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0140] Furthermore, each of the above configurations, functions, processing units, and processing means may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. Alternatively, each of the above configurations and functions may be implemented in software that allows the processor to interpret and execute programs that implement each function. Information such as programs, tables, and files that implement each function can be stored in memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC (Integrated Circuit) card, an SD (Secure Digital) card, or an optical disc.

[0141] 2 Metaverse (Virtual Space) 3 Avatar 20 Hardware (HW) 21 Processor 22 NIC 30 OS 40 User Space 100, 100A, 100B, 100C, 100D Real-time communication control device (metaverse management device) 110 Integrated management unit (power saving function unit) 120 Processor idle state management unit (power saving function unit) 130 Room thread (thread for media processing, etc.) 131 Queuing 132 Session control thread (session control unit) 133 Media processing thread (thread for processing) (power saving function unit) 140 Packet forwarding unit (media processing unit) (power saving function unit) 141 Packet receiving unit 142 Packet receiving queue 150 Processor idle state setting unit (power saving function unit) 200A, 200B, 200C Power saving function unit 1000 Real-time communication control system

Claims

1. A real-time communication control device comprising a session control unit and a media processing unit for real-time communication, wherein the media processing unit comprises an integrated management unit that controls the idle state so that when it reserves processor resources for media processing in advance, the processor transitions to a deep idle state and remains in the deep idle state so as not to consume extra power.

2. The real-time communication control device according to claim 1, further comprising a processor idle state management unit that configures the processor to enter a deep idle state when a processing thread is in a reserve resource reservation state.

3. The real-time communication control device according to claim 1, characterized in that the processing thread does not perform monitoring by polling so that the processor it uses can maintain a sleep state.

4. The real-time communication control device according to claim 1, characterized in that, if a processing thread has tasks that need to be processed, it executes those tasks by shifting them to one side in the time direction to secure sleep time for the processor.

5. The media processing unit has a packet receiving unit that processes packets that have been queued in a packet receiving queue when a packet arrives at the NIC (Network Interface Card), and when the media processing unit is in a preliminary resource reservation state, if packet data for session updates or statistical information arrives, the packet receiving unit aligns the packets in the time axis direction by queuing, so as not to wake up threads while the processor is sleeping, as described in claim 1.

6. The real-time communication control device according to claim 4 or 5, characterized in that the processing thread determines the granularity of the time axis alignment within the range of the delay requirements and media processing bitrate defined in the service.

7. A metaverse management device that provides an application for communication between multiple users, characterized in that it includes a power saving function unit that performs power saving settings for threads related to a room in accordance with the actions of avatars in a room within the application.

8. A program for causing a computer to function as a real-time communication control device as described in claim 1.

Images