Information processing device, learning method, and program
Patent Information
- Application Number
- PCT/JP2025/005486
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-02-18
- Publication Date
- 2026-08-27
Smart Images

Figure JP2025005486_27082026_PF_FP_ABST
Abstract
Description
Information processing device, learning method, and program
[0001] This invention relates to the technology of video distribution services, such as web conferencing services.
[0002] With the increase in remote work and other factors, the use of web conferencing services has increased. To ensure that users can use the service comfortably, it is necessary to maintain an appropriate quality of experience for the user. In order to maintain the quality of experience for the user, it is effective to appropriately control the video bitrate of the video transmitted from the client in a web conference, and video bitrate control technologies have been proposed in the past (Non-Patent Documents 1 and 2).
[0003] The technology disclosed in Non-Patent Documents 1 and 2 searches for a video bitrate such that the future perceived quality exceeds the target perceived quality. Specifically, it considers the reception quality (bitrate, resolution, frame rate) over the past T1 seconds and searches for and determines the necessary video quality for the future T2 seconds in order to achieve the target perceived quality. At this time, it is assumed that the same video bitrate continues for T2 seconds. In this way, it is possible to determine a future bitrate that will achieve the target perceived quality, and the client can maintain appropriate perceived quality by transmitting a bitrate that matches it.
[0004] M. Yokota and K. Yamagishi, "Quality-based Video Bitrate Control for WebRTC-based Teleconference Services", IS&T International Symposium on Electronic Imaging 2022M. Yokota and K. Yamagishi, "Quality and Transferred Data based Video Bitrate Control Method for Web-Conferencing," IEICE Transactions on Communications, vol.E107-B, No.1, pp.272-285, Jan. 2024.
[0005] The technologies disclosed in Non-Patent Documents 1 and 2 allow for the determination of a video bitrate that maintains appropriate perceived quality. However, when instructing the client on the determined bitrate, some web conferencing systems may not be able to immediately control the system according to the instructed bitrate.
[0006] For example, the bitrate may drop below the instructed bitrate at the moment of instruction, and then gradually improve. In such cases, the prior art disclosed in Non-Patent Documents 1 and 2 searches for the bitrate based on the assumption that the determined video bitrate (instructed bitrate) will continue for T2 seconds in the future. As a result, the control may not be as expected, and the target perceived quality may not always be maintained.
[0007] This invention has been made in view of the above points, and aims to provide a technique for calculating time-series data of the actual bitrate relative to the instructed bitrate in a client transmitting video.
[0008] According to the disclosed technology, an information processing device is provided for learning a model for calculating time-series data of bitrate when transmitting video, comprising: an input unit that collects an instructed bitrate instructed to a client transmitting video and time-series data of the actual bitrate relative to the instructed bitrate; and a calculation unit that determines the parameters of the model so as to minimize the error between the time-series data of the bitrate output from the model that has the instructed bitrate as input and the time-series data of the actual bitrate.
[0009] According to the disclosed technology, a technique is provided for calculating time-series data of the actual bitrate relative to the instructed bitrate at a client transmitting video.
[0010] This figure shows an example of the system configuration in an embodiment of the present invention. This figure shows an example of the configuration of the information processing device 100. This figure shows an example of the configuration of the control device 50. This is a flowchart for explaining the operation of the bitrate error learning device 10. This figure explains the initial value and the fluctuation model. This figure explains the operation of the instruction bitrate correction device 20. This figure shows an example of the hardware configuration of the device.
[0011] Hereinafter, embodiments of the present invention (this embodiment) will be described with reference to the drawings. The embodiments described below are merely examples, and the embodiments to which the present invention is applied are not limited to the embodiments described below.
[0012] Although the following explanation focuses on web conferencing services, the technology according to the present invention is applicable to video distribution services in general, not limited to web conferencing services. In this application, "video distribution service" refers to any service in which a client transmits video to a server, and the server transmits that video to another client. Furthermore, a system having such a client and such a server is called a video distribution system. A web conferencing service is an example of a video distribution service, and a web conferencing system is an example of a video distribution system.
[0013] (Outline of the Embodiment) The web conferencing system targeted in this embodiment is assumed to be a system that performs video communication between web conferencing clients via a server, such as WebRTC-SFU. Hereinafter, a web conferencing client will be referred to as a client.
[0014] In this web conferencing system, a client that transmits video (transmitting client) sends video to the server at a certain bitrate, and the server selects a bitrate and sends the video to another client. The instructed bitrate described below is the bitrate that the transmitting client instructs the server to use when transmitting video.
[0015] In this embodiment, in order to solve the aforementioned problems, when controlling the client's transmission bitrate according to the reception quality of the client participating in the web conference, the bitrate control is implemented to maintain the optimal perceived quality for the receiving user by taking into account the error between the instructed bitrate instructed to the client and the client's actual transmission bitrate.
[0016] In this embodiment, the bitrate error learning device 10 learns the error between the actual bitrate and the instructed bitrate at the client and its fluctuation trend. Subsequently, the instructed bitrate correction device 20 searches for instructed bitrate candidates based on the learning results from the bitrate error learning device 10, for example, based on the methods disclosed in Non-Patent Documents 1 and 2, and determines a future video bitrate that would exceed the target quality.
[0017] Specifically, the bitrate error learning device 10 collects data on actual bitrates relative to past instructed bitrates and learns the error and fluctuation trends of the actual bitrates relative to the instructed bitrates. Based on this error and fluctuation data, the instructed bitrate correction device 20 corrects the bitrate for the future T2 seconds as described in Non-Patent Documents 1 and 2 to the bitrate that is expected to be actually transmitted. Based on the corrected bitrate, the video bitrate control device 30 searches for an appropriate bitrate (instructed bitrate) to achieve video bitrate control that maintains optimal perceived quality.
[0018] The system configuration and operation of this embodiment will be described in detail below.
[0019] (System Configuration Example) Figure 1 shows an example of the system configuration for realizing the correction of the candidate instructed bitrate in this embodiment.
[0020] As shown in Figure 1, this system includes a bitrate error learning device 10 and an indicated bitrate correction device 20. Figure 1 also shows a video bitrate control device 30 and a web conferencing server 40.
[0021] The video bitrate control device 30 corresponds to the controller disclosed in Non-Patent Documents 1 and 2. Multiple clients (not shown) are connected to the web conferencing server 40, which is the server in the web conferencing system.
[0022] The bitrate error learning device 10 collects logs of the instructed bitrate for the client from the video bitrate control device 30. Furthermore, the bitrate error learning device 10 collects time-series data of the client's actual transmission bitrate relative to the instructed bitrate from the web conferencing server 40 as historical values. Based on this historical information, the bitrate error learning device 10 learns a model that models the error and fluctuation trends of the actual transmission bitrate relative to the instructed bitrate.
[0023] The instruction bitrate correction device 20 uses the model learned by the bitrate error learning device 10 to correct the instruction bitrate candidate being searched for by the video bitrate control device 30, and returns the corrected value to the video bitrate control device 30.
[0024] (Example of device configuration) The bitrate error learning device 10 and the indicated bitrate correction device 20 may each be standalone devices, or a standalone device may be configured that includes the functions of both the bitrate error learning device 10 and the indicated bitrate correction device 20.
[0025] When the bitrate error learning device 10 and the instructed bitrate correction device 20 are separate devices, the bitrate error learning device 10 and the instructed bitrate correction device 20 each have the configuration of the information processing device 100 shown in Figure 2.
[0026] When the information processing device 100 functions as a bitrate error learning device 10, it may be called a learning device. When the information processing device 100 functions as an indicated bitrate correction device 20, it may be called a correction device.
[0027] As shown in Figure 2, the information processing device 100 includes an input unit 110, a calculation unit 120, an output unit 130, and a data storage unit 140.
[0028] The input unit 110 receives the information necessary for the bitrate error learning / instructed bitrate correction calculation. The input information is stored in the data storage unit 140. The calculation unit 120 uses the information read from the data storage unit 140 to perform bitrate error learning / instructed bitrate correction. The output unit 130 outputs the calculation result from the calculation unit 120.
[0029] Figure 3 shows an example of the configuration of a control device 50, which is a standalone device that includes the functions of the bitrate error learning device 10 and the instructed bitrate correction device 20.
[0030] As shown in Figure 3, the control device 50 includes a bitrate error learning unit 11 and an indicated bitrate correction unit 21. The operation of the bitrate error learning unit 11 is the same as the operation of the bitrate error learning device 10, and the operation of the indicated bitrate correction unit 21 is the same as the operation of the indicated bitrate correction device 20.
[0031] The following describes in detail the operation of the bitrate error learning device 10 and the instructed bitrate correction device 20.
[0032] (Operation of the bitrate error learning device 10) The operation of the bitrate error learning device 10 will be explained according to the procedure in the flowchart of Figure 4.
[0033] <S1 (Step 1): Data Collection> The bitrate error learning device 10 collects time-series data of the instructed bitrate for a client when the video bitrate control device 30 instructs one or more clients among the multiple clients participating in the web conference to change the bitrate.
[0034] In Figure 1, the video bitrate control device 30 issues a bitrate instruction (notification of the instructed bitrate) to the web conferencing server 40, but the final bitrate instruction is received by the client transmitting the video. Alternatively, the video bitrate control device 30 may also notify the client of the instructed bitrate.
[0035] Furthermore, the bitrate error learning device 10 collects time-series data of the actual transmission bitrate (referred to as the actual transmitted bitrate) with respect to the instructed bitrate at the client from the Web conference server 40.
[0036] As will be described later, since models for clients instructed with a bitrate and models for clients not receiving a bitrate instruction are constructed, the collection of the time-series data of the above-described actual transmitted bitrates is performed for each of the clients instructed with a bitrate and the clients not receiving a bitrate instruction.
[0037] <S2: Model determination> The bitrate error learning device 10 determines a calculation model (referred to as an initial bitrate model) for calculating (estimating) the actual transmitted bitrate immediately after the instruction and a calculation model (referred to as a bitrate variation model) for calculating (estimating) the actual transmitted bitrate after a certain time has elapsed from the time of the instruction from the collected time-series data.
[0038] The bitrate error learning device 10 constructs each of the above models for clients that have received a bitrate instruction and clients that have not received an instruction. That is, in the present embodiment, it is assumed that there are patterns in which the tendency of variation is different between the case of receiving an instruction and the case of not receiving an instruction, and thus it is assumed that the models are divided according to whether an instruction has been received.
[0039] Depending on the implementation, there may be cases where the bitrate does not change unless the client receives an instruction. In that case, a model in which the bitrate for clients not receiving an instruction is the same as the past instruction is constructed as the model for clients not receiving an instruction.
[0040] FIG. 5 shows an image of the initial bitrate model and the bitrate variation model. The vertical axis in FIG. 5 is the bitrate when the client transmits video, and the horizontal axis is time. It is assumed that the instructed bitrate is instructed to the client at the "current time".
[0041] The initial bitrate model is a model that estimates the initial value (1) at the current time. The bitrate fluctuation model is a model that estimates the time fluctuation of the bitrate at the current time, as shown in "Fluctuation Model (2)".
[0042] Examples of specific model determination methods (model learning methods) are explained below.
[0043] The bitrate immediately after the instruction (initial value) is determined from the instruction bitrate, or from the instruction bitrate and the actual transmitted bitrate immediately before the instruction. Therefore, the bitrate error learning device 10 finds the initial bitrate model by searching for coefficients (a, b, c) that minimize the error with respect to the collected data, using equation (1) or equation (2) below.
[0044] Initial value i = a i *Instructed bitrate (1) Initial value i = b i * Instruction bitrate + c i * Actual transmission bitrate immediately prior to transmission i (2) The above i is an index that distinguishes between a model for clients that have been given a bitrate specification and a model for clients that have not been given a bitrate specification.
[0045] The bitrate error learning device 10 performs the above process for both clients that have received instructions to change the bitrate and clients that have not received instructions to change the bitrate. As a result, initial bitrate models are constructed for clients that have received bitrate instructions and for clients that have not received bitrate instructions. At this time, the same value is used for the instructed bitrate in both of these models.
[0046] That is, the initial bit rate model in the present embodiment is the formula (1) or formula (2) having coefficients (parameters) obtained by search. The bit rate error learning device 10 determines the coefficients so that the error between the output (left side of the formula) from the initial bit rate model (right side of the formula) with the "indicated bit rate, or the indicated bit rate and the actual transmission bit rate immediately before the indication" and the actual initial value is minimized.
[0047] Subsequently, the bit rate error learning device 10 determines the bit rate fluctuation model from the time series fluctuation data of the actual transmission bit rate. Specifically, the bit rate error learning device 10 searches for coefficients (x, y, z, a, b) that minimize the error with respect to the collected time series data for the following formula (3), formula (4) or formula (5).
[0048] Actual transmission bit rate i = x i * Time + Initial value i (3) Actual transmission bit rate i = y i 時刻 + Initial value i - z i (4) Actual transmission bit rate i = log(a i * Time) + Initial value i - b i (5) Here, the time indicates the elapsed time starting from the bit rate indication time. This process is also executed for each of the client that has received the bit rate change instruction and the client that has not received the bit rate change instruction. Thereby, for the bit rate fluctuation model, a model for the client to which the bit rate is indicated and a model for the client that has not received the bit rate indication are respectively constructed. The above i is a subscript for identifying the model for the client to which the bit rate is indicated and the model for the client that has not received the bit rate indication.
[0049] In other words, the bitrate fluctuation model in this embodiment is equation (3), equation (4), or equation (5), which has coefficients (parameters) obtained by the search. The bitrate error learning device 10 determines the coefficients so as to minimize the error between the output (left side of the equation) from the bitrate fluctuation model (right side of the equation) to which time and initial values have been input, and the time-series data of the actual transmitted bitrate.
[0050] By using the learned initial bitrate model and bitrate fluctuation model determined by the bitrate error learning device 10, it becomes possible to calculate the actual transmission bitrate for the instructed bitrate. The learned initial bitrate model and bitrate fluctuation model are input to the instructed bitrate correction device 20.
[0051] It should be noted that the "initial bitrate model and bitrate fluctuation model" can be considered as a single model. Furthermore, the "initial bitrate model and bitrate fluctuation model" are not limited to models using the above-mentioned formulas; for example, they could be neural network models.
[0052] In the configuration of the information processing device 100 described above, the input unit 110 collects the instructed bitrate given to the client transmitting the video, and time-series data of the actual bitrate relative to the instructed bitrate. The collected data is stored in the data storage unit 140.
[0053] The calculation unit 120 uses the data read from the data storage unit 140 to determine the parameters of the model such that the error between the time-series data of the bitrate output from the model to which the instructed bitrate was input and the time-series data of the actual bitrate is minimized. If the model is a neural network, the parameters can be determined using, for example, backpropagation.
[0054] (Operation of the instruction bitrate correction device 20) The instruction bitrate correction device 20 corrects the instruction bitrate candidates used in the video bitrate control device 30 using the initial bitrate model and bitrate fluctuation model determined by the bitrate error learning device 10.
[0055] The operation of the instruction bitrate correction device 20 will be explained according to the procedure in the flowchart of Figure 6. This process is performed for each client. Also, S13 is an operation by the video bitrate control device 30.
[0056] <S11: Acquisition of Candidate Instructed Bitrate> The video bitrate control device 30 searches for an instruction bitrate that exceeds the appropriate perceived quality for the client. The instruction bitrate correction device 20 acquires the candidate instruction bitrate obtained during this instruction bitrate search from the video bitrate control device 30.
[0057] <S12: Time-series data calculation> The indicated bitrate correction device 20 uses the above-mentioned candidate indicated bitrate as the indicated bitrate and calculates (estimates) time-series data of the actual bitrate for that indicated bitrate by utilizing the initial bitrate model and the bitrate fluctuation model.
[0058] In other words, the instruction bitrate correction device 20 calculates an initial value by using an instruction bitrate candidate as the "instruction bitrate" in the initial bitrate model described above, and then calculates (estimates) the actual bitrate time series data by inputting this initial value into the bitrate fluctuation model.
[0059] The instruction bitrate correction device 20 transmits the time-series data of the actual bitrate calculated as described above to the video bitrate control device 30 as an instruction bitrate candidate correction value.
[0060] As mentioned above, the "initial bitrate model and bitrate fluctuation model" can be considered as a single model.
[0061] In the configuration of the information processing device 100 described above, the input unit 110 acquires a candidate for the instructed bitrate. The acquired candidate for the instructed bitrate is stored in the data storage unit 140. The calculation unit 120 reads the candidate for the instructed bitrate from the data storage unit 140 and inputs the candidate for the instructed bitrate into the model described above, thereby calculating (estimating) time-series data of the actual bitrate for the candidate for the instructed bitrate. The output unit 130 transmits the time-series data to the video bitrate control device 30.
[0062] <S13: Bitrate Search> The video bitrate control device 30 searches for an instruction bitrate by calculating whether it is possible to maintain appropriate perceived quality based on the bitrate time-series data received from the instruction bitrate correction device 20. Steps S11 to S13 may be repeated for the searched candidate instruction bitrates.
[0063] The video bitrate control device 30 transmits the instructed bitrate determined by the search to the web conferencing server 40. The web conferencing server 40 transmits the instructed bitrate to the target client.
[0064] Furthermore, the video bitrate control device 30 may choose not to transmit the instructed bitrate obtained through the search for the target client to the web conferencing server 40 if the instructed bitrate is the same as the previous instructed bitrate.
[0065] <For clients where no bitrate has been specified> The above processing will also be performed for each client where no bitrate has been specified. An example of the processing for each client where no bitrate has been specified (clients where the candidate bitrate is the same as the previously specified bitrate and no instruction is needed) is as follows.
[0066] In S12, the instruction bitrate correction device 20 uses, for example, the client's previous instruction bitrate as the "instruction bitrate" in a model for clients that have not received a bitrate instruction, and uses that model to calculate (estimate) the time-series data of the client's actual bitrate. In other words, the initial bitrate model and bitrate fluctuation model used here are models for clients that have not received an instruction.
[0067] The instruction bitrate correction device 20 transmits the time-series data of the actual bitrate calculated as described above to the video bitrate control device 30.
[0068] In S13, the video bitrate control device 30 calculates whether it is possible to maintain appropriate perceived quality based on the bitrate time-series data received from the instruction bitrate correction device 20, and searches for an instruction bitrate for the client.
[0069] The video bitrate control device 30 transmits the instructed bitrate obtained through the search to the web conferencing server 40. The web conferencing server 40 transmits the instructed bitrate to the target client.
[0070] Furthermore, the video bitrate control device 30 may choose not to transmit the instructed bitrate obtained through the search for the target client to the web conferencing server 40 if the instructed bitrate is the same as the previous instructed bitrate.
[0071] (Example Hardware Configuration) Any of the devices described in this embodiment (bitrate error learning device, instructed bitrate correction device, information processing device, control device, etc.) can be realized, for example, by having a computer run a program. This computer may be a physical computer or a virtual machine on the cloud.
[0072] In other words, the device can be realized by using hardware resources such as the CPU and memory built into a computer to execute a program corresponding to the processing performed by the device. The program can be recorded on a computer-readable recording medium (such as portable memory), saved, and distributed. It can also be provided via a network, such as the Internet or email.
[0073] Figure 7 shows an example of the hardware configuration of the computer described above. The computer in Figure 7 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., all of which are interconnected by bus B. The computer may also be equipped with a GPU.
[0074] The program that enables processing on the computer is provided on a recording medium 1001, such as a CD-ROM or memory card. When the recording medium 1001 containing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001; it may also be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files and data.
[0075] The memory device 1003 reads and stores a program from the auxiliary storage device 1002 when a program startup command is received. The CPU 1004 implements the functions related to the memory device 1003 according to the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) etc., based on a program. The input device 1007 consists of a keyboard and mouse, buttons, or a touch panel, etc., and is used to input various operation commands. The output device 1008 outputs the calculation results.
[0076] (Summary of Embodiments) As described above, in the technology described in this embodiment, the bitrate error learning device 10 models the error between past instructed bitrates and the actual bitrate from past control data. Furthermore, when the video bitrate control device 30 searches for an appropriate bitrate, the instructed bitrate correction device 20 corrects the instructed bitrate to the actual bitrate using the learned model. This makes it possible to achieve bitrate control that maintains optimal perceived quality.
[0077] More specifically, the bitrate error learning device 10 uses past instructed bitrates and actual bitrates to determine an initial bitrate model that estimates the initial bitrate after the bitrate instruction, and a bitrate change model that estimates the degree of change over time.
[0078] (Effects of the technology according to the embodiment) The technology described in this embodiment makes it possible to calculate time-series data of the actual bitrate relative to the instructed bitrate in a client transmitting video.
[0079] When controlling the bitrate in video distribution services such as web conferencing services, the technology according to this embodiment can be used to capture appropriate bitrate changes relative to the instructed bitrate and estimate future perceived quality, thereby enabling control that maintains appropriate perceived quality.
[0080] The following additional information is disclosed regarding the embodiments described above.
[0081] <Notes> (Note 1) An information processing device for learning a model for calculating time-series data of bitrate when transmitting video, comprising: an input unit that collects an instructed bitrate instructed to a client transmitting video and time-series data of the actual bitrate for the instructed bitrate; and a calculation unit that determines the parameters of the model so as to minimize the error between the time-series data of bitrate output from the model that has been input with the instructed bitrate and the time-series data of the actual bitrate. (Note 2) An information processing device for calculating time-series data of bitrate when transmitting video using a learned model, wherein the model is a model that has been learned so as to minimize the error between the time-series data of bitrate output from the model that has been input with the instructed bitrate and the time-series data of the actual bitrate for the instructed bitrate, comprising: an input unit that acquires an instructed bitrate candidate; and a calculation unit that calculates time-series data of the actual bitrate for the instructed bitrate candidate by inputting the instructed bitrate candidate into the model. (Note 3) A learning method executed by an information processing device that learns a model for calculating time-series data of bitrate when transmitting video, comprising the steps of: collecting an instructed bitrate instructed to a client transmitting video and time-series data of the actual bitrate relative to the instructed bitrate; and determining the parameters of the model such that the error between the time-series data of bitrate output from the model that has the instructed bitrate as input and the time-series data of the actual bitrate is minimized. (Note 4) A non-temporary storage medium storing a program for causing a computer to function as a part of the information processing device described in Note 1 or 2.
[0082] Although this embodiment has been described above, the present invention is not limited to this specific embodiment, and various modifications and changes are possible within the scope of the gist of the invention as described in the claims.
[0083] 10 Bitrate error learning device 20 Instruction bitrate correction device 30 Video bitrate control device 40 Web conferencing server 100 Information processing device 110 Input unit 120 Calculation unit 130 Output unit 140 Data storage unit 50 Control device 11 Bitrate error learning unit 21 Instruction bitrate correction unit 1000 Drive device 1001 Recording medium 1002 Auxiliary storage device 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input device 1008 Output device
Claims
1. An information processing device for learning a model for calculating time-series data of bitrate when transmitting video, comprising: an input unit that collects an instructed bitrate instructed to a client transmitting video and time-series data of the actual bitrate relative to the instructed bitrate; and a calculation unit that determines the parameters of the model so as to minimize the error between the time-series data of bitrate output from the model that has the instructed bitrate as input and the time-series data of the actual bitrate.
2. An information processing device for calculating time-series data of bitrate when transmitting video using a trained model, wherein the model is trained to minimize the error between time-series data of bitrate output from the model when an instruction bitrate is input and time-series data of the actual bitrate for the instruction bitrate, and the information processing device comprises an input unit for acquiring an instruction bitrate candidate and a calculation unit for calculating time-series data of the actual bitrate for the instruction bitrate candidate by inputting the instruction bitrate candidate into the model.
3. A learning method executed by an information processing device that learns a model for calculating time-series data of bitrate when transmitting video, comprising the steps of: collecting an instructed bitrate instructed to a client transmitting video and time-series data of the actual bitrate relative to the instructed bitrate; and determining the parameters of the model such that the error between the time-series data of bitrate output from the model that has the instructed bitrate as input and the time-series data of the actual bitrate is minimized.
4. A program for causing a computer to function as a component of the information processing apparatus described in claim 1 or 2.