Client device and information system

Abstract

A client device (10) is provided with: an information reception unit (12) that receives, from a cloud server (20), display information required to display characters / images to be displayed and background data rendered by the cloud server (20); and a content generation unit (13) that generates, from character / image data to be displayed acquired on the basis of the display information received by the information reception unit (12) and the received background data, a content to be displayed on the basis of the display information.

Description

Client Device and Information System

[0001] The present disclosure relates to a client device and an information system including a client device and a cloud server.

[0002] In conventional cloud rendering, data transfer typically involves using lossy compression techniques to reduce transmission costs. In actual processing, the cloud server typically embeds text and moving images into a three-dimensional background, and then renders the three-dimensional background image, text, and moving images together, outputting them as a single video data set to a client device (such as a mobile terminal or PC).

[0003] Regarding such cloud rendering, the technology disclosed in Patent Document 1 below has the primary purpose of load distribution, in which in addition to the cloud server, many client devices participate and the rendering task is load-distributed across many processing entities (cloud servers, client devices).

[0004] Japanese Patent Application Laid-Open No. 2015-215908

[0005] However, in the conventional cloud rendering described above, when transmitting data using the above compression method, in areas where there is a large difference in pixel values ​​between the pixels of the character string and the background (for example, areas where black character strings are displayed on a white background), there is a problem that the character becomes blurred after decoding and the compression efficiency cannot be improved, which may result in a decrease in the quality of the generated content and a decrease in the user experience (UX). However, Patent Document 1 does not take the above problem into consideration and does not make any proposals to solve the problem.

[0006] Therefore, an object of the present disclosure is to solve the above problems, suppress degradation in the quality of generated content, and suppress degradation in user experience.

[0007] The client device according to the present disclosure includes an information receiving unit that receives, from a cloud server, display information required for displaying characters and images to be displayed and background data rendered by the cloud server, and a content generating unit that generates content to be displayed based on the display information from the characters and image data to be displayed that are obtained based on the display information received by the information receiving unit and the received background data.

[0008] According to the present disclosure, by avoiding the collective rendering of background data that requires pre-rendering and other data (text and image data), the generation process is distributed between the cloud server and the client device, thereby solving the conventional problems caused by collective rendering and suppressing degradation in the quality of the generated content, thereby realizing the delivery of high-quality content without impairing the user experience.

[0009] It is a functional block diagram of an information system including a client device and related devices according to an embodiment. It is a flow diagram showing processing executed in the information system of Figure 1. It is a diagram showing an example of content to be displayed. It is a diagram showing an example of the hardware configuration of a client device.

[0010] Hereinafter, an embodiment of a client device according to the present disclosure and an information system including the client device will be described with reference to the drawings.

[0011] As shown in FIG. 1 , the information system 1 in this embodiment includes a cloud server 20 that performs cloud rendering and a client device 10. The information system 1 also includes, although not necessarily included, a platform that includes an image database 31 that stores various image data and is composed of multiple servers (e.g., cache servers) that distribute image data in response to external requests. In this embodiment, a Content Delivery Network (hereinafter referred to as "CDN") 30 is used as the platform. This CDN refers to a mechanism for quickly delivering various content on a website to end users (here, client devices 10) by using a platform composed of multiple cache servers and the like. Note that "images" in this embodiment broadly include both moving images and still images, and the CDN 30 stores and manages various moving image data and still image data using the image database 31.

[0012] The cloud server 20 is configured by at least one server located on the so-called cloud side, and includes a rendering execution unit 21 and an information transmission unit 22 as components for performing the functions related to the present disclosure.

[0013] Of these, the rendering execution unit 21 is a functional unit that renders background data that requires pre-rendering, for example, in response to a request from the client device 10. In this case, "background data" refers to all data (mainly background display data) that represents display objects other than characters and images and that requires pre-rendering.

[0014] The information sending unit 22 is a functional unit that transmits to the client device 10 background data obtained by rendering by the rendering execution unit 21 and display information (described later) required for displaying the characters and images to be displayed. The "display information" here includes character information indicating the characters to be displayed, character display control information related to display control of the characters, and storage destination information for the image to be displayed (e.g., identification information for the image data and identification information for the CDN 30 storing the image data), and image display control information related to display control of the image. In this embodiment, the "character display control information" is character markup information, which is information regarding character display control described in a markup language. The character markup information describes, for example, information regarding the display position and timing of the characters, information regarding character decoration such as font and size, etc. The "image display control information" is image markup information, which is information regarding image display control described in a markup language. The image markup information describes, for example, information regarding the display position and timing of the image, etc.

[0015] On the other hand, the client device 10 is composed of various information processing devices such as mobile terminals (smartphones, tablet terminals, mobile phones, etc.), portable or stationary personal computers, and is equipped with an acquisition request unit 11, an information receiving unit 12, a content generation unit 13, and an output unit 14 as components for performing the functions related to the present disclosure.

[0016] Of these, the acquisition request unit 11 is a functional unit that requests the cloud server 20 to acquire the background data described above and the display information required to display the character image to be displayed.

[0017] The information receiving unit 12 is a functional unit that receives the background data obtained by rendering and the above-mentioned display information transmitted from the information transmitting unit 22 of the cloud server 20 .

[0018] The content generation unit 13 is a functional unit that generates content to be displayed based on the display information from the character and image data acquired based on the received display information and the received background data. Specifically, the content generation unit 13 generates character data to be displayed based on the character information, requests the CDN 30 to acquire image data to be displayed based on storage destination information included in the display information, and acquires the image data from the CDN 30. Furthermore, the content generation unit 13 generates content to be displayed by synchronizing and combining the character data and image data with the background data based on the character markup information and image markup information included in the display information.

[0019] The output unit 14 is a functional unit that outputs the content generated by the content generation unit 13. Note that "output" can take various forms, such as display output, print output, or data transmission to an external device of the client device 10.

[0020] The processing executed in the information system 1 of FIG. 1 will be described below with reference to the flow chart of FIG.

[0021] First, the acquisition request unit 11 of the client device 10 transmits an acquisition request for background data that requires pre-rendering and display information (including the aforementioned character information, character markup information, image storage destination information, and image markup information) to the cloud server 20 (step S1). Here, as an example, it is assumed that a moving image representing people P1 and P2, a character string P3, and a background P4 (e.g., multiple trees that appear behind the people in a three-dimensional space, the sky behind them, etc.) shown in Fig. 3 is to be output, and data for displaying the background P4 corresponds to the background data that requires pre-rendering, the images of the people P1 and P2 correspond to the image to be displayed, and the characters of the character string P3 correspond to the characters to be displayed.

[0022] In the cloud server 20 that receives the request, the rendering execution unit 21 performs rendering of the background data (step S2), and the information transmission unit 22 transmits the background data obtained by the rendering in step S2 and the requested display information to the client device 10 (step S3).

[0023] In the client device 10, the information receiving unit 12 receives the background data and display information transmitted in step S3 (step S4), and sends the received data to the content generating unit 13. The content generating unit 13 references the "image storage destination information (such as identification information for the image data and identification information for the CDN that stores and manages the image data)" contained in the received display information, and transmits a request to the CDN 30, which is the storage destination, to acquire the image data to be displayed (step S5). Furthermore, the content generating unit 13 generates character data to be displayed based on the "character information" contained in the received display information (step S6).

[0024] On the other hand, upon receiving the acquisition request in step S5, the CDN 30 reads out the image data to be displayed from the image database 31 and transmits it to the content generator 13 (step S7).

[0025] Then, when the content generation unit 13 receives image data to be displayed from the CDN 30 (step S8), it generates content to be displayed by synchronizing and combining the character data generated in step S6 and the image data received in step S8 with the background data received in step S4 (step S9). In step S9, the content generation unit 13 synchronizes and combines the character data with the background data using character decoration, character display position and timing, etc. based on the "character markup information" included in the display information. Also, the content generation unit 13 synchronizes and combines the image data with the background data using the image display position and timing, etc. based on the "image markup information" included in the display information. The content generated in this manner is output by the output unit 14 (step S10).

[0026] In the embodiment described above, the cloud server 20 generates only background data that requires prior rendering by rendering, and the client device 10 obtains or generates other data (text and image data) from the CDN 30, thereby distributing the processing. This solves the problems that arise when a conventional cloud server renders background data and text and image data together (i.e., (1) the problem that characters become blurred after decoding in areas where there is a large difference in pixel values ​​between the text and background, and (2) the problem that compression efficiency cannot be improved, resulting in a decrease in the quality of the generated content and a decrease in UX), and it is possible to suppress the decrease in the quality of the generated content and deliver high-quality content without compromising UX.

[0027] Specifically, the content generation unit 13 of the client device 10 generates content to be displayed by synchronizing and combining character data with background data using character decoration, character display position and timing, etc. based on the "character markup information" included in the display information, and synthesizing image data with background data using image display position and timing, etc. based on the "image markup information" included in the display information. In this way, it is possible to appropriately synchronize and combine character and image data with background data based on the character markup information and image markup information.

[0028] Furthermore, the content generation unit 13 of the client device 10 requests the CDN 30 to acquire the image data to be displayed based on the "storage destination information" included in the display information, and acquires the image data. Therefore, the client device 10 does not need to store and manage large amounts of image data in advance, and this can be realized with an inexpensive and simple configuration.

[0029] [Explanation of Terms, Explanation of Hardware Configuration (FIG. 4), etc.] The block diagrams used in the description of the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method for realizing each functional block is not particularly limited. That is, each functional block may be realized using a single device that is physically or logically coupled, or may be realized using two or more physically or logically separated devices that are directly or indirectly connected (e.g., wired, wireless, etc.) and these multiple devices. The functional block may also be realized by combining software with the single device or multiple devices.

[0030] Functions include, but are not limited to, judgment, determination, judgment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, election, establishment, comparison, assumption, expectation, regard, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, and assignment. For example, a functional block (component) that performs transmission functions is called a transmitting unit, transmitter, etc. As mentioned above, there are no particular limitations on how these functions are implemented.

[0031] For example, a client device according to an embodiment of the present disclosure may function as a computer that executes the processing of the present disclosure. Fig. 4 is a diagram illustrating an example of a hardware configuration of a client device 10 according to an embodiment of the present disclosure. The above-described client device 10 may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like. Note that the cloud server 20 and the CDN 30 illustrated in Fig. 1 may also be configured in the same manner as the client device 10.

[0032] In the following description, the term "apparatus" can be interpreted as a circuit, a device, a unit, etc. The hardware configuration of the client apparatus 10 may be configured to include one or more of the apparatuses shown in the drawings, or may be configured to exclude some of the apparatuses.

[0033] Each function of the client device 10 is realized by loading specified software (programs) onto hardware such as the processor 1001 and memory 1002, causing the processor 1001 to perform calculations, control communication via the communication device 1004, and control at least one of reading and writing data in the memory 1002 and storage 1003.

[0034] The processor 1001 controls the entire computer by running, for example, an operating system, and may be configured as a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc.

[0035] The processor 1001 also reads programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes in accordance with these. The programs used are those that cause a computer to execute at least some of the operations described in the above-described embodiments. While the various processes have been described as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The programs may be transmitted from a network via a telecommunications line.

[0036] The memory 1002 is a computer-readable recording medium and may be configured by, for example, at least one of a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a random access memory (RAM), etc. The memory 1002 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 1002 can store executable programs (program codes), software modules, etc. for implementing a wireless communication method according to an embodiment of the present disclosure.

[0037] Storage 1003 is a computer-readable recording medium, and may be composed of at least one of, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, etc. Storage 1003 may also be referred to as an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, a server, or other appropriate medium including at least one of memory 1002 and storage 1003.

[0038] The communication device 1004 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 1004 may be configured to include a high-frequency switch, a duplexer, a filter, a frequency synthesizer, etc. to realize at least one of frequency division duplex (FDD) and time division duplex (TDD).

[0039] The input device 1005 is an input device (e.g., a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives input from the outside. The output device 1006 is an output device (e.g., a display, a speaker, an LED lamp, etc.) that outputs to the outside. The input device 1005 and the output device 1006 may be integrated into one device (e.g., a touch panel).

[0040] Furthermore, each device, such as the processor 1001 and the memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or may be configured using different buses between each device.

[0041] The client device 10 may also be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 1001 may be implemented using at least one of these pieces of hardware.

[0042] The notification of information is not limited to the aspects / embodiments described in the present disclosure and may be performed using other methods. For example, the notification of information may be performed by physical layer signaling (e.g., Downlink Control Information (DCI) and Uplink Control Information (UCI)), higher layer signaling (e.g., Radio Resource Control (RRC) signaling, Medium Access Control (MAC) signaling, broadcast information (Master Information Block (MIB) and System Information Block (SIB))), other signals, or a combination thereof. Furthermore, the RRC signaling may be referred to as an RRC message, and may be, for example, an RRC Connection Setup message, an RRC Connection Reconfiguration message, or the like.

[0043] Each aspect / embodiment described in the present disclosure may be implemented using any of the following standards: LTE (Long Term Evolution), LTE-Advanced (LTE-A), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), 6th generation mobile communication system (6G), xth generation mobile communication system (xG) (xG (x is, for example, an integer or a decimal number)), FRA (Future Radio Access), NR (new Radio), New radio access (NX), Future generation radio access (FX), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.17 (WiMAX (registered trademark)), IEEE 802.19 (WiMAX (registered trademark)), IEEE 802.20 (WiMAX (registered trademark)), IEEE 802.21 (Wi-Fi (registered trademark)), IEEE 802.22 (WiMAX (registered trademark)), IEEE 802.23 (WiMAX (registered trademark)), IEEE 802.24 (WiMAX (registered trademark)), IEEE 802.25 (WiMAX (registered trademark)), IEEE 802.26 (WiMAX (registered trademark)), IEEE 802.27 (WiMAX (registered trademark)), IEEE 802.28 (WiMAX (registered trademark)), IEEE 802.29 (WiMAX (registered trademark)), IEEE 802.30 (WiMAX (registered trademark)), IEEE 802.31 (Wi-Fi (registered trademark)), IEEE 802.32 (WiMAX (registered trademark)), IEEE 802.33 (WiMAX (registered trademark)), IEEE 802.34 ( The present invention may be applied to at least one of systems using 802.20, UWB (Ultra-Wide Band), Bluetooth (registered trademark), or other suitable systems, and next-generation systems that are extended, modified, created, or defined based on these systems. The present invention may also be applied to a combination of multiple systems (e.g., a combination of LTE and / or LTE-A with 5G).

[0044] The order of the procedures, sequences, flowcharts, etc. of each aspect / embodiment described in this disclosure may be changed unless it is consistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0045] Input and output information may be stored in a specific location (for example, memory) or may be managed using a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be transmitted to another device.

[0046] The determination may be made based on a value represented by one bit (0 or 1), a Boolean value (true or false), or a numerical comparison (e.g., comparison with a predetermined value).

[0047] The aspects / embodiments described in this disclosure may be used alone, in combination, or switched depending on the implementation. Notification of predetermined information (e.g., notification that "X is true") is not limited to explicit notification, but may be implicit (e.g., not notifying the predetermined information).

[0048] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure.

[0049] Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0050] Software, instructions, information, etc. may also be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then these wired and / or wireless technologies are included within the definition of transmission media.

[0051] The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0052] Note that terms described in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings. For example, at least one of a channel and a symbol may be a signal (signaling). Furthermore, a signal may be a message. Furthermore, a component carrier (CC) may be called a carrier frequency, a cell, a frequency carrier, etc.

[0053] As used in this disclosure, the terms "system" and "network" are used interchangeably.

[0054] Furthermore, the information, parameters, etc. described in the present disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or other corresponding information. For example, a radio resource may be indicated by an index.

[0055] The names used for the above-described parameters are not intended to be limiting in any way. Furthermore, the mathematical expressions using these parameters may differ from those explicitly disclosed in this disclosure. The various channels (e.g., PUCCH, PDCCH, etc.) and information elements may be identified by any suitable names, and therefore the various names assigned to these various channels and information elements are not intended to be limiting in any way.

[0056] As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" and "determining" may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in memory), and the like. Furthermore, "judgment" and "decision" can include regarding resolving, selecting, choosing, establishing, comparing, etc. as having been "judged" or "decided." In other words, "judgment" and "decision" can include regarding some action as having been "judged" or "decided." Furthermore, "judgment (decision)" can be interpreted as "assuming," "expecting," "considering," etc.

[0057] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0058] As used in this disclosure, any reference to an element using a designation such as "first," "second," etc. does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed or that the first element must in some way precede the second element.

[0059] When the terms "include," "including," and variations thereof are used in this disclosure, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, when the term "or" is used in this disclosure, it is not intended to be an exclusive or.

[0060] In this disclosure, where articles are added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural form.

[0061] In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "coupled" may also be interpreted in the same way as "different."

[0062] 1...information system, 10...client device, 11...acquisition request unit, 12...information receiving unit, 13...content generation unit, 14...output unit, 20...cloud server, 21...rendering execution unit, 22...information transmitting unit, 30...CDN, 31...image database, 1001...processor, 1002...memory, 1003...storage, 1004...communication device, 1005...input device, 1006...output device, 1007...bus.

Claims

1. A client device comprising: an information receiving unit that receives from a cloud server display information required for displaying characters and images to be displayed, and background data rendered by the cloud server; and a content generating unit that generates content to be displayed based on the display information from the characters and image data to be displayed, which is obtained based on the display information received by the information receiving unit, and the received background data.

2. The client device of claim 1, wherein the display information includes character information indicating characters to be displayed and character display control information relating to display control of the characters, as well as save destination information for an image to be displayed and image display control information relating to display control of the image, and the content generation unit generates character data to be displayed based on the character information, acquires image data to be displayed based on the save destination information, and generates the content to be displayed by synchronizing and combining the acquired character data and image data with the background data based on the character display control information and the image display control information.

3. The client device of claim 2, wherein the content generation unit makes a request to a platform composed of a plurality of servers that distribute various image data in response to requests to obtain the image data to be displayed based on the storage destination information, and obtains the image data from the platform.

4. An information system including: a cloud server that renders background data and provides display information and the rendered background data required for displaying characters and images to be displayed; an information receiving unit that receives the display information and the rendered background data from the cloud server; and a client device including: a content generating unit that generates content to be displayed based on the display information from the character and image data to be displayed, which is obtained based on the display information received by the information receiving unit, and the received background data.

Images