Client devices and information systems

Abstract

A client device (10) is provided with: an information reception unit (12) that receives, from a cloud server (20), display information required to display characters / images to be displayed and background data rendered by the cloud server (20); and a content generation unit (13) that generates, from character / image data to be displayed acquired on the basis of the display information received by the information reception unit (12) and the received background data, a content to be displayed on the basis of the display information.

Description

Technical Field

[0001] The present disclosure relates to a client device, and an information system including the client device and a cloud server.

Background Art

[0002] In conventional cloud rendering, it is common to perform data transfer aiming at reduction of transmission cost using a lossy compression method, and as an actual process, it is general that a cloud server embeds character strings and moving images into the background of a three-dimensional space, collectively renders the background image of the three-dimensional space, the character strings and the moving images, and outputs the result as single video data to a client device (mobile terminal, personal computer, etc.).

[0003] Regarding such cloud rendering, a technique in which many client devices participate in addition to a cloud server mainly for the purpose of load distribution to distribute rendering tasks to many processing entities (the cloud server and the client devices) is disclosed in Patent Document 1 below.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, with the conventional cloud rendering described above, when transmitting data using the above compression method, there is a problem in that the decoded text becomes blurry in areas where there is a large difference in pixel values ​​between the text and the background (for example, when black text is displayed on a white background), and the compression efficiency cannot be improved. This can lead to a decrease in the quality of the generated content and a decline in the user experience (UX). However, Patent Document 1 does not take the above problems into consideration and does not offer any solutions.

[0006] Therefore, this disclosure aims to solve the above-mentioned problems, suppress the deterioration of the quality of generated content, and suppress the decline in user experience. [Means for solving the problem]

[0007] The client device relating to this disclosure includes an information receiving unit that receives display information necessary for displaying characters and images to be displayed, and background data rendered by the cloud server, from the cloud server, and a content generating unit that generates content to be displayed based on the display information, obtained from the character and image data to be displayed based on the display information received by the information receiving unit, and the received background data. [Effects of the Invention]

[0008] According to this disclosure, by avoiding the combined rendering of background data requiring pre-rendering and other data (text and image data), and distributing the generation process between the cloud server and client devices, the conventional problems caused by combined rendering are solved, and the degradation of the generated content quality can be suppressed. As a result, high-quality content can be delivered without compromising the user experience. [Brief explanation of the drawing]

[0009] [Figure 1] This is a functional block diagram of an information system including a client device and related devices according to the embodiment. [Figure 2] This is a flowchart showing the processes executed in the information system shown in Figure 1. [Figure 3] This figure shows an example of the content that will be displayed. [Figure 4] This figure shows an example of the hardware configuration of a client device. [Modes for carrying out the invention]

[0010] Hereinafter, with reference to the drawings, one embodiment of the client device and the information system including said client device relating to this disclosure will be described.

[0011] As shown in Figure 1, the information system 1 in this embodiment includes a cloud server 20 that performs cloud rendering and a client device 10. In addition, although not an essential component, the information system 1 includes a platform consisting of multiple servers (e.g., cache servers) that store various image data in an image database 31 and deliver image data in response to external requests. In this embodiment, a Contents Delivery Network (hereinafter referred to as "CDN") 30 is used as the above platform. A CDN is a mechanism that uses a platform consisting of multiple cache servers to quickly deliver various content on a website to end users (in this case, client devices 10). In this embodiment, "images" broadly includes both moving and still images, and the CDN 30 stores and manages various moving and still image data in the image database 31.

[0012] The above-mentioned cloud server 20 is composed of at least one server located on the so-called cloud side, and includes a rendering execution unit 21 and an information transmission unit 22 as components for performing the functions related to this disclosure.

[0013] Of these, the rendering execution unit 21 is a functional unit that renders background data that needs to be rendered in advance, for example, in response to a request from the client device 10. In this case, "background data" refers to all data that represents display targets other than text and images and that needs to be rendered in advance (mainly background display data).

[0014] The information transmission unit 22 is a functional unit that transmits background data obtained by rendering by the rendering execution unit 21, and display information, described later, necessary for displaying the characters and images to be displayed, to the client device 10. Here, "display information" includes character information indicating the characters to be displayed, character display control information relating to the display control of said characters, and storage location information for the images to be displayed (for example, identification information of the image data and identification information of the CDN 30 storing said image data) and image display control information relating to the display control of said images. In this embodiment, the "character display control information" uses character markup information, which is information written in a markup language regarding the display control of characters. The character markup information describes, for example, information regarding the display position and timing of characters, font, size, and other information regarding character decoration. In addition, the "image display control information" uses image markup information, which is information written in a markup language regarding the display control of images. The image markup information describes, for example, information regarding the display position and timing of images.

[0015] On the other hand, the client device 10 is configured by various information processing apparatuses such as a mobile terminal (a smartphone, a tablet terminal, a mobile phone, etc.), a portable or stationary personal computer, and includes an acquisition request unit 11, an information receiving unit 12, a content generation unit 13, and an output unit 14 as components for exerting the functions according to the present disclosure.

[0016] Among these, the acquisition request unit 11 is a functional unit that requests the cloud server 20 to acquire the aforementioned background data and the aforementioned display information required for displaying the character image to be displayed.

[0017] The information receiving unit 12 is a functional unit that receives the background data obtained by rendering and the aforementioned display information transmitted from the information transmitting unit 22 of the cloud server 20.

[0018] The content generation unit 13 is a functional unit that generates content to be displayed from the character and image data acquired based on the received aforementioned display information and the received background data, based on the aforementioned display information. Specifically, the content generation unit 13 generates character data to be displayed based on the character information, and requests the CDN 30 to acquire image data to be displayed based on storage destination information included in the display information, and acquires the relevant image data from the CDN 30. Further, the content generation unit 13 generates the content to be displayed by synchronizing and compositing the character data and the image data with the background data based on character markup information and image markup information included in the display information.

[0019] The output unit 14 is a functional unit that outputs the content generated by the content generation unit 13. Note that "output" can take various forms such as display output, print output, and data transmission to the outside of the client device 10.

[0020] Hereinafter, the processing executed by the information system 1 in Fig. 1 will be described along the flowchart of Fig. 2.

[0021] First, the acquisition request unit 11 of the client device 10 transmits an acquisition request for background data that requires pre-rendering and display information (including the aforementioned character information, character markup information, image storage destination information, and image markup information) to the cloud server 20 (step S1). Here, as an example, it is assumed that a moving image representing persons P1, P2, a character string P3, and a background (for example, a plurality of trees appearing behind the persons or the like in a three-dimensional space, a sky appearing behind, etc.) P4 shown in FIG. 3 is to be output. The data for displaying the background P4 corresponds to the above-mentioned background data that requires pre-rendering, the images of the persons P1 and P2 correspond to the images to be displayed, and the characters of the character string P3 correspond to the characters to be displayed, respectively.

[0022] In the cloud server 20 that has received the above request, the rendering execution unit 21 executes rendering of the background data (step S2), and the information transmission unit 22 transmits the background data obtained by the rendering in step S2 and the requested display information to the client device 10 (step S3).

[0023] In the client device 10, the information receiving unit 12 receives the background data and the display information transmitted in step S3 (step S4), and the received data is sent to the content generation unit 13. The content generation unit 13 refers to the "image storage destination information (such as identification information of image data and identification information of a CDN that stores and manages the image data)" included in the received display information, and transmits an acquisition request for the image data to be displayed to the CDN 30 that is the storage destination (step S5). Further, the content generation unit 13 generates character data to be displayed based on the "character information" included in the received display information (step S6).

[0024] On the other hand, the CDN 30 that has received the acquisition request in step S5 reads out the image data to be displayed from the image database 31 and transmits it to the content generation unit 13 (step S7).

[0025] Then, when the content generation unit 13 receives the image data to be displayed from the CDN 30 (step S8), it generates the content to be displayed by synchronizing the text data generated in step S6 and the image data received in step S8 with the background data received in step S4 (step S9). In step S9, the content generation unit 13 synchronizes the text data with the background data based on the "text markup information" included in the display information, including text decoration, text display position, and timing. The content generation unit 13 also synchronizes the image data with the background data based on the "image markup information" included in the display information, including image display position and timing. The content generated in this manner is output by the output unit 14 (step S10).

[0026] In the embodiment described above, the cloud server 20 generates only the background data that requires prior rendering through rendering, while the client device 10 acquires or generates other data (text and image data) from the CDN 30, thus distributing the processing. This solves the problems caused by conventional cloud servers rendering background data and text / image data together (i.e., (1) the problem of decoded text becoming blurry in areas where there is a large difference in pixel values ​​between text and background, and (2) the inconvenience of not being able to improve compression efficiency, leading to a decrease in the quality of the generated content and a decline in UX), and enables the delivery of high-quality content without compromising the UX by suppressing the decline in the quality of the generated content.

[0027] Specifically, the content generation unit 13 of the client device 10 generates content to be displayed by synchronizing text data with background data based on the "text markup information" included in the display information, including text decoration, text display position, and timing, and synchronizing image data with background data based on the "image markup information" included in the display information, including image display position and timing. In this way, it is possible to appropriately synchronize and synthesize text and image data with background data based on text markup information and image markup information.

[0028] Furthermore, the content generation unit 13 of the client device 10 requests the CDN 30 to acquire the image data to be displayed based on the "storage location information" included in the display information, and acquires the image data. Therefore, the client device 10 does not need to store and manage a large amount of image data in advance, and this can be realized with an inexpensive and simple configuration.

[0029] [Explanation of terms, explanation of hardware configuration (Figure 4), etc.] The block diagrams used in the description of the above embodiments show functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining the above one device or the above multiple devices with software.

[0030] Functions include, but are not limited to, judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmitting unit, transmitter, etc. As mentioned above, the method of implementation is not particularly limited.

[0031] For example, a client device in one embodiment of this disclosure may function as a computer that performs the processing of this disclosure. Figure 4 is a diagram showing an example of the hardware configuration of a client device 10 according to one embodiment of this disclosure. The client device 10 described above may be physically configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc. The cloud server 20 and CDN 30 shown in Figure 1 may also be configured in the same way as the client device 10.

[0032] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of the client device 10 may include one or more of the devices shown in the figure, or it may be configured to omit some of the devices.

[0033] Each function in the client device 10 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, which causes the processor 1001 to perform calculations, control communication by the communication device 1004, and control at least one of the reading and writing of data in the memory 1002 and storage 1003.

[0034] The processor 1001 controls the entire computer, for example, by running an operating system. The processor 1001 may consist of a central processing unit (CPU) that includes interfaces with peripheral devices, control units, arithmetic units, registers, and so on.

[0035] Furthermore, the processor 1001 reads programs (program code), software modules, data, etc., from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. Although it has been explained that the various processes are executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may also be transmitted from a network via a telecommunications line.

[0036] Memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 1002 may also be called a register, cache, main memory, etc. Memory 1002 can store executable programs (program code), software modules, etc., for carrying out a wireless communication method according to one embodiment of the present disclosure.

[0037] Storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. Storage 1003 may also be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, server, or other suitable medium including at least one of memory 1002 and storage 1003.

[0038] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc. The communication device 1004 may be configured to include, for example, a high-frequency switch, duplexer, filter, frequency synthesizer, etc., in order to implement at least one of frequency division duplex (FDD) and time division duplex (TDD).

[0039] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).

[0040] Furthermore, each device, such as the processor 1001 and memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or different buses may be configured for each device.

[0041] Furthermore, the client device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, the processor 1001 may be implemented using at least one of these hardware components.

[0042] Information notification is not limited to the embodiments described herein and may be carried out by other means. For example, information notification may be carried out by physical layer signaling (e.g., DCI (Downlink Control Information), UCI (Uplink Control Information)), upper layer signaling (e.g., RRC (Radio Resource Control) signaling, MAC (Medium Access Control) signaling, broadcast information (MIB (Master Information Block), SIB (System Information Block))), other signals, or combinations thereof. RRC signaling may also be called RRC messages, and may be, for example, RRC Connection Setup messages, RRC Connection Reconfiguration messages, etc.

[0043] Each aspect / embodiment described in this disclosure includes LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), 6th generation mobile communication system (6G), xth generation mobile communication system (xG) (xG (where x is, for example, an integer or decimal)), FRA (Future Radio Access), NR (new Radio), New radio access (NX), Future generation radio access (FX), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), and IEEE This may apply to at least one system utilizing 802.20, UWB (Ultra-WideBand), Bluetooth®, or other appropriate systems, and to next-generation systems extended, modified, created, or defined based thereon. It may also apply to a combination of multiple systems (for example, a combination of at least one of LTE and LTE-A with 5G).

[0044] The processing procedures, sequences, flowcharts, etc., of each aspect / embodiment described herein may be reordered, provided they are consistent with each other. For example, the methods described herein present various step elements in an exemplary order and are not limited to that specific order.

[0045] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be transmitted to other devices.

[0046] The determination may be made by a value represented by 1 bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0047] Each aspect / embodiment described herein may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of specific information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0048] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0049] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, and so on, whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name.

[0050] Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technology (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technology (such as infrared or microwave), then at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0051] The information, signals, etc. described in this disclosure may be represented using any of the various different techniques. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0052] In addition, terms used in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings. For example, at least one of the channel and symbol may be a signal (signaling). Also, a signal may be a message. Furthermore, a component carrier (CC) may be called a carrier frequency, cell, frequency carrier, etc.

[0053] The terms “system” and “network” as used in this disclosure are interchangeable.

[0054] Furthermore, the information, parameters, etc., described in this disclosure may be expressed using absolute values, relative values ​​from a given value, or other corresponding information. For example, wireless resources may be indicated by an index.

[0055] The names used for the parameters described above are not restrictive in any way. Furthermore, the formulas and other expressions using these parameters may differ from those expressly disclosed in this disclosure. Various channels (e.g., PUCCH, PDCCH, etc.) and information elements can be identified by any suitable name, and therefore, the various names assigned to these various channels and information elements are not restrictive in any way.

[0056] As used in this disclosure, the terms “determining” and “determining” may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiry (e.g., searching in a table, database, or other data structure), and ascertaining. “Determining” may also include, for example, receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, and accessing (e.g., accessing data in memory). Furthermore, "judgment" and "decision" can include considering something as having been "judged" or "decided" after resolving, selecting, choosing, establishing, comparing, etc. In other words, "judgment" and "decision" can include considering something as having been "judged" or "decided" after some action. Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."

[0057] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0058] Any reference to elements using the designations “first,” “second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Accordingly, references to the first and second elements do not imply that only two elements may be employed, or that the first element must precede the second element in any way.

[0059] Where the terms “include,” “including,” and variations thereof are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to mean exclusive OR.

[0060] In this disclosure, if articles are added through translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0061] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different." [Explanation of Symbols]

[0062] 1... Information system, 10... Client device, 11... Acquisition request unit, 12... Information receiving unit, 13... Content generation unit, 14... Output unit, 20... Cloud server, 21... Rendering execution unit, 22... Information transmission unit, 30... CDN, 31... Image database, 1001... Processor, 1002... Memory, 1003... Storage, 1004... Communication device, 1005... Input device, 1006... Output device, 1007... Bus.

Claims

1. An information receiving unit that receives display information necessary for displaying the characters and images to be displayed, and background data rendered by the cloud server, from the cloud server. A content generation unit generates content to be displayed based on the display information, using the character and image data to be displayed acquired based on the display information received by the information receiving unit, and the background data received. Equipped with, The aforementioned display information is, Character display control information including character information indicating the characters to be displayed and character markup information which is information written in a markup language regarding the display control of said characters, and Image display control information, including image markup information which is information written in a markup language regarding the storage location of the image to be displayed and the display control of the said image, Includes, The content generation unit, Based on the aforementioned character information, character data to be displayed is generated, and based on the aforementioned storage location information, image data to be displayed is acquired. Based on the character markup information and the image markup information, the acquired character data and image data are synchronized with the background data and combined to generate the content to be displayed. Client device.

2. The content generation unit, A platform consisting of multiple servers that distribute various image data in response to requests is used to request the acquisition of the image data to be displayed, based on the storage location information, and the image data is acquired from the platform. The client device according to claim 1.

3. A cloud server that renders background data and provides display information and rendered background data necessary for displaying the characters and images to be displayed, An information receiving unit that receives the aforementioned display information and the rendered background data from the cloud server, and, A content generation unit generates content to be displayed based on the display information, using the character and image data to be displayed acquired based on the display information received by the information receiving unit, and the received background data. A client device equipped with, Includes, The aforementioned display information is, Character display control information including character information indicating the characters to be displayed and character markup information which is information written in a markup language regarding the display control of said characters, and Image display control information, including image markup information which is information written in a markup language regarding the storage location of the image to be displayed and the display control of the said image, Includes, The content generation unit, Based on the aforementioned character information, character data to be displayed is generated, and based on the aforementioned storage location information, image data to be displayed is acquired. Based on the character markup information and the image markup information, the acquired character data and image data are synchronized with the background data and combined to generate the content to be displayed. Information systems.

Images