Content evaluation apparatus, program, method, and system

Abstract

To provide a content evaluation device, a program, a method, and a system that can evaluate contents more precisely than when evaluating contents simply by using drawn contents of a finished product.SOLUTION: A content evaluation device (16) calculates a state feature amount related to a drawing state within a creation period from a start point to an end point of creation of contents (80), and generates a picture pattern (100) that is a set or a trajectory of points on a feature amount space (90) for expressing the state feature amount.SELECTED DRAWING: Figure 13

Description

Technical Field

[0001] The present invention relates to a content evaluation apparatus, a program, a method, and a system.

Background Art

[0002] Conventionally, techniques for sharing digital content (hereinafter also simply referred to as "content"), which is an intangible object, among a plurality of users using a computer system have been known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Recently, with the progress of artificial intelligence technology, various machine learning models (as an example, adversarial generative networks; GANs) for generating content have been proposed. For example, if counterfeits of content can be automatically and elaborately generated using such machine learning models, it is expected that it will be difficult to determine the authenticity of the content simply by comparing the drawing contents of the finished products.

[0005] The present invention has been made in view of such problems, and an object thereof is to provide a content evaluation apparatus, a program, a method, and a system capable of evaluating content more precisely as compared with the case of evaluating using only the drawing content of the finished product.

Means for Solving the Problems

[0006] A content evaluation device according to a first aspect of the present invention includes a feature calculation unit that calculates state features relating to the drawing state during the creation period from the start to the end of content creation, and a picture generation unit that generates a picture, which is a set of points or a trajectory in a feature space for representing the state features calculated by the feature calculation unit.

[0007] A content evaluation device according to a second aspect of the present invention comprises a first calculation unit that calculates a time series of state features relating to the rendering state of content created through a series of operations, and a second calculation unit that uses the time series of state features calculated by the first calculation unit to determine the amount of change in state features before and after one operation, and calculates operation features relating to the one operation from the amount of change.

[0008] A content evaluation program in a third aspect of the present invention causes one or more computers to perform a first calculation step of calculating a time series of state features relating to the rendering state of content created through a series of operations, and a second calculation step of using the calculated time series of state features to determine the amount of change in state features before and after one operation, and to calculate operation features relating to the one operation from the amount of change.

[0009] A content evaluation method according to a fourth aspect of the present invention includes a first calculation step of calculating a time series of state features relating to the rendering state of content created through a series of operations, One or more computers perform a second calculation step in which they use the time series of the calculated state features to determine the change in the state features before and after an operation, and calculate operation features related to the operation from the change in state features.

[0010] A content evaluation system according to a fifth aspect of the present invention comprises a user device capable of generating content data indicating content created through a series of operations, and a server device configured to communicate with the user device, wherein the server device comprises a first calculation unit that calculates a time series of state features relating to the rendering state of the content, and a second calculation unit that uses the time series of state features calculated by the first calculation unit to determine the amount of change in state features before and after one operation, and calculates operation features relating to the one operation from the amount of change. [Effects of the Invention]

[0011] According to the present invention, content can be evaluated more precisely compared to simply using the drawing content of the finished product for evaluation. [Brief explanation of the drawing]

[0012] [Figure 1] This is an overall configuration diagram of a content evaluation system in one embodiment of the present invention. [Figure 2] This block diagram shows an example of the server device configuration in Figure 1. [Figure 3] Figure 2 is a detailed functional block diagram of the feature calculation unit. [Figure 4] This flowchart shows an example of how a server device calculates feature information. [Figure 5] This figure shows an example of content created using the user device shown in Figure 1. [Figure 6] This figure shows the transitions in the drawing state of the artwork in Figure 5. [Figure 7] This figure shows an example of the data structure of the content data in Figures 1 and 2. [Figure 8] This figure shows an example of the data structure of the graph data in Figure 3. [Figure 9] This figure shows an example of the data structure of the first emblem data. [Figure 10] This figure shows an example of a method for calculating state features. [Figure 11] It is a diagram showing an example of the data structure of the second pattern data. [Figure 12] It is a diagram showing an example of a method for calculating operation feature amounts. [Figure 13] It is a diagram showing a first example of a method for identifying creation steps. [Figure 14] It is a diagram showing a second example of a method for identifying creation steps.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. For ease of understanding of the description, the same reference numerals are attached to the same components in each drawing as much as possible, and duplicate descriptions are omitted.

[0014] [Configuration of Content Evaluation System 10] [Overall Configuration] FIG. 1 is an overall configuration diagram of a content evaluation system 10 according to an embodiment of the present invention. The content evaluation system 10 is provided to provide a "content evaluation service" for evaluating digitized content (so-called digital content). Specifically, this content evaluation system 10 includes one or more user devices 12, one or more electronic pens 14, and a server device 16 (corresponding to a "content evaluation device"). Each user device 12 and the server device 16 are communicably connected via a network NT.

[0015] The user device 12 is a computer owned by a user (for example, a content creator) who uses the content evaluation service, and is composed of, for example, a tablet, a smartphone, a personal computer, etc. Each user device 12 generates content data D1 and related data D2, which will be described later, and is configured to be able to supply various data generated by itself to the server device 16 via the network NT. Specifically, this user device 12 includes a processor 21, a memory 22, a communication unit 23, a display unit 24, and a touch sensor 25.

[0016] The processor 21 is composed of arithmetic processing units including a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an MPU (Micro-Processing Unit). The processor 21 reads programs and data stored in memory 22 and performs generation processing to generate ink data (hereinafter also called digital ink) that describes content, rendering processing to display the content indicated by the digital ink, and so on.

[0017] Memory 22 stores programs and data necessary for the processor 21 to control each component. Memory 22 is non-transient and consists of a computer-readable storage medium. Here, the computer-readable storage medium consists of [1] storage devices such as hard disk drives (HDDs) and solid state drives (SSDs) built into the computer system, and [2] portable media such as magneto-optical disks, ROMs (Read Only Memory), CDs (Compact Disk)-ROMs, and flash memory.

[0018] The communication unit 23 has a communication function that enables wired or wireless communication with external devices. This allows the user device 12 to exchange various types of data, such as content data D1, related data D2, or presentation data D3, with, for example, the server device 16.

[0019] The display unit 24 is capable of visually displaying content including images or videos, and is composed of, for example, a liquid crystal panel, an organic EL (Electro-Luminescence) panel, or electronic paper. By making the display unit 24 flexible, the user can perform various writing operations while the touch surface of the user device 12 is curved or bent.

[0020] The touch sensor 25 is a capacitive sensor comprising multiple sensor electrodes arranged in a planar configuration. This touch sensor 25 is composed of, for example, multiple X-line electrodes for detecting the position along the X axis of the sensor coordinate system and multiple Y-line electrodes for detecting the position along the Y axis. Alternatively, the touch sensor 25 may be a self-capacitive sensor in which block-shaped electrodes are arranged in a two-dimensional grid, instead of the mutual-capacitive sensor described above.

[0021] The electronic pen 14 is a pen-type pointing device configured to communicate with the user device 12 in one direction or bidirectionally. This electronic pen 14 is, for example, an active electrostatic coupling (AES) or electromagnetic induction (EMR) stylus. A student, as a user, can write pictures, letters, etc., on the user device 12 by holding the electronic pen 14 and moving the pen tip against the touch surface of the user device 12.

[0022] The server device 16 is a computer that provides overall control over content evaluation, and may be either cloud-based or on-premises. Here, the server device 16 is illustrated as a single computer, but the server device 16 may instead be a group of computers that form a distributed system.

[0023] <Block diagram of server device 16> Figure 2 is a block diagram showing an example of the configuration of the server device 16 in Figure 1. Specifically, the server device 16 comprises a communication unit 30, a control unit 32, and a storage unit 34.

[0024] The communication unit 30 is an interface for sending and receiving electrical signals to and from external devices. This allows the server device 16 to acquire at least one of the content data D1 and related data D2 from the user device 12, and to provide the presentation data D3 that it has generated to the user device 12.

[0025] The control unit 32 is composed of a processor including a CPU and a GPU. The control unit 32 reads and executes programs and data stored in the memory unit 34, thereby functioning as a data acquisition unit 40, a feature calculation unit 42, a content evaluation unit 44 (corresponding to the "authenticity determination unit" and the "process identification unit"), an information generation unit 46 (corresponding to the "picture pattern generation unit"), and a display instruction unit 48.

[0026] The data acquisition unit 40 acquires various data related to the content being evaluated (for example, content data D1 and related data D2). The data acquisition unit 40 may acquire various data from an external device via communication, or it may acquire various data by reading them from the storage unit 34.

[0027] The feature calculation unit 42 calculates features related to the content from at least one of the content data D1 and related data D2 acquired by the data acquisition unit 40. These features include [1] features related to the rendering state of the content (hereinafter referred to as "state features") or [2] features related to individual operations performed to create the content (hereinafter referred to as "operation features"). The specific configuration of the feature calculation unit 42 is explained in detail in Figure 3.

[0028] The content evaluation unit 44 performs an evaluation process to evaluate the content using the time series of state features or operation features calculated by the feature calculation unit 42. The content evaluation unit 44 evaluates, for example, [1] the style of the content, [2] the creator's habits, [3] the creator's psychological state, or [4] the state of the external environment. Here, "style" means the creator's individuality or ideas as expressed in the content. Examples of "habits" include the use of color, drawing tendencies of strokes, tendencies in how equipment is used, and the degree of operational errors. Examples of "psychological state" include emotions such as joy, anger, sadness, and pleasure, as well as various states such as drowsiness, relaxation, and tension. Examples of "external environment" include ambient brightness, temperature, weather, and season.

[0029] Furthermore, the content evaluation unit 44 calculates the similarity between the time series of features corresponding to the content being evaluated (i.e., the first time series features) and the time series of features corresponding to the genuine content (i.e., the second time series features), and determines the authenticity of the content being evaluated based on this similarity. Various indicators, including correlation coefficients and norms, can be used to calculate this similarity.

[0030] Furthermore, the content evaluation unit 44 can estimate the type of creative process corresponding to the rendering state of the content using the time series of state features or operation features calculated by the feature calculation unit 42. Examples of creative process types include composition, line drawing, coloring, and finishing. The coloring process may also be subdivided into, for example, underpainting and final coloring.

[0031] The information generation unit 46 uses the time series of various features (more specifically, state features or operation features) calculated by the feature calculation unit 42 to generate either pictogram information 54 or derived information 56, which will be described later. Alternatively, the information generation unit 46 generates evaluation result information 58 that shows the evaluation result by the content evaluation unit 44.

[0032] The display instruction unit 48 instructs the display of the information generated by the information generation unit 46. This "display" includes not only displaying the information on an output device (not shown) provided on the server device 16, but also transmitting the presentation data D3, which includes the pictogram information 54, derived information 56, or evaluation result information 58, to an external device such as the user device 12 (Figure 1).

[0033] The memory unit 34 stores programs and data necessary for the control unit 32 to control each component. The memory unit 34 is composed of a non-transient and computer-readable storage medium. Here, the computer-readable storage medium consists of [1] storage devices such as HDDs and SSDs built into the computer system, and [2] portable media such as magneto-optical disks, ROMs, CD-ROMs, and flash memory.

[0034] In the example shown in Figure 2, the memory unit 34 contains a database of word concepts (hereinafter referred to as "concept graph 50"), a database of content (hereinafter referred to as "content DB 52"), and also stores pictogram information 54, derivative information 56, and evaluation result information 58.

[0035] Conceptual graph 50 is a graph that shows the relationships between words (i.e., an ontology graph), and is composed of nodes and links (or edges). Each word that makes up a node is associated with a coordinate value in an N-dimensional (e.g., N≧3) feature space. In other words, each word is quantified as a "distributed representation" in natural language processing.

[0036] Conceptual graph 50 includes nouns, adjectives, adverbs, verbs, or compound words combining these. Furthermore, conceptual graph 50 is not limited to words that directly express the form of content (e.g., type of object, color, shape, pattern, etc.), but may also include words related to mental expressions such as emotions and states. Moreover, conceptual graph 50 is not limited to words used in everyday life, but may also include words not used in everyday life (e.g., fictional objects, types of creative processes). In addition, conceptual graph 50 may be provided for each language, such as Japanese, English, and Chinese. By using different conceptual graphs 50, it is possible to more accurately reflect cultural differences between countries or regions.

[0037] Content DB52 stores [1] content data D1, [2] related data D2, and [3] information generated using content data D1 or related data D2 (hereinafter referred to as "generated information") in association with each other. This "generated information" includes pictogram information 54, derivative information 56, and evaluation result information 58.

[0038] Content data D1 is a collection of content elements that constitute the content, and is configured to express the content creation process. Content data D1 consists, for example, of ink data (hereinafter referred to as digital ink) for representing handwritten content. Examples of "ink description languages" for describing digital ink include WILL (Wacom Ink Layer Language), InkML (Ink Markup Language), and ISF (Ink Serialized Format). The content may be artwork (or digital art) including, for example, paintings, calligraphy, illustrations, and text.

[0039] Related data D2 includes various information related to the creation of content. Examples of related data D2 include: [1] creator information including the identification information and attributes of the content creator; [2] "device driver-side setting conditions" including the resolution, size, and type of the display unit 24, the detection performance and type of the touch sensor 25, and the shape of the pen pressure curve; [3] "drawing application-side setting conditions" including the type of content, color palette and brush color information, and visual effect settings; [4] "creator's operation history" which is sequentially stored through the execution of the drawing application; or [5] "vital data" which indicates the creator's biological state.

[0040] The pictogram information 54 includes pictograms defined on the feature space described above or processed pictograms. Here, "pictogram" means a set of points or a trajectory on the feature space for representing state features. Examples of "processed pictograms" include pictograms with reduced dimensionality (i.e., cross-sectional views) and pictograms with reduced point counts. The pictogram information 54 is stored in association with the creator information described above, specifically, the identification information of the content or creator.

[0041] Derived information 56 is information derived from pictogram information 54, and includes, for example, visible information (hereinafter referred to as "awareness information") intended to give the content creator an idea. Examples of awareness information include [1] a group of words as state features, and [2] another expression that abstracts or euphemizes the words included in this group of words (for example, a symbol indicating the strength of a feature, another word with high similarity, etc.). Derived information 56 is stored in association with creator information (specifically, identification information of the content or creator), similar to pictogram information 54.

[0042] The evaluation result information 58 shows the content evaluation results by the content evaluation unit 44. Examples of evaluation results include [1] the results of individual evaluations including classification categories and scores, and [2] the results of comparative evaluations including similarity and authenticity determination.

[0043] <Functional block diagram of the feature calculation unit 42> Figure 3 is a detailed functional block diagram of the feature calculation unit 42 shown in Figure 2. This feature calculation unit 42 functions as a data formatting unit 60, a rasterization processing unit 62, a word conversion unit 64, a data integration unit 66, a state feature calculation unit 68 (corresponding to the "first calculation unit"), and an operation feature calculation unit 70 (corresponding to the "second calculation unit").

[0044] The data formatting unit 60 processes the content data D1 and related data D2 acquired by the data acquisition unit 40 and outputs formatted data (hereinafter referred to as "non-raster data"). Specifically, the data formatting unit 60 performs the following: [1] association processing to link both sets of data together, [2] assignment processing to assign order to a series of operations during the content creation period, and [3] removal processing to remove unnecessary data. Examples of "unnecessary data" include: [1] operation data related to user operations that were canceled during the creation period, [2] operation data related to user operations that do not contribute to the completion of the content, and [3] various data that are found to be inconsistent as a result of the association processing described above.

[0045] The rasterization processing unit 62 performs a "rasterization process" that converts the vector data contained in the content data D1 acquired by the data acquisition unit 40 into raster data. Vector data refers to stroke data that indicates the form of the stroke (for example, shape, thickness, color, etc.). Raster data refers to image data composed of multiple pixel values.

[0046] The word conversion unit 64 performs data conversion processing to convert input data into one or more words (hereinafter referred to as a word group). The word conversion unit 64 includes a first converter for outputting a first word group and a second converter for outputting a second word group.

[0047] The first converter consists of a learner that takes raster data from the rasterization processing unit 62 as input and outputs tensor data indicating the detection result of the image (probability of existence regarding the type and position of the object). This learner may be constructed, for example, by a machine learning-based convolutional neural network (e.g., "Mask R-CNN"). The word conversion unit 64 refers to the graph data 72 describing the concept graph 50 and determines the group of words registered in the concept graph 50 from the group of words indicating the type of object detected by the first converter as the "first word group".

[0048] The second converter consists of a learner that takes non-raster data from the data formatting unit 60 as input and outputs a score for each word. This learner may be constructed using, for example, a machine learning-based neural network (e.g., "LightGBM" or "XGBoost"). The word conversion unit 64 refers to the graph data 72 that describes the concept graph 50 and determines the group of words registered in the concept graph 50 from the group of words converted by the second converter as the "second word group".

[0049] The data integration unit 66 integrates the operation data (more specifically, the first word group and the second word group) obtained sequentially by the word conversion unit 64. This operation is a "stroke operation" for drawing a single stroke, but it may also be various user operations that may influence the creation of content, either in conjunction with or separately from this. Furthermore, this integration may be performed on an individual operation unit or on two or more consecutive operation units.

[0050] The state feature calculation unit 68 calculates, in a time series, features relating to the drawing state of the content created through a series of operations, based on at least both the raster data and stroke data of the content (hereinafter referred to as "state features"). This time series of state features corresponds to a "picture pattern" that is unique to the content. These state features may be, for example, [1] the types and number of words constituting a word group, or [2] coordinate values ​​in the feature space defined by the conceptual graph 50. Alternatively, the state feature calculation unit 68 may identify the type of language from the content data D1 or related data D2 and calculate the time series of state features using the conceptual graph 50 corresponding to the type of language.

[0051] The operation feature calculation unit 70 uses the time series of state features calculated by the state feature calculation unit 68 to determine the change in state features before and after a single or consecutive operation, and calculates operation features related to the operation in a time series from the amount of change. This time series of operation features corresponds to a "picture pattern" which is unique to the content. These operation features are, for example, the magnitude or direction of a vector that starts from the first drawing state immediately before an operation is performed and ends from the second drawing state immediately after an operation is performed.

[0052] [How the content evaluation system 10 works] The content evaluation system 10 in this embodiment is configured as described above. Next, the operation of the server device 16, which constitutes a part of the content evaluation system 10, specifically the operation of calculating feature information, will be explained with reference to the functional block diagram in Figure 3, the flowchart in Figure 4, and Figures 5 to 12.

[0053] In step SP10 of Figure 4, the data acquisition unit 40 acquires various data related to the content to be evaluated, for example, at least one of content data D1 and related data D2.

[0054] Figure 5 shows an example of content created using the user device 12 shown in Figure 1. The content in this figure is artwork 80 created by hand. The content creator completes the desired artwork 80 by making full use of the user device 12 and the electronic pen 14. The artwork 80 is created through a series of operations by the creator, or through multiple types of creative processes.

[0055] Figure 6 shows the transitions in the drawing state of artwork 80 in Figure 5. The first work in progress 80a shows the drawing state during the "composition process," where the overall composition is determined. The second work in progress 80b shows the drawing state during the "line drawing process," where the line drawing is created. The third work in progress 80c shows the drawing state during the "coloring process," where coloring is performed. The fourth work in progress 80d shows the drawing state during the "finishing process," where the artwork is completed.

[0056] Figure 7 shows an example of the data structure of content data D1 in Figures 1 and 2. In this example, content data D1 is shown as digital ink. Digital ink has a data structure consisting of a sequential arrangement of [1] document metadata, [2] ink semantics, [3] devices, [4] strokes, [5] groups, and [6] contexts.

[0057] Stroke data 82 is data for describing individual strokes that make up handwritten content, and indicates the shape of the strokes that make up the content and the order in which the strokes are written. As can be seen from Figure 7, one stroke is<trace>It is described by multiple point data arranged sequentially within a tag. Each point data consists of at least an indicated position (X coordinate, Y coordinate) and is separated by a delimiter such as a comma. For the sake of illustration, only the point data indicating the start and end points of the stroke are shown, and point data indicating multiple intermediate points are omitted. In addition to the indicated positions mentioned above, this point data may also include the order of stroke generation or editing, the pressure and orientation of the electronic pen 14, etc.

[0058] In step SP12 of Figure 4, the data formatting unit 60 performs formatting on the content data D1 and related data D2 acquired in step SP10. This formatting associates non-raster data (hereinafter also referred to as "non-raster data") with each stroke operation.

[0059] In step SP14, the feature calculation unit 42 specifies one drawing state that has not yet been selected within the content creation period. The feature calculation unit 42 specifies the drawing state in which the first stroke operation was performed during the first processing.

[0060] In step SP16, the rasterization processing unit 62 performs rasterization to reproduce the drawing state specified in step SP14. Specifically, the rasterization processing unit 62 performs drawing to add one stroke to the most recent image. This updates the raster data (i.e., the image) that is being converted.

[0061] In step SP18, the word conversion unit 64 converts each data created in steps SP14 and SP16 into a word group consisting of one or more words. Specifically, the word conversion unit 64 converts the raster data from the rasterization processing unit 62 into a first word group and converts the non-raster data from the data formatting unit 60 into a second word group. When performing [1] the conversion of raster data and [2] the conversion of non-raster data, the word conversion unit 64 refers to the graph data 72 that describes the conceptual graph 50.

[0062] Figure 8 shows an example of the data structure of the graph data 72 in Figure 3. This graph data 72 is a table-formatted data that shows the correspondence between "node information" about the nodes that make up the graph and "link information" about the links that make up the graph. The node information includes, for example, the node ID (identification), word name, distributed representation (coordinate values ​​in the feature space), and display flag. The link information includes whether or not there are links between nodes and the labels assigned to each link.

[0063] In step SP20 of Figure 4, the feature calculation unit 42 checks whether the data transformation for all stroke operations is complete. Since the transformation is not yet complete in the first process (step SP20: NO), the feature calculation unit 42 returns to step SP14.

[0064] In step SP14, the feature calculation unit 42 specifies the drawing state in the second processing where the second stroke operation was performed. Subsequently, the feature calculation unit 42 sequentially repeats the operations of steps SP14 to SP20 until data conversion is completed for all drawing states. While this operation is repeated, the data integration unit 66 aggregates and integrates the data for each stroke operation. After that, if data conversion for all stroke operations is completed (step SP20: YES), the process proceeds to the next step SP22.

[0065] In step SP22, the state feature calculation unit 68 calculates a time series of state features using the integrated data that has been combined through the execution of steps SP14 to SP20. This generates the first pictogram data 74 that represents the first pictogram.

[0066] Figure 9 shows an example of the data structure of the first pictogram data 74. This first pictogram data 74 is a table-formatted data that shows the correspondence between "state IDs," which are identification information of the drawing state of the artwork 80, and "state features" related to the drawing state. The state features include, for example, a first word group, a second word group, and coordinate values. These coordinate values ​​are defined on an N-dimensional feature space 90. The number of dimensions N is an integer of 3 or greater, and it is desirable that it be a number on the order of several hundred.

[0067] Figure 10 shows an example of a method for calculating state features. For illustrative purposes, the feature space 90 is represented in a planar coordinate system with the first and second components as two axes. The word group G1, corresponding to the first word group, consists of multiple words (6 in this example) 92. The word group G2, corresponding to the second word group, consists of multiple words (7 in this example) 94.

[0068] Here, the state feature calculation unit 68 calculates the union of the two word groups G1 and G2, and calculates the coordinate values ​​of the representative point 96 of the point set as features in the plotted state (i.e., state features). The state feature calculation unit 68 may calculate the union using all words belonging to word groups G1 and G2, or it may calculate the union after excluding words that have little relationship with other words (specifically, independent nodes without links). In addition, the state feature calculation unit 68 may, for example, identify the centroid of the point set as the representative point 96, or it may identify the representative point 96 using other statistical methods.

[0069] In step SP24 of Figure 4, the operational feature calculation unit 70 calculates a time series of operational features using the time series of state features calculated in step SP22. This generates second pictogram data 76 that represents the second pictogram.

[0070] Figure 11 shows an example of the data structure of the second pictogram data 76. This second pictogram data 76 is a table-formatted data showing the correspondence between "stroke IDs," which are identification information for stroke operations, and "operation features" related to each stroke operation. Operation features include, for example, increased words, decreased words, and displacement amounts. Displacement amounts are defined on an N-dimensional feature space 90, similar to the "coordinate values" in Figure 9.

[0071] Figure 12 schematically illustrates an example of a method for calculating operational features. For illustrative purposes, as in Figure 10, the feature space 90 is represented in a planar coordinate system with the first and second components as two axes. The stars in this figure indicate the position of a word defined by the conceptual graph 50 (so-called distributed representation). The circles in this figure indicate the position of a drawing state (i.e., a state feature).

[0072] For example, suppose that performing the i-th stroke operation from the i-th drawing state transitions to the (i+1)th drawing state. In this case, the vector (or displacement) with position P as the starting point and position Q as the ending point corresponds to the i-th operation feature. Similarly, the vector (or displacement) with position Q as the starting point and position R as the ending point corresponds to the (i+1)th operation feature.

[0073] In step SP26 of Figure 4, the feature calculation unit 42 stores the feature information calculated in steps SP22 and SP24, respectively. Specifically, the feature calculation unit 42 supplies the first image data 74 and the second image data 76 to the storage unit 34, associated with the content or creator. As a result, the first image data 74 and the second image data 76 are registered in the content DB 52 in a usable state. In this way, the server device 16 completes the operation shown in the flowchart of Figure 4.

[0074] [Example of using emblem data] <Example 1: Identifying the creative process> The content evaluation unit 44 may use a time series of state features or operation features (i.e., pictogram data) to identify the type of creative process corresponding to the drawing state of the artwork 80. For example, if words indicating creative processes are defined in the conceptual graph 50, the content evaluation unit 44 can identify the creative process depending on whether or not the words are included in the first word group or the second word group. The method for identifying the creative process when words indicating creative processes are not defined in the conceptual graph 50 will be explained below with reference to Figures 13 and 14.

[0075] Figure 13 shows a first example of a method for identifying the creative process. For ease of illustration, as in Figures 10 and 12, the feature space is represented in a planar coordinate system with the first and second components as two axes. The pattern 100 in this figure represents a series of creative processes from the start to the end of the creation of the artwork 80. More specifically, the pattern 100 is a collection of representative points 96 (Figure 10) calculated for each stroke operation. Each point constituting the pattern 100 is drawn with different shades of gray depending on the type of creative process. As can be seen from this figure, clusters of point sets tend to be formed for each creative process. Therefore, the content evaluation unit 44 can perform clustering on the point set of the pattern 100 and identify the creative process according to whether or not it belongs to a divided group. Note that the pattern 100 may be a trajectory consisting of a single line instead of a collection of points (i.e., a scatter plot).

[0076] Figure 14 shows a second example of a method for identifying the creative process. The horizontal axis of the graph represents the stroke ID, and the vertical axis represents the operation feature. This operation feature corresponds to the magnitude of the displacement of the state feature, i.e., the "norm" of the vector. As can be seen from this figure, there is a tendency for the feature points to move significantly and the norm to increase temporarily and rapidly when transitioning to the next creative process. Therefore, the content evaluation unit 44 performs peak detection processing on the time profile of the operation feature and can identify the timing of the transition to the creative process from the positional relationship of the multiple detected peaks.

[0077] <2. Presentation of insightful information> The server device 16 may present various information related to the creative activity to the creator of the artwork 80. In this case, the display instruction unit 48 instructs the display of the pictogram information 54 or derived information 56 generated by the information generation unit 46. More specifically, the display instruction unit 48 transmits the presentation data D3, which includes the pictogram information 54 or derived information 56 related to the artwork 80, to the user device 12 owned by the creator of the artwork 80.

[0078] Then, the processor 21 of the user device 12 generates a display signal using the presentation data D3 received from the server device 16 and supplies the display signal to the display unit 24. As a result, the pictogram information 54 or derived information 56 is visualized and displayed on the display screen of the display unit 24.

[0079] For example, the user device 12 may display insight information, which is one aspect of derived information 56, together with the artwork 80. By visualizing the insight information, it becomes possible to encourage the creator to make new discoveries and interpretations of the artwork 80, thereby providing the creator with new inspiration. As a result, a "positive spiral" in the creative activity of art is created.

[0080] [Summary of Embodiments] As described above, the content evaluation system 10 in this embodiment comprises one or more user devices 12 capable of generating content data D1 representing content (e.g., artwork 80), and a content evaluation device (in this case, a server device 16) configured to communicate with each of the user devices 12.

[0081] [1] The server device 16 includes a feature calculation unit 42 that calculates state features related to the drawing state during the creative period from the start to the end of the creation of the artwork 80, and an image generation unit (here, an information generation unit 46) that generates an image pattern 100, which is a set of points or a trajectory on the feature space 90 to represent the state features calculated by the feature calculation unit 42.

[0082] Furthermore, according to the content evaluation method and program in this embodiment, one or more computers (here, a server device 16) calculate state features relating to the drawing state during the creation period from the start to the end of the creation of the artwork 80 (step SP22 in Figure 4), and generate a pictogram 100 which is a set of points or a trajectory on the feature space 90 to represent the calculated state features.

[0083] In this way, a set of points or a trajectory on the feature space 90, i.e., a pictogram 100, is generated to represent state features related to the drawing state. Therefore, the artwork 80 can be evaluated more precisely compared to simply evaluating it using the drawing content of the finished product.

[0084] The server device 16 may also further include a display instruction unit 48 that instructs the display of pictogram information 54 related to the pictogram 100 or derived information 56 derived from the pictogram information 54. Furthermore, if the state feature quantity has a number of dimensions greater than 3, the pictogram information 54 may be a pictogram with a reduced number of dimensions of 3 or less. Furthermore, the derived information 56 may be insight information intended to give insight to the creator of the artwork 80. The server device 16 may also further include a content evaluation unit 44 that evaluates the artwork 80 using pictogram data representing the pictogram 100 (here, first pictogram data 74 or second pictogram data 76).

[0085] [2] The server device 16 includes a first calculation unit (here, a state feature calculation unit 68) that calculates a time series of state features relating to the drawing state of content (here, artwork 80) created through a series of operations, and a second calculation unit (here, an operation feature calculation unit 70) that uses the time series of state features calculated by the state feature calculation unit 68 to determine the amount of change in state features before and after one operation, and calculates operation features relating to one operation from the amount of change.

[0086] Furthermore, according to the content evaluation method and program in this embodiment, one or more computers (here, a server device 16) perform a first calculation step (step SP22 in Figure 4) in which a time series of state features relating to the drawing state of artwork 80 created through a series of operations is calculated, and a second calculation step (step SP24 in Figure 4) in which the amount of change in the state features before and after a single or consecutive operation is determined using the calculated time series of state features, and operation features relating to a single or consecutive operation are calculated from the amount of change.

[0087] In this way, by determining the change in state features before and after a single operation, and calculating operation features related to that operation from these changes, the artwork 80 can be evaluated more precisely compared to simply evaluating using the rendering content of the finished product.

[0088] Furthermore, state features may be coordinate values ​​on a feature space 90 defined by a conceptual graph 50 that shows the relationships between words. Also, if a conceptual graph 50 is provided for each language type, the state feature calculation unit 68 may identify the language type from at least one of the content data D1 representing the artwork 80 and related data D2 related to the creation of the artwork 80, and calculate a time series of state features using the conceptual graph 50 corresponding to the language type. In addition, the state feature calculation unit 68 may calculate a time series of state features based on at least both the raster data and stroke data of the artwork 80.

[0089] Furthermore, the operation feature may be the magnitude or direction of a vector that starts from the first drawing state immediately before a stroke operation is performed to draw a single stroke and ends at the second drawing state immediately after the stroke operation is performed. In addition, the content evaluation unit 44 may use the time series of the state feature or operation feature to identify the type of creative process corresponding to the drawing state of the artwork 80.

[0090] [Differentiation] It should be noted that the present invention is not limited to the embodiments described above, and can be freely modified without departing from the spirit of the invention. Alternatively, the various components may be combined in any way that does not create a technical inconsistency. Alternatively, the execution order of each step constituting the flowchart may be changed as long as it does not create a technical inconsistency. [Explanation of Symbols]

[0091] 10...Content evaluation system, 12...User device, 14...Electronic pen, 16...Server device (Content evaluation device), 40...Data acquisition unit, 42...Feature calculation unit, 44...Content evaluation unit (Process identification unit), 46...Information generation unit (Picture generation unit), 48...Display instruction unit, 50...Conceptual graph, 54...Picture information, 56...Derived information, 80...Artwork (Content), 90...Feature space, 100...Picture, D1...Content data, D2...Related data, D3...Presented data< / trace>

Claims

1. A feature calculation unit calculates state features, which are coordinate values ​​in a feature space defined based on the relationships between words, that relate to the rendering state during the creative period from the start to the end of content creation. A picture pattern generation unit generates a picture pattern, which is a set of points or a trajectory in the feature space, based on the time series of state features calculated by the feature calculation unit. A content evaluation device equipped with the following features.

2. The state feature has a number of dimensions greater than 3, The system further includes a display instruction unit that instructs the display of the pictogram in a form in which the number of dimensions is reduced to three or less. The content evaluation device according to claim 1.

3. A conceptual graph showing the relationships between the words is constructed on the feature space, The aforementioned conceptual graph is provided for each type of language. The content evaluation device according to claim 1.

4. The content is digital art including paintings, calligraphy, illustrations, or text. The content evaluation device according to claim 1.

5. A content evaluation unit further comprises a content evaluation unit that evaluates the authenticity of the content to be evaluated based on the similarity between a first time-series feature, which is a time-series of the state feature corresponding to the content to be evaluated, and a second time-series feature, which is a time-series of the state feature corresponding to the genuine content. The content evaluation device according to claim 1.

6. A calculation step of calculating state features, which are coordinate values ​​on a feature space defined based on the relationships between words, relating to the drawing state during the creative period from the start to the end of content creation, A generation step of generating a picture, which is a set of points or a trajectory in the feature space, based on the time series of the calculated state features, A content evaluation program that runs on one or more computers.

7. A calculation step of calculating state features, which are coordinate values ​​on a feature space defined based on the relationships between words, relating to the drawing state during the creative period from the start to the end of content creation, A generation step of generating a picture, which is a set of points or a trajectory in the feature space, based on the time series of the calculated state features, A content evaluation method performed by one or more computers.

8. A user device capable of generating content data that shows the content created through a series of operations, A server device configured to communicate with the aforementioned user device, Equipped with, The server device is A feature calculation unit calculates state features, which are coordinate values ​​in a feature space defined based on the relationships between words, relating to the drawing state during the creative period from the start to the end of the creation of the aforementioned content. A picture pattern generation unit generates a picture pattern, which is a set of points or a trajectory in the feature space, based on the time series of state features calculated by the feature calculation unit. A content evaluation system equipped with the following features.

Images