Information processing method, information processing system, and program

Abstract

Provided are an information processing method, an information processing system, a program, and the like that can improve color reproducibility after color reduction processing compared to the conventional methods. [Solution] According to one aspect of the present invention, there is provided an information processing method comprising an acquisition step, a color reduction step, and an output step, in which the acquisition step acquires third image data which is colored image data, the color reduction step performs a clustering process on the colors contained in the third image data and reduces the colors of the third image data according to post-color reduction information indicating the colors corresponding to each cluster, the colors corresponding to each cluster being colors determined using the most frequent values ​​of each cluster, and the output step outputs fourth image data which is the image data after color reduction.

Description

[Technical field]

[0001] The present invention relates to an information processing method, an information processing system, and a program. [Background technology]

[0002] Patent Document 1 discloses an image processing device.

[0003] The scanner includes a data acquisition unit, a memory, a selection unit, and an image generation unit. The data acquisition unit acquires image data. The memory stores, for each of a plurality of types of color reduction processes that are executable by the image processing device and have different processing methods, a combination of processing information indicating the color reduction process and specific information for identifying the type of image data on which the color reduction process should be executed. The selection unit selects the color reduction process to be executed on the region image data using image information related to the region image data and the combination of the processing information and the specific information in the memory. The image generation unit executes the selected color reduction process on the region image data to generate color-reduced image data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-226278 A Summary of the Invention [Problem to be solved by the invention]

[0005] The image processing device disclosed in Patent Document 1 performs color reduction processing using the k-means method. However, color reduction processing using the k-means method has limitations in color reproducibility.

[0006] In view of the above circumstances, the present invention provides an information processing method, an information processing system, a program, and the like that can improve color reproducibility after color reduction processing compared to the conventional art. [Means for solving the problem]

[0007] According to one aspect of the present invention, there is provided an information processing method comprising an acquisition step, a color reduction step, and an output step, in which the acquisition step acquires third image data which is colored image data, the color reduction step clusters colors contained in the third image data, and reduces the colors of the third image data according to post-color reduction information indicating the colors corresponding to each cluster, the colors corresponding to each cluster being colors determined using the most frequent values ​​of each cluster, and the output step outputs fourth image data which is the image data after color reduction.

[0008] The method of the above embodiment uses the most frequent value of each cluster when performing clustering processing on the colors in the painted portion and extracting the colors corresponding to each cluster. It has been found that when the most frequent value of each cluster is used to extract the color, the color reproducibility of the image data before the color reduction processing is improved compared to when the average value of each cluster is used to extract the color (k-means method). Since there is no need to prepare a special device to execute this method, the configuration is accordingly simple.

[0009] Therefore, the method of this aspect can improve the functionality of a computer to achieve at least one of the following (1) to (4): (1) The computer's processing speed can be increased. (2) Power consumption of the computer can be reduced. (3) The communication speed of the computer can be increased. (4) The resources saved in the computer can be used for other core functions.

[0010] According to this aspect, it is possible to improve color reproducibility after the color subtraction process compared to the conventional art. [Brief description of the drawings]

[0011] [Figure 1] 1 is a configuration diagram illustrating an information processing system 100. [Diagram 2]FIG. 2 is a block diagram showing a hardware configuration of an information processing device 200. [Diagram 3] FIG. 2 is a block diagram showing a hardware configuration of a terminal 300. [Figure 4] FIG. 2 is a block diagram showing functions realized by an information processing device 200 (a control unit 210). [Diagram 5] 2 is an activity diagram showing the flow of an information processing method executed by information processing device 200. FIG. [Figure 6] 2 is an activity diagram showing the flow of an information processing method executed by information processing device 200. FIG. [Figure 7] 2 is an activity diagram showing the flow of an information processing method executed by information processing device 200. FIG. [Figure 8] FIG. 13 illustrates an example of a graph editor used when inputting adjustment information. [Figure 9] FIG. 13 is a diagram showing a process of generating a second image sequence after adjusting for changes in motion from the first image data and the second image data. [Figure 10] FIG. 13 is a diagram supplementing the processing of activity A280. [Figure 11] This is a diagram supplementing the processing of activities A310 to A340. [Figure 12] This is a diagram supplementing the processing of activities A310 to A340. [Figure 13] This is a diagram supplementing the processing of activities A310 to A340. [Figure 14] This is a diagram supplementing the processing of activities A310 to A340. [Figure 15] FIG. 13 is a diagram supplementing the processing of activity A430. [Figure 16] 11A and 11B are diagrams illustrating a comparison between image data subjected to color reduction processing by the method of the present embodiment and image data subjected to color reduction processing by a conventional method. [Figure 17] 11A and 11B are diagrams illustrating a comparison between image data subjected to color reduction processing by the method of the present embodiment and image data subjected to color reduction processing by a conventional method. [Figure 18] FIG. 13 is a diagram illustrating a prompt, which is an example of state information. [Figure 19] FIG. 13 is a diagram showing an example of setting a line graph in the graph editor. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. Various characteristic features shown in the following embodiments can be combined with each other.

[0013] Incidentally, a program for realizing the software appearing in one embodiment may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server, or may be provided so that the program is launched on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0014] In addition, in various information processing according to an embodiment, an input and an output according to the input can be realized. Here, as long as an output is obtained as a result of the input, the form of information referenced in such information processing (hereinafter referred to as reference information) is not limited. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a judgment formula such as a regression formula constructed by a statistical method), or may be a trained model that has previously trained the correlation between the input and the output, or may be a large-scale language model that can output a desired result by inputting a prompt.

[0015] In one embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In one embodiment, various information is handled, and this information is represented by, for example, physical values ​​of signal values ​​representing voltage and current, high and low signal values ​​as a binary bit collection consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculation can be performed on the circuit in the broad sense.

[0016] Furthermore, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, and a memory. The processor may be a general-purpose processor or a dedicated circuit. In other words, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0017] 1. Hardware Configuration In the first section, the hardware configuration of this embodiment will be described.

[0018] 1-1. Information processing system 100 FIG. 1 is a configuration diagram showing an information processing system 100. The information processing system 100 includes an information processing device 200 and a terminal 300, which are connected via a network. These components will be further described. Here, a system exemplified as the information processing system 100 is composed of one or more devices or components. Therefore, for example, even the information processing device 200 alone can become a system exemplified as the information processing system 100.

[0019] 1-2. Information processing device 200 2 is a block diagram showing a hardware configuration of the information processing device 200. The information processing device 200 has a control unit 210, a storage unit 220, and a communication unit 250, and these components are electrically connected via a communication bus 260 inside the information processing device 200. The information processing device 200 is, for example, a server. Each component will be further described.

[0020] The control unit 210 performs processing and control of the overall operation related to the information processing device 200. The control unit 210 is, for example, a central processing unit (CPU) not shown. The control unit 210 realizes various functions related to the information processing device 200 by reading out a predetermined program stored in the storage unit 220. That is, information processing by software stored in the storage unit 220 can be specifically realized by the control unit 210, which is an example of hardware, and executed as each functional unit included in the control unit 210. These will be further described in Section 2. Note that the control unit 210 is not limited to being single, and may be implemented to have multiple control units 210 for each function. Also, a combination of these may be used.

[0021] The storage unit 220 stores various information necessary for information processing of the information processing device 200. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs related to the information processing device 200 executed by the control unit 210, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the calculation of the program. Also, it may be a combination of these.

[0022] The communication unit 250 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., but may also include wireless LAN network communication, mobile communication such as 5G / LTE / 3G, Bluetooth (registered trademark) communication, etc. as necessary. In other words, it is more preferable to implement it as a collection of multiple communication means. In other words, the information processing device 200 communicates various information with the terminal 300 via the communication unit 250 and the network.

[0023] 1-3. Terminal 300 3 is a block diagram showing a hardware configuration of the terminal 300. The terminal 300 has a control unit 310, a storage unit 320, a display unit 330, an input unit 340, and a communication unit 350, and these components are electrically connected inside the terminal 300 via a communication bus 360. The terminal 300 is, for example, a desktop personal computer. Descriptions of the control unit 310, the storage unit 320, and the communication unit 350 are omitted because they are substantially similar to the descriptions of the control unit 210, the storage unit 220, and the communication unit 250 in the information processing device 200.

[0024] The display unit 330 may be included in the housing of the terminal 300 or may be externally attached. The display unit 330 displays a screen of a graphical user interface (GUI) that can be operated by a user. This is preferably implemented by using display devices such as a CRT display, a liquid crystal display, an organic EL display, and a plasma display depending on the type of the terminal 300. In the following, the display unit 330 will be described as being included in the housing of the terminal 300.

[0025] The input unit 340 may be included in the housing of the terminal 300, or may be externally attached. For example, the input unit 340 may be implemented as a touch panel integrated with the display unit 330. If it is a touch panel, the user can input a tap operation, a swipe operation, or the like. Of course, a switch button, a mouse, a QWERT keyboard, or the like may be adopted instead of the touch panel. That is, the input unit 340 accepts an operation input made by the user. The input is transferred to the control unit 310 via the communication bus 360 as a command signal. Then, the control unit 310 can execute a predetermined control or calculation as necessary.

[0026] 2. Functional configuration The functional configuration of this embodiment will be described in Section 2. As described above, information processing by software stored in the storage unit 220 is specifically realized by the control unit 210, which is an example of hardware, and can be executed as each functional unit included in the control unit 210.

[0027] FIG. 4 is a block diagram showing functions realized by the information processing device 200 (control unit 210). As described above, the information processing device 200 (information processing system 100) includes the control unit 210. Specifically, the information processing device 200 (control unit 210) is configured to execute each step in the information processing method of this embodiment. The information processing device 200 (control unit 210) includes an acquisition unit 211, a generation unit 212, an adjustment unit 213, an output unit 214, a deletion unit 215, a separation unit 216, a color reduction unit 217, and an integration unit 218, corresponding to each step of the information processing method of this embodiment. Each unit and each step will be described below in association with each other.

[0028] The acquisition unit 211 is configured to acquire various information. The acquisition unit 211 is configured to execute an acquisition step. For example, the acquisition unit 211 acquires first image data and state information. The first image data is assumed to be reference image data. Here, the state information is at least one of second image data indicating an image after the first image data has changed and a prompt indicating a state after the first image data has changed. In addition, the acquisition unit 211 acquires third image data which is image data that has been colored.

[0029] The generating unit 212 is configured to generate various information. The generating unit 212 is configured to execute a generating step. For example, the generating unit 212 generates a plurality of intermediate image data, which is image data that interpolates between the first image data and the status information. The generating unit 212 also generates a first image sequence, which is data in which the first image data, the plurality of intermediate image data, and the status information are arranged in chronological order.

[0030] The adjustment unit 213 is configured to adjust various information. The adjustment unit 213 is configured to execute an adjustment step. For example, the adjustment unit 213 adjusts a change in movement expressed in the first image sequence based on the first image sequence and the input adjustment information. Here, the adjustment information is information for adjusting a change in movement expressed in the image sequence.

[0031] The output unit 214 is configured to output various information. The output unit 214 is configured to execute an output step. For example, the output unit 214 outputs a second image sequence that is an adjusted image sequence. The output unit 214 also outputs fourth image data that is image data after color reduction.

[0032] The deletion unit 215 is configured to delete various information. The deletion unit 215 is configured to execute a deletion step. For example, when the second image sequence is confirmed, the deletion unit 215 deletes data related to the second image sequence from the storage unit. Also, when the fourth image data is confirmed, the deletion unit 215 deletes data related to the fourth image data from the storage unit.

[0033] The separation unit 216 is configured to separate various pieces of information. The separation unit 216 is configured to perform a separation step. For example, the separation unit 216 separates a line drawing portion and a painted portion from the third image data.

[0034] The color reduction unit 217 is configured to reduce the colors of various information. The color reduction unit 217 is configured to execute a color reduction step. For example, the color reduction unit 217 performs clustering processing on the colors included in the third image data, and reduces the colors of the third image data according to the color reduction information indicating the colors corresponding to each cluster. Here, the colors corresponding to each cluster are colors determined using the most frequent values ​​of each cluster. Note that the colors after color reduction are, for example, colors such as blue, yellow, and red, and are also referred to as the types of colors after color reduction in the following description.

[0035] The integrating unit 218 is configured to integrate various pieces of information. The integrating unit 218 is configured to execute an integration step. For example, the integrating unit 218 integrates a line drawing portion and a painted portion after color reduction.

[0036] 3. Information processing method In Section 3, we will explain the flow of the information processing method of the information processing device 200 described above. This information processing method includes an acquisition step, a generation step, an adjustment step, an output step, a deletion step, a separation step, a color reduction step, and an integration step. Each of these steps is executed by the control unit 210 in the information processing device 200, as described in Section 2.

[0037] 5 to 7 are activity diagrams showing the flow of an information processing method executed by the information processing device 200. The following description will be given along with each activity of these activity diagrams. In this embodiment, the first image data and the second image data are assumed to be prepared in advance in the terminal 300. The first image data is assumed to be reference image data. The second image data is data showing an image after the first image data has been changed. Here, the first image data and the second image data are assumed to be colored image data. In this manner, according to the aspect in which colored image data is used as an input image, the accuracy of the finally generated image sequence can be improved compared to the conventional art.

[0038] First, the control unit 310 in the terminal 300 transmits the first image data and the second image data to the information processing device 200 (activity A110). The second image data is an example of "status information" in the claims. The status information is at least one of the second image data showing an image after the change of the first image data, and a prompt showing the state after the change of the first image data. The prompt will be described later. In the activity A110, for example, the following two-stage information processing is executed. (1) The control unit 310 reads out the first image data and the second image data from the storage unit 320. (2) The control unit 310 transmits the first image data and the second image data to the information processing device 200 via the communication unit 350.

[0039] Next, the control unit 210 in the information processing device 200 receives the first image data and the second image data from the terminal 300 (activity A120). In other words, in the acquisition step, the first image data and the status information are acquired. In the activity A120, for example, the following two-stage information processing is executed. (1) The communication unit 250 receives the first image data and the second image data from the terminal 300. (2) The control unit 210 stores the first image data and the second image data in the memory unit 220.

[0040] Next, the control unit 210 in the information processing device 200 generates image data (hereinafter also referred to as "intermediately divided image data") that interpolates between the first image data and the second image data (activity A130). In other words, in the generation step, a plurality of intermediately divided image data, which is image data that interpolates between the first image data and the status information, is generated. In activity A130, for example, the following three-stage information processing is executed. (1) The control unit 210 reads out the first image data and the second image data from the storage unit 220. (2) The control unit 210 executes a generation process and generates a plurality of intermediately divided image data. (3) The control unit 210 stores the plurality of intermediately divided image data in the storage unit 220.

[0041] Next, the control unit 210 in the information processing device 200 generates an image sequence (hereinafter also referred to as "first image sequence") in which the first image data, the plurality of intermediately divided image data, and the second image data are arranged in chronological order (activity A140). In other words, in the generation step, the first image sequence is generated, which is data in which the first image data, the plurality of intermediately divided image data, and the state information are arranged in chronological order. In the activity A140, for example, the following three-stage information processing is executed. (1) The control unit 210 reads out the first image data, the plurality of intermediately divided image data, and the second image data from the storage unit 220. (2) The control unit 210 executes a generation process, and arranges the first image data, the plurality of intermediately divided image data, and the second image data in chronological order to generate the first image sequence. (3) The control unit 210 stores the first image sequence in the storage unit 220.

[0042] Next, the control unit 210 in the information processing device 200 transmits the first image sequence to the terminal 300 (activity A150). In activity A150, for example, the following two-stage information processing is executed. (1) The control unit 210 reads out the first image sequence from the storage unit 220. (2) The control unit 210 transmits the first image sequence to the terminal 300 via the communication unit 250.

[0043] Next, the control unit 310 in the terminal 300 receives the first image sequence from the information processing device 200 (activity A160). In activity A160, for example, the following two-stage information processing is executed. (1) The communication unit 350 receives the first image sequence from the information processing device 200. (2) The control unit 310 stores the first image sequence in the memory unit 320.

[0044] Next, control unit 310 in terminal 300 causes display unit 330 to display the first image sequence (activity A170). In activity A170, for example, the following two-stage information processing is executed. (1) Control unit 310 reads out the first image sequence from storage unit 320. (2) Control unit 310 causes display unit 330 to display the first image sequence.

[0045] Next, the control unit 310 in the terminal 300 receives adjustment information for adjusting the change in the movement expressed in the first image sequence (activity A180). Here, the adjustment information is information for adjusting the change in the movement expressed in the first image sequence. The adjustment information also includes image number information that is information indicating the number of image data included in the second image sequence.

[0046] If the control unit 310 determines that the adjustment information has been received, it transitions to processing of activity A190 (YES in activity A180). If the control unit 310 determines that the adjustment information has not been received, it transitions to processing of activity A260 (NO in activity A180). In activity A180, for example, the following four stages of information processing are executed. (1) If the input unit 340 receives the adjustment information, it transmits an acceptance signal to the control unit 310 via the communication bus 360. (2) If the input unit 340 does not receive the adjustment information, it transmits a non-acceptance signal to the control unit 310 via the communication bus 360. (3) If the control unit 310 receives an acceptance signal, it stores the adjustment information in the storage unit 320 and transitions to processing of activity A190. (4) If the control unit 310 receives a non-acceptance signal, it transitions to processing of activity A260.

[0047] Next, the control unit 310 in the terminal 300 transmits adjustment information, which is information for adjusting changes in movement expressed in the first image sequence, to the information processing device 200 (activity A190). In the activity A190, for example, the following two-stage information processing is executed. (1) The control unit 310 reads out the adjustment information from the storage unit 320. (2) The control unit 310 transmits the adjustment information to the information processing device 200 via the communication unit 350.

[0048] FIG. 8 is a diagram showing an example of a graph editor used when inputting adjustment information. A line graph 411, a setting area 412, and a graph area 413 are displayed in the graph editor 410. The line graph 411 is a graph for adjusting the degree of change between the first image data A and the second image data B. The setting area 412 is an area for setting the number of image data (image number information) included in the second image sequence output after adjusting the change in movement. The graph area 413 is a variable area of ​​the line graph 411. The lower base of the graph area 413 is the first image data A, and the upper base is the second image data B. That is, the closer to the lower base of the line graph 411, the closer to the representation of the first image data A, and the closer to the upper base, the closer to the representation of the second image data B.

[0049] The shape of the line graph 411 can be changed by dragging and dropping the six movable points displayed in total. The example shown in Fig. 8 shows a gradual change from the first image data A to the second image data B over time.

[0050] In this way, the adjustment information is information input via the graph editor 410. According to this aspect, it is possible to adjust the change in movement while visually checking. Here, the graph editor 410 is an editor for inputting the adjustment information using a line graph. According to this aspect, it is possible to more easily adjust the change in movement while visually checking.

[0051] Returning to the explanation of Fig. 5, the control unit 210 in the information processing device 200 receives adjustment information from the terminal 300, which is information for adjusting changes in movement expressed in the first image sequence (activity A200). In activity A200, for example, the following two-stage information processing is executed. (1) The communication unit 250 receives the adjustment information from the terminal 300. (2) The control unit 210 stores the adjustment information in the storage unit 220.

[0052] Next, the control unit 210 in the information processing device 200 deletes some of the intermediately divided image data included in the first image sequence based on the first image sequence and the input adjustment information (activity A210). In other words, in the adjustment step, the change in movement expressed in the first image sequence is adjusted based on the first image sequence and the input adjustment information. More specifically, in the adjustment step, the change in movement is adjusted by deleting some of the intermediately divided image data among the plurality of intermediately divided image data from the first image sequence. According to this aspect, the data capacity of the image sequence can be reduced to adjust the change in movement. Also, in the adjustment step, the number of image data included in the second image sequence is adjusted based on image number information that is information indicating the number of image data included in the second image sequence. According to this aspect, the second image sequence can be changed quickly in a short time or slowly in a relatively long time by balancing the frame rate of the second image sequence and the number of image data included in the second image sequence.

[0053] In activity A210, for example, the following three-stage information processing is executed. (1) The control unit 210 reads out the first image sequence and adjustment information from the storage unit 220. (2) The control unit 210 executes an adjustment process and deletes some of the in-between image data included in the first image sequence. (3) The control unit 210 stores the image sequence after the adjustment process (hereinafter also referred to as the "second image sequence") in the storage unit 220.

[0054] 9 is a diagram showing a process of generating a second image sequence after adjusting for changes in motion from the first image data and the second image data. First, the first image data A and the second image data B are prepared. Next, intermediate image data 1 to 10 are generated between the first image data A and the second image data B. Next, the intermediate image data 2, 5, 8, and 9 are deleted to generate the second image sequence.

[0055] In the example of Fig. 9, it is assumed that the frame rate of the first image sequence is set to 12 fps. Also, it is assumed that the number of pieces of image data included in the second image sequence is set to 8. The second image sequence can show an operation in which the intermediate divided image data changes from 1 to 3 at an accelerated rate, changes from 4 to 6 at an accelerated rate, and changes from 7 to 10 at an accelerated rate.

[0056] Here, when the frame rate of the second image sequence is set to 12 fps, the speed of change of the second image sequence is changed so as to accelerate by skipping the portions where the in-between image data has been deleted, with the speed of change of the first image sequence being used as the standard. When the frame rate of the second image sequence is set to 8 fps, the speed of change of the second image sequence becomes gentler overall, and the speed of change of the second image sequence becomes equivalent to the speed of change of the first image sequence, with the portions where the in-between image data has been deleted being skipped.

[0057] Returning to the explanation of FIG. 5, the control unit 210 in the information processing device 200 then transmits the second image sequence to the terminal 300 (activity A220). In other words, in the output step, the second image sequence, which is the adjusted image sequence, is output. In activity A220, for example, the following two-stage information processing is executed. (1) The control unit 210 reads out the second image sequence from the storage unit 220. (2) The control unit 210 transmits the second image sequence to the terminal 300 via the communication unit 250.

[0058] Next, the control unit 310 in the terminal 300 receives the second image sequence from the information processing device 200 (activity A230). In activity A230, for example, the following two-stage information processing is executed. (1) The communication unit 350 receives the second image sequence from the information processing device 200. (2) The control unit 310 stores the second image sequence in the memory unit 320.

[0059] Next, the control unit 310 in the terminal 300 causes the second image sequence to be displayed on the display unit 330 (activity A240). In activity A240, for example, the following two-stage information processing is executed. (1) The control unit 310 reads out the second image sequence from the storage unit 320. (2) The control unit 310 causes the display unit 330 to display the second image sequence.

[0060] We move on to the explanation of FIG. 6. Next, the control unit 310 in the terminal 300 accepts adjustment information, which is information for adjusting the change in the movement expressed in the first image sequence (activity A250). If the control unit 310 determines that the adjustment information has been accepted, it proceeds to processing of activity A190 (YES in activity A250). If the control unit 310 determines that the adjustment information has not been accepted, it proceeds to processing of activity A260 (NO in activity A250). In activity A250, for example, the following four stages of information processing are executed. (1) If the input unit 340 accepts the adjustment information, it transmits an acceptance signal to the control unit 310 via the communication bus 360. (2) If the input unit 340 does not accept the adjustment information, it transmits a non-acceptance signal to the control unit 310 via the communication bus 360. (3) If the control unit 310 receives an acceptance signal, it stores the adjustment information in the storage unit 320 and proceeds to processing of activity A190. (4) When the control unit 310 receives a non-acceptance signal, the control unit 310 proceeds to the processing of activity A260.

[0061] Next, the control unit 310 in the terminal 300 transmits setting information including the number of colors after color reduction to the information processing device 200 (activity A260). In this embodiment, it is assumed that the setting information is set as a default. In activity A260, for example, the following two-stage information processing is executed. (1) The control unit 310 reads out the setting information from the storage unit 320. (2) The control unit 310 transmits the setting information to the information processing device 200 via the communication unit 350.

[0062] Next, the control unit 210 in the information processing device 200 receives the setting information from the terminal 300 (activity A270). In the activity A270, for example, the following two-stage information processing is executed. (1) The communication unit 250 receives the setting information from the terminal 300. (2) The control unit 210 stores the setting information in the storage unit 220.

[0063] Next, the control unit 210 in the information processing device 200 separates each image data included in the second image sequence into a line drawing portion and a painted portion (activity A280). In other words, in the separation step, the line drawing portion and the painted portion are separated from the image data. In activity A280, for example, the following three-stage information processing is executed. (1) The control unit 210 reads out the second image sequence and an image separation program from the storage unit 220. (2) The control unit 210 executes a separation process and separates each image data included in the second image sequence into a line drawing portion and a painted portion. (3) The control unit 210 stores the line drawing portion and the painted portion (hereinafter simply referred to as "line drawing portion" and "painted portion") of each image data in the storage unit 220.

[0064] 10 is a diagram supplementing the process of activity A280. Image data 510 is assumed to be any one of the image data included in the second image sequence. In activity A280, image data 510 is separated into a line drawing portion 520 and a painted portion 530.

[0065] Returning to the description of FIG. 6, the control unit 210 in the information processing device 200 subsequently performs color reduction processing on the painted portion (activity A290). In other words, in the color reduction step, the colors in the painted portion (colors included in the third image data) are clustered, and the painted portion (third image data) is color reduced according to the color-reduced information indicating the color corresponding to each cluster. Here, the color corresponding to each cluster is a color determined using the most frequent value of each cluster. The number of clusters in the clustering process corresponds to the number of colors after color reduction included in the setting information. That is, in the color reduction step, the painted portion (image data) is color reduced according to the set number of colors. According to this aspect, the number of colors after color reduction can be set arbitrarily. In addition, in the color reduction step, first color-reduced information and second color-reduced information are generated according to the set number of colors. Here, both the first color-reduced information and the second color-reduced information indicate colors corresponding to each cluster, and are equal to each other.

[0066] In activity A290, for example, the following three-stage information processing is executed. (1) The control unit 210 reads out the setting information and the painted portion from the storage unit 220. (2) The control unit 210 performs a clustering process on the colors in the painted portion, and reduces the colors of the painted portion according to the colors corresponding to each cluster. (3) The control unit 210 stores in the storage unit 220 the painted portion after the color reduction and the type of color in the painted portion after the color reduction (hereinafter also referred to as "post-color reduction information").

[0067] Next, the control unit 210 in the information processing device 200 transmits the portion painted after color reduction and the type of color in the portion painted after color reduction to the terminal 300 (activity A300). In activity A300, for example, the following three-stage information processing is executed. (1) The control unit 210 reads out the portion painted after color reduction and the second color-reduction information from the storage unit 220. (2) The control unit 210 transmits the portion painted after color reduction and the second color-reduction information to the terminal 300 via the communication unit 250.

[0068] Next, the control unit 310 in the terminal 300 receives the painted portion after color reduction and the type of color in the painted portion after color reduction from the information processing device 200 (activity A310). In activity A310, for example, the following two-stage information processing is executed. (1) The communication unit 350 receives the painted portion after color reduction and second color-reduction information from the information processing device 200. (2) The control unit 310 stores the painted portion after color reduction and the second color-reduction information in the storage unit 320.

[0069] Next, the control unit 310 in the terminal 300 causes the display unit 330 to display the painted portion after color reduction and the type of color in the painted portion after color reduction (activity A320). In other words, in the color reduction step, the painted portion after color reduction and the second color-reduction information are displayed according to the set number of colors. In activity A320, for example, the following two-stage information processing is executed. (1) The control unit 310 reads out the painted portion after color reduction and the second color-reduction information from the storage unit 320. (2) The control unit 310 causes the display unit 330 to display the painted portion after color reduction and the second color-reduction information.

[0070] Next, the control unit 310 in the terminal 300 accepts a change operation of the type of color after color reduction (second post-color-reduction information) (activity A330). If the control unit 310 determines that the change operation has been accepted, it proceeds to processing of activity A340 (YES in activity A330). If the control unit 310 determines that the change operation has not been accepted, it proceeds to processing of activity A410 (NO in activity A330). In activity A330, for example, the following four stages of information processing are executed. (1) If the input unit 340 accepts the change operation, it transmits an acceptance signal to the control unit 310 via the communication bus 360. (2) If the input unit 340 does not accept the change operation, it transmits a non-acceptance signal to the control unit 310 via the communication bus 360. (3) If the control unit 310 receives the acceptance signal, it stores the change operation in the storage unit 320 and proceeds to processing of activity A340. (4) When the control unit 310 receives a non-acceptance signal, the control unit 310 proceeds to the processing of activity A410.

[0071] Next, the control unit 310 in the terminal 300 generates modification information for changing the type of color after color reduction (second color-reduction information) (activity A340). In activity A340, for example, the following three-stage information processing is executed. (1) The control unit 310 reads out a modification operation from the storage unit 320. (2) The control unit 310 executes a generation process to generate modification information for changing the type of color after color reduction (second color-reduction information). (3) The control unit 310 stores the modification information in the storage unit 320.

[0072] 11 to 14 are diagrams supplementing the processing of activities A310 to A340. FIG. 11 shows a palette 600 displayed before data received by activity A310 is read. The palette 600 displays a number of colors area 610, a color reduction method area 620, and an add button 630. In the number of colors area 610, the number of colors after color reduction can be set. In the color reduction method area 620, the algorithm for color reduction processing can be set, and here, the algorithm of this embodiment (clustering processing using the most frequent value) is set. The add button 630 is a button for increasing the number of colors after color reduction.

[0073] 12 shows a palette 600 that is displayed after moving to the processing of activity A320. Palette 600 corresponds to "second color-reduction information" in the claims. Palette 600 displays a color number area 610, a color-reduction method area 620, an add button 630, and color types 640. Each color type 640 displays a delete button 641. The delete button 641 is a button for deleting that color. Each color in color types 640 is a color extracted by a clustering process that uses the most frequent value.

[0074] 13 shows the state where the process moves to activity A330, and the delete buttons 641 for three of the colors displayed in color types 640 are clicked, resulting in a reduced number of colors of 15. In the color reduction step, if any color is deleted from the colors after color reduction, the deleted color is further deleted from the painted portion.

[0075] Fig. 14(A) shows a state in which the add button 630 in Fig. 13 is clicked and color 642 is added. Then, when color 642 is selected, a color setting box 650 is displayed as shown in Fig. 14(B). Areas 651, 652, and 653 are displayed in color setting box 650. Color 642 is set by entering an arbitrary color in area 651, operating the slider in area 652, and inputting a numerical value in area 653. In the color reduction step, if an arbitrary color is added to the color after color reduction, the added color is added to the painted portion.

[0076] Here, the first color-reduction information is held in the storage unit 220 in the information processing device 200. Therefore, the first color-reduction information is not changed, and the colors corresponding to each cluster can be maintained.

[0077] Returning to the explanation of Fig. 6, the control unit 310 in the terminal 300 then transmits modification information for changing the type of color after color reduction (second post-color-reduction information) to the information processing device 200 (activity A350). In activity A350, for example, the following two-stage information processing is executed. (1) The control unit 310 reads out the modification information from the storage unit 320. (2) The control unit 310 transmits the modification information to the information processing device 200 via the communication unit 350.

[0078] Next, the control unit 210 in the information processing device 200 receives modification information for changing the type of color after color reduction (second color-reduction information) from the terminal 300 (activity A360). In activity A360, for example, the following two-stage information processing is executed. (1) The communication unit 250 receives the modification information from the terminal 300. (2) The control unit 210 stores the modification information in the storage unit 220.

[0079] The explanation of FIG. 7 will be shifted to. Next, the control unit 210 in the information processing device 200 reflects the type of color after color reduction included in the modification information in the painted portion (activity A370). In other words, in the color reduction step, when the second color-reduced information is changed, the second color-reduced information after the change is reflected in the painted portion (image data). In the activity A370, for example, the following three-stage information processing is executed. (1) The control unit 210 reads out the painted portion after color reduction and the modification information from the storage unit 220. (2) The control unit 210 executes a reflection process, and reflects the type of color after color reduction included in the modification information in the painted portion. (3) The control unit 210 stores the painted portion after the reflection process (hereinafter also referred to as the "reflected painted portion") in the storage unit 220. According to this aspect, while maintaining each cluster in the first color-reduced information, it is possible to change the second color-reduced information to change the color corresponding to each cluster.

[0080] Next, the control unit 210 in the information processing device 200 transmits the painted portion after the reflection process to the terminal 300 (activity A380). In activity A380, for example, the following two-stage information processing is executed. (1) The control unit 210 reads out the reflected painted portion from the storage unit 220. (2) The control unit 210 transmits the reflected painted portion to the terminal 300 via the communication unit 250.

[0081] Next, the control unit 310 in the terminal 300 receives the painted portion after the reflection process from the information processing device 200 (activity A390). In activity A390, for example, the following two-stage information processing is executed. (1) The communication unit 350 receives the reflected painted portion from the information processing device 200. (2) The control unit 310 stores the reflected painted portion in the memory unit 320.

[0082] Next, the control unit 310 in the terminal 300 accepts a change operation of the type of color after color reduction (second post-color-reduction information) (activity A400). If the control unit 310 determines that the change operation has been accepted, it transitions to processing of activity A340 (YES in activity A400). If the control unit 310 determines that the change operation has not been accepted, it transitions to processing of activity A410 (NO in activity A400). In activity A400, for example, the following four stages of information processing are executed. (1) If the input unit 340 accepts the change operation, it transmits an acceptance signal to the control unit 310 via the communication bus 360. (2) If the input unit 340 does not accept the change operation, it transmits a non-acceptance signal to the control unit 310 via the communication bus 360. (3) If the control unit 310 receives the acceptance signal, it stores the change operation in the storage unit 320 and transitions to processing of activity A340. (4) When the control unit 310 receives a non-acceptance signal, the control unit 310 proceeds to the processing of activity A410.

[0083] Next, the control unit 310 in the terminal 300 transmits an end signal, which is a signal for terminating the color reduction process, to the information processing device 200 (activity A410). The end signal is generated, for example, by executing an end operation in the palette 600. In the activity A410, for example, the following two-stage information processing is executed. (1) The control unit 310 reads out the end signal from the storage unit 320. (2) The control unit 310 transmits the end signal to the information processing device 200 via the communication unit 350.

[0084] Next, the control unit 210 in the information processing device 200 receives an end signal, which is a signal to end the color reduction processing, from the terminal 300 (activity A420). In activity A420, for example, the following two-stage information processing is executed. (1) The communication unit 250 receives the end signal from the terminal 300. (2) The control unit 210 stores the end signal in the storage unit 220.

[0085] Next, the control unit 210 in the information processing device 200 integrates the line drawing portion of each image data included in the second image sequence with the painted portion of each image data after the reflection process (activity A430). In other words, in the integration step, the line drawing portion is integrated with the painted portion after the color reduction. In activity A430, for example, the following three-stage information processing is executed. (1) The control unit 210 reads out the line drawing portion and the reflected painted portion from the storage unit 220. (2) The control unit 210 executes the integration step and integrates the line drawing portion with the reflected painted portion. (3) The control unit 210 stores the integrated image data (hereinafter also referred to as "fourth image data") in the storage unit 220.

[0086] 15 is a diagram supplementing the processing of activity A430. In activity A430, line drawing portion 520 and color-reduced painted portion 540 are integrated to generate integrated image data 550 (fourth image data). Line drawing portion 520, color-reduced painted portion 540, and integrated image data 550 are assumed to be any one of the image data. In activity A430, line drawing portion 520 and color-reduced painted portion 540 of each image data in the second image sequence are integrated, respectively, to generate an integrated image sequence in which each integrated image data 550 is an image sequence.

[0087] Next, the control unit 210 in the information processing device 200 transmits the combined image sequence to the terminal 300 (activity A440). In other words, in the output step, the fourth image data, which is the image data after color reduction, is output. In activity A440, for example, the following two-stage information processing is executed. (1) The control unit 210 reads out the combined image sequence from the storage unit 220. (2) The control unit 210 transmits the combined image sequence to the terminal 300 via the communication unit 250.

[0088] Next, the control unit 210 in the information processing device 200 deletes data relating to the second image sequence and data relating to the fourth image data from the storage unit 220 (activity A450). In other words, in the deleting step, when the second image sequence is confirmed, data relating to the second image sequence is deleted from the storage unit. According to this aspect, it is possible to protect information relating to the image sequence in which a change in movement has been confirmed. Also, in the deleting step, when the fourth image data is confirmed, data relating to the fourth image data is deleted from the storage unit. According to this aspect, it is possible to protect information relating to the image data after the color reduction process.

[0089] In activity A450, for example, the following two-stage information processing is executed: (1) The control unit 210 reads out data related to the second image sequence and data related to the fourth image data from the storage unit 220. (2) The control unit 210 executes a deletion process and deletes the data related to the second image sequence and data related to the fourth image data from the storage unit 220.

[0090] Next, the control unit 310 in the terminal 300 receives the integrated image sequence from the information processing device 200 (activity A460). In activity A460, for example, the following two-stage information processing is executed. (1) The communication unit 350 receives the integrated image sequence from the information processing device 200. (2) The control unit 310 stores the integrated image sequence in the storage unit 320.

[0091] Next, control unit 310 in terminal 300 causes display unit 330 to display the integrated image sequence (activity A470). In activity A470, for example, the following two-stage information processing is executed. (1) Control unit 310 reads out the integrated image sequence from storage unit 320. (2) Control unit 310 causes display unit 330 to display the integrated image sequence.

[0092] 16 and 17 are diagrams showing a comparison between image data subjected to color reduction processing using the method of this embodiment and image data subjected to color reduction processing using a conventional method. Original image 700 is image data before color reduction processing. Comparative example data 710 is image data subjected to color reduction processing using a conventional method (k-means). Comparative example data 720 is image data subjected to color reduction processing using another conventional method (DBSCAN). Example data 730 is image data subjected to color reduction processing using the method of this embodiment.

[0093] Here, when checking Fig. 17 which is an enlarged view of region 740 in Fig. 16, color noise occurs in both comparative example data 710 and comparative example data 720. On the other hand, no color noise occurs in example data 730, similar to original image 700. This shows that the color reduction process of this embodiment can improve color reproducibility after color reduction process compared to the conventional method.

[0094] According to the information processing method of this embodiment, it is possible to adjust the change in the movement expressed in the generated image sequence. Also, according to the information processing method of this embodiment, it is possible to improve the reproducibility of the color after the color reduction process more than before. Furthermore, because of the simple configuration, the saved resources can be used for other core functions.

[0095] Although the embodiment of the present invention has been described above, the present invention is not limited to this, and can be modified as appropriate without departing from the technical concept of the invention.

[0096] 4. Variations Modifications of this embodiment will be described in Section 4. The following modifications can be combined as appropriate.

[0097] An aspect of the present embodiment may be a program. The program is configured to cause a computer to execute each step of the information processing method of the present embodiment.

[0098] The control unit 210 performs write (storage) and read processes for various data and information in the memory unit 220, but this is not limited to this, and for example, information processing for each activity may be performed using a register or cache memory within the control unit 210.

[0099] In the present embodiment, the second image data is described as being one image, but is not limited to this. The second image data may be two or more images. In this case, intermediate image data is generated between the first image data and each of the second image data.

[0100] Fig. 18 is a diagram for explaining a prompt, which is an example of state information. Fig. 18(A) is a diagram showing an input box for the prompt. Fig. 18(B) is a diagram showing an example of a graph editor corresponding to the prompt.

[0101] When the state information is a prompt, an arbitrary prompt is input into the input box 420 in activity A110. Here, it is assumed that "Angry" is input. After that, a graph editor 430 is displayed in activity A180. In the graph editor 430, a line graph 431, a setting area 432, and a graph area 433 are displayed.

[0102] Line graph 431 is a graph for adjusting the degree of change between the first image data A and Anger. Setting area 432 is an area for setting the number of image data (image number information) included in the second image sequence to be output after adjusting the change in movement. Graph area 433 is a variable area of ​​line graph 431. Graph area 433 has the first image data A as its lower base and Anger as its upper base. That is, the closer line graph 411 is to the lower base, the closer it is to the representation of the first image data A, and the closer it is to the upper base, the closer it is to the representation of Anger.

[0103] In the example of FIG. 18, if the first image data A is an image showing a normal facial expression, the image shows a state in which the facial expression gradually changes to an angry facial expression over time. Here, the type of prompt is not particularly limited as long as it is in a natural language. Therefore, for example, it may be "sorrow" showing sadness, "joy" showing joy, or a natural language other than English.

[0104] FIG. 19 is a diagram showing an example of setting a line graph in a graph editor. A line graph 441, a setting area 442, and a graph area 443 are displayed in the graph editor 440. The line graph 441 shows a state in which the image data changes suddenly from the first image data A to the vicinity of the second image data B, then changes suddenly from the vicinity of the second image data B to the vicinity of the first image data A, then gradually changes from the vicinity of the first image data A to the second image data B side, and then changes from the second image data B side to the first image data A side. In this way, the adjustment information may be information for adjusting a movement that changes from the first image data to the state information, and a movement that changes from the state information to the first image data. According to this aspect, it is possible to adjust the change in a movement that requires repetition, such as blinking or lip-syncing.

[0105] The type of color after the change included in the change information generated in the activity A350 may be applicable to image data other than the third image data or the fourth image data. In this case, the type of color after the change may be stored in the storage unit 220 in the information processing device 200, in the storage unit 320 in the terminal 300, or in an external hard disk drive. According to such an embodiment, the type of color set as the color after color reduction can be applied to another image.

[0106] In this embodiment, the color-reduced information has been described as the first color-reduced information and the second color-reduced information, but is not limited to this. In the color-reduced step, the color-reduced information may be displayed according to the set number of colors. In the color-reduced step, if the color-reduced information is changed, the color-reduced information after the change may be reflected in the painted portion (image data). According to this aspect, the color after color reduction can be changed arbitrarily.

[0107] In the present embodiment, an example has been shown in which the adjustment information is input by operating a line graph, but the present invention is not limited to this. For example, the adjustment information may be input by operating a bar graph or a pie chart, or may be input by inputting something other than a graph, such as a predetermined index.

[0108] Activities A110 to A250 and activities A260 to A470 may be executed independently. In this case, in activity A260, colored image data may be transmitted to information processing device 200, and in activity A270, colored image data may be received. In other words, in the acquisition step, third image data that is colored image data is acquired.

[0109] In the present embodiment, the information processing method of the present embodiment has been described as being executed between the information processing device 200 and the terminal 300, but the present invention is not limited to this. For example, when the program of the present embodiment is installed in the terminal 300, the information processing method of the present embodiment may be executed by the terminal 300 alone.

[0110] In this embodiment, a CPU is given as an example of the control unit 210 in the information processing device 200 and the control unit 310 in the terminal 300, but the present invention is not limited thereto. The control unit 210 and the control unit 310 may each be a CPU, a graphics processing unit (GPU), a neural processing unit (NPU), a tensor processing unit (TPU), or a combination of these processors. That is, the control unit 210 and the control unit 310 refer to one or more of the above processors, and the information processing method of this embodiment may be executed by making these processors work together.

[0111] 5.Other It may be provided in any of the following ways:

[0112] (1) An information processing method comprising an acquisition step, a color reduction step, and an output step, wherein in the acquisition step, third image data which is colored image data is acquired, in the color reduction step, colors contained in the third image data are clustered, and the third image data is color-reduced according to post-color reduction information indicating a color corresponding to each cluster, the color corresponding to each cluster is a color determined using the most frequent value of each cluster, and in the output step, fourth image data which is image data after color reduction is output.

[0113] According to this aspect, it is possible to improve color reproducibility after color reduction processing compared to the conventional art. In addition, because of the simple configuration, the saved resources can be used for other core functions.

[0114] (2) The information processing method according to (1) above, wherein the color reduction step reduces the number of colors of the image data in accordance with a set number of colors.

[0115] According to this embodiment, the number of colors after color reduction can be set arbitrarily.

[0116] (3) An information processing method according to (1) or (2) above, wherein in the color reduction step, the color-reduced information is displayed according to a set number of colors, and in the color reduction step, if the color-reduced information is changed, the color-reduced information after the change is reflected in the image data.

[0117] According to this embodiment, the color after the color subtraction can be changed arbitrarily.

[0118] (4) In the information processing method described in (3) above, in the color reduction step, first color-reduced information and second color-reduced information are generated according to the set number of colors, in the color reduction step, the second color-reduced information is displayed, and in the color reduction step, if the second color-reduced information is changed, the second color-reduced information after the change is reflected in the image data.

[0119] According to this aspect, while each cluster is maintained in the first color-reduced information, the second color-reduced information can be modified to change the color corresponding to each cluster.

[0120] (5) In the information processing method described in (3) or (4) above, the type of color after the change is also applicable to image data other than the third image data or the fourth image data.

[0121] According to this aspect, the type of color set as the color after color reduction can be applied to another image.

[0122] (6) An information processing method according to any one of (1) to (5) above, further comprising a deletion step, in which, when the fourth image data is confirmed, data relating to the fourth image data is deleted from a memory unit.

[0123] According to this aspect, information relating to image data after the color reduction process can be protected.

[0124] (7) An information processing system comprising a control unit, the control unit being configured to execute each step of the information processing method according to any one of (1) to (6) above.

[0125] According to this aspect, it is possible to improve color reproducibility after color reduction processing compared to the conventional art. In addition, because of the simple configuration, the saved resources can be used for other core functions.

[0126] (8) A program configured to cause a computer to execute each step of the information processing method according to any one of (1) to (6) above.

[0127] According to this aspect, it is possible to improve color reproducibility after color reduction processing compared to the conventional art. In addition, because of the simple configuration, the saved resources can be used for other core functions. Of course, this is not the case. [Explanation of symbols]

[0128] 1: Intermediate image data 2: Intermediate image data 3: Intermediate image data 4: Intermediate image data 5: Intermediate image data 6: Intermediate image data 7: Intermediate image data 8: Intermediate image data 9: Intermediate image data 10: Intermediate image data 100: Information processing systems 200: Information processing device 210: Control section 211: Acquisition Department 212 :Generation part 213: Adjustment section 214: Output section 215: Deleted section 216: Separation section 217: Color reduction part 218: Integration Department 220: Storage section 250: Communications Department 260: Communication bus 300: Terminal 310: Control section 320: Storage section 330: Display section 340: Input section 350: Communications Department 360: Communication bus 410: Graph Editor 411: Line graph 412: Setting area 413 :Graph area 420: Input box 430: Graph Editor 431: Line graph 432: Setting area 433: Graph area 440: Graph Editor 441: Line graph 442: Setting area 443: Graph area 510: Image data 520: Line drawing 530: Painted area 540: Painted area 550: Integrated image data 600: Palette 610: Color area 620: Color reduction method area 630: Add button 640: Type 641: Delete button 642: Color 650: Color setting box 651 :Area 652: area 653: area 700: Original image 710: Comparative example data 720: Comparative example data 730: Example data 740: area A: First image data B: Second image data

Claims

1. 1. An information processing method, comprising: The method includes an acquisition step, a color reduction step, and an output step, In the acquiring step, third image data which is colored image data is acquired, In the color reduction step, a clustering process is performed on the colors included in the third image data, the number of clusters in the clustering process corresponds to a set number of colors; The number of colors can be set arbitrarily, In the color reduction step, first color-reduced information and second color-reduced information are generated as color-reduced information indicating colors corresponding to each cluster; the color corresponding to each cluster is determined using the most frequent value of each cluster, In the color reduction step, the second color-reduced information is displayed, In the color reduction step, when the second color-reduced information is changed, the second color-reduced information after the change is reflected in the third image data to reduce the colors of the third image data; The first color-reduced information is configured to be unchangeable so as to maintain the colors corresponding to the clusters; In the output step, fourth image data which is image data after color reduction is output. Information processing methods.

2. 2. The information processing method according to claim 1, The type of color after the change is applicable to image data other than the third image data or the fourth image data. Information processing methods.

3. 2. The information processing method according to claim 1, Further, a deletion step is provided, In the deleting step, when the fourth image data is confirmed, data related to the fourth image data is deleted from a storage unit. Information processing methods.

4. An information processing system, A control unit is provided, The control unit is configured to execute each step of the information processing method according to any one of claims 1 to 3. Information processing system.

5. A program, A method for processing information according to any one of claims 1 to 3, program.

Images