Drawing data generation method, program, drawing data generation device, drawing device, and data structure

Abstract

The present invention provides a new visual effect when drawing an image. A drawing data generation method according to the present disclosure causes a computer to execute: acquiring attribute information based on an input image; generating route information about a particle route in which the positions of a particle drawn in a drawing area are set in a time series by connecting generated points in the drawing area and end points in the drawing area; generating drawing information used for drawing the particle at each time point of the time series on the basis of the attribute information; and generating drawing data in which the route information is associated with the drawing information.

Description

Drawing data generation method, program, drawing data generation device, drawing device, and data structure

[0001] The present invention relates to a drawing data generation method, a program, a drawing data generation device, a drawing device, and a data structure.

[0002] One method of displaying data is to convert the data into particles and display them on a drawing device. Patent Document 1 describes an information processing device that can display a particle three-dimensional image in which particles corresponding to a predetermined number of data items are distributed three-dimensionally with particle sizes determined according to the number of data items in order to display the data in an easy-to-understand manner.

[0003] Patent No. 6911843

[0004] As described in Patent Literature 1, in addition to displaying images using particles, there is a technique for enhancing the visual effect by expressing part of the shape of an object as fluid when displaying an image of the object on a display or the like. In this technique, for example, a fluid shape is drawn based on pixel information of the original object image. While there are techniques for drawing part of an image as fluid, there is a demand for providing newer visual effects.

[0005] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide new visual effects when drawing images.

[0006] A drawing data generation method according to one aspect of the present disclosure includes a computer acquiring attribute information based on an input image, generating path information for a particle path that connects a generation point within a drawing area with an end point within the drawing area and sets the position of a particle to be drawn in the drawing area along a time series, generating drawing information used to draw particles at each point in the time series based on the attribute information, and generating drawing data in which the path information and the drawing information are associated.

[0007] A program according to another aspect of the present disclosure causes a computer to perform the following operations: acquire attribute information based on an input image; generate path information for a particle path that connects a generation point within a drawing area with an end point within the drawing area and sets the position of a particle drawn in the drawing area along a time series; generate drawing information used to draw particles at each point in the time series based on the attribute information; and generate drawing data associated with the path information and the drawing information.

[0008] A drawing data generation device according to another aspect of the present disclosure includes an attribute information acquisition unit that acquires attribute information based on an input image, a path information generation unit that generates path information for a particle path that connects a generation point within a drawing area with an end point within the drawing area and sets the position of a particle to be drawn in the drawing area along a time series, a drawing information generation unit that generates drawing information used to draw particles at each point in the time series based on the attribute information, and a drawing data generation unit that generates drawing data in which the path information and the drawing information are associated.

[0009] A program according to another aspect of the present disclosure causes a computer to acquire drawing data generated by the drawing data generation method according to the above aspect, acquire the position of a particle on a particle path at each point in time in the time series based on the drawing data, generate drawing images in a time series in which the particle is drawn in a drawing area at each point in time in the time series based on the position and drawing information, and display the drawing images in a time series in a time series.

[0010] A drawing device according to another aspect of the present disclosure includes a drawing data acquisition unit that acquires drawing data generated by the drawing data generation method according to the above aspect, a position acquisition unit that identifies the position of a particle on a particle path at each point in time in the time series based on the drawing data, a drawing image generation unit that generates drawing images in chronological order in which the particle is drawn in a drawing area at each point in time in the time series based on the position and drawing information, and a drawing image display unit that displays the drawing images in chronological order.

[0011] A data structure according to another aspect of the present disclosure is a data structure of drawing data that includes path information of a particle path that connects a generation point within a drawing area with an end point within the drawing area and sets the position of a particle to be drawn in the drawing area along a time series, and drawing information used to draw the particle moving along the particle path at each point in the time series, wherein a computer that acquires the drawing data performs the following operations: based on the drawing data, identifying the position of the particle on the particle path at each point in the time series; based on the position and the drawing information, continuously generating drawing images along the time series in which the particle is drawn in the drawing area at each point in the time series; and displaying the drawing images continuously along the time series.

[0012] According to the present invention, it is possible to provide new visual effects when drawing images.

[0013] FIG. 1 is a schematic diagram of a drawing system according to the present embodiment. FIG. 2 is a diagram illustrating generation of drawing data according to the present embodiment. FIG. 3 is a diagram illustrating generation of drawing data according to the present embodiment. FIG. 4 is a diagram illustrating an example of a drawn image generated based on drawing data according to the present embodiment. FIG. 5 is a block diagram of a drawing data generating device according to the present embodiment. FIG. 6 is an example of attribute information according to the present embodiment. FIG. 7 is a flowchart of processing in the drawing data generating device according to the present embodiment. FIG. 8 is a diagram illustrating generation processing of drawing data according to the present embodiment. FIG. 9 is a diagram illustrating generation processing of background image according to the present embodiment. FIG. 10 is an example of drawing data according to the present embodiment. FIG. 11 is a block diagram of a drawing device according to the present embodiment. FIG. 12 is a diagram illustrating an example of the appearance of the drawing device according to the present embodiment. FIG. 13 is a flowchart of drawing processing according to the present embodiment. FIG. 14 is a diagram illustrating drawing processing according to the present embodiment. FIG. 15 is an example of a drawn image by drawing processing according to the present embodiment. FIG. 16 is an example of another drawn image by drawing processing according to the present embodiment. FIG. 17 is a flowchart illustrating drawn image display processing according to the present embodiment. FIG. 18 is a diagram illustrating updating of a drawn image according to the present embodiment. FIG. 19 is a diagram illustrating updating of a drawn image according to the present embodiment. FIG. 19 is a diagram illustrating updating of a drawn image according to the present embodiment. FIG. 19 is a diagram illustrating another example of updating of a drawn image according to the present embodiment. FIG. 19 is a diagram illustrating another example of updating of a drawn image according to the present embodiment.

[0014] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A preferred embodiment of the present invention will be described with reference to the accompanying drawings. In the drawings, components with the same reference numerals have the same or similar configurations.

[0015] 1 shows a drawing system 10 according to this embodiment. The drawing system 10 includes a drawing data generating device 101 and a drawing device 102. The drawing data generating device 101 is a device that generates drawing data to be drawn by the drawing device 102. The drawing data generated by the drawing data generating device 101 is transmitted to the drawing device 102, for example, via a network N, and the drawing data is processed in the drawing device 102 and displayed on the drawing device 102. Note that the drawing data generating device 101 and the drawing device 102 do not necessarily have to be separate devices, and may be realized as a single device.

[0016] 2 to 4, the generation of drawing data in the drawing data generating device 101 and the drawing image displayed on the drawing device 102 will be described.

[0017] 2, the drawing data generating device 101 generates attribute information based on an input image I. The drawing data generating device 101 acquires the input image I, for example, through an information processing device that can communicate with the drawing data generating device 101, or acquires the input image I by reading an information storage medium on which the input image I is stored.

[0018] The input image I has information on a plurality of objects corresponding to a plurality of regions (first region, second region), for example, region A, region B, and region C. The drawing data generating device 101 performs object recognition processing on the input image I to generate the region that each object occupies in the input image I, i.e., shape information for each object. The drawing data generating device 101 also acquires color information for pixels in the input image I that correspond to each object. In step 2A, the drawing data generating device 101 generates attribute information including shape information and color information based on the input image I.

[0019] Generation of drawing data using particles will be described with reference to Fig. 3. First, the drawing data generating device 101 generates particles P in the drawing data to which at least one of material information, texture information, and color information is added.

[0020] In step 3A, the drawing data generating device 101 generates a particle path PH along which the particle P moves. The particle path PH is a path connecting a generation point G in the drawing area DA and an end point E in the drawing area DA. The particle path PH is a path that sets the position of the particle P in time series. The particle path PH is determined based on an initial velocity vector V0 that indicates the initial velocity of the particle at the generation point G, and a vector D that connects the generation point G and the end point E. As shown in FIG. 3 , when frame F transitions from F0 to F1, the position of the particle P in frame F1 is calculated based on a vector V1 that is a normalized difference vector between vector D and initial velocity vector V0. Similarly, when frame F transitions from F1 to F2, a difference vector is calculated using a vector connecting the position of particle P in frame F1 and the end point E, and a velocity vector that is the sum of initial velocity vector V0 and vector V1. The position of particle P in frame F2 is calculated by adding a vector obtained by normalizing the difference vector to the velocity vector. Similarly, the position of particle P is calculated using the difference between the particle's velocity vector and the vector connecting the current position and the end point. FIG. 3 illustrates the positions of particle P calculated in this manner when frames F are F0, F1, F2, and F3. That is, particle P moves along particle path PH from generation point G toward end point E over time. In this way, the drawing data generating device 101 generates or acquires the particle's generation point G, end point E, and initial velocity vector V at generation point G to generate particle path PH.

[0021] In step 3B, the drawing data generating device 101 generates drawing information used to draw particles P at each position on the particle path PH based on the attribute information. For example, the drawing data generating device 101 identifies the position of particles P on the particle path PH in each frame. Based on the attribute information, the drawing data generating device 101 identifies an area indicated by shape information in which each particle P in the frame is included. The drawing data generating device 101 associates color information associated with the identified area as drawing information with particles P in each frame. Here, the color information includes, for example, information on color parameters (r, g, b) of pixels indicating particles in the drawing device and information on a parameter (α) indicating the transparency of the particle. The drawing data generating device 101 also associates texture as drawing information with particles P in each frame.

[0022] In the example of FIG. 3 , a texture TX1 is associated with a particle P in frame F1, a texture TX2 is associated with a particle P in frame F2, and a texture TX3 is associated with a particle P in frame F3. A common texture may be associated with the particle P in each frame. For example, the drawing data generation device 101 may associate texture TX3 with the particle P as a common texture in each frame. The drawing data generation device 101 may also associate color information with the particle P in each frame. For example, color information that sets a common color in each frame based on color information corresponding to the end point E1 of the particle path PH may be associated with the particle P. In this case, color information based on the color C of the region C is associated with the particle in each frame. In this case, the information of the pixel color parameters (r, g, b) may be common, but the transparency (α) may vary.

[0023] The texture can have any shape, and can be any texture that has an expression such as smoke, water, or fire. Figure 3 shows an example in which a smoke texture is associated. By combining the texture with color information, the shape can be changed by the texture while the color of the texture remains the same.

[0024] 3 is repeatedly performed to assign particle paths and textures, thereby generating drawing data based on the input image I. The drawing data is information in which information on particles P, information on particle paths PH, and at least one of material information, texture information, and color information are associated with each other.

[0025] A plurality of drawing images are generated by the drawing device 102 based on the drawing data generated by the drawing data generating device 101. The drawing device 102 can continuously display the drawing images as a moving image. Fig. 4 shows an example of a drawing image DI displayed by the drawing device 102. In the drawing image DI, particles on a plurality of particle paths (not shown) are simultaneously drawn, thereby displaying an image based on the input image I.

[0026] At this time, particles P are drawn as if springing out from generation point G located at the bottom of drawn image DI. In the example of Fig. 4, a smoke texture is applied to each particle, so people and objects shown in input image I are expressed as if they were smoke rising from a candle. This provides a new visual effect when drawing an image.

[0027] The configuration of the drawing data generating device 101 will be described with reference to Fig. 5. Fig. 5 shows a block diagram of the drawing data generating device 101 according to this embodiment. The drawing data generating device 101 includes a communication unit 1011, a storage unit 1012, an image acquisition unit 1013, an attribute information acquisition unit 1014, a path information generation unit 1015, a drawing information generation unit 1016, and a drawing data generation unit 1017. Each unit of the drawing data generating device 101 can be realized by a processor executing a program stored in a storage device in an information processing device such as a personal computer.

[0028] The communication unit 1011 controls communication between the drawing data generating device 101 and an external information processing device including the drawing device 102. The communication unit 1011 is capable of communicating with, for example, a terminal (such as a personal computer, a smartphone, or a tablet) operated by a user.

[0029] The storage unit 1012 stores various types of information used in processing by the drawing data generating device 101. The storage unit 1012 has a material DB 10121, a texture DB 10122, and a color DB 10123. The material DB 10121 stores information on the material of an object (plastic, metal, cloth, person, space, etc.) that the drawing data generating device 101 assigns to particles as drawing information when generating drawing data. The texture DB 10122 stores information on textures (shapes of smoke, fire, water, etc.) that the drawing data generating device 101 assigns to particles as drawing information when generating drawing data. The color DB 10123 stores information on colors (RGB settings, transparency settings, etc.).

[0030] The image acquisition unit 1013 acquires an input image from a terminal operated by a user. The image acquisition unit 1013 may refer to an input image stored in advance in the storage unit 1012.

[0031] The attribute information acquisition unit 1014 generates attribute information based on the input image. In addition to generating attribute information internally within the drawing data generation device 101, the attribute information acquisition unit 1014 may acquire the input image and attribute information from external information processing. The attribute information generated based on the input image may be updated by a user. Alternatively, the user may input attribute information based on the input image to the drawing data generation device 101. The attribute information acquisition unit 1014 may also generate attribute information based on drawing settings, which are information added to the input image. Here, the drawing settings are information added to the input image, such as a filter added to the input image to change the appearance of the input image. Alternatively, the drawing settings are information indicating an object to be additionally displayed when drawing data is drawn. The attribute information acquisition unit 1014 acquires the drawing settings from a user, for example.

[0032] FIG. 6 shows an example of attribute information. The attribute information is information including, for example, shape information, color information, object information, object type information, and material information. The shape information is information indicating the shape of each object in the input image. The attribute information acquisition unit 1014 performs object recognition processing on the input image I to generate shape information. The shape information is, for example, coordinate information for setting the range of an object in the drawing area. The color information is information about the color of pixels belonging to each area in the input image. The color information for a certain area is not limited to one color, and multiple colors may be set. The object information is information indicating the name of each object. The object information is generated by the object recognition processing by the attribute information acquisition unit 1014. The object type information is information indicating the category of each object. The object type information is generated by the object recognition processing by the attribute information acquisition unit 1014. The material information is information indicating the material of each object.

[0033] In the example of FIG. 6 , for an area where "area A" is stored as shape information, the color is "color A" and the object is stored as "M1." M1, for example, corresponds to area A in FIG. 2 and is a computer. "Device" is stored as object type information and "Material A" is stored as material information. Material A is, for example, plastic. Similarly, for an area where "area B" is stored as shape information, the color is "color B" and the object is stored as "M2." M2, for example, corresponds to area B in FIG. 2 and is clothing. "Clothing" is stored as object type information and "Material B" is stored as material information. Material B is, for example, cloth. Similarly, for an area where "area C" is stored as shape information, the color is "color C" and the object is stored as "M3." M3, for example, corresponds to area C in FIG. 2 and is a human face. "Person" is stored as object type information and "Material C" is stored as material information. Material C is, for example, a human body.

[0034] The path information generation unit 1015 generates path information for particle paths that connect generation points within the drawing area with end points within the drawing area and set particle positions in time series. The path information generation unit 1015 may set end points based on attribute information. For example, the path information generation unit 1015 may generate path information such that end points are located on the boundaries between each object-recognized area and the background area. Here, the coordinates for setting the generation points, particle paths, and end points may directly use coordinates of the input image, or may use coordinates extracted and thinned from the input image (referred to as drawing coordinates). This allows the data size of the final drawing data to be reduced, even if the input image has a large number of pixels. The number of particle paths, i.e., the number of particles, is configurable. For example, the number of particles may be set by the path information generation unit 1015 according to the number of coordinates for drawing, or may be set by user input.

[0035] The drawing information generating unit 1016 generates drawing information to be used for drawing particles at each time point in the time series based on the attribute information. The generation of drawing information will be described later.

[0036] The drawing information generation unit 1016 generates drawing information including material information and texture information. The drawing information generation unit 1016 may also generate drawing information including color information in which color information at the end point of a particle path is set based on the color of a pixel in the input image corresponding to the end point. The drawing information generation unit 1016 may also generate a background image based on the input image and generate drawing information further including the background image. The background image may also be a group of images that change over time and are associated with each frame of drawing data. The background image may be, for example, an image that combines an image for setting the color of the background region and a silhouette image that shows the outline of an object to be drawn in the foreground. The generation of a background image will be described later. The color of the background region (described below) and the silhouette image may also be set based on user input without being generated based on the input image.

[0037] The drawing data generating unit 1017 generates drawing data in which particle information, path information, and drawing information are associated with each other. The drawing data is a set of a series of images that can be displayed consecutively.

[0038] The drawing data generation process will be described with reference to FIG.

[0039] In step S701, the image acquisition unit 1013 acquires an input image. In step S702, the attribute information acquisition unit 1014 performs object recognition processing to recognize objects included in the input image. In step S703, the attribute information acquisition unit 1014 generates attribute information for each recognized object. Note that the processing from step S701 to step S703 may be performed by an information processing device different from the drawing data generation device 101, and the drawing data generation device 101 may acquire the attribute information from the other information processing device or through input from the user.

[0040] In step S704, the path information generating unit 1015 generates particle information. For example, the path information generating unit 1015 generates a plurality of pieces of particle information including information on particle P1 and information on particle P2 as shown in FIG.

[0041] In step S705, the path information generation unit 1015 generates path information of particle paths for the generated particles. For example, as shown in step 8A of Fig. 8, the path information generation unit 1015 generates a particle path PH1 for particle P1 that travels from the generation point G to the end point E1. Also, the path information generation unit 1015 generates a particle path PH2 for particle P2 that travels from the generation point G to the end point E2.

[0042] Particle path PH1 sets the position of particle P1 in each of frames F (=F1, F2, F3). Particle path PH2 sets the position of particle P2 in each of frames F (=F1, F2). Particle P2 has disappeared in frame F3. In this way, the generation and disappearance of each particle can be set in any frame.

[0043] In step S706, the drawing information generation unit 1016 generates material information to be drawn by each particle based on the attribute information. In step S707, the drawing information generation unit 1016 generates texture information for each particle based on the attribute information. In step S708, the drawing information generation unit 1016 generates color information for each particle based on the attribute information.

[0044] In step S709, the drawing information generating unit 1016 generates drawing information including material information, texture information, and color information based on the attribute information. Note that the material information does not necessarily need to be included in the drawing information, and only the texture information and color information may be included in the drawing information.

[0045] For example, as shown in step 8B of FIG. 8, the drawing information generator 1016 associates material information, texture information, and color information with particles on each particle path in each frame.

[0046] In step S710, the drawing data generation unit 1017 generates drawing data including particle information, path information, and drawing information. Here, for example, as shown in step 8C of FIG. 8 , the drawing data generation unit 1017 may generate drawing data including a background image BG in addition to the particle information, path information, and drawing information. By adding the background image BG, the boundary between the drawing object and the background becomes clearer, improving the visibility of the drawn image. If the background image is a group of images that change over time, the drawing data is generated so that multiple types of background images are associated with each frame.

[0047] The generation of a background image BG based on an input image I will be described with reference to FIG. 9 . The drawing information generation unit 1016 performs processing to recognize objects in the input image I and distinguish between an object region and a background region in the input image I. The drawing information generation unit 1016 generates a background color image CI indicating the color of the background region based on the input image I (9A). The drawing information generation unit 1016 generates a silhouette image SI including a silhouette S indicating the outline of the object region based on the input image I (9B). The silhouette images may be generated as a silhouette image group including multiple silhouettes. The drawing information generation unit 1016 may also generate a silhouette image group including multiple silhouettes of different shapes, such as a silhouette image SI0 including a silhouette S0 and a silhouette image SI1 including a silhouette S1, based on the input image I (9C). The drawing information generation unit 1016 combines the background color image CI with the silhouette image SI or the silhouette image group to generate a background image BG (9D). When combining a background color image CI with a group of silhouette images, the background image BG is a group of multiple types of background images including different silhouettes, and drawing data is generated so that each background image is associated with each frame.

[0048] The silhouette in the background image BG may be set to a darker color than the color set in the background color image. For example, the silhouette color may be set to a color with a lower V (brightness) in the HSV parameters than the background color. Furthermore, in addition to color, the silhouette image may also set transparency as a parameter in the silhouette area. By making the transparency configurable, an image that blends more seamlessly with the background can be generated, enabling the representation of a shadow. In this way, by cutting out the outline of the object rendered by the particles and rendering a dark silhouette behind the particles (in the layer between the background color image and the particles), the particle color is not visually overpowered by the background color, even if the background color is light, thereby improving visibility. Therefore, any color can be set as the background color, increasing the degree of freedom in color setting.

[0049] The silhouettes generated in the silhouette image group may be generated so that they have contours that are modified from the contours of objects in the original input image. This makes it possible to express a fluctuating silhouette, and when the drawing data is drawn, it becomes possible to express a situation in which a shadow is moving. Furthermore, for example, if the texture is smoke, the silhouette may move in a manner that is consistent with the attributes of smoke, such as smoke rising from below and disappearing, making it easier to grasp that the image is a representation of smoke when drawn.

[0050] FIG. 10 shows an example of drawing data. The drawing data includes particle information, path information, drawing information, and disappearance settings. The particle information includes the particle name, particle generation frequency, and particle size. For example, a particle with the particle name "P1" has a generation frequency set to "G01" and a size set to "S01." The generation frequency here is information that sets how many frames after the particle is generated that the particle is displayed again from the generation point of the particle path and begins moving toward the end point. The size is the size of the particle displayed in the drawing area. The smaller the size, the more detailed the drawing. The generation frequency and size can be set and changed by the user operating the drawing data generating device 101.

[0051] The path information includes the following fields: frame, position, initial velocity vector, and initial velocity setting. The frame is information indicating a time axis for setting the position of a particle along a time series. The position is set as coordinates in the drawing area. For example, for particle P1, the particle positions at the generation point PO01 and the end point PO04 are set. The initial velocity vector is set as a vector indicating the initial velocity at the generation point PO01. The particle position in the intermediate frame is calculated based on the generation point PO01, the end point PO04, and the initial velocity vector V01. The particle position in the intermediate frame may be calculated by the path information generation unit 1015 and included in the drawing data. Alternatively, the particle position in the intermediate frame may not be included in the drawing data, but may be calculated during drawing by the drawing device 102 (described later). The initial velocity setting is information indicating that the initial velocity vector when the particle is drawn is randomly set by the drawing device. When at least one of the direction and magnitude of the initial velocity vector is set randomly and drawn, the initial velocity vector may be set randomly by the drawing device based on the initial velocity vector initially set in the path information. When the position of the particle in the intermediate frame is calculated during drawing by the drawing device 102, the drawing device 102 can calculate the position of the particle based on the initial velocity vector that is set randomly each time the particle is drawn, if the particle is repeatedly drawn.

[0052] The drawing information includes material information and texture information added to particles in each frame. For example, particle P1 in frame F0 is associated with "M0" corresponding to space as a material, TX0 corresponding to "smoke" as a texture, and C0 corresponding to "color 4," the color of end point P004, as a color. Similarly, particle P1 in frame F1 is associated with "M1" corresponding to plastic as a material, TX1 corresponding to "smoke" as a texture, and C1 corresponding to "color 4" as a color. Particle P1 in frame F2 is associated with "M2" corresponding to cloth as a material, TX3 corresponding to "smoke" as a texture, and C2 corresponding to "color 4" as a color. Particle P1 in frame F3 is associated with "M3" corresponding to a person as a material, TX3 corresponding to "smoke" as a texture, and C3 corresponding to "color 4" as a color. Similarly, material information, texture information, and color information are associated with particle P2 in each frame.

[0053] Here, the color information of each particle is set based on the color at the end point. For example, the color of each particle may be set to a color that is the same as the color at the end point (color 4). Furthermore, the color of each particle may be set by an animation curve that defines the color change from a certain color to color 4 as the particle progresses along the path. Similarly, the texture, particle size and orientation, and particle transparency may be set by animation curves. Furthermore, the color information may set the transparency of the particle at the time indicated by each frame. If the particle has transparency, the colors of multiple particles will be superimposed and drawn in one pixel.

[0054] Furthermore, an extinction setting is associated with the drawing data. The extinction setting is information indicating a timing for changing the timing at which the drawing of a particle is terminated when the drawing data is rendered by the rendering device. For example, particle P1 is rendered from frame F=F0 to frame F=F3. Assume that this rendering continues for five seconds. Here, if the extinction setting is set to, for example, "random," the rendering device 102 terminates the drawing of particle P1 at a random time between 0 and 5 seconds (for example, three seconds after the start of drawing). When particle P1 is rendered again, the drawing of particle P1 is terminated at another random time. In this way, by randomly setting the time at which the drawing of particle P1 is terminated within a single rendering time, the rendering state can be changed, improving the visual effect. Note that the extinction setting may not be set, as with particle P2. Note that the drawing end time may be randomly set to a value longer than the time it takes for the particle to reach its end point. In this case, the particle that has reached the end point continues to be drawn so that it remains at the end point, and the drawing can be completed at a randomly set drawing end point.

[0055] The drawing data generated by the drawing data generating device 101 is information that can generate a drawing image DI as shown in Figure 4 by the drawing device 102 drawing particles on the particle path in each frame based on the drawing information.

[0056] The drawing device 102 will be described with reference to Fig. 11. The drawing device 102 includes a communication unit 1021, a storage unit 1022, a display device 1023, a drawing data acquisition unit 1024, a particle position acquisition unit 1025, a drawing image generation unit 1026, a drawing image display processing unit 1027, an environmental information acquisition unit 1028, an updated image generation unit 1029, an environmental information acquisition device 10210, and an audio output device 10211.

[0057] The communication unit 1021 controls communication between the drawing device 102 and an external information processing device including the drawing data generating device 101. The communication unit 1021 is also capable of communicating with, for example, a terminal (such as a personal computer, a smartphone, or a tablet) operated by a user.

[0058] The storage unit 1022 stores various types of information used in processing by the drawing device 102. The storage unit 1022 has a material DB 10221, a texture DB 10222, and a color DB 10223. The material DB 10221 stores information on the material of an object (plastic, metal, cloth, person, space, etc.) associated with drawing information read from drawing data. The texture DB 10222 stores information on textures (shapes of smoke, fire, water, etc.) associated with drawing information read from drawing data. The color DB 10223 stores information on colors (RGB settings, transparency settings, etc.).

[0059] The display device 1023 is a device for displaying a drawn image, and may be any display device capable of displaying an image, such as an LED display, an organic EL display, or a projector device.

[0060] The drawing data acquisition unit 1024 acquires drawing data from the drawing data generation device 101. The drawing data acquisition unit 1024 stores the acquired drawing data in the storage unit 1022. Note that the drawing data acquisition unit 1024 does not need to acquire drawing data directly from the drawing data generation device 101, and may acquire drawing data from another information processing device.

[0061] The particle position acquisition unit 1025 acquires the positions of particles on particle paths at each time point in the time series based on the drawing data. At this time, the particle position acquisition unit 1025 acquires the positions of multiple particles on multiple particle paths in a certain frame. Furthermore, the particle position acquisition unit 1025 can calculate the position of each particle according to the initial velocity vector. If the initial velocity vector is set randomly each time a particle is drawn, the particle position acquisition unit 1025 can calculate a random particle position each time the particle is drawn.

[0062] The drawn image generation unit 1026 generates drawn images in chronological order, in which particles are drawn in a drawing area at each time point in the time series, based on the particle positions and drawing information. The drawn image generation unit 1026 draws each particle based on the drawing information associated with each particle in a certain frame. When a background image is associated with the drawing data, the drawn image generation unit 1026 generates a drawn image that includes the background image, with the particles as the foreground. When the background image is a group of images that change along the time series and are associated with each frame, the drawn image generation unit 1026 generates a drawn image using the background image in each frame.

[0063] The drawn image display processing unit 1027 displays the drawn images generated successively by the drawn image generating unit 1026 in chronological order. The drawn image display processing unit 1027, for example, displays the drawn images successively on the display device 1023. For example, the drawn image display processing unit 1027 displays the drawn images as a moving image, or continues to generate drawn images infinitely and displays the drawn images successively.

[0064] The environmental information acquisition unit 1028 acquires environmental information around the drawing device 102 through an environmental information acquisition device 10210 (described later).

[0065] The updated image generating unit 1029 updates at least one of the particle paths and the drawing information after the environmental information is acquired based on the environmental information, and generates a plurality of updated images. The process of generating updated images will be described later.

[0066] As an example, the environmental information acquisition device 10210 includes a sound acquisition device 10210a and an imaging device 10210b. The sound acquisition device 10210a is a device for acquiring environmental information about the environment surrounding the drawing device 102, and includes a sound collection device such as a microphone and a sound generation device such as a speaker. The sound acquisition device 10210a acquires sound information as environmental information. The sound acquisition device 10210a may also function as a music playback device and may acquire sound information emitted by the sound acquisition device 10210a itself as environmental information. The environmental information acquisition device 10210 may be any device capable of acquiring information about the surrounding environment, and may be a device capable of acquiring information other than sound and light. For example, the environmental information acquisition device 10210 may be a vibration sensor or an acceleration sensor. In this case, the drawing device 102 may generate an effect such as shaking the drawn image in response to the shaking of the environment detected by the environmental information acquisition device 10210.

[0067] The imaging device 10210b is a device for acquiring information about the environment surrounding the drawing device 102. The imaging device 10210b is, for example, a camera having an image sensor. The imaging device 10210b acquires light information (optical information) as environmental information.

[0068] The sound output device is a sound generating device such as a speaker for emitting sound around the drawing device 102 .

[0069] 12 shows a schematic diagram of the rendering device 102. The rendering device 102 has sound acquisition devices 10210a1 and 10210a2 on the left and right. The rendering device 102 can acquire the position of a sound source, the volume, and the frequency of the sound (the pitch of the sound) through the sound acquisition devices 10210a1 and 10210a2. The rendering device 102 also has an image capture device 10210b in front of the display device 1023. Note that the number and positions of the sound acquisition devices can be changed as desired.

[0070] The processing of the drawing data will be described with reference to Fig. 13. In step S1301, the drawing data acquisition unit 1024 acquires the drawing data.

[0071] In step S1302, the particle position acquisition unit 1025 acquires the position of each particle in a certain frame. When the particle position acquisition unit 1025 acquires the position of a particle for the first time, the particle position acquisition unit 1025 acquires the position of each particle in the first frame. The particle position acquisition unit 1025 may calculate and acquire the position of each particle in a certain frame based on the initial velocity vector.

[0072] In step S1303, the drawn image generating unit 1026 refers to the drawing information associated with each particle whose position has been acquired from the drawing data.

[0073] In step S1304, the drawn image generating unit 1026 generates a drawn image in which each particle whose position has been acquired is drawn.

[0074] In step S1305, the drawn image display processing unit 1027 performs a drawn image display process for drawing the generated drawn image.

[0075] In step S1306, the drawn image generation unit 1026 determines whether or not to end drawing of the drawn image. The drawn image generation unit 1026 determines to end drawing, for example, when an operation signal to end drawing is input to the drawing device 102. Alternatively, the drawn image generation unit 1026 determines to end drawing when drawing is set to continue and end after a predetermined time has elapsed. For example, when drawing is set to be performed continuously and the drawn image generation unit 1026 determines not to end drawing, the drawing device 102 performs the processes from step S1302 onwards for the incremented frame.

[0076] A schematic diagram of the processing up to this point is shown in FIG. 14. The drawn image generation unit 1026 generates a drawn image DI0 corresponding to a certain frame (frame F0). Next, the drawn image display processing unit 1027 displays the drawn image DI0 on the display device 1023. Thereafter, drawn images DI1, DI2, and DI3 corresponding to each frame (frames F1, F2, and F3) are generated and drawn. Here, the drawn image generation unit 1026 may draw particles so as to disappear in accordance with the disappearance setting associated with each particle.

[0077] Here, for each particle, a frame in which the particle is at its generation point and a frame in which the particle is at its end point are set. For example, as shown in FIG. 10 , particle P1 is set to be located at its generation point in frame F0 and at its end point in frame F3. Assume that the rendering device 102 performs the process shown in FIG. 13 and completes rendering from frame F0 to frame F3. If it is determined in step S1306 that rendering is to continue, the rendering device 102 renders particle P1 in the next frame based on the path information and rendering information set in frame F0. Thereafter, rendering is performed up to step S1305, the frame is incremented, and the process from step S1302 onward is repeated. At this time, the particle position acquisition unit 1025 calculates and acquires the position of particle P1 in each frame based on an initial velocity vector randomly set for particle P1. Therefore, the position of particle P1 stored for the initial frame F1 may differ from the position of particle P1 in a frame corresponding to the frame F1 that is subsequently recalculated. In this way, the drawing device 102 can repeatedly and continuously draw the particle P1 from frame F0 to frame F3 based on the drawing data in which information related to drawing the particle P1 is set.

[0078] Similarly, continuous particle drawing is possible for other particles. For example, as shown in FIG. 10 , particle P2 is set to be located at the generation point in frame F0 and at the end point in frame F2. Assume that the drawing of frame F2 is completed. In the next frame, the drawing device 102 draws particle P2 based on the path information and drawing information set in frame F0. Thereafter, drawing is performed up to step S1305, the frame is incremented, and the processing from step S1302 onward is repeated. At this time, the particle position acquisition unit 1025 calculates and acquires the position of particle P2 in each frame based on the initial velocity vector randomly set for particle P2. The drawing device 102 can repeatedly and continuously draw particle P2 from frame F0 to frame F2 based on drawing data in which information related to the drawing of particle P2 is set. Here, the repeated drawing of particle P1 and the repeated drawing of particle P2 can be performed separately. That is, when particle P1 is drawn up to frame F3 and repeatedly drawn, particle P2 has been drawn up to frame F2 and may already be drawn repeatedly.

[0079] In this way, when repeatedly drawing each particle, the drawing device 102 can update and draw the position of the particle associated with each frame having a predetermined range for each series of drawing. In other words, the drawing device 102 can perform random drawing using the drawing data generated by the drawing data generating device 101 as an initial value. This makes it possible to generate drawn images continuously and randomly. Note that when particles are repeatedly drawn, the positions do not necessarily have to be random, and the particles may follow a common position each time they are drawn.

[0080] The drawn image generation unit 1026 may also generate drawn images corresponding to a predetermined number of frames and generate video data including the generated drawn images. In this case, the drawn image display processing unit 1027 displays the video data on the display device 1023. The drawing device 102 may store or transmit the generated drawn images or video data so that they can be displayed on another device (e.g., a smartphone, a tablet terminal, etc.).

[0081] FIG. 15 shows a more specific example of a drawn image. FIG. 15 is an example of an image of a person drawn using a smoke texture. The smoke is drawn so as to rise from a generation point G to a predetermined end point. The drawn image DI includes an additional object AM. Here, information about the additional object AM may be generated by the drawing data generation unit 1017 based on the input image. If the attribute information includes, for example, a drawing setting that the additional object AM is to be drawn at the time of drawing, the drawn image generation unit 1026 generates the drawn image DI including the additional object AM.

[0082] The rendering aspect changes depending on the attribute data. FIG. 16 shows an example of a rendered image generated based on rendering data different from the rendering data described in FIG. 14 . In the example of FIG. 16 , the rendered image DI is information in which texture information with a shape different from that of the example of FIG. 14 is added to particles on multiple particle paths, including particle paths PH3 and PH4 from generation point G2 located at the top of the rendering area DA. The rendered image of FIG. 16 is a rendered image DI that expresses an input image with water pouring down from above. In this way, while the input image is the same, various expressions are possible by changing the rendering data. Furthermore, the position of the generation point can be any position within the rendering area DA, and the number of generation points is not limited to one, but may be multiple. This enables a greater variety of expressions.

[0083] Generation of an updated image in the drawn image display process will be described with reference to Fig. 17. The updated image is generated while the drawn image is being displayed on the display device 1023 as a moving image.

[0084] In step S1701, the environmental information acquisition unit 1028 acquires environmental information. For example, the environmental information acquisition unit 1028 acquires information about sounds around the drawing device 102 from the sound acquisition device 10210a. Here, an example will be described in which sound from a sound source SD1 is acquired via the sound acquisition devices 10210a and 10210b as shown in FIG.

[0085] In step S1702, the updated image generation unit 1029 identifies particle paths or drawing information to be updated based on the environmental information. For example, the updated image generation unit 1029 identifies particle paths or drawing information associated with frames after the point in time when the environmental information is acquired.

[0086] In step S1703, the updated image generating unit 1029 updates the particle paths or drawing information associated with frames subsequent to the point in time when the environmental information was acquired.

[0087] The updating of particle paths or drawing information will be described with reference to Fig. 19. Fig. 19 shows an example of updating particle paths or drawing information when sound information is acquired from sound source SD1 at a timing after frame F2.

[0088] Based on the environmental information from the sound acquisition devices 10210a and 10210b, the updated image generation unit 1029 calculates the position of the sound source SD1 relative to the drawing device 102. Here, it is assumed that it is calculated that the sound source SD1 is located at a height H1 from the bottom surface.

[0089] The updated image generation unit 1029 identifies an intersection CP1 on the particle path PH1 at height H1 and an intersection CP5 on the particle path PH5 at height H1. The updated image generation unit 1029 updates the particle path or drawing information associated with frames subsequent to the frame in which particles are located at intersections CP1 and CP5. FIG. 19 shows, as an example, particle paths PH1a and PH5a in which the end positions of the particle paths in frame F3 have been updated. The particle path is updated based on information obtained by converting environmental information into external forces in a physics engine that moves particles according to a texture, for example. In addition to updating the position of the particle path, the updated image generation unit 1029 may also change the texture information and material information associated with the particle according to the environmental information.

[0090] In step S1704, the updated image generating unit 1029 generates an updated image based on the updated particle paths or drawing information. The image displayed on the display device 1023 is updated with the updated image.

[0091] In step S1705, the drawn image display processing unit 1027 causes the display device 1023 to display the drawn image using the updated image.

[0092] In step S1706, the drawn image display processing unit 1027 determines whether to display a drawn image using the original particle path and drawing information. If the drawn image display processing unit 1027 determines that updating of the particle path or drawing information based on the environmental information has not been completed, that is, that the reflection of the influence based on the environmental information has not been completed, a negative determination is made in step S1706. Thereafter, the process returns to step S1702, and drawing based on the updated particle path or drawing information is repeated. Here, the drawn image display processing unit 1027 may reflect the influence of the environmental information based on the environmental information acquired in step S1701 and calculate in advance the number of frames to be updated. At this time, if display of the drawn screen using the updated image covering the frames to be updated has not been completed, the drawn image display processing unit 1027 makes a negative determination in step S1706.

[0093] For example, if the generation of a series of updated images is completed, the drawn image display processing unit 1027 makes a positive determination in step S1706. Thereafter, in step S1707, the drawn image display processing unit 1027 displays the drawn image on the display device 1023 based on the initial particle path and drawing information. As a result, the drawing is restored to the initial drawing state after the image is updated by the sound source SD1. This provides a visual effect in which the drawn image displayed on the display device 1023 can be interactively changed in response to user input.

[0094] FIG. 20 schematically illustrates an example of a drawn image after generating an updated image and the subsequent restoration of the drawn image based on the initial data. A portion of the drawn image changes in response to sound information from the sound source SD1, and after subsequent video playback, a drawn image based on the initial data is generated. At this time, the updated image generating unit 1029 may generate the updated image UI so that the drawn image includes, as drawing information, an updated background image in which the background image BG has been updated to include a silhouette S2 whose outline has been updated based on environmental information. For example, in FIG. 20 , the updated image UI is generated based on environmental information (sound information) so that the outline of the silhouette changes as the position of the particle changes. This allows for improved visibility by providing a silhouette, even when the shape of the particle's drawing target changes.

[0095] 21 shows a more specific example of changes in the drawn image. First, drawn image DIa is displayed in frame Fa. Then, in frame Fb, updated images U1b and UIc generated in response to sound information are displayed. When drawn images are continuously generated, if it is determined that the original drawing data should be used for drawing, drawn image DIa is displayed again. By repeatedly displaying drawn images in this manner, a visual effect is achieved in which the video displayed on display device 1023 can be interactively changed in response to changes in the environment, and the change can be repeated.

[0096] Another example of generating an updated image according to environmental information will be described with reference to Figures 22 and 23. Figure 22 shows an example of generating an updated image based on information from two sound sources SD1 and SD2. Here, it is assumed that sound source SD2 emits a lower-pitched sound than sound source SD1. It is also assumed that the volume of sound source SD1 is lower than in the example of Figure 20.

[0097] The updated image generation unit 1029 generates an updated image UI by, for example, updating the particle path corresponding to the height H2 of the sound source SD2 and updating the particle path corresponding to the height H1 of the sound source SD1. In the updated image UI, the particle path is updated so that the lower part of the rendering target is compressed in accordance with the low-frequency sound from the sound source SD2. Furthermore, in the updated image UI, the particle path near the height H1 is updated in accordance with the sound from the sound source SD1. At this time, the updated image generation unit 1029 can change the degree of change in the particle path in accordance with the volume of the sound from the sound source SD1. Alternatively, the updated image generation unit 1029 may change the particle path in accordance with the frequency of the sound from the sound source SD1. Note that the updated image generation unit 1029 may generate an updated image by updating the particle path or rendering information in accordance with the sound source of the sound emitted by the audio output device 10211 by the rendering device 102 itself.

[0098] FIG. 23 illustrates an example of generating an updated image based on light information from the light source LS. The light information is acquired by the environmental information acquisition unit 1028 via the imaging device 10210b. The updated image generation unit 1029 updates the drawing information assigned to each particle according to the light information (e.g., light frequency and intensity) to generate an updated image. Here, as shown as an example, the updated image generation unit 1029 may generate the updated image UI such that the background image, which has been updated based on the environmental information, includes as drawing information an updated background image. For example, in FIG. 23 , the updated image generation unit 1029 generates an updated background image with a darker hue than the background image in the drawing image DI based on the environmental information, thereby generating the updated image UI. By updating the image of the main drawing object and the image of the background based on the environmental information, for example, an updated background image is generated in accordance with the update of the foreground image based on the environmental information, without blurring the outline. This clarifies the boundary between the drawing object and the background, improves the visibility of the drawing image, and makes it easier for the user to see the outline of the drawing object. The updated background image may be generated based on various environmental information such as the above-mentioned sound information, in addition to the light information.

[0099] Here, the imaging device 10210b may be an imaging device for image recognition. For example, the updated image generation unit 1029 may generate an updated image in accordance with a recognized object (e.g., a human face or hand). Alternatively, the imaging device 10210b may be a sensor capable of infrared sensing, and may generate an updated image in accordance with a detected object (e.g., a person passing in front of the drawing device).

[0100] The examples described with reference to FIGS. 22 and 23 also bring about the visual effect of being able to interactively change the moving image displayed on the display device 1023.

[0101] The drawing data generating device 101 according to this embodiment includes an attribute information acquiring unit 1014 that acquires attribute information based on an input image, a path information generating unit 1015 that generates path information for particle paths that connect generation points within a drawing area with end points within the drawing area and that set positions of particles to be drawn in the drawing area along a time series, a drawing information generating unit 1016 that generates drawing information used to draw particles at each point in time based on the attribute information, and a drawing data generating unit 1017 that generates drawing data in which the path information and the drawing information are associated with each other.

[0102] By generating drawing information for particles moving along particle paths based on attribute information, drawing data that can be used for drawing in various ways is generated, which makes it possible to provide new visual effects when drawing images.

[0103] In addition, in the drawing data generating device 101, the drawing information generating unit 1016 may generate drawing information including material information indicating the substance to be drawn by the particles.

[0104] This makes it possible to generate rendering data that can express the reflection of materials, etc., and increases the realism of rendering.

[0105] In the drawing data generating device 101, the drawing information generating unit 1016 may generate drawing information including texture information indicating the texture of the particles.

[0106] By setting texture information, for example, smoke texture, fire texture, or water texture can be set for particles, which can provide new visual effects when drawing images.

[0107] In the drawing data generating device 101, the drawing information generating unit 1016 may generate drawing information including color information indicating the color of a particle. Also, in the drawing data generating device 101, the drawing information generating unit 1016 may generate drawing information including color information in which the color of a particle at an end point of a particle path is set based on the color of a pixel in the input image corresponding to the end point.

[0108] Setting color information enables a variety of renderings according to the input image, providing new visual effects when rendering images. Furthermore, when an image based on the rendering data is rendered, the color of the particles at the end of the path can be made to match the color of the input image. Therefore, by making the color at the end point (outline position) the color of the input image, the rendered image becomes closer to the input image, improving the visibility of the rendered image.

[0109] In the drawing data generating device 101, the route information generating unit 1015 may set the end point based on the attribute information.

[0110] The attribute information may include first attribute information for region A in the input image I, and the drawing information generation unit 1016 may generate drawing information including the first drawing information based on the first attribute information. The attribute information may include attribute information for region A in the input image I and second attribute information for region B in the input image I that is different from region A, and the drawing information generation unit 1016 may generate drawing information including the first drawing information based on the first attribute information and the second drawing information based on the second attribute information.

[0111] This makes it possible to generate drawing information for each of a plurality of regions of the input image, and therefore, for example, to generate drawing information according to an object in the input image.

[0112] Furthermore, the drawing information generating unit 1016 may generate drawing information including a background image having information on the background color and information on the silhouette of the target to be drawn by the particles, based on the input image.

[0113] As a result, when drawing based on drawing data, the drawing by particles may be difficult to see depending on the color of the input image, for example, but by providing a background image that contains color information and silhouette information, the object being drawn by the particles becomes easier to see.

[0114] The drawing information generating unit 1016 may generate drawing information including a plurality of background images each having a plurality of silhouettes with different contours.

[0115] This generates a silhouette that has a contour that is a modified version of the contour of the object in the original input image, making it possible to express a fluctuating silhouette, and thus a moving shadow.

[0116] The drawing information generating unit 1016 may generate attribute information based on an input image and drawing settings that are information added to the input image. The drawing information generating unit 1016 may also acquire drawing settings from a user and generate attribute information based on the acquired drawing settings.

[0117] This makes it possible to generate and render different rendering data by varying the rendering settings even for a common input image, thereby producing a variety of visual effects.

[0118] The drawing information generating unit 1016 may also acquire information for setting the drawing information from the user, and generate the drawing information based on the acquired information for setting the drawing information.

[0119] This makes it possible to generate drawing information that takes into account input from the user, and to produce a variety of visual effects.

[0120] Furthermore, the drawing data generation unit 1017 may generate drawing data that is further associated with information indicating a change in the timing for ending drawing of particles when the drawing data is drawn by the drawing device 102. This makes it possible to randomly set the time at which drawing of particles ends within one drawing time, thereby changing the drawn state and improving the visual effect.

[0121] Furthermore, the path information generating unit 1015 may generate path information including an initial velocity vector of the particle at the generation point. Furthermore, the path information generating unit 1015 may further include information indicating that the initial velocity vector is to be randomly changed when the drawing data is drawn by the drawing device 102. By generating particle paths based on the initial velocity vector, a path is generated that allows the particles to move in a more visually natural manner. Furthermore, by allowing the initial velocity vector to be randomly set when drawing, it is possible to prevent the same drawing from being repeated when drawing data, thereby providing visual novelty. Therefore, the visual effect can be improved.

[0122] The drawing device 102 according to this embodiment also includes a drawing data acquisition unit 1024 that acquires drawing data generated by the drawing data generation method, a particle position acquisition unit 1025 that acquires the position of a particle on a particle path at each point in time in the time series based on the drawing data, a drawing image generation unit 1026 that generates drawing images in chronological order in which particles are drawn in a drawing area at each point in time in the time series based on the position and drawing information, and a drawing image display processing unit 1027 that displays the drawing images in chronological order.

[0123] By using drawing data generated based on attribute information for drawing information of particles moving along particle paths, drawing can be performed in various ways, providing new visual effects when drawing images.

[0124] The drawing device 102 further includes an environmental information acquisition unit 1028 that acquires environmental information, and an updated image generation unit 1029 that generates, based on the environmental information, multiple updated images that are displayed as multiple drawing images from the time the environmental information was acquired onwards.

[0125] An updated image is generated in accordance with the environmental information and displayed by the drawn image display processing unit 1027, so that interactive changes according to the environment can be imparted to the drawn information, providing new visual effects when drawing an image. Instead of updating the drawing data, the updated image generating unit 1029 may generate updated drawing data as new drawing data and update the image.

[0126] In addition, in the drawing device 102, the updated image generation unit 1029 may update at least one of the particle path and drawing information after the environmental information is acquired based on the environmental information, and generate multiple updated images.

[0127] This allows interactive changes based on the environment to be reflected in particle paths and particle textures, providing new visual effects when drawing images.

[0128] The environmental information may be sound information, and the updated image generation unit 1029 may update at least one of the particle paths and the drawing information based on the sound information. The environmental information acquisition unit 1028 may acquire the sound information from a sound collection device that collects sound. The environmental information acquisition unit 1028 may acquire the sound information based on sound source data.

[0129] As a result, for example, an updated image is generated and drawn in response to sounds around the drawing device 102. This makes it possible to impart interactive changes to the drawn information in response to sounds, providing new visual effects when drawing images.

[0130] The sound information may also include the incident direction of the sound, and the updated image generating unit 1029 may update at least one of the particle path and the drawing information according to the incident direction.

[0131] It becomes possible to give more interactive changes to the rendered information depending on the direction of sound incidence, providing new visual effects when rendering images.

[0132] The sound information may also include a volume of the sound, and the updated image generation unit 1029 may update at least one of the particle path and the drawing information in accordance with the volume. The sound information may also include a frequency of the sound, and the updated image generation unit 1029 may update at least one of the particle path and the drawing information in accordance with the frequency. This makes it possible to impart more interactive changes to the information to be drawn, and provides new visual effects when drawing images.

[0133] Furthermore, the environmental information is light information, and the environmental information acquisition unit 1028 may acquire the light information from a light-receiving device that receives light, and the updated image generation unit 1029 may update at least one of the particle paths and the drawing information based on the light information. This allows an updated image to be generated and drawn in accordance with ambient light as well as sound. This makes it possible to impart interactive changes in accordance with light to the drawn information, providing new visual effects when drawing images.

[0134] Furthermore, the drawn image generation unit 1026 may generate a drawn image DI in which particles are drawn in the drawing area as the foreground of the background image BG, based on drawing information including a background image BG based on the input image I. As a result, for example, although the drawing by particles may be difficult to see depending on the color of the input image, providing a background image clarifies the boundary between the drawing target and the background, improving the visibility of the drawn image and making it easier for the user to see the target drawn by the particles of the drawing target.

[0135] In addition, in the drawing device 102, the updated image generation unit 1029 may generate a plurality of updated images including, as drawing information, a background image having information on the background color and information on the silhouette of the target to be drawn by the particles, and an updated background image in which the background image is updated based on environmental information.

[0136] This allows, for example, an updated background image to be generated so that the boundary between the drawing target and the background is clearer, improving the visibility of the drawing image and making it easier for the user to see the target being drawn by the particles of the drawing target.

[0137] In the drawing device 102, the updated image generating unit 1029 may generate a plurality of updated images by updating silhouette information in the background image based on environmental information.

[0138] This allows the silhouette information in the background image to be updated as the shape of the particle's rendering target changes based on the environmental information, so that even if the shape of the particle's rendering target changes, the visibility can be improved by providing a silhouette.

[0139] The above-described embodiments are intended to facilitate understanding of the present invention and are not intended to limit the present invention. The elements of the embodiments, as well as their arrangement, conditions, shape, size, etc., are not limited to those illustrated and can be modified as appropriate. Furthermore, configurations shown in different embodiments can be partially substituted or combined with each other.

[0140] 10...drawing system, 101...drawing data generating device, 102...drawing device, 1014...attribute information acquiring unit, 1015...path information generating unit, 1016...drawing information generating unit, 1017...drawing data generating unit, 1024...drawing data acquiring unit, 1025...particle position acquiring unit, 1026...drawing image generating unit, 1027...drawing image display processing unit

Claims

1. A drawing data generation method, comprising: a computer acquiring attribute information based on an input image; generating path information for particle paths that connect a generation point in a drawing area with an end point in the drawing area and set the positions of particles to be drawn in the drawing area along a time series; generating drawing information to be used for drawing the particles at each point in the time series based on the attribute information; and generating drawing data in which the path information and the drawing information are associated.

2. A drawing data generation method according to claim 1, wherein generating the drawing information includes generating drawing information including material information indicating the substance to be drawn by the particles.

3. A drawing data generation method according to claim 1, wherein generating the drawing information includes generating drawing information including texture information indicating the texture of the particles.

4. A drawing data generation method according to claim 1, wherein generating the drawing information includes generating drawing information including color information indicating the color of the particles.

5. A drawing data generation method according to claim 4, wherein generating the drawing information includes generating drawing information including color information in which the color of the particle at the end point of the particle path is set based on the color of the pixel of the input image corresponding to the end point.

6. A drawing data generation method according to claim 1, wherein generating the route information includes setting the end point based on the attribute information.

7. A drawing data generation method as described in claim 1, wherein the attribute information includes first attribute information for a first region in the input image, and generating the drawing information includes generating the drawing information including first drawing information based on the first attribute information.

8. A drawing data generation method as described in claim 7, wherein the attribute information further includes second attribute information for a second region in the input image that is different from the first region, and generating the drawing information includes generating the drawing information including the first drawing information and second drawing information based on the second attribute information.

9. A drawing data generation method according to claim 1, wherein generating the drawing information includes generating, based on an input image, the drawing information further including a background image having background color information and silhouette information of an object to be drawn by the particles.

10. A drawing data generation method according to claim 9, wherein generating the drawing information includes generating the drawing information including a plurality of background images each having a plurality of silhouettes with different contours.

11. A drawing data generation method according to claim 1, wherein generating the attribute information includes generating the attribute information based on the input image and drawing settings, which are information added to the input image.

12. A drawing data generation method according to claim 11, wherein generating the attribute information further comprises acquiring the drawing settings from a user and generating the attribute information based on the acquired drawing settings.

13. A drawing data generation method according to claim 1, wherein generating the drawing information includes obtaining information for setting the drawing information from a user, and generating the drawing information based on the obtained information for generating the drawing information.

14. A drawing data generation method according to claim 1, wherein generating the drawing data includes generating drawing data further associated with information indicating that the timing for ending drawing of the particles is to be changed when the drawing data is drawn by a drawing device.

15. A drawing data generation method according to claim 1, wherein generating the path information includes generating the path information including information on the initial velocity vector of the particle at the generation point.

16. A drawing data generation method according to claim 15, wherein generating the path information includes generating the path information further including information indicating that the initial velocity vector is to be changed randomly when the drawing data is drawn by a drawing device.

17. A program that causes a computer to perform the following steps: acquire attribute information based on an input image; generate path information for a particle path that connects a generation point in a drawing area with an end point in the drawing area and sets the position of a particle to be drawn in the drawing area along a time series; generate drawing information used to draw the particle at each point in the time series based on the attribute information; and generate drawing data in which the path information and the drawing information are associated.

18. A drawing data generation device comprising: an attribute information acquisition unit that acquires attribute information based on an input image; a path information generation unit that generates path information for a particle path that connects a generation point in a drawing area with an end point in the drawing area and sets the position of a particle to be drawn in the drawing area along a time series; a drawing information generation unit that generates drawing information used to draw the particle at each point in the time series based on the attribute information; and a drawing data generation unit that generates drawing data in which the path information and the drawing information are associated.

19. A program that causes a computer to perform the following steps: acquire the drawing data generated by the drawing data generation method described in any one of claims 1 to 16; acquire the positions of particles on the particle path at each point in time in the time series based on the drawing data; generate drawing images in the time series in which particles are drawn in the drawing area at each point in time in the time series based on the positions and the drawing information; and display the drawing images in the time series in the time series.

20. A program as described in claim 19, further causing the computer to acquire environmental information, and based on the environmental information, generate a plurality of updated images to be displayed as a plurality of the drawing images after the time when the environmental information was acquired.

21. A program as described in claim 20, wherein generating the updated images includes updating at least one of the particle path and the drawing information after the environmental information is acquired based on the environmental information, thereby generating the multiple updated images.

22. A program according to claim 21, wherein the environmental information is sound information, and generating the updated image includes updating at least one of the particle path and the drawing information based on the sound information.

23. A program according to claim 22, wherein the environmental information is sound information, and acquiring the environmental information includes acquiring the sound information from a sound collection device that collects the sound.

24. A program as described in claim 23, wherein the sound information includes the incident direction of the sound, and generating the updated image includes updating at least one of the particle path and the drawing information according to the incident direction.

25. A program according to claim 20, wherein the environmental information is sound information, and acquiring the environmental information includes acquiring the sound information based on sound source data.

26. A program according to claim 23, wherein the sound information includes the volume of the sound, and generating the updated image includes updating at least one of the particle path and the drawing information according to the volume.

27. A program according to claim 23, wherein the sound information includes a frequency of the sound, and generating the updated image includes updating at least one of the particle path and the drawing information in accordance with the frequency.

28. A program as described in claim 20, wherein the environmental information is light information, acquiring the environmental information includes acquiring the light information from a light receiving device that receives the light, and generating the updated image includes updating at least one of the particle path and the drawing information based on the light information.

29. A program as described in claim 19, wherein generating the drawn image includes generating the drawn image in which the particles are drawn in the drawing area as a foreground of a background image based on the input image, based on the drawing information including the background image.

30. A program as described in claim 20, wherein the drawing data further includes, as the drawing information, a background image having background color information and silhouette information of an object to be drawn by the particles based on the input image, and generating the updated images includes generating the plurality of updated images including, as the drawing information, an updated background image in which the background image is updated based on the environmental information.

31. A program according to claim 30, wherein generating the updated images includes updating the silhouette information in the background image based on the environmental information to generate the plurality of updated images.

32. A drawing device comprising: a drawing data acquisition unit that acquires the drawing data generated by the drawing data generation method of any one of claims 1 to 16; a position acquisition unit that identifies the position of the particle on the particle path at each point in time in the time series based on the drawing data; a drawing image generation unit that generates drawing images in the time series in which the particle is drawn in the drawing area at each point in time in the time series based on the position and the drawing information; and a drawing image display unit that displays the drawing images in the time series in the time series.

33. A data structure of drawing data including: path information of a particle path that connects a generation point within a drawing area with an end point within the drawing area and sets the position of a particle to be drawn in the drawing area along a time series; and drawing information used to draw the particle moving along the particle path at each point in the time series, wherein a computer that acquires the drawing data performs the following operations: based on the drawing data, identifies the position of the particle on the particle path at each point in the time series; based on the position and the drawing information, continuously generates drawing images along the time series in which the particle is drawn in the drawing area at each point in the time series; and continuously displays the drawing images along the time series.

Images