Image generating apparatus, image generating method and program

Abstract

To provide an image generating apparatus or the like for enabling a user to easily grasp characteristics of an input image.SOLUTION: The image generating apparatus includes: an acquisition unit for acquiring an image; a generation unit for generating a converted image in which a graphic is arranged based on characteristics of the image; and an output unit for outputting the converted image.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an image generating device, an image generating method, and a program. [Background technology]

[0002] When copying a photograph or painting, it is important to grasp the size and relative positions of the characteristic parts of the object on the canvas. The size and relative positions of the characteristic parts can be grasped, for example, by sketching simple shapes such as ellipses or lines at the positions on the canvas where the characteristic parts should be drawn. Understanding the size and relative positions of the characteristic parts is also called "making a rough sketch." Those with little experience in copying may not be able to make a rough sketch properly, which may result in an unsuccessful copy.

[0003] Patent Document 1 describes a method for assisting copying practice by projecting a photograph or picture onto a canvas. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-65603 A Summary of the Invention [Problem to be solved by the invention]

[0005] To aid in the development of copying skills, it may be preferable to present only characteristic portions of a photograph or painting rather than projecting the entire image.

[0006] The present invention has been made to solve the above-mentioned problems, and aims to provide an image generation device, an image generation method, and a program that enable a user to easily grasp the characteristics of an input image. [Means for solving the problem]

[0007] An image generating device according to an embodiment of the present invention is characterized by having an acquisition unit that acquires an image, a generation unit that generates a converted image in which figures are arranged based on the characteristics of the image, and an output unit that outputs the converted image.

[0008] Also, the shape of the figure is preferably selected from a plurality of geometric shapes.

[0009] In addition, it is preferable that the generation unit extracts a foreground region from the image, divides the foreground region based on a positional relationship between multiple pixels included in the foreground region, thereby extracting multiple regions from the image, and calculates a figure based on the shape of each region.

[0010] An image generating method according to an embodiment of the present invention is characterized by including the steps of acquiring an image, generating a converted image in which a figure is arranged based on a feature of the image, and outputting the converted image.

[0011] A program according to an embodiment of the present invention is characterized in that it causes a computer to acquire an image, generate a converted image in which a figure is arranged based on the characteristics of the image, and output the converted image. Effect of the Invention

[0012] The image generating device, the image generating method, and the program according to the present invention enable a user to easily grasp the features of an input image. [Brief description of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of an image generation system 1. FIG. [Diagram 2] FIG. 1A is a diagram showing an example of an input image, and FIG. 1B is a diagram showing an example of a converted image. [Diagram 3] FIG. 13 is a diagram showing an example of the data structure of a shape table T1. [Figure 4] FIG. 11 is a flowchart showing an example of the flow of an image output process. [Diagram 5]FIG. 11 is a flow diagram showing an example of the flow of a generation process. [Figure 6] FIG. 11 is a flowchart showing an example of the flow of a graphic calculation process. [Figure 7] FIG. 13 is a schematic diagram for explaining calculation of a curve. [Figure 8] FIG. 13 is a diagram illustrating an example of a converted image. [Figure 9] FIG. 13 is a diagram illustrating an example of a converted image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, various embodiments of the present invention will be described with reference to the drawings. It should be noted that the technical scope of the present invention is not limited to these embodiments, but covers the inventions described in the claims and their equivalents.

[0015] FIG. 1 is a diagram showing an example of a schematic configuration of an image generation system 1 according to an embodiment of the present invention. The image generation system 1 generates a converted image in which figures are arranged based on an input image including an object. The figures are arranged, for example, so as to show characteristic parts of the object. By referring to the converted image, a user can grasp the characteristic parts of the object and appropriately copy the input image. The image generation system 1 has an image generation device 2 and a communication terminal 3. The image generation device 2 and the communication terminal 3 communicate with each other via a network N, which is the Internet or an intranet.

[0016] The image generating device 2 is an information processing device such as a server, a PC (Personal Computer), a tablet terminal, etc. The image generating device 2 has a storage unit 21, a communication unit 22, and a processing unit .

[0017] The storage unit 21 is configured to store programs and data, and includes, for example, a semiconductor memory. The storage unit 21 stores, as programs, an operating system program, a driver program, an application program, and the like used in processing by the processing unit 23. The programs are installed in the storage unit 21 from a computer-readable, non-transitory, portable storage medium such as a CD-ROM (Compact Disc Read Only Memory) or a DVD-ROM (Digital Versatile Disc Read Only Memory).

[0018] The communication unit 22 is configured to enable the image generating device 2 to communicate with other devices, and includes a communication interface circuit. The communication interface circuit is a communication interface circuit for a wired LAN (Local Area Network) or a wireless LAN. The communication unit 22 receives data from other devices and supplies the data to the processing unit 23, and transmits data supplied from the processing unit 23 to other devices.

[0019] The processing unit 23 is configured to comprehensively control the operation of the image generating device 2, and includes one or more processors and their peripheral circuits. The processing unit 23 includes, for example, a CPU (Central Processing Unit). The processing unit 23 may include a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc. The processing unit 23 controls the operation of each component and executes various processes so that various processes of the image generating device 2 are executed in an appropriate order based on the programs stored in the storage unit 21.

[0020] The processing unit 23 has an acquisition unit 231, a setting unit 232, a generation unit 233, and an output unit 234. Each of these units is a functional module realized by the processing unit 23 executing a program. Each of these units may be implemented in the image generating device 2 as a dedicated processing circuit.

[0021] The communication terminal 3 is an information processing terminal such as a PC, a tablet terminal, a smartphone, a mobile phone, a portable game machine, etc. owned by a user. The communication terminal 3 displays an image based on the display data transmitted from the image generating device 2. The communication terminal 3 also transmits a signal to the image generating device 2 in response to an operation by the user.

[0022] FIG. 2(A) is a diagram showing an example of an input image, and FIG. 2(B) is a diagram showing an example of a converted image. The input image shown in FIG. 2(A) includes a dog as an object. In the converted image shown in FIG. 2(B), the dog's head, torso, left front leg, and tail are extracted as characteristic parts of the object. In the converted image, the head, torso, and left front leg are represented by ellipses, and the tail is represented by a curved line. By referring to the converted image, the user can easily grasp the positional relationship between the head, torso, left front leg, and tail, and copy the input image.

[0023] 3 is a diagram showing an example of the data structure of the shape table T1 stored in the storage unit 21. The image generating device 2 converts the characteristic portion of the object into a figure having a shape according to the level of the user. The shape table T1 stores the user's level and the shape in association with each other.

[0024] In the example shown in FIG. 3, five levels are set as user levels. Also, for each level, a shape selected from a plurality of geometric shapes set in advance is set as the shape of a figure to be placed in the converted image. For example, the plurality of geometric shapes include a convex polygon, an ellipse, and a simple curve. Here, an ellipse includes a perfect circle. Also, a simple curve includes a straight line. Hereinafter, a convex polygon and a simple curve are simply referred to as a polygon and a curve, respectively.

[0025] For example, at level 1, which indicates that the user has the least experience in copying, the characteristic parts are shown as polygons. At level 3, which indicates that the user has medium experience in copying, the characteristic parts are shown as ellipses. At level 5, which indicates that the user has the most experience in copying, the characteristic parts are shown as curves. That is, since a user with little experience in copying may not be able to grasp the outline of the characteristic parts, the characteristic parts are shown as polygons so that the outline of the characteristic parts can be roughly grasped in the converted image. On the other hand, a user with much experience in copying can grasp the outline of the characteristic parts, so the characteristic parts are shown as curves in the converted image.

[0026] At level 2, which is intermediate between level 1 and level 3, features are represented by either ellipses or polygons. At level 4, which is intermediate between level 3 and level 5, features are represented by either curves or ellipses. In these cases, one of the geometric shapes is selected depending on the shape of the feature.

[0027] The data in the shape table T1 is set in advance.

[0028] 4 is a flow diagram showing an example of the flow of image output processing executed by image generating device 2. The image output processing is processing for generating and outputting a converted image in which figures showing characteristic parts of an object are arranged based on an input image. The image output processing is realized by processing unit 23 cooperating with the configuration of image generating device 2 based on a program stored in storage unit 21.

[0029] First, the acquisition unit 231 acquires an input image (step S11). For example, the acquisition unit 231 acquires the input image by receiving the input image from the communication terminal 3 via the communication unit 22. The acquisition unit 231 may acquire an input image stored in advance in the storage unit 21.

[0030] Next, the setting unit 232 sets the number and shape of figures indicating characteristic parts of the object (step S12). For example, the setting unit 232 transmits display data of a screen for inputting the number of figures and the user's level to the communication terminal 3 via the communication unit 22. The display data is, for example, HTML (Hypertext markup Language). The setting unit 232 acquires the number of figures and the user's level input to the communication terminal 3 by receiving them from the communication terminal 3. The setting unit 232 acquires a shape associated with the user's level from the shape table T1. The setting unit 232 sets the number and shape of figures based on the acquired information. Note that the setting unit 232 may set the number and shape of figures to default values ​​without receiving information from the communication terminal 3.

[0031] Next, the generating unit 233 executes a generating process (step S13). The generating process is a process for generating a converted image in which a figure is arranged based on the characteristics of the input image. The generating process will be described in detail later.

[0032] Next, the output unit 234 outputs the converted image (step S14). For example, the output unit 234 outputs the converted image by transmitting it to the communication terminal 3 via the communication unit 22. The output unit 234 may output the converted image by printing it via a printer or the like. This completes the image output process.

[0033] 5 is a flow diagram showing an example of the flow of the generation process executed by the image generation device 2. The generation process is executed in step S13 of the image output process.

[0034] First, the generating unit 233 extracts a plurality of regions from the input image based on the features of the input image (step S21). The generating unit 233 extracts a foreground region including an object from the input image. For example, the generating unit 233 applies a Watershed algorithm, a GrabCut algorithm, or the like to extract a region including an object as a foreground region. The generating unit 233 extracts a plurality of regions by dividing the foreground region into the number of regions set in step S12. For example, the generating unit 233 divides the foreground region by applying spectral clustering to classify pixels included in the foreground region into the number of clusters set in step S12. In the spectral clustering, the similarity between pixels is set based on the distance between pixels.

[0035] Next, the generating unit 233 selects one of the multiple regions as a region to be processed (step S22).

[0036] Next, the generating unit 233 executes a graphic calculation process (step S23). The graphic calculation process is a process for calculating a graphic corresponding to each region based on the shapes of the multiple regions. The graphic calculation process will be described in detail later.

[0037] Next, the generating unit 233 determines whether or not all of the multiple regions have been selected as regions to be processed (step S24).

[0038] If all of the multiple areas have not been selected as areas to be processed (step S24-No), the generation process returns to step S22, and the generation unit 233 selects one of the areas that have not yet been selected as areas to be processed as the area to be processed.

[0039] When all of the multiple regions are selected as the regions to be processed (step S24-Yes), the generating unit 233 judges whether or not the error condition is satisfied (step S25). The error condition is that the total value of the error between the figure calculated for each of the multiple regions and each region is less than a threshold value. As described later, since the input image and the converted image have the same number of pixels, the error is calculated based on the difference between the pixels included in the region extracted from the input image and the pixels included in the figure arranged in the converted image. For example, when the figure is an ellipse or a polygon, the error is the total value of the number of pixels that are included in the ellipse or polygon in the converted image and are not included in the region corresponding to the ellipse or polygon in the input image, and the number of pixels that are not included in the ellipse or polygon in the converted image and are included in the region corresponding to the ellipse or polygon in the input image. When the figure is a curve, the error is the number of pixels that form the curve in the converted image and are not included in the region corresponding to the curve in the input image.

[0040] If the error condition is not satisfied (step S25-No), the generating unit 233 changes the number of figures (step S26). For example, the generating unit 233 increases the number of figures by 1. After that, the generation process proceeds to step S21, and the generating unit 233 extracts multiple regions from the input image again.

[0041] If the error condition is satisfied (step S25-Yes), the generating unit 233 generates a converted image (step S27). The generating unit 233 generates a binary image as the converted image, in which the pixel values ​​of pixels located on the curve, the circumference of the ellipse, and the sides of the polygon are 0, and the pixel values ​​of other pixels are 1. This ends the generation process.

[0042] 6 is a flow diagram showing an example of the flow of the geometry calculation process. The geometry calculation process is executed in step S23 of the generation process.

[0043] First, the generating unit 233 judges whether the curve calculation condition is satisfied (step S31). The curve calculation condition is a condition for calculating a curve as a figure corresponding to the area to be processed. For example, the curve calculation condition is (1) that only a curve is set as the shape of the figure in step S12, or (2) that a curve and another figure are set as the shape of the figure and the aspect ratio of the area to be processed is a predetermined value or more. The aspect ratio is a value obtained by dividing the long side length of the circumscribed rectangle by the short side length when the area to be processed is fitted to the circumscribed rectangle while taking rotation into consideration. In other words, when a curve and another figure are set as the shape of the figure, a curve is calculated only for an area where the aspect ratio is large and it does not look strange to associate the curve.

[0044] If the curve calculation condition is satisfied (step S31-Yes), the generating unit 233 calculates a curve as a graphic corresponding to the area to be processed (step S32). This ends the graphic calculation process.

[0045] FIG. 7 is a schematic diagram for explaining the calculation of the curve in step S32. In FIG. 7, the long side length X of the outline rectangle of the region S to be processed is greater than the short side length Y. That is, the aspect ratio of the region S is X / Y. At this time, the curve C corresponding to the region S is calculated by connecting the points at both ends of the region S in the x direction and the midpoints of two intersections between a straight line extending in the y direction and the contour of the region S. In FIG. 7, the points P1 and P2 at both ends of the region S in the x direction and points M1-M9 as representative midpoints are illustrated. Note that, when the width of the region S in the y direction is greater than the width in the x direction, the curve C is calculated by connecting the midpoints of two intersections between a straight line extending in the x direction and the contour of the region S. That is, the curve corresponding to the region is calculated by connecting the midpoints of two intersections between a straight line extending in the x direction and the contour of the region that are mutually orthogonal and extend in the direction in which the width of the region to be processed is smaller, among two directions along the pixel array. In addition, the curve corresponding to the area may be calculated by connecting the midpoints of two intersections between a straight line extending in the direction in which the width of the area to be processed is smaller, and the contour line of the area, which are mutually perpendicular and parallel to the long and short sides of the outline rectangle, respectively.

[0046] Returning to FIG. 6, if the curve calculation condition is not satisfied, the generating unit 233 judges whether or not the ellipse calculation condition is satisfied (step S33). The ellipse calculation condition is a condition for calculating a curve as a figure corresponding to the area to be processed. For example, the ellipse calculation condition is that an ellipse is set as the shape of the figure in step S12.

[0047] If the ellipse calculation condition is satisfied (step S33-Yes), the generation unit 233 calculates an ellipse as a figure corresponding to the area to be processed (step S34). The generation unit 233 detects edge pixels that form the contour of the area to be processed. For example, the edge pixels are detected using the Canny algorithm or a Sobel filter. The generation unit 233 calculates an ellipse by fitting an ellipse to the edge pixels. The fitting of the ellipse is performed by, for example, the least squares method. Since an ellipse is specified by the center coordinates, the major axis, the minor axis, and the inclination, the fitting of an ellipse is possible if there are five or more edge pixels.

[0048] If the ellipse calculation condition is not satisfied (step S33-No), or following step S34, the generation unit 233 determines whether or not the polygon calculation condition is satisfied (step S35). The polygon calculation condition is a condition for calculating a polygon as a figure corresponding to the area to be processed. For example, the polygon calculation condition is that a polygon is set as the shape of the figure in step S12.

[0049] If the polygon calculation conditions are not satisfied (step S35-No), the graphic calculation process ends.

[0050] If the polygon calculation condition is satisfied (step S35-Yes), the generation unit 233 calculates a polygon as a figure corresponding to the area to be processed (step S36). The generation unit 233 calculates a convex hull that contains the pixels that make up the area to be processed as a polygon corresponding to the area to be processed. The convex hull is calculated by, for example, the gift wrapping method or the Graham scan.

[0051] Next, the generating unit 233 determines whether or not both an ellipse and a polygon have been calculated as the graphic form corresponding to the area to be processed (step S37).

[0052] If neither the ellipse nor the polygon has been calculated (step S37-No), the geometry calculation process ends.

[0053] If both an ellipse and a polygon have been calculated (step S37-Yes), the generation unit 233 selects one of the ellipse and the polygon as a figure corresponding to the region to be processed based on the error between the figure and the region to be processed (step S38). The generation unit 233 selects, from the ellipse and the polygon, the one with the smaller error from the region to be processed as the figure corresponding to the region. The error is the total value of the number of pixels that are included in the ellipse or the polygon but not included in the corresponding region, and the number of pixels that are not included in the ellipse or the polygon but included in the corresponding region. This ends the figure calculation process.

[0054] 8(A)-(E) and 9(A)-(E) are diagrams showing examples of converted images generated based on the input image shown in FIG. 2(A). FIG. 8(A)-(E) show examples of converted images when the number of shapes is 4. FIG. 8(A) shows converted images when the user's level is 1, FIG. 8(B) shows converted images when the user's level is 2, FIG. 8(C) shows converted images when the user's level is 3, FIG. 8(D) shows converted images when the user's level is 4, and FIG. 8(E) shows converted images when the user's level is 5. Similarly, FIG. 9(A)-(E) show converted images when the user's level is 1, FIG. 9(B) shows converted images when the user's level is 2, FIG. 9(C) shows converted images when the user's level is 3, FIG. 9(D) shows converted images when the user's level is 4, and FIG. 9(E) shows converted images when the user's level is 5. In this way, various converted images are generated according to the user's level.

[0055] The user can copy the input image based on the generated converted image. For example, the user executes a drawing application program on the communication terminal 3 and displays the converted image as a background, thereby copying the input image in the drawing application. The user prints the converted image on paper or the like, and copies the input image by overlaying it on the converted image.

[0056] As described above, the generation unit 233 of the image generation device 2 generates a converted image in which a figure is arranged based on the features of the input image. This allows the user to easily grasp the features of the input image.

[0057] Furthermore, the generating unit 233 selects the shape of the figure from among a plurality of geometric shapes, which allows the user to easily grasp the outline of the characteristics of the input image.

[0058] Furthermore, the generating unit 233 extracts a plurality of regions from the image based on the positional relationship of a plurality of pixels included in the input image, and calculates a figure based on the shape of each region. This allows the user to appropriately grasp the outline of the features extracted based on the pixel values ​​of the input image.

[0059] The image generation system 1 may have the following modified examples.

[0060] In the above-described embodiment, the user's level and the shape of the figure are associated in the shape table T1, and in step S12 of the image output process, the setting unit 232 sets the shape of the figure based on information indicating the user's level. This is not a limitation, and the setting unit 232 may obtain information indicating the shape of the figure. In this case, in step S12, the setting unit 232 transmits display data of a screen for inputting the number and shape of the end figures to the communication terminal 3.

[0061] In the above-described embodiment, in step S21 of the generation process, the generation unit 233 applies spectral clustering using similarity based on the distance between pixels to extract multiple regions from the input image. The present invention is not limited to this example, and the similarity between pixels may be set based on the distance between pixels and the difference in luminance value. For example, the similarity may be set so that the larger the difference in luminance value between pixels, the smaller the similarity. In this way, pixels that have similar luminance values ​​and are likely to constitute the same feature portion are classified into the same cluster, so that the feature portion of the input image is more appropriately extracted.

[0062] Furthermore, the generating unit 233 may extract a plurality of regions from the input image by applying a method other than spectral clustering. For example, the generating unit 233 may apply agglomerative hierarchical clustering to classify pixels included in the foreground region into a predetermined number of clusters. Alternatively, clustering may not be used. For example, the pixels included in the foreground region can be classified into a predetermined number of regions by using any method, such as region division using a Watershed algorithm or a GrabCut algorithm, region division based on edge information, or region division based on an inscribed figure. In this way, the user can easily grasp the features of the input image.

[0063] In the above-described embodiment, in step S21 of the generation process, the generation unit 233 extracts a foreground region from the input image and extracts multiple regions by dividing the foreground region. The present invention is not limited to such an example, and the generation unit 233 may directly extract multiple regions from the input image without extracting the foreground region. For example, the generation unit 233 may extract multiple regions from the input image using a trained model that has been trained to detect characteristic parts (head, torso, etc.) of an object when an image is input. In this case, the number of figures may not be set in step S12 of the image output process.

[0064] In the above-described embodiment, the shape of the figure is set to any one of a curve, an ellipse, and a convex polygon. The shape of the figure is not limited to such an example, and may include other shapes. For example, an area surrounded by a quadratic curve may be set as the shape of the figure. In addition, other classification methods may be applied as a classification method for the shape of the figure. For example, the shape of the figure may be set to any one of a curve, an ellipse, a triangle, and a convex polygon having four or more vertices.

[0065] The functions of the image generation device 2 described above may be realized by a plurality of devices. Also, the functions of the image generation device 2 and the communication terminal 3 described above may be realized by a single device.

[0066] It should be understood by those skilled in the art that various changes, substitutions, and modifications can be made to the present invention without departing from the scope of the present invention. For example, the above-described embodiments and modifications may be appropriately combined within the scope of the present invention. [Explanation of symbols]

[0067] 2. Image Generation Device 231 Acquisition Department 232 Setting section 233 Generation part 234 Output section

Claims

1. An acquisition unit that acquires an image; A generating unit that generates a converted image in which a figure is arranged based on the image; an output unit that outputs the converted image; An image generating device comprising:

2. The shape of the figure is selected from a plurality of geometric shapes.

2. The image generating device of claim 1.

3. The generation unit is Extracting a foreground region from the image; extracting a plurality of regions from the image by dividing the foreground region based on a positional relationship of a plurality of pixels included in the foreground region; calculating the figure based on the shape of each region; 2. The image generating device of claim 1.

4. Acquire an image, generating a transformed image in which a graphic is arranged based on the features of the image; outputting the transformed image; 13. An image generating method comprising:

5. Acquire an image, generating a transformed image in which a graphic is arranged based on the features of the image; outputting the transformed image; A program that causes a computer to execute the following:

Images